Focus following method and system based on screen direction, electronic equipment and storage medium

Abstract

The invention provides a focus following method and system based on a screen direction, electronic equipment and a storage medium, and relates to the technical field of shooting processing, and the method comprises the steps: obtaining a screen direction state and a real-time video frame of shooting equipment; performing target detection and feature extraction on the real-time video frame to obtain at least one candidate target and feature information of the candidate target; performing dynamic weighting according to the screen direction state and the feature information, and calculating a comprehensive priority score of each candidate target; determining an optimal tracking target according to the comprehensive priority score, and calculating a target deviation between the optimal tracking target and the center of the image frame based on the feature information of the optimal tracking target; and generating a control signal according to the target deviation, and driving a lens of the shooting equipment to perform target automatic tracking and focusing. According to the invention, the intelligent level of automatic tracking and focusing and the user experience in the shooting process are obviously improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of photographing processing, in particular to a screen direction-based follow focus method and system, electronic equipment and storage medium. BACKGROUND

[0002] The current mainstream gimbal camera and smartphone automatic focusing and tracking functions continue to develop, greatly improving the convenience of shooting. At present, the mainstream automatic focusing and tracking methods mainly rely on the saliency features of the target, such as color contrast and motion speed, or are based on user manual selection of the tracking target.

[0003] However, these methods have obvious limitations and lack a deep understanding of the user's shooting intention. Users hold the shooting device in different ways, often showing differentiated shooting intentions and composition preferences. Horizontal holding is mostly used for shooting wide scenes such as landscapes and vehicles, while vertical holding is more suitable for shooting vertical subjects such as people and high-rise buildings. When the physical direction of the shooting device changes dynamically, the existing technology has the problem of poor automatic focusing and tracking effect, which cannot meet the creative needs of professional users. SUMMARY

[0004] In view of the above shortcomings of the prior art, the present application provides a screen direction-based follow focus method and system, electronic equipment and storage medium, which effectively solves the problem of poor automatic focusing and tracking effect when the physical direction of the shooting device changes dynamically.

[0005] In a first aspect, the present application provides a screen direction-based follow focus method, which comprises: obtaining the screen direction state of the shooting device and the real-time video frame; performing target detection and feature extraction on the real-time video frame to obtain at least one candidate target and feature information of the candidate target; performing dynamic weighting according to the screen direction state and the feature information to calculate the comprehensive priority score of each candidate target; determining the optimal tracking target according to the comprehensive priority score, and calculating the target deviation of the optimal tracking target from the image center based on the feature information of the optimal tracking target; generating a control signal according to the target deviation to drive the lens of the shooting device to perform target automatic tracking and focusing.

[0006] In an optional embodiment, the target detection and feature extraction on the real-time video frame to obtain at least one candidate target and feature information of the candidate target comprises: performing image preprocessing on the real-time video frame to obtain an image tensor; The image tensor is input into the object detection model to obtain at least one initial candidate object and initial feature parameters. The initial feature parameters include at least bounding box parameters, object confidence, and class probability. Post-processing decoding and motion vector calculation are performed based on the initial feature parameters to obtain the candidate target and its category identifier, detection confidence, bounding box information, and motion vector.

[0007] In an optional implementation, the formula for calculating the comprehensive priority score is as follows: The formula for calculating the overall priority score is as follows:

[0008] In the above formula, S i This indicates the overall priority score. This represents the confidence score. Cnf i Indicates the first i The detection confidence of each candidate target. Represents the centrality fraction. x i and y i Indicates the first i The center point coordinates of the bounding boxes of the candidate targets This indicates the sizing fit fraction. w i and h i They represent the first i The width and height of the bounding boxes of each candidate target. This represents the fractional terms indicating the significance of the motion. Mv i Indicates the first i The motion vectors of the candidate targets Indicates category score items, C i Indicates category, O This indicates the screen orientation state. , , , and These represent the weighting coefficients for the confidence score, centrality score, size fitness score, motion significance score, and category score, respectively.

[0009] In an optional implementation, the confidence score term weight coefficient, the center degree score term weight coefficient, the size adaptability score term weight coefficient, the motion saliency score term weight coefficient and the category score term weight coefficient are dynamically adjusted according to the screen orientation state, including: When the screen orientation state is landscape, the motion saliency score term weight coefficient is increased, and sensitivity to horizontal direction motion is increased. When the screen orientation state is portrait, the center degree score term weight coefficient is increased, and sensitivity to vertical direction motion is increased.

[0010] In an optional implementation, the sensitivity to horizontal direction motion is increased, including: The motion vector of the candidate target is decomposed into a horizontal component and a vertical component, the horizontal component is given a higher weight than the vertical component, and the motion saliency score term is re-determined based on the weighted motion vector length; The sensitivity to vertical direction motion is increased, including: The motion vector of the candidate target is decomposed into a horizontal component and a vertical component, the vertical component is given a higher weight than the horizontal component, and the motion saliency score term is re-determined based on the weighted motion vector length.

[0011] In an optional implementation, the optimal tracking target is determined according to the comprehensive priority score, and a target deviation of the optimal tracking target from the image frame center is calculated based on feature information of the optimal tracking target, including: The candidate target with the highest comprehensive priority score is determined as the optimal tracking target; The target deviation is calculated according to the bounding box information of the optimal tracking target and the frame center coordinate information of the image frame center.

[0012] In an optional implementation, the target deviation is calculated according to the bounding box information of the optimal tracking target and the coordinate information of the image frame center, including: The target center coordinate information of the optimal tracking target in an image pixel coordinate system is calculated according to the bounding box information; A pixel deviation is calculated according to the target center coordinate information and the frame center coordinate information; The pixel deviation is converted into an angle deviation based on the field of view angle of the shooting device, and the target deviation is obtained.

[0013] In a second aspect, the application provides a screen orientation-based focus tracking system, including: A data acquisition module is configured to acquire a screen orientation state of a shooting device and a real-time video frame; a target detection module, configured to perform target detection and feature extraction on the real-time video frame to obtain at least one candidate target and feature information of the candidate target; a score calculation module, configured to perform dynamic weighting according to the screen orientation state and the feature information, and calculate a comprehensive priority score of each candidate target; a deviation calculation module, configured to determine an optimal tracking target according to the comprehensive priority score, and calculate a target deviation of the optimal tracking target from a center of an image frame based on feature information of the optimal tracking target; a focus control module, configured to generate a control signal according to the target deviation, and drive a lens of the shooting device to perform target automatic tracking and focusing.

[0014] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the screen orientation based focus method according to the first aspect of the present application.

[0015] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the screen orientation based focus method according to the first aspect of the present application.

[0016] The screen orientation based focus method, system, electronic device and storage medium provided by the present application realize context perception and intention understanding of intelligent focus composition based on the screen orientation of a shooting device and multi-dimensional target features, dynamically adjust target priority weights according to a horizontal screen or a vertical screen state, make the shooting composition more in line with user intentions, and significantly improve the intelligent level of automatic tracking and focusing and user experience in the shooting process. At the same time, the method effectively suppresses the interference of non-intended targets, improves the accuracy and scene adaptability of subject selection, realizes accurate follow-up shooting and continuous focusing, ensures stable pictures and clear subjects, has stronger semantic understanding ability in complex multi-target scenes, reduces false tracking and frequent switching, and is suitable for application occasions with high requirements for composition aesthetics and operation convenience. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0018] Figure 1 is a first schematic diagram of the screen orientation based focus method provided by the embodiments of the present application; Figure 2 is a second schematic diagram of a screen direction-based tracking focus method provided by an embodiment of the present application; Figure 3 is a third schematic diagram of a screen direction-based tracking focus method provided by an embodiment of the present application; Figure 4 is a schematic diagram of a tracking process of a shooting device in a horizontal screen state in an embodiment of the present application; Figure 5 is a schematic diagram of a tracking process of a shooting device in a vertical screen state in an embodiment of the present application; Figure 6 is a schematic diagram of a screen direction-based tracking focus system structure provided by an embodiment of the present application; Figure 7 is a schematic diagram of a structure of an electronic device provided by an embodiment of the present application.

[0019] Main element symbol description: 200, a screen direction-based tracking focus system; 210, a data acquisition module; 220, a target detection module; 230, a score calculation module; 240, a deviation calculation module; 250, a tracking focus control module; 300, an electronic device; 310, a processor; 320, a communication interface; 330, a memory; 340, a communication bus. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the present application will be further clearly and completely described below in combination with the drawings in the embodiments of the present application. It should be noted that the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0021] In addition, the terms "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used in the specification of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application.

[0023] Current autofocus and tracking methods in shooting devices primarily rely on the salient features of the target or are based on manual selection of the target by the user, such as simply tracking the largest face in the frame, the fastest-moving object, or an object clicked by the user. However, these methods have significant limitations, lacking a deep understanding of the user's shooting intent. Different ways of holding the shooting device lead to significant differences in shooting intent and compositional preferences. When holding the device horizontally, users tend to shoot wide scenes such as landscapes and vehicles, pursuing a grand and expansive visual effect. When holding the device vertically, they tend to shoot vertical subjects such as people and tall buildings, emphasizing the height or elongation of the subject. When the physical orientation of the shooting device changes dynamically, existing technologies suffer from poor automatic autofocus and tracking performance, failing to meet the creative needs of professional users.

[0024] Example 1 This application provides a screen orientation-based focusing method that effectively solves the problem of poor automatic focusing and tracking performance when the physical orientation of the shooting device changes dynamically. Figure 1 This is a first schematic diagram of the focusing method based on screen orientation provided in the embodiments of this application, as shown below. Figure 1 As shown, the method includes the following steps: S100: Acquire the screen orientation status and real-time video frames of the shooting device.

[0025] In this embodiment, the shooting device can be a camera, mobile phone, or camcorder, and the screen orientation state can be determined by a microswitch, which includes at least a landscape orientation and a portrait orientation. Simultaneously, real-time video frames can be acquired using an image sensor.

[0026] S200. Perform target detection and feature extraction on real-time video frames to obtain at least one candidate target and its feature information.

[0027] In this embodiment of the application, target detection and feature extraction are performed on real-time video frames to identify one or more candidate targets in the real-time video frames and generate feature information for each candidate target. Figure 2 This is a second schematic diagram of the focusing method based on screen orientation provided in the embodiments of this application, as shown below. Figure 2 As shown, target detection specifically includes the following steps: S210. Perform image preprocessing on real-time video frames to obtain image tensors.

[0028] Optionally, real-time video frames can be scaled to a fixed size, such as 640×640, and then normalized and transformed using tensors to obtain image tensors that meet the input requirements of the object detection model.

[0029] S220. Input the image tensor into the object detection model to obtain at least one initial candidate object and initial feature parameters. The initial feature parameters include at least bounding box parameters, object confidence and class probability.

[0030] As an optional implementation of this application, the object detection model can employ a YOLOv5 network model, which is obtained by training the YOLOv5 network model using a public dataset. The preprocessed image tensor is input into the object detection model to obtain a prediction output with a shape of [1, 25200, 15]. This prediction output includes one or more initial candidate targets and initial feature parameters. The initial candidate targets are predicted bounding boxes, and their initial feature parameters include, but are not limited to, bounding box parameters, object confidence, and class probability.

[0031] In this embodiment, in [1,25200,15], 1 represents the batch size, 25200 represents the total number of anchor boxes predicted by the object detection model at all scales, and 15 represents the dimensional parameters of each predicted bounding box, which are composed of: [x_center, y_center, width, height, object_confidence, class_probability_0,...,class_probability_9]. The first four values ​​represent the center coordinates, width, and height of the bounding box, respectively, which are offsets relative to the grid cells and need to be decoded later. The fifth value represents the object confidence, indicating the probability that an object exists within the bounding box. The last 10 values ​​represent the class probabilities, and the specific number of classes can be set according to the actual situation.

[0032] S230. Perform post-processing decoding and motion vector calculation based on the initial feature parameters to obtain candidate targets and their category identifiers, detection confidence, bounding box information, and motion vectors.

[0033] In this embodiment of the application, the post-processing encoding includes the following steps: First, confidence filtering is performed, discarding predicted bounding boxes whose object confidence is below a threshold. Optionally, this threshold can be set to 0.4.

[0034] Then, class determination is performed. For the remaining predicted bounding boxes, the object confidence score corresponding to each predicted bounding box is multiplied by the class probability, and the class with the highest score is taken as its class label. C i This category identifier C i Objects including, but not limited to, vehicles, ships, horizons, people, tall buildings, and trees are considered, and the highest score is used as the detection confidence of the predicted bounding box. Cnf i .

[0035] Next, a non-maximum suppression algorithm is applied to remove duplicate predicted bounding boxes, obtaining candidate targets and ensuring that each candidate target has only one optimal detection result. Specifically, first, scores are sorted, with all predicted bounding boxes ordered from highest to lowest score. Then, a greedy selection process is performed, choosing the predicted bounding box with the highest score as the baseline box. Next, overlap suppression is performed, calculating the intersection-union ratio (IUU) of the baseline box with other boxes; if it exceeds a threshold, other boxes are suppressed. Finally, a loop is executed, repeating the greedy selection and overlap suppression steps until all predicted bounding boxes have been processed.

[0036] Finally, bounding box information decoding and coordinate transformation are performed. The selected predicted bounding box coordinates are decoded and mapped back to the pixel coordinate system of the original image to obtain the final bounding box information. x i ,y i ,w i ,h i ),in( x i ,y i () represents the coordinates of the center point of the bounding box. w i and h i These represent the width and height of the bounding box, respectively.

[0037] In this embodiment of the application, the motion vector can be calculated using optical flow or inter-frame difference methods. For example: First, target association is performed by associating the bounding boxes of candidate targets in the current real-time video frame with the tracking trajectories of the previous video frame. Then, displacement is calculated: for the same successfully associated candidate target, the displacement vector of its bounding box center point between the two frames is calculated to obtain the motion vector. ,in:

[0038] In the above formula, dx express x Displacement vector in the axial direction, x t and y t This represents the coordinates of the center point of the bounding box in the current real-time video frame. x t-1 and y t-1 This represents the coordinates of the center point of the bounding box in the previous video frame.

[0039] This application's embodiments achieve high-precision identification of real-time video frames through target detection and feature extraction, outputting key features such as category, bounding box information, object confidence, and motion vectors, providing comprehensive data support for subsequent intelligent decision-making. Through confidence filtering, non-maximum suppression for deduplication, and coordinate mapping in post-processing, the accuracy and stability of target detection are effectively improved, avoiding false detections and duplicate tracking.

[0040] S300: Dynamically weight the data based on screen orientation and feature information to calculate the overall priority score for each candidate target.

[0041] In this embodiment of the application, for each detected candidate target i Calculate a comprehensive priority score S i This comprehensive priority score S i The weighted calculation formula is as follows:

[0042] In the above formula, S i This indicates the overall priority score. This represents the confidence score. Cnf i Indicates the first i The detection confidence of each candidate target. Represents the centrality fraction. x i and y i Indicates the first i The center point coordinates of the bounding boxes of the candidate targets This indicates the sizing fit fraction. w i and h i They represent the first i The width and height of the bounding boxes of each candidate target. This represents the fractional terms indicating the significance of the motion. Mv i Indicates the first i The motion vectors of the candidate targets Indicates category score items, C i Indicates category, O Indicates the screen orientation. , , , and These represent the weighting coefficients for the confidence score, centrality score, size fitness score, motion significance score, and category score, respectively.

[0043] In the embodiments of this application, the confidence score item You can directly take the first one. i Detection confidence of each candidate target Cnf i Prioritize high-confidence test results.

[0044] In this embodiment, the centrality score term gives higher scores to candidate targets closer to the center of the image. The calculation formula is as follows:

[0045] In the above formula, ( x c , y c () represents the coordinates of the center of the image. D max This represents the maximum distance from the center of the image to the opposite corner.

[0046] In this embodiment, the size fitness fraction avoids tracking objects that are too large or too small. The calculation formula is as follows:

[0047] In the above formula, W frame This represents the total pixel width of the image. H frame This indicates the total pixel height of the image. W frame × H frame This indicates the total number of pixels in the image. R ideal This represents the percentage of the area of ​​an ideal object.

[0048] In the embodiments of this application, the motion saliency score term This can make candidate targets for a movement more prominent, thus increasing the significance score of the movement. Motion vectors can be obtained Mv i The modulus is obtained by normalizing it to the [0,1] interval.

[0049] In the embodiments of this application, the category score item Based on the current screen orientation O Perform dynamic assignment, where: When the screen orientation is O When the screen is in landscape mode, assign high scores, such as 1, to horizontal categories like vehicles, ships, and horizons, and low scores, such as 0.3, to vertical categories like people and tall buildings.

[0050] When the screen orientation is O When the screen is in portrait mode, assign high scores, such as 1, to vertical categories like people, tall buildings, and trees, and low scores, such as 0.3, to horizontal categories like vehicles, ships, and the horizon.

[0051] As an optional implementation of this application, the weight coefficients of the confidence score, centrality score, size fitness score, motion salience score, and category score can be dynamically adjusted according to the screen orientation.

[0052] For example, when the screen orientation is landscape, the weight coefficient of the motion saliency score is increased, and the sensitivity to horizontal motion is improved, while the weight coefficient of the corresponding category score is also increased. When the screen orientation is portrait, the weight coefficient of the centrality score is increased, and the sensitivity to vertical motion is improved, while maintaining a relatively high weight coefficient of the corresponding category score.

[0053] In this embodiment of the application, the sensitivity of motion direction is improved by performing direction weighting within the motion saliency score term, and the steps are as follows: First, decompose the motion vectors. Mv i Decompose it into horizontal and vertical components to obtain the absolute value of the horizontal motion. The absolute value of the vertical direction of motion .

[0054] Then, based on the screen orientation... O Define a direction weighting function To calculate direction-sensitive motion fractions ,in: When the screen orientation is O When in landscape mode, ,in > When the screen orientation is... O When in portrait mode, ,in < .

[0055] For example, landscape parameters =0.9, =0.1 means that the weight of horizontal motion is 9 times the weight of vertical motion, in portrait mode parameters. =0.1, =0.9 means that the weight of vertical motion is 9 times that of horizontal motion.

[0056] Finally, use motion scores Significance of alternative motion fractions Calculate the overall priority score.

[0057] In this application embodiment, improving horizontal or vertical motion sensitivity can better achieve intent understanding and semantic matching. When a user holds the shooting device horizontally, they typically intend to capture content with a wide and expansive feel, such as landscapes, traffic, or group activities. The typical movement pattern of the subjects in these scenes is horizontal. Improving horizontal motion sensitivity allows the shooting device to actively seek out and match these typical moving subjects in horizontal scenes. When a user holds the shooting device vertically, they typically intend to capture tall, deep, and prominent content, such as people, tall buildings, trees, and jumping athletes. The typical movement pattern of these subjects is vertical. Improving vertical motion sensitivity allows the shooting device to prioritize these typical moving subjects in vertical scenes.

[0058] Furthermore, improving motion sensitivity can enhance the accuracy of subject selection and the signal-to-noise ratio. Without directional sensitivity, a small bird hopping vertically up and down in a landscape view might steal the tracking rights of a car moving smoothly in a horizontal direction, because the bird's motion vector magnitude might be larger, which clearly does not align with the user's shooting intentions. In this case, directional sensitivity can be used as a filter; by introducing directional weights, it can effectively suppress interfering motions in non-dominant directions.

[0059] Improving motion sensitivity can also optimize composition and visual comfort. Tracking horizontally moving objects in landscape mode produces smooth and continuous horizontal follow shots, resulting in stable footage that conforms to horizontal composition principles such as the golden ratio. Tracking vertically moving objects in portrait mode produces impactful vertical rise and fall or follow shots, emphasizing height and depth, which aligns with the consumption habits of portrait-oriented content.

[0060] S400. Determine the optimal tracking target based on the comprehensive priority score, and calculate the target deviation between the optimal tracking target and the center of the image based on the feature information of the optimal tracking target.

[0061] Figure 3 This is a third schematic diagram of the focusing method based on screen orientation provided in the embodiments of this application, as shown below. Figure 3 As shown, the calculation of the target deviation specifically includes the following steps: S410. The candidate target with the highest comprehensive priority score is determined as the optimal tracking target.

[0062] In this embodiment of the application, the comprehensive priority scores of all candidate targets are compared, and the candidate target with the highest comprehensive priority score is selected as the optimal tracking target.

[0063] S420. Calculate the target deviation based on the bounding box information of the optimal tracking target and the coordinate information of the center of the image.

[0064] In this embodiment of the application, the target deviation between the optimal tracking target and the center of the image is calculated based on the bounding box position of the optimal tracking target, including the following steps: First, calculate the target center coordinates in the image pixel coordinate system based on the bounding box information. The calculation formula is as follows:

[0065] In the above formula, x target and y target These represent the optimal tracking target in the image pixel coordinate system. x shaft and y The target center coordinates corresponding to the axis.

[0066] Then, the pixel deviation is calculated based on the target center coordinates and the image center coordinates. Specifically, the origin of the coordinate system is moved to the image center, assuming the total pixel width of the image is... W frame The total pixel height of the image is H frame The coordinates of the center of the screen are ( x center , y center )=( W frame / 2, H frame / 2). The formula for calculating pixel deviation is as follows:

[0067] In the above formula, dx pixel Represents the pixel coordinate system of the image x Pixel deviation in the axial direction, dy pixel This represents the pixel deviation along the y-axis in the image pixel coordinate system.

[0068] Finally, based on the field of view of the shooting device, the pixel deviation is converted into angular deviation to obtain the target deviation. Specifically, the field of view of the shooting device is used as the conversion coefficient to obtain the horizontal deviation angle and the vertical deviation angle as the target deviation. The formula for calculating the horizontal deviation angle is as follows:

[0069] In the above formula, dPan Indicates the horizontal deviation angle. This indicates the field of view of the shooting device in the horizontal direction.

[0070] The formula for calculating the vertical deviation angle is as follows:

[0071] In the above formula, dTilt Indicates the vertical deviation angle. This indicates the field of view of the shooting device in the vertical direction.

[0072] The S500 generates control signals based on target deviation to drive the lens of the shooting device to automatically track and focus on the target.

[0073] In this embodiment, a control signal is generated based on the target deviation to drive the lens of the shooting device to perform tracking motion, so that the optimal tracking target is kept in the center of the image, and at the same time the lens focusing motor is driven to ensure that the optimal tracking target is clear.

[0074] For example, Figure 4 This is a schematic diagram of the tracking device in landscape mode in an embodiment of this application, such as... Figure 4 As shown, when the screen orientation is landscape, the optimal tracking target is determined to be the vehicle based on the comprehensive priority score. At this time, the camera lens of the driving shooting device is used to shoot the vehicle according to the tracking frame. Figure 5 This is a schematic diagram of the tracking device in portrait mode in an embodiment of this application, as shown below. Figure 5 As shown, when the screen orientation is portrait, the optimal tracking target is determined to be a person based on the comprehensive priority score. At this time, the camera lens of the shooting device is driven to shoot the person according to the tracking frame.

[0075] As an optional implementation of this application, the step of setting the bounding box of the optimal tracking target as the active focus area and generating a control signal to drive the lens focusing motor includes: firstly, extracting the contrast value of the image within the focus area; then, issuing a command to the lens focusing motor to slowly move a small step in one direction; and recalculating the contrast value of the image within the focus area after the movement. If the contrast increases, it indicates that the focusing direction is correct, and the movement continues in that direction. If the contrast decreases, it indicates that the focusing direction is incorrect, and the lens focusing motor should move in the opposite direction until a contrast peak point is found, ensuring that the optimal tracking target remains sharp. In continuous tracking mode, this process continues to achieve continuous autofocus.

[0076] The screen orientation-based focusing method provided in this application realizes context awareness and intent understanding of intelligent focusing composition based on the screen orientation of the shooting device and multi-dimensional target features. It dynamically adjusts the target priority weight according to the horizontal or vertical screen state, making the shooting composition more in line with the user's intent, and significantly improving the intelligence level of automatic tracking focus and user experience during the shooting process.

[0077] Example 2 Based on the same technical concept as the above embodiments, this application provides a screen orientation-based focus tracking system. Figure 6 This is a schematic diagram of the screen orientation-based focus system structure provided in an embodiment of this application, as shown below. Figure 6 As shown, the screen orientation-based focus system 200 includes: The data acquisition module 210 is used to acquire the screen orientation status and real-time video frames of the shooting device.

[0078] The target detection module 220 is used to perform target detection and feature extraction on real-time video frames to obtain at least one candidate target and its feature information.

[0079] The score calculation module 230 is used to dynamically weight and calculate the comprehensive priority score of each candidate target based on the screen orientation state and feature information.

[0080] The deviation calculation module 240 is used to determine the optimal tracking target based on the comprehensive priority score, and to calculate the target deviation between the optimal tracking target and the center of the image based on the feature information of the optimal tracking target.

[0081] The focus control module 250 is used to generate control signals based on the target deviation, driving the lens of the shooting device to automatically track and focus on the target.

[0082] The screen orientation-based focusing system provided in this application effectively suppresses interference from non-intended targets, improves the accuracy of subject selection and scene adaptability, achieves precise tracking and continuous focusing, and ensures image stability and subject sharpness. It possesses stronger semantic understanding capabilities in complex multi-target scenes, reducing false tracking and frequent switching, making it suitable for applications requiring high compositional aesthetics and ease of operation.

[0083] It is understood that the implementation methods of the screen orientation-based focusing method in the above embodiments are also applicable to the embodiments of this application and can achieve the same technical effect, so they will not be described again here.

[0084] Example 3 Based on the same concept, this application also provides an electronic device. Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as...Figure 7 As shown, the electronic device 300 may include a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute the steps of the screen orientation-based focus method as described in the above embodiments. For example, this includes: S100: Acquire the screen orientation status and real-time video frames of the shooting device.

[0085] S200. Perform target detection and feature extraction on real-time video frames to obtain at least one candidate target and its feature information.

[0086] S300: Calculate the comprehensive priority score of each candidate target by dynamically weighting the results based on the screen orientation and feature information. S400: Determine the optimal tracking target based on the comprehensive priority score, and calculate the target deviation between the optimal tracking target and the center of the image based on the feature information of the optimal tracking target; The S500 generates control signals based on target deviation to drive the lens of the shooting device to automatically track and focus on the target.

[0087] The processor 310 can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0088] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0089] The memory 330 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0090] Example 4 Based on the same concept, embodiments of this application also provide a computer-readable storage medium storing a computer program containing at least one piece of code executable by a master control device to control the master control device to implement the screen orientation-based focus method steps as described in the above embodiments. For example, it includes: S100: Acquire the screen orientation status and real-time video frames of the shooting device.

[0091] S200. Perform target detection and feature extraction on real-time video frames to obtain at least one candidate target and its feature information.

[0092] S300: Calculate the comprehensive priority score of each candidate target by dynamically weighting the results based on the screen orientation and feature information. S400: Determine the optimal tracking target based on the comprehensive priority score, and calculate the target deviation between the optimal tracking target and the center of the image based on the feature information of the optimal tracking target; The S500 generates control signals based on target deviation to drive the lens of the shooting device to automatically track and focus on the target.

[0093] Based on the same technical concept, this application also provides a computer program, which, when executed by a main control device, is used to implement the above-described method embodiments.

[0094] The computer program may be stored, in whole or in part, on a computer-readable storage medium packaged with the processor, or in part or in whole on a memory not packaged with the processor.

[0095] Based on the same technical concept, this application also provides a processor for implementing the above-described method embodiments. The processor can be a chip.

[0096] In summary, the screen orientation-based focusing method, system, electronic device, and storage medium provided in this application, based on the screen orientation of the shooting device and multi-dimensional target features, achieve contextual awareness and intent understanding in intelligent focusing composition. It dynamically adjusts target priority weights according to the horizontal or vertical screen state, making the shooting composition more in line with the user's intent, significantly improving the intelligence level of automatic tracking focus and the user experience during shooting. Simultaneously, it effectively suppresses interference from non-intended targets, improves the accuracy of subject selection and scene adaptability, achieves precise tracking and continuous focusing, and ensures image stability and subject sharpness. It possesses stronger semantic understanding capabilities in complex multi-target scenes, reducing mistracking and frequent switching, making it suitable for applications with high requirements for compositional aesthetics and ease of operation.

[0097] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0098] The embodiments described above are merely examples of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A screen direction-based follow focus method, characterized in that, The method comprises: acquiring a screen orientation state of a shooting device and a real-time video frame; performing target detection and feature extraction on the real-time video frame to obtain at least one candidate target and feature information of the candidate target; performing dynamic weighting according to the screen orientation state and the feature information to calculate a comprehensive priority score of each candidate target; determining an optimal tracking target according to the comprehensive priority score, and calculating a target deviation of the optimal tracking target from a center of an image frame based on feature information of the optimal tracking target; generating a control signal according to the target deviation to drive a lens of the shooting device to perform target automatic tracking and focusing. 2.The screen direction-based follow focus method of claim 1, wherein, The target detection and feature extraction on the real-time video frame to obtain at least one candidate target and feature information of the candidate target comprises: performing image preprocessing on the real-time video frame to obtain an image tensor; inputting the image tensor into a target detection model to obtain at least one initial candidate target and initial feature parameters, wherein the initial feature parameters at least include a bounding box parameter, an object confidence and a class probability; performing post-processing decoding and motion vector calculation according to the initial feature parameters to obtain the candidate target and class identification, detection confidence, bounding box information and motion vector of the candidate target. 3.The screen direction based follow focus method of claim 2, wherein, The calculation formula of the comprehensive priority score is as follows: In the above formulae, S i denotes a comprehensive priority score, denotes a confidence score term, Cnf i denotes a detection confidence of the i th candidate target, denotes a center degree score term, x i and y i denotes a center point coordinate of the i th candidate target bounding box, denotes a size fitness score term, w i and h i denotes a width and a height of the i th candidate target bounding box, respectively, denotes a motion saliency score term, Mv i denotes a motion vector of the i th candidate target, denotes a category score term, C i denotes a category, O denotes the screen orientation state, , , , and denote a confidence score term weight coefficient, a center degree score term weight coefficient, a size fitness score term weight coefficient, a motion saliency score term weight coefficient, and a category score term weight coefficient, respectively. 4.The screen direction based focus following method of claim 3, wherein, The confidence score item weight coefficient, the center degree score item weight coefficient, the size adaptability score item weight coefficient, the motion saliency score item weight coefficient and the class score item weight coefficient are dynamically adjusted according to the screen orientation state, comprising: when the screen orientation state is horizontal, the motion saliency score item weight coefficient is increased, and the sensitivity to horizontal direction motion is increased; when the screen orientation state is vertical, the center degree score item weight coefficient is increased, and the sensitivity to vertical direction motion is increased.

5. The screen direction-based follow focus method of claim 4, wherein, The increase of the sensitivity to horizontal direction motion comprises: decomposing the motion vector of the candidate target into a horizontal component and a vertical component, giving the horizontal component a higher weight than the vertical component, and recalculating the motion saliency score item based on the weighted motion vector length; The increase of the sensitivity to vertical direction motion comprises: decomposing the motion vector of the candidate target into a horizontal component and a vertical component, giving the vertical component a higher weight than the horizontal component, and recalculating the motion saliency score item based on the weighted motion vector length. 6.The screen direction based focus following method of claim 3, wherein, The determination of the optimal tracking target according to the comprehensive priority score and the calculation of the target deviation of the optimal tracking target from the center of the image frame based on the feature information of the optimal tracking target comprises: determining the candidate target with the highest comprehensive priority score as the optimal tracking target; calculating the target deviation according to the bounding box information of the optimal tracking target and the picture center coordinate information of the center of the image frame. 7.The screen direction based follow focus method of claim 6, wherein, The calculation of the target deviation according to the bounding box information of the optimal tracking target and the coordinate information of the center of the image frame comprises: According to the boundary box information, target center coordinate information of the optimal tracking target in an image pixel coordinate system is calculated; According to the target center coordinate information and the picture center coordinate information, pixel deviation is calculated; According to the field of view angle of the shooting device, the pixel deviation is converted into angle deviation, and the target deviation is obtained.

8. A screen direction based follow focus system, characterized by, The system comprises: A data acquisition module is configured to acquire a screen orientation state of a shooting device and a real-time video frame; A target detection module is configured to perform target detection and feature extraction on the real-time video frame, and obtain at least one candidate target and feature information of the candidate target; A score calculation module is configured to perform dynamic weighting according to the screen orientation state and the feature information, and calculate a comprehensive priority score of each candidate target; A deviation calculation module is configured to determine an optimal tracking target according to the comprehensive priority score, and calculate target deviation of the optimal tracking target from a picture center based on feature information of the optimal tracking target; A focus control module is configured to generate a control signal according to the target deviation, and drive a lens of the shooting device to perform target automatic tracking and focusing.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the screen orientation based focus method according to any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the screen orientation based focus method according to any one of claims 1-7.

Images