Following zoom control method and device, electronic equipment and storage medium

Abstract

The invention relates to a following zoom control method and device, electronic equipment and a storage medium. According to the technical scheme provided by the invention, the target frame marked in the video frame is taken as a reference, the preset target frame proportion threshold value is taken as a benchmark, and the zoom moving direction is determined according to the relationship between the target frame in the plurality of video frames and the target frame proportion threshold value along with the continuity of the video shooting process; the frame number is adjusted according to the preset multiplying power to control the zooming speed, so that the moving direction and the moving step length of the zooming motor are determined when each frame is shot in the future, and finally the target frame in the video frame is stably adjusted to be close to the size corresponding to the target frame proportion threshold value along with video shooting. By means of the technical scheme, smooth zooming in the tracking process of the target object is achieved, and the target object can be presented in the video in a proper size by setting the target frame proportion threshold value of the proper size.

Description

Technical Field

[0001] The present disclosure relates to the field of camera technology, and in particular to a follow-up zoom control method, device, electronic device and storage medium. Background Art

[0002] In the field of camera technology, when capturing large scene images or long-distance images, when a target object appears in the image (the target object may be stationary or moving), in order to observe the target object more clearly, it is necessary to control the zoom (ZOOM) motor to adjust the focal length, and then scale the target object to the appropriate size in the image. Here, scaling to the appropriate size is zoom (zoom). During the zoom (zoom) process, it is necessary to keep the target object clear in real time. When the target object is adjusted to the appropriate size, it is necessary to use the control of the focus (FOCUS) motor to achieve rapid focus on the target object and keep the image of the target object clear. In this process, in order to ensure that the target object is not lost, the pan / tilt can also be mobilized to ensure that the target object is in the center of the picture in real time. In this way, the target object can be tracked throughout the process and the target object can be kept clear in real time to ensure image quality. Summary of the invention

[0003] In view of this, the present disclosure provides a follow-up zoom control method, device, electronic device and storage medium to achieve smooth zooming in the process of tracking a target object, and on this basis, ensure clear imaging of the target object in the process of smooth zooming.

[0004] The technical solution of the present disclosure is achieved as follows:

[0005] A follow-up zoom control method includes the following follow-up zoom control process:

[0006] Acquire a video frame captured in real time by a camera device and a target frame marked in the video frame;

[0007] In the first set number of consecutive video frames most recently shot, an average target frame ratio value is obtained according to the size of the target frame in each video frame and the size of the video frame, wherein the average target frame ratio value is an average value of the proportion of the target frame in the first set number of consecutive video frames in the video frame;

[0008] Obtaining a zoom movement direction of the zoom motor according to the target frame average ratio value, a preset target frame ratio threshold, and a current magnification of the camera device;

[0009] Obtaining a moving step length of the zoom motor in a process from shooting a current video frame to shooting a next video frame according to a proportion of a target frame in a current video frame, a target frame ratio threshold, a preset magnification adjustment frame number, the current magnification, a zoom magnification table of the camera device, and a zoom movement direction of the zoom motor;

[0010] According to the zoom movement direction and the movement step length of the zoom motor, the zoom motor is controlled to move from the current zoom position to the next frame zoom position, so that the camera device shoots the next video frame when the zoom motor is in the next frame zoom position.

[0011] Furthermore, in the first set number of consecutive video frames captured most recently, according to the size of the target frame in each video frame and the size of the video frame, obtaining the average ratio value of the target frame includes:

[0012] Obtaining, according to the geometric measurement values ​​of the target frames in the respective video frames, the deviations of the geometric measurement values ​​of the target frames in the respective video frames;

[0013] From the first set number of continuous video frames, filter out the video frames where the target frames whose absolute values ​​of the deviations are greater than a set screening threshold are located;

[0014] For each video frame that is not filtered out in the first set number of continuous video frames, dividing the geometric measurement value of the target frame in the video frame by the geometric measurement value of the image in the video frame to obtain a target frame ratio value;

[0015] An average value of the target frame ratio values ​​of all video frames that are not filtered out of the first continuous set number of video frames is determined as the target frame average ratio value.

[0016] Further, obtaining the zoom movement direction of the zoom motor according to the target frame average ratio value, a preset target frame ratio threshold, and the current magnification of the camera device includes:

[0017] The average difference ratio of the target box is obtained according to the following formula:

[0018] RRavg=T1 / Ravg

[0019] Wherein, T1 is the target frame ratio threshold, Ravg is the target frame average ratio value, and RRavg is the target frame average difference ratio;

[0020] The target magnification of the camera device is obtained according to the following formula:

[0021] tarLensRatio=curLensRatio*RRavg

[0022] Wherein, curLensRatio is the current magnification, and tarLensRatio is the target magnification;

[0023] According to the target magnification, obtaining a target zoom position of the zoom motor corresponding to the target magnification by querying a zoom ratio table of the camera device;

[0024] The zoom movement direction of the zoom motor is obtained according to the current zoom position of the zoom motor and the target zoom position, wherein:

[0025] If the target zoom position is greater than the current zoom position, the zoom movement direction of the zoom motor is to zoom in the direction of a larger magnification;

[0026] If the target zoom position is smaller than the current zoom position, the zoom movement direction of the zoom motor is to zoom in a direction of smaller magnification; or

[0027] If the target zoom position is equal to the current zoom position, the zoom motor does not perform zoom movement.

[0028] Further, the step length of movement of the zoom motor in the process from shooting the current video frame to shooting the next video frame is obtained according to the proportion of the target frame in the current video frame, the target frame ratio threshold, the preset magnification adjustment frame number, the current magnification, the zoom magnification table of the camera device and the zoom movement direction of the zoom motor, including:

[0029] According to the proportion of the target frame in the current video frame, the target frame ratio threshold, the preset magnification adjustment frame number, the current magnification and the zoom movement direction of the zoom motor, the next frame magnification is obtained, wherein the next frame magnification is the magnification corresponding to shooting the next video frame;

[0030] According to the current magnification, the next frame magnification and the zoom ratio table of the camera device, the moving step length of the zoom motor in the process from shooting the current video frame to shooting the next video frame is obtained.

[0031] Further, the magnification of the next frame is obtained according to the proportion of the target frame in the current video frame, the target frame ratio threshold, the preset magnification adjustment frame number, the current magnification and the zoom movement direction of the zoom motor, including:

[0032] Obtaining a reference speed for rate adjustment according to a proportion of a target frame in the current video frame and a target frame ratio threshold;

[0033] The next frame magnification is obtained according to the magnification adjustment reference speed, the magnification adjustment frame number, the current magnification and the zoom movement direction of the zoom motor.

[0034] Further, obtaining the reference speed for rate adjustment according to the proportion of the target frame in the current video frame and the target frame ratio threshold comprises:

[0035] The current difference ratio of the target frame is obtained according to the following formula:

[0036] RRcur=T1 / Rcur

[0037] And, if RRcur is less than 1, let

[0038] RRcur=1 / RRcur

[0039] Wherein, T1 is the target frame ratio threshold, Rcur is the proportion in the current video frame, and RRcur is the current difference ratio of the target frame;

[0040] The ratio adjustment reference speed is obtained according to the following formula:

[0041] Speed ​​= A RRcur -B

[0042] Among them, A and B are both adjustment parameters, A is equal to B, Speed ​​is the ratio adjustment reference speed, and when the ratio adjustment reference speed is less than the first set speed, the ratio adjustment reference speed is set to the first set speed, and when the ratio adjustment reference speed is greater than the second set speed, the ratio adjustment reference speed is set to the second set speed, wherein the first set speed is less than the second set speed.

[0043] Further, obtaining the next frame magnification according to the magnification adjustment reference speed, the magnification adjustment frame number, the current magnification and the zoom movement direction of the zoom motor includes:

[0044] The single-frame moving speed is obtained according to the following formula:

[0045] fpsSpeed=Speed / NUM

[0046] Wherein, Speed ​​is the reference speed for the magnification adjustment, NUM is the number of frames for the magnification adjustment, and fpsSpeed ​​is the single-frame moving speed;

[0047] When the zoom movement direction of the zoom motor is to zoom in the direction of a larger magnification, the magnification of the next frame is obtained according to the following formula:

[0048] nextLensRatio=curLensRatio*(1+fpsSpeed)

[0049] When the zoom movement direction of the zoom motor is to zoom in the direction of small magnification, the magnification of the next frame is obtained according to the following formula:

[0050] nextLensRatio=curLensRatio*(1-fpsSpeed)

[0051] When the zoom motor does not perform zoom movement, the next frame magnification is obtained according to the following formula:

[0052] nextLensRatio=curLensRatio

[0053] Wherein, curLensRatio is the current magnification ratio, and nextLensRatio is the next frame magnification ratio.

[0054] Further, obtaining the moving step length of the zoom motor in the process from shooting the current video frame to shooting the next video frame according to the current magnification, the next frame magnification and the zoom ratio table of the camera device includes:

[0055] Obtaining a current zoom position of a zoom motor in the camera device according to the current magnification and a zoom ratio table of the camera device;

[0056] Obtaining a zoom position for a next frame according to the next frame magnification and the zoom ratio table of the camera device, wherein the zoom position for the next frame is a zoom position of the zoom motor when shooting a next video frame;

[0057] The moving step length of the zoom motor is obtained according to the current zoom position and the next frame zoom position.

[0058] Further, obtaining the moving step length of the zoom motor according to the current zoom position and the next frame zoom position includes:

[0059] The moving step length of the zoom motor is obtained according to the following formula:

[0060] curZoomStep=|nextZoomPos-curZoomPos|

[0061] Among them, nextZoomPos is the zoom position of the next frame, curZoomPos is the current zoom position, and curZoomStep is the moving step length of the zoom motor.

[0062] Further, the following zoom control method further includes:

[0063] In the case that the target frame is not included in the second consecutive set number of video frames of the latest shot, end the following zoom, and clear the size of the target frame and the average target frame ratio value in each video frame associated with the first consecutive set number of video frames;

[0064] In the case that the zoom motor zooms and moves in the direction of a large magnification, when Rcur>T1+Ttol / 2 and Ravg>T1, stop the movement of the zoom motor; or

[0065] In the case that the zoom motor zooms and moves in the direction of a small magnification, when Rcur<T1-Ttol / 2 and Ravg<T1, stop the movement of the zoom motor;

[0066] Wherein, Rcur is the ratio of the target frame in the current video frame to the current video frame, T1 is the target frame ratio threshold, Ttol is the preset tolerance ratio threshold, Ravg is the average target frame ratio value, and the target zoom position is obtained according to the target magnification and the zoom ratio table of the imaging device.

[0067] Further, the following zoom control method further includes:

[0068] When the target frame marked in the video frame is first obtained, according to the object category marked by the target frame, obtain the target frame ratio threshold associated with the object category.

[0069] Further, before acquiring the video frames captured in real time by the imaging device and the target frames marked in the video frames, the following zoom control method further includes:

[0070] When a first video frame containing a target frame that meets the set ratio condition appears in the video frames captured in real time by the imaging device, obtain the clarity evaluation value of the target area where the target frame is located, and perform fast autofocus according to the clarity evaluation value to focus the target area where the detection frame is located clearly.

[0071] Further, the set ratio condition is:

[0072] Rcur≥T1 / 2+Ttol

[0073] Wherein, Rcur is the ratio of the target frame in the current video frame to the current video frame, T1 is the target frame ratio threshold, and Ttol is the preset tolerance ratio threshold.

[0074] Further, the fast autofocus includes:

[0075] Obtaining a theoretical focus value from a lens curve table of the camera device according to the object distance value before the fast autofocus starts and the current zoom position of the zoom motor;

[0076] Obtaining a focus value variation range according to the theoretical focus value and a preset focal length extension parameter;

[0077] The focus motor is driven to move within the focus value variation range, and as the camera device shoots, focusing is completed within a preset number of video frames according to the variation of the clarity evaluation value.

[0078] Further, the step of driving the focus motor to move within the focus value variation range and completing the focus according to the change of the clarity evaluation value within a preset number of video frames as the camera device shoots, includes:

[0079] The number of steps of the focus motor moving corresponding to each frame in the preset number of video frames is obtained according to the following formula:

[0080] fStep1=(B2-curFocusPos) / M

[0081] Wherein, fStep1 is the number of steps of the focus motor corresponding to each frame in the preset number of video frames, curFocusPos is the current focus position of the focus motor, M is the preset number, and B2 is the upper limit value of the focus value variation range;

[0082] The focus motor is driven to move fStep1, and a definition evaluation value of a target area where the target frame is located in the video frame after the focus motor moves is obtained;

[0083] When the clarity evaluation value obtained with the movement of the focus motor decreases, the number of steps of the focus motor movement corresponding to each frame that has not been moved in the preset number of video frames is obtained according to the following formula:

[0084] fStep2=(curFocusPos-B1) / (MN)

[0085] Wherein, fStep2 is the number of steps of the focus motor corresponding to each frame that has not been passed in the preset number of video frames, N is the number of frames that have been passed in the preset number of video frames, and B1 is the lower limit value of the focus value variation range;

[0086] Drive the focus motor to move fStep2, and obtain a definition evaluation value of a target area where the target frame is located in the video frame after the focus motor moves;

[0087] When all the preset number of video frames have passed, the focus motor is controlled to move to a focus position of the focus motor corresponding to the maximum clarity evaluation value obtained in the preset number of video frames, so as to complete the focusing.

[0088] Furthermore, the following zoom control method further includes the following steps which are simultaneously performed during the following zoom control process:

[0089] Obtaining a current object distance value according to a current zoom position of the zoom motor, a current focus position of a focus motor in the camera device, and a lens curve table of the camera device;

[0090] Obtaining a predicted object distance for the next frame according to the proportion of the target frame in the current video frame, the target frame ratio threshold, the frame rate of the video frame, a preset object distance adjustment factor, and the current object distance value;

[0091] According to the next frame predicted object distance and the current zoom position of the zoom motor, the next frame predicted focus position of the focus motor is obtained through the lens curve table;

[0092] The focus motor is controlled to move to the next frame predicted focus position, so as to shoot the next video frame when the focus motor is located at the next frame predicted focus position.

[0093] Further, the predicted object distance of the next frame is obtained according to the proportion of the target frame in the current video frame, the target frame ratio threshold, the frame rate of the video frame, the preset object distance adjustment factor, and the current object distance value, including the following process performed when the zoom movement direction of the zoom motor is zoom movement in the direction of a large magnification:

[0094] The current difference ratio of the target frame is obtained according to the following formula:

[0095] RRcur=T1 / Rcur

[0096] Wherein, RRcur is the current difference ratio of the target frame;

[0097] The object distance limit offset is obtained according to the following formula:

[0098] distOffset = RRcur * adjFactor1

[0099] Wherein, adjFactor1 is the object distance adjustment factor, and distOffset is the object distance limit offset;

[0100] According to the current object distance value and the object distance limit offset, an object distance limit range is obtained, wherein the object distance limit range is [curDist-distOffset, curDist+distOffset];

[0101] A zoom tracking method is used to obtain the predicted object distance of the next frame, wherein if the predicted object distance of the next frame is less than the lower limit value of the object distance limit range, the predicted object distance of the next frame is set to the lower limit value of the object distance limit range; if the predicted object distance of the next frame is greater than the upper limit value of the object distance limit range, the predicted object distance of the next frame is set to the upper limit value of the object distance limit range.

[0102] Further, the predicted object distance of the next frame is obtained according to the proportion of the target frame in the current video frame, the target frame ratio threshold, the frame rate of the video frame, the preset object distance adjustment factor, and the current object distance value, including the following process performed when the zoom movement direction of the zoom motor is zoom movement in the direction of small magnification:

[0103] The current difference ratio of the target frame is obtained according to the following formula:

[0104] RRcur=T1 / Rcur

[0105] Wherein, RRcur is the current difference ratio of the target frame;

[0106] The object distance adjustment speed is obtained according to the following formula:

[0107] distSpeed ​​= RRcur / fps

[0108] Wherein, fps is the frame rate of the video frame, and distSpeed ​​is the object distance adjustment speed;

[0109] The predicted object distance of the next frame is obtained according to the following formula:

[0110] predictDist=curDist-distSpeed

[0111] Wherein, curDist is the current object distance value, predictDist is the predicted object distance of the next frame, wherein, if the predicted object distance of the next frame is less than the minimum object distance supported by the camera device, the predicted object distance of the next frame is set to the minimum object distance supported by the camera device.

[0112] A follow-up zoom control device, comprising:

[0113] A video frame acquisition module is configured to acquire a video frame captured in real time by a camera device and a target frame marked in the video frame;

[0114] The target frame average ratio value obtaining module is configured to execute, in the first consecutive set number of video frames taken most recently, obtaining the target frame average ratio value according to the size of the target frame in each video frame and the size of the video frame, wherein the target frame average ratio value is the average value of the proportion of the target frame in the first consecutive set number of video frames in the video frame;

[0115] A zoom movement direction obtaining module is configured to obtain the zoom movement direction of the zoom motor according to the target frame average ratio value, a preset target frame ratio threshold value, and the current magnification of the camera device;

[0116] A zoom motor movement step length obtaining module is configured to obtain the movement step length of the zoom motor in the process from shooting the current video frame to shooting the next video frame according to the proportion of the target frame in the current video frame, the target frame ratio threshold, the preset magnification adjustment frame number, the current magnification, the zoom magnification table of the camera device and the zoom movement direction of the zoom motor;

[0117] The zoom motor movement control module is configured to control the zoom motor to move from the current zoom position to the next frame zoom position according to the zoom movement direction of the zoom motor and the movement step of the zoom motor, so that the camera device can shoot the next video frame when the zoom motor is in the next frame zoom position.

[0118] An electronic device, comprising:

[0119] processor;

[0120] a memory for storing executable instructions for the processor;

[0121] Wherein, the processor is configured to execute the executable instructions to implement the follow-up zoom control method as described in any one of the above items.

[0122] A computer-readable storage medium, when at least one instruction in the computer-readable storage medium is executed by a processor of an electronic device, enables the electronic device to implement any of the above-mentioned follow-up zoom control methods.

[0123] It can be seen from the above scheme that the following zoom control method, device, electronic device and storage medium disclosed in the present invention use the target frame marked in the video frame as a reference and the preset target frame ratio threshold as a benchmark. As the video shooting process continues, the zoom movement direction is determined according to the relationship between the target frame and the target frame ratio threshold in multiple video frames, and the zoom speed is controlled by adjusting the number of frames according to the preset magnification, thereby determining the movement direction and movement step of the zoom motor when shooting each frame in the future, and finally making the target frame in the video frame smoothly adjusted to the size corresponding to the target frame ratio threshold as the video is shot. Since the target frame marked in the video frame is obtained according to the target object in the video frame, therefore, making the adjustment in the video frame to the size corresponding to the target frame ratio threshold means that the target object in the video frame is smoothly adjusted to the size corresponding to the target frame ratio threshold as the zoom motor moves, therefore, setting the target frame ratio threshold of an appropriate size can obtain the presentation of the target object in the video at the appropriate size. It can be seen that the present invention realizes smooth zooming in the process of tracking the target object.

[0124] On this basis, in the following zoom control method, device, electronic device and storage medium disclosed in the present invention, when the first frame containing a target frame that meets the set ratio condition appears in the video frame captured in real time by the camera device, fast auto focus is performed within a preset number of video frames according to the clarity evaluation value of the target area where the target frame is located, so as to clearly focus the target area where the detection frame is located. Moreover, in the following zoom process, the following zoom control method, device, electronic device and storage medium disclosed in the present invention, under the constraints of the target frame ratio threshold, the frame rate of the video frame and other conditions, use the lens curve table of the camera device and follow the zoom position change of the zoom motor to obtain the corresponding focus position of the focus motor, so as to control and adjust the focus position of the focus motor in real time, and realize the smooth adjustment of the dynamic focus and dynamic zoom process.

[0125] In summary, the follow-up zoom control method, device, electronic device and storage medium disclosed in the present invention realize smooth adjustment of the zoom motor and the focus motor by following the changes in the size of the target frame in the video frame, so that the video image can be presented smoothly and silkily while keeping the clarity of the target object as the target object in the target frame moves away from or approaching the camera device. BRIEF DESCRIPTION OF THE DRAWINGS

[0126] Figure 1 is a flow chart of a follow-up zoom control method according to an exemplary embodiment;

[0127] Figure 2 is a flow chart showing a method of obtaining an average ratio value of a target frame according to an exemplary embodiment;

[0128] Figure 3is a flow chart showing a method of obtaining a zoom movement direction of a zoom motor according to an exemplary embodiment;

[0129] Figure 4 is a flow chart showing a method of obtaining a moving step length of a zoom motor according to an exemplary embodiment;

[0130] Figure 5 is a flow chart showing a method of obtaining a magnification ratio of a next frame according to an exemplary embodiment;

[0131] Figure 6 is a flow chart showing a method of obtaining a moving step length of a zoom motor according to an exemplary embodiment;

[0132] Figure 7 is a flow chart showing a fast auto-focusing process according to an exemplary embodiment;

[0133] Figure 8 is a flow chart showing the control of the movement of a focus motor during a fast autofocus process according to an exemplary embodiment;

[0134] Fig. 9 is a flow chart showing a method of controlling a focus motor to achieve focusing during a follow zoom process according to an exemplary embodiment;

[0135] Fig.10 is a logical structure diagram of a follow-up zoom control device according to an exemplary embodiment;

[0136] Fig.11 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0137] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below with reference to the accompanying drawings and examples.

[0138] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0139] In the application fields of video surveillance and video conferencing, the technology of real-time zooming following the movement of the target object is of great help in improving the image presentation effect. For example, in the field of video conferencing, after obtaining the target speaker information, the camera device automatically adjusts the magnification and focuses to keep a clear picture of the speaker in real time, and the same is true after switching speakers; in the education industry, during the video course recording process, the camera device can follow the teacher's movement during the teacher's lecture to record clearly.

[0140] However, there are still some problems in the current technology of real-time zooming following the movement of the target object. For example, the whole process of zoom tracking a moving object is intermittent, the image is stuck, the image of the target object is blurred during the zoom tracking process, the focusing process is slow after the tracking is completed, and the image is easily blurred after the target object tracking is lost.

[0141] In response to the above problems, the embodiments of the present disclosure provide a follow-up zoom control method, device, electronic device and storage medium, which realize smooth zooming in the process of tracking the target object. On this basis, clear imaging of the target object can also be ensured during the smooth zooming process.

[0142] Figure 1 is a flow chart of a follow zoom control method according to an exemplary embodiment. Figure 1 As shown, the following zoom control method mainly includes the following following zoom control process from step 101 to step 105.

[0143] Step 101: Obtain a video frame captured in real time by a camera device and a target frame marked in the video frame.

[0144] In an illustrative embodiment, the target frame may be a target frame obtained by analyzing a video frame according to an AI (artificial intelligence) method, wherein the AI ​​method may be a target detection method based on a neural network model such as YOLO. The AI ​​method obtains the target frame and the type of the target object by detecting the target object in the video frame. The target object includes but is not limited to: a human face, a human body, a non-motor vehicle, a motor vehicle, a license plate, an animal, etc.

[0145] Step 102: In the first consecutive set number of video frames most recently shot, an average target frame ratio value is obtained according to the size of the target frame in each video frame and the size of the video frame, wherein the average target frame ratio value is the average value of the proportion of the target frame in the first consecutive set number of video frames in the video frame.

[0146] In the embodiment of the present disclosure, the first continuous setting number is represented as T2, and the target frame average ratio value is represented as Ravg.

[0147] When the target object appears in a video frame, it will continue to exist in the video frame for a period of time, and then the AI ​​method marks the target box in continuous video frames by continuously monitoring the video frames.

[0148] In the disclosed embodiment, starting from the average ratio value of the target frame, follow-up zoom control is implemented according to the average ratio value of the target frame, which can achieve smooth zooming in the process of tracking the target object, and can reduce the disturbance of the zooming process caused by the sudden enlargement or sudden reduction of the target frame in a certain video frame, thereby avoiding the problem of video presentation effect being interrupted due to zoom jump caused by the sudden change of the target frame.

[0149] In an exemplary embodiment, the first continuous set number may be 4 to 8, preferably 4, that is, the first continuous set number of video frames may be 4 to 8 consecutive frames, preferably, the first continuous set number of video frames is 4 consecutive frames.

[0150] Figure 2 is a flow chart showing a method of obtaining an average ratio value of a target frame according to an exemplary embodiment, as shown in FIG. Figure 2 As shown, the step 102 of "obtaining an average target frame ratio value according to the size of the target frame in each video frame and the size of the video frame in the first set number of consecutive video frames shot most recently" may include the following steps 201 to 204.

[0151] Step 201: Obtain the degree of deviation of the geometric metric value of the target frame in each video frame according to the geometric metric value of the target frame in each video frame.

[0152] Step 202: From a first set number of continuous video frames, filter out the video frames where the target frames have an absolute value of the deviation greater than a set screening threshold.

[0153] Step 203: For each video frame that is not screened out in the first set number of consecutive video frames, divide the geometric measurement value of the target frame in the video frame by the geometric measurement value of the image in the video frame to obtain a target frame ratio value.

[0154] Step 204 : Determine the average of the target frame ratio values ​​of all video frames that are not filtered out in the first continuous set number of video frames as the target frame average ratio value.

[0155] In an exemplary embodiment, the geometrical measurement value is a width value, a height value or a diagonal length value, and an appropriate geometrical measurement value can be set according to the type of the target object. The following description takes the width value as an example.

[0156] Take the first continuous set number of video frames as 4 consecutive frames as an example:

[0157] In step 201, in the latest 4 consecutive video frames, according to the width value of the target frame in each video frame, the deviation of the width value of the target frame in each video frame in the 4 consecutive video frames is obtained;

[0158] In step 202, the video frames where the target frames whose absolute values ​​of deviations are greater than a set screening threshold are located are screened out from the four consecutive video frames;

[0159] In step 203, for each video frame that is not filtered out in the four consecutive video frames, the width value of the target frame in the video frame is divided by the width value of the image in the video frame to obtain a target frame ratio value.

[0160] In step 204, the average value of the target frame ratio values ​​of all the video frames that are not filtered out in the four consecutive video frames is determined as the target frame average ratio value.

[0161] In an exemplary embodiment, the calculation formula of the deviation is:

[0162] Deviation = (width - average) / standard deviation

[0163] The standard deviation can also be called mean square error.

[0164] For example, the width values ​​of the target frame in each of the four consecutive video frames are 50, 55, 45, and 70, respectively, and the screening threshold is set to 1.5. Then, according to the embodiment of the present disclosure, in step 201, the average value of the width values ​​of the target frame in the four consecutive video frames is 55, and the standard deviation of the width values ​​of the target frame in each of the four consecutive video frames is 9.35. According to the above deviation formula, the deviations of the width values ​​of the target frame in the four consecutive video frames are -0.53, 0, -1.07, and 1. 6, where the absolute value of the deviation degree 1.6 of the target frame width value of 70 exceeds the set screening threshold, then in step 202, the video frames corresponding to the target frame width value of 70 are screened out, leaving the video frames corresponding to the three target frame width values ​​of 50, 55, and 45, and in step 203, the target frame ratio values ​​corresponding to the three target frame width values ​​of 50, 55, and 45 are obtained, and in step 204, the average value of the target frame ratio values ​​corresponding to the three target frame width values ​​of 50, 55, and 45 is determined as the target frame average ratio value.

[0165] The above steps 201 to 204 implement an abnormality check on the target frame, and determine the target frame whose absolute value of deviation is greater than the set screening threshold as an abnormal target frame for screening out, which can avoid the interference of abnormal size target frame on follow zoom control.

[0166] In an exemplary embodiment, in addition to the determination of whether the target frame between the continuous multiple video frames is abnormal in the above steps 201 to 204, a step of determining and filtering out the abnormal size of the target frame itself may be further included. In an exemplary embodiment, this step is performed before step 201. This step may include: comparing the aspect ratio of the target frame with a preset aspect ratio range; if the aspect ratio is outside the aspect ratio range, determining that the size of the target frame itself is abnormal, filtering out video frames with abnormal target frame sizes; if the aspect ratio is within the aspect ratio range, determining that the size of the target frame itself is normal, and retaining video frames with normal target frame sizes. The length-to-width ratio of the target frame obtained for the target object is usually within a reasonable ratio range corresponding to the ratio of the target object. If the length-to-width ratio of the target frame exceeds the reasonable ratio range, it can be determined that the target frame is an abnormal target frame. For example, if the width of the target frame obtained by face recognition is much greater than the length or the length is much greater than the width, it can be determined that the area in the target frame may not be a face or the shooting angle of the face is too large to make it meaningful to perform follow-up zoom control.

[0167] Step 103 : Obtain the zoom movement direction of the zoom motor according to the target frame average ratio value, the preset target frame ratio threshold value, and the current magnification of the camera device.

[0168] The magnification refers to the ratio between the image size of an object and the actual object size. The magnification is usually determined by the size of the image sensor and the focal length of the lens. The larger the magnification, the larger the size of the same object in the imaging area.

[0169] In the disclosed embodiment, the target frame ratio threshold is represented as T1, and the current magnification is represented as curLensRatio.

[0170] Figure 3 FIG. 1 is a flow chart showing a method for obtaining a zoom movement direction of a zoom motor according to an exemplary embodiment. Figure 3 As shown, step 103 may include the following steps 301 to 304 .

[0171] Step 301: Obtain the average difference ratio of the target frame according to the following formula:

[0172] RRavg=T1 / Ravg

[0173] Among them, T1 is the target box ratio threshold, Ravg is the average ratio value of the target box, and RRavg is the average difference ratio of the target box.

[0174] Among them, the target frame ratio threshold reflects the expected ratio of the target frame presented in the video to the video frame. The expected ratio can be the best ratio that the video viewer hopes to present. For example, the video viewer hopes that the face presented in the video image is not too large or too small, and it is best to control the width of the face to account for 1 / 6 to 1 / 3 of the width of the video frame. The target frame ratio threshold can be set in the range of 1 / 6 to 1 / 3. The target frame average difference ratio reflects the difference between the target frame ratio threshold and the target frame average ratio value. According to the difference, that is, according to the target frame average difference ratio, the target frame average ratio value can be made as close to the target frame ratio threshold as possible by following the zoom control. For example, if the target frame average difference ratio is less than 1, it indicates that the target frame ratio threshold is less than the target frame average ratio value, that is, the target frame average ratio value is too large (meaning that the target object in the video image is too large). At this time, it is necessary to move the zoom motor in the camera device to a small magnification direction to make the target frame average ratio value smaller, thereby reducing the target object in the video image; if the target frame average difference ratio is greater than 1, it indicates that the target frame ratio threshold is greater than the target frame average ratio value, that is, the target frame average ratio value is too small (meaning that the target object in the video image is too small). At this time, it is necessary to move the zoom motor in the camera device to a large magnification direction to make the target frame average ratio value larger, thereby enlarging the target object in the video image.

[0175] Based on the above explanation of the average difference ratio of the target frame, a suitable target magnification can be obtained through the following step 302 .

[0176] Step 302: Obtain the target magnification of the camera device according to the following formula:

[0177] tarLensRatio=curLensRatio*RRavg

[0178] Among them, curLensRatio is the current magnification, and tarLensRatio is the target magnification.

[0179] Step 303: According to the target magnification, a target zoom position of the zoom motor corresponding to the target magnification is obtained by querying a zoom ratio table of the camera device.

[0180] Step 304: Obtain the zoom movement direction of the zoom motor according to the current zoom position and the target zoom position of the zoom motor.

[0181] In the exemplary embodiment, step 304 is divided into three cases, including:

[0182] Case 1: The target zoom position is greater than the current zoom position;

[0183] Case 2: the target zoom position is smaller than the current zoom position; and

[0184] Case 3: The target zoom position is equal to the current zoom position.

[0185] Among them, in the target frame presentation of the image, case 1 corresponds to the target frame ratio threshold being greater than the target frame average ratio value, case 2 corresponds to the target frame ratio threshold being less than the target frame average ratio value, and case 3 corresponds to the target frame ratio threshold being equal to the target frame average ratio value.

[0186] Based on the above three situations, step 103 includes the following processing solutions:

[0187] Solution 101: If the target zoom position is greater than the current zoom position, the zoom movement direction of the zoom motor is to zoom in the direction of a larger magnification;

[0188] Solution 102: If the target zoom position is smaller than the current zoom position, the zoom movement direction of the zoom motor is to zoom in a direction of smaller magnification; or

[0189] Solution 103: If the target zoom position is equal to the current zoom position, the zoom motor does not perform zoom movement.

[0190] Among them, situation 1 means that the target object in the video image is too small, so the target object in the video image is enlarged through solution 1; situation 2 means that the target object in the video image is too large, so the target object in the video image is reduced through solution 2; situation 3 means that the target object in the video image is of moderate size, so the size of the target object in the video image is maintained through solution 3.

[0191] The above schemes 101 to 103 are to control the zoom movement of the zoom motor after the zoom movement of the zoom motor is triggered. As to whether to trigger the zoom movement of the zoom motor, if it is designed that the zoom movement of the zoom motor is triggered when the average ratio value of the target frame changes from being equal to the target frame ratio threshold value to being slightly less than or greater than the average ratio value of the target frame, then there will be continuous zoom movement control of the zoom motor as the target object moves. This situation will produce a visual effect that the content in the video frame is enlarged or reduced at any time, especially when the target object frequently and repeatedly moves away from and approaches the camera device in a small range. This situation is very likely to cause repeated zooming of video content outside the target frame range, which is very likely to cause visual fatigue. Therefore, considering this point, it is preferably necessary to appropriately reduce the trigger sensitivity of the zoom movement of the zoom motor. In an illustrative embodiment, the zoom movement of the zoom motor is triggered in the following manner to reduce the trigger sensitivity and eliminate visual fatigue:

[0192] 1) When T1 - Ttol ≤ Ravg ≤ T1 + Ttol, the zoom movement of the zoom motor is not triggered. This situation is equivalent to the size of the current target object in the video frame being within the tolerance range;

[0193] 2) When Ravg < T1 - Ttol, that is, when the target is smaller than the lower limit of the tolerance range, triggering zooming in to a larger magnification; that is, if the target zoom position is greater than the current zoom position and Ravg < T1 - Ttol, the zoom motor zooms in the direction of a larger magnification;

[0194] 3) When Ravg > T1 + Ttol, that is, when the target is larger than the upper limit of the tolerance range, triggering zooming out to a smaller magnification; that is, if the target zoom position is smaller than the current zoom position and Ravg > T1 + Ttol, the zoom motor zooms out in the direction of a smaller magnification.

[0195] Wherein, Ttol is a preset tolerance ratio threshold.

[0196] Step 104: Obtain the moving step of the zoom motor in the process from shooting the current video frame to shooting the next video frame according to the ratio of the target box in the current video frame in the current video frame, the target box ratio threshold, the preset number of frames for magnification adjustment, the current magnification, the zoom ratio table of the imaging device, and the zoom movement direction of the zoom motor.

[0197] In the embodiment of the present disclosure, the ratio of the target box in the current video frame in the current video frame is denoted as Rcur, the number of frames for magnification adjustment is denoted as NUM, and the moving step of the zoom motor in the process from shooting the current video frame to shooting the next video frame is denoted as curZoomStep.

[0198] Consistent with obtaining the average ratio value of the target box in the above - mentioned embodiment, in the exemplary embodiment, the ratio of the target box in the current video frame in the current video frame is obtained by dividing the geometric measurement value (such as the width value) of the target box in the current video frame by the geometric measurement value (such as the width value) of the image in the current video frame.

[0199] In the exemplary embodiment, before obtaining the ratio of the target box in the current video frame in the current video frame, it may further include a step of judging whether there is an abnormality in the size of the target box itself in the current video frame. This step may include: comparing the aspect ratio of the target box in the current video frame with a preset aspect ratio range; if the aspect ratio is outside the aspect ratio range, it is judged that the size of the target box itself is abnormal; if the aspect ratio is within the aspect ratio range, it is judged that the size of the target box itself is normal.

[0200] In this case, if the size of the target frame itself is abnormal, it indicates that the target object in the current video frame may be misidentified or distorted. At this time, if the subsequent process is performed based on the proportion of the target frame in the current video frame, the zoom position of the zoom motor may deviate from the ideal zoom position. Therefore, in the exemplary embodiment, when the size of the target frame in the current video frame itself is abnormal, the video frame containing a normal-sized target frame that is closest to the current video frame before the current video frame is used as the current video frame.

[0201] The magnification adjustment frame number refers to the number of frames that the target frame is required to go through in order to change to a predetermined size in the video frame. In an exemplary embodiment, the magnification adjustment frame number can be set to 30, so that the target frame in the video frame is changed to a predetermined size by controlling the movement of the zoom motor during the playback of 30 frames of video.

[0202] The predetermined size of the target frame is determined by the target frame ratio threshold. In an exemplary embodiment, the target frame ratio threshold is the ratio of the target frame width threshold to the video frame image width. For example, if the target frame ratio threshold is set to 0.18, the target frame is changed to a proportion of 0.18 in the video frame by controlling the movement of the zoom motor during the playback of the video with the magnification adjusted frame number.

[0203] In an exemplary embodiment, the zoom ratio table is stored in the electronic hardware of the camera device. The zoom ratio table stores the correspondence between the zoom position of the zoom motor and the focal length, and the magnification is obtained by the focal length: magnification = focal length / basic focal length. The basic focal length is the focal length when the zoom position of the zoom motor is 0.

[0204] Figure 4 is a flow chart showing a method for obtaining a moving step length of a zoom motor according to an exemplary embodiment, such as Figure 4 As shown, in the exemplary embodiment, step 104 mainly includes the following process from step 401 to step 402.

[0205] Step 401, obtain the next frame magnification ratio according to the proportion of the target frame in the current video frame, the target frame ratio threshold, the preset magnification adjustment frame number, the current magnification ratio and the zoom movement direction of the zoom motor, wherein the next frame magnification ratio is the magnification ratio corresponding to shooting the next video frame.

[0206] Step 402: Obtain the moving step length of the zoom motor in the process from shooting the current video frame to shooting the next video frame according to the current magnification, the next frame magnification and the zoom ratio table of the camera device.

[0207] In the embodiment of the present disclosure, the next frame magnification ratio is expressed as nextLensRatio.

[0208] It can be seen from the above steps 401 to 402 that in the embodiment of the present disclosure, before obtaining the moving step length of the zoom motor, the next frame magnification is first obtained through step 401, and then the zoom magnification table is queried and calculated in step 402 to obtain the moving step length of the zoom motor based on the current magnification and the next frame magnification. Among them, the magnification is obtained by the focal length in the zoom magnification table: magnification = focal length / basic focal length. For example, the current magnification = current focal length / basic focal length, and in the zoom magnification table, the current focal length corresponds to the current zoom position of the zoom motor, so that the current zoom position of the zoom motor can be matched with the current magnification; for another example, the next frame magnification = next frame focal length / basic focal length, in the zoom magnification table, the next frame focal length corresponds to the next frame zoom position of the zoom motor, so that the next frame zoom position of the zoom motor can be matched with the next frame magnification. Through such a corresponding relationship, the current zoom position of the zoom motor and the next frame zoom position of the zoom motor are obtained. The difference between the current zoom position of the zoom motor and the next frame zoom position of the zoom motor is the moving step of the zoom motor in the process from shooting the current video frame to shooting the next video frame.

[0209] Figure 5 is a flow chart of obtaining the magnification of the next frame according to an exemplary embodiment, as shown in FIG. Figure 5 As shown, in the exemplary embodiment, step 401 mainly includes the following process from step 501 to step 502.

[0210] Step 501: Obtain a reference speed for rate adjustment according to a proportion of a target frame in a current video frame and a target frame ratio threshold.

[0211] In the embodiment of the present disclosure, the ratio adjustment reference speed is represented as Speed.

[0212] Among them, the magnification adjustment reference speed is mainly used to control the moving speed of the zoom motor.

[0213] In the exemplary embodiment, step 501 specifically includes the following calculation process:

[0214] The current difference ratio of the target frame is obtained according to the following formula:

[0215] RRcur=T1 / Rcur

[0216] And, if RRcur is less than 1, let

[0217] RRcur=1 / RRcur

[0218] The above formula finally makes RRcur≥1, where T1 is the target frame ratio threshold, Rcur is the proportion in the current video frame, and RRcur is the current difference ratio of the target frame. The above formula can ensure that the current difference ratio of the target frame is not less than 1, so as to ensure that the subsequent magnification adjustment reference speed can be not less than 0;

[0219] The reference speed for rate adjustment is obtained according to the following formula:

[0220] Speed ​​= A RRcur -B

[0221] Wherein, A and B are both adjustment parameters, A and B are equal, for example, A and B are both set to 2.8, Speed ​​is the ratio adjustment reference speed, and when the ratio adjustment reference speed is less than the first set speed, the ratio adjustment reference speed is set to the first set speed, and when the ratio adjustment reference speed is greater than the second set speed, the ratio adjustment reference speed is set to the second set speed, wherein the first set speed is less than the second set speed. The purpose of A and B being equal is to make the speed 0 when RRcur is 1, that is, when Rcur is equal to T1, Speed ​​= 0, and to control the ratio adjustment reference speed in combination with the first set speed and the second set speed.

[0222] For example, when T1=0.18, Rcur=0.13, A=2.8, B=2.8, the first set speed is 0.5, and the second set speed is 5, we can get RRcur=0.18 / 0.13=1.38, Speed=2.8 1.38 -2.8=1.34.

[0223] Step 502: Obtain the magnification of the next frame according to the magnification adjustment reference speed, the magnification adjustment frame number, the current magnification and the zoom movement direction of the zoom motor.

[0224] In the exemplary embodiment, step 502 specifically includes the following calculation process:

[0225] The single-frame moving speed is obtained according to the following formula:

[0226] fpsSpeed=Speed / NUM

[0227] Wherein, Speed ​​is the reference speed for rate adjustment, NUM is the number of frames for rate adjustment, and fpsSpeed ​​is the moving speed of a single frame. The above formula is used to decompose the reference speed for rate adjustment into each frame in the number of frames for rate adjustment. In the exemplary embodiment, NUM can also be set to the video frame rate. If fpsSpeed ​​needs to be changed, fpsSpeed ​​can be adjusted by changing NUM.

[0228] When the zoom movement direction of the zoom motor is to zoom in the direction of a larger magnification, the magnification of the next frame is obtained according to the following formula:

[0229] nextLensRatio=curLensRatio*(1+fpsSpeed)

[0230] When the zoom movement direction of the zoom motor is to zoom in the direction of small magnification, the magnification of the next frame is obtained according to the following formula:

[0231] nextLensRatio=curLensRatio*(1-fpsSpeed)

[0232] When the zoom motor does not perform zoom movement, the magnification of the next frame is obtained according to the following formula:

[0233] nextLensRatio=curLensRatio

[0234] Among them, curLensRatio is the current magnification, and nextLensRatio is the next frame magnification.

[0235] Zooming toward a larger magnification and zooming toward a smaller magnification are two opposite directions. In the above formula, (1+fpsSpeed) and (1-fpsSpeed) reflect these two opposite directions.

[0236] Figure 6 is a flow chart showing a method for obtaining a moving step length of a zoom motor according to an exemplary embodiment, such as Figure 6 As shown, in the exemplary embodiment, step 402 mainly includes the following process from step 601 to step 603.

[0237] Step 601, obtaining the current zoom position of the zoom motor in the camera device according to the current magnification and the zoom ratio table of the camera device;

[0238] Step 602, obtaining a zoom position of a next frame according to a magnification ratio of a next frame and a zoom ratio table of the camera device, wherein the zoom position of the next frame is a zoom position of a zoom motor when shooting a next video frame;

[0239] Step 603: Obtain the moving step length of the zoom motor according to the current zoom position and the zoom position of the next frame.

[0240] In the embodiment of the present disclosure, the current zoom position of the zoom motor is represented as curZoomPos, and the next frame zoom position is represented as nextZoomPos.

[0241] In an exemplary embodiment, step 603 specifically includes:

[0242] The moving step of the zoom motor is obtained according to the following formula:

[0243] curZoomStep = |nextZoomPos - curZoomPos|

[0244] where nextZoomPos is the zoom position of the next frame, curZoomPos is the current zoom position, and curZoomStep is the moving step of the zoom motor.

[0245] Step 105: Control the zoom motor to move from the current zoom position to the zoom position of the next frame according to the zoom moving direction of the zoom motor and the moving step of the zoom motor, so that the imaging device captures the next video frame when the zoom motor is at the zoom position of the next frame.

[0246] Among them, the zoom moving direction of the zoom motor is obtained in step 103, and the moving step of the zoom motor is obtained in step 104. Through the zoom moving direction of the zoom motor and the moving step of the zoom motor, it is possible to determine in which direction and by how many steps the zoom motor needs to move at the current zoom position to capture the next video frame.

[0247] Since the target object does not always exist in the video image, it is also necessary to stop the movement of the zoom motor after the target object leaves the video image. In addition, when the target box reaches the expected size (i.e., the proportion of the target box in the video frame reaches the expectation), it is also necessary to stop the movement of the zoom motor or end the follow zoom. Therefore, in the illustrative embodiment, the follow zoom control method of the present disclosure embodiment further includes the following four schemes for stopping the movement of the zoom motor:

[0248] Scheme 401: When the target box is not included in the latest second consecutive set number of video frames, end the follow zoom, and clear the size of the target box and the average proportion value of the target box in each video frame associated with the first consecutive set number of video frames;

[0249] Scheme 402: When the zoom motor zooms and moves in the direction of a large magnification, stop the movement of the zoom motor when Rcur > (T1 + Ttol / 2) and Ravg > T1; or

[0250] Scheme 403: When the zoom motor zooms and moves in the direction of a small magnification, stop the movement of the zoom motor when Rcur < (T1 - Ttol / 2) and Ravg < T1;

[0251] Wherein, Rcur is the ratio of the target box in the current video frame to the current video frame, T1 is the target box ratio threshold, Ttol is the preset tolerance ratio threshold, Ravg is the average ratio value of the target box, and the target zoom position is obtained according to the target magnification and the zoom ratio table of the imaging device.

[0252] In the embodiment of the present disclosure, the target zoom position is denoted as tarZoomPos.

[0253] Among them, in Solution 401, the latest second consecutive set number of video frames do not contain a target box, indicating that the target object has left the video area, so the target box disappears. In Solution 401, the condition that the latest second consecutive set number of video frames do not contain a target box can avoid the problem that the target object may not actually leave the video area while the target box is lost in individual video frames, thus wrongly ending the follow-up zoom. In Solution 401, when ending the follow-up zoom, the relevant data regarding the follow-up zoom is no longer needed. Therefore, Solution 401 further includes clearing the size of the target box and the average ratio value of the target box in each video frame associated with the first consecutive set number of video frames.

[0254] In Solution 402, when the zoom motor zooms and moves in the direction of a larger magnification, Rcur > (T1 + Ttol / 2) and Ravg > T1 indicate that the target box in the video frame is already large enough, that is, the target object in the video frame is already large enough. Therefore, it is also necessary to stop the movement of the zoom motor.

[0255] In Solution 403, when the zoom motor zooms and moves in the direction of a smaller magnification, Rcur < (T1 + Ttol / 2) and Ravg < T1 indicate that the target box in the video frame cannot be any smaller. If it gets any smaller, it is no longer suitable to observe the target object in the target box through the video. Therefore, it is also necessary to stop the movement of the zoom motor.

[0256] Since there may be significant differences in the shapes of target objects with different object categories, there are different optimal ratio relationships between target objects of different object categories and the image, and this ratio relationship will be reflected in the ratio between the target box and the image. Therefore, in order to be able to adapt to the optimal ratio relationship between target objects of different object categories and the image, in the exemplary embodiment, the follow-up zoom control method of the embodiment of the present disclosure further includes: when the target box marked in the video frame is first obtained, obtaining the target box ratio threshold associated with the object category according to the object category marked by the target box.

[0257] In the exemplary embodiment, there may be a situation where the target object does not appear in the video picture captured by the camera in real time, so the target frame may not always exist in the video captured by the camera in real time. Therefore, when the target object begins to enter the video image and the target frame is generated, in order to quickly focus on the target object, the follow-up zoom control method of the embodiment of the present disclosure may further include the following process performed before step 101:

[0258] When the first video frame containing a target frame that meets the set ratio condition appears in the video frames captured in real time by the camera device, a clarity evaluation value of the target area where the target frame is located is obtained, and fast auto focus is performed based on the clarity evaluation value to clearly focus the target area where the detection frame is located.

[0259] The clarity evaluation value reflects the clarity of the corresponding area of ​​the video frame, and a higher clarity evaluation value indicates a higher clarity of the corresponding area. For example, when the object category is a face, the target area where the target frame is located is the face area in the video frame, and the higher the clarity evaluation value of the target area where the target frame is located, the clearer the face in the video. The clarity evaluation value can be obtained from the electronic hardware of the camera device, such as SoC (System on Chip).

[0260] In an illustrative embodiment, when there is no target frame in the video frame captured in real time by the camera device, a global clarity evaluation value of the video frame is obtained, and fast autofocus in the embodiment of the present disclosure is performed based on the clarity evaluation value. The process of fast autofocus is detailed in the subsequent description.

[0261] In an exemplary embodiment, the ratio condition is set as:

[0262] Rcur≥T1 / 2+Ttol

[0263] Among them, Rcur is the proportion of the target box in the current video frame in the current video frame, T1 is the target box ratio threshold, and Ttol is the preset tolerance ratio threshold.

[0264] Figure 7 is a flowchart of a fast auto-focusing process according to an exemplary embodiment. Figure 7 As shown, in the exemplary embodiment, fast auto focusing includes the following process from step 701 to step 703 .

[0265] Step 701, obtaining a theoretical focus value from a lens curve table of a camera device according to an object distance value before the start of fast autofocus and a current zoom position of a zoom motor;

[0266] Step 702: Obtain a focus value variation range according to a theoretical focus value and a preset focal length extension parameter;

[0267] Step 703: driving the focus motor to move within the focus value variation range, and completing the focus according to the variation of the clarity evaluation value within a preset number of video frames as the camera device shoots.

[0268] In the disclosed embodiment, the theoretical focus value is represented by F1, the focal length extension parameter is represented by S1, and the focus value variation range is represented by [B1, B2]. In the exemplary embodiment, B1=F1-S1, B2=F1+S1, so the focus value variation range is [F1-S1, F1+S1]. For example, when S1=100, the focus value variation range is [F1-100, F1+100], and the focus value variation range is the range of the number of steps of the focus motor.

[0269] Among them, the lens curve table is recorded in the electronic hardware of the camera device. The lens curve table stores the correspondence between the zoom position of the zoom motor and the focus position of the focus motor at different object distances. According to any two quantities among the object distance, the zoom position of the zoom motor and the focus position of the focus motor, another quantity can be obtained from the lens curve table. In this way, in step 701, the focus position of the focus motor corresponding to the current zoom position of the zoom motor, that is, the theoretical focus value, can be obtained from the curve corresponding to the object distance value before the start of fast autofocus according to the current zoom position of the zoom motor by querying the lens curve table. Note that the theoretical focus value (theoretical focus position of the focus motor) may not be the current actual focus position of the focus motor.

[0270] Since the clarity of the target area where the target frame in the obtained video frame is located is not ideal when the focus motor is in the theoretical focus position, the theoretical focus value is expanded to a range in step 702, and the optimal focus position of the focus motor is found within this range in step 703, so that the image of the target area where the target frame in the video frame is clear.

[0271] Since the embodiment of the present disclosure implements fast autofocus, it is necessary to complete autofocus in a very short time, which corresponds to the need to complete focusing in the process of playing a small number of video frames. Therefore, the preset number of video frames is a small number of video frames. In the exemplary embodiment, the preset number is 5 to 15 frames, preferably 9 frames. If the number of frames is too small, there will be a problem of poor focusing effect, and if the number of frames is too large, there will be a problem of too long focusing time.

[0272] Figure 8 FIG. 1 is a flow chart showing the control of the movement of the focus motor during the fast autofocus process according to an exemplary embodiment. Figure 8 As shown, in an exemplary embodiment, step 703 may specifically include the following process of step 801 to step 805.

[0273] Step 801: Obtain the number of steps of the focus motor corresponding to each of the preset number of video frames according to the following formula:

[0274] fStep1=(B2-curFocusPos) / M

[0275] Wherein, fStep1 is the number of steps of the focus motor corresponding to each frame in the preset number of video frames, curFocusPos is the current focus position of the focus motor, M is the preset number, and B2 is the upper limit of the focus value variation range;

[0276] Step 802, driving the focus motor to move fStep1, and obtaining a definition evaluation value of a target area where a target frame is located in a video frame after the focus motor moves;

[0277] Step 803: When the clarity evaluation value obtained with the movement of the focus motor decreases, the number of steps of the focus motor movement corresponding to each frame that has not been moved in the preset number of video frames is obtained according to the following formula:

[0278] fStep2=(curFocusPos-B1) / (MN)

[0279] Wherein, fStep2 is the number of steps of the focus motor corresponding to each frame that has not been passed in the preset number of video frames, N is the number of frames that have been passed in the preset number of video frames, and B1 is the lower limit value of the focus value variation range;

[0280] Step 804, driving the focus motor to move fStep2, and obtaining a definition evaluation value of the target area where the target frame is located in the video frame after the focus motor moves;

[0281] Step 805 , when all the preset number of video frames have passed, the focus motor is controlled to move to the focus position of the focus motor corresponding to the maximum clarity evaluation value obtained in the preset number of video frames, so as to complete the focus.

[0282] In step 802, each time the focus motor moves fStep1, a new video frame is obtained accordingly, that is, the video frame after the focus motor moves. Therefore, when the focus motor moves the first fStep1, the first video frame in the preset number of video frames is obtained, and then the clarity evaluation value of the target area where the target frame is located in the first video frame is obtained. When the focus motor moves the second fStep1, the second video frame in the preset number of video frames is obtained, and then the clarity evaluation value of the target area where the target frame is located in the second video frame is obtained, and so on.

[0283] Similarly, in step 804, when N frames have been passed in the preset number of video frames, the focus motor moves the first fStep2, and the N+1th video frame in the preset number of video frames is obtained, and then the clarity evaluation value of the target area where the target frame in the N+1th video frame is located is obtained, and the focus motor moves the second fStep2, and the N+2th video frame in the preset number of video frames is obtained, and then the clarity evaluation value of the target area where the target frame in the N+2th video frame is located is obtained, and so on, until the Mth video frame is passed.

[0284] In step 803, the clarity evaluation value obtained by the movement of the focus motor decreases, indicating that after the Nth frame, the clarity evaluation value has an inflection point and is decreasing, which means that the target area where the target frame is located has changed from clear to blurred. If the focus motor continues to move forward instead of moving in the opposite direction as controlled by the formula in step 803, it will continue to blur. Therefore, through the control process of steps 801 to 805, the focus position of the focus motor with better imaging quality in the target area where the target frame is located can be obtained in the shortest possible time (within the smallest possible number of frames). In the disclosed embodiment, fast autofocus is performed as soon as the target frame appears, and the focusing time can be completed within 300ms (milliseconds). From the visual eye observation angle, the target object in the target frame can be clearly presented in an instant.

[0285] In the process of the above steps 101 to 105 of the follow-up zoom control method of the embodiment of the present disclosure, only the control of the zoom motor is involved, which is to ensure that the target object in the target frame can always maintain a relatively appropriate size, and the clarity needs to be adjusted in real time by the focus motor according to the movement of the zoom motor. For example, the above-mentioned fast autofocus process is to ensure that the target object can be quickly presented clearly when it just appears. In the process of the movement of the target object, the target object may be far away from or close to the camera device, so that the object distance will continue to change, and the zoom motor will also move according to the follow-up zoom control method of the embodiment of the present disclosure. In an illustrative embodiment, in order to clearly present the target object in the target frame during the follow-up zoom process, the embodiment of the present disclosure may further include determining the focus position (focus value) of the corresponding focus motor in real time according to the change of the zoom motor, the change of the object distance, and through the lens curve table to achieve focusing.

[0286] Fig. 9 FIG. 1 is a flow chart showing a method of controlling a focus motor to achieve focusing during a follow-up zoom process according to an exemplary embodiment. Fig. 9 As shown, in the exemplary embodiment, the process of controlling the focus motor to achieve focusing during the follow-up zoom process mainly includes the following process from step 901 to step 904.

[0287] Step 901, obtaining a current object distance value according to a current zoom position of a zoom motor, a current focus position of a focus motor in a camera device, and a lens curve table of the camera device;

[0288] Step 902: Obtain the predicted object distance of the next frame according to the proportion of the target frame in the current video frame, the target frame ratio threshold, the frame rate of the video frame, the preset object distance adjustment factor, and the current object distance value;

[0289] Step 903, according to the next frame predicted object distance and the current zoom position of the zoom motor, the next frame predicted focus position of the focus motor is obtained through the lens curve table;

[0290] Step 904 : Control the focus motor to move to the predicted focus position of the next frame, so as to shoot the next video frame when the focus motor is located at the predicted focus position of the next frame.

[0291] In the disclosed embodiment, the current object distance value is represented as curDist, the frame rate of the video frame is represented as fps, the object distance adjustment factor is represented as adjFactor1, the predicted object distance of the next frame is represented as predictDist, and the predicted focus position of the focus motor in the next frame is represented as preFocusPos.

[0292] In an exemplary embodiment, in step 901, an object distance curve corresponding to the current zoom position of the zoom motor and the current focus position of the focus motor in the camera device can be obtained by querying a lens curve table, and the object distance curve is the corresponding current object distance value.

[0293] Since the distances between the various object distance curves in the lens curve table will become farther and farther when moving in the direction of large magnification, there is a situation where the appropriate object distance curve is selected; while when moving in the direction of small magnification, the distances between the various object distance curves in the lens curve table will become closer and closer, so different object distance curves have little effect on the selection of the focus position of the focus motor. Therefore, the method of obtaining the focus position of the focus motor when moving in the direction of large magnification and moving in the direction of small magnification is slightly different, as follows.

[0294] In an exemplary embodiment, step 902 includes the following process performed when the zoom movement direction of the zoom motor is zoom movement toward a larger magnification direction:

[0295] The current difference ratio of the target frame is obtained according to the following formula:

[0296] RRcur=T1 / Rcur

[0297] Among them, RRcur is the current difference ratio of the target frame;

[0298] The object distance limit offset is obtained according to the following formula:

[0299] distOffset = RRcur * adjFactor1

[0300] Among them, adjFactor1 is the object distance adjustment factor, and distOffset is the object distance limit offset;

[0301] According to the current object distance value and the object distance limit offset, the object distance limit range is obtained, wherein the object distance limit range is [curDist-distOffset, curDist+distOffset];

[0302] A zoom tracking method is used to obtain the predicted object distance of the next frame, wherein, if the predicted object distance of the next frame is less than the lower limit of the object distance limit range, the predicted object distance of the next frame is set to the lower limit of the object distance limit range; if the predicted object distance of the next frame is greater than the upper limit of the object distance limit range, the predicted object distance of the next frame is set to the upper limit of the object distance limit range.

[0303] Among them, the zoom tracking method can be implemented using existing technologies, which will not be described in detail here.

[0304] In an exemplary embodiment, step 902 further includes the following process performed when the zoom movement direction of the zoom motor is zoom movement toward a smaller magnification direction:

[0305] The current difference ratio of the target frame is obtained according to the following formula:

[0306] RRcur=T1 / Rcur

[0307] Among them, RRcur is the current difference ratio of the target frame;

[0308] The object distance adjustment speed is obtained according to the following formula:

[0309] distSpeed ​​= RRcur / fps

[0310] Among them, fps is the frame rate of the video frame, and distSpeed ​​is the object distance adjustment speed;

[0311] The predicted object distance for the next frame is obtained according to the following formula:

[0312] predictDist=curDist-distSpeed

[0313] Wherein, curDist is the current object distance value, predictDist is the predicted object distance of the next frame, wherein, if the predicted object distance of the next frame is less than the minimum object distance supported by the camera device, the predicted object distance of the next frame is set to the minimum object distance supported by the camera device.

[0314] Through the process from step 901 to step 904, focusing can be achieved in the follow-up zoom process.

[0315] The following zoom control method of the embodiment of the present disclosure uses the target frame marked in the video frame as a reference and the preset target frame ratio threshold as a benchmark. As the video shooting process continues, the zoom movement direction is determined according to the relationship between the target frame and the target frame ratio threshold in multiple video frames, and the zoom speed is controlled by adjusting the number of frames according to the preset magnification, thereby determining the movement direction and movement step of the zoom motor when shooting each frame in the future, and finally making the target frame in the video frame smoothly adjusted to the size corresponding to the target frame ratio threshold as the video is shot. Since the target frame marked in the video frame is obtained according to the target object in the video frame, therefore, making the adjustment in the video frame to the size corresponding to the target frame ratio threshold means that the target object in the video frame is smoothly adjusted to the size corresponding to the target frame ratio threshold as the zoom motor moves, therefore, setting the target frame ratio threshold of an appropriate size can obtain the presentation of the target object in the video at the appropriate size. It can be seen that the following zoom control method of the embodiment of the present disclosure realizes smooth zooming in the process of tracking the target object.

[0316] On this basis, in the follow-up zoom control method of the embodiment of the present disclosure, when the first frame containing a target frame that meets the set ratio condition appears in the video frames captured in real time by the camera device, fast auto focus is performed within a preset number of video frames according to the clarity evaluation value of the target area where the target frame is located, so as to focus the target area where the detection frame is located clearly. Moreover, in the process of follow-up zoom, the follow-up zoom control method of the embodiment of the present disclosure, under the constraints of the target frame ratio threshold, the frame rate of the video frame and other conditions, uses the lens curve table of the camera device and follows the zoom position change of the zoom motor to obtain the corresponding focus position of the focus motor, so as to control and adjust the focus position of the focus motor in real time, thereby realizing the smooth adjustment of the dynamic focus and dynamic zoom process.

[0317] In summary, the follow-up zoom control method of the embodiment of the present disclosure realizes smooth adjustment of the zoom motor and the focus motor by following the changes in the size of the target frame in the video frame, so that the video image can be presented smoothly and silkily while keeping the clarity of the target object as the target object in the target frame moves away from or approaching the camera device.

[0318] Fig.10 is a logical structure diagram of a follow-up zoom control device according to an exemplary embodiment. Fig.10As shown, the follow-up zoom control device mainly includes a video frame acquisition module 1001, a target frame average ratio value acquisition module 1002, a zoom movement direction acquisition module 1003, a zoom motor movement step acquisition module 1004 and a zoom motor movement control module 1005.

[0319] The video frame acquisition module 1001 is configured to acquire the video frame captured by the camera device in real time and the target frame marked in the video frame;

[0320] The target frame average ratio value obtaining module 1002 is configured to execute, in the first consecutive set number of video frames captured most recently, obtaining the target frame average ratio value according to the size of the target frame in each video frame and the size of the video frame, wherein the target frame average ratio value is the average value of the proportion of the target frame in the first consecutive set number of video frames in the video frame;

[0321] The zoom movement direction obtaining module 1003 is configured to obtain the zoom movement direction of the zoom motor according to the target frame average ratio value, the preset target frame ratio threshold value, and the current magnification of the camera device;

[0322] The zoom motor movement step length obtaining module 1004 is configured to obtain the movement step length of the zoom motor in the process from shooting the current video frame to shooting the next video frame according to the proportion of the target frame in the current video frame, the target frame ratio threshold, the preset magnification adjustment frame number, the current magnification, the zoom magnification table of the camera device and the zoom movement direction of the zoom motor;

[0323] The zoom motor movement control module 1005 is configured to control the zoom motor to move from the current zoom position to the next frame zoom position according to the zoom movement direction and the movement step of the zoom motor, so that the camera device shoots the next video frame when the zoom motor is in the next frame zoom position.

[0324] In an exemplary embodiment, the follow-up zoom control device further includes:

[0325] The fast autofocus module is configured to execute, when the first frame of the video frame containing the target frame that meets the set ratio condition appears in the video frames taken in real time by the camera device, obtain the clarity evaluation value of the target area where the target frame is located, and perform fast autofocus according to the clarity evaluation value to clearly focus the target area where the detection frame is located.

[0326] In an exemplary embodiment, the follow-up zoom control device further includes:

[0327] A current object distance value acquisition module is configured to obtain a current object distance value according to a current zoom position of the zoom motor, a current focus position of a focus motor in the camera device, and a lens curve table of the camera device;

[0328] The predicted object distance acquisition module is configured to obtain the predicted object distance of the next frame according to the proportion of the target frame in the current video frame, the target frame ratio threshold, the frame rate of the video frame, the preset object distance adjustment factor, and the current object distance value;

[0329] A predicted focus position acquisition module is configured to execute the predicted object distance of the next frame and the current zoom position of the zoom motor, and obtain the predicted focus position of the focus motor in the next frame through a lens curve table;

[0330] The focus motor control module is configured to control the focus motor to move to the next frame predicted focus position, so as to shoot the next video frame when the focus motor is located at the next frame predicted focus position.

[0331] Regarding the follow-up zoom control device in the above embodiment, the specific manner in which each unit performs operations has been described in detail in the embodiment of the follow-up zoom control method, and will not be elaborated here.

[0332] It should be noted that: the above embodiment only illustrates the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0333] Fig.11 It is a structural diagram of an electronic device provided in an embodiment of the present disclosure. In some embodiments, the electronic device is a server. The electronic device 1100 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 1101 and one or more memories 1102, wherein the memory 1102 stores at least one program code, and the at least one program code is loaded and executed by the processor 1101 to implement the follow-up zoom control method provided in the above-mentioned embodiments. Of course, the electronic device 1100 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The electronic device 1100 may also include other components for implementing device functions, which will not be repeated here.

[0334] In an exemplary embodiment, a computer-readable storage medium including at least one instruction is also provided, such as a memory including at least one instruction. The at least one instruction can be executed by a processor in a computer device to complete the follow-up zoom control method in the above embodiment.

[0335] Optionally, the above-mentioned computer-readable storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may include ROM (Read-Only Memory), RAM (Random-Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk and optical data storage device, etc.

[0336] In an exemplary embodiment, a computer program product is also provided, including one or more instructions, which can be executed by a processor of a computer device to implement the follow-up zoom control method provided by each of the above embodiments.

[0337] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A follow-up zoom control method, comprising the following follow-up zoom control process: Acquire a video frame captured in real time by a camera device and a target frame marked in the video frame; In the first set number of consecutive video frames captured most recently, an average target frame ratio value is obtained according to the size of the target frame in each video frame and the size of the video frame, where: The target frame average ratio value is an average value of the proportion of the target frame in the first set number of consecutive video frames in the video frames; Obtaining a zoom movement direction of the zoom motor according to the target frame average ratio value, a preset target frame ratio threshold, and a current magnification of the camera device; Obtaining a moving step length of the zoom motor in a process from shooting a current video frame to shooting a next video frame according to a proportion of a target frame in a current video frame, a target frame ratio threshold, a preset magnification adjustment frame number, the current magnification, a zoom magnification table of the camera device, and a zoom movement direction of the zoom motor; According to the zoom movement direction and the movement step length of the zoom motor, the zoom motor is controlled to move from the current zoom position to the next frame zoom position, so that the camera device shoots the next video frame when the zoom motor is in the next frame zoom position.

2. The follow zoom control method according to claim 1, characterized in that: The method of obtaining the target frame average ratio value according to the size of the target frame in each video frame and the size of the video frame in the first set number of consecutive video frames taken most recently includes: Obtaining, according to the geometric measurement values ​​of the target frames in the respective video frames, the deviations of the geometric measurement values ​​of the target frames in the respective video frames; From the first set number of continuous video frames, filter out the video frames where the target frames whose absolute values ​​of the deviations are greater than a set screening threshold are located; For each video frame that is not filtered out in the first set number of continuous video frames, dividing the geometric measurement value of the target frame in the video frame by the geometric measurement value of the image in the video frame to obtain a target frame ratio value; An average value of the target frame ratio values ​​of all video frames that are not filtered out of the first continuous set number of video frames is determined as the target frame average ratio value.

3. The follow zoom control method according to claim 1, characterized in that: The step of obtaining the zoom movement direction of the zoom motor according to the target frame average ratio value, a preset target frame ratio threshold, and the current magnification of the camera device includes: The average difference ratio of the target box is obtained according to the following formula: RRavg=T1 / Ravg Wherein, T1 is the target frame ratio threshold, Ravg is the target frame average ratio value, and RRavg is the target frame average difference ratio; The target magnification of the camera device is obtained according to the following formula: tarLensRatio=curLensRatio*RRavg Wherein, curLensRatio is the current magnification, and tarLensRatio is the target magnification; According to the target magnification, obtaining a target zoom position of the zoom motor corresponding to the target magnification by querying a zoom ratio table of the camera device; The zoom movement direction of the zoom motor is obtained according to the current zoom position of the zoom motor and the target zoom position, wherein: If the target zoom position is greater than the current zoom position, the zoom movement direction of the zoom motor is to zoom in the direction of a larger magnification; If the target zoom position is smaller than the current zoom position, the zoom movement direction of the zoom motor is to zoom in a direction of smaller magnification; or If the target zoom position is equal to the current zoom position, the zoom motor does not perform zoom movement.

4. The follow zoom control method according to claim 1, characterized in that: The method of obtaining the moving step length of the zoom motor in the process from shooting the current video frame to shooting the next video frame according to the proportion of the target frame in the current video frame, the target frame ratio threshold, the preset magnification adjustment frame number, the current magnification, the zoom magnification table of the camera device and the zoom movement direction of the zoom motor includes: According to the proportion of the target frame in the current video frame, the target frame ratio threshold, the preset magnification adjustment frame number, the current magnification and the zoom movement direction of the zoom motor, the next frame magnification is obtained, wherein the next frame magnification is the magnification corresponding to shooting the next video frame; According to the current magnification, the next frame magnification and the zoom ratio table of the camera device, the moving step length of the zoom motor in the process from shooting the current video frame to shooting the next video frame is obtained.

5. The follow-up zoom control method according to claim 4, characterized in that: The method of obtaining the magnification of the next frame according to the proportion of the target frame in the current video frame, the target frame ratio threshold, the preset magnification adjustment frame number, the current magnification and the zoom movement direction of the zoom motor includes: Obtaining a reference speed for rate adjustment according to a proportion of a target frame in the current video frame and a target frame ratio threshold; The next frame magnification is obtained according to the magnification adjustment reference speed, the magnification adjustment frame number, the current magnification and the zoom movement direction of the zoom motor.

6. The follow-up zoom control method according to claim 5, characterized in that: The obtaining the reference speed for rate adjustment according to the proportion of the target frame in the current video frame and the target frame ratio threshold comprises: The current difference ratio of the target frame is obtained according to the following formula: RRcur=T1 / Rcur And, if RRcur is less than 1, let RRcur=1 / RRcur Wherein, T1 is the target frame ratio threshold, Rcur is the proportion in the current video frame, and RRcur is the current difference ratio of the target frame; The ratio adjustment reference speed is obtained according to the following formula: Speed=A RRcur -B Among them, A and B are both adjustment parameters, A is equal to B, Speed ​​is the ratio adjustment reference speed, and when the ratio adjustment reference speed is less than the first set speed, the ratio adjustment reference speed is set to the first set speed, and when the ratio adjustment reference speed is greater than the second set speed, the ratio adjustment reference speed is set to the second set speed, wherein the first set speed is less than the second set speed.

7. The follow-up zoom control method according to claim 5, characterized in that: The step of obtaining the next frame magnification ratio according to the magnification ratio adjustment reference speed, the magnification ratio adjustment frame number, the current magnification ratio and the zoom movement direction of the zoom motor comprises: The single-frame moving speed is obtained according to the following formula: fpsSpeed=Speed / NUM Wherein, Speed ​​is the reference speed for the magnification adjustment, NUM is the number of frames for the magnification adjustment, and fpsSpeed ​​is the single-frame moving speed; When the zoom movement direction of the zoom motor is to zoom in the direction of a larger magnification, the magnification of the next frame is obtained according to the following formula: nextLensRatio=curLensRatio*(1+fpsSpeed) When the zoom movement direction of the zoom motor is to zoom in the direction of small magnification, the magnification of the next frame is obtained according to the following formula: nextLensRatio=curLensRatio*(1-fpsSpeed) When the zoom motor does not perform zoom movement, the next frame magnification is obtained according to the following formula: nextLensRatio=curLensRatio Wherein, curLensRatio is the current magnification ratio, and nextLensRatio is the next frame magnification ratio.

8. The follow-up zoom control method according to claim 4, characterized in that: The step of obtaining the moving step of the zoom motor in the process from shooting the current video frame to shooting the next video frame according to the current magnification, the next frame magnification and the zoom ratio table of the camera device comprises: Obtaining a current zoom position of a zoom motor in the camera device according to the current magnification and a zoom ratio table of the camera device; Obtaining a zoom position for a next frame according to the next frame magnification and the zoom ratio table of the camera device, wherein the zoom position for the next frame is a zoom position of the zoom motor when shooting a next video frame; The moving step length of the zoom motor is obtained according to the current zoom position and the next frame zoom position.

9. The follow zoom control method according to claim 8, characterized in that: The step of obtaining the moving step length of the zoom motor according to the current zoom position and the next zoom position includes: The moving step length of the zoom motor is obtained according to the following formula: curZoomStep=|nextZoomPos-curZoomPos| Among them, nextZoomPos is the zoom position of the next frame, curZoomPos is the current zoom position, and curZoomStep is the moving step length of the zoom motor.

10. The follow zoom control method according to claim 1, characterized in that: The following zoom control method also includes: When the target frame is not included in the second consecutive set number of video frames shot most recently, the follow zoom is terminated, and the size and average ratio of the target frame in each video frame associated with the first consecutive set number of video frames are cleared; When the zoom motor moves in a zooming direction with a large magnification, when Rcur>T1+Ttol / 2 and Ravg>T1, stop the movement of the zoom motor; or When the zoom motor zooms and moves in the direction of a smaller magnification, if Rcur < T1 - Ttol / 2 and Ravg < T1, stop the movement of the zoom motor; wherein, Rcur is the ratio of the target box in the current video frame to the current video frame, T1 is the target box ratio threshold, Ttol is the preset tolerance ratio threshold, Ravg is the average ratio value of the target box, and the target zoom position is obtained according to the target magnification and the zoom ratio table of the imaging device.

11. The follow zoom control method according to claim 1, characterized in that: The following zoom control method further includes: When the target box marked in the video frame is first obtained, according to the object category marked by the target box, obtain the target box ratio threshold associated with the object category.

12. The follow zoom control method according to claim 1, characterized in that: Before obtaining the video frame captured in real time by the imaging device and the target box marked in the video frame, the following zoom control method further includes: When a first video frame containing a target box that meets the set ratio condition appears in the video frame captured in real time by the imaging device, obtain the clarity evaluation value of the target area where the target box is located, and perform fast autofocus according to the clarity evaluation value to focus the target area where the detection box is located clearly.

13. The following zoom control method according to claim 12, wherein: The set ratio condition is: Rcur ≥ T1 / 2 + Ttol wherein, Rcur is the ratio of the target box in the current video frame to the current video frame, T1 is the target box ratio threshold, and Ttol is the preset tolerance ratio threshold.

14. The follow-up zoom control method according to claim 12, characterized in that: The fast autofocus includes: According to the object distance value before the start of the fast autofocus and the current zoom position of the zoom motor, obtain the theoretical focus value from the lens curve table of the imaging device; According to the theoretical focus value and the preset focal length extension parameter, obtain the focus value change range; Drive the focus motor to move within the focus value change range, and with the shooting of the imaging device, complete the focus according to the change of the clarity evaluation value within a preset number of video frames.

15. The follow-up zoom control method according to claim 14, characterized in that: The driving the focus motor to move within the focus value change range, and with the shooting of the imaging device, completing the focus according to the change of the clarity evaluation value within a preset number of video frames includes: Obtain the number of steps of movement of the focus motor corresponding to each frame in the preset number of video frames according to the following formula: fStep1 = (B2 - curFocusPos) / M wherein, fStep1 is the number of steps of movement of the focus motor corresponding to each frame in the preset number of video frames, curFocusPos is the current focus position of the focus motor, M is the preset number, and B2 is the upper limit value of the focus value change range; Drive the focus motor to move fStep1 to obtain the clarity evaluation value of the target area where the target box is located in the video frame after the movement of the focus motor; When the clarity evaluation value obtained with the movement of the focus motor decreases, obtain the number of steps of movement of the focus motor corresponding to each unpassed frame in the preset number of video frames according to the following formula fStep2=(curFocusPos-B1) / (MN) Wherein, fStep2 is the number of steps of the focus motor corresponding to each frame that has not been passed in the preset number of video frames, N is the number of frames that have been passed in the preset number of video frames, and B1 is the lower limit value of the focus value variation range; Drive the focus motor to move fStep2, and obtain a definition evaluation value of a target area where the target frame is located in the video frame after the focus motor moves; When all the preset number of video frames have passed, the focus motor is controlled to move to a focus position of the focus motor corresponding to the maximum clarity evaluation value obtained in the preset number of video frames, so as to complete the focusing.

16. The follow zoom control method according to claim 1, characterized in that: The following zoom control method further includes the following steps performed simultaneously during the following zoom control process: Obtaining a current object distance value according to a current zoom position of the zoom motor, a current focus position of a focus motor in the camera device, and a lens curve table of the camera device; Obtaining a predicted object distance for the next frame according to the proportion of the target frame in the current video frame, the target frame ratio threshold, the frame rate of the video frame, a preset object distance adjustment factor, and the current object distance value; According to the next frame predicted object distance and the current zoom position of the zoom motor, the next frame predicted focus position of the focus motor is obtained through the lens curve table; The focus motor is controlled to move to the next frame predicted focus position, so as to shoot the next video frame when the focus motor is located at the next frame predicted focus position.

17. The follow zoom control method according to claim 16, characterized in that: The method of obtaining the predicted object distance of the next frame according to the proportion of the target frame in the current video frame, the target frame ratio threshold, the frame rate of the video frame, the preset object distance adjustment factor, and the current object distance value includes the following process performed when the zoom movement direction of the zoom motor is zoom movement in the direction of a larger magnification: The current difference ratio of the target frame is obtained according to the following formula: RRcur=T1 / Rcur Wherein, RRcur is the current difference ratio of the target frame; The object distance limit offset is obtained according to the following formula: distOffset = RRcur * adjFactor1 Wherein, adjFactor1 is the object distance adjustment factor, and distOffset is the object distance limit offset; According to the current object distance value and the object distance limit offset, an object distance limit range is obtained, wherein the object distance limit range is [curDist-distOffset, curDist+distOffset]; A zoom tracking method is used to obtain the predicted object distance of the next frame, wherein if the predicted object distance of the next frame is less than the lower limit value of the object distance limit range, the predicted object distance of the next frame is set to the lower limit value of the object distance limit range; if the predicted object distance of the next frame is greater than the upper limit value of the object distance limit range, the predicted object distance of the next frame is set to the upper limit value of the object distance limit range.

18. The follow zoom control method according to claim 16, characterized in that: The method of obtaining the predicted object distance of the next frame according to the proportion of the target frame in the current video frame, the target frame ratio threshold, the frame rate of the video frame, the preset object distance adjustment factor, and the current object distance value includes the following process performed when the zoom movement direction of the zoom motor is zoom movement in a direction of a small magnification: The current difference ratio of the target frame is obtained according to the following formula: RRcur=T1 / Rcur Wherein, RRcur is the current difference ratio of the target frame; The object distance adjustment speed is obtained according to the following formula: distSpeed ​​= RRcur / fps Wherein, fps is the frame rate of the video frame, and distSpeed ​​is the object distance adjustment speed; The predicted object distance of the next frame is obtained according to the following formula: predictDist=curDist-distSpeed Wherein, curDist is the current object distance value, predictDist is the predicted object distance of the next frame, wherein, if the predicted object distance of the next frame is less than the minimum object distance supported by the camera device, the predicted object distance of the next frame is set to the minimum object distance supported by the camera device.

19. A follow-up zoom control device, characterized in that: include: A video frame acquisition module is configured to acquire a video frame captured in real time by a camera device and a target frame marked in the video frame; The target frame average ratio value obtaining module is configured to execute, in the first consecutive set number of video frames taken most recently, obtaining the target frame average ratio value according to the size of the target frame in each video frame and the size of the video frame, wherein the target frame average ratio value is the average value of the proportion of the target frame in the first consecutive set number of video frames in the video frame; A zoom movement direction obtaining module is configured to obtain the zoom movement direction of the zoom motor according to the target frame average ratio value, a preset target frame ratio threshold value, and the current magnification of the camera device; A zoom motor movement step length obtaining module is configured to obtain the movement step length of the zoom motor in the process from shooting the current video frame to shooting the next video frame according to the proportion of the target frame in the current video frame, the target frame ratio threshold, the preset magnification adjustment frame number, the current magnification, the zoom magnification table of the camera device and the zoom movement direction of the zoom motor; The zoom motor movement control module is configured to control the zoom motor to move from the current zoom position to the next frame zoom position according to the zoom movement direction of the zoom motor and the movement step of the zoom motor, so that the camera device can shoot the next video frame when the zoom motor is in the next frame zoom position.

20. An electronic device, characterized in that: include: processor; a memory for storing executable instructions for the processor; The processor is configured to execute the executable instructions to implement the follow-up zoom control method as described in any one of claims 1 to 18.

21. A computer-readable storage medium, characterized in that: When at least one instruction in the computer-readable storage medium is executed by a processor of an electronic device, the electronic device is enabled to implement the follow-up zoom control method as described in any one of claims 1 to 18.

Images