Automatic focusing method and system for optical camera in target tracking process
By adaptively adjusting the focal length of the optical camera and the position of the focusing lens group, the problems of focusing overshoot and background interference during target tracking of the optical camera are solved, and accurate and high-quality imaging of the target area is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-10
- Publication Date
- 2026-03-10
AI Technical Summary
Existing optical cameras lack intelligent focusing capabilities during target tracking, which can easily cause overshoot when the target is at a distance, making it difficult to suppress background interference and resulting in a decrease in image quality.
By adaptively adjusting the focal length of the optical camera, the zoom step size is determined using a logarithmic relationship based on the deviation between the area ratio of the target region and the preset expected ratio range, and the position of the focusing lens group is optimized in the focusing execution image to achieve localized precise focusing.
It effectively suppresses background interference, improves the imaging clarity of the target object and the overall image quality, and ensures stable tracking and imaging in complex scenes.
Smart Images
Figure CN121644989A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computers, and specifically to an optical camera autofocusing method and system for target tracking. Background Technology
[0002] In existing technologies, optical cameras typically employ autofocus strategies based on fixed step sizes or preset parameters in target imaging and tracking applications, lacking intelligent control capabilities tailored to the dynamic characteristics of the target.
[0003] However, during actual tracking, the relative distance between the target and the camera continuously changes, causing its scale in the image to constantly change. When the target is at a distance and the camera is operating at a telephoto range, a large focusing step size can easily cause "focus overshoot," resulting in momentary image blurring or even target loss. Furthermore, although some systems have introduced feedback control mechanisms based on global sharpness evaluation, dynamic targets typically occupy a small portion of the overall image, while background areas may have stronger texture or higher contrast. This causes the focusing system to incorrectly treat the background as the optimization target, making it difficult to achieve precise focusing on specific targets and thus affecting image quality. Summary of the Invention
[0004] This application provides an automatic focusing method and system for an optical camera during target tracking. It can adaptively adjust the focal length of the optical camera and achieve precise local focusing on the target area, effectively suppressing background interference and improving the imaging clarity and overall image quality of the target object.
[0005] This application provides an embodiment of an optical camera autofocusing method during target tracking, including:
[0006] The focusing decision image of the optical camera is processed to detect the target object, and the target region corresponding to the target object in the focusing decision image is obtained.
[0007] Based on the deviation between the area ratio of the target region in the focusing decision image and the preset expected ratio range, and the current focal length of the optical camera, the adaptive zoom step size is determined. The area ratio is the ratio of the area of the target region in the focusing decision image to the area of the focusing decision image. The adaptive zoom step size has a logarithmic relationship with the current focal length.
[0008] Adjust the current focal length of the optical camera according to the adaptive zoom step size;
[0009] After completing the focus adjustment, acquire the focusing execution image of the optical camera;
[0010] Adjust the position of the focusing lens group of the optical camera, and search for the target focusing position that maximizes the sharpness evaluation value based on the sharpness evaluation value of the target area in the image to complete autofocus.
[0011] This application also provides an optical camera autofocus system for target tracking, including:
[0012] The target detection unit is used to perform target object detection processing on the focusing decision image of the optical camera to obtain the target area corresponding to the target object in the focusing decision image;
[0013] The zoom step size determination unit is used to determine the adaptive zoom step size based on the deviation between the area ratio of the target region in the focus decision image and the preset expected ratio range, as well as the current focal length value of the optical camera. The area ratio is the ratio of the area of the target region in the focus decision image to the area of the focus decision image. The adaptive zoom step size has a logarithmic relationship with the current focal length value.
[0014] The focusing unit is used to adjust the current focal length of the optical camera according to the adaptive zoom step size;
[0015] The image acquisition unit is used to acquire the focusing execution image of the optical camera after the focus adjustment is completed;
[0016] The focusing unit is used to adjust the position of the focusing lens group of the optical camera and, based on the sharpness evaluation value corresponding to the target area in the image where the target object is focused, search for the target focusing position that maximizes the sharpness evaluation value in order to complete autofocus.
[0017] This application also provides an electronic device, including a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to execute steps in any of the optical camera autofocusing methods provided in this application during target tracking.
[0018] This application also provides a computer-readable storage medium storing multiple instructions adapted for loading by a processor to execute steps in any of the optical camera autofocusing methods provided in the target tracking process according to this application.
[0019] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of any of the optical camera autofocusing methods provided in the target tracking process according to this application.
[0020] This application embodiment can perform target object detection processing on the focusing decision image of an optical camera to obtain the target area corresponding to the target object in the focusing decision image; based on the deviation between the area ratio of the target area in the focusing decision image and the preset expected ratio range, and the current focal length value of the optical camera, an adaptive zoom step size is determined, where the area ratio is the ratio of the area of the target area in the focusing decision image to the area of the focusing decision image, and the adaptive zoom step size has a logarithmic relationship with the current focal length value; the current focal length value of the optical camera is adjusted according to the adaptive zoom step size; after the focal length adjustment is completed, the focusing execution image of the optical camera is obtained; the position of the focusing lens group of the optical camera is adjusted, and based on the sharpness evaluation value corresponding to the target area in the focusing execution image, the target focusing position that maximizes the sharpness evaluation value is searched to complete the autofocus.
[0021] In this application, the target region corresponding to the target object in the focusing decision image of the optical camera is first obtained. Then, based on the deviation between the area ratio of the target region in the focusing decision image and the preset expected ratio range, and the current focal length value of the optical camera, the adaptive zoom step size is determined. Since the calculation of the adaptive zoom step size introduces the logarithm of the current focal length value, it can effectively suppress abrupt changes in the focal length adjustment process, making the zoom action smoothly transition with the focal length change, thereby achieving adaptive and stable adjustment of the optical camera's focal length. After completing the focal length adjustment, the focusing execution image of the optical camera is acquired. During the adjustment of the focusing lens group position of the optical camera, only for the sharpness evaluation value corresponding to the target region in the focusing execution image of the target object, the target focusing position that maximizes the sharpness evaluation value is searched, achieving localized precise focusing of the target region, effectively suppressing background interference, and improving the imaging sharpness of the target object and the overall image quality. Attached Figure Description
[0022] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1a This is a schematic flowchart of an optical camera autofocusing method for target tracking provided in an embodiment of this application.
[0024] Figure 1b This is a closed-loop flowchart of focusing and autofocus control provided in the embodiments of this application;
[0025] Figure 2This is a schematic diagram of the structure of an optical camera autofocus system in the target tracking process provided in an embodiment of this application; Detailed Implementation
[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0027] This application provides an optical camera autofocusing method and system for target tracking.
[0028] Specifically, the optical camera autofocus system during target tracking can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet, smart Bluetooth device, laptop, or personal computer (PC); the server can be a single server or a server cluster consisting of multiple servers.
[0029] In some embodiments, the optical camera autofocus system during target tracking can also be integrated into multiple electronic devices. For example, the optical camera autofocus system during target tracking can be integrated into multiple servers, and the optical camera autofocus method during target tracking of this application can be implemented by multiple servers.
[0030] In some embodiments, the server may also be implemented as a terminal.
[0031] The following sections provide detailed descriptions of each example. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.
[0032] In this embodiment, an automatic focusing method for an optical camera during target tracking is provided, such as... Figure 1a The specific process of the optical camera autofocusing method in the target tracking process can be shown as follows:
[0033] 101. Perform target object detection processing on the focusing decision image of the optical camera to obtain the target area corresponding to the target object in the focusing decision image.
[0034] An optical camera is an imaging device with target tracking capabilities. It can not only capture visible light images but also continuously lock onto and follow a specific target object by combining target detection and tracking algorithms. This optical camera can be used as a standalone intelligent vision unit or integrated into various mobile or fixed platforms. For example, it can be a vision module mounted on a security robot or inspection robot, or a camera system integrated into a drone or autonomous vehicle, and so on.
[0035] The focus decision image is a frame used in the autofocus process to determine whether zoom operation is needed and to determine the zoom step size. The focus decision image is usually selected from the video stream output by the optical camera.
[0036] The target object refers to the specific object in the imaging scene that needs to be identified, tracked, and focused. For example, the target object can be single or multiple, and these objects can be static objects (such as power poles, traffic signs, and building structures) or dynamically moving objects (such as pedestrians, vehicles, drones, or other moving targets).
[0037] The target region corresponding to the target object in the focus decision image refers to the specific location range of the target object in the focus decision image identified by the target detection algorithm. It is usually represented by a rectangular bounding box, which is uniquely determined by the coordinates of its upper left corner, width, and height, or by the coordinates of its two diagonal vertices.
[0038] In some embodiments, in order to enable the optical camera to periodically perform adaptive focusing and autofocusing operations during operation, the method further includes, before performing target object detection processing on the focusing decision image of the optical camera:
[0039] Based on a preset time interval, target object tracking processing is performed on the video stream output by the optical camera to obtain the target tracking confidence level;
[0040] When the target tracking confidence is lower than the preset confidence, the current frame is selected from the video stream as the focus decision image.
[0041] The preset time interval refers to the time period set in advance for periodic autofocus. This time interval can be configured according to the real-time requirements of the actual application scenario. For example, it can trigger a tracking confidence calculation every 200 milliseconds, 500 milliseconds, or every certain number of video frames (such as every 10 frames). Its purpose is to balance the consumption of computing resources and the frequency of status monitoring, and to avoid frequent calls to the target detection model.
[0042] A video stream refers to a sequence of images continuously output by an optical camera during operation, typically transmitted at a fixed frame rate (such as 15fps or 30fps). Each frame is a complete digital image, and the video stream serves as a real-time sensing data source for processing stages such as target tracking, focus decisions, and focus execution.
[0043] Target tracking confidence level refers to a numerical indicator used to quantify the reliability of the current tracking results during the target tracking process.
[0044] Preset reliability refers to a pre-set threshold used to determine whether target tracking is reliable.
[0045] The current frame refers to the latest frame of image acquired from the video stream output by the optical camera at the moment the focus adjustment decision is triggered. This frame has not yet undergone subsequent zoom or focus operations and is used as the input for the next round of focus adjustment decisions, ensuring that the system responds based on the latest scene state.
[0046] In some embodiments, to ensure that the optical camera can quickly enter a stable focusing state during the startup phase and avoid focusing failure or misadjustment due to unconfigured parameters, the method further includes the following before performing target object detection processing on the focusing decision image of the optical camera:
[0047] When the optical camera is started, the system parameters related to adjusting the focal length of the optical camera are initialized;
[0048] Based on the initialized system parameters, obtain the video stream captured by the optical camera;
[0049] Select the current frame from the video stream as the focus decision image.
[0050] Here, system parameters refer to parameters related to adjusting the focal length of the optical camera. These system parameters include the initial value of the optical camera's focal length. Preset expected proportion range equal Preset adaptive zoom step size range ,etc
[0051] The preset expected proportion range is a reasonable range of values for the ratio of the target area to the entire image area. It is used to guide the adaptive zoom process, ensuring that the target is neither too small (difficult to identify) nor too large (outside the field of view), thereby achieving a balance between tracking and imaging (e.g., 10% to 25%).
[0052] The preset adaptive zoom step size coefficient range is used to calculate the upper and lower limits of the zoom step size ratio coefficient. Specifically, it can be [10, 20].
[0053] The initialized system parameters refer to the system parameters set to preset default values or initial valid values loaded from non-volatile memory after the optical camera is powered on or reset, so that it is in a state that can perform adaptive focusing and focusing operations. The initialization process ensures that the system can still start the target detection, zoom decision and focus search process based on reasonable default configuration without relying on historical state, thereby improving the system startup robustness and the first image quality.
[0054] It is understandable that, such as Figure 1b As shown, after the optical camera system starts up, it first performs parameter initialization. These parameters include: the initial value of the optical camera's focal length. The preset expected proportion range of the target object in the image Focal length adjustment adaptive step size coefficient range (Usually set to [10,20]). After initialization, the camera begins capturing the video stream.
[0055] 102. Based on the deviation between the area ratio of the target region in the focusing decision image and the preset expected ratio range, and the current focal length of the optical camera, determine the adaptive zoom step size. The area ratio is the ratio of the area of the target region in the focusing decision image to the area of the focusing decision image. The adaptive zoom step size has a logarithmic relationship with the current focal length.
[0056] The area ratio is the ratio of the area of the target region in the focus decision image to the area of the focus decision image itself. The formula for calculating this area ratio R is:
[0057] R= Where w is the width of the target region, h is the height of the target region, W is the width of the focus decision image, and H is the height of the focus decision image.
[0058] Deviation refers to the degree of deviation between the area proportion of the target region in the focusing decision image and the preset expected proportion range.
[0059] The current focal length value refers to the actual focal length of the optical camera at the moment of making a zoom decision.
[0060] Adaptive zoom step size refers to dynamically adjusting the lens focal length by incrementing or decrementing based on the deviation between the area proportion of the target region in the focusing decision image and the preset desired proportion range, combined with the current focal length value of the optical camera. This step size aims to optimize image quality and target recognition performance by precisely controlling the zoom operation so that the image size of the target region in the image is as close as possible to the preset ideal proportion.
[0061] In some embodiments, to avoid system oscillation caused by sudden changes in step size during high-magnification zoom, and to combine deviation dynamic adjustment sensitivity to improve the smoothness of the zoom process and the continuity of target imaging while ensuring adjustment responsiveness, the adaptive zoom step size is determined based on the deviation between the area ratio corresponding to the target region in the focusing decision image and the preset expected ratio range, and the current focal length value of the optical camera, including:
[0062] Based on the deviation between the area ratio of the target region in the focusing decision image and the preset expected ratio range, the target adaptive zoom step coefficient is determined from the preset adaptive zoom step coefficient range.
[0063] The adaptive zoom step size is obtained by multiplying the target adaptive zoom step size coefficient by the logarithm of the current focal length value of the optical camera.
[0064] The preset adaptive zoom step size coefficient range refers to a pre-configured set of proportional coefficient ranges used to adjust zoom sensitivity, denoted as... For example, [10, 20]. This range determines the scaling capability of the adaptive zoom step size.
[0065] The target adaptive zoom step size coefficient refers to the deviation between the area proportion of the target region in the focusing decision image and the preset expected proportion range. The system value determined from the preset adaptive zoom step size coefficient range is within this range; the larger the deviation, the closer the selected coefficient is to the desired proportion. To speed up the adjustment process, the smaller the deviation, the closer the coefficient is to the desired value. This allows for fine-tuning.
[0066] The formula for calculating the target adaptive zoom step size coefficient k is:
[0067] ,in, It is the area proportion of the target region in the focusing decision image. This is the preset expected proportion range, and 10 is the adaptive zoom step size coefficient range. For low-speed, small target objects, the ideal expectation has a relatively small proportion, such as 0.003.
[0068] The logarithm of the current focal length value is ,in, It is the current focal length value of the optical camera.
[0069] Adaptive zoom step size refers to the amount of focal length adjustment used to drive the lens zoom mechanism. The calculation formula is:
[0070] ,in, The adaptive zoom step size coefficient for the target. This is used to counteract the effect of focal length sensitivity increasing with f. For example, when f = 100mm, if k = 15, then... When f=1000mm, the same coefficient k... This design allows for a longer step size. It increases moderately with increasing f, but the growth rate is suppressed by the logarithmic function to avoid abrupt changes. The adjustment direction is determined by the area ratio R: if If so, increase the focal length; If so, then reduce the focal length.
[0071] In some embodiments, to achieve smooth and efficient adjustment of the focus length, a target adaptive zoom step coefficient is determined from a preset adaptive zoom step coefficient range based on the deviation between the area proportion corresponding to the target region in the focus decision image and a preset desired proportion range, including:
[0072] Determine the difference between the area proportion of the target region in the focusing decision image and the preset expected proportion range;
[0073] The ratio of the percentage difference to the width of the preset expected percentage range is used as the normalized deviation, where the width is the difference between the maximum and minimum expected percentages within the preset expected percentage range.
[0074] The normalized deviation is multiplied by the minimum adaptive zoom step coefficient in the preset adaptive zoom step coefficient range to obtain the dynamic increment.
[0075] The target adaptive zoom step size coefficient is obtained by adding the dynamic increment to the minimum adaptive zoom step size coefficient.
[0076] Wherein, the proportion difference refers to the absolute deviation between the area proportion corresponding to the target region in the focusing decision image and the preset expected proportion range, where the proportion difference = .
[0077] The width of the preset expected proportion range refers to the difference between the maximum and minimum expected proportions within the preset expected proportion range. It is used to characterize the allowable fluctuation range of the target imaging scale and serves as the benchmark scale for normalized deviation calculation.
[0078] Normalization bias is a dimensionless quantity obtained by dividing the difference in proportions by the width of the preset expected proportion range, where normalization bias = .
[0079] The minimum adaptive zoom step size coefficient refers to the minimum value within the preset range of adaptive zoom step size coefficients, denoted as . It is used to provide basic zoom sensitivity when the target is close to the ideal imaging scale, ensuring fine adjustment can still be made with small deviations.
[0080] Dynamic increment refers to the additional adjustment amount obtained by multiplying the normalized deviation by the minimum adaptive zoom step size coefficient. It is used to dynamically enhance the zoom response intensity on top of the basic sensitivity, and its magnitude increases linearly with the degree of target deviation.
[0081] The target adaptive zoom step size coefficient refers to the proportional coefficient used to calculate the adaptive zoom step size. It is obtained by adding the minimum adaptive zoom step size coefficient and the dynamic increment. Its value changes adaptively with the degree to which the area ratio of the target region deviates from the preset expected ratio range.
[0082] For example, when f current When the diameter is 100mm, if k=15, then Δf=30; while when f current At a focal length of 1000mm, with the same coefficient k=15, Δf=45. This design allows the zoom step size Δf to increase moderately as the current focal length fcurrent increases, but its growth rate is suppressed by the nonlinearity of the logarithmic function, effectively avoiding overshoot or oscillation caused by sudden changes in step size during high-magnification zooming, thus improving the stability and continuity of the zooming process. Simultaneously, the zoom direction is dynamically determined by the area ratio R: if R≤R min If the target is too small in the image, increase the focal length to magnify the target; if R ≥ R max If the target is too large in the image or exceeds the field of view, the focal length is reduced to restore a reasonable imaging scale. In summary, this mechanism ensures a rapid response to changes in target scale while suppressing drastic adjustments at high focal lengths through nonlinear step size control, balancing zoom efficiency, control stability, and imaging robustness. It is particularly suitable for complex visual scenarios such as long-distance, small targets, and dynamic tracking.
[0083] It is understandable that, such as Figure 1b As shown, after the optical camera system initialization, the optical camera captures video frames and inputs them into the YOLOv5s object detection model. This pre-trained model can identify specific targets (such as drones) and output their bounding box coordinates (x, y, w, h), where x and y represent the coordinates of the top-left corner of the target region's bounding box in the image (focus decision image), and w and h represent the width and height of the target region's bounding box, respectively. YOLOv5s' lightweight design ensures real-time performance. Its network structure consists of three parts: Backbone, Neck, and Head, and it enhances multi-scale feature extraction through cross-stage local connections and spatial pyramids. After object detection, the system calculates the area ratio R, where w and h are the total width and height of the image. This area ratio R serves as a trigger indicator for focusing requirements. Below the threshold (e.g., 0.001) or higher than the threshold If the value is 0.03, then focus adjustment will be initiated. Focus adjustment aims to adjust the target's proportion. Maintain within the ideal range .in, This represents the ideal area ratio range (for low-speed, small targets, the ideal ratio value is relatively small, such as 0.003). This is used to counteract the effect of focal length sensitivity increasing with f.
[0084] 103. Adjust the current focal length of the optical camera according to the adaptive zoom step size.
[0085] The current focal length value refers to the actual effective focal length of the optical camera at the moment of making a zoom decision.
[0086] In some embodiments, such as Figure 1b As shown, the current focal length of the optical camera can be adjusted by the control module inside the optical camera or by an external processing unit (such as a server, edge computing device, etc.) that communicates with the optical camera, based on the adaptive zoom step size.
[0087] In some embodiments, zoom commands are sent to the stepper motor driving the lens via a camera control interface (such as USB or SDK). After the focal length is adjusted, the system re-detects the area ratio and repeats the above process until... It has entered the ideal range.
[0088] 104. After completing the focus adjustment, acquire the focusing execution image of the optical camera.
[0089] The focus execution image refers to a real-time image output by the image sensor of the optical camera after it has completed adaptive focus adjustment, used for performing local focusing operations. This image reflects the imaging state of the target object and its surrounding scene at the current focal length, serving as input for subsequent focusing lens group position search and sharpness evaluation. Unlike the focus decision image, the focus execution image is used in the fine focusing stage, focusing on the imaging sharpness of the target area rather than the target scale or zoom decision.
[0090] In some embodiments, the focus-performing image is typically acquired after the focus has stabilized to avoid image blurring caused by lens movement.
[0091] In some embodiments, to effectively avoid unnecessary lens movements, reduce mechanical wear and power consumption, and improve the response efficiency and stability of the overall focusing process, an adaptive zoom step size is determined based on the deviation between the area ratio corresponding to the target region in the focusing decision image and the preset expected ratio range, and the current focal length value of the optical camera, including:
[0092] If the area ratio of the target region in the focusing decision image does not fall within the preset expected ratio range, the adaptive zoom step size is determined based on the deviation between the area ratio of the target region in the focusing decision image and the preset expected ratio range, as well as the current focal length value of the optical camera.
[0093] Also includes:
[0094] If the area ratio of the target region in the focusing decision image falls within the preset expected ratio range, then the focus adjustment is not performed, and the focusing execution image of the optical camera is directly obtained.
[0095] Understandably, this area percentage R serves as a trigger indicator for focusing requirements: if the area percentage R falls below a threshold... (e.g., 0.001) or higher than the threshold If the value is 0.03, then focus adjustment will be initiated.
[0096] 105. Adjust the position of the focusing lens group of the optical camera, and search for the target focusing position that maximizes the sharpness evaluation value based on the sharpness evaluation value of the target area in the focusing execution image, so as to complete the autofocus.
[0097] In this context, the focusing lens group position refers to the physical or logical position of the movable lens group (i.e., the focusing lens group) in an optical camera, which is used to achieve the focusing function, along the optical axis. This position is typically controlled by a stepper motor, voice coil motor (VCM), or other drive mechanism inside the lens, and can be quantified (e.g., displacement in micrometers, or discrete position in motor steps) via encoders, position sensors, or control commands. Adjusting the focusing lens group position changes the relative distance between the imaging plane and the image sensor, thus affecting image sharpness.
[0098] A sharpness evaluation value is a numerical metric used to quantify the sharpness of a local region in an image. It is calculated using pixel information of the target region in the image being focused. Common sharpness evaluation functions include, but are not limited to, the sum of squared gradients (Tenengrad), Laplacian variance, Brenner gradient, and energy entropy. A larger value generally indicates a sharper region; conversely, a smaller value indicates a blurrier region. In this application, the sharpness evaluation value is calculated solely based on the target region of the object in the image being focused, to eliminate background interference and improve focusing accuracy.
[0099] The target focus position refers to the position of the focusing lens group that maximizes the sharpness evaluation value of the target area within the adjustable travel range of the focusing lens group. This target focus position is the optimal focus point obtained through the search, which is used to drive the focusing lens group to complete the final positioning and achieve accurate autofocus on the target object.
[0100] In some embodiments, to effectively avoid interference from complex backgrounds, motion blur, or irrelevant objects on global focus judgment, and significantly improve the image sharpness of the target area and the system's focusing robustness, especially suitable for precise imaging needs in small targets, long distances, or high dynamic scenes, the position of the focusing lens group of the optical camera is adjusted, and based on the sharpness evaluation value corresponding to the target area in the image where the focus is performed, the target focus position that maximizes the sharpness evaluation value is searched, including:
[0101] The target object is detected in the image being focused, and the target region corresponding to the target object in the image being focused is obtained.
[0102] The sharpness of the target region in the focused image is evaluated using the gradient sum of squares function, and the sharpness evaluation value of the target region in the focused image is obtained.
[0103] A hill-climbing search strategy is adopted to gradually adjust the position of the focusing lens group near the current focusing position of the optical camera and compare the corresponding sharpness evaluation values until the target focusing position that maximizes the sharpness evaluation value locally is found.
[0104] It is understandable that the gradient sum of squares function E= Where m and n are the target region dimensions, and the partial derivatives are calculated using the Sobel operator. Focus control is optimized using a hill-climbing method, fine-tuning the position of the focusing lens group, and searching to achieve... Maximized focal length value During this process, the system only calculates the target area, ignoring the background area that occupies most of the area, which can effectively avoid interference from clouds and other factors.
[0105] Hill Climbing Search Strategy refers to a local optimization search algorithm that, during the focusing process, starts from the current position of the focusing lens group and gradually moves the focusing lens group along the focusing travel direction (such as forward or backward). It calculates the sharpness evaluation value of the target area corresponding to each step in real time, and retains the position with the higher sharpness evaluation value as the new starting point. It continues to iterate until the evaluation value no longer increases or reaches the upper limit of the preset search range.
[0106] In some embodiments, in order to focus on the imaging quality of the target itself and effectively avoid interference from cluttered backgrounds, lighting changes, or irrelevant objects on focus determination, a gradient squared sum function is used to evaluate the sharpness of the target region in the focusing execution image, obtaining a sharpness evaluation value corresponding to the target region in the focusing execution image, including:
[0107] The target region in the focused image is cropped to obtain a target region image block;
[0108] The sharpness of the target region image patch is evaluated using the gradient sum of squares function, and the sharpness evaluation value corresponding to the target region image patch is obtained.
[0109] The target region image patch refers to a sub-image region containing only the target object, extracted from the image after focusing based on the target detection results. This image patch is defined by the bounding box of the target region, and its size and location are identical to the target region in the original image. It is used for subsequent local sharpness calculations. By limiting the sharpness evaluation to this image patch, interference from other areas of the image can be eliminated, enabling accurate focus assessment of the target object.
[0110] In some embodiments, such as Figure 1b As shown, in order to improve the self-recovery capability, imaging stability, and target tracking continuity of the optical camera in complex dynamic scenes, and to avoid the limitation of "one-shot success or failure," after completing autofocus, it also includes:
[0111] Acquire the target image output by the optical camera after focusing;
[0112] The sharpness evaluation value of the target region corresponding to the target object in the target image is compared with a preset sharpness threshold.
[0113] If the sharpness evaluation value of the target region in the target image is lower than the preset sharpness threshold, the target image will be used as the new focusing decision image.
[0114] Based on the deviation between the area ratio of the target region in the new focusing decision image and the preset expected ratio range, and the current focal length of the optical camera, a new adaptive zoom step size is determined.
[0115] Adjust the current focal length of the optical camera according to the new adaptive zoom step size;
[0116] After completing the focus adjustment, acquire the new focus execution image from the optical camera;
[0117] Adjust the position of the focusing lens group of the optical camera, and search for the target focusing position that maximizes the sharpness evaluation value based on the sharpness evaluation value of the target area in the new focusing execution image, so as to complete the autofocus.
[0118] The target image refers to the final imaging result output by the optical camera after completing a full autofocus operation. This target image reflects the target imaging state under the combined effect of the current focal length and the position of the focusing lens group. It is used to evaluate whether the focus quality meets the standard and serves as the input basis for subsequent closed-loop verification.
[0119] The preset sharpness threshold is a minimum evaluation value set by the system to determine whether the image sharpness of a target area is acceptable. This threshold can be calibrated according to the application scenario (such as security monitoring, power line inspection), target size, or ambient lighting conditions. Typical values are determined based on historical data statistics or experimental tests. When the sharpness evaluation value is lower than the preset sharpness threshold, it is considered a focus failure or insufficient image quality, and a readjustment process needs to be initiated.
[0120] The new focus decision image refers to the current target image that is reused as the input image for the next zoom decision when the focus quality is not up to standard. Its function is the same as the initial focus decision image: to recalculate the area ratio of the target region, and then assess whether the focus needs to be adjusted again, thus achieving closed-loop control of "focusing - assessment - re-zooming".
[0121] The new adaptive zoom step size refers to the adaptive zoom step size recalculated after closed-loop verification is triggered, based on the area ratio deviation of the target region in the new focus decision image and the updated current focal length value. Its calculation logic is the same as the initial zoom step size, but the input parameters are the latest state, ensuring that the adjustment strategy matches the current scene.
[0122] The new focus execution image refers to the image output by the optical camera after the focal length adjustment is completed according to the new adaptive zoom step size, used to perform a new round of local focusing operations. This image is used to extract the target area and calculate the sharpness evaluation value to search for a new optimal focus position and complete two (or more) focusing iterations.
[0123] Understandably, the entire optical camera system operates on a "zoom priority, focus subsequent" principle. The focusing and autofocus modules work together through a state machine. If the target's proportion is abnormal, the focal length is adjusted first; then, the focus value is optimized. The system periodically evaluates the target tracking confidence (e.g., based on detection scores) at preset time intervals. If the target tracking confidence is lower than the preset confidence, the detection model is reinitialized. The entire solution achieves low-latency processing through hardware interfaces (such as FPGAs or embedded processors), ensuring real-time control within a 30-frame-per-second video stream, forming a highly robust autofocus system capable of handling changes in target distance and size, as well as complex background interference.
[0124] During target tracking, changes in target distance alter its image size. If the camera's focal length fails to adjust in time, the target may become too large and exceed the field of view, or too small and lose detail. Both scenarios hinder the target recognition algorithm from quickly and stably extracting target information from the image, and can easily lead to target tracking failure. Traditional systems use a fixed motor step size to adjust the focal length. However, since the focusing step size of a telephoto camera increases with the focal length f, a fixed motor step size at large f can cause excessively large abrupt changes in focal length, resulting in image blurring, loss of the tracked target, and interruption of the tracking process.
[0125] In camera autofocus, traditional methods rely on the overall image sharpness. When the target is extremely small (e.g., a drone against a sky background), the background area contributes far more to the sharpness than the target area, causing the focusing system to focus on the background while the target remains blurry. For example, the gradient energy of cloud textures may account for more than 99% of the entire image, but the focusing algorithm may misjudge the clouds as the main subject that needs to be clearly imaged, causing the drone target to go out of focus.
[0126] To address the aforementioned shortcomings, this application utilizes a lightweight deep learning object detection model (such as YOLOv5s) to detect the target to be tracked from the video stream in real time and obtain its precise bounding box. Then, for the zoom control problem, an adaptive zoom step size algorithm is designed. This step size is no longer a fixed value but rather has a logarithmic relationship with the current focal length, ensuring a gradual increase in step size at the telephoto end to avoid abrupt focal length changes, and a relatively larger step size at the short focal length end to guarantee adjustment efficiency, thereby smoothly and stably adjusting the target size to the preset desired proportion range. Finally, for the autofocus problem, the traditional full-image focusing strategy is abandoned. Only the image region within the target bounding box is extracted, and the sharpness evaluation function of this local region is calculated. This function drives the focusing module to search, ensuring that the focusing operation is always target-centric and effectively eliminating background interference.
[0127] In summary, this application introduces an adaptive step size adjustment mechanism for the camera focal length. By using a logarithmic function to constrain the focal length adjustment, the step size increases slowly within the telephoto range, avoiding image blurring caused by abrupt changes and thus improving the stability of target tracking during camera zoom. Based on the sharpness of the local focus area of the target region, background interference is eliminated, ensuring that the sharpness evaluation function only responds to changes in target texture, resulting in precise focus on the target.
[0128] To better implement the above methods, this application also provides an optical camera autofocus system during target tracking. This system can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet, smart Bluetooth device, laptop, or personal computer; the server can be a single server or a server cluster consisting of multiple servers.
[0129] For example, in this embodiment, the method of this application embodiment will be described in detail by taking an optical camera autofocus system specifically integrated into an electronic device during target tracking as an example.
[0130] For example, such as Figure 2 As shown, the optical camera autofocus system in the target tracking process may include a target detection unit 201, a zoom step size determination unit 202, a focusing unit 203, an image acquisition unit 204, and a focusing unit 205, as follows:
[0131] (I) Target detection unit 201.
[0132] The target detection unit 201 is used to perform target object detection processing on the focusing decision image of the optical camera to obtain the target area corresponding to the target object in the focusing decision image.
[0133] In some embodiments, prior to performing target object detection processing on the focusing decision image of the optical camera, the method further includes:
[0134] Based on a preset time interval, target object tracking processing is performed on the video stream output by the optical camera to obtain the target tracking confidence level;
[0135] When the target tracking confidence is lower than the preset confidence, the current frame is selected from the video stream as the focus decision image.
[0136] In some embodiments, prior to performing target object detection processing on the focusing decision image of the optical camera, the method further includes:
[0137] When the optical camera is started, the system parameters related to adjusting the focal length of the optical camera are initialized;
[0138] Based on the initialized system parameters, obtain the video stream captured by the optical camera;
[0139] Select the current frame from the video stream as the focus decision image.
[0140] (ii) Zoom step size determination unit 202.
[0141] The zoom step size determination unit 202 is used to determine the adaptive zoom step size based on the deviation between the area ratio of the target region in the focus decision image and the preset expected ratio range, and the current focal length value of the optical camera. The area ratio is the ratio of the area of the target region in the focus decision image to the area of the focus decision image. The adaptive zoom step size has a logarithmic relationship with the current focal length value.
[0142] In some embodiments, the adaptive zoom step size is determined based on the deviation between the area proportion corresponding to the target region in the focus decision image and a preset desired proportion range, and the current focal length value of the optical camera, including:
[0143] Based on the deviation between the area ratio of the target region in the focusing decision image and the preset expected ratio range, the target adaptive zoom step coefficient is determined from the preset adaptive zoom step coefficient range.
[0144] The adaptive zoom step size is obtained by multiplying the target adaptive zoom step size coefficient by the logarithm of the current focal length value of the optical camera.
[0145] In some embodiments, determining the target adaptive zoom step coefficient from a preset adaptive zoom step coefficient range based on the deviation between the area proportion corresponding to the target region in the focus decision image and a preset expected proportion range includes:
[0146] Determine the difference between the area proportion of the target region in the focusing decision image and the preset expected proportion range;
[0147] The ratio of the percentage difference to the width of the preset expected percentage range is used as the normalized deviation, where the width is the difference between the maximum and minimum expected percentages within the preset expected percentage range.
[0148] The normalized deviation is multiplied by the minimum adaptive zoom step coefficient in the preset adaptive zoom step coefficient range to obtain the dynamic increment.
[0149] The target adaptive zoom step size coefficient is obtained by adding the dynamic increment to the minimum adaptive zoom step size coefficient.
[0150] (III) Focusing unit 203.
[0151] The focusing unit 203 is used to adjust the current focal length value of the optical camera according to the adaptive zoom step size.
[0152] (iv) Image acquisition unit 204.
[0153] The image acquisition unit 204 is used to acquire the focusing execution image of the optical camera after the focus adjustment is completed.
[0154] In some embodiments, the adaptive zoom step size is determined based on the deviation between the area proportion corresponding to the target region in the focus decision image and a preset desired proportion range, and the current focal length value of the optical camera, including:
[0155] If the area ratio of the target region in the focusing decision image does not fall within the preset expected ratio range, the adaptive zoom step size is determined based on the deviation between the area ratio of the target region in the focusing decision image and the preset expected ratio range, as well as the current focal length value of the optical camera.
[0156] Also includes:
[0157] If the area ratio of the target region in the focusing decision image falls within the preset expected ratio range, then the focus adjustment is not performed, and the focusing execution image of the optical camera is directly obtained.
[0158] (v) Focusing unit 205.
[0159] The focusing unit 205 is used to adjust the position of the focusing lens group of the optical camera and search for the target focusing position that maximizes the sharpness evaluation value based on the sharpness evaluation value corresponding to the target area in the focusing execution image, so as to complete the autofocus.
[0160] In some embodiments, adjusting the position of the focusing lens group of the optical camera and searching for the target focusing position that maximizes the sharpness evaluation value based on the sharpness evaluation value corresponding to the target area in the focusing execution image includes:
[0161] The target object is detected in the image being focused, and the target region corresponding to the target object in the image being focused is obtained.
[0162] The sharpness of the target region in the focused image is evaluated using the gradient sum of squares function, and the sharpness evaluation value of the target region in the focused image is obtained.
[0163] A hill-climbing search strategy is adopted to gradually adjust the position of the focusing lens group near the current focusing position of the optical camera and compare the corresponding sharpness evaluation values until the target focusing position that maximizes the sharpness evaluation value locally is found.
[0164] In some embodiments, the sharpness evaluation of the target region in the focusing execution image is performed using the gradient sum of squares function to obtain a sharpness evaluation value corresponding to the target region in the focusing execution image, including:
[0165] The target region in the focused image is cropped to obtain a target region image block;
[0166] The sharpness of the target region image patch is evaluated using the gradient sum of squares function, and the sharpness evaluation value corresponding to the target region image patch is obtained.
[0167] In some embodiments, after autofocus is completed, the method further includes:
[0168] Acquire the target image output by the optical camera after focusing;
[0169] The sharpness evaluation value of the target region corresponding to the target object in the target image is compared with a preset sharpness threshold.
[0170] If the sharpness evaluation value of the target region in the target image is lower than the preset sharpness threshold, the target image will be used as the new focusing decision image.
[0171] Based on the deviation between the area ratio of the target region in the new focusing decision image and the preset expected ratio range, and the current focal length of the optical camera, a new adaptive zoom step size is determined.
[0172] Adjust the current focal length of the optical camera according to the new adaptive zoom step size;
[0173] After completing the focus adjustment, acquire the new focus execution image from the optical camera;
[0174] Adjust the position of the focusing lens group of the optical camera, and search for the target focusing position that maximizes the sharpness evaluation value based on the sharpness evaluation value of the target area in the new focusing execution image, so as to complete the autofocus.
[0175] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.
[0176] As described above, in the target tracking process of this embodiment, the optical camera autofocus system uses a target detection unit to detect the target object in the focusing decision image of the optical camera, obtaining the target area corresponding to the target object in the focusing decision image. A zoom step size determination unit determines an adaptive zoom step size based on the deviation between the area ratio of the target area in the focusing decision image and a preset expected ratio range, as well as the current focal length of the optical camera. The area ratio is the ratio of the area of the target area in the focusing decision image to the area of the focusing decision image itself, and the adaptive zoom step size is logarithmically related to the current focal length. The focusing unit adjusts the current focal length of the optical camera according to the adaptive zoom step size. After completing the focal length adjustment, the image acquisition unit acquires the focusing execution image of the optical camera. The focusing unit adjusts the position of the focusing lens group of the optical camera and, based on the sharpness evaluation value corresponding to the target area in the focusing execution image, searches for the target focusing position that maximizes the sharpness evaluation value, thereby completing autofocus.
[0177] Therefore, the embodiments of this application adaptively adjust the focal length of the optical camera and achieve localized and precise focusing on the target area, effectively suppressing background interference and improving the imaging clarity of the target object and the overall image quality.
[0178] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.
[0179] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the optical camera autofocusing methods provided in the target tracking process according to embodiments of this application.
[0180] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0181] Since the instructions stored in the storage medium can execute the steps in any of the UAV optimal interception decision methods based on model predictive control provided in the embodiments of this application, the beneficial effects that any of the optical camera autofocusing methods in the target tracking process provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.
[0182] According to one aspect of this application, a computer program product or computer program is provided, comprising a computer program / instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer program / instructions from the computer-readable storage medium and executes the computer program / instructions, causing the electronic device to perform the method provided in the above embodiments for autofocusing of an optical camera during target tracking.
[0183] The above provides a detailed description of an optical camera autofocusing method and system for target tracking provided by the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. An automatic focusing method of an optical camera in a target tracking process, characterized by, The method comprises the following steps: detecting a target object in a focus decision image of an optical camera to obtain a target area corresponding to the target object in the focus decision image; determining an adaptive zoom step based on a deviation between an area ratio of the target area in the focus decision image and a preset expected area ratio range and a current focal length value of the optical camera, wherein the area ratio is a ratio of an area of the target area in the focus decision image to an area of the focus decision image, and the adaptive zoom step is in a logarithmic relationship with the current focal length value; adjusting the current focal length value of the optical camera according to the adaptive zoom step; after the focal length adjustment is completed, obtaining a focus execution image of the optical camera; adjusting a focus lens group position of the optical camera, and searching for a target focus position that maximizes a sharpness evaluation value of the target area of the target object in the focus execution image to complete automatic focusing.
2. The method of claim 1, wherein, The method comprises the following steps: determining a target adaptive zoom step coefficient from a preset adaptive zoom step coefficient range according to a deviation between the area ratio of the target area in the focus decision image and the preset expected area ratio range; multiplying the target adaptive zoom step coefficient and a logarithm of the current focal length value of the optical camera to obtain the adaptive zoom step.
3. The method of claim 2, wherein, The method comprises the following steps: determining an area ratio difference between the area ratio of the target area in the focus decision image and the preset expected area ratio range; taking a ratio of the area ratio difference and a width of the preset expected area ratio range as a normalized deviation, wherein the width is a difference between a maximum expected area ratio and a minimum expected area ratio in the preset expected area ratio range; multiplying the normalized deviation and a minimum adaptive zoom step coefficient in the preset adaptive zoom step coefficient range to obtain a dynamic increment; adding the dynamic increment and the minimum adaptive zoom step coefficient to obtain the target adaptive zoom step coefficient.
4. The method of claim 1, wherein, The method comprises the following steps: detecting a target object in the focus execution image to obtain a target area corresponding to the target object in the focus execution image; performing sharpness evaluation processing on the target area in the focus execution image by using a sum of squared gradients function to obtain a sharpness evaluation value of the target area in the focus execution image; and The hill climbing search strategy is adopted to gradually adjust the focusing lens position near the current focusing position of the optical camera, and the corresponding sharpness evaluation value is compared until the target focusing position is found, which makes the sharpness evaluation value reach the local maximum.
5. The method of claim 4, wherein, The evaluation processing of the sharpness of the target region in the focusing execution image is performed by using the gradient square sum function, and the sharpness evaluation value corresponding to the target region in the focusing execution image is obtained. The target region image block is obtained by performing the cropping processing on the target region in the focusing execution image. The evaluation processing of the sharpness of the target region image block is performed by using the gradient square sum function, and the sharpness evaluation value corresponding to the target region image block is obtained.
6. The method of claim 1, wherein, The adaptive zoom step is determined based on the deviation between the area ratio corresponding to the target region in the focusing decision image and the preset expected ratio range, and the current focal length value of the optical camera. If the area ratio corresponding to the target region in the focusing decision image does not fall within the preset expected ratio range, the adaptive zoom step is determined based on the deviation between the area ratio corresponding to the target region in the focusing decision image and the preset expected ratio range, and the current focal length value of the optical camera. Further comprising: If the area ratio corresponding to the target region in the focusing decision image falls within the preset expected ratio range, the focal length adjustment is not performed, and the focusing execution image of the optical camera is directly obtained.
7. The method of claim 1, wherein, Before the detection processing of the target object on the focusing decision image of the optical camera is performed, further comprising: The target tracking confidence is obtained by performing the tracking processing of the target object on the video stream output by the optical camera based on a preset time interval; When the target tracking confidence is lower than a preset confidence, the current frame is selected from the video stream as the focusing decision image.
8. The method of claim 1, wherein, Before the detection processing of the target object on the focusing decision image of the optical camera is performed, further comprising: When the optical camera is started, the system parameters related to the adjustment of the focal length of the optical camera are initialized; The video stream photographed by the optical camera is obtained based on the initialized system parameters; The current frame is selected from the video stream as the focusing decision image.
9. The method of claim 1, wherein, After the automatic focusing is completed, further comprising: The target image output after the focusing of the optical camera is completed is obtained; The sharpness evaluation value of the target region corresponding to the target object in the target image is compared with a preset sharpness threshold value; If the sharpness evaluation value of the target region in the target image is lower than the preset sharpness threshold value, the target image is taken as a new focusing decision image; A new adaptive zoom step is determined based on the deviation between the area ratio corresponding to the target region in the new focusing decision image and the preset expected ratio range, and the current focal length value of the optical camera; The current focal length value of the optical camera is adjusted according to the new adaptive zoom step; After the focal length adjustment is completed, a new focusing execution image of the optical camera is obtained; Adjust a focus lens group position of the optical camera, and search for a target focus position that maximizes a sharpness evaluation value of a target region corresponding to the target object in a focus execution image of the optical camera based on the sharpness evaluation value, to complete auto-focusing.
10. An optical camera auto focus system in a target tracking process, characterized in that, The method comprises the steps of: a target detection unit configured to perform target object detection processing on a focus decision image of an optical camera to obtain a target region corresponding to the target object in the focus decision image; a zoom step determination unit configured to determine an adaptive zoom step based on a deviation between an area proportion corresponding to the target region in the focus decision image and a preset expected proportion range, and a current focal length value of the optical camera, the area proportion being a ratio of an area of the target region in the focus decision image to an area of the focus decision image, the adaptive zoom step being in a logarithmic relationship with the current focal length value; a focus unit configured to adjust the current focal length value of the optical camera according to the adaptive zoom step; an image acquisition unit configured to acquire a focus execution image of the optical camera after the focal length adjustment is completed; a focus unit configured to adjust a focus lens group position of the optical camera, and search for a target focus position that maximizes a sharpness evaluation value of a target region corresponding to the target object in a focus execution image of the optical camera based on the sharpness evaluation value, to complete auto-focusing.