Wide-area scene optimal focal length optimizing method based on stepless zooming

Abstract

The invention discloses a wide-area scene optimal focal length optimizing method based on stepless zooming, relates to the technical field of machine vision and target tracking, and aims to change the focal length by using a pan-tilt camera and adaptively adjusting the size of a target in a picture, so that mounting equipment can be effectively utilized. By means of the stepless zooming optimization technology, the focal length can be dynamically adjusted according to the real-time relative distance between the target and the camera and the change of the field angle, and therefore it is ensured that the size of the target in the picture is matched with the optimal confidence degree of the tracking model. And the definition of the image and the high confidence coefficient of target tracking are always ensured no matter how far the target distance or the size changes. The method does not need to manually adjust the focal length of the lens, does not need to deploy additional equipment, and is suitable for various visual detection scenes under unmanned aerial vehicles and long-focus monitoring view angles.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision and target tracking, and in particular to a method for finding the best focal length of a wide-area scene based on stepless zoom. Background Art

[0002] With the rapid development of the low-altitude economy, drones and high-altitude cameras are increasingly used in many fields such as security monitoring, agricultural monitoring and disaster relief, playing an important role. Drones can efficiently and flexibly collect real-time data over a large range, providing strong support for various tasks. However, when tracking targets in high altitude and wide-area scenes, traditional fixed focal length or limited zoom cameras often cannot effectively cope with changes in target distance and motion. The acquired target size is usually small and the texture details are sparse, which makes it difficult to capture its rich features, thus posing a challenge to visual detection. Especially in dynamic scenes, the distance and size of the target are constantly changing, and ensuring the clarity and details of the target has become a major difficulty in technical implementation. Summary of the invention

[0003] In order to overcome the deficiencies of the above technologies, the present invention provides a method for dynamically adjusting the focal length according to the real-time relative distance and field of view angle changes of a target domain camera.

[0004] The technical solution adopted by the present invention to overcome the technical problems is:

[0005] A method for finding the best focal length of a wide-area scene based on stepless zoom, comprising:

[0006] S1. The gimbal camera detects the target and obtains a target bounding box for the selected target;

[0007] S2. Determine whether the center of the image captured by the gimbal camera is within the target bounding box. If yes, execute step S3. If not, adjust the pitch angle and yaw angle of the gimbal camera until the center of the image captured by the gimbal camera is within the target bounding box.

[0008] S3. The gimbal camera zooms step by step according to the focal length value in the focal length zoom coefficient Zoom list and records the detection frame information of the target;

[0009] S4. Calculate the best zoom focal length best_zoom when the target is static according to the detection frame information of the target;

[0010] S5. Calculate the real length r of the target;

[0011] S6. Calculate the optimal dynamic focal length l′ of the target based on the actual length r of the target.

[0012] Furthermore, in step S1, if the gimbal camera is installed on the drone, the drone enters a hovering mode.

[0013] Furthermore, in step S1, the YOLOv8 and ReID models are used to perform target detection and feature matching on the image captured by the gimbal camera to obtain a target bounding box of the selected target, thereby completing the target locking.

[0014] Preferably, in step S1, the initial focal length zoom factor of the gimbal camera is Zoom=1.0.

[0015] Further, step S2 includes the following steps:

[0016] S2-1. The center coordinates of the image captured by the PTZ camera are calculated by the formula Center_image = (W / 2, H / 2), where W is the width of the image captured by the PTZ camera, and H is the height of the image captured by the PTZ camera;

[0017] S2-2. The center point coordinates of the target bounding box are calculated by the formula Center_box=(w / 2,h / 2), where w=x2-x1, h=y2-y1, x1 is the horizontal coordinate of the upper left corner of the target bounding box, y1 is the vertical coordinate of the upper left corner of the target bounding box, x2 is the horizontal coordinate of the lower right corner of the target bounding box, and y2 is the vertical coordinate of the lower right corner of the target bounding box;

[0018] S2-3. If the center point coordinates Center_image of the image captured by the PTZ camera coincide with the center point coordinates Center_box of the target bounding box, it is determined that the center of the image captured by the PTZ camera is within the target bounding box. Further, step S3 includes the following steps:

[0019] S3-1. Traverse the index values ​​of the focal length zoom coefficient Zoom list provided by the gimbal camera manufacturer. The gimbal camera zooms step by step according to the corresponding focal length value in the focal length zoom coefficient Zoom list. The focal length zoom coefficient of the i-th zoom is Z i , i∈{1,...,m}, m is the total number of focal lengths of the gimbal camera;

[0020] S3-2. Create a list, and after the pan / tilt camera completes the i-th zoom, the width w of the target bounding box is updated every Nms. i , the height of the target bounding box h i , after the i-th zoom is completed, the model mixed confidence p is used to detect the picture taken by the gimbal camera using the YOLOv8 and ReID models i , the focal length scaling factor Z of the i-th zoom i Store in a list;

[0021] S3-3. Select the target bounding box width w after the i-th zoom is completed in the list i, height h i Compare and select the maximum value as the target long side pixel value l i .

[0022] In step S3-2, N is set to 500, and the formula w i =x2 i -x1 i Calculate the width w of the target bounding box after the pan / tilt camera completes the i-th zoom i , where x2 i is the horizontal coordinate of the lower right corner of the target bounding box, x1 i is the horizontal coordinate of the upper left corner of the target bounding box, through the formula h i =y2 i -y1 i Calculate the height h of the target bounding box after the pan / tilt camera completes the i-th zoom i , where, y2 i is the ordinate of the lower right corner of the target bounding box, y1 i The ordinate of the upper left corner of the target bounding box.

[0023] Further, step S4 includes the following steps:

[0024] S4-1. By formula Calculate the harmonic weight pf, where l 1.0 is the target long side pixel value when the initial focal length zoom factor of the gimbal camera is Zoom = 1.0, l j is the target long side pixel value of the gimbal camera after the jth zoom is completed, j∈{1,...,m} and j≠1, L is the image long side pixel value of the picture taken by the gimbal camera;

[0025] S4-2. Through the formula p i ′=pf×p i Calculate the adjustment confidence p of the gimbal camera after the i-th zoom is completed i ′;

[0026] S4-3. Select all m maximum values ​​of the adjusted confidence, and adjust the target long side pixel value l of the gimbal camera corresponding to the maximum value of the adjusted confidence after the oth zoom is completed o , o∈{1,...,m}, if The focal length scaling factor at the oth zooming is used as the best zoom focal length best_zoom when the target is static.

[0027] Further, step S5 includes the following steps:

[0028] S5-1. The straight-line distance dis between the gimbal camera and the target is calculated by the formula dis=alt / cosθ, where alt is the height from the gimbal camera to the target, θ=90°+pitch, and pitch is the pitch angle of the gimbal camera;

[0029] S5-2. By formula r = (l o ×R) / L to calculate the real length r of the target, where R is the real length of the long side of the image, R = tanα×dis, α is the field of view of the gimbal camera, l o / r=960 / R.

[0030] Further, step S6 includes the following steps:

[0031] S6-1. The angle β between the straight-line distance from the gimbal camera to the center of the target and the side length of the target is calculated by the formula β = arctan (r / dis);

[0032] S6-2. By formula l i ′=l o -lb i Calculate the target dynamic difference l after the i-th zoom is completed i ′, where lb i =(s i ×L) / S i , si is the size of the target in the gimbal camera image sensor after the i-th zoom is completed, s i =f i ×tanβ,f i The focal length scaling factor of the pan-tilt camera for the i-th zoom is Z i The corresponding focal length value, S i S is the size of the gimbal camera image sensor after the i-th zoom is completed, i =f i ×tanα, take the focal length value corresponding to the minimum value of all m target dynamic differences as the optimal focal length.

[0033] The beneficial effect of the present invention is that by using a gimbal camera to adaptively adjust the size of the target in the picture to change the focal length, the mounting equipment can be effectively utilized. With the help of stepless zoom optimization technology, the focal length can be dynamically adjusted according to the real-time relative distance between the target and the camera and the change of the field of view angle, thereby ensuring that the target size in the picture matches the best confidence of the tracking model. Regardless of the distance of the target or the size change, the clarity of the image and the high confidence of the target tracking are always guaranteed. This method does not require manual adjustment of the lens focal length or deployment of additional equipment, and is suitable for various visual detection scenarios under drones and telephoto monitoring perspectives. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a flow chart of the method of the present invention;

[0035] Figure 2 A schematic diagram of the relationship between the focal length scaling factor and the target side length of the present invention;

[0036] Figure 3 A schematic diagram of calculating the straight-line distance between the pan-tilt camera and the target of the present invention;

[0037] Figure 4 A schematic diagram of calculating the actual size of the target of the present invention;

[0038] Figure 5 This is a schematic diagram of focal length adjustment calculation of the present invention. DETAILED DESCRIPTION

[0039] The following is combined with Figure 1 To Attachment Figure 5 The present invention is further described.

[0040] As attached Figure 1 As shown, a method for finding the best focal length of a wide-area scene based on stepless zoom includes:

[0041] S1. The gimbal camera detects the target and obtains a target bounding box for the selected target.

[0042] S2. Determine whether the center of the image captured by the gimbal camera is within the target bounding box. If yes, execute step S3. If not, adjust the pitch angle and yaw angle of the gimbal camera until the center of the image captured by the gimbal camera is within the target bounding box.

[0043] S3. The gimbal camera zooms step by step according to the focal length value in the focal length zoom coefficient Zoom list and records the detection frame information of the target.

[0044] S4. Calculate the best zoom focal length best_zoom when the target is static according to the detection frame information of the target.

[0045] S5. Calculate the true length r of the target.

[0046] S6. Calculate the optimal dynamic focal length l′ of the target based on the actual length r of the target.

[0047] With a gimbal camera, when the target is first locked, the algorithm automatically adjusts the focal length to ensure that the target is in the center of the picture and reaches the optimal pixel size. During the movement of the target, the camera or camera can continuously ensure that the target maintains the appropriate pixel size in the picture. The entire process uses adaptive stepless zoom technology, which can dynamically adjust the focal length according to the real-time relative distance between the target and the camera and the change in the field of view angle, ensuring that the image clarity and high precision of target tracking are always guaranteed regardless of the distance or size of the target. Not only can it perform adaptive focusing when the target is first locked, but it can also continuously and stably track the target during the movement of the target, ensuring that the image is always clear and avoiding the impact of out-of-focus or blur on the tracking model. This technology significantly improves the operating efficiency and accuracy of drones and high-altitude cameras in complex environments, and is particularly suitable for scenes that require continuous tracking of targets, and has broad application prospects.

[0048] In one embodiment of the present invention, if the gimbal camera is installed on the drone in step S1, the drone enters a hovering mode.

[0049] In one embodiment of the present invention, in step S1, the YOLOv8 and ReID models are used to perform target detection and feature matching on the image captured by the gimbal camera, and a target bounding box for the selected target is obtained to complete the target locking. In one embodiment of the present invention, the initial focal length zoom factor of the gimbal camera in step S1 is Zoom=1.0. In one embodiment of the present invention, step S2 includes the following steps:

[0050] S2-1. The center point coordinates Center_image of the image captured by the gimbal camera are calculated by the formula Center_image=(W / 2,H / 2), where W is the width of the image captured by the gimbal camera, and H is the height of the image captured by the gimbal camera.

[0051] S2-2. The center point coordinates of the target bounding box Center_box are calculated by the formula Center_box = (w / 2, h / 2), where w = x2-x1, h = y2-y1, x1 is the horizontal coordinate of the upper left corner of the target bounding box, y1 is the vertical coordinate of the upper left corner of the target bounding box, x2 is the horizontal coordinate of the lower right corner of the target bounding box, and y2 is the vertical coordinate of the lower right corner of the target bounding box.

[0052] S2-3. If the center point coordinates Center_image of the image captured by the PTZ camera coincide with the center point coordinates Center_box of the target bounding box, it is determined that the center of the image captured by the PTZ camera is within the target bounding box. In one embodiment of the present invention, step S3 includes the following steps:

[0053] S3-1. Traverse the index values ​​of the focal length zoom coefficient Zoom list provided by the gimbal camera manufacturer. The gimbal camera zooms step by step according to the corresponding focal length value in the focal length zoom coefficient Zoom list. The focal length zoom coefficient of the i-th zoom is Z i , i∈{1,...,m}, m is the total focal length of the gimbal camera.

[0054] S3-2. Create a list, and after the pan / tilt camera completes the i-th zoom, the width w of the target bounding box is updated every Nms. i , the height of the target bounding box h i , after the i-th zoom is completed, the model mixed confidence p is used to detect the picture taken by the gimbal camera using the YOLOv8 and ReID models i , the focal length scaling factor Z of the i-th zoom i Store in a list.

[0055] S3-3. Select the target bounding box width w after the i-th zoom is completed in the list i , height h i Compare and select the maximum value as the target long side pixel value l i .

[0056] In one embodiment of the present invention, in step S3-2, N is set to 500, and the formula w i =x2 i -x1 i Calculate the width w of the target bounding box after the pan / tilt camera completes the i-th zoom i , where x2 i is the horizontal coordinate of the lower right corner of the target bounding box, x1 i is the horizontal coordinate of the upper left corner of the target bounding box, through the formula h i =y2 i -y1 i Calculate the height h of the target bounding box after the pan / tilt camera completes the i-th zoom i , where, y2 i is the ordinate of the lower right corner of the target bounding box, y1 i The ordinate of the upper left corner of the target bounding box.

[0057] In one embodiment of the present invention, as shown in the attached Figure 2 As shown, step S4 includes the following steps:

[0058] S4-1. By formula Calculate the harmonic weight pf, where l 1.0 is the target long side pixel value when the initial focal length zoom factor of the gimbal camera is Zoom = 1.0, l jis the target long side pixel value of the gimbal camera after the jth zoom is completed, j∈{1,...,m} and j≠1, and L is the long side pixel value of the image captured by the gimbal camera.

[0059] S4-2. Through the formula p i ′=pf×p i Calculate the adjustment confidence p of the gimbal camera after the i-th zoom is completed i ′.

[0060] S4-3. Select all m maximum values ​​of the adjusted confidence, and adjust the target long side pixel value l of the gimbal camera corresponding to the maximum value of the adjusted confidence after the oth zoom is completed o , o∈{1,...,m}, if The focal length scaling factor at the oth zooming is used as the best zoom focal length best_zoom when the target is static.

[0061] In one embodiment of the present invention, as shown in the attached Figure 3 and attached Figure 4 As shown, step S5 includes the following steps:

[0062] S5-1. The straight-line distance dis between the gimbal camera and the target is calculated by the formula dis=alt / cosθ, where alt is the height from the gimbal camera to the target, θ=90°+pitch, and pitch is the pitch angle of the gimbal camera.

[0063] S5-2. By formula r = (l o ×R) / L to calculate the real length r of the target, where R is the real length of the long side of the image, R = tanα×dis, α is the field of view of the gimbal camera, l o / r=960 / R. According to the benchmark of 5 meters corresponding to 100pix, the actual length error of the long side of the image is verified by checking whether the ratio of the target's real long side r to the target pixel long side l is equal to 0.05.

[0064] In one embodiment of the present invention, as shown in the attached Figure 5 As shown, step S6 includes the following steps:

[0065] S6-1. The angle β between the straight-line distance from the gimbal camera to the target center and the side length of the target is calculated using the formula β=arctan(r / dis).

[0066] S6-2. By formula l i ′=l o -lb i Calculate the difference l of the target dynamics after the i-th zoom is completed i ′, where lb i =(s i×L) / S i , si is the size of the target in the gimbal camera image sensor after the i-th zoom is completed, s i =f i ×tanβ,f i The focal length scaling factor of the pan-tilt camera for the i-th zoom is Z i The corresponding focal length value, S i S is the size of the gimbal camera image sensor after the i-th zoom is completed, i =f i ×tanα, take the focal length value corresponding to the minimum value of all m target dynamic differences as the optimal focal length.

[0067] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for finding the best focal length of a wide-area scene based on stepless zoom, characterized in that: include: S1. The gimbal camera detects the target and obtains a target bounding box for the selected target; S2. Determine whether the center of the image captured by the gimbal camera is within the target bounding box. If yes, execute step S3. If not, adjust the pitch angle and yaw angle of the gimbal camera until the center of the image captured by the gimbal camera is within the target bounding box. S3. The gimbal camera zooms step by step according to the focal length value in the focal length zoom coefficient Zoom list and records the detection frame information of the target; S4. Calculate the best zoom focal length best_zoom when the target is static according to the detection frame information of the target; S5. Calculate the real length r of the target; S6. Calculate the optimal dynamic focal length l′ of the target based on the actual length r of the target.

2. The method for finding the best focal length for a wide-area scene based on stepless zoom according to claim 1, characterized in that: In step S1, if the gimbal camera is installed on the drone, the drone enters a hovering mode.

3. The method for finding the best focal length for a wide-area scene based on stepless zoom according to claim 1, characterized in that: In step S1, the YOLOv8 and ReID models are used to perform target detection and feature matching on the images captured by the gimbal camera to obtain a target bounding box for the selected target and complete the target locking.

4. The method for finding the best focal length for a wide-area scene based on stepless zoom according to claim 1, characterized in that: In step S1, the initial focal length zoom factor of the gimbal camera is Zoom=1.

0.

5. The method for optimizing the best focal length of a wide-area scene based on stepless zoom according to claim 1, characterized in that: Step S2 includes the following steps: S2-1. The center coordinates of the image captured by the PTZ camera are calculated by the formula Center_image = (W / 2, H / 2), where W is the width of the image captured by the PTZ camera, and H is the height of the image captured by the PTZ camera; S2-2. The center point coordinates of the target bounding box are calculated by the formula Center_box=(w / 2,h / 2), where w=x2-x1, h=y2-y1, x1 is the horizontal coordinate of the upper left corner of the target bounding box, y1 is the vertical coordinate of the upper left corner of the target bounding box, x2 is the horizontal coordinate of the lower right corner of the target bounding box, and y2 is the vertical coordinate of the lower right corner of the target bounding box; S2-3. If the center point coordinates Center_image of the image captured by the gimbal camera coincide with the center point coordinates Center_box of the target bounding box, it is determined that the center of the image captured by the gimbal camera is within the target bounding box.

6. The method for finding the best focal length for a wide-area scene based on stepless zoom according to claim 1, characterized in that: Step S3 includes the following steps: S3-1. Traverse the index values ​​of the focal length zoom coefficient Zoom list provided by the gimbal camera manufacturer. The gimbal camera zooms step by step according to the corresponding focal length value in the focal length zoom coefficient Zoom list. The focal length zoom coefficient of the i-th zoom is Z i , i∈{1,...,m}, m is the total number of focal lengths of the gimbal camera; S3-2. Create a list, and after the pan / tilt camera completes the i-th zoom, the width w of the target bounding box is updated every Nms. i , the height of the target bounding box h i , after the i-th zoom is completed, the model mixed confidence p is used to detect the picture taken by the gimbal camera using the YOLOv8 and ReID models i , the focal length scaling factor Z of the i-th zoom i Store in a list; S3-3. Select the target bounding box width w after the i-th zoom is completed in the list i , height h i Compare and select the maximum value as the target long side pixel value l i .

7. The method for finding the best focal length for a wide-area scene based on stepless zoom according to claim 6, characterized in that: In step S3-2, N is set to 500, and the formula w i =x2 i -x1 i Calculate the width w of the target bounding box after the pan / tilt camera completes the i-th zoom i , where x2 i is the horizontal coordinate of the lower right corner of the target bounding box, x1 i is the horizontal coordinate of the upper left corner of the target bounding box, through the formula h i =y2 i -y1 i Calculate the height h of the target bounding box after the pan / tilt camera completes the i-th zoom i , where, y2 i is the ordinate of the lower right corner of the target bounding box, y1 i The ordinate of the upper left corner of the target bounding box.

8. The method for optimizing the best focal length of a wide-area scene based on stepless zoom according to claim 6, characterized in that: Step S4 includes the following steps: S4-1. By formula Calculate the harmonic weight pf, where l 1.0 is the target long side pixel value when the initial focal length zoom factor of the gimbal camera is Zoom = 1.0, l j is the target long side pixel value of the gimbal camera after the jth zoom is completed, j∈{1,...,m} and j≠1, L is the image long side pixel value of the picture taken by the gimbal camera; S4-2. Through the formula p i ′=pf×p i Calculate the adjustment confidence p of the gimbal camera after the i-th zoom is completed i ′; S4-3. Select all m maximum values ​​of the adjusted confidence, and adjust the target long side pixel value l of the gimbal camera corresponding to the maximum value of the adjusted confidence after the oth zoom is completed o , o∈{1,...,m}, if The focal length scaling factor at the oth zooming is used as the best zoom focal length best_zoom when the target is static.

9. The method for finding the best focal length for a wide-area scene based on stepless zoom according to claim 8, characterized in that: Step S5 includes the following steps: S5-1. The straight-line distance dis between the gimbal camera and the target is calculated by the formula dis=alt / cosθ, where alt is the height from the gimbal camera to the target, θ=90°+pitch, and pitch is the pitch angle of the gimbal camera; S5-2. By formula r = (l o ×R) / L to calculate the real length r of the target, where R is the real length of the long side of the image, R = tanα×dis, α is the field of view of the gimbal camera, l o / r=960 / R.

10. The method for finding the best focal length for a wide-area scene based on stepless zoom according to claim 9, characterized in that: Step S6 includes the following steps: S6-1. The angle β between the straight-line distance from the gimbal camera to the center of the target and the side length of the target is calculated by the formula β = arctan (r / dis); S6-2. By formula l i ′=l o -lb i Calculate the target dynamic difference l after the i-th zoom is completed i ′, where lb i =(s i ×L) / S i , si is the size of the target in the gimbal camera image sensor after the i-th zoom is completed, s i =f i ×tanβ,f i The focal length scaling factor of the pan-tilt camera for the i-th zoom is Z i The corresponding focal length value, S i S is the size of the gimbal camera image sensor after the i-th zoom is completed, i =f i ×tanα, take the focal length value corresponding to the minimum value of all m target dynamic differences as the optimal focal length.

Images