A Target Distance Prediction Method and System Based on Gimbal Optical Characteristics

Abstract

This application provides a target distance estimation method and system based on the optical characteristics of a gimbal. The method includes automatic target detection and tracking of the target under test from the current viewpoint to acquire a target video; simultaneously acquiring the gimbal attitude and camera focal length in real time; performing target detection based on the target video to obtain key feature points of the target; correcting distortion of the key feature points through image coordinate mapping to form a unified image coordinate system; combining the image coordinate system with the gimbal attitude information to map it onto a three-dimensional virtual space to construct a virtual coordinate system; and calculating the target distance of the target under test based on the geometric position of the target under test in the virtual coordinate system. This application realizes a gimbal system with dynamic attitude adjustment capabilities, enabling accurate estimation of target distance in complex scenes and under different viewpoints, and exhibiting real-time performance and robustness.

Description

Technical Field

[0001] This application belongs to the field of computer vision, specifically relating to a target distance prediction method and system based on the optical characteristics of a gimbal. Background Technology

[0002] In the fields of computer vision, optical measurement, and electromechanical control, accurate target distance measurement and positioning are key to the efficient operation of the system.

[0003] Traditional monocular camera ranging methods typically rely on fixed camera intrinsic parameters and geometric models. However, these methods have limited accuracy in scenarios with frequent changes in camera pose, especially in complex environments where pose errors and limited field of view can lead to inaccurate ranging. Alternatively, traditional ranging methods often rely on multi-sensor technologies such as binocular vision or LiDAR. However, the high cost and system complexity of these methods limit their application in resource-constrained devices.

[0004] Therefore, a new distance measurement technology is urgently needed to meet the accuracy requirements of complex scenarios when measuring target distance and locating positions.

[0005] The foregoing statements are for informational purposes only and are not intended to provide background information in connection with this application. Unless otherwise stated herein, the content described in this section is not prior art to the rest of this application. Summary of the Invention

[0006] To address the aforementioned technical limitations, this application provides a target distance prediction method and system based on the optical characteristics of a gimbal. By fusing the dynamic optical characteristics of the gimbal with attitude information, the ranging accuracy in dynamic scenes is improved, adapting to the need for frequent camera angle adjustments.

[0007] This application utilizes a gimbal system with dynamic attitude adjustment capabilities, enabling accurate target distance estimation in complex scenes and from different perspectives, while exhibiting real-time performance and robustness. Furthermore, a virtual coordinate system mapping algorithm expands the ranging range and reduces system complexity. Image coordinate mapping is also employed for distortion correction, eliminating image distortion interference and improving the accuracy of geometric model derivation.

[0008] According to a first aspect of the embodiments of this application, a target distance prediction method based on the optical characteristics of a gimbal is provided, comprising:

[0009] Automatic target detection and tracking are performed on the target under test from the current perspective to obtain target video of the target under test; at the same time, the gimbal attitude and field of view information are collected in real time; the gimbal attitude includes rotation angle, pitch angle and position translation data, and the field of view information includes the field of view angle and focal length of the camera;

[0010] Target detection is performed on the target video to obtain the key feature points of the target;

[0011] Distortion correction is performed on key feature points through image coordinate mapping to form a unified image coordinate system;

[0012] By combining the image coordinate system with the gimbal's attitude information, a virtual coordinate system is constructed by mapping it to a three-dimensional virtual space.

[0013] Calculate the target distance based on the geometric position of the target in the virtual coordinate system.

[0014] In some embodiments of this application, the method further includes:

[0015] After adjusting the gimbal attitude and field of view information multiple times, the distances to the target under test were calculated multiple times from different perspectives.

[0016] The target distance is corrected by fusing multiple real-time distances from different perspectives using a filtering algorithm.

[0017] In some embodiments of this application, the target distance of the target to be measured is calculated based on the geometric position of the target to be measured in the virtual coordinate system, including: calculating the horizontal deflection angle and vertical pitch angle of any point of the target to be measured in the virtual coordinate system and compensating for them;

[0018] Calculate the vertical and horizontal projected distances based on the camera height and pitch angle; calculate the spatial depth of the point to be measured along the vertical direction based on the camera height and pitch angle; calculate the Euclidean distance from the point to be measured to the camera based on the vertical and horizontal projected distances.

[0019] Based on the angle between the current frame and the centerline, and the current deflection angle of the gimbal, calculate the total azimuth angle of the test point relative to the centerline of the field; change the test point of the test target, repeat the above steps, and output the vertical projection distance, horizontal projection distance, total azimuth angle, and Euclidean distance of all test points of the test target.

[0020] In some embodiments of this application, the method further includes:

[0021] Based on the virtual coordinate system, multiple targets to be measured are mapped onto the same virtual top-view plane through projection algorithms and geometric mapping, enabling observation and analysis of multi-target motion formations.

[0022] In some embodiments of this application, key feature points of the target are obtained, including feature point estimation when key feature points are missing; key feature points include geometric features of the object; geometric features include the edges and foot points of the target to be measured.

[0023] In some embodiments of this application, feature point estimation includes foot point estimation; foot point estimation includes the following steps:

[0024] Extract bounding box information from images using object detection models;

[0025] If the bottom edge of the target is visible, extract the coordinates of the foot points based on the bottom edge of the target bounding box;

[0026] If the bottom edge of the target is not visible, estimate the coordinates of the foot point based on the average aspect ratio of the target box.

[0027] In some embodiments of this application, key feature points are distorted through image coordinate mapping to form a unified image coordinate system, including:

[0028] Based on the target frame image from the current perspective, obtain the image coordinates of any test point of the target to be tested;

[0029] Based on the image coordinates of any point to be measured, the uniform image coordinates after distortion correction are calculated using a distortion mapping table or a distortion model.

[0030] This continues until distortion correction of all test points of the target objects is completed, forming a unified image coordinate system.

[0031] In some embodiments of this application, an image coordinate system is combined with the gimbal's attitude information and mapped onto a three-dimensional virtual space to construct a virtual coordinate system, including:

[0032] Obtain the image coordinate system of the target frame image from the current viewpoint and the corresponding gimbal posture;

[0033] Add a z-direction component to the coordinates of any point to be measured in the target frame image, so that the vector falls on a sphere with a radius equal to the focal length, and obtain the three-dimensional coordinates of the point to be measured.

[0034] Calculate the spatial rotation coordinates of the 3D vector based on the three-dimensional coordinates of the point to be measured and the gimbal attitude;

[0035] Based on the spatial rotation coordinates, all the points to be measured in the target frame image from the current viewpoint are virtually mapped in three dimensions to obtain a virtual coordinate system.

[0036] According to a second aspect of the embodiments of this application, a target distance prediction system based on the optical characteristics of a gimbal is provided, comprising:

[0037] Target acquisition module: used for automatic target detection and tracking of the target under test, to acquire target video of the target under test; at the same time, it acquires gimbal attitude and field of view information in real time; gimbal attitude includes rotation angle, pitch angle and position translation data, and field of view information includes the camera's field of view angle and focal length;

[0038] Target detection module: Used to detect targets in the target video and obtain the key feature points of the targets;

[0039] Coordinate mapping module: used to correct distortion of key feature points through image coordinate mapping, forming a unified image coordinate system;

[0040] Virtual coordinate module: used to combine the image coordinate system with the gimbal's attitude information and map it to a three-dimensional virtual space to construct a virtual coordinate system;

[0041] Distance estimation module: used to calculate the target distance of the target based on the geometric position of the target in the virtual coordinate system.

[0042] According to a third aspect of the embodiments of this application, a target distance prediction device based on the optical characteristics of a gimbal is provided, comprising: a storage unit for storing executable instructions; and a processing unit for connecting to the storage unit to execute the executable instructions to complete the target distance prediction method based on the optical characteristics of a gimbal.

[0043] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored; the computer program is executed by a processor to implement a target distance prediction method based on the optical characteristics of a gimbal.

[0044] The target distance estimation method and system based on the optical characteristics of a gimbal, as described in this application, includes automatic target detection and tracking of the target under test from the current viewpoint to acquire target video; simultaneously, real-time acquisition of gimbal attitude and field of view information; target detection based on the target video to obtain key feature points of the target; distortion correction of the key feature points through image coordinate mapping to form a unified image coordinate system; mapping the image coordinate system with the gimbal attitude information to a three-dimensional virtual space to construct a virtual coordinate system; and calculating the target distance of the target under test based on the geometric position of the target under test in the virtual coordinate system. This achieves a gimbal system with dynamic attitude adjustment capabilities, enabling accurate target distance estimation in complex scenes and from different viewpoints, and exhibiting real-time performance and robustness. Attached Figure Description

[0045] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0046] Figure 1 The diagram illustrates the steps of a target distance estimation method based on gimbal optical characteristics according to an embodiment of this application.

[0047] Figure 2The diagram illustrates the steps of another target distance estimation method based on the optical characteristics of a gimbal according to an embodiment of this application;

[0048] Figure 3 The diagram shows a step-by-step illustration of another target distance estimation method based on the optical characteristics of a gimbal according to an embodiment of this application;

[0049] Figure 4 The diagram illustrates the steps of footpoint estimation according to an embodiment of this application;

[0050] Figure 5 The diagram illustrates the steps of image coordinate mapping according to an embodiment of this application;

[0051] Figure 6 The diagram illustrates the steps for constructing virtual coordinates according to an embodiment of this application;

[0052] Figure 7 The diagram shows a flowchart of a target distance estimation method based on the optical characteristics of a gimbal according to an embodiment of this application;

[0053] Figure 8 The flowchart of the ranging and angle measurement algorithm of the target distance estimation method according to an embodiment of this application is shown;

[0054] Figure 9 The diagram shows a schematic representation of a target distance prediction system based on the optical characteristics of a gimbal according to an embodiment of this application.

[0055] Figure 10 The diagram shows a schematic representation of a target distance estimation device based on the optical characteristics of a gimbal according to an embodiment of this application. Detailed Implementation

[0056] Regarding this application, in the field of computer vision technology, traditional monocular camera ranging methods typically rely on fixed camera intrinsic parameters and geometric models. However, these methods have limited accuracy in scenarios where camera pose changes frequently, especially in complex environments where pose errors and limited viewing angles can lead to inaccurate ranging. Alternatively, traditional ranging methods often rely on multi-sensor technologies such as binocular vision or LiDAR. However, these methods are limited in their application in some resource-constrained devices due to their high cost and system complexity.

[0057] The gimbal has a flexible angle adjustment function and can dynamically change the optical characteristics of the camera, such as field of view and focal length, so that the camera can capture images at different angles and positions, which can provide additional geometric information for target distance estimation.

[0058] However, how to effectively utilize the optical characteristics and motion capabilities of the gimbal to accurately estimate the target distance remains a significant technical challenge.

[0059] This invention proposes a target distance estimation method based on the optical characteristics of a gimbal. By combining camera intrinsic parameters, attitude information, and dynamically adjusted optical parameters such as field of view and focal length, the target distance can be accurately estimated.

[0060] This invention provides a target distance estimation method based on the optical characteristics of a gimbal, which realizes accurate estimation of the distance and position of a target object by using an image coordinate mapping device combined with a virtual coordinate system mapping algorithm.

[0061] This application uses the camera's attitude information dynamically acquired by the gimbal to map the two-dimensional image coordinates to a three-dimensional virtual coordinate system. Combined with the camera's optical characteristics, it accurately predicts the target distance and adapts to complex dynamic scenes.

[0062] The system architecture of this application includes the following core elements: a gimbal-based image coordinate mapping module, a virtual coordinate system composition module, and a target distance estimation module; others also include a camera optical characteristic adjustment module.

[0063] This application can dynamically adjust the field of view and focal length according to the camera's posture, use a virtual coordinate system to accurately estimate the target distance, and perform multi-view information fusion and calculation, which greatly improves the accuracy and stability of ranging.

[0064] Compared with the prior art, the present invention has the following advantages:

[0065] 1. High-precision distance measurement: By combining the optical parameters dynamically adjusted by the gimbal with the mapping algorithm of the virtual coordinate system, high-precision distance measurement is achieved from different perspectives.

[0066] 2. High real-time performance: By utilizing real-time attitude acquisition and optical characteristic adjustment, the algorithm can maintain efficient ranging capabilities in rapidly changing scenes.

[0067] 3. Wide adaptability: The algorithm is not only applicable to static scenes, but also adaptable to dynamic ranging tasks in moving objects or complex environments.

[0068] 4. Error correction: Real-time error correction is performed through multi-frame data fusion and filtering, which significantly reduces ranging errors caused by attitude changes.

[0069] 5. Strong system integration: The image coordinate mapping device and virtual coordinate system algorithm are seamlessly integrated into the system architecture, ensuring the efficient operation of the algorithm and the stability of the system.

[0070] To make the technical solutions and advantages of the embodiments of this application clearer, the exemplary embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0071] Furthermore, the coordinate system establishment method and direction used in the various embodiments of this application are merely examples. Those skilled in the art should know that such coordinate system settings are not absolutely fixed. When the specific setting method of the coordinate system changes or transforms, including but not limited to the axis direction, the relative position of the axis, or the position of the coordinate origin, the corresponding technical solutions that can be obtained by simple adjustments to this solution without creative effort are all within the protection scope of this application.

[0072] Example 1

[0073] Figure 1 The diagram illustrates the steps of a target distance prediction method based on gimbal optical characteristics according to an embodiment of this application.

[0074] like Figure 1 As shown in the figure, a target distance estimation method based on the optical characteristics of a gimbal, according to an embodiment of this application, includes:

[0075] S1: Automatically detect and track the target under test from the current perspective to obtain target video of the target under test; at the same time, collect the attitude information of the gimbal and the focal length of the camera in real time.

[0076] The gimbal's attitude information includes rotation angle, pitch angle, and position translation data.

[0077] S2: Perform target detection based on the target video to obtain the key feature points of the target;

[0078] S3: Correct the distortion of key feature points through image coordinate mapping to form a unified image coordinate system;

[0079] S4: Combine the image coordinate system with the gimbal's attitude information and map it to a three-dimensional virtual space to construct a virtual coordinate system;

[0080] S5: Calculate the target distance of the target based on the geometric position of the target in the virtual coordinate system and the gimbal geometric model.

[0081] This application integrates the dynamic optical characteristics and attitude information of the gimbal to improve ranging accuracy in dynamic scenes and adapt to the need for frequent camera angle adjustments. Based on a gimbal system with dynamic attitude adjustment capabilities, this application can accurately estimate target distances in complex scenes and from different perspectives, exhibiting real-time performance and robustness. Furthermore, a virtual coordinate system mapping algorithm expands the ranging range and reduces system complexity. Image coordinate mapping is also used for distortion correction to eliminate image distortion interference and improve the accuracy of geometric model derivation.

[0082] Figure 2 The diagram illustrates the steps of another target distance estimation method based on the optical characteristics of a gimbal, according to an embodiment of this application.

[0083] like Figure 2 As shown, in other embodiments, the target distance estimation method of this application further includes:

[0084] S6: After adjusting the gimbal attitude and field of view information multiple times, calculate the distances of the target under test at different viewpoints multiple times.

[0085] S7: Use a filtering algorithm to fuse multiple real-time distances from different perspectives to obtain the corrected target distance.

[0086] This application reduces ranging errors caused by attitude changes by fusing multi-frame data and correcting filtering errors, thereby enhancing the real-time performance and robustness of the algorithm.

[0087] Figure 3 The diagram illustrates the steps of another target distance estimation method based on the optical characteristics of a gimbal, according to an embodiment of this application.

[0088] like Figure 3 As shown, in other embodiments, the target distance estimation method of this application further includes:

[0089] S8: Based on the virtual coordinate system, multiple targets to be measured are mapped onto the same virtual top-view plane through projection algorithms and geometric mapping, realizing the observation and analysis of multi-target motion formations.

[0090] This application achieves intuitive analysis of multi-target planes by combining a virtual coordinate system with multi-target detection, thereby increasing the application scenarios and functionality.

[0091] 1) First, obtain the camera and gimbal integration and parameters.

[0092] A camera is mounted on a multi-degree-of-freedom gimbal, which has functions such as rotation, pitch, and translation. By acquiring the optical characteristic parameters of the gimbal, including the camera's field of view, focal length, and relative attitude to the target object, a complete geometric model of the gimbal is constructed. Specifically, the camera's field of view is calculated based on the focal length and the physical lens size; specifically, the camera's field of view is twice the arctangent of the quotient of the physical lens width divided by twice the focal length.

[0093] 2) Then, optical characteristics and attitude information are acquired in real time, and image and video streams of the target object are acquired.

[0094] The system utilizes a gimbal sensor to collect real-time attitude change information, such as the camera's rotation angle, pitch angle, and translation. Simultaneously, it dynamically adjusts the camera's optical parameters, such as focal length and field of view, to adapt to different shooting scenarios and target objects.

[0095] 3) Then, image acquisition and feature extraction.

[0096] Based on the acquired image and video streams, target detection of the target is performed, and key feature points of the target are extracted. Feature points may include the object's edges, corners, or other geometric features, which are used for subsequent geometric derivation and distance calculation.

[0097] 4) Next, the coordinates of target feature points in the real-time acquired images are correlated with the gimbal's attitude information using an image coordinate mapping device to form a unified image coordinate system. This device, combined with attitude information obtained from gimbal sensors (such as angle sensors and accelerometers), ensures coordinate accuracy during the mapping process.

[0098] 5) The image coordinates are mapped to a three-dimensional virtual space using a virtual coordinate system mapping algorithm.

[0099] The virtual coordinate system mapping algorithm constructs a virtual coordinate system corresponding to the physical space based on the gimbal attitude and field of view information, such as focal length, principal point coordinates, attitude, and position.

[0100] 6) Regarding the combined calculation of field of view and focal length, this application calculates the field of view range of the target object under different viewing angles based on the field of view and focal length of the camera after gimbal adjustment.

[0101] By combining this field-of-view information with the geometric positions of object feature points, and using a geometric model combined with virtual coordinate system information, the distance to the target object can be accurately derived.

[0102] 7) Dynamic attitude correction and error correction.

[0103] This application takes into account the dynamic movement of the gimbal and adjusts the attitude information in real time to affect distance prediction. It utilizes multi-frame data, fuses the predicted position data of the target from different viewpoints, and employs filtering algorithms (such as Kalman filtering) to correct attitude and viewpoint errors, further improving measurement accuracy.

[0104] 8) Distance optimization algorithm based on deep learning.

[0105] Preferably, in complex scenarios, such as when the target object has an irregular shape or is occluded, this application uses a deep learning algorithm to analyze the extracted image features, learn the nonlinear relationship between the target distance and the camera pose under different optical characteristics, and further optimize the distance prediction results.

[0106] 9) Output and application of estimated distance data.

[0107] The final estimated target distance data in this application is output through an interface and can be used in systems such as navigation, obstacle avoidance, and augmented reality to ensure the real-time performance and accuracy of the target distance.

[0108] This application can be applied to autonomous driving, drone navigation, or industrial robot scenarios. For example, in autonomous driving, target distance data is output through an interface for obstacle avoidance or path planning.

[0109] In practice, the key feature points of the target are obtained through step S2, including feature point estimation when key feature points are missing; key feature points include the geometric features of the object; geometric features include the edges and foot points of the target to be measured.

[0110] Figure 4 The diagram illustrates the steps of footpoint estimation according to an embodiment of this application.

[0111] Feature point estimation includes foot point estimation; such as Figure 4 As shown, foot point estimation includes the following steps:

[0112] S21: Extract bounding box information from the image using an object detection model;

[0113] S22: If the bottom edge of the target is visible, extract the coordinates of the foot points based on the bottom edge of the target bounding box;

[0114] If the bottom edge of the target is not visible, estimate the coordinates of the foot point based on the aspect ratio of the detection frame.

[0115] Figure 5 The diagram illustrates the steps of image coordinate mapping according to an embodiment of this application.

[0116] like Figure 5As shown, in some embodiments of this application, key feature points are distorted through image coordinate mapping to form a unified image coordinate system, including:

[0117] S31: Based on the target frame image from the current viewpoint, obtain the image coordinates of any test point of the target to be tested;

[0118] S32: Based on the image coordinates of any point to be measured, calculate the uniform image coordinates after distortion correction using a distortion mapping table or distortion model;

[0119] S33: Until the distortion correction of all test points of the target is completed and a unified image coordinate system is formed.

[0120] Figure 6 The diagram illustrates the steps for constructing virtual coordinates according to an embodiment of this application.

[0121] Next, as Figure 6 As shown, the virtual coordinate system is constructed by combining the image coordinate system with the gimbal's attitude information and mapping it to a three-dimensional virtual space, including the following steps:

[0122] S41: Obtain the image coordinate system of the target frame image from the current viewpoint and the corresponding gimbal attitude;

[0123] S42: Add the z-direction component to the coordinates of any point to be measured in the target frame image to obtain a 3D vector, and make the vector fall on a sphere with a radius equal to the focal length to obtain the three-dimensional coordinates of the point to be measured.

[0124] S43: Calculate the spatial rotation coordinates of the 3D vector based on the 3D coordinates of the point to be measured and the gimbal attitude. That is, combine the yaw and pitch angles of the gimbal, and use a rotation matrix to rotate the 3D vector in the reverse direction to the initial attitude to calculate the spatial rotation coordinates of the 3D vector.

[0125] S44: Based on the spatial rotation coordinates, perform three-dimensional virtual mapping on all the points to be measured in the target frame image of the current viewpoint to obtain a virtual coordinate system.

[0126] Figure 7 The diagram shows a flowchart of a target distance estimation method based on the optical characteristics of a gimbal according to an embodiment of this application.

[0127] like Figure 7 As shown, after starting, the YOLO target detection box is first performed to determine whether the target box is in the foreground (whether the midpoint of the foot is on the screen) and the corresponding foot point to be tested is calculated.

[0128] Distortion correction is performed on the point to be measured to obtain the distortion-corrected point to be measured. The distance and angle estimation of the corrected point to be measured are then performed from a "God's-eye view" based on the measured distance and angle information.

[0129] Then, perform a 2D top-down projection to obtain the projected image. Repeating this process and drawing the image yields the projected video.

[0130] This invention provides an algorithm for estimating the distance to a target based on the optical characteristics of a gimbal. The algorithm collects motion information of the gimbal, intrinsic parameters of the camera, and dynamically adjusted optical characteristics, and combines a field-of-view geometry model and deep learning methods to estimate the distance to the target object in real time.

[0131] In summary, the target distance estimation method based on gimbal optical characteristics proposed in this application includes: automatically detecting and tracking the target from the current viewpoint to acquire a target video; simultaneously acquiring gimbal attitude and field-of-view information in real time; performing target detection based on the target video to obtain key feature points of the target; correcting distortion of the key feature points through image coordinate mapping to form a unified image coordinate system; combining the image coordinate system with the gimbal attitude information to map it onto a three-dimensional virtual space to construct a virtual coordinate system; and calculating the target distance of the target based on the geometric position of the target in the virtual coordinate system and the gimbal geometric model. This achieves accurate target distance estimation in complex scenes and under different viewpoints based on a gimbal system with dynamic attitude adjustment capabilities, demonstrating real-time performance and robustness.

[0132] Furthermore, the gimbal device of this application integrates a camera, and the gimbal has functions such as rotation and pitch. It can also acquire the optical characteristics of the gimbal and camera in real time, including the gimbal's six degrees of freedom attitude information, the camera's field of view, focal length, and feature point information of the target in the image.

[0133] This application achieves accurate distance measurement by combining the acquired target video with image coordinates and a virtual coordinate mapping system.

[0134] The following is a detailed explanation of the target distance prediction scheme of this application, using a motion pan-tilt unit with an integrated camera to measure the distance of football players at the edge of a football field.

[0135] (a) Set up the pan-tilt unit and camera at the edge of the center line of the court, with the camera facing the center line of the court, to ensure that the pan-tilt unit and camera can observe all the players on the court when they rotate.

[0136] Using a pan-tilt camera, automatic target detection and tracking are performed on the field.

[0137] (ii) Detect the target in real-time images captured by the camera. The target includes athletes and footballs.

[0138] It also extracts geometric feature points according to requirements. Feature points can include the edges, corners or other geometric features of an object.

[0139] In scenarios where feature points cannot be fully acquired at different field of view angles (field of view scaling), feature point estimation is required for the missing feature points, such as the estimation of the foot points of a nearby target.

[0140] The specific calculation for estimating the foot point to be measured is as follows:

[0141] First, the object detection model is used to extract the bounding box information (x, y, w, h) from the image, where x, y, w, and h are the coordinates of the center point of the bounding box, the width of the bounding box, and the height of the bounding box, respectively.

[0142] If there are foot points in the target bounding box, that is, when the bottom of the target touches the ground (such as a far-field target), and the bottom edge of the target bounding box is not close to the bottom edge of the image, then the coordinates of the foot points are directly extracted as follows:

[0143] fp = (x, y + h / 2 - σ)

[0144] Where σ is the detection box error value, which is the average value of multiple frames of static target, and is generally 5% of the target box height h.

[0145] If there are no foot points in the target frame, i.e., the target is so close to the camera that the foot points are not visible (such as a close-up target), the foot points are estimated based on the average aspect ratio (ratioWH) of the target frame as follows:

[0146]

[0147] The average aspect ratio of the target bounding box is obtained from the dataset statistics. It represents the aspect ratio of the detected target bounding box of a complete human target, typically fluctuating slightly around 0.39. This allows for the inference of the vertical offset of the foot points by using the aspect ratio, thus adapting to feature loss caused by field-of-view scaling.

[0148] (iii) Using an image coordinate mapping device, the extracted key feature points are distorted by image coordinate mapping to form a unified image coordinate system.

[0149] Because the image from the lens exhibits slight distortion under different motion postures and field of view angles, further distortion correction is required for the point to be measured. This involves performing high-precision image coordinate mapping by combining the coordinates of the point to be measured in the image with the camera's optical characteristics, such as focal length and field of view angle, to improve ranging accuracy.

[0150] First, read the 2D image from the camera's current viewpoint and obtain the coordinates (x_real, y_real) of any point A_real to be measured on the image;

[0151] Then, distortion correction is performed. Distortion correction is applied to any point A_real(x_real,y_real) on the 2D image to obtain the distorted coordinates A(x,y).

[0152] The principle of distortion correction is to calculate the corrected coordinates based on the distortion model using pre-calibrated distortion parameters, such as radial distortion coefficients k1 and k2, and tangential distortion coefficients p1 and p2.

[0153] One example of a distortion model is as follows:

[0154]

[0155] Where r is the distance from the pixel to the image center, C x and C y The coordinates of the center point are given.

[0156] In practice, to avoid iterative calculations, a mapping table is directly constructed, and the lookup table method is used.

[0157] (iv) Apply virtual coordinate mapping algorithm, combine the sensor information of the gimbal, such as pitch angle, rotation angle, height, speed information, etc., to map the image coordinate system to the three-dimensional virtual space to construct a virtual coordinate system.

[0158] The image coordinates of the test point in the image are mapped to three-dimensional virtual space coordinates in different gimbal postures and different field of view angles. The specific calculation process is as follows:

[0159] First, read the 2D image of the camera's current viewpoint to obtain the current gimbal attitude angle information, including yaw and pitch angles. For any point on the image to be measured, after distortion correction, take the coordinates A(x,y). Since the virtual coordinate mapping algorithm's rotation and 3D projection operations require the optical axis (image center) as a reference, the coordinates are first translated from a coordinate system with the top left corner of the image as the origin to a coordinate system with the image center as the origin. If the center point coordinates (C... x C y If the coordinates of point A are (x, C), then the coordinates of point A after the coordinate system is translated are (x, C). x yC y ).

[0160] Then, calculate the 3D transformation of any point A on the current image to obtain the 3D coordinates A of point A in the 3D virtual coordinate system. 3d .

[0161] Specifically: Increase the z-direction component so that the vector falls on a sphere with radius R fixed at the focal length f (in pixels), then A 3d The coordinates are

[0162] Next, the spatial rotation of the 3D vector is calculated to obtain the spherical virtual coordinates B. 3d .

[0163] Specifically, by combining the yaw and pitch angles of the gimbal, the spatial rotation coordinates of the 3D vector are calculated by rotating the 3D vector in reverse to the initial attitude using a rotation matrix.

[0164] Rotate the current 3D vector back to its initial coordinates B when yaw = 0 and pitch = 0, according to yaw and pitch. 3d (xb,yb,zb).

[0165] The specific calculation process is as follows:

[0166] 1. Establish a right-handed coordinate system: The x-axis of the 2D imaging plane is to the right, the y-axis is downward, and the optical axis is forward, forming a right-handed coordinate system;

[0167] 2. Camera rotation is equivalent to the spatial vector of any point on the 2D screen rotating in the opposite direction (to the initial screen).

[0168] 3. Yaw rotation, which is rotation around the Y-axis; Pitch rotation, which is rotation around the X-axis;

[0169] 4. Assuming a rotation of -θ(yaw) around the Y-axis, the rotation matrix is:

[0170]

[0171] 5. Assuming a rotation of -Φ(pitch) around the X-axis, the rotation matrix is:

[0172]

[0173] 6. The total rotation is:

[0174] R = R y (-θ)·R x (-φ);

[0175] 7. The corresponding 3D coordinates after rotation are:

[0176] B 3d =R·A 3d =R y (-θ)·R x (-φ)·[x,y,z).

[0177] Theoretically, B 3d This refers to the virtual coordinates on the spherical surface.

[0178] Next, based on the spherical virtual coordinates B 3d Calculate the virtual coordinates of a point (e.g., point B). First, based on the virtual coordinates of the sphere B... 3d(xb, yb, zb) Calculate the orientation (total_yaw) and pitch (total_pitch) of point B in the virtual coordinates; then calculate using the arc length formula to obtain the virtual coordinates of the target point B.

[0179] First, extract the azimuth angle:

[0180] The horizontal direction, total_yaw, is determined by x. b With z b The specific value can be determined using the arctan2 function in a computer programming language, i.e., total_yaw = arctan2(x b ,2b);

[0181] `arctan2(y,x)` is a four-quadrant arctangent function. Its output represents the azimuth angle (in radians) of a point (x,y) relative to the origin in a Cartesian coordinate system, covering the range (-π,π). Its core logic is to determine the specific quadrant based on the sign combination of the input coordinates (x,y) and adjust the angle value to avoid the limitations of a standard arctangent function. Its calculation method is as follows:

[0182] When x>0: directly calculate arctan(y / x), the result range is (-π / 2, π / 2).

[0183] When x < 0:

[0184] y≥0: The result is arctan(y / x)+π, adjusted to the second quadrant (π / 2,π).

[0185] y<0: The result is arctan(y / x)-π, adjusted to the third quadrant (-π,-π / 2).

[0186] When x = 0:

[0187] y>0: The result is π / 2 (in the positive Y-axis direction).

[0188] y<0: The result is -π / 2 (in the negative Y-axis direction).

[0189] y = 0: usually defined as 0 (origin).

[0190] Vertical direction total_pitch:

[0191] Next, arc length mapping is performed.

[0192] Assuming the radius of the virtual sphere is M, the formulas for mapping virtual coordinates in the x and y directions are as follows:

[0193] X vir =total_yaw·M,Y vir=total_pitch·M.

[0194] (V) Target distance estimation: Combining the geometric position of the target point with the three-dimensional virtual coordinate information of the target obtained in (IV) and the gimbal attitude and field of view information, the target distance is accurately derived through the geometric model, that is, the three-dimensional geometric relationship under spherical coordinates.

[0195] Figure 8 The diagram shows a flowchart of the ranging and angle measurement algorithm for the target distance estimation method according to an embodiment of this application.

[0196] like Figure 8 As shown, the specific calculation process is as follows:

[0197] 1) Read the 2D image of the camera's current viewpoint to obtain the current gimbal attitude angle information, such as yaw and pitch. If the coordinates of any point to be measured on the image are B after virtual coordinate mapping... 3d (x b ,y b ,z b Other input parameters include: camera mounting height camH, and lens focal length f.

[0198] 2) Define a coordinate system and establish a virtual coordinate system for the sphere.

[0199] Ground coordinate system: with camera projection point O as the origin, X w The axis points directly forward (aligned with the initial optical axis), Y w The axis is horizontal to the right, Z w The axis is vertically downward.

[0200] Spherical coordinate system: B based on virtual coordinates 3d (x b ,y b ,z b Its direction is aligned with the ground coordinate system.

[0201] 3) Calculate and compensate for the horizontal deflection angle and vertical pitch angle of any point of the target under test in the virtual coordinate system. That is, calculate the azimuth angle and geometric parameters.

[0202] First, extract the virtual azimuth angle.

[0203] Obtain the total azimuth of the point to be measured from the virtual coordinate system:

[0204] β=total_yaw,γ=total_pitch;

[0205] Where: β is the horizontal yaw angle, and γ is the vertical pitch angle.

[0206] Then, the gimbal attitude angle is superimposed for compensation.

[0207] If the gimbal currently has yaw and pitch offsets, the virtual angle needs to be compensated:

[0208] γ′=γ+α(α=pitch);

[0209] Here, it is assumed that γ already includes the effect of the gimbal pitch angle, and needs to be adjusted according to the actual calibration.

[0210] 4) Calculate the ground projection distance. That is, calculate the vertical projection distance and the horizontal projection distance based on the camera height and pitch angle.

[0211] First, calculate the vertical projection distance Y. w .

[0212] Based on the camera height camH and pitch angle γ', calculate the distance of the point to be measured on the ground along the optical axis:

[0213] Y w = camH / tan(γ').

[0214] Then, calculate the horizontal projection distance X. w That is, using the horizontal deflection angle β and the longitudinal distance Y w Calculate the lateral offset X w :

[0215] X w =Y w *tan(β);

[0216] Where β corresponds to the horizontal offset angle in the virtual coordinate system.

[0217] 5) Calculate the three-dimensional spatial depth and total distance. That is, calculate the spatial depth of the point to be measured along the vertical direction based on the camera height and pitch angle; and calculate the Euclidean distance from the point to be measured to the camera based on the vertical and horizontal projected distances.

[0218] First, calculate the depth Z. deep Based on the camera height and pitch angle, calculate the depth of the point to be measured along the optical axis:

[0219] Z deep =camH / sin(γ')

[0220] Then, calculate the total ground projection distance Distance.

[0221] By combining the longitudinal and lateral projection components, the Euclidean distance from the point to be measured to the camera is calculated:

[0222] Distance = sqrt(Yw 2 + Xw 2 ).

[0223] 6) Align the azimuth angle with the centerline. Calculate the total azimuth angle of the point to be measured relative to the centerline of the field based on the angle θ between the current frame and the centerline and the current deflection angle of the gimbal.

[0224] First, calculate the angle θ between the current frame and the center line.

[0225] Based on the lateral offset X w and longitudinal distance Y w Calculate the local deflection angle of the point to be measured relative to the optical axis:

[0226] θ = arctan(Y) w / X w ).

[0227] Then, calculate the total rotation angle (total_yaw), which is the total azimuth angle.

[0228] By superimposing the current yaw angle of the gimbal, the total azimuth angle of the point to be measured relative to the centerline of the field is obtained:

[0229] total_yaw = θ + yaw.

[0230] 7) Output the distance measurement information of the point to be measured, including the vertical projection distance, horizontal projection distance, total azimuth angle and Euclidean distance of the point to be measured.

[0231] Regarding the ground projection coordinates. X w (Horizontal offset, in meters), Y w (Vertical distance, unit: meters), Total distance: Distance (unit: meters).

[0232] Regarding the total yaw: total_yaw (unit: radians or degrees).

[0233] Finally, the output yields the projected distance from the measured point (detection frame foot point) to the camera and the XY components. w Y w Distance and the total yaw, which is the angle between the point to be measured and the rotation angle between the point and the site.

[0234] Then, change the test points of the target and repeat the above steps to output the vertical projection distance, horizontal projection distance, total azimuth angle and Euclidean distance of multiple test points of the target.

[0235] In summary, S5 calculates the target distance of the target based on its geometric position in the virtual coordinate system, including: First, establishing a spherical virtual coordinate system; then, calculating and compensating for the horizontal deflection angle and vertical pitch angle of any point of the target in the virtual coordinate system; next, calculating the vertical and horizontal projected distances based on the camera height and pitch angle; then, calculating the spatial depth of the target point along the vertical direction based on the camera height and pitch angle; and calculating the Euclidean distance from the target point to the camera based on the vertical and horizontal projected distances; finally, calculating the total azimuth angle of the target point relative to the centerline of the field based on the angle θ between the current frame and the centerline and the current deflection angle of the gimbal.

[0236] Change the measurement point of the target to be measured, repeat the above steps, and output the vertical projection distance, horizontal projection distance, total azimuth angle and Euclidean distance of multiple measurement points.

[0237] Therefore, the target distance estimation method of this application combines dynamic optical parameters and virtual coordinate system algorithm to achieve ultra-low error ranging accuracy under complex viewing angles; it improves ranging accuracy in dynamic scenes and adapts to the need for frequent camera viewing angle adjustments.

[0238] (vi) Dynamic error correction: By utilizing the gimbal's motion attitude, the system integrates target ranging information from multiple frames under different perspectives, and combines it with filtering algorithms to perform ranging correction and improve accuracy.

[0239] After adjusting the gimbal attitude and field of view information multiple times, the target distance of the target under test is calculated multiple times under different viewpoints. Then, a filtering algorithm is used to fuse the multiple real-time distances under different viewpoints to obtain the corrected target distance.

[0240] The following explanation will further illustrate the specific implementation process.

[0241] Step 1: System calibration and initialization.

[0242] First, the gimbal attitude parameters are calibrated by measuring parameters such as the gimbal's horizontal angle (yaw), pitch angle (pitch), focal length (f), and mounting height (camH), and then establishing a three-dimensional coordinate system according to the three-dimensional kinematic model.

[0243] A panoramic view of the scene is generated by image stitching (e.g., using the OpenCV-based Stitching method for stitching), and a global reference frame for the target location is established.

[0244] Then, a feature extraction network is pre-trained, and a feature point detection model is pre-trained using algorithms such as SIFT and ORB. The feature point detection model can, for example, extract 500 candidate points and select 50 optimal matching points.

[0245] Among them, an adaptive feature fusion module (such as VMFE multi-scale feature extraction) is introduced to improve the accuracy of cross-view feature matching.

[0246] Step 2: Multi-view information collection and feature matching.

[0247] Regarding dynamic frame acquisition, the pan-tilt unit is controlled to cruise along a preset trajectory to acquire multiple frames of images. It is preferable to use a pre-set cruise mode to acquire multiple frames of images.

[0248] Each frame of the image is converted to grayscale, and feature points are extracted. The FLANN matching algorithm is preferably used for feature point extraction.

[0249] Regarding multi-frame feature fusion, perspective transformation is used to map images from different viewpoints to a unified coordinate system. Cross-attention feature fusion is preferred for this mapping.

[0250] Then, the pixel offset (Δx, Δy) of the target in adjacent frames is calculated, and the spatial displacement is derived by combining the gimbal attitude angle change. Preferably, multi-sensor fusion can be used to determine the gimbal attitude angle change.

[0251] Step 3: Three-dimensional geometric model and distance measurement calculation.

[0252] First, a coordinate system transformation is performed to convert the gimbal attitude angle (yaw, pitch) into a rotation matrix. Then, the three-dimensional coordinates of the target are derived by combining the camera intrinsic parameters, such as in virtual 3D synthesis methods.

[0253] Calculate the initial distance to the target using the formula:

[0254]

[0255] Where pitch′ is the pitch angle after offset compensation.

[0256] Then, multi-frame data fusion optimization is performed, and the ranging results from different perspectives are weighted and averaged. The weights are determined by the feature matching confidence (e.g., using 50 optimal feature points) and the gimbal attitude stability.

[0257] Step 4: Implement distance measurement correction using a filtering algorithm.

[0258] Regarding the Kalman filter design, assuming the target moves at a constant velocity, with the state variables being distance s and velocity v, the state equation is obtained. as follows:

[0259]

[0260] Where F is the state transition matrix, w k This is process noise.

[0261] Observation equation: The distance after fusing multiple frames is used as the observation value, and the covariance matrix is ​​dynamically adjusted according to the matching error.

[0262] Regarding adaptive parameter adjustment, when the target moves rapidly, the process noise covariance is increased.

[0263] When determining whether a target is moving rapidly, a judgment logic is introduced: if the prediction error exceeds a threshold for N consecutive frames, the filter is reset.

[0264] Step 5: Dynamic error compensation and optimization.

[0265] Regarding gimbal mechanical error compensation, a gimbal repetitive positioning error model is established, and a nonlinear error curve is fitted using historical data to correct the attitude angle reading in real time.

[0266] Regarding the optimization of adaptive loss function, a loss function based on feature similarity is designed to dynamically balance the weights of multiple frames.

[0267] Contrastive learning is introduced to enhance cross-perspective feature consistency.

[0268] Step 6: Performance Verification and Result Output.

[0269] First, accuracy evaluation metrics can be used to quantify ranging accuracy using mean absolute error (MAE) and root mean square error (RMSE); or by comparing the fluctuation of data before and after filtering.

[0270] Then, when outputting the optimization results in real time, an embedded system is used to achieve low-latency processing.

[0271] Finally, the target's three-dimensional coordinates (Xw, Yw, Zdeep) and total azimuth (total_yaw) are output.

[0272] This application improves the robustness of spatial positioning by using multi-sensor fusion, combining gimbal attitude angle, image features, and IMU data.

[0273] This application uses adaptive filtering to dynamically adjust the Kalman filter parameters, balancing accuracy and real-time performance.

[0274] This application solves the problem of inconsistency in information across multiple frames by using cross-view feature alignment, perspective transformation, and attention mechanisms.

[0275] In summary, this application reduces ranging errors caused by attitude changes by fusing multi-frame data and correcting filtering errors, thereby enhancing the real-time performance and robustness of the algorithm. The target distance prediction method and system based on gimbal optical characteristics in this application can achieve sub-pixel-level ranging accuracy and are suitable for high-precision scenarios such as security monitoring and UAV navigation.

[0276] (vii) For target points that have completed distance estimation, they can be connected to modules such as projection algorithms for application: for example, to measure the distance and orientation of different football players on a football field, and through geometric mapping, project them onto a virtual overhead plane for planar projection, which can realize the observation and analysis of movement formations, etc.

[0277] Example 2

[0278] This embodiment provides a target distance prediction system based on the optical characteristics of a gimbal. For details not disclosed in this embodiment, please refer to the specific implementation of the target distance prediction method based on the optical characteristics of a gimbal in other embodiments.

[0279] Figure 9 The diagram shows a schematic representation of a target distance prediction system based on the optical characteristics of a gimbal, according to an embodiment of this application.

[0280] like Figure 9 As shown, the target distance prediction system based on the optical characteristics of the gimbal includes a target acquisition module 10, a target detection module 20, a coordinate mapping module 30, a virtual coordinate module 40, and a distance prediction module 50.

[0281] Target acquisition module 10: used for automatic target detection and tracking of the target under test, to acquire target video of the target under test; at the same time, it acquires gimbal attitude and field of view information in real time; gimbal attitude includes rotation angle, pitch angle and position translation data, and field of view information includes the field of view angle and focal length of the camera;

[0282] Target detection module 20: Used to perform target detection based on the target video and obtain the key feature points of the target;

[0283] Coordinate mapping module 30: Used to correct distortion of key feature points through image coordinate mapping, forming a unified image coordinate system;

[0284] Virtual coordinate module 40: used to combine the image coordinate system with the gimbal's attitude information and map it to a three-dimensional virtual space to construct a virtual coordinate system;

[0285] Distance estimation module 50: Used to calculate the target distance of the target based on the geometric position of the target in the virtual coordinate system and the gimbal geometric model.

[0286] The system architecture of this invention includes the following core elements: a gimbal-based image coordinate mapping device, a virtual coordinate system mapping algorithm and a target distance estimation module, and also includes a camera optical characteristic adjustment module.

[0287] This system can not only dynamically adjust the field of view and focal length according to the camera's posture, but also use a virtual coordinate system to achieve the fusion and calculation of multi-view information, thereby improving the accuracy and stability of distance measurement.

[0288] The target distance prediction system based on the optical characteristics of a gimbal, as described in this application, includes a target acquisition module 10 that automatically detects and tracks the target from the current viewpoint to acquire a target video; simultaneously, it acquires gimbal attitude and field of view information in real time; a target detection module 20 performs target detection based on the target video to acquire key feature points of the target; a coordinate mapping module 30 performs distortion correction on the key feature points through image coordinate mapping to form a unified image coordinate system; a virtual coordinate module 40 combines the image coordinate system with the gimbal attitude information to map it onto a three-dimensional virtual space to construct a virtual coordinate system; and a distance prediction module 50 calculates the target distance of the target based on the geometric position of the target in the virtual coordinate system.

[0289] This application implements a gimbal system with dynamic attitude adjustment function, which can accurately estimate the target distance in complex scenes and from different perspectives, and has the technical effects of real-time performance and robustness.

[0290] Example 3

[0291] This embodiment provides a target distance prediction device based on the optical characteristics of a gimbal. For details not disclosed in this embodiment, please refer to the specific implementation of the target distance prediction method or system based on the optical characteristics of a gimbal in other embodiments.

[0292] Figure 9 The diagram shows a schematic of a target distance estimation device 400 based on the optical characteristics of a gimbal according to an embodiment of this application.

[0293] like Figure 9 As shown, the target distance prediction device 400 based on the optical characteristics of the gimbal includes: a storage unit 402 for storing executable instructions; and a processing unit 401 for connecting to the storage unit 402 to execute the executable instructions to complete the target distance prediction method based on the optical characteristics of the gimbal.

[0294] Those skilled in the art will understand that the illustration Figure 9 This is merely an example of a target distance estimation device 400 based on the optical characteristics of a gimbal, and does not constitute a limitation on the target distance estimation device 400 based on the optical characteristics of a gimbal. It may include more or fewer components than shown, or combine certain components, or different components. For example, the target distance estimation device 400 based on the optical characteristics of a gimbal may also include input / output devices, network access devices, buses, etc.

[0295] The processing unit 401 (Central Processing Unit, CPU) can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processing unit 401 can be any conventional processor. The processing unit 401 is the control center of the target distance estimation device 400 based on the optical characteristics of the gimbal, connecting all parts of the target distance estimation device 400 based on the optical characteristics of the gimbal through various interfaces and lines.

[0296] Storage unit 402 can be used to store computer-readable instructions. Processing unit 401 implements various functions of the target distance estimation device 400 based on gimbal optical characteristics by running or executing the computer-readable instructions or modules stored in storage unit 402 and calling the data stored in storage unit 402. Storage unit 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the target distance estimation device 400 based on gimbal optical characteristics. In addition, storage unit 402 may include hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, read-only memory (ROM), random access memory (RAM), or other non-volatile / volatile storage devices.

[0297] If the target distance estimation device 400 based on the optical characteristics of a gimbal is implemented as a software functional module and sold or used as an independent product, the integrated module can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium, and when executed by a processor, they can implement the steps of the various method embodiments described above.

[0298] Example 4

[0299] This embodiment provides a computer-readable storage medium having a computer program stored thereon; the computer program is executed by a processor to implement the target distance estimation method based on gimbal optical characteristics in other embodiments.

[0300] The target distance estimation device and medium based on the optical characteristics of a gimbal, as described in this application, include automatic target detection and tracking of the target under test from the current viewpoint to acquire target video; simultaneously, real-time acquisition of gimbal attitude and field of view information; target detection based on the target video to obtain key feature points of the target; distortion correction of the key feature points through image coordinate mapping to form a unified image coordinate system; mapping the image coordinate system with the gimbal attitude information to a three-dimensional virtual space to construct a virtual coordinate system; and calculating the target distance of the target under test based on the geometric position of the target under test in the virtual coordinate system and the gimbal geometric model. This achieves a gimbal system with dynamic attitude adjustment capabilities, enabling accurate estimation of target distance in complex scenes and from different viewpoints, and exhibiting real-time performance and robustness.

[0301] Those skilled in the art will understand that the terminology used in this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” as used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0302] It should be understood that although the terms first, second, third, etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of this invention, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0303] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0304] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A target distance prediction method based on the optical characteristics of a gimbal, characterized in that, include: Automatic target detection and tracking are performed on the target under test from the current viewpoint to obtain target video of the target under test; Simultaneously, the gimbal's attitude information and the camera's focal length are collected in real time; the gimbal's attitude information includes rotation angle, pitch angle, and position translation data; Target detection is performed based on the target video to obtain the key feature points of the target; The key feature points are corrected for distortion through image coordinate mapping to form a unified image coordinate system; Based on the image coordinate system, the coordinates of each point to be measured in the target frame image are transformed in 3D to obtain the three-dimensional coordinates of the point to be measured. Based on the three-dimensional coordinates and the gimbal's attitude information, construct the spatial rotation coordinates of the point to be measured in the current viewpoint; Based on the spatial rotation coordinates, each of the points to be measured in the current viewpoint is mapped to a three-dimensional virtual space to construct a virtual coordinate system; The target distance is calculated based on the geometric position of the target in the virtual coordinate system.

2. The target distance prediction method according to claim 1, characterized in that, Also includes: After adjusting the gimbal attitude and field of view information multiple times, the distances to the target under test were calculated multiple times from different perspectives. The target distance is obtained by fusing multiple real-time distances from different perspectives using a filtering algorithm.

3. The target distance prediction method according to claim 1, characterized in that, The step of calculating the target distance of the target under test based on the geometric position of the target under test in the virtual coordinate system includes: Calculate and compensate for the horizontal deflection angle and vertical pitch angle of any point of the target under test in the virtual coordinate system. Calculate the vertical and horizontal projection distances based on the camera height and pitch angle. Calculate the spatial depth of the point to be measured along the vertical direction based on the camera height and pitch angle; Calculate the Euclidean distance from the point to be measured to the camera based on the vertical projection distance and the horizontal projection distance. Calculate the total azimuth of the point to be measured relative to the centerline of the field based on the angle between the current frame and the centerline and the current deflection angle of the gimbal. Change the test points of the target and repeat the previous steps to output the vertical projection distance, horizontal projection distance, total azimuth angle and Euclidean distance of all test points of the target.

4. The target distance prediction method according to claim 1, characterized in that, The acquisition of key feature points of the target includes feature point estimation when the key feature points are missing; the key feature points include the geometric features of the object; the geometric features include the edges and foot points of the target to be measured.

5. The target distance prediction method according to claim 4, characterized in that, The feature point estimation includes foot point estimation; the foot point estimation includes the following steps: Extract bounding box information from images using object detection models; If the bottom edge of the target is visible, extract the coordinates of the foot points based on the bottom edge of the target bounding box; If the bottom edge of the target is not visible, estimate the coordinates of the foot point based on the average aspect ratio of the target box.

6. The target distance prediction method according to claim 1, characterized in that, The step of correcting distortion of the key feature points through image coordinate mapping to form a unified image coordinate system includes: Based on the target frame image from the current perspective, obtain the image coordinates of any test point of the target to be tested; Based on the image coordinates of any point to be measured, the uniform image coordinates after distortion correction are calculated using a distortion mapping table or a distortion model. This continues until distortion correction of all test points of the target objects is completed, forming a unified image coordinate system.

7. The target distance prediction method according to claim 1, characterized in that, The step of performing a 3D transformation on the coordinates of each point to be measured in the target frame image based on the image coordinate system includes: The coordinates of each point to be measured in the target frame image are increased by adding a z-direction component to obtain a 3D vector, and the vector is made to fall on a sphere with a radius equal to the focal length to obtain the three-dimensional coordinates of the point to be measured.

8. A target distance prediction system based on the optical characteristics of a gimbal, characterized in that, include: Target acquisition module: used for automatic target detection and tracking of the target under test, and to acquire target video of the target under test; Simultaneously, the gimbal's attitude information and the camera's focal length are collected in real time; the gimbal's attitude information includes rotation angle, pitch angle, and position translation data; Target detection module: used to perform target detection based on the target video and obtain the key feature points of the target; Coordinate mapping module: used to perform distortion correction on the key feature points through image coordinate mapping to form a unified image coordinate system; Virtual coordinate module: used to perform 3D transformation on the coordinates of each test point in the target frame image based on the image coordinate system to obtain the three-dimensional coordinates of the test point; construct the spatial rotation coordinates of the test point in the current view based on the three-dimensional coordinates and the gimbal's attitude information; and map each test point in the current view to a three-dimensional virtual space to construct a virtual coordinate system based on the spatial rotation coordinates. Distance estimation module: used to calculate the target distance of the target based on the geometric position of the target in the virtual coordinate system.

9. A target distance prediction device based on the optical characteristics of a gimbal, characterized in that, include: Storage unit, used to store executable instructions; as well as A processing unit is configured to be connected to a memory to execute executable instructions to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a computer program thereon; the computer program is executed by a processor to implement the method as described in any one of claims 1-7.

Images