Target localization methods, devices, electronic equipment and storage media in UAV images

Abstract

This invention relates to a target localization method, apparatus, electronic device, and storage medium in UAV images. The method includes: acquiring UAV images, corresponding laser rangefinder values, and spatial state data collected by the UAV while observing the same target from multiple observation points; determining a relative depth map of the UAV images; performing scale transformation on the corresponding relative depth map based on the laser rangefinder values ​​to obtain an absolute depth map; selecting multiple depth positioning points from the absolute depth map, and calculating the initial GPS coordinates corresponding to each depth positioning point based on the pixel coordinates, depth values, and spatial state data of each depth positioning point to obtain an initial GPS coordinate set for each observation point; and fusing and optimizing each initial GPS coordinate set to obtain the GPS coordinates of the target. Using this invention, the positioning accuracy and real-time performance of the target's GPS coordinates can be improved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and unmanned aerial vehicle (UAV) application technology, and in particular to a method, apparatus, electronic device and storage medium for target localization in UAV images. Background Technology

[0002] With the rapid development of drone technology, drones have been widely used in traffic monitoring, forest fire prevention, power line inspection, and agricultural plant protection. In these application scenarios, when a drone detects abnormal targets (such as fire spots, accident sites, or areas affected by pests and diseases) through its onboard camera, it is necessary to quickly obtain the precise geographical coordinates of the target so that ground personnel can respond in a timely manner.

[0003] Existing UAV systems are equipped with sensors such as GPS modules, cameras, and laser rangefinders, which can be used to calculate target positions. However, current technologies still have significant shortcomings in achieving high-precision real-time target localization. Because they primarily rely on the UAV's own GPS coordinates and laser rangefinder values ​​for simple triangulation, the positioning accuracy is low, typically only providing a rough range of the target. Some high-precision localization methods based on binocular vision or SLAM are computationally complex and difficult to implement in real-time on UAV embedded computing platforms. Furthermore, single-point observation methods are susceptible to interference from random errors such as sensor noise and GPS signal drift, resulting in insufficient reliability and robustness of single measurements.

[0004] Therefore, how to quickly and accurately calculate the precise GPS coordinates of targets in UAV images is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] In view of this, it is necessary to provide a target localization method, device, electronic device and storage medium in UAV images to solve the technical problems of poor real-time performance and low GPS coordinate positioning accuracy of the existing GPS coordinate determination method for targets in UAV images.

[0006] To address the aforementioned problems, in a first aspect, the present invention provides a target localization method in UAV images, comprising: The system acquires multiple frames of UAV images containing the target collected when the UAV observes the same target from multiple observation points, as well as the laser range value corresponding to each frame of the UAV image and the spatial state data of the UAV. Each frame of the UAV image is input into the monocular depth estimation model to obtain the corresponding relative depth map. Based on the laser ranging value, the corresponding relative depth map is scaled to obtain the absolute depth map; Multiple depth positioning points are selected from the absolute depth map. Based on the pixel coordinates and depth value of each depth positioning point, as well as the spatial state data, the initial GPS coordinates corresponding to each depth positioning point are calculated to obtain the initial GPS coordinate set for each observation point. The initial GPS coordinate set at each observation point is fused and optimized to obtain the GPS coordinates of the target.

[0007] In one possible implementation, the monocular depth estimation model is a Depth Anything model, which includes an encoder, a neck feature fusion module, and a head depth regression module. The encoder includes a ConvNeXt-V2 backbone network, a feature refinement layer, and a Transformer module connected in series. The feature refinement layer includes cascaded depthwise separable convolutions and GLU activation functions. The Transformer module employs a simplified attention head and sequence length reduction mechanism. The step of inputting each frame of the UAV image into the monocular depth estimation model to obtain the corresponding relative depth map includes: The ConvNeXt-V2 backbone network is used to extract multi-scale features from the UAV images to obtain initial feature maps of different resolutions. The initial feature map is refined by the feature refinement layer to obtain a refined feature map. The Transformer module is used to perform global context modeling on the refined feature map to obtain the encoded feature map. The neck feature fusion module performs hierarchical fusion on the encoded feature map, aligning and fusing low-resolution high semantic features with high-resolution detail features to obtain fused multi-scale features. The head depth regression module performs depth regression prediction on the fused multi-scale features and outputs the corresponding relative depth map.

[0008] In one possible implementation, scaling the corresponding relative depth map based on the laser ranging value includes: The depth value of the central region is extracted from the relative depth map, and the average depth value of the central region is determined to obtain the model average depth. The ratio of the laser ranging value to the average depth of the model is calculated to obtain the scale factor; The absolute depth map is generated by multiplying the depth value of each pixel in the relative depth map by the scale factor.

[0009] In one possible implementation, the pixel coordinates include pixel horizontal coordinates and pixel vertical coordinates, and the spatial state data includes the differential GPS coordinates and attitude angles of the UAV; the step of calculating the initial GPS coordinates corresponding to each depth positioning point based on the pixel coordinates and depth value of each depth positioning point, and the spatial state data, includes: Obtain the intrinsic parameters of the camera mounted on the drone, including the principal point x-coordinate, principal point y-coordinate, lateral focal length, and longitudinal focal length; Subtract the principal point's horizontal coordinate from the horizontal coordinate of each pixel and divide by the horizontal focal length to obtain the normalized horizontal coordinate; subtract the principal point's vertical coordinate from the vertical coordinate of each pixel and divide by the vertical focal length to obtain the normalized vertical coordinate. Multiply the normalized lateral coordinate by the depth value of the corresponding depth positioning point to obtain the rotation matrix of the camera coordinate system from the camera coordinate system to the UAV body coordinate system. Transform the camera lateral coordinate, camera longitudinal coordinate, and camera depth coordinate in the camera coordinate system to the camera lateral coordinate under the UAV body. Multiply the normalized longitudinal coordinate by the depth value of the corresponding depth positioning point to obtain the camera longitudinal coordinate in the camera coordinate system. Use the depth value of the corresponding depth positioning point as the camera depth coordinate in the camera coordinate system. Using the camera's gimbal angle to construct a reference system, the three-dimensional coordinates of the machine body in the machine body coordinate system are obtained; A rotation matrix from the body coordinate system to the navigation coordinate system is constructed using the attitude angle, the three-dimensional coordinates of the body are transformed into the navigation coordinate system, and the position coordinates of the UAV in the navigation coordinate system are added to the position coordinates of the UAV in the navigation coordinate system. The position is obtained by coordinate transformation based on the differential GPS coordinates. The position coordinates in the navigation coordinate system are converted into GPS coordinates to obtain the initial GPS coordinates corresponding to the depth positioning point.

[0010] In one possible implementation, after obtaining the initial set of GPS coordinates for each observation point, the method further includes: Based on the pixel coordinates of each depth positioning point in the absolute depth map, calculate the distance from the depth positioning point to the center of the absolute depth map; The weight of the initial GPS coordinates corresponding to the depth positioning point is determined based on the distance, wherein the distance is negatively correlated with the weight.

[0011] In one possible implementation, the distance is a Euclidean distance; the step of determining the weights includes: Based on the Euclidean distance, the weight corresponding to the depth positioning point is calculated according to the exponential decay method, wherein the decay coefficient in the exponential decay method ranges from 0.01 to 0.1.

[0012] In one possible implementation, the fusion optimization of the initial GPS coordinate set for each observation point includes: The RANSAC algorithm is used to remove outliers from each initial GPS coordinate in the initial GPS coordinate set to obtain the interior point set. The initial GPS coordinates in the set of interior points are weighted and averaged to obtain the GPS coordinates of the target.

[0013] Secondly, the present invention also provides a target localization device in UAV images, comprising: The acquisition unit is used to acquire multiple frames of UAV images containing the target collected by the UAV when the UAV observes the same target from multiple observation points, as well as the laser range value and the spatial state data of the UAV corresponding to each frame of the UAV image; The estimation unit is used to input the UAV images of each frame into the monocular depth estimation model to obtain their respective relative depth maps. A conversion unit is used to perform scale conversion on the corresponding relative depth map based on the laser ranging value to obtain an absolute depth map; The calculation unit is used to select multiple depth positioning points from the absolute depth map, and calculate the initial GPS coordinates corresponding to each depth positioning point according to the pixel coordinates and depth value of each depth positioning point and the spatial state data, so as to obtain the initial GPS coordinate set under each observation point. The fusion unit is used to fuse and optimize the initial GPS coordinate set under each observation point to obtain the GPS coordinates of the target.

[0014] Thirdly, the present invention also provides an electronic device, including a memory and a processor, wherein, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the target localization method in UAV images described in any of the above implementations.

[0015] Fourthly, the present invention also provides a computer-readable storage medium for storing a computer-readable program or instructions, which, when executed by a processor, can implement the steps in the target localization method in UAV images described in any of the above implementations.

[0016] The beneficial effects of this invention are: The target localization method in UAV images provided by this invention acquires multiple frames of UAV images containing the target collected when the UAV observes the same target from multiple observation points, along with the laser ranging value and spatial state data of the UAV corresponding to each frame of the UAV image. This helps to eliminate random errors that may be introduced by a single observation, thereby improving the reliability and stability of the localization results. Each frame of the UAV image is input into a monocular depth estimation model to obtain its corresponding relative depth map, avoiding the high computational overhead of binocular matching or SLAM algorithms. Furthermore, the forward inference computation of the monocular depth estimation model is reduced, which is beneficial for real-time processing on embedded platforms, ensuring the real-time performance of subsequent localization. Based on the laser ranging value, the corresponding relative depth map is scaled to obtain an absolute depth map. The system achieves absolute scale calibration of the relative depth map, giving the entire depth map a true physical scale. Multiple depth positioning points are selected from the absolute depth map. Based on the pixel coordinates, depth values, and spatial state data of each depth positioning point, the initial GPS coordinates corresponding to each depth positioning point are calculated, resulting in an initial GPS coordinate set for each observation point. This fully utilizes the depth information of different regions of the image, improving the robustness of coordinate estimation in a single observation. The initial GPS coordinate sets for each observation point are then fused and optimized to obtain the target's GPS coordinates. This effectively integrates the initial GPS coordinates generated by multiple observation points and multiple depth positioning points, reducing the impact of single measurement errors, sensor noise, and environmental interference, and improving the positioning accuracy and real-time performance of the target's GPS coordinates. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the positioning system provided by the present invention; Figure 2 This is a schematic flowchart of an embodiment of the target localization method in UAV images provided by the present invention; Figure 3 Provided by the present invention Figure 2 A schematic diagram of an embodiment of S202; Figure 4 Provided by the present invention Figure 2 A schematic diagram of an embodiment of S203; Figure 5 Provided by the present invention Figure 2 A schematic diagram of an embodiment of S204; Figure 6 Provided by the present invention Figure 2 A schematic diagram of an embodiment of S205; Figure 7 This is a schematic diagram of the target positioning device in UAV images provided by the present invention; Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0019] In the description of the embodiments of the present invention, unless otherwise stated, "a plurality of" means two or more.

[0020] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] This invention provides a target localization method, device, electronic device, and storage medium in UAV images, which are described below.

[0023] The target localization method in UAV images according to the embodiments of this application can be used in scenarios such as traffic congestion monitoring, forest fire prevention patrol, power line inspection, and agricultural pest and disease monitoring to achieve rapid, accurate, and real-time localization of specific targets (such as fire points, accident vehicles, defective insulators, and pest and disease areas) in images taken by UAVs.

[0024] The execution entity of the target localization method in UAV images according to the embodiments of this application can be the target localization device in UAV images provided in the embodiments of this application, or different types of electronic devices such as server equipment, physical host, or user equipment (UE) that integrate the target localization device in UAV images, an embedded computing platform mounted on the UAV, or a ground station server. The target localization device in UAV images can be implemented in hardware or software. Specifically, the UE can be a smartphone, tablet computer, laptop computer, PDA, desktop computer, or personal digital assistant (PDA) or other terminal device.

[0025] The target localization method in UAV images provided in this embodiment can be applied to, for example... Figure 1 The positioning system shown is used as an example for illustration. This positioning system is mounted on a drone flight platform, which carries various sensors and flies along a preset route. The positioning system includes the following hardware components: The RTK-GPS module 101 is electrically connected to the embedded computing platform 106 and is used to send the UAV's own precise three-dimensional GPS coordinates to the embedded computing platform 106. The horizontal positioning accuracy of the RTK-GPS module reaches 1cm+1ppm and the vertical positioning accuracy reaches 1.5cm+1ppm. The inertial measurement unit 102 is electrically connected to the embedded computing platform 106 and is used to acquire attitude angles (roll angle, pitch angle, yaw angle) in the spatial state data of the UAV in real time and send them to the embedded computing platform 106. The gimbal angle sensor 103 is electrically connected to the embedded computing platform 106 and is used to acquire the pitch angle and yaw angle of the camera gimbal and send them to the embedded computing platform 106. The laser rangefinder 104 is electrically connected to the embedded computing platform 106 and is used to measure the straight-line distance from the UAV to the target and send the measured distance value to the embedded computing platform 106; the rangefinder has a range of 0.5m-3000m and a measurement accuracy of ±1m; The dual-camera system 105 includes a wide-angle camera and a telephoto camera, both of which are electrically connected to the embedded computing platform 106. It is used to synchronously acquire images of the UAV containing the target and transmit the image data to the embedded computing platform 106. The wide-angle camera has a focal length of 24mm, the telephoto camera has a focal length of 162mm, and both have a resolution of 1920×1080. The embedded computing platform 106, as a computing processing unit, is connected to each of the above-mentioned sensor modules to receive and process all sensor data and execute the target localization method. The embedded computing platform adopts NVIDIA Jetson OrinNX and is equipped with a Depth Anything depth estimation model accelerated by TensorRT quantization. The data transmission communication module 107 is electrically connected to the embedded computing platform 106 and is used to send the precise GPS coordinates of the target calculated by the embedded computing platform 106 to the ground station or cloud server 108. The ground station or cloud server 108 is connected to the data transmission communication module 107 to receive and display the GPS coordinates of the target.

[0026] It should be noted that the connecting lines in the diagram represent signal transmission relationships, and in actual implementation, wired or wireless methods can be used. The power supply relationships of the various modules are not shown in the diagram and can be uniformly powered by the power system of the UAV flight platform.

[0027] The method steps of this embodiment will be described in detail below with reference to the above hardware configuration.

[0028] Figure 2 This is a schematic flowchart of an embodiment of the target localization method in UAV images provided by the present invention, as shown below. Figure 2 As shown, the target localization methods in UAV images include: S201. Acquire multiple frames of UAV images containing the target collected when the UAV observes the same target from multiple observation points, as well as the laser ranging value corresponding to each frame of the UAV image and the spatial state data of the UAV.

[0029] The spatial state data of the UAV includes: differential GPS coordinates of the UAV obtained by the RTK-GPS module, attitude angles of the UAV obtained by the inertial measurement unit, and camera gimbal angles obtained by the gimbal angle sensor; the attitude angles of the UAV include roll angle, pitch angle and yaw angle, and the camera gimbal angles include pitch angle and yaw angle.

[0030] The drone images were acquired by a dual-camera system with both wide-angle and telephoto lenses mounted on the drone. The dual-camera system includes a wide-angle camera and a telephoto camera, both of which have been jointly calibrated, and their intrinsic and extrinsic parameters are known.

[0031] Specifically, the drone can fly along a preset route, taking pictures and measuring distances to the same target area from multiple different locations. At each observation point, the drone acquires at least one frame of an image containing the target using its onboard camera, simultaneously measures the straight-line distance from the drone to the target using a laser rangefinder, and records the drone's spatial state data using onboard sensors. This spatial state data may include the drone's differential GPS coordinates, attitude angles, and camera gimbal angles, and may also include other sensor data related to the drone's pose; this embodiment does not specifically limit this. As an example, the drone can sequentially pass through three observation points P1, P2, and P3 during its flight, acquiring one frame of image at each observation point and simultaneously recording the corresponding laser rangefinder value and spatial state data, thereby obtaining three sets of raw data.

[0032] In this embodiment, data collection from multiple observation points provides abundant raw data for subsequent positioning, which helps to eliminate random errors that may be introduced by a single observation, thereby improving the reliability and stability of the positioning results.

[0033] S202. Input the UAV images of each frame into the monocular depth estimation model to obtain the corresponding relative depth map.

[0034] Monocular depth estimation models are deep learning-based computer vision models. Their input is a single-frame RGB image, such as a drone image, and their output is the relative depth value of each pixel in the drone image (i.e., the distance of each pixel from the camera, not based on actual physical scale). Monocular depth estimation models include MiDaS and DepthAnything.

[0035] The value of each pixel in the relative depth map represents the relative distance of the scene point corresponding to that pixel from the camera. The larger the value, the closer (or farther, depending on the model definition) the distance. However, there is only an unknown scale ratio between this value and the actual physical distance (such as meters).

[0036] Specifically, for each UAV image acquired at each observation point, the image is input into the monocular depth estimation model for inference, and the corresponding relative depth map is output. This embodiment does not rely on binocular vision or multi-view images; dense depth information of the entire scene can be obtained from a single frame image, providing a rich data foundation for subsequent pixel-level coordinate calculations. It also avoids the high computational overhead of binocular matching or SLAM algorithms, and reduces the forward inference computation of the monocular depth estimation model, facilitating real-time processing on embedded platforms and ensuring the real-time performance of subsequent positioning.

[0037] S203. Based on the laser ranging value, perform scale transformation on the corresponding relative depth map to obtain an absolute depth map.

[0038] An absolute depth map is a depth map in which the depth value of each pixel in an image has a real physical scale unit (e.g., meters).

[0039] Specifically, since the laser rangefinder can directly measure the physical distance from the UAV to the target (or the area near the target), this measurement can serve as a precise scale benchmark. The region corresponding to the measurement direction of the laser rangefinder can be selected from the relative depth map, the average relative depth value of the region can be calculated, and then the laser rangefinder value can be divided by the average relative depth value to obtain the scale factor. Finally, the relative depth value of each pixel in the relative depth map can be multiplied by the scale factor, thereby converting the dimensionless relative depth into an absolute depth in meters.

[0040] Understandably, in this embodiment, the single-point physical measurement value of the laser rangefinder is used to solve the inherent scale ambiguity problem of monocular depth estimation, realize the absolute scale calibration of the relative depth map, and enable the entire depth map to have a real physical scale. This ensures that the depth value of each pixel in the subsequent coordinate calculation corresponds to the actual spatial distance, thereby improving the accuracy of target positioning.

[0041] S204. Select multiple depth positioning points from the absolute depth map, and calculate the initial GPS coordinates corresponding to each depth positioning point based on the pixel coordinates and depth value of each depth positioning point and the spatial state data, so as to obtain the initial GPS coordinate set under each observation point.

[0042] The depth positioning points can be multiple representative pixel locations in the absolute depth map, such as the center point, four corner points, and several points on the edge of the image.

[0043] The initial GPS coordinates are coarse GPS coordinates obtained by transforming each depth positioning point selected from the absolute depth map of a single observation point. These coarse GPS coordinates are raw estimates without fusion optimization.

[0044] Specifically, for each selected depth positioning point, using the pinhole camera imaging model, combined with camera intrinsic parameters and UAV spatial state data, the GPS coordinates of the actual ground feature corresponding to that pixel are calculated through a series of transformations: from pixel coordinates to camera coordinates, from camera coordinates to body coordinates, from body coordinates to navigation coordinates, and from navigation coordinates to geographic coordinates. Since multiple depth positioning points are selected under one observation point, each observation point can obtain a set of initial GPS coordinates. For example, selecting 25 depth positioning points can obtain 25 initial coordinates. This embodiment selects multiple depth positioning points in a single frame image, making full use of the depth information of different regions of the image, avoiding excessive influence of individual pixel depth estimation errors on the final result, and improving the robustness of coordinate estimation under a single observation.

[0045] S205. The initial GPS coordinate set under each observation point is fused and optimized to obtain the GPS coordinates of the target.

[0046] The fusion optimization process integrates all initial GPS coordinates generated from all observation points, removing outliers that significantly deviate from the true location and consolidating valid estimates to output the final result. Fusion involves merging and aggregating all initial GPS coordinate sets (i.e., multiple sets of coarse GPS coordinates) obtained by the UAV from multiple observation points into a single dataset. The data comes from independent estimations at different spatial locations and image depths, exhibiting redundancy and complementarity. Optimization involves further processing the aggregated initial GPS coordinate dataset, such as removing outliers caused by sensor transient noise, GPS signal drift, and image matching errors.

[0047] Specifically, a random sampling consensus algorithm can be used to iteratively filter all initial GPS coordinates, identify and remove outliers, and then perform a weighted average or arithmetic average on the selected interior points to obtain the final accurate GPS coordinates.

[0048] In this embodiment, the initial GPS coordinates generated by multiple observation points and multiple depth positioning points are effectively integrated, thereby combining the advantages of each estimation result, reducing the impact of single measurement error, sensor noise and environmental interference, and improving the positioning accuracy and reliability of the final output positioning result.

[0049] In summary, the target localization method in UAV images provided by this invention, by acquiring multiple frames of UAV images containing the target collected when the UAV observes the same target from multiple observation points, as well as the laser ranging value and spatial state data of the UAV corresponding to each frame of the UAV image, helps to eliminate random errors that may be introduced by a single observation, thereby improving the reliability and stability of the localization results. Inputting each frame of the UAV image into a monocular depth estimation model to obtain its corresponding relative depth map avoids the high computational overhead of binocular matching or SLAM algorithms, and reduces the forward inference computation of the monocular depth estimation model, which is beneficial for real-time processing on embedded platforms and ensures the real-time performance of subsequent localization. Based on the laser ranging value, the corresponding relative depth map is scaled to obtain the absolute depth. The depth map achieves absolute scale calibration of the relative depth map, giving the entire depth map a true physical scale. Multiple depth positioning points are selected from the absolute depth map. Based on the pixel coordinates, depth values, and spatial state data of each depth positioning point, the initial GPS coordinates corresponding to each depth positioning point are calculated, resulting in an initial GPS coordinate set for each observation point. This fully utilizes the depth information of different regions of the image, improving the robustness of coordinate estimation in a single observation. The initial GPS coordinate sets for each observation point are then fused and optimized to obtain the target's GPS coordinates. This effectively integrates the initial GPS coordinates generated by multiple observation points and multiple depth positioning points, reducing the impact of single measurement errors, sensor noise, and environmental interference, and improving the positioning accuracy and real-time performance of the target's GPS coordinates.

[0050] In some embodiments of the present invention, such as Figure 3 As shown, the monocular depth estimation model is a Depth Anything model, which includes an encoder, a neck feature fusion module, and a head depth regression module. The encoder includes a ConvNeXt-V2 backbone network, a feature refinement layer, and a Transformer module connected in series. The feature refinement layer includes cascaded depthwise separable convolutions and GLU activation functions. The Transformer module employs a simplified attention head and sequence length shortening mechanism. Step S202 includes: S301. Multi-scale feature extraction is performed on the UAV image through the ConvNeXt-V2 backbone network to obtain initial feature maps of different resolutions; S302. The initial feature map is refined through the feature refinement layer to obtain a refined feature map. S303. The refined feature map is modeled globally using the Transformer module to obtain the encoded feature map; S304. The coded feature map is fused in layers by the neck feature fusion module, and the low-resolution high semantic features and high-resolution detail features are aligned and fused to obtain the fused multi-scale features. S305. The head depth regression module performs depth regression prediction on the fused multi-scale features and outputs the corresponding relative depth map.

[0051] Among them, the ConvNeXt-V2 backbone network is a modern convolutional neural network architecture that uses depthwise separable convolution and GLU activation function for multi-scale feature extraction.

[0052] Depthwise separable convolution breaks down standard convolution into two steps: depthwise convolution and pointwise convolution, which reduces computational cost and the number of parameters.

[0053] The GLU activation function is a gated linear unit used to enhance the nonlinear expressive power of a model.

[0054] The Transformer module employs simplified attention heads and sequence length reduction mechanisms to reduce model computation for deployment on embedded platforms. This can be achieved through DINOv2 (Vision Transformer). DINOv2 is a self-supervised vision Transformer that segments the input image into patches (e.g., 14×14 patches) and then extracts features using a multi-layer Transformer encoder.

[0055] The Neck Feature Fusion Module (LFFM) is a layered feature fusion module used to efficiently fuse low-resolution high-semantic features with high-resolution detail features. It can be implemented based on the Dense Prediction Transformer (DPT) decoder structure.

[0056] The Multi-scale Depth Refinement Module (MDRM) employs a multi-scale depth refinement module to improve the depth estimation accuracy for small objects and edge regions. It can be implemented based on the Head Stage of the DPT decoder, converting the fused features into the final depth map. Its multi-scale depth refinement capability stems from: extracting multi-scale features from multiple layers of the encoder (typically the last four layers, i.e., stages 9-12), capturing semantic and texture information of different granularities; and the multi-resolution fusion mechanism in the DPT architecture preserves edge details, improving the prediction accuracy for small objects.

[0057] Specifically, after inputting each frame of UAV image into the Depth Anything model, the model performs the following operations: First, it extracts multi-scale features from the input UAV image using the ConvNeXt-V2 backbone network to obtain initial feature maps at different resolutions. Then, it refines the initial feature maps using a feature refinement layer, which includes cascaded depthwise separable convolutions and GLU activation functions to improve feature expressiveness while maintaining computational efficiency. Next, it models the global context of the refined feature maps using a Transformer module. This Transformer module employs a simplified attention head and sequence length shortening mechanism to reduce the overall computational load, resulting in encoded feature maps. Then, it performs layered fusion of the encoded feature maps using a neck feature fusion module, aligning and fusing low-resolution high-semantic features with high-resolution detail features to obtain fused multi-scale features. Finally, it performs depth regression prediction on the fused multi-scale features using a head depth regression module, outputting the corresponding relative depth map with dimensions [B, 1, H, W], where B is the batch size, and H and W are the height and width of the input image, respectively.

[0058] Understandably, this embodiment, through the optimized design of each module of the Depth Anything model, can achieve efficient depth estimation on the UAV embedded computing platform, ensuring both the accuracy of depth estimation and meeting the requirements of real-time processing.

[0059] In some embodiments of the present invention, such as Figure 4 As shown, step S203 includes: S401. Extract the depth value of the central region from the relative depth map, determine the average depth value of the central region, and obtain the model average depth. S402. Calculate the ratio of the laser ranging value to the average depth of the model to obtain the scale factor; S403. Multiply the depth value of each pixel in the relative depth map by the scale factor to generate the absolute depth map.

[0060] The scale factor is a proportional coefficient used to convert relative depth values ​​into absolute depth values. Its physical meaning is the actual physical distance corresponding to a unit of relative depth.

[0061] Specifically, the depth values ​​of the central region are extracted from the relative depth map, and the average depth value of this region is calculated to obtain the model average depth d_model. For example, the central region can be a square region with a side length of 64 pixels, that is, all depth values ​​within a 64×64 pixel range in the center of the image are averaged. Then, the distance value measured by a laser rangefinder at the same time as the relative depth map is obtained and denoted as the laser rangefinder value d_lidar, with the unit being meters. The scale factor is calculated as scale = d_lidar / d_model; the depth value D_model(x,y) of each pixel in the relative depth map is multiplied by the scale factor to obtain the absolute depth map: D_absolute(x,y) = D_model(x,y) × scale, where D_absolute(x,y) represents the depth value at pixel (x,y) in the absolute depth map, and D_model(x,y) represents the depth value of the corresponding pixel in the relative depth map.

[0062] Understandably, by using the single-point physical measurement value provided by the laser rangefinder as a global scale benchmark, the entire relative depth map is uniformly linearly scaled, thereby converting the relative depth into an absolute depth with real physical meaning, thus improving the positioning accuracy of the target's GPS coordinates.

[0063] In some embodiments of the present invention, such as Figure 5 As shown, the pixel coordinates include pixel horizontal coordinates and pixel vertical coordinates, and the spatial state data includes the differential GPS coordinates and attitude angles of the UAV; step S204 includes: S501. Obtain the intrinsic parameters of the camera mounted on the UAV, the intrinsic parameters including the principal point x-coordinate, principal point y-coordinate, lateral focal length and longitudinal focal length; S502. Subtract the principal point's horizontal coordinate from the horizontal coordinate of each pixel and divide by the horizontal focal length to obtain a normalized horizontal coordinate. Subtract the principal point's vertical coordinate from the vertical coordinate of each pixel and divide by the vertical focal length to obtain a normalized vertical coordinate. S503. Multiply the normalized lateral coordinate by the depth value of the corresponding depth positioning point to obtain the rotation matrix of the camera coordinate system from the camera coordinate system to the UAV body coordinate system. Transform the camera lateral coordinate, camera longitudinal coordinate, and camera depth coordinate in the camera coordinate system to the camera lateral coordinate under the UAV body. Multiply the normalized longitudinal coordinate by the depth value of the corresponding depth positioning point to obtain the camera longitudinal coordinate in the camera coordinate system. Use the depth value of the corresponding depth positioning point as the camera depth coordinate in the camera coordinate system. S504. Using the camera's gimbal angle to construct a reference system, obtain the three-dimensional coordinates of the machine body in the machine body coordinate system; S505. Construct a rotation matrix from the body coordinate system to the navigation coordinate system using the attitude angle, transform the three-dimensional coordinates of the body to the navigation coordinate system, and add it to the position of the UAV in the navigation coordinate system to obtain the position coordinates in the navigation coordinate system, wherein the position is obtained by coordinate transformation based on the differential GPS coordinates; S506. Convert the position coordinates in the navigation coordinate system into GPS coordinates to obtain the initial GPS coordinates corresponding to the depth positioning point.

[0064] Intrinsic parameters refer to the set of parameters representing the camera's internal optical characteristics, including the principal point's x-coordinate (cx), principal point's y-coordinate (cy), lateral focal length (fx), and longitudinal focal length (fy). The principal point is located near the image center and represents the coordinates of the intersection of the optical axis and the imaging plane. The focal length represents the distance from the camera lens center to the imaging plane, expressed in pixels. These intrinsic parameters can be obtained in advance using camera calibration algorithms, such as the Zhang Zhengyou checkerboard calibration method.

[0065] Gimbal angle refers to the rotation angle of the camera gimbal relative to the drone body, including pitch and yaw angles. Attitude angle refers to the rotation angle of the drone body relative to the navigation coordinate system, including roll angle φ, pitch angle θ, and yaw angle ψ. Differential GPS coordinates refer to the drone's latitude, longitude, and altitude coordinates with centimeter-level accuracy obtained through an RTK-GPS module.

[0066] Specifically, first, the camera intrinsic parameters are obtained, namely, the principal point's x-coordinate cx, principal point's y-coordinate cy, lateral focal length fx, and axial focal length fy. For each depth positioning point's pixel coordinates (x, y) and its depth value D, the following transformations are performed: Calculate the normalized lateral coordinate u: u = (x - cx) / fx, and calculate the normalized axial coordinate v: v = (y - cy) / fy. u and v are the normalized lateral and axial coordinates, respectively, and are dimensionless.

[0067] Then, the normalized coordinates are multiplied by the depth value to obtain the three-dimensional coordinates in the camera coordinate system: camera horizontal coordinate X_cam = u × D, camera vertical coordinate Y_cam = v × D, and camera depth coordinate Z_cam = D.

[0068] Furthermore, a rotation matrix R_cam2body is constructed using the camera's gimbal angle to transform the 3D coordinates (X_cam, Y_cam, Z_cam) in the camera coordinate system to the UAV body coordinate system, resulting in the 3D body coordinates (X_body, Y_body, Z_body). The specific transformation relationship is: P_body = R_cam2body × P_cam, where P_cam = [X_cam, Y_cam, Z_cam]^T and P_body = [X_body, Y_body, Z_body]^T.

[0069] Next, using the UAV's attitude angles (roll angle φ, pitch angle θ, yaw angle ψ), a rotation matrix R_body2ned is constructed from the body coordinate system to the navigation coordinate system. The 3D coordinates of the UAV are then transformed to the navigation coordinate system: P_ned_raw = R_body2ned × P_body. The UAV's differential GPS coordinates are then converted to position coordinates P_uav_ned in the navigation coordinate system. The two coordinates are added together to obtain the absolute position coordinates of the depth positioning point in the navigation coordinate system: P_ned = P_ned_raw + P_uav_ned.

[0070] Finally, the position coordinates in the navigation coordinate system are converted to GPS coordinates. This is done by calculating the longitude, latitude, and altitude increments based on the northward displacement Δy_ned, eastward displacement Δx_ned, and groundward displacement Δz_ned in the navigation coordinate system, combined with the Earth's radius R_earth and the UAV's latitude lat. This yields the initial GPS coordinates corresponding to the depth positioning point. The conversion formulas are: Δlat = Δy_ned / (R_earth + h) × (180 / π), Δlon = Δx_ned / [(R_earth + h) × cos(lat)] × (180 / π), Δh = -Δz_ned.

[0071] Wherein, the depth value D is taken from the actual physical depth value of the corresponding pixel position in the absolute depth map; Δx_ned, Δy_ned, and Δz_ned represent the eastward, northward, and groundward displacements of the target relative to the UAV in the navigation coordinate system, respectively, and these displacement values ​​are obtained from the difference between P_ned and P_uav_ned; R_earth is the Earth's radius, which is 6,378,137 meters; h is the current altitude of the UAV; and π is the constant of pi.

[0072] Understandably, this embodiment, through the complete conversion chain described above—from pixel coordinates to camera coordinates, from camera coordinates to drone coordinates, from drone coordinates to navigation coordinates, and from navigation coordinates to GPS coordinates—precisely combines the depth information of any pixel in the image with the spatial state data of the UAV itself, achieving a mapping from two-dimensional images to three-dimensional geographic coordinates. Camera intrinsic parameters ensure accurate mapping from pixel coordinates to camera coordinates, gimbal angles and attitude angles ensure the correctness of rotational relationships between coordinate systems, and differential GPS coordinates provide a high-precision position reference.

[0073] In some embodiments of the present invention, after obtaining the initial GPS coordinate set for each observation point, the method further includes: calculating the distance from the depth positioning point to the center of the absolute depth map based on the pixel coordinates of each depth positioning point in the absolute depth map; and determining the weight of the initial GPS coordinates corresponding to the depth positioning point based on the distance, wherein the distance is negatively correlated with the weight.

[0074] The distance from the depth positioning point to the center of the absolute depth map refers to the Euclidean distance between that pixel and the center point of the image.

[0075] Weight refers to a coefficient used to measure the importance of each initial GPS coordinate in subsequent fusion optimization. This coefficient is negatively correlated with distance, that is, the closer the depth positioning point is to the center of the image, the higher the weight of the initial GPS coordinate, and the farther the depth positioning point is from the center of the image, the lower the weight of the initial GPS coordinate.

[0076] The decay coefficient α is a parameter that controls how quickly the weight decays with distance. Its value ranges from 0.01 to 0.1. The larger the value of α, the faster the weight decays with distance.

[0077] Specifically, for each frame of the absolute depth map at each observation point, the Euclidean distance from the pixel coordinates of each selected depth positioning point to the center pixel coordinates of the absolute depth map is calculated. Then, the weight of the initial GPS coordinates corresponding to that depth positioning point is calculated based on the Euclidean distance using an exponential decay method.

[0078] Understandably, this embodiment introduces an image position weighting strategy, giving higher weight to the initial GPS coordinates of depth positioning points closer to the image center. Due to lens distortion and perspective effects, the depth estimation accuracy of image edge regions is usually lower than that of the center region. Therefore, the weighting method can effectively suppress the negative impact of low-quality depth data in edge regions on the positioning results, thereby improving the accuracy of the final target GPS coordinates. At the same time, the exponential decay method makes the weights change smoothly with distance, avoiding instability that may be introduced by weight jumps.

[0079] In some embodiments of the present invention, the distance is Euclidean distance; the step of determining the weight includes: calculating the weight corresponding to the depth positioning point according to the Euclidean distance and in an exponential decay manner, wherein the decay coefficient in the exponential decay manner ranges from 0.01 to 0.1.

[0080] Here, Euclidean distance refers to the straight-line distance between two pixels in the image plane; in this embodiment, it specifically refers to the Euclidean distance from the depth positioning point to the image center point. Exponential decay refers to a function where the weight decreases exponentially as the distance of the independent variable increases. Its mathematical expression is that the weight equals the negative power of the natural constant, and the decay coefficient is used to control the rate of decay.

[0081] Specifically, weights are assigned to the initial GPS coordinates corresponding to each depth positioning point. First, the Euclidean distance from the depth positioning point to the image center is calculated using the formula: dist_i = sqrt((x_i - cx)^2 + (y_i - cy)^2), where (x_i, y_i) are the pixel coordinates of the depth positioning point, and (cx, cy) are the coordinates of the image center. Then, the weights are calculated based on this Euclidean distance using an exponential decay method, using the formula w_i = exp(-α × dist_i), where α is the decay coefficient, ranging from 0.01 to 0.1. Through this calculation, depth positioning points closer to the image center receive higher weights, while those closer to the image edge receive lower weights.

[0082] Understandably, this embodiment effectively suppresses the impact of lens distortion and perspective effects on the accuracy of depth values ​​in image edge regions by introducing an exponential decay weighting method based on Euclidean distance. This allows more reliable depth information in the central region to play a greater role in subsequent fusion, thereby improving the overall accuracy and stability of coordinate calculation.

[0083] In some embodiments of the present invention, such as Figure 6 As shown, step S205 includes: S601. Use the RANSAC algorithm to remove outliers from each initial GPS coordinate in the initial GPS coordinate set to obtain the interior point set. S602. Perform a weighted average of the initial GPS coordinates in the set of interior points to obtain the GPS coordinates of the target.

[0084] Among them, RANSAC (Random Sample Consensus) algorithm refers to the random sampling consensus algorithm, which is a robust method for estimating mathematical model parameters from a dataset containing outliers through iterative sampling.

[0085] Weighted averaging refers to assigning different weight coefficients to each value in a set of data and then averaging them. The higher the weight, the greater the contribution of that value to the final result. In this embodiment, the weight of each initial GPS coordinate can be predetermined based on its position in the image; for example, coordinates closer to the image center are given higher weights.

[0086] Specifically, the RANSAC algorithm is first used to remove outliers from all initial GPS coordinates obtained from multiple observation points. For example, suppose a UAV obtains 75 initial GPS coordinates from three observation points. The maximum number of iterations K is set to 100, and the inlier threshold ε is set to 10 meters. In each iteration, three coordinate points are randomly selected from the 75 as the minimum sample set, and the geometric center coordinates of these three points are calculated as candidate models. Then, the Euclidean distance from each initial GPS coordinate to the candidate center is calculated. If the distance is less than 10 meters, the coordinate is identified as an inlier. The number of inliers in this iteration is counted. If it exceeds the historical best number of inliers, the best inlier set is updated. After 100 iterations, the resulting set of inliers is the effective coordinate set after outlier removal. For example, after iterations, 65 inliers may be selected, while the remaining 10 are identified as outliers and removed.

[0087] Furthermore, after obtaining the optimal set of inliers, a weighted average is applied to the initial GPS coordinates within this set to calculate the final precise GPS coordinates of the target. Specifically, the longitude coordinate of each inlier is multiplied by its corresponding weight, summed, and then divided by the sum of the weights to obtain the final longitude value. Latitude and altitude coordinates are calculated in the same way. It should be noted that the weight of each initial GPS coordinate was determined in the previous image position weighting step, and this weight reflects the reliability of the corresponding depth positioning point's position in the image. Through weighted averaging, coordinates with higher reliability in the inlier set contribute more to the final result, thereby further improving positioning accuracy.

[0088] Understandably, this embodiment uses the RANSAC algorithm to iteratively filter the initial GPS coordinates generated from multiple observation points, effectively eliminating outliers caused by instantaneous jitter in laser ranging, image motion blur, or GPS signal drift, thus ensuring good consistency of the coordinate data used in the final calculation. Combined with weighted averaging, the final output target GPS coordinates are more accurate and reliable, improving the robustness of the positioning system.

[0089] To better implement the target localization method in UAV images in this embodiment of the invention, based on the target localization method in UAV images, correspondingly, as follows: Figure 7 As shown, this embodiment of the invention also provides a target localization device in a drone image. The target localization device 700 in a drone image includes: The acquisition unit 701 is used to acquire multiple frames of UAV images containing the target collected by the UAV when the UAV observes the same target at multiple observation points, as well as the laser range value and the spatial state data of the UAV corresponding to each frame of the UAV image. Estimation unit 702 is used to input the UAV images of each frame into the monocular depth estimation model to obtain their respective relative depth maps; The conversion unit 703 is used to perform scale conversion on the corresponding relative depth map based on the laser ranging value to obtain an absolute depth map; The calculation unit 704 is used to select multiple depth positioning points from the absolute depth map, and calculate the initial GPS coordinates corresponding to each depth positioning point according to the pixel coordinates and depth value of each depth positioning point and the spatial state data, so as to obtain the initial GPS coordinate set under each observation point. The fusion unit 705 is used to perform fusion optimization on the initial GPS coordinate set under each observation point to obtain the GPS coordinates of the target.

[0090] The target positioning device 700 in the UAV image provided in the above embodiments can realize the technical solution described in the above embodiments of the target positioning method in the UAV image. The specific implementation principle of each module or unit can be found in the corresponding content in the above embodiments of the target positioning method in the UAV image, and will not be repeated here.

[0091] like Figure 8 As shown, the present invention also provides an electronic device 800. The electronic device 800 includes a processor 801, a memory 802, and a display 803. Figure 8 Only some components of the electronic device 800 are shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0092] In some embodiments, processor 801 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 802 or process data, such as the target localization method in UAV images in this invention.

[0093] In some embodiments, processor 801 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 801 may be local or remote. In some embodiments, processor 801 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, intranet, multi-cloud, etc., or any combination thereof.

[0094] In some embodiments, memory 802 may be an internal storage unit of electronic device 800, such as a hard disk or memory of electronic device 800. In other embodiments, memory 802 may also be an external storage device of electronic device 800, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 800.

[0095] Furthermore, the memory 802 may include both internal storage units of the electronic device 800 and external storage devices. The memory 802 is used to store application software and various types of data installed on the electronic device 800.

[0096] In some embodiments, display 808 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 808 is used to display information from electronic device 800 and to display a visual user interface. Components 801-808 of electronic device 800 communicate with each other via a system bus.

[0097] It should be understood that when the processor 801 executes the target localization program in the UAV image stored in the memory 802, in addition to the functions mentioned above, it can also perform other functions, as detailed in the description of the corresponding method embodiments above.

[0098] Furthermore, this embodiment of the invention does not specifically limit the type of electronic device 800 mentioned. Electronic device 800 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the invention, electronic device 800 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0099] Accordingly, this application also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions of the target localization method in the UAV image provided in the above-described method embodiments.

[0100] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0101] The above provides a detailed description of the target localization method, apparatus, electronic device, and storage medium in UAV images provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for target localization in UAV images, characterized in that, include: The system acquires multiple frames of UAV images containing the target collected when the UAV observes the same target from multiple observation points, as well as the laser range value corresponding to each frame of the UAV image and the spatial state data of the UAV. Each frame of the UAV image is input into the monocular depth estimation model to obtain the corresponding relative depth map. Based on the laser ranging value, the corresponding relative depth map is scaled to obtain the absolute depth map; Multiple depth positioning points are selected from the absolute depth map. Based on the pixel coordinates and depth value of each depth positioning point, as well as the spatial state data, the initial GPS coordinates corresponding to each depth positioning point are calculated to obtain the initial GPS coordinate set for each observation point. The initial GPS coordinate set at each observation point is fused and optimized to obtain the GPS coordinates of the target.

2. The target localization method in UAV images according to claim 1, characterized in that, The monocular depth estimation model is the Depth Anything model, which includes an encoder, a neck feature fusion module, and a head depth regression module. The encoder includes a ConvNeXt-V2 backbone network, a feature refinement layer, and a Transformer module connected in series. The feature refinement layer includes cascaded depthwise separable convolutions and GLU activation functions. The Transformer module employs a simplified attention head and sequence length reduction mechanism. The step of inputting each frame of the UAV image into the monocular depth estimation model to obtain the corresponding relative depth map includes: The ConvNeXt-V2 backbone network is used to extract multi-scale features from the UAV images to obtain initial feature maps at different resolutions. The initial feature map is refined by the feature refinement layer to obtain a refined feature map. The Transformer module is used to perform global context modeling on the refined feature map to obtain the encoded feature map. The neck feature fusion module performs hierarchical fusion on the encoded feature map, aligning and fusing low-resolution high semantic features with high-resolution detail features to obtain fused multi-scale features. The head depth regression module performs depth regression prediction on the fused multi-scale features and outputs the corresponding relative depth map.

3. The target localization method in UAV images according to claim 1, characterized in that, Based on the laser ranging value, scale transformation is performed on the corresponding relative depth map, including: The depth value of the central region is extracted from the relative depth map, and the average depth value of the central region is determined to obtain the model average depth. The ratio of the laser ranging value to the average depth of the model is calculated to obtain the scale factor; The absolute depth map is generated by multiplying the depth value of each pixel in the relative depth map by the scale factor.

4. The target localization method in UAV images according to claim 1, characterized in that, The pixel coordinates include pixel horizontal and pixel vertical coordinates, and the spatial state data includes the differential GPS coordinates and attitude angles of the UAV; the step of calculating the initial GPS coordinates corresponding to each depth positioning point based on the pixel coordinates and depth value of each depth positioning point, and the spatial state data, includes: Obtain the intrinsic parameters of the camera mounted on the drone, including the principal point x-coordinate, principal point y-coordinate, lateral focal length, and longitudinal focal length; Subtract the principal point's horizontal coordinate from the horizontal coordinate of each pixel and divide by the horizontal focal length to obtain the normalized horizontal coordinate; subtract the principal point's vertical coordinate from the vertical coordinate of each pixel and divide by the vertical focal length to obtain the normalized vertical coordinate. Multiply the normalized lateral coordinate by the depth value of the corresponding depth positioning point to obtain the rotation matrix of the camera coordinate system from the camera coordinate system to the UAV body coordinate system. Transform the camera lateral coordinate, camera longitudinal coordinate, and camera depth coordinate in the camera coordinate system to the camera lateral coordinate under the UAV body. Multiply the normalized longitudinal coordinate by the depth value of the corresponding depth positioning point to obtain the camera longitudinal coordinate in the camera coordinate system. Use the depth value of the corresponding depth positioning point as the camera depth coordinate in the camera coordinate system. Using the camera's gimbal angle to construct a reference system, the three-dimensional coordinates of the machine body in the machine body coordinate system are obtained; A rotation matrix from the body coordinate system to the navigation coordinate system is constructed using the attitude angle, the three-dimensional coordinates of the body are transformed into the navigation coordinate system, and the position coordinates of the UAV in the navigation coordinate system are added to the position coordinates of the UAV in the navigation coordinate system. The position is obtained by coordinate transformation based on the differential GPS coordinates. The position coordinates in the navigation coordinate system are converted into GPS coordinates to obtain the initial GPS coordinates corresponding to the depth positioning point.

5. The target localization method in UAV images according to claim 1, characterized in that, After obtaining the initial GPS coordinate set for each observation point, the following is also included: Based on the pixel coordinates of each depth positioning point in the absolute depth map, calculate the distance from the depth positioning point to the center of the absolute depth map; The weight of the initial GPS coordinates corresponding to the depth positioning point is determined based on the distance, wherein the distance is negatively correlated with the weight.

6. The target localization method in UAV images according to claim 5, characterized in that, The distance is a Euclidean distance; the steps for determining the weights include: Based on the Euclidean distance, the weight corresponding to the depth positioning point is calculated according to the exponential decay method, wherein the decay coefficient in the exponential decay method ranges from 0.01 to 0.

1.

7. The target localization method in UAV images according to claim 1, characterized in that, The process of fusing and optimizing the initial GPS coordinate set for each observation point includes: The RANSAC algorithm is used to remove outliers from each initial GPS coordinate in the initial GPS coordinate set to obtain the interior point set. The initial GPS coordinates in the set of interior points are weighted and averaged to obtain the GPS coordinates of the target.

8. A target localization device in UAV images, characterized in that, include: The acquisition unit is used to acquire multiple frames of UAV images containing the target collected by the UAV when the UAV observes the same target from multiple observation points, as well as the laser range value corresponding to each frame of the UAV image and the spatial state data of the UAV. The estimation unit is used to input the UAV images of each frame into the monocular depth estimation model to obtain their respective relative depth maps. A conversion unit is used to perform scale conversion on the corresponding relative depth map based on the laser ranging value to obtain an absolute depth map; The calculation unit is used to select multiple depth positioning points from the absolute depth map, and calculate the initial GPS coordinates corresponding to each depth positioning point according to the pixel coordinates and depth value of each depth positioning point and the spatial state data, so as to obtain the initial GPS coordinate set under each observation point. The fusion unit is used to fuse and optimize the initial GPS coordinate set under each observation point to obtain the GPS coordinates of the target.

9. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the target localization method in UAV images according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps in the target localization method in UAV images as described in any one of claims 1 to 7.

Images