Unmanned aerial vehicle positioning method and apparatus, computer device and storage medium

Abstract

The application relates to a UAV positioning method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring visual data and basic data of a UAV; the basic data comprises speed data and positioning data; performing image recognition on the visual data to obtain reference positioning of the UAV in a target area; in the case that the confidence of the reference positioning is greater than a preset data confidence threshold, performing data fusion on the reference positioning and the basic data to obtain target data; in the case that the confidence of the reference positioning is not greater than the preset data confidence threshold, performing data fusion on the speed data and the positioning data to obtain the target data; and determining target positioning of the UAV in the target area according to the target data. The method can accurately position the UAV.

Description

Technical Field

[0001] This application relates to the field of positioning technology, and in particular to a method, apparatus, computer equipment, and storage medium for locating unmanned aerial vehicles (UAVs). Background Technology

[0002] With the widespread application of drone technology in industrial inspection, power facility monitoring, and other fields, the demand for high-precision autonomous positioning of drones in complex indoor environments is becoming increasingly urgent. In special scenarios such as valve halls in power systems, there are harsh conditions such as strong electromagnetic interference, dense metal structures, weak textures, and multipath reflections, which cause traditional Global Navigation Satellite System (GNSS) signals to fail. Single sensors (such as vision, lidar, or inertial measurement units) are easily affected by environmental interference, resulting in low positioning accuracy and serious error accumulation, which cannot meet the requirements of long-term, high-reliability autonomous inspection.

[0003] Traditional UAV positioning primarily relies on GNSS to provide absolute position information, or uses relative positioning methods such as visual odometry and lidar odometry in indoor environments. However, in environments with strong electromagnetic interference, such as valve halls, GNSS signals are severely shielded; single visual sensors are prone to failure under weak textures or changes in lighting; lidar is prone to point cloud degradation in areas with strong metallic reflections; and inertial measurement units accumulate errors over time due to integration drift. Furthermore, existing methods lack multi-sensor adaptive fusion and global error correction mechanisms, making it difficult to achieve stable and continuous centimeter-level positioning in complex dynamic environments.

[0004] However, current positioning methods cannot provide reliable positioning in environments with strong electromagnetic interference or dense metal concentrations. Therefore, there is an urgent need for a method that can enable autonomous positioning of UAVs in harsh environments such as those with strong electromagnetic interference. Summary of the Invention

[0005] Therefore, it is necessary to provide a drone positioning method, device, computer equipment, and storage medium that can accurately locate the above-mentioned technical problems.

[0006] Firstly, this application provides a method for locating a drone, including:

[0007] Acquire visual and basic data from the drone; the basic data includes speed and positioning data.

[0008] Visual data is used for image recognition to obtain the reference positioning of the UAV in the target area;

[0009] If the confidence level of the reference positioning is greater than the preset data confidence threshold, the reference positioning and the basic data are fused to obtain the target data.

[0010] If the confidence level of the reference positioning is not greater than the preset data confidence threshold, the velocity data and positioning data are fused to obtain the target data.

[0011] Based on the target data, determine the target location of the drone in the target area.

[0012] In one embodiment, the visual data includes: video data and point cloud data; image recognition is performed on the visual data to obtain the reference positioning of the UAV in the target area, including:

[0013] Image recognition is performed on the video data to obtain the initial location of the drone in the target area; and,

[0014] Point cloud recognition is performed on the point cloud data to obtain the second location of the UAV in the target area;

[0015] The reference position of the UAV in the target area is determined based on the confidence levels of the first and second positioning.

[0016] In one embodiment, determining the reference position of the UAV in the target area based on the confidence levels of the first and second positions includes:

[0017] If the confidence scores of the first and second locations are both greater than a preset location confidence threshold, the first and second locations are fused to obtain a reference location.

[0018] If either the first or second positioning has a confidence level that is not greater than a preset location confidence threshold, the positioning with a confidence level greater than the preset location confidence threshold is selected as the reference positioning.

[0019] In one embodiment, speed data and positioning data are fused to obtain target data, including:

[0020] Based on the current speed data and the drone's historical positioning from the previous moment, determine the drone's relative movement distance within the target area;

[0021] Based on the relative travel distance and historical positioning, the speed and location of the UAV in the target area are determined;

[0022] The velocity positioning and positioning data are fused to obtain the target data.

[0023] 5. In one embodiment, the method further includes:

[0024] Determine the location differences between the target location of the UAV and its historical locations;

[0025] If there are historical locations with a positioning difference less than a preset difference threshold, the target positioning and historical positioning are fused to update the target node on the UAV's flight path in the target area; the target node is the location node in the target area where the historical positioning was located.

[0026] In one embodiment, the method further includes:

[0027] The visual data corresponding to historical positioning and the visual data corresponding to target positioning are fused to obtain fused visual data.

[0028] Based on the fused visual data, update the partial map of the target node's location in the target area; the regional map is used to represent the geographic information of the target area.

[0029] Secondly, this application also provides a drone positioning device, comprising:

[0030] The acquisition module is used to acquire the drone's visual data and basic data; the basic data includes: speed data and positioning data;

[0031] The recognition module is used to perform image recognition on visual data to obtain the reference positioning of the UAV in the target area;

[0032] The determination module is used to perform data fusion on the reference positioning and basic data to obtain target data when the confidence level of the reference positioning is greater than the preset data confidence threshold; and to perform data fusion on the velocity data and positioning data to obtain target data when the confidence level of the reference positioning is not greater than the preset data confidence threshold.

[0033] The positioning module is used to determine the target location of the UAV in the target area based on the target data.

[0034] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0035] Acquire visual and basic data from the drone; the basic data includes speed and positioning data.

[0036] Visual data is used for image recognition to obtain the reference positioning of the UAV in the target area;

[0037] If the confidence level of the reference positioning is greater than the preset data confidence threshold, the reference positioning and the basic data are fused to obtain the target data.

[0038] If the confidence level of the reference positioning is not greater than the preset data confidence threshold, the velocity data and positioning data are fused to obtain the target data.

[0039] Based on the target data, determine the target location of the drone in the target area.

[0040] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0041] Acquire visual and basic data from the drone; the basic data includes speed and positioning data.

[0042] Visual data is used for image recognition to obtain the reference positioning of the UAV in the target area;

[0043] If the confidence level of the reference positioning is greater than the preset data confidence threshold, the reference positioning and the basic data are fused to obtain the target data.

[0044] If the confidence level of the reference positioning is not greater than the preset data confidence threshold, the velocity data and positioning data are fused to obtain the target data.

[0045] Based on the target data, determine the target location of the drone in the target area.

[0046] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0047] Acquire visual and basic data from the drone; the basic data includes speed and positioning data.

[0048] Visual data is used for image recognition to obtain the reference positioning of the UAV in the target area;

[0049] If the confidence level of the reference positioning is greater than the preset data confidence threshold, the reference positioning and the basic data are fused to obtain the target data.

[0050] If the confidence level of the reference positioning is not greater than the preset data confidence threshold, the velocity data and positioning data are fused to obtain the target data.

[0051] Based on the target data, determine the target location of the drone in the target area.

[0052] The aforementioned UAV positioning method, device, computer equipment, and storage medium utilize visual data to acquire a reference positioning with environmental awareness capabilities as the core observation source. Subsequently, a confidence judgment mechanism is introduced to evaluate the reliability of the reference positioning in real time: when the visual information is reliable, the system prioritizes the fusion of visual, inertial, and satellite data from multiple sources, fully leveraging the high precision advantage of vision in texture-rich areas, while suppressing errors from a single sensor through multi-source complementarity; when the reliability of the vision decreases due to interference, weak texture, or other factors, it automatically switches to a trajectory recursion mode based on velocity estimation and historical positioning, avoiding the introduction of positioning deviations by erroneous visual information, ensuring the continuity of the positioning process, and further improving the stability, reliability, and accuracy of UAV positioning in complex scenarios such as strong electromagnetic interference and dynamic occlusion. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is an application environment diagram of a UAV positioning method provided in this embodiment;

[0055] Figure 2A This is a flowchart illustrating a drone positioning method provided in this embodiment;

[0056] Figure 2B This embodiment provides a block diagram of a drone positioning system.

[0057] Figure 3 This is a flowchart illustrating a reference positioning determination step provided in this embodiment;

[0058] Figure 4 This is a flowchart illustrating a target data determination step provided in this embodiment;

[0059] Figure 5A This is a flowchart illustrating a target node update step provided in this embodiment;

[0060] Figure 5B This embodiment provides a visual bag-of-words flowchart;

[0061] Figure 6 This is a flowchart illustrating a geographic information update step provided in this embodiment;

[0062] Figure 7 This is a structural block diagram of a drone positioning device provided in this embodiment;

[0063] Figure 8 This is an internal structural diagram of a computer device provided in this embodiment. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0065] The UAV positioning method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. The computer device acquires the drone's visual data and basic data; the basic data includes speed data and positioning data; image recognition is performed on the visual data to obtain the drone's reference positioning in the target area; if the confidence level of the reference positioning is greater than a preset data confidence threshold, the reference positioning and basic data are fused to obtain target data; if the confidence level of the reference positioning is not greater than the preset data confidence threshold, the speed data and positioning data are fused to obtain target data; based on the target data, the target positioning of the drone in the target area is determined. The computer device can be either a terminal or a server. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0066] In one exemplary embodiment, such as Figure 2A As shown, a method for locating unmanned aerial vehicles (UAVs) is provided, which can be applied to... Figure 1 Taking a computer device as an example, the explanation includes the following steps S201 to S204. Wherein:

[0067] The S201 acquires visual and basic data from the drone.

[0068] Visual data refers to image or video information acquired by visual sensors (such as cameras) mounted on drones, as well as point cloud data acquired by sensors such as LiDAR (Light Detection and Ranging). This data is used for environmental perception, feature extraction, and localization reference. Visual data includes video data and point cloud data; video data consists of continuous image sequences captured by cameras, used for image recognition and scene understanding. Point cloud data is a set of three-dimensional spatial points generated by sensors such as LiDAR, where each point represents a specific location in the environment, used to construct a three-dimensional model of the environment.

[0069] Basic data refers to the fundamental motion and environmental information acquired by the UAV during flight. This data is typically used as input to positioning algorithms to assist visual data in achieving more accurate positioning. Basic data includes velocity data and positioning data. Velocity data refers to the UAV's speed relative to a reference frame (such as the ground) at a given moment, including linear and angular velocity information. Positioning data represents the UAV's spatial location information at a given moment and is either a direct output or an intermediate result of the Global Navigation Satellite System (GNSS). For example, the GNSS could be the Global Positioning System (GPS).

[0070] In some embodiments, continuous image sequences captured by a camera are acquired to obtain video data of the UAV; a three-dimensional spatial point set generated by sensors such as LiDAR is acquired, where each point represents a specific location in the environment, to obtain point cloud data of the UAV. The Inertial Measurement Unit (IMU) calculates velocity information by measuring the three-axis angular velocity and three-axis acceleration of the carrier to obtain velocity data of the UAV. Positioning data output by the Huqiu satellite positioning system is also included.

[0071] S202 performs image recognition on the visual data to obtain the reference positioning of the UAV in the target area.

[0072] Reference positioning is intermediate location information derived from specific sensor data or algorithm processing results, used to assist in the final positioning.

[0073] In some embodiments, image recognition is performed on visual data based on an image recognition model to obtain the reference positioning of the UAV in the target area.

[0074] S203 If the confidence level of the reference positioning is greater than the preset data confidence threshold, the reference positioning and the basic data are fused to obtain the target data; if the confidence level of the reference positioning is not greater than the preset data confidence threshold, the speed data and the positioning data are fused to obtain the target data.

[0075] In this context, target data typically refers to the precise state information about a target (such as a drone, robot, or monitoring object) that the system ultimately hopes to obtain or output.

[0076] In one optional embodiment, when the confidence level of the reference positioning is greater than a preset data confidence threshold, a joint state vector is formed with the reference positioning and basic data through time synchronization. .in, Represents the attitude rotation matrix. For location, For linear velocity, It is a gyroscope. To achieve zero bias in the accelerometer; based on the preset factor graph optimization function, the target data is obtained according to the joint state vector.

[0077] For example, the factor graph optimization function is shown in the following formula (1):

[0078] (1)

[0079] in, Constraints characterizing velocity data Constraints representing point cloud data Constraints characterizing video data Constraints characterizing the location data For the residual vector, Let be the noise covariance matrix. This is solved by solving a nonlinear least squares optimization problem. The optimal state estimate can then be obtained. .

[0080] It should be noted that, as Figure 2BThe UAV positioning block diagram shown addresses the challenges of strong electromagnetic interference, strong metal reflection, and weak texture in the indoor valve hall. This project employs a tightly coupled multi-sensor fusion of camera, LiDAR, and IMU (GPS is only used as an auxiliary in a few available locations and is not relied upon) to achieve high-precision and robust UAV positioning within the valve hall. The positioning technology route includes four modalities: camera, LiDAR, IMU, and GPS. The overall system framework uses a tightly coupled Error State Iterative Kalman Filter (ESIKF) as the core of data fusion. The filter's state vector contains key state variables such as the UAV's position, attitude, velocity, and IMU zero bias. Simultaneously, it estimates the extrinsic calibration parameters between the camera and IMU, and between the LiDAR and IMU, online to ensure that the system maintains calibration accuracy even under conditions of electromagnetic interference, mechanical vibration, and airflow disturbance within the valve hall. The visual observation model is constructed based on a semi-direct method. Pose constraints are provided by minimizing the photometric error of image patches; geometric constraints are provided by the matching error between the point cloud features of the LiDAR observation model and the local map. Tightly coupled architectures can make fuller use of sensor information and can still operate even when there are fewer than four GNSS satellites, significantly improving system availability and accuracy, but they also bring higher computational complexity and implementation difficulty. At the same time, GPS information is fused with the SLAM system in a loosely coupled manner, thereby ensuring stable autonomous flight operations even when GPS signals are weak or unavailable.

[0081] In one optional embodiment, when the confidence level of the reference positioning is not greater than a preset data confidence threshold, the speed data and positioning data are fused based on a preset fusion model to obtain the target data.

[0082] It should be noted that, under normal operating conditions, the GPS-IMU subsystem achieves coarse state estimation by periodically updating the IMU integration results. While the IMU can provide high-precision attitude and velocity change information in a short time, its inherent integration drift leads to accumulated errors over a long period. GPS, although updated less frequently, offers global consistency and absolute reference capability. The fusion of the two effectively complements each other: GPS suppresses the IMU's zero-bias drift, while the IMU provides high dynamic response within the GPS signal gaps, enabling the system to obtain a smooth and stable global pose estimation. When environmental factors within the valve hall cause strong interference to the visual or lidar sensors (e.g., strong electromagnetic fields, metal reflections, partial obstruction, or uneven lighting), the system of this invention possesses an adaptive mode switching mechanism. The system dynamically evaluates the fusion weights based on the confidence level of the sensor data: if both vision and radar are working properly, the GPS-IMU module serves as a weak constraint factor, providing a long-term consistent benchmark for the factor map; if vision or radar fails, it automatically switches to LiDAR-IMU mode or GPS-IMU mode; when both fail simultaneously, the system degenerates into GPS-IMU navigation mode, relying solely on GPS and inertial information to maintain track estimation.

[0083] It should be noted that this multimodal adaptive design ensures that the UAV can maintain continuous navigation even in indoor environments with strong electromagnetic interference, dense metal, and complex spatial structures, such as valve halls, without interrupting state estimation due to local sensor failure. The GPS-IMU subsystem not only enhances the global consistency and robustness of the entire system, but also provides reliable initial values ​​and constraints for subsequent nonlinear optimization, enabling the system to achieve stable positioning and mapping functions even under such interference.

[0084] S204 determines the target location of the UAV in the target area based on the target data.

[0085] In some embodiments, the target data is mapped onto a regional map corresponding to the target area to determine the target location of the UAV in the target area.

[0086] In the above embodiments, a reference positioning with environmental awareness is obtained using visual data as the core observation source. Subsequently, a confidence judgment mechanism is introduced to evaluate the reliability of the reference positioning in real time: when the visual information is reliable, the system prioritizes the fusion of visual, inertial, and satellite data to fully leverage the high precision advantage of vision in texture-rich areas, while suppressing the error of a single sensor through multi-source complementarity; when the reliability of the vision decreases due to factors such as interference or weak texture, the system automatically switches to a trajectory recursion mode based on velocity estimation and historical positioning to avoid introducing positioning deviations with erroneous visual information, ensuring the continuity of the positioning process, and further improving the stability, reliability, and accuracy of UAV positioning in complex scenarios such as strong electromagnetic interference and dynamic occlusion.

[0087] Figure 3 This is a flowchart illustrating the reference positioning determination step in one embodiment. In this embodiment, the visual data includes video data and point cloud data. Based on this, the steps for performing image recognition on the visual data to obtain the reference positioning of the UAV in the target area, as described in the above embodiment, are refined, including the following steps:

[0088] S301 performs image recognition on video data to obtain the first location of the UAV in the target area; and performs point cloud recognition on point cloud data to obtain the second location of the UAV in the target area.

[0089] In some embodiments, based on an image recognition model, image recognition is performed on video data to obtain a first location of the UAV in the target area; and based on a point cloud recognition model, point cloud recognition is performed on point cloud data to obtain a second location of the UAV in the target area.

[0090] It should be noted that in this embodiment, an improved Iterative Closest Point (ICP) algorithm is used to match frames with the local map during the determination of the second positioning process. Considering the structural characteristics of the valve hall environment, normal vector constraints and an adaptive distance threshold adjustment mechanism are added to improve matching accuracy and robustness. To correct for accumulated odometer errors, the matching results are incorporated into a graph optimization framework in the form of constraints.

[0091] S302 determines the reference position of the UAV in the target area based on the confidence levels of the first and second positioning.

[0092] In one alternative embodiment, the location with higher confidence is selected from the confidence levels of the first location and the second location, and used as the reference location for the UAV in the target area.

[0093] In one optional embodiment, if the confidence levels of both the first and second locations are greater than a preset location confidence threshold, the first and second locations are fused to obtain a reference location; if either the first or second location has a confidence level less than the preset location confidence threshold, the location with a confidence level greater than the preset location confidence threshold is selected as the reference location.

[0094] For example, if either the first location or the second location has a confidence level that is not greater than a preset location confidence threshold, the location with a confidence level greater than the preset location confidence threshold is selected as the reference location.

[0095] For example, when the confidence levels of the first and second positioning are both greater than a preset position confidence threshold, a first constraint term for the first positioning and a second constraint term for the second positioning are determined (as shown in Formula 3); based on the factor graph optimization function, the first and second constraint terms are constrained and fused to obtain the reference positioning of the UAV in the target area.

[0096] For example, the first constraint term of the first localization uses the semi-direct visual odometry (SVO) algorithm, which achieves more efficient pose estimation by minimizing image brightness error rather than feature matching error. The error term in the first constraint term is defined as shown in the following formula (2):

[0097] (2)

[0098] in, Let be the grayscale value of the k-th frame image. and These are the projection and back-projection functions, respectively, where d is the pixel depth.

[0099] For example, the second constraint term of the second positioning is optimized based on the point-to-surface residual model, and the error term in the second constraint term is defined as shown in the following formula (3):

[0100] (3)

[0101] in, Let be the coordinates of a point in the point cloud, and n be the normal vector of the corresponding plane. Let be the coordinates of a point on the plane. This residual directly reflects the geometric consistency between the point cloud and the map.

[0102] It should be noted that by using the high-precision point cloud map generated from the point cloud data as a depth prior to the video data, the system can significantly reduce triangulation errors and improve the matching stability of feature points in low-texture regions. The fused visual constraints are directly introduced into the factor graph framework and solved together with the constraints from LiDAR and IMU, achieving true tightly coupled fusion.

[0103] It should be noted that, to meet the real-time positioning requirements of UAVs, the system introduces a sliding window optimization mechanism, optimizing only the most recent N keyframe states. Historical keyframes outside the window retain their statistical information through marginalization, thus maintaining global consistency in the estimation. The system optimization employs an incremental nonlinear solver (such as iSAM2) to achieve the following iterative updates: Where H is the Jacobian matrix and r is the residual vector. This represents the state update operator on a Lie group.

[0104] In the above embodiments, by utilizing the rich visual information of video data and the precise spatial three-dimensional information of point cloud data, the complementary use of multi-source data can effectively improve the comprehensiveness and accuracy of positioning. The reference positioning is determined based on the confidence level, which can flexibly adapt to different environmental conditions and data quality. Even when some data is interfered with or the accuracy is limited, the reliability of the positioning results can still be guaranteed, providing strong support for the stable and precise operation of UAVs in the target area.

[0105] Figure 4 This is a flowchart illustrating the target data determination steps in one embodiment. This embodiment refines the steps of fusing speed data and positioning data to obtain target data from the above embodiments, including the following steps:

[0106] Based on the current speed data and the drone's historical positioning from the previous moment, S401 determines the relative movement distance of the drone in the target area.

[0107] Among them, the relative movement distance reflects the change in the spatial position of the UAV within the time interval, and is an important basic parameter for subsequent speed positioning and data fusion to obtain target data.

[0108] In some embodiments, based on the distance determination function shown in the following formula (4), the relative movement distance of the UAV in the target area is determined according to the speed data at the current moment and the historical positioning of the UAV at the previous moment.

[0109] (4)

[0110] in, Characterizes the velocity data between two keyframes i and j. For linear velocity, It is a gyroscope. These quantities are used as IMU factors in subsequent factor graph optimization to connect motion constraints between consecutive keyframes, thereby maintaining system accuracy while ensuring computational efficiency.

[0111] S402 determines the speed and location of the UAV in the target area based on the relative travel distance and historical positioning.

[0112] In some embodiments, the relative movement distance is superimposed along the direction of the drone's movement based on the drone's historical positioning in the target area to determine the drone's speed positioning in the target area.

[0113] S403 fuses velocity positioning and positioning data to obtain target data.

[0114] In some embodiments, velocity positioning and positioning data representing positioning are fused to obtain target data.

[0115] In the above embodiments, the relative movement distance is calculated based on the current speed data and the historical positioning at the previous moment, thereby determining the speed positioning. Finally, the speed positioning and positioning data are fused to obtain the target data. This can make full use of speed information to achieve continuous calculation of relative positioning and reduce the dependence on a single absolute positioning method. By fusing with positioning data and combining the advantages of multi-source information, the accuracy and reliability of target data are effectively improved, and the positioning stability and adaptability of UAVs in complex environments are enhanced.

[0116] Figure 5A This is a flowchart illustrating the target node update step in one embodiment. This embodiment refines the steps of the above embodiment, including the following steps:

[0117] S501 determines the positioning differences between the UAV's target positioning and its historical positioning.

[0118] In some embodiments, for each historical location, the location difference between the UAV's target location in the target area and each historical location is determined to determine whether the UAV passed through the same location in the target area.

[0119] For example, the positioning difference between the target positioning of the UAV and each historical positioning is determined based on the following formula (5).

[0120] (5)

[0121] Where v1 represents the target location and v2 represents the historical location.

[0122] It should be noted that, as Figure 5BThe visual bag-of-words flowchart shown in this embodiment first uses a pre-trained convolutional neural network to extract global descriptors and local keypoint features of the image. Global descriptors are obtained through the fully connected layers of the network, forming a compact image representation vector; local features are extracted from the intermediate layers of the network, preserving rich spatial structure information. These deep features are more robust to changes in illumination, viewpoint, and partial occlusion, effectively addressing challenges such as uneven lighting and equipment occlusion within the valve hall. Based on this, the system constructs an incremental visual bag-of-words model, forming visual vocabulary through clustering deep features and employing a TF-IDF weighted mechanism to evaluate the importance of the vocabulary. During similarity calculation, the system simultaneously considers the cosine similarity of the global descriptors and the Euclidean distance of the bag-of-words vectors, obtaining the final similarity score through weighted fusion. To improve retrieval efficiency, the system employs a fast approximate nearest neighbor search algorithm, capable of quickly finding candidate loop closure frames in a large-scale image database. The system also introduces temporal consistency checks and geometric verification mechanisms to analyze the similarity of consecutive frames, avoiding instantaneous mismatches and ensuring the reliability of loop closure detection.

[0123] If there are historical locations with a positioning difference less than a preset difference threshold, S502 performs positioning fusion on the target positioning and the historical positioning to update the target node on the UAV's flight trajectory in the target area.

[0124] The target node is the location node in the target area where the historical positioning was located.

[0125] In some embodiments, if there are historical locations with a positioning difference less than a preset difference threshold, it proves that the UAV has passed through the same location in the target area and has looped back; the target positioning and historical positioning can be fused to update the target node on the UAV's flight trajectory in the target area; the target node is the location node in the target area where the historical positioning was located.

[0126] In the above embodiments, by determining the difference between the UAV target location and each historical location, and fusing the target location and historical locations when the difference is less than a preset threshold to update the target node on the flight trajectory, this method can effectively improve the continuity and smoothness of the trajectory and avoid trajectory jumps caused by single positioning errors. At the same time, by selecting reliable historical locations for fusion, the robustness of the positioning results can be enhanced, the impact of outliers on the trajectory can be reduced, thereby providing the UAV with a more stable and accurate flight path reference.

[0127] Figure 6 This is a flowchart illustrating the geographic information update steps in one embodiment. This embodiment refines the steps of the above embodiment, including the following steps:

[0128] S601 performs fusion processing on the visual data corresponding to historical positioning and the visual data corresponding to target positioning to obtain fused visual data.

[0129] In some embodiments, the consistency between the visual data corresponding to the historical location and the visual data corresponding to the target location is verified based on the following formula (6); if the verification is successful, the visual data corresponding to the historical location and the visual data corresponding to the target location are fused to obtain fused visual data.

[0130] For example, the fundamental matrix or homography matrix between the current frame and candidate frames is calculated through geometric verification. False matching points are eliminated using the RANSAC algorithm to ensure the correctness of loop closure relationships. After successful verification, the system adds loop closure constraints to the pose graph, constructing an optimization problem that includes all keyframe nodes and loop closure edges.

[0131] (6)

[0132] Where m represents the minimum number of samples, p represents the probability that the algorithm reaches the target, and w represents the proportion of matching points. This formula is used to estimate the number of iterations of the RANSAC algorithm.

[0133] S602 updates the partial map of the target node's location in the target area based on the fused visual data.

[0134] Among them, the regional map is used to represent the geographical information of the target area.

[0135] In some embodiments, a graph-based optimization method is employed, using an optimization library to solve for globally consistent motion trajectories and map point positions. During optimization, the system adaptively adjusts the weights of loop closure constraints, assigning greater weight to loops with high confidence and appropriately reducing the impact of loops with uncertainty. The system also employs a sliding window optimization mechanism to control computational complexity while ensuring accuracy and meeting real-time requirements. After optimization, the system updates the poses and map point coordinates of all keyframes and performs consistency propagation for subsequent positioning processes, ensuring global consistency across the entire system. This optimization process not only corrects the current position error but also significantly improves the accuracy of historical trajectories, providing a more accurate map foundation for subsequent navigation.

[0136] In the above embodiments, the visual data corresponding to historical positioning and target positioning are fused and the map of a portion of the area is updated. The beneficial effect is that by integrating multi-temporal visual information, the local error of a single positioning can be effectively corrected, and the geometric accuracy and texture consistency of the map can be improved. At the same time, the dynamically updated area map can reflect environmental changes in real time (such as obstacle movement and structural adjustment), providing the UAV with a more accurate environmental perception capability, thereby enhancing the safety and adaptability of path planning.

[0137] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0138] Based on the same inventive concept, this application also provides a drone positioning device for implementing the drone positioning method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more drone positioning device embodiments provided below can be found in the limitations of the drone positioning method described above, and will not be repeated here.

[0139] In one exemplary embodiment, such as Figure 7 As shown, a drone positioning device is provided, comprising:

[0140] The acquisition module 701 is used to acquire visual data and basic data of the UAV; the basic data includes: speed data and positioning data;

[0141] The recognition module 702 is used to perform image recognition on visual data to obtain the reference positioning of the UAV in the target area;

[0142] The determination module 703 is used to perform data fusion on the reference positioning and the basic data to obtain the target data when the confidence level of the reference positioning is greater than the preset data confidence threshold; and to perform data fusion on the velocity data and the positioning data to obtain the target data when the confidence level of the reference positioning is not greater than the preset data confidence threshold.

[0143] The positioning module 704 is used to determine the target location of the UAV in the target area based on the target data.

[0144] In some embodiments, the identification module 702 is further configured to perform image recognition on video data to obtain a first location of the UAV in the target area; and to perform point cloud recognition on point cloud data to obtain a second location of the UAV in the target area; and to determine a reference location of the UAV in the target area based on the confidence level of the first location and the confidence level of the second location.

[0145] In some embodiments, the identification module 702 is further configured to perform data fusion on the first location and the second location to obtain a reference location when both the confidence level of the first location and the confidence level of the second location are greater than a preset location confidence threshold; and to select the location with a confidence level greater than the preset location confidence threshold as the reference location when either the first location or the second location has a confidence level not greater than the preset location confidence threshold.

[0146] In some embodiments, the determination module 703 is further configured to determine the relative movement distance of the UAV in the target area based on the speed data at the current moment and the historical positioning of the UAV at the previous moment; determine the speed positioning of the UAV in the target area based on the relative movement distance and the historical positioning; and perform data fusion on the speed positioning and positioning data to obtain target data.

[0147] In some embodiments, the UAV positioning device further includes: a node update module, used to determine the positioning difference between the UAV's target positioning and each historical positioning; if there is a historical positioning with a positioning difference less than a preset difference threshold, then the target positioning and the historical positioning are fused to update the target node on the UAV's flight trajectory in the target area; the target node is the location node in the target area where the historical positioning is located.

[0148] In some embodiments, the UAV positioning device further includes: a map update module, configured to perform fusion processing on the visual data corresponding to historical positioning and the visual data corresponding to target positioning to obtain fused visual data; and update a partial area map of the location of the target node in the target area based on the fused visual data; the area map is used to represent the geographic information of the target area.

[0149] Each module in the aforementioned UAV positioning device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0150] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When the computer program is executed by the processor, it implements a UAV positioning method.

[0151] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0152] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0153] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0154] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0155] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0156] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0157] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0158] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for locating unmanned aerial vehicles (UAVs), characterized in that, The method includes: Acquire visual data and basic data of the drone; the basic data includes: speed data and positioning data; Image recognition is performed on the visual data to obtain the reference positioning of the UAV in the target area; If the confidence level of the reference positioning is greater than a preset data confidence threshold, the reference positioning and the basic data are fused to obtain the target data. If the confidence level of the reference positioning is not greater than a preset data confidence threshold, the velocity data and positioning data are fused to obtain the target data. Based on the target data, the target location of the UAV in the target area is determined.

2. The method according to claim 1, characterized in that, The visual data includes: video data and point cloud data; the step of performing image recognition on the visual data to obtain the reference positioning of the UAV in the target area includes: Image recognition is performed on the video data to obtain the first location of the drone in the target area; and, Point cloud recognition is performed on the point cloud data to obtain the second location of the UAV in the target area; The reference position of the UAV in the target area is determined based on the confidence levels of the first and second positioning.

3. The method according to claim 2, characterized in that, Determining the reference position of the UAV in the target area based on the confidence levels of the first and second positions includes: If the confidence levels of the first location and the second location are both greater than a preset location confidence threshold, the first location and the second location are fused to obtain a reference location. If either the first location or the second location has a confidence level that is not greater than a preset location confidence threshold, the location with a confidence level greater than the preset location confidence threshold is selected as the reference location.

4. The method according to claim 1, characterized in that, The process of fusing the speed data and positioning data to obtain target data includes: Based on the speed data at the current moment and the historical positioning of the UAV at the previous moment, the relative movement distance of the UAV in the target area is determined; Based on the relative travel distance and the historical positioning, the speed and positioning of the UAV in the target area are determined; The speed positioning and positioning data are fused to obtain the target data.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Determine the positioning differences between the target positioning of the UAV and its historical positioning; If there is a historical location with a location difference less than a preset difference threshold, the target location and the historical location are fused to update the target node on the flight trajectory of the UAV in the target area; the target node is the location node in the target area where the historical location is located.

6. The method according to claim 5, characterized in that, The method further includes: The visual data corresponding to the historical location and the visual data corresponding to the target location are fused to obtain fused visual data. Based on the fused visual data, update the partial area map of the target node's location in the target area; the area map is used to represent the geographic information of the target area.

7. A drone positioning device, characterized in that, The device includes: The acquisition module is used to acquire visual data and basic data of the UAV; the basic data includes: speed data and positioning data; The recognition module is used to perform image recognition on the visual data to obtain the reference positioning of the UAV in the target area; The determination module is used to perform data fusion on the reference positioning and the basic data to obtain target data when the confidence level of the reference positioning is greater than a preset data confidence threshold; and to perform data fusion on the speed data and the positioning data to obtain target data when the confidence level of the reference positioning is not greater than the preset data confidence threshold. The positioning module is used to determine the target location of the UAV in the target area based on the target data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images