Satellite positioning and visual positioning fusion verification unmanned aerial vehicle positioning method and system

Abstract

The application discloses a kind of unmanned plane positioning method and system of satellite positioning and visual positioning fusion check, method includes: preacquiring the image of several with high-precision positioning information in the fixed route of unmanned plane flight, fixed route is divided into several intervals according to first length, the image feature information corresponding to each interval and two end extension second length size route area is acquired, the visual feature library of segmented storage is obtained;Unmanned plane is in real-time flight process on route, the data output by satellite positioning equipment on unmanned plane, airborne downward-looking camera is acquired in real time, is combined with visual feature library, after signal quality evaluation, mode decision, visual matching and fusion calculation, unmanned plane positioning data is output, high-quality positioning data in real-time flight process is fed back to visual feature library and is updated.The present application is positioned by hierarchical strategy, sequence consistency visual positioning and segmented feature library design, to ensure that unmanned plane is continuously, stably, efficiently positioned on fixed route.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) positioning technology, and in particular to a UAV positioning method and system that integrates satellite positioning and visual positioning for verification. Background Technology

[0002] Unmanned aerial vehicles (UAVs) have the advantages of being able to adapt to complex terrain environments, flying at high speeds, and being flexible in deployment. They have been successfully applied in many fields such as agricultural plant protection, power line inspection, emergency rescue, geographic surveying, express logistics, and military operations.

[0003] Current UAV positioning technologies primarily rely on Global Navigation Satellite System (GNSS) positioning, visual positioning, or a combination of these methods. However, satellite positioning suffers from signal obstruction and interference spoofing in environments with dense buildings or electromagnetic interference, leading to decreased accuracy or even failure. Visual positioning, which often uses satellite maps, suffers from drawbacks such as significant perspective differences and inaccuracies in areas lacking texture; furthermore, the construction and maintenance of satellite maps are costly.

[0004] Furthermore, existing fusion positioning methods still have shortcomings in the following aspects: First, they lack a graded evaluation mechanism for satellite positioning signal quality, making it impossible to fully utilize computing resources when the signal is good and to switch positioning strategies in a timely manner when the signal deteriorates; second, visual positioning algorithms are highly complex and difficult to run in real time on embedded platforms; third, large-scale flight path satellite maps are difficult to use on memory-constrained devices; and fourth, maintaining the timeliness of satellite maps is difficult, and map failure due to changes in the flight path environment leads to positioning failure. Therefore, there is an urgent need for a UAV positioning solution that can adapt to fixed flight paths, has low computational complexity, low memory footprint, and can be automatically updated. Summary of the Invention

[0005] Technical objective: To address the shortcomings of existing technologies, this invention discloses a UAV positioning method and system that integrates satellite positioning and visual positioning verification. This solves the problems of limited adaptability to complex scenarios, high computational complexity, and difficulty in managing feature libraries in existing UAV positioning systems. Through hierarchical positioning strategies, sequence-consistent visual positioning, and segmented feature library design, it ensures continuous, stable, and efficient positioning of UAVs on fixed routes, improves the environmental adaptability of UAVs, and reduces computational resource consumption.

[0006] Technical Solution: To achieve the above-mentioned technical objectives, this invention proposes a UAV positioning method based on the fusion verification of satellite positioning and visual positioning. This method includes the following steps:

[0007] Step S1: Pre-acquire several images with positioning information within the fixed flight path of the UAV, divide the fixed flight path into several intervals according to the first length, acquire the image feature information corresponding to each interval and the second length of the flight path area at both ends, and obtain a segmented visual feature library.

[0008] Step S2: During the real-time flight of the UAV along the flight path, the data output by the satellite positioning device and the airborne downward-looking camera on the UAV is acquired in real time. Combined with the visual feature library in step S1, after signal quality assessment, mode decision, visual matching and fusion calculation, the UAV positioning data is output and the positioning data during real-time flight is fed back to the visual feature library for updating.

[0009] Furthermore, step S1 includes the following steps:

[0010] Step S1-1: Use a drone equipped with a downward-looking camera of a fixed model, focal length and resolution to collect video with positioning information within a fixed flight path.

[0011] Step S1-2: Perform time-synchronized sampling on the video; sample at consecutive video frame intervals to obtain several image frames, and align them with the GNSS timestamp based on the image frame timestamp; for image frames that do not directly correspond to GNSS positioning data, use linear interpolation of adjacent GNSS positioning points or nearest neighbor mapping to complete the position information.

[0012] Steps S1-3: Obtain image feature information of all image frames; preprocess all image frames and extract corresponding image feature information, including positional features and visual features;

[0013] Steps S1-4: Segment and store all image frames and their image feature information to establish a visual feature library; divide the fixed flight path into several intervals according to a fixed first length, and extend the second length between each interval to form an overlapping area. Store the image frames and their image feature information corresponding to the flight path area of ​​each interval separately to obtain the segmented visual feature library.

[0014] Furthermore, the real-time data acquisition and visual positioning calculation in step S2 are as follows:

[0015] Step S2-1, Real-time data acquisition: During the real-time flight of the UAV on the flight path, satellite positioning data is obtained by receiving satellite signals through the onboard satellite positioning equipment on the UAV, and images of the scene where the UAV is located are collected through the onboard downward-looking camera.

[0016] Step S2-2, Visual Positioning Calculation: The satellite signal quality is assessed, and a mode decision is made based on the assessment results to determine the positioning mode; after visual matching and fusion calculation in the positioning mode, the UAV positioning data is output.

[0017] Furthermore, the satellite signal quality is assessed, and a mode decision is made based on the assessment results to determine the positioning mode, including setting an excellent threshold as follows: and And the number of satellites is ≥8, among which, Here, C is the position accuracy factor, C is the carrier-to-noise ratio, and N0 is the number of visible satellites; the good threshold is: and ; Obtain the current satellite signal quality, when the signal quality is: and When the number of satellites is ≥8 and remains so for 3 seconds, the system enters satellite-dominated mode; when the signal quality is: and When the signal quality is: and If satellite signal is lost, it will enter a pure visual positioning mode.

[0018] Furthermore, dual threshold control is adopted for positioning mode switching in mode decision-making.

[0019] Furthermore, there are several modes: Satellite-dominated mode: Satellite positioning data is directly used as the drone's positioning result, with the positioning frequency matching the output frequency of the satellite positioning equipment; Fusion mode: Satellite positioning data and visual positioning data are fused using methods such as Kalman filtering to calculate the drone's positioning data; Pure visual positioning mode: Visual positioning data is directly used as the drone's positioning result.

[0020] Furthermore, visual positioning data is directly used as the UAV positioning result output, including: calculating the comprehensive confidence score; if the comprehensive confidence score is greater than or equal to the confidence score threshold, the satellite coordinates stored in the image are output as the visual positioning result.

[0021] The formula for calculating the overall confidence level includes:

[0022]

[0023]

[0024]

[0025]

[0026]

[0027]

[0028] in, To assess the overall confidence level, These are, respectively, similarity score weight, temporal consistency weight, and geometric rationality weight; For geometric rationality confidence level; For the theoretical maximum displacement, This represents the geometric distance between the candidate location and the localization result of the previous frame. For sequence consistency score, This is a descriptor for the candidate image at its corresponding temporal position in the library. For the current image descriptor, For candidate image descriptors in the feature library, For the first time sequence window Frame descriptor, For timing window, The normalized cosine similarity. The cosine similarity function is calculated within the range [-1, 1]. The normalized similarity function is calculated, with a range of [0,1].

[0029] Furthermore, S2 also includes safety decisions. In satellite-dominated mode and fusion mode, when the deviation between satellite positioning and visual positioning exceeds a preset threshold, or the confidence level of the visual positioning result is lower than a preset threshold, a mode switch or safety alarm is triggered. When both satellite positioning and visual positioning fail and continue for more than a preset time threshold, the UAV performs safety measures such as deceleration, stopping, hovering, or landing.

[0030] Furthermore, updating the visual feature library in step S2 includes: analyzing image data collected during daily flights in satellite-dominated mode; if the flight path range covered by the image data contains a complete range divided according to the method described in steps S1-4, then the image data within the range of the divided range is used to update the feature library file of the corresponding divided range, thereby realizing dynamic maintenance and timeliness guarantee of the feature library.

[0031] Furthermore, this invention proposes a UAV positioning system for fusion verification of satellite positioning and visual positioning. This system includes an airborne computing device and an airborne satellite positioning device and an airborne downward-looking camera connected to the airborne computing device. The airborne satellite positioning device is used to receive satellite navigation signals and output satellite positioning data. The airborne downward-looking camera is used to acquire scene images during the UAV's flight. The airborne computing device is used to execute any of the UAV positioning methods for fusion verification of satellite positioning and visual positioning described in this invention.

[0032] Beneficial effects: Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

[0033] 1. This invention uses satellite positioning signal quality grading assessment and a three-level positioning mode design. When the signal is excellent, it directly uses satellite positioning and performs low-frequency visual verification, which significantly reduces computational complexity and power consumption. When the signal deteriorates, it automatically switches to fusion mode or pure visual mode, ensuring the continuity and reliability of positioning.

[0034] 2. By introducing sequence consistency evaluation and geometric rationality analysis, this invention enables visual localization to no longer rely on the direct determination of single-frame cosine similarity, but combines continuous frame temporal constraints and geometric constraints, thereby improving robustness in scenarios with changes in viewpoint, lighting, and texture repetition. At the same time, it reduces the computational burden on embedded platforms through segmented feature libraries.

[0035] 3. This invention divides the flight path into 100m intervals for independent storage through segmented feature library design and dynamic loading mechanism. During flight, only the data of 3 intervals (within 10MB of memory) are stored continuously, which solves the problem of limited memory in embedded devices. The 20m overlap area between adjacent intervals ensures the continuity of positioning during switching.

[0036] 4. This invention uses a feature library update mechanism driven by daily flight data to automatically update the corresponding intervals using high-quality data collected when satellite signals are excellent. It preserves overlapping areas to ensure continuity with adjacent intervals, achieving low-cost dynamic maintenance and timeliness of the feature library, and avoiding the high cost of manual re-collection.

[0037] 5. This invention avoids frequent switching when satellite signals fluctuate near the threshold by adopting a dual-threshold control mode switching strategy, thereby improving system stability. Through the deviation verification mechanism between satellite and visual positioning, as well as the joint judgment mechanism of confidence and sequence consistency of visual positioning results, abnormal data sources can be detected in a timely manner and safety protection measures can be triggered, thereby improving flight safety. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the flight path bottom view image acquisition and segmented storage in an embodiment of the present invention;

[0039] Figure 2 This is an overall framework diagram of an embodiment of the present invention;

[0040] Figure 3 This is a detailed diagram of the program architecture of the airborne computing device in an embodiment of the present invention;

[0041] Figure 4 This is a diagram showing the switching status of the three-level positioning mode in an embodiment of the present invention;

[0042] Figure 5 This is a schematic diagram illustrating the dynamic loading of the feature library in an embodiment of the present invention. Detailed Implementation

[0043] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0044] Figure 2 This is an overall framework diagram of an embodiment of the present invention. The framework is divided into a pre-built visual feature library on the left and a real-time flight path section on the right. The visual feature library provides matching data support for the real-time flight path section on the right, and simultaneously receives high-quality positioning data output from the right for dynamic updates.

[0045] As attached Figure 2 As shown in the figure, a UAV positioning method for satellite positioning and visual positioning fusion verification in this embodiment includes the following steps:

[0046] Step S1: Construct a visual feature library; Acquire several images with high-precision positioning information along a fixed flight path of the UAV in advance. Divide the fixed flight path into several intervals according to a first length, and acquire image feature information corresponding to each interval and the flight path regions extending to both ends by a second length, thus obtaining a segmented visual feature library. The second length is less than the first length.

[0047] Step S1 includes the following steps:

[0048] Step S1-1: A drone equipped with a downward-looking camera of a fixed model, focal length, and resolution is used to collect video with positioning information within a fixed flight path. The optical axis of the downward-looking camera is vertically downward or maintains a fixed tilt angle to collect surface images directly below the flight path. During the acquisition process, the parameters of the downward-looking camera remain unchanged, resulting in clear video resolution and a stable frame rate. The positioning information includes GNSS positioning data and GNSS timestamps.

[0049] Step S1-2: Perform time-synchronized sampling on the video; sample at consecutive video frame intervals to obtain several image frames, and align them with the GNSS timestamp based on the image frame timestamp; for image frames that do not directly correspond to GNSS positioning data, use linear interpolation of adjacent GNSS positioning points or nearest neighbor mapping to complete the position information.

[0050] Steps S1-3: Obtain image feature information for all image frames; preprocess all image frames to extract corresponding image feature information, including location features and visual features; feature extraction is achieved through image feature extraction algorithms, such as ResNet, ConvNeXt, ViT, DINOv2, or NetVLAD, to obtain a 4096-dimensional global descriptor. The float32 data is quantized to float16 to save storage space; local features and temporal features of adjacent frames are extracted simultaneously and saved as a NumPy compressed file in float16 format. Location features include satellite coordinate data, i.e., GNSS positioning data.

[0051] Steps S1-4: Segment and store all image frames and their image feature information to establish a visual feature library; divide the fixed route into several intervals according to the first length, and extend the two ends of each interval by the second length. Store the image frames and their image feature information corresponding to the route area of ​​each interval separately to obtain the segmented visual feature library.

[0052] like Figure 1 The diagram illustrates the acquisition and segmented storage of flight path images in an embodiment of the present invention. A segmented storage structure is adopted, dividing the flight path into several intervals with a first length of 100m. Each interval is stored independently as a data file. A second length of 10m is extended to each end of each interval to form a 120m actual storage interval, including overlapping areas. The actual overlap area between adjacent intervals is 20m, establishing a visual feature library. For example, the first segment stores data from 0-120m, and the second segment stores data from 80-220m, with a 20m overlap between the two segments in the 80-100m interval. Each data segment is stored independently as a .npz format file, containing: descriptors (global descriptor matrix, shape [N, 4096], float16), local features, satellite_coords (satellite coordinates, shape [N, 2], float64), distances (distance from the segment start point, shape [N], float32), sequence window index, and metadata (metadata dictionary).

[0053] Step S2: During the real-time flight of the UAV along the flight path, the data output by the satellite positioning equipment and the airborne downward-looking camera on the UAV is acquired in real time. Combined with the visual feature library in step S1, after signal quality assessment, pattern decision, visual matching and fusion calculation, the UAV positioning data is output. In abnormal situations, safety measures are implemented, including deceleration and stopping, hovering and landing. The high-quality positioning data during the real-time flight is fed back to the visual feature library for updating.

[0054] Step S2 includes the following steps:

[0055] Step S2-1, Real-time data acquisition: During the real-time flight of the UAV on the flight path, satellite positioning data is obtained by receiving satellite signals through the onboard satellite positioning equipment on the UAV, and images of the scene where the UAV is located are collected through the onboard downward-looking camera.

[0056] Step S2-2: Assess the signal quality of the satellite signal, make a mode decision based on the assessment results, and determine the positioning mode; output the UAV positioning data after visual matching and fusion calculation in the positioning mode.

[0057] The satellite signal quality is assessed, and a mode decision is made based on the assessment results to determine the positioning mode, including setting an excellent threshold. and And the number of satellites is ≥8, among which, Here, C is the position accuracy factor, C is the carrier-to-noise ratio, and N0 is the number of visible satellites; the good threshold is: and Obtain the current satellite signal quality. When the signal quality is: and When the number of satellites is ≥8 and remains so for 3 seconds, the system enters satellite-dominated mode; when the signal quality is: and When the signal quality is: and Or, if satellite signals are lost, it will enter a pure visual positioning mode;

[0058] In the mode decision-making process, positioning mode switching employs dual threshold control. Switching from pure vision positioning mode to fusion positioning mode, or from fusion positioning mode to satellite positioning mode, requires meeting stricter threshold conditions to avoid frequent switching. or At that time, it will downgrade from satellite-dominated mode to filter fusion mode, and upgrading from filter fusion mode to satellite-dominated mode requires meeting certain conditions. and And lasts for 3 seconds; if GPS fails or or When this happens, the system will downgrade from the filtering and fusion mode to the pure visual positioning mode. Upgrading from the pure visual positioning mode to the filtering and fusion mode requires meeting certain conditions. and And the condition must last for 3 seconds.

[0059] Among them, satellite-dominated mode refers to: when the signal quality is: and When the number of satellites is ≥8 and lasts for 3 seconds, it enters satellite-dominated mode and directly uses satellite positioning data as the drone's positioning result. The positioning frequency is consistent with the output frequency of the satellite positioning device.

[0060] Fusion mode refers to: when the signal quality is: and When the time comes, it enters fusion mode; it uses filtering to fuse satellite positioning data and visual positioning data to calculate UAV positioning data;

[0061] When the satellite positioning signal quality improves to the preset excellent threshold and lasts for 3 seconds, it switches back to satellite-dominated mode; the visual positioning result adopts a dual-threshold mode to achieve dual-threshold control (high confidence threshold and low confidence threshold). When satellite positioning fails and the visual matching confidence is greater than or equal to the preset threshold, it switches to pure visual positioning mode.

[0062] Pure vision positioning mode refers to: when the signal quality is and If satellite signals are lost, the system will enter pure visual positioning mode and directly use visual positioning data as the drone's positioning result.

[0063] In pure vision positioning mode, visual positioning data is directly used as the UAV's positioning result output, and the positioning frequency is consistent with the output frequency of the image feature extraction algorithm; satellite signal quality is continuously monitored, and when the signal recovers... and And after 3 seconds, switch back to fusion mode.

[0064] The visual positioning data is directly used as the UAV positioning result output, including: calculating the comprehensive confidence score; if the comprehensive confidence score is greater than or equal to the confidence score threshold, the satellite coordinates stored in the image are output as the visual positioning result. The visual positioning algorithm includes global descriptor extraction, feature library retrieval, temporal consistency evaluation, and geometric rationality analysis; the calculation process is as follows:

[0065] The system acquires images of the scene where the drone is located, collected by the airborne downward-looking camera, and extracts global descriptors using the image feature extraction algorithm used when building the feature library. Cosine similarity is then calculated between these global descriptors and the descriptors in the feature library data files for the currently loaded interval and its adjacent intervals (three intervals in total) in the device's memory.

[0066]

[0067]

[0068]

[0069] in, For cosine similarity, For the feature descriptor of the currently queried image, These are feature descriptors extracted from images when building the feature library.

[0070] Based on similarity scores, the feature entries corresponding to the k feature descriptors with the highest scores are selected, namely the Top-k candidate entries. Each candidate entry corresponds to a frame of image and its global descriptor, local features, and associated satellite coordinates.

[0071] After removing outlier frames using the above method, the similarity consistency between the candidate image and its neighboring frames (indexed by distance) in the feature library and the corresponding frames within the time window is calculated. A sequence consistency score is defined. The calculation formula is:

[0072]

[0073] in, The timing window corresponding to the current frame; For the first time sequence window Global descriptor of a frame image This is a descriptor for the candidate image at its corresponding temporal position in the library. Normalized similarity function calculation, range [0,1]:

[0074] Only when When the (temporal consistency threshold) is reached, the candidate is accepted as a valid match.

[0075] Geometric rationality confidence level Based on the kinematic constraints of the UAV, calculate the geometric distance between the candidate position and the localization result of the previous frame. and with the theoretical maximum displacement Compare:

[0076]

[0077] like It was determined to be a geometric anomaly. .

[0078] The overall confidence level fusion uses a weighted geometric mean to fuse three levels of confidence:

[0079]

[0080] in, These are weights for similarity score, temporal consistency, and geometric plausibility, respectively. Overall confidence score. The closer the confidence level is to 1, the higher the confidence level. If the overall confidence level is greater than or equal to the confidence threshold... Output the satellite coordinates stored in the image as the visual positioning result; otherwise, return a matching failure.

[0081] In this step, feature extraction and matching are performed. Global descriptors of the images from the airborne downward-looking camera are extracted, and Top-k candidate images are retrieved from the feature library. The global descriptor results of the n adjacent frames preceding the current frame are retained to form a sequence window. The cosine similarity between the Top-k candidate images and the descriptors of the n adjacent frames is calculated. Ideally, the similarity between adjacent frames should gradually decrease during UAV flight. After removing abnormal frames, the similarity between the candidate images and their adjacent frames in the feature library (indexed by distance) and the corresponding frames within the temporal window is calculated to evaluate temporal consistency. Based on UAV kinematic constraints, the geometric distance between the Top-k candidate images and the localization results of the n adjacent frames is calculated and compared with the theoretical maximum displacement to analyze geometric rationality. Considering feature similarity, temporal consistency, and geometric rationality, when the confidence of the final matching result is greater than or equal to a preset threshold, the coordinate data corresponding to the image is output as the visual localization result; when the confidence of the final matching result is less than the preset threshold, visual localization is deemed to have failed.

[0082] The safety decision in step S2 refers to the following: In satellite-dominated mode and fusion mode, when the deviation between satellite positioning and visual positioning exceeds a preset threshold, or the confidence level of the visual positioning result is lower than a preset threshold, a mode switch or safety alarm is triggered; when both satellite positioning and visual positioning fail and continue for more than a preset time threshold, the UAV performs safety measures such as deceleration, stopping, hovering, or landing.

[0083] The preset time threshold for safety decision-making is 2 to 5 seconds. When satellite positioning fails and the visual positioning reliability is less than the preset threshold, or the distance deviation between the two exceeds the preset distance threshold and continues to exceed the threshold, the UAV will perform safety measures such as deceleration, hovering, or landing. The deceleration includes gradually reducing the flight speed until it comes to a stop, the hovering includes maintaining the current position altitude, and the landing includes landing on the ground or a safe area at a preset descent speed.

[0084] In step S2, the onboard computing device performs memory cache management. During real-time flight, the onboard computing device dynamically loads feature library data and keeps the data files of the current interval and its adjacent intervals in memory. When the UAV's position crosses the interval boundary, it automatically switches to use the data files of the adjacent interval for matching, while unloading the historical interval files that are not adjacent to the new interval and loading the new adjacent interval files.

[0085] Feature library cache management includes: three resident data files in the onboard computing device's memory, including the current interval and its adjacent intervals before and after; each interval's data file contains a descriptor for the image feature extraction algorithm of a 120m flight path, satellite coordinates, and distance information; when the UAV's position crosses the boundary of the current interval and enters an adjacent interval, the historical interval file that is not adjacent to the new interval from the three resident intervals is unloaded, and the interval file that is adjacent to the new interval is loaded, so that there are always three intervals of data available for matching in memory.

[0086] In step S2, the visual feature library is updated. Image data acquired during periods of excellent satellite signal quality in daily flights is quality-assessed. If the data covers a complete interval, it is used to update the feature library file for that interval, achieving dynamic maintenance and timeliness of the feature library. Specifically, during daily flights, when in satellite-dominated mode, the current frame image, satellite coordinates, and image feature extraction algorithm descriptors are marked as high-quality data. High-quality data is clustered by interval. When high-quality data covers a complete interval, it can be used to update the feature library for that interval, generating a new version of the .npz file. The two most recent versions of the data file are maintained for rollback, and earlier historical versions are deleted.

[0087] This embodiment also discloses a UAV positioning system that fuses and verifies satellite positioning and visual positioning, including an airborne computing device and an airborne satellite positioning device and an airborne downward-looking camera connected to the airborne computing device; the airborne satellite positioning device is used to receive satellite navigation signals and output satellite positioning data; the airborne downward-looking camera is used to acquire scene images during the flight of the UAV; the airborne computing device is used to execute the positioning method that fuses and verifies satellite positioning and visual positioning to achieve continuous and stable positioning and navigation of the UAV.

[0088] The airborne downward-looking camera has a fixed focal length and resolution, and a frame rate of 30fps. The airborne computing device is an embedded computing platform equipped with a GPU or NPU, which is used to perform tasks such as global descriptor extraction, cosine similarity calculation, filtering fusion, pattern decision-making, and dynamic management of the feature library in real time.

[0089] In the embodiments of this application, the terms "first" and "second" (if they exist) are used only as name identifiers and do not represent the order of first and second.

[0090] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This computer software product can be stored in a storage medium. The storage medium can be various types of storage, such as random access memory, read-only memory, flash memory, etc., like read-only memory, magnetic disk, optical disk, etc. It includes several instructions to cause a computer device (which can be a personal computer, a server, or a network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0091] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. The components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment solution according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0092] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for UAV positioning based on the fusion and verification of satellite positioning and visual positioning, characterized in that, The method includes the following steps: Step S1: Pre-acquire several images with positioning information within the fixed flight path of the UAV, divide the fixed flight path into several intervals according to the first length, acquire the image feature information corresponding to each interval and the second length of the flight path area at both ends, and obtain a segmented visual feature library. Step S2: During the real-time flight of the UAV along the flight path, the data output by the satellite positioning device and the airborne downward-looking camera on the UAV is acquired in real time. Combined with the visual feature library in step S1, after signal quality assessment, mode decision, visual matching and fusion calculation, the UAV positioning data is output and the positioning data during real-time flight is fed back to the visual feature library for updating.

2. The UAV positioning method for satellite positioning and visual positioning fusion verification according to claim 1, characterized in that, Step S1 includes the following steps: Step S1-1: Use a drone equipped with a downward-looking camera of a fixed model, focal length and resolution to collect video with positioning information within a fixed flight path. Step S1-2: Perform time-synchronized sampling on the video; sample at consecutive video frame intervals to obtain several image frames, and align them with the GNSS timestamp based on the image frame timestamp; for image frames that do not directly correspond to GNSS positioning data, use linear interpolation of adjacent GNSS positioning points or nearest neighbor mapping to complete the position information. Steps S1-3: Obtain image feature information of all image frames; preprocess all image frames and extract corresponding image feature information, including positional features and visual features; Steps S1-4: Segment and store all image frames and their image feature information to establish a visual feature library; divide the fixed flight path into several intervals according to a fixed first length, and extend the two ends of each interval by a second length to form an overlapping area. Store the image frames and their image feature information corresponding to the flight path area of ​​each interval separately to obtain the segmented visual feature library.

3. The UAV positioning method for satellite positioning and visual positioning fusion verification according to claim 2, characterized in that, The real-time data acquisition and visual positioning calculation in step S2 are as follows: Step S2-1, Real-time data acquisition: During the real-time flight of the UAV on the flight path, satellite positioning data is obtained by receiving satellite signals through the onboard satellite positioning equipment on the UAV, and images of the scene where the UAV is located are collected through the onboard downward-looking camera. Step S2-2, Visual Positioning Calculation: The satellite signal quality is assessed, and a mode decision is made based on the assessment results to determine the positioning mode; after visual matching and fusion calculation in the positioning mode, the UAV positioning data is output.

4. The UAV positioning method for satellite positioning and visual positioning fusion verification according to claim 3, characterized in that: The satellite signal quality is assessed, and a mode decision is made based on the assessment results to determine the positioning mode, including setting an excellent threshold. and And the number of satellites is ≥8, among which, Here, C is the position accuracy factor, C is the carrier-to-noise ratio, and N0 is the number of visible satellites; the good threshold is: and ; Obtain the current satellite signal quality, when the signal quality is: and When the number of satellites is ≥8 and remains so for 3 seconds, the system enters satellite-dominated mode; when the signal quality is: and When the signal quality is: and If satellite signal is lost, it will enter a pure visual positioning mode.

5. The UAV positioning method for satellite positioning and visual positioning fusion verification according to claim 4, characterized in that: The switching of positioning modes in different decision-making modes adopts dual threshold control.

6. The UAV positioning method for satellite positioning and visual positioning fusion verification according to claim 4, characterized in that: Satellite-dominated mode: directly uses satellite positioning data as the drone's positioning result, and the positioning frequency is consistent with the output frequency of the satellite positioning equipment; Fusion mode: Using methods such as Kalman filtering to fuse satellite positioning data and visual positioning data to calculate UAV positioning data; Pure vision positioning mode: Directly uses visual positioning data as the drone's positioning result.

7. The UAV positioning method for satellite positioning and visual positioning fusion verification according to claim 6, characterized in that: The visual positioning data is directly used as the UAV positioning result output, including: calculating the comprehensive confidence score; if the comprehensive confidence score is greater than or equal to the confidence score threshold, the satellite coordinates stored in the image are output as the visual positioning result. The formula for calculating the overall confidence level includes: in, To assess the overall confidence level, These are, respectively, similarity score weight, temporal consistency weight, and geometric rationality weight; For geometric rationality confidence level; For the maximum displacement, This represents the geometric distance between the candidate location and the localization result of the previous frame. For sequence consistency score, This is a descriptor for the candidate image at its corresponding temporal position in the library. For the current image descriptor, For candidate image descriptors in the feature library, For the first time sequence window Frame descriptor, For timing window, The normalized cosine similarity. The cosine similarity function is calculated, with a range of [-1, 1]. The normalized similarity function is calculated, with a range of [0,1].

8. The UAV positioning method for satellite positioning and visual positioning fusion verification according to claim 1, characterized in that: S2 also includes safety decisions, namely, in satellite-dominated mode and fusion mode, when the deviation between satellite positioning and visual positioning exceeds a preset threshold, or the confidence level of the visual positioning result is lower than a preset threshold, a mode switch or safety alarm is triggered; when both satellite positioning and visual positioning fail and continue for more than a preset time threshold, the UAV performs safety measures such as deceleration, stopping, hovering, or landing.

9. The UAV positioning method for satellite positioning and visual positioning fusion verification according to claim 1, characterized in that: The visual feature library update in step S2 includes: analyzing image data collected during flight in satellite-dominated mode; if the flight path range covered by the image data contains a complete range divided according to the method described in steps S1-4, then the image data within the range of the divided range is used to update the feature library file of the corresponding divided range, thereby realizing dynamic maintenance and timeliness guarantee of the feature library.

10. A UAV positioning system that integrates satellite positioning and visual positioning for verification, characterized in that: The system includes an airborne computing device and an airborne satellite positioning device and an airborne downward-looking camera connected to the airborne computing device; the airborne satellite positioning device is used to receive satellite navigation signals and output satellite positioning data; the airborne downward-looking camera is used to acquire scene images during the flight of the UAV; the airborne computing device is used to execute a UAV positioning method for satellite positioning and visual positioning fusion verification as described in any one of claims 1-9.

Images