Unmanned aerial vehicle global positioning method and device, electronic equipment and storage medium

Abstract

The application relates to a UAV global positioning method and device, electronic equipment and storage medium, comprising: processing a transformed image of an overhead aerial image through a preset convolutional neural network to obtain a vectorized feature image; searching a preset tile map global image feature database to obtain a candidate position object set, and screening a target candidate position object set; screening an image satisfying that the number of feature point pairs between images is greater than a preset number and the target homography matrix satisfies a preset requirement from the target candidate position object set to obtain a final candidate position set, determining a final candidate position according to a position overlap result, calculating an actual translation amount between images according to a translation component of a homography matrix matched from the final candidate position and an overhead aerial image of a flight area at a current moment, and performing global positioning. The application solves the problem of how to perform global positioning without any position prior condition, improves the correctness of the positioning result while ensuring the rapidity of the algorithm.

Description

Technical Field

[0001] This application relates to the field of positioning technology, and in particular to a global positioning method, device, electronic device and storage medium for unmanned aerial vehicles (UAVs). Background Technology

[0002] With the development of airborne processing capabilities and image sensors, map-assisted visual positioning technology for UAVs has gradually become a necessary supplement to GNSS (Global Navigation Satellite System) positioning. UAVs operating in complex environments require stronger global autonomous positioning capabilities. While relative positioning methods, primarily SLAM (Simultaneous Localization and Mapping), have made significant progress, they are prone to drift in positioning results due to their inability to acquire absolute pose. Map-assisted visual positioning technology provides an effective solution for absolute positioning of UAVs. UAVs can acquire their current absolute flight position using only a top-down camera and a pre-stored reference map with geographic location information, exhibiting strong autonomy and robustness. In particular, the effective encoding of remote sensing maps to construct a feature map library, followed by retrieval methods to achieve global positioning, is gradually becoming a mainstream positioning method.

[0003] For retrieval-based localization problems, map-assisted visual localization techniques are similar to Visual Place Recognition (VPR) in computer vision. Researchers primarily focus on constructing effective image feature description methods. For map retrieval problems, the difference between the captured image and the reference image often leads to retrieval failures due to various factors such as sensor differences, seasonal differences, lighting differences, and image resolution differences. Geographical consistency does not equate to image consistency. Therefore, researchers aim to propose image feature construction schemes that are more stable to temporal environmental changes. Early global image descriptor construction was mainly based on manually constructed features, such as the well-known SIFT (Scale-invariant feature transform) and SURF (Speeded Up Robust Feature). Building upon this, in 2003, Sivic and Zisserman of Oxford University proposed the Bag of Visual Words (BoVW) method, which implemented a global image description based on feature clustering statistics on SIFT feature points. This has become the foundation for many subsequent visual image retrieval schemes. Building upon visual word sets, the Visual Local Aggregated Vector Descriptor (VLAD) method expands the descriptive dimensions by employing the idea of ​​descriptor residual aggregation to construct visual features. It has also found wide application as a more effective retrieval feature. Furthermore, in 2003, Oliva et al. from MIT used Gabor filters to process images, integrating gradient information to provide a global description of the image, thus offering a new perspective for solving scene recognition tasks beyond feature point methods.

[0004] In recent years, the field of computer vision has made remarkable progress thanks to the emergence of deep learning methods. An increasing number of visual tasks can be solved to some extent with the help of deep learning, and various network architectures evolved from Convolutional Neural Networks (CNNs) have been widely applied in computer vision tasks. In 2016, Arandjelovic et al. improved the structure of the VLAD operator, modifying it into a differentiable VLAD implementation, and combined it with the CNN image extraction architecture to achieve the end-to-end trainable NetVLAD method. Since then, deep learning-based VLAD global image descriptors have become the mainstream solution to the scene re-identification problem, and researchers have made many additional designs based on it, all of which have achieved significant progress.

[0005] Although image retrieval has been developing for a relatively long time, with good solutions at both the algorithm and application levels, previous solutions have primarily focused on ground-view platforms, with few works proposing solutions for drone applications. Compared to ground-view objects like vehicle-mounted platforms, drone objects present a far greater complexity in their surrounding environment, making them more susceptible to temporal variations. Furthermore, since drone platforms use publicly available remote sensing images rather than pre-collected scene map databases, they exhibit stronger image heterogeneity and platform differences, posing significant challenges to scene re-identification tasks on drone platforms. Further exploration and solutions are needed to better integrate scene re-identification with drone visual matching systems. Summary of the Invention

[0006] This application provides a global positioning method, device, electronic device, and storage medium for unmanned aerial vehicles (UAVs), which solves the problem of how to perform global positioning when there is no prior location information, and improves the accuracy of positioning results while ensuring the speed of the algorithm.

[0007] The first aspect of this application provides a global positioning method for an unmanned aerial vehicle (UAV), comprising the following steps: acquiring an aerial view of the flight area at the current moment, transforming the aerial view based on a preset homography matrix to obtain a transformed image of the aerial view, and processing the transformed image of the aerial view through a preset convolutional neural network to obtain a vectorized feature image; searching from a preset tile map global image feature database based on the vectorized feature image to obtain a set of candidate locations, and selecting a set of target candidate locations from the set of candidate locations based on a first filtering strategy; selecting images from the set of target candidate locations that satisfy the condition that the number of feature point pairs between images is greater than a preset number and that the target homography matrix meets preset requirements to obtain a final set of candidate locations, determining the final candidate location based on the position overlap results in the final set of candidate locations, and calculating the actual translation amount between images based on the translation component of the homography matrix obtained by matching the final candidate location with the aerial view of the flight area at the current moment, so as to perform global positioning based on the actual translation amount and the final candidate location.

[0008] Optionally, the step of selecting images from the target candidate location object set that satisfy the condition that the number of feature point pairs is greater than a preset number and the target homography matrix satisfies a preset requirement, to obtain the final candidate location set, includes: associating the vectorized feature image with the images in the target candidate location object set by feature points to obtain the number of feature point pairs between the images; obtaining the target homography matrix that matches the vectorized feature image with the images in the target candidate location object set; and selecting images that satisfy the condition that the number of feature point pairs is greater than a preset number and the target homography matrix satisfies a preset requirement, to obtain the final candidate location set.

[0009] Optionally, before searching from the preset global image feature database of tile maps based on the vectorized feature image, the method further includes: converting the original remote sensing map into a tile map library that meets preset requirements; inputting the tile map library into the preset convolutional neural network for processing to obtain the preset global image feature database of tile maps.

[0010] Optionally, determining the final candidate position based on the position overlap results in the final candidate position set includes: if there are position overlap results in the final candidate position set, then the candidate position with the most pairs of associated points between images with position overlap results is taken as the final candidate position; if there are no position overlap results in the final candidate position set, then the candidate position with the most pairs of associated points between images in the final candidate position set is taken as the final candidate position.

[0011] Optionally, the first screening strategy is:

[0012] {P} = {p1, p2, ..., p k′}={p j′ |num(p j′ )>t};

[0013] Where, {P}={p1,p2,…,p k′ Let} be the set of final candidate results corresponding to the image, and k′ be the number of candidate location objects. j′ is the index of the target location, num() is the counting function, and t is the threshold number of times the candidate location appears in the set.

[0014] Optionally, the preset homography matrix is:

[0015]

[0016] Where a is the scaling factor of the image transformation, θ is the required rotation angle of the image represented by the homography, and t x t represents the horizontal translation of the image. y This represents the amount of translation of the image in the vertical direction.

[0017] A second aspect of this application provides a global positioning device for an unmanned aerial vehicle (UAV), comprising: an acquisition module, configured to acquire an aerial view of the flight area at a current moment, transform the aerial view based on a preset homography matrix to obtain a transformed image of the aerial view, and process the transformed image of the aerial view through a preset convolutional neural network to obtain a vectorized feature image; a filtering module, configured to search a preset tile map global image feature database based on the vectorized feature image to obtain a set of candidate location objects, and filter a set of target candidate location objects from the set of candidate location objects based on a first filtering strategy; and a positioning module, configured to filter images from the set of target candidate location objects that satisfy the condition that the number of feature point pairs between images is greater than a preset number and the target homography matrix meets preset requirements, to obtain a final candidate location set, and determine the final candidate location based on the position overlap results in the final candidate location set, and calculate the actual translation amount between images based on the translation component of the homography matrix obtained by matching the final candidate location with the aerial view of the flight area at the current moment, so as to perform global positioning based on the actual translation amount and the final candidate location.

[0018] Optionally, the positioning module is specifically used to: associate feature points of the vectorized feature image and the images in the target candidate location object set to obtain the number of feature point pairs between the images; obtain the target homography matrix that matches the vectorized feature image and the images in the target candidate location object set; and filter out images that satisfy the condition that the number of feature point pairs is greater than a preset number and the target homography matrix satisfies a preset requirement to obtain the final candidate location set.

[0019] Optionally, before searching from the preset global image feature database of tile maps based on the vectorized feature image, the positioning module is further configured to: convert the original remote sensing map into a tile map library that meets preset requirements; input the tile map library into the preset convolutional neural network for processing to obtain the preset global image feature database of tile maps.

[0020] Optionally, in determining the final candidate position based on the position overlap results in the final candidate position set, the positioning module is further configured to: if there are position overlap results in the final candidate position set, then take the candidate position with the most pairs of associated points between images that have position overlap results as the final candidate position; if there are no position overlap results in the final candidate position set, then take the candidate position with the most pairs of associated points between images in the final candidate position set as the final candidate position.

[0021] Optionally, the first screening strategy is:

[0022] {P} = {p1, p2, ..., p k′}={p j′ |num(p j′ )>t};

[0023] Where, {P}={p1,p2,…,p k′ Let} be the set of final candidate results corresponding to the image, and k′ be the number of candidate location objects. j′ is the index of the target location, num() is the counting function, and t is the threshold number of times the candidate location appears in the set.

[0024] Optionally, the preset homography matrix is:

[0025]

[0026] Where a is the scaling factor of the image transformation, θ is the required rotation angle of the image represented by the homography, and t x t represents the horizontal translation of the image. y This represents the amount of translation of the image in the vertical direction.

[0027] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the UAV global positioning method as described in the above embodiments.

[0028] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the UAV global positioning method as described in the above embodiments.

[0029] This application acquires an aerial view of the flight area at the current moment, transforms the aerial view based on a preset homography matrix, and obtains a transformed image of the aerial view. This transformed image is then processed by a preset convolutional neural network to obtain a vectorized feature image. Based on the vectorized feature image, a search is performed on a preset tile map global image feature database to obtain a set of candidate locations. A first filtering strategy is used to select a set of target candidate locations from this set. From the target candidate location set, images that satisfy the condition that the number of feature point pairs between images is greater than a preset number and that the target homography matrix meets preset requirements are selected, resulting in a final set of candidate locations. The final candidate locations are determined based on the positional overlap results in the final set. The actual translation between images is calculated based on the translation components of the homography matrix obtained by matching the final candidate locations with the aerial view of the flight area at the current moment. Global positioning is then performed based on the actual translation and the final candidate locations. This solves the problem of how to perform global positioning without any prior location information, improving the accuracy of the positioning results while maintaining the algorithm's speed.

[0030] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0031] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0032] Figure 1 This is a flowchart of a global positioning method for unmanned aerial vehicles (UAVs) provided according to an embodiment of this application;

[0033] Figure 2 This is an overall flowchart of a global positioning method for unmanned aerial vehicles according to an embodiment of this application;

[0034] Figure 3 This is a flowchart of a model enhancement method using homography transformation according to an embodiment of this application;

[0035] Figure 4 This is a schematic diagram illustrating an example of candidate confirmation using local descriptors according to one embodiment of this application;

[0036] Figure 5 This is a schematic diagram illustrating an example of the retrieval results of the dataset verification results according to one embodiment of this application;

[0037] Figure 6 This is a block diagram of a global positioning device for unmanned aerial vehicles according to an embodiment of this application;

[0038] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0039] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0040] The following description, with reference to the accompanying drawings, describes a UAV global positioning method, apparatus, electronic device, and storage medium according to embodiments of this application. To address the problem of global positioning in the absence of any prior location information mentioned in the background technology center, this application provides a global positioning method for unmanned aerial vehicles (UAVs). In this method, an aerial view of the flight area at the current moment is acquired, and the aerial view is transformed based on a preset homography matrix to obtain a transformed image. This transformed image is then processed by a preset convolutional neural network to obtain a vectorized feature image. Based on the vectorized feature image, a search is performed on a preset tile map global image feature database to obtain a set of candidate locations. A target candidate location set is then selected from the candidate location set based on a first filtering strategy. Images that satisfy the condition that the number of feature point pairs between images is greater than a preset number and that the target homography matrix meets preset requirements are selected from the target candidate location set to obtain a final candidate location set. The final candidate location is determined based on the position overlap results in the final candidate location set. The actual translation amount between images is calculated based on the translation component of the homography matrix obtained by matching the final candidate location with the aerial view of the flight area at the current moment, so as to perform global positioning based on the actual translation amount and the final candidate location. This solves the problem of how to perform global localization when there is no prior location information, improving the accuracy of the localization results while ensuring the speed of the algorithm.

[0041] Prior to implementing the embodiments of this application, this embodiment selected map resources provided by the USGS public satellite map data network. The test area was a 3km*3km urban area. The downloaded map size was 10000*10000, with a resolution of 0.3m per pixel. The downloaded data years were 2003, 2005, 2011, and 2015. In this embodiment, the 2015 map was selected as the baseline remote sensing map, and a sliding window method was used to segment the entire map into a tile library. The window size was 400*400, the sliding step size was 100 pixels, and overlapping sampling was performed, generating a total of 8464 tile maps. A convolutional network was used to construct a global descriptor for the entire library, generating a vector of length 4096 for each tile, forming a coded feature map library for global search. At the same time, this embodiment also randomly sampled 1200 images from the remote sensing maps of the other three years, 400 images for each time period, to simulate real-time images taken at different time periods. Both the real-time map and the baseline tile map library have been labeled accordingly to determine whether the search results in this invention are correct. Figure 2 This illustrates a schematic diagram of the search process, showing how a three-step tile filtering process yields the final search location results. Among these steps... Figure 3 The flowchart of the real-time image homography matrix transformation enhancement method designed in the embodiments of this application is shown, which mainly includes two parts: image transformation construction of homography matrix and aggregation of retrieval results. Figure 4 This demonstrates the matching pairs between candidate results from the encoded feature map library and real-time aerial photographs when using local descriptors for inter-image feature matching. The figures show successful matching results for simulated real-time images at three different time periods. It can be observed that when the matching objects are correct, the resulting feature point pairs are abundant and evenly distributed. Finally, the qualitative results of this embodiment are as follows: Figure 5 As shown, the results display three search results, including correctly and incorrectly retrieved map tiles. Using the local feature-related point pairs and colocation criteria given in this embodiment, the final correct candidate location results can be obtained. Regarding the quantitative test results of this embodiment, the dataset constructed in this embodiment contains 1200 search targets. After reasonable parameter configuration, this embodiment can achieve a positioning success rate of up to 75.06%, and the average positioning time in the test environment of this embodiment is approximately 6 seconds, demonstrating that this embodiment has a certain degree of efficiency and accuracy.

[0042] In this embodiment, a drone flight system adapted to the present method is constructed. At the hardware level, in addition to commonly used sensor units, the target drone platform should also have a top-down camera and an onboard computing unit. At the software level, the onboard platform needs to store a baseline map with labeled geographic information within the target flight area, and needs to pre-deploy the relevant trained network model and software operating environment.

[0043] Furthermore, Figure 1 This is a flowchart illustrating a global positioning method for unmanned aerial vehicles (UAVs) provided in an embodiment of this application.

[0044] like Figure 1 As shown, the UAV global positioning method includes the following steps:

[0045] In step S101, an aerial view of the flight area at the current moment is acquired, and the aerial view is transformed based on a preset homography matrix to obtain a transformed image of the aerial view. The transformed image of the aerial view is then processed by a preset convolutional neural network to obtain a vectorized feature image.

[0046] After the airborne visual positioning module starts working, the algorithm input end reads the current aerial image I of the flight area from the overhead camera on board the system. Because the overhead aerial image is taken at a high altitude, the height variations of ground objects can be approximately ignored, thus it can be considered a planar image. For planar images, homography matrices can characterize the relationships between images. Each homography matrix corresponds to a transformation operation of an image, and homography matrices H1, H2, ..., H... can be constructed. N (N is typically an integer around 5), to construct a series of special homography matrices to simulate relatively small image changes, such as short translations or small rotations. These homography matrices can transform the real-time acquired image I to obtain a set of vectorized feature images I1, I2, ..., I... N Since the changes in these images are relatively minor, it can be assumed that these N images still represent the same location, with only differences in the images themselves. Therefore, the search results should also be consistent.

[0047] In some embodiments, the preset homography matrix is:

[0048]

[0049] Where a is the scaling factor of the image transformation, θ is the required rotation angle of the image represented by the homography, and t x t represents the horizontal translation of the image. y This represents the amount of translation of the image in the vertical direction.

[0050] In step S102, based on the vectorized feature image, a search is performed from the preset tile map global image feature database to obtain a set of candidate location objects, and based on the first filtering strategy, a set of target candidate location objects is selected from the set of candidate location objects.

[0051] Optionally, in some embodiments, before searching from a preset global image feature database of tile maps based on vectorized feature images, the method further includes: converting the original remote sensing map into a tile map library that meets preset requirements; and inputting the tile map library into a preset convolutional neural network for processing to obtain the preset global image feature database of tile maps.

[0052] In some embodiments, the first screening strategy is:

[0053] {P} = {p1, p2, ..., p k′}={p j′ |num(p j′ (2)

[0054] Where, {P}={p1,p2,…,p k′ Let} be the set of final candidate results corresponding to the image, and k′ be the number of candidate location objects. j′ is the index of the target location, num() is the counting function, and t is the threshold number of times the candidate location appears in the set.

[0055] Those skilled in the art should understand that, for the sake of facilitating subsequent retrieval and positioning steps, the original remote sensing map needs to be converted into a format of map tiles with the same orientation and similar area size as the real-time captured image. Based on the flight altitude information and the resolution information of the original acquired remote sensing map, the geographical area that each tile should occupy during the division can be determined. Furthermore, a flight gimbal can be used to fix the shooting direction of the overhead camera so that the captured image maintains a consistent orientation even when the UAV's heading changes. This allows for the corresponding rotation adjustment of the original remote sensing map based on the initial angle. After correcting the image size and heading information, a sliding window sampling method is used to overlay and acquire data on the original remote sensing image, resulting in a tile map library that meets the preset requirements.

[0056] Due to image differences, the acquired tile map database is not directly used in the retrieval process. Instead, it is described by a convolutional network and used as image description vectors to form a pre-defined global image feature database of tile maps. This vector description method encodes a remote sensing image tile into an n-dimensional (typically thousands) vector, and can effectively extract more robust image features, overcoming the temporal differences inherent in the image itself. These numerous n-dimensional vectors constitute the pre-defined global image feature database of tile maps for subsequent retrieval. Further organized using a kd-tree data format, it allows for rapid retrieval and matching. The encoded feature database can be stored on the onboard device as the basis for subsequent retrieval.

[0057] Specifically, this application embodiment uses a convolutional network constructed in the same way as the preset tile map global image feature database for this set of vectorized feature images I1, I2, ..., I N After processing, we obtain the n-dimensional vectors of the N images after homography transformation, which can be denoted as q1,q2,…,q N For the tile map library, we can assume there are M tile maps in total, and denote their original images as S1, S2, ..., S... M This represents the possible location of the drone at point M. The feature map library can be represented as d1, d2, ..., d M Therefore, the retrieval and location process based on map coding can be defined by the following formula:

[0058] {P} i =argmin j∈{1,2,…,M} {(q i -d j ) 2}; (3)

[0059] Equation (3) uses the L2 norm to characterize the distance, indicating that the nearest neighbor retrieval is performed between the real-time map after the i-th transformation and the benchmark map library, and the target location index j of the nearest neighbor is recorded. Where, {P} i This represents a set of possible candidate targets, meaning that the selected target location index may not be unique for a given search object q. i The k nearest neighbors of the search results can all be used as candidate final results, as shown below:

[0060] {P} i ={p1,p2,…,p k} i (4)

[0061] Equation (4) represents the number of candidate locations found for the search target. For the real-time graph I, since N sets of homography transformations have been performed, there are actually N candidate sets, namely: {P}1, {P}2, ..., {P} N Because the homography matrix H1, H2, ..., H N The corresponding homology changes are relatively small. From the retrieval results, under ideal conditions, different candidate sets: {P}1, {P}2, ..., {P} N This should represent the same group of positions. Therefore, we can require that the final candidate positions should be those common candidate positions in the N candidate sets. Based on this, we can filter the k candidate position objects. The specific filtering formula is as follows:

[0062] {p} = {p1, p2, ..., p k′}={p j′ |num(p j′ )>t}; (5)

[0063] In equation (5), {P} = {p1, p2, ..., p k′} represents the final candidate result set corresponding to image I, where k' should be no greater than k, thus serving as a filter for erroneous candidates. The num() function is a counting function, calculated as follows:

[0064]

[0065] In equation (6), the if function represents the position p. j Does it belong to the location set {P}? i In this context, a value belonging to a candidate location is recorded as 1, otherwise as 0. The `num()` function represents the number of times the location to be verified appears in the set of N candidate locations. Combining the specific forms of equations (5) and (4), it can be seen that if a certain location is a final candidate location for image I, it needs to appear more than a threshold number t in the set of N candidate locations. Through this homography construction method, the candidate location sets {p1, p2, ..., p...} can be filtered out. k} i A subset of interference terms are used to obtain the final candidate position set {p1, p2, ..., p...}. k′}

[0066] In step S103, images that satisfy the condition that the number of feature point pairs between images is greater than a preset number and the target homography matrix meets the preset requirements are selected from the target candidate position object set to obtain the final candidate position set. The final candidate position is determined according to the position overlap results in the final candidate position set. The actual translation amount between images is calculated based on the translation component of the homography matrix obtained by matching the final candidate position with the aerial image of the flight area at the current time, so as to perform global positioning based on the actual translation amount and the final candidate position.

[0067] Optionally, in some embodiments, images that satisfy the condition that the number of feature point pairs is greater than a preset number and the target homography matrix meets a preset requirement are selected from the target candidate location object set to obtain the final candidate location set. This includes: associating feature points between the vectorized feature image and the images in the target candidate location object set to obtain the number of feature point pairs between the images; obtaining the target homography matrix that matches between the vectorized feature image and the images in the target candidate location object set; and selecting images that satisfy the condition that the number of feature point pairs is greater than a preset number and the target homography matrix meets a preset requirement to obtain the final candidate location set.

[0068] Those skilled in the art should understand that for the final candidate positions {p1,p2,…,p} k′ Since the candidate positions at this point have been filtered, they will be fewer than the original number of selected candidates, k, making it easier to quickly identify the correct position. To improve the accuracy of the retrieval and localization algorithm, local image descriptor matching is used for the candidate results {p1, p2, ..., p...}. k′ Further filtering is performed. Candidate position p j Corresponding to the original tile map block S pj The correct candidate location should have a stable matching relationship with the real-time aerial image through local descriptors. Therefore, for candidate locations, the superpoint image local descriptor based on deep learning is used to extract features from the corresponding tile map, and features are simultaneously extracted from the real-time aerial image. Subsequent filtering and verification processes are then performed based on these extraction results.

[0069] Specifically, in constructing matching point pairs, a brute-force matching method is used to associate the extracted image features of the two images. During the matching search, the algorithm uses a nearest neighbor search strategy, and sets a condition that if the distance between the nearest neighbor pair is less than 0.75 times the distance of the second nearest neighbor pair, this feature point pair is considered a pair that can be associated. When there are more than 10 pairs of feature points that can be associated between the images, this candidate position is considered a correct candidate position, and the homography matrix between the two images is calculated using RANSAC. When there are fewer than 10 pairs of points that can be associated, the two images are considered not associated, that is, the candidate position is an incorrect candidate position, and the filtering result needs to be removed.

[0070] After obtaining the homography matrix between the two images, and combining it with the definition of homography change between two planar images in Equation (1), the last element of the first row and the last element of the second row of the homography matrix represent the actual translation amount between the two images in the horizontal and vertical directions. For the calculated homography matrix, when the distance between these two images is greater than the side length of the image, such candidate results with abnormal values ​​are filtered out, and the final candidate position set is obtained.

[0071] Optionally, in some embodiments, determining the final candidate position based on the position overlap results in the final candidate position set includes: if there are position overlap results in the final candidate position set, then the candidate position with the most pairs of associated points between images with position overlap results is taken as the final candidate position; if there are no position overlap results in the final candidate position set, then the candidate position with the most pairs of associated points between images in the final candidate position set is taken as the final candidate position.

[0072] After filtering using the image local descriptor method, the number of candidate locations can generally be controlled to within ten. Due to the overlapping sampling method used during feature map library construction, overlapping candidate locations may still occur, meaning multiple filtering results exist within the same region. If overlapping results exist in the final set, the candidate location with the most matching associated point pairs is selected as the final candidate result. Conversely, if there are no overlapping locations or the overlapping locations are not unique, the candidate location with the most matching associated point pairs from the entire candidate location set is output as the final candidate result.

[0073] After obtaining the final candidate position, the translation component of the homography matrix obtained by matching it with the real-time aerial image is extracted. The actual translation amount between the images is calculated using the calculated true resolution of the image pixels. Combined with the latitude and longitude of the target candidate position itself, the latitude and longitude coordinates of the position represented by the current aerial image can be calculated, thus achieving the final positioning.

[0074] In summary, the embodiments of this application have the following beneficial effects: These embodiments are applicable to multiple scenario types and a wide range of flight environments, and while ensuring retrieval accuracy, they also incorporate corresponding design considerations for algorithm robustness. Vector encoding descriptions extracted through convolutional networks are used to describe the features of the divided tile reference map library, and the same convolutional network is used to describe the vectors of the real-time image, thereby enabling retrieval and positioning within the reference encoded map library. During retrieval and positioning, these embodiments also employ homography matrices to enhance the real-time image, improving the feature retrieval performance of the convolutional network model. For candidate locations, image local descriptors are further used for image matching. The number of associated point pairs established between the real-time aerial view and the candidate tile map is used to determine whether the candidate is the correct target location. Finally, overlapping sampled map library feature points are used to determine the correct retrieval location, and the positioning result is deduced and output. This approach can provide a certain degree of absolute positioning information even when external positioning devices fail. It can also be combined with mature relative navigation systems, such as inertial navigation systems and SLAM, to form a more stable positioning system that can eliminate relative positioning drift. This application embodiment also possesses three advantages: it eliminates the need for a complex baseline map preparation process, requiring only the use of openly available satellite map resources on the internet; the system's sensor end relies solely on a monocular camera, without overly complex equipment requirements; and it exhibits excellent adaptability to complex flight scenarios and large flight ranges. These advantages enable this application embodiment to be easily deployed on unmanned aerial vehicle (UAV) systems, while also demonstrating high robustness and accuracy.

[0075] The UAV global positioning method proposed in this application transforms the overhead aerial image of the current flight area based on a preset homography matrix, obtaining a transformed image of the overhead aerial image. This transformed image is then processed by a preset convolutional neural network to obtain a vectorized feature image. A search is performed on a preset tile map global image feature database to obtain a set of candidate locations. Based on a first filtering strategy, a set of target candidate locations is selected from this set. From the target candidate location set, images satisfying a greater than preset number of feature point pairs and a target homography matrix meeting preset requirements are selected, resulting in a final set of candidate locations. The final candidate locations are determined based on the position overlap results in the final set. The actual translation between images is calculated based on the translation components of the homography matrix obtained by matching the overhead aerial image of the current flight area. Global positioning is then performed based on the actual translation and the final candidate locations. This solves the problem of how to perform global positioning without any prior location information, improving the accuracy of the positioning results while maintaining the algorithm's speed.

[0076] Next, with reference to the accompanying drawings, a global positioning device for unmanned aerial vehicles (UAVs) according to an embodiment of this application is described.

[0077] Figure 6 This is a block diagram of a global positioning device for unmanned aerial vehicles (UAVs) according to an embodiment of this application.

[0078] like Figure 6 As shown, the UAV global positioning device 10 includes: an acquisition module 100, a filtering module 200, and a positioning module 300.

[0079] The acquisition module 100 is used to acquire an aerial view of the flight area at the current moment, transform the aerial view based on a preset homography matrix to obtain a transformed image of the aerial view, and process the transformed image of the aerial view through a preset convolutional neural network to obtain a vectorized feature image; the filtering module 200 is used to search from a preset tile map global image feature database based on the vectorized feature image to obtain a set of candidate location objects, and filter out a set of target candidate location objects from the set of candidate location objects based on a first filtering strategy; the positioning module 300 is used to filter out images from the set of target candidate location objects that meet the requirement that the number of feature point pairs between images is greater than a preset number and the target homography matrix meets preset requirements, to obtain a final set of candidate locations, and determine the final candidate location based on the position overlap results in the final candidate location set, and calculate the actual translation amount between images based on the translation component of the homography matrix obtained by matching the final candidate location with the aerial view of the flight area at the current moment, so as to perform global positioning based on the actual translation amount and the final candidate location.

[0080] Optionally, in some embodiments, the positioning module 300 is specifically used to: associate feature points of the vectorized feature image and the images in the target candidate location object set to obtain the number of feature point pairs between the images; obtain the target homography matrix that matches between the vectorized feature image and the images in the target candidate location object set; and filter out images that satisfy the condition that the number of feature point pairs is greater than a preset number and the target homography matrix satisfies a preset requirement to obtain the final candidate location set.

[0081] Optionally, in some embodiments, before searching from a preset global image feature database of tile maps based on vectorized feature images, the positioning module 300 is further configured to: convert the original remote sensing map into a tile map library that meets preset requirements; input the tile map library into a preset convolutional neural network for processing to obtain a preset global image feature database of tile maps.

[0082] Optionally, in some embodiments, the positioning module 300 determines the final candidate position based on the position overlap results in the final candidate position set, and is further configured to: if there are position overlap results in the final candidate position set, then the candidate position with the most pairs of associated points between images with position overlap results is taken as the final candidate position; if there are no position overlap results in the final candidate position set, then the candidate position with the most pairs of associated points between images in the final candidate position set is taken as the final candidate position.

[0083] Optionally, in some embodiments, the first screening strategy is:

[0084] {P} = {p1, p2, ..., p k′}={p j′ |num(p j′ )>t};

[0085] Where, {P}={p1,p2,…,p k′ Let} be the set of final candidate results corresponding to the image, and k′ be the number of candidate location objects. j′ is the index of the target location, num() is the counting function, and t is the threshold number of times the candidate location appears in the set.

[0086] Optionally, in some embodiments, the preset homography matrix is:

[0087]

[0088] Where a is the scaling factor of the image transformation, θ is the required rotation angle of the image represented by the homography, and t x t represents the horizontal translation of the image. y This represents the amount of translation of the image in the vertical direction.

[0089] It should be noted that the foregoing explanation of the UAV global positioning method embodiment also applies to the UAV global positioning device of this embodiment, and will not be repeated here.

[0090] According to the UAV global positioning device proposed in this application, the device transforms the overhead aerial image of the current flight area based on a preset homography matrix to obtain a transformed image of the overhead aerial image. This transformed image is then processed by a preset convolutional neural network to obtain a vectorized feature image. A search is performed on a preset tile map global image feature database to obtain a set of candidate locations. Based on a first filtering strategy, a set of target candidate locations is selected from the candidate location set. From the target candidate location set, images satisfying a greater than preset number of feature point pairs and a target homography matrix meeting preset requirements are selected to obtain the final candidate location set. The final candidate location is determined based on the position overlap results in the final candidate location set. The actual translation amount between images is calculated based on the translation component of the homography matrix obtained by matching the overhead aerial image of the current flight area. Global positioning is then performed based on the actual translation amount and the final candidate location. This solves the problem of how to perform global positioning without any prior location information, improving the accuracy of the positioning results while ensuring the algorithm's speed.

[0091] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0092] The memory 701, the processor 702, and the computer program stored on the memory 701 and executable on the processor 702.

[0093] When the processor 702 executes the program, it implements the UAV global positioning method provided in the above embodiments.

[0094] Furthermore, electronic devices also include:

[0095] Communication interface 703 is used for communication between memory 701 and processor 702.

[0096] The memory 701 is used to store computer programs that can run on the processor 702.

[0097] The memory 701 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0098] If the memory 701, processor 702, and communication interface 703 are implemented independently, then the communication interface 703, memory 701, and processor 702 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0099] Optionally, in a specific implementation, if the memory 701, processor 702, and communication interface 703 are integrated on a single chip, then the memory 701, processor 702, and communication interface 703 can communicate with each other through an internal interface.

[0100] The processor 702 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0101] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described UAV global positioning method.

[0102] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0103] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0104] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0105] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0106] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0107] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0108] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0109] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A global positioning method for unmanned aerial vehicles (UAVs), characterized in that, The method comprises the following steps: obtaining an aerial view image of a flight area at a current time, transforming the aerial view image based on a preset homography matrix to obtain a transformed image of the aerial view image, and processing the transformed image of the aerial view image through a preset convolutional neural network to obtain a vectorized feature image; searching a preset tile map global image feature database based on the vectorized feature image to obtain a candidate position object set, and screening a target candidate position object set from the candidate position object set based on a first screening strategy; and screening images that satisfy a condition that a number of feature point pairs between the images is greater than a preset number and a target homography matrix satisfies a preset requirement from the target candidate position object set to obtain a final candidate position set, determining a final candidate position according to a position overlap result in the final candidate position set, and calculating an actual translation amount between images according to a translation component of the homography matrix matched based on the final candidate position and the aerial view image of the flight area at the current time to perform global positioning according to the actual translation amount and the final candidate position; the preset homography matrix is: ； wherein, is a scaling size of the image transformation, is a rotation angle size of the image required by the homographic change, is a translation amount of the image in the horizontal direction, is a translation amount of the image in the vertical direction.

2. The method of claim 1, wherein, the screening of the images that satisfy the condition that the number of feature point pairs is greater than the preset number and the target homography matrix satisfies the preset requirement from the target candidate position object set to obtain the final candidate position set comprises: associating feature points in the vectorized feature image and images in the target candidate position object set to obtain the number of feature point pairs between the images; obtaining the target homography matrix matched between the vectorized feature image and the images in the target candidate position object set; screening the images that satisfy the condition that the number of feature point pairs is greater than the preset number and the target homography matrix satisfies the preset requirement to obtain the final candidate position set.

3. The method according to claim 1 or 2, characterized in that, Before the searching of the preset tile map global image feature database based on the vectorized feature image, the method further comprises: transforming an original remote sensing map into a tile map library that satisfies a preset requirement; inputting the tile map library into the preset convolutional neural network for processing to obtain the preset tile map global image feature database.

4. The method of claim 3, wherein, The determination of the final candidate position according to the position overlap result in the final candidate position set comprises: if there is a position overlap result in the final candidate position set, taking a candidate position with the most associated point pairs between images as the final candidate position; if there is no position overlap result in the final candidate position set, taking a candidate position with the most associated point pairs between images in the final candidate position set as the final candidate position.

5. The method of claim 1, wherein, the first screening strategy is: ； wherein, is a final candidate result set corresponding to the image, is the number of candidate location objects, is a target location index, num() is a counting function, and t is a threshold number of occurrences in the candidate location set.

6. A global positioning apparatus for a drone, comprising: comprises: an obtaining module, configured to obtain an aerial view image of a flight area at a current time, transform the aerial view image based on a preset homography matrix to obtain a transformed image of the aerial view image, and process the transformed image of the aerial view image through a preset convolutional neural network to obtain a vectorized feature image; The screening module is configured to search, based on the vectorized feature image, a preset tile map global image feature database to obtain a candidate position object set, and screen, based on a first screening strategy, a target candidate position object set from the candidate position object set. And The positioning module is configured to screen, from the target candidate position object set, images that satisfy that a number of inter-image feature point pairs is greater than a preset number and that a target homography matrix satisfies a preset requirement, to obtain a final candidate position set, determine a final candidate position according to a position overlap result in the final candidate position set, and calculate an actual translation amount between images according to a translation component of a homography matrix matched from the final candidate position and a bird's-eye view aerial image of the flight area at the current time, to perform global positioning according to the actual translation amount and the final candidate position. The preset homography matrix is: ； wherein, is a scaling size of the image transformation, is a required rotation angle size of the image represented by the homography change, is a translation amount of the image in the horizontal direction, is a translation amount of the image in the vertical direction.

7. The apparatus of claim 6, wherein, The positioning module is specifically configured to: associate feature points in the vectorized feature image and images in the target candidate position object set to obtain a number of inter-image feature point pairs; obtain the target homography matrix matched between the vectorized feature image and the images in the target candidate position object set; screen images that satisfy that the number of feature point pairs is greater than the preset number and that the target homography matrix satisfies the preset requirement, to obtain the final candidate position set.

8. An electronic device, comprising: The device comprises a memory and a processor. The processor reads executable program codes stored in the memory to run programs corresponding to the executable program codes, to implement the global positioning method of the unmanned aerial vehicle according to any one of claims 1-5.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The program is executed by the processor to implement the global positioning method of the unmanned aerial vehicle according to any one of claims 1-5.

Images