An unmanned aerial vehicle navigation method based on image matching

Abstract

This invention relates to the field of unmanned aerial vehicle (UAV) navigation technology and discloses a UAV navigation method based on image matching. The method includes the following steps: obtaining a precise region original map set; performing boundary fusion on the precise region feature original map set to obtain a precise terrain original map; conducting on-site measurements of the UAV's location to obtain captured images and perform calculations to obtain a real-time feature terrain map; matching the real-time feature terrain map with the precise terrain original map to obtain the current precise location; receiving the flight path or original target location set on the precise terrain original map to generate a flight path navigation map or a flight target navigation map; analyzing the current flight area to generate and calculate an overlapping navigation area; and performing correction operations on the precise terrain original map. This invention combines GPS navigation to correct GPS navigation errors and can also perform offline navigation, enabling autonomous navigation within the operational area with precise positioning.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) navigation technology, and more particularly to an UAV navigation method based on image matching. Background Technology

[0002] Drone navigation technology, which relies on GPS, suffers from severe error accumulation over time, rendering drones unable to complete their missions or even return to base. Therefore, researching an autonomous navigation method capable of operating without GPS is crucial for completing flight missions in areas with strong or blocked GPS, such as jungles, urban clusters, and mountain valleys. Summary of the Invention

[0003] This invention provides an image matching-based drone navigation method to solve at least one of the aforementioned technical problems.

[0004] To achieve the above objectives, the present invention provides an image matching-based unmanned aerial vehicle (UAV) navigation method, comprising:

[0005] Step S1: Use visual odometry to take and measure the operating range of the UAV at the first altitude to obtain the first captured image. Extract keyframes from the first captured image to obtain the first keyframe set. Send the first keyframe set to the server so that the server can stitch the images together to obtain the blurred original image.

[0006] Step S2: Divide and sort the blurred original image in matrix form to obtain the original image set of blurred regions;

[0007] Step S3: Perform panoramic photography at multiple locations based on the geographical area range of each fuzzy region original image in the fuzzy region original image set to obtain a second captured image, where each location includes altitude and latitude and longitude. Extract keyframes from the second captured image to obtain a second keyframe set and an altitude information set.

[0008] Step S4: Send the second keyframe set and height information set to the server so that the server can perform feature extraction based on the second keyframe set, obtain the regional feature map set, aggregate the regional feature map set to build a three-dimensional data model, thereby obtaining the accurate original regional map set;

[0009] Step S5: Based on the precise region original map set and the height information set, calculate the height information in the relevant height information set for each precise region original map in the precise region original map set to obtain the region feature point set. Mark the region feature point set on the precise region original map to obtain the precise region feature original map set.

[0010] Step S6: Perform boundary fusion operation on the original map set of precise regional features to obtain the original map of precise terrain, and store the original map of precise terrain in the guidance system of the UAV.

[0011] Step S7: Conduct on-site measurements of the drone's location to obtain the third captured image, atmospheric visibility value, and light intensity value. Calculate the first weight value based on the atmospheric visibility value and the preset visibility value. Calculate the second weight value based on the light intensity value and the preset light intensity value. Compare the first weight value and the second weight value to obtain the maximum weight. Use the maximum weight to select and switch the captured image type.

[0012] Step S8: Calculate the height information of the third captured image based on the third captured image, extract key frames from the third captured image, and obtain the on-site measurement key frames.

[0013] Step S9: Based on the keyframes measured in the field, a panoramic image is obtained by calculation using the PP-MSVSR algorithm. The panoramic image is then convolved to obtain a real-time feature map set. The real-time feature map set is then aggregated to obtain a real-time terrain map.

[0014] Step S10: Calculate the real-time terrain map based on the height information of the third captured image to obtain real-time feature points, mark the real-time feature points on the real-time terrain map, and obtain the real-time feature terrain map.

[0015] Step S11: The real-time feature map and the original precise map are matched multiple times using a preset matching order to obtain multiple sets of matching degree. The matching degree sets are filtered to obtain the highest matching degree. The highest matching degree is compared with the expected value set to obtain the region corresponding to the matching degree set with the highest similarity. This region is reset as the current flight area of ​​the UAV. The current precise position of the UAV is obtained in the original precise map. The current precise positions that cannot be matched are sent back to the server for processing.

[0016] Step S12: Receive the flight path set on the precise terrain map, generate a flight path navigation map based on the current precise location, and perform autonomous navigation flight; or, receive the original target location set on the precise terrain map, generate a flight target navigation map based on the current precise location, and perform autonomous navigation flight.

[0017] Step S13: Analyze the current flight area of ​​the UAV to generate a navigation path set; analyze the navigation path set, and when the number of times the navigation overlap area appears in the navigation path set reaches the overlap threshold, generate an overlapping navigation area; analyze the overlapping navigation area, count the number of times each real-time feature point belongs to the real-time feature point set but not to the regional feature point set, and mark the real-time feature points whose occurrence count reaches the first threshold as corrected feature points; count the number of times each real-time feature point belongs to the regional feature point set but not to the real-time feature point set, and mark the real-time feature points whose occurrence count reaches the second threshold as eliminated feature points; based on the corrected feature points and eliminated feature points, perform a correction operation on the original accurate terrain map to obtain a more accurate original terrain map.

[0018] This embodiment determines the current location by comparing the UAV with the precise original terrain map, thus making navigation and positioning more accurate. The precise terrain map is modified based on multiple flight paths to better match the current work area. Based on the stored precise terrain map, UAV navigation can operate without GPS, enabling network-free operations. This allows the UAV to adapt to harsher geographical environments for functional operations. Data transmission based on keyframes requires significantly less data than directly transmitting video images. Data transmission using small amounts of data and discontinuous keyframe images is much faster, and the anti-interference capability of discontinuous keyframe data transmission is much stronger than that of video.

[0019] In one embodiment of this specification, step S4, which involves sending a second set of keyframes and a set of height information to the server so that the server can perform feature extraction based on the second set of keyframes to obtain a region feature map, includes the following steps:

[0020] The second keyframe set and altitude information set are sent to the server so that the server can highlight the feature regions of the keyframes in the second keyframe set based on the second keyframe set returned by the UAV, and use preset spatial weights and channel weights to extract features from different keyframes using convolution operations to obtain a set of regional feature maps.

[0021] This embodiment extracts keyframes from UAV images and transmits them back to the server, thereby reducing the amount of data transmitted and improving transmission efficiency. The data is processed back to the server, reducing the workload of the UAV, making the UAV more compact and increasing its working efficiency. The server can perform calculations, reducing factors that interfere with the calculation, thus making the calculation more accurate and efficient.

[0022] In one embodiment of this specification, step S4, which involves aggregating the regional feature map set to establish a three-dimensional data model, thereby obtaining the accurate original regional map set, includes the following steps:

[0023] A deep convolutional feature aggregation framework is used to aggregate regional feature maps and establish a three-dimensional data model, thereby obtaining an accurate original regional map set.

[0024] This embodiment uses a region feature map set for aggregation, combining the features of various feature maps to form a whole, making it easier for the UAV to identify features; it also establishes a three-dimensional data model, making it easier for the UAV to identify its current location. The deep convolution feature aggregation framework has lower computational difficulty and faster speed, and its training difficulty is lower compared to other neural networks.

[0025] In one embodiment of this specification, step S6 specifically includes the following steps:

[0026] Extract the edge regions from the original precise region feature map set. Then, perform convolution operations on the overlapping or similar regions of the edge regions in the order of the original precise region feature map set to make the edges transition smoothly, thereby achieving the fusion operation and obtaining the accurate original topographic map.

[0027] This embodiment uses boundary fusion to synthesize a large-scale, precise topographic map by combining the original images of various small-scale, precise regional features. This transforms a messy map into a regular one, making navigation more convenient and expanding the working range. Boundary fusion can also be performed on the precise topographic map to make the map even wider. The map in question is not necessarily a map with regular edges and can be adjusted according to the needs of the working area.

[0028] In one embodiment of this specification, step S9, which involves calculating the panoramic image based on on-site measurements and using the PP-MSVSR algorithm, includes the following steps:

[0029] The PP-MSVSR algorithm is used to align the pixels of the field measurement keyframes with the pixels of the accurate original terrain image, thereby obtaining a panoramic image.

[0030] This embodiment aligns the pixels of the keyframe with the pixels of the accurate original terrain map through calculation, which makes the comparison between the keyframe and the real-time feature map more accurate when comparing the keyframe and the regional feature map on the original terrain map.

[0031] In one embodiment of this specification, step S11 involves multiple matching operations between the real-time feature map and the precise original map using a preset matching order, which can be categorized into three types:

[0032] (1) Matching is performed by gradually expanding the range based on the location of the original map corresponding to the GPS navigation positioning;

[0033] (2) In the absence of a network, matching is performed using the matrix order of the accurate original topographic map;

[0034] (3) Sort the real-time feature map matching order from high to low according to the feature point matching degree;

[0035] Step S11 includes the following steps:

[0036] Under the condition of the same height information, the real-time feature points and regional feature points are matched for the first time according to the matching order (1) or (2). The maximum likelihood estimation algorithm is used to match the real-time feature points on the real-time topographic map with the regional feature points on the precise topographic map to obtain the first set of matching degree.

[0037] When the latest matching degree of the first matching degree set is higher than the first expected value during the first matching, the matching is stopped immediately, and the position of the stop matching point is determined on the accurate original topographic map as the current position;

[0038] When the first matching does not have a full map match with a matching degree higher than the first expected value in the first matching set, the highest matching degree in the first matching set is obtained, and the matching point with the highest matching degree higher than the second expected value is determined as the current position on the accurate original topographic map.

[0039] When the highest matching degree is lower than the second expected value, the real-time feature map and the regional feature map are matched for the second time according to the matching order (3), and the real-time feature map and the regional feature map are matched for similarity using the maximum likelihood estimation algorithm to obtain the second set of matching degree sets;

[0040] Obtain the highest matching degree in the second set of matching degrees, and determine the matching point on the accurate original topographic map where the highest matching degree is higher than the third expected value as the current position;

[0041] If the highest matching degree is lower than the third expected value, the UAV autonomously raises its flight altitude to the second altitude to obtain scattered real-time feature points on the real-time terrain map. According to the matching order (1) or (2), the real-time feature points and regional feature points are matched for the third time. The maximum likelihood estimation algorithm is used to match the real-time feature points on the real-time terrain map with the regional feature points on the accurate terrain map to obtain the third set of matching degree sets, where the first altitude is higher than the second altitude.

[0042] Obtain the highest matching degree in the third set of matching degrees, and determine the matching point on the accurate original topographic map where the highest matching degree is higher than the fourth expected value as the current position;

[0043] If the highest matching degree is lower than the fourth expected value, the real-time feature map and the regional feature map are matched four times according to the matching order (1) or (2), and the real-time feature map and the regional feature map are matched for similarity using the maximum likelihood estimation algorithm to obtain the fourth set of matching degree sets.

[0044] Obtain the highest matching degree in the fourth set of matching degrees, and determine the matching point on the accurate original topographic map where the highest matching degree is higher than the fifth expected value as the current position;

[0045] If the highest matching degree is lower than the fifth expected value, the first to fourth matching degree sets will be sent back to the server for processing, with the first, second, third, fourth, and fifth expected values ​​decreasing gradually.

[0046] This embodiment can obtain the current location more efficiently through different matching methods; by matching the current location in a hierarchical manner, the matching accuracy and precision are improved. If the first expected value is reached during the first comparison, the matching is stopped to determine the current precise location, which can improve the matching speed, especially when GPS-assisted positioning is used. The matching efficiency is higher. If the second expected value is reached during the second comparison, the required accuracy standard is also met. During the third comparison, the feature map is used for matching, which can better compare the feature areas to obtain the precise location. During the fourth comparison, the drone is first raised to the second altitude before shooting, which is conducive to obtaining a larger range of shooting images, including more feature points and more complex feature maps. The increase in feature points makes the similarity obtained by matching more accurate. During the fifth comparison, the accuracy of matching is higher by using more complex feature maps. The server is used to process the matching that cannot be matched to obtain the precise location. The server can be a console within the working range. No network is required for transmission. Electromagnetic waves can be used. The console has stronger processing capabilities and higher efficiency.

[0047] In one embodiment of this specification, step S12, which involves receiving the flight path set on the precise terrain map, generating a flight path navigation map based on the current precise location, and performing autonomous navigation flight, specifically includes the following steps:

[0048] Receive the flight path set on the accurate original terrain map, and generate a flight path navigation map on the accurate original terrain map of the UAV based on the flight path;

[0049] Construct a return flight path based on the current precise location and the starting point of the generated flight path navigation map, and fly to the starting point of the flight path navigation map according to the return flight path to conduct autonomous navigation flight.

[0050] This embodiment sets the flight path map on the precise topographic map of the server and sends it to the precise topographic map of the UAV. By sending the flight path based on the information on the precise topographic map, the amount of data transmission is reduced and the transmission efficiency is accelerated. Based on the transmitted flight path information, the flight path can be accurately generated on the precise topographic map of the UAV. Autonomous return navigation is established based on the precise position before the point and the starting point of the flight path, which reduces a lot of manual operation and increases work efficiency.

[0051] In one embodiment of this specification, step S12, which involves receiving the original target location set on the precise terrain map, generating a flight target navigation map based on the current precise location, and performing autonomous navigation flight, includes the following steps:

[0052] The system receives the original target location set on the precise terrain map, marks the precise target location on the precise terrain map of the UAV based on the original target location, generates a flight target navigation map based on the precise target location, and performs autonomous navigation flight.

[0053] This embodiment determines the precise target location by setting the original target location on the precise terrain map of the control console and sending it to the precise terrain map of the UAV, thereby autonomously generating a navigation path. This reduces the amount of data transmission, speeds up transmission efficiency, improves the autonomous navigation capability of the UAV, and reduces a lot of manual operation, thus increasing work efficiency.

[0054] In one embodiment of this specification, step S13, which involves correcting the original accurate topographic map based on corrected feature points and eliminated feature points to obtain a more accurate original topographic map, includes the following steps:

[0055] Searching based on the corrected feature points, obtaining the real-time feature map corresponding to the corrected feature points, and replacing the real-time feature map with the corresponding regional feature map, thereby realizing the correction and adjustment operation on the accurate original topographic map;

[0056] The search is performed based on the disappearing feature points to obtain the real-time feature map corresponding to the disappearing feature points. The real-time feature map is then replaced with the corresponding regional feature map to perform correction and adjustment operations on the accurate original topographic map.

[0057] This embodiment corrects overlapping areas through multiple navigations, making the original accurate terrain map more suitable for the current working area. This ensures that the working environment of the UAV remains in optimal condition even as the environment changes over time. By using the corrected feature points and disappearing feature points obtained from multiple navigations to correct the original accurate terrain map, it can be determined that only small-scale permanent changes in the area that need correction are effective modifications to the original accurate terrain map.

[0058] In this application example, by obtaining a precise original terrain map based on the detection of the working environment, the UAV can navigate autonomously within the specified working area without the GPS navigation system. Based on the precise original terrain map, the current precise location can be determined to better assist navigation. Multiple matching methods improve navigation accuracy and provide more autonomous solutions to deal with emergencies. The precise original terrain map is autonomously corrected so that the UAV can remain in the best condition even when the working environment changes over time. Attached Figure Description

[0059] Figure 1This is a schematic diagram of some steps in an image matching-based UAV navigation method according to the present invention.

[0060] Figure 2 This is a schematic diagram of another part of the steps of an image matching-based UAV navigation method of the present invention;

[0061] Figure 3 for Figure 2 A detailed flowchart illustrating the implementation steps of one of the processes;

[0062] Figure 4 A height comparison diagram of an example drone-captured image;

[0063] Figure 5 This is a schematic diagram illustrating the method of cutting a blurred original image for a single instance.

[0064] Figure 6 A schematic diagram of a flight path navigation example;

[0065] Figure 7 A schematic diagram of flight target navigation for an example;

[0066] Figure 8 This is a schematic diagram illustrating the feature map aggregation method for a single instance.

[0067] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0068] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0069] This application provides an image matching-based drone navigation method. The execution entity of the image matching navigation method includes, but is not limited to, at least one of the following devices: a server, a drone, or other devices that can be configured to execute the method provided in this application. The server includes, but is not limited to, a console, a single server, a server cluster, a cloud server, or a cloud server cluster. The internal equipment units of the drone include, but are not limited to, a storage unit, a shooting unit, a data processing unit, a data transmission unit, an atmospheric transparency detection unit, a light intensity detection unit, and a GPS unit.

[0070] Reference Figure 1 The above is a partial flowchart illustrating a drone navigation method based on image matching according to the present invention. In this example, the drone navigation method based on image matching includes:

[0071] Step S1: Obtain the first captured image, extract keyframes, and then stitch the keyframes together to obtain the blurred original image;

[0072] Specifically, the operating range of the UAV is photographed and measured on-site at a first altitude using a visual odometry system to obtain a first captured image. Keyframes are extracted from the first captured image to obtain a first keyframe set. The first keyframe set is then sent to the server so that the server can stitch the images together to obtain a blurred original image.

[0073] In this embodiment of the invention, the visual odometry refers to an instrument used to calculate the altitude of the UAV and the captured images. The first altitude refers to the altitude at which the reconnaissance UAV first takes a rough picture of the area where the UAV needs to operate. This altitude is set manually and can be modified according to the terrain. It is used to obtain a blurred original image. The on-site shooting and measurement refer to capturing images and measuring altitude, respectively. The captured image is named the first captured image. The measured altitude is used to determine whether the UAV is at the first altitude for shooting. The keyframe extraction from the first captured image refers to performing a frame segmentation operation on the image to form an image frame set. The images that meet the set sharpness requirements are then filtered through a convolution operation with a custom convolution kernel to obtain an initial keyframe set. The initial keyframes that meet the set similarity requirements are deduplicated through a convolution operation to obtain the first keyframe set. The method of sending the data to the server is to connect to a network such as a local area network, metropolitan area network, wide area network, or personal network through the UAV's network interface. The server can be a console, a single server, a server cluster, a cloud server, or a cloud server cluster. The stitching of the first keyframe set refers to matching the overlapping areas of the keyframes and merging the overlapping areas. This process is repeated to obtain the blurred original image.

[0074] The first height can be set to 300 meters or a preset value can be obtained through training. The sharpness can be set to 0.90 or a preset value can be obtained through training. The similarity can be set to 0.4 or a preset value can be obtained through training.

[0075] Furthermore, the images captured in this embodiment of the invention can be captured using imaging devices such as monocular cameras, binocular cameras, and tricular cameras.

[0076] As an example of the present invention, reference is made to Figure 4 The image shown is a height comparison diagram of an example drone-captured image. The content described in this example includes:

[0077] S401: This indicates the normal operating altitude of the drone. The area within the elliptical dashed line below the drone is the normal shooting range of the drone.

[0078] S402: This refers to the altitude at which the drone is repositioned when it is unable to accurately locate its current position at its normal operating altitude. This altitude is called the second altitude.

[0079] S403: This refers to the height at which the work area is captured in a general overview, known as the first height.

[0080] There are two methods for obtaining the second altitude. The first method is simpler: manual setting. The second altitude is generally set between 50-100 meters, depending on the different operating areas. For example, when operating in orchards, the drone's operating altitude is generally low and the fruit trees are relatively uniform in height, so the second altitude should be set relatively low, generally around 30 meters. If the operating area is in an urban area, where the height of the area varies greatly and the height of the buildings is uneven, the second altitude needs to be set higher, generally around 100 meters. The second method is more complex. It uses data from a precise original terrain map and performs convolution training to obtain the most suitable second altitude. The second altitude obtained through this method varies depending on the terrain. For example, in a cliff terrain, the second altitude can be divided into two parts based on the cliff edge: the first part is the second altitude above the cliff, and the second part is the second altitude below the cliff. If the drone's operating area is a steep slope, the second altitude obtained through training can be tilted according to the slope of the terrain to achieve the most suitable second altitude.

[0081] The first altitude is higher than the second altitude, and the second altitude is definitely higher than the altitude at which the drone usually operates.

[0082] Step S2: Divide and sort the blurred original image in matrix form to obtain the original image set of blurred regions;

[0083] In this embodiment of the invention, segmenting the blurred original image in matrix form refers to segmenting a large area into a matrix of squares arranged side by side. The segmented images are called blurred original images. The images are sorted from left to right and top to bottom according to the segmentation matrix. The set of sorted images is called the blurred original image set. The purpose is to reduce the detection range of a single area, reduce the detection difficulty, and also allow multiple UAVs to detect different areas simultaneously without affecting each other, thereby improving detection efficiency.

[0084] As an example of the present invention, reference is made to Figure 5 The image shown is a schematic diagram of a blurred original image segmentation method for an example. The content described in this example includes:

[0085] S501: This refers to the blurred original image obtained in step S1 based on the first altitude.

[0086] S502: This refers to the original image of the blurred region obtained by segmenting the blurred original image in step S2.

[0087] according to Figure 5The dashed lines represent the dividing lines of the blurred original image, and the solid lines represent the edge regions of the blurred original image. Each small region has a number used to sort the small regions, for example... Figure 5 The sorting sequence numbered 1-12 is used to form a set of original blurred regions, which is called the blurred region original image set. The size and number of segments are different for different types of blurring. The number can be manually set as needed, or a more suitable blurred region original image can be obtained by calculating based on the size of the blurred original image and the terrain features according to the number obtained through training.

[0088] The blurring of the original image does not necessarily involve cutting in a straight line; it can also be done based on terrain features. The resulting blurry area original image is not necessarily a shape with neat edges. The blurry original image can also be used to capture and synthesize maps of different shapes according to the required work area.

[0089] Step S3: Take a panoramic photo of the geographic area to which the original image of the blurred area belongs to obtain a second photo image, and obtain a second keyframe set and a height information set from the second photo image;

[0090] Specifically, panoramic images are taken at multiple locations based on the geographical area range of each fuzzy region in the original image set, resulting in a second image. Each location includes altitude and latitude / longitude. Keyframes are extracted from the second image to obtain a second keyframe set and an altitude information set.

[0091] In this embodiment of the invention, the geographical region to which the original image of the blurred area belongs refers to, as shown in the reference... Figure 4 The original images of the blurred areas at positions 1-12, such as the entire range of the blurred area at position 1, are captured in panoramic shots from multiple locations. Each location includes altitude and latitude / longitude. Shooting at different altitudes ensures that the drone obtains different feature points at different altitudes within the same location. To improve the accuracy of matching at different altitudes, it is necessary to detect different altitudes. Similarly, shooting at different locations also improves the accuracy of matching at different locations. The panoramic shooting refers to 360-degree shooting from the same location without blind spots, which can better collect information data clearly. The captured images are called low-angle captured images. The second captured image keyframe extraction involves performing a frame segmentation operation on the second captured image to form an image frame set. Images that meet the set clarity requirements are selected through a convolution operation using a custom convolution kernel to obtain the initial keyframe set. The initial keyframes that meet the set resemblance requirements are then deduplicated through a convolution operation to obtain the second keyframe set. The altitude information set refers to the altitude corresponding to the shooting, so there is a correspondence between altitude and keyframes in the keyframe set, which can be one-to-one or one-to-many.

[0092] The sharpness can be set to 0.90 or a preset value can be obtained through training. The similarity can be set to 0.4 or a preset value can be obtained through training.

[0093] Step S4: Extract features from the second keyframe set to obtain a region feature map set, and aggregate the region feature map set to obtain the accurate region original map set;

[0094] Specifically, a second set of keyframes and a set of height information are sent to the server so that the server can perform feature extraction based on the second set of keyframes to obtain a regional feature map. The regional feature map is then aggregated to build a three-dimensional data model, thereby obtaining an accurate original regional map.

[0095] In this embodiment of the invention, sending to the server refers to the reconnaissance drone extracting keyframes from the captured images and sending them to the server. The reconnaissance drone operates by capturing, processing, and sending images simultaneously, performing parallel work and continuously transmitting small amounts of data to save transmission time. Feature extraction of the second keyframe set refers to extracting features from each keyframe in the keyframe set. The feature extraction method involves performing a convolution operation on the keyframes using a set convolution kernel to obtain corresponding feature maps, which are named region feature maps. The set of region feature maps is called the region feature map set. Aggregating the region feature map set refers to aggregating region feature maps at the same location and height. All results of the aggregation operation are used to build a three-dimensional data model, and the generated model is called the accurate region original map. The set of accurate region original maps is called the accurate region original map set.

[0096] The convolution operation extracts feature maps by performing a convolution operation on the input image based on the convolution kernel obtained from training, and the number of convolution layers is set according to the required accuracy.

[0097] The relationship between the aggregation operation and the result of the feature maps of the same location and the same height is that the feature map of the region is a single feature value, and the result of the aggregation is all the feature values ​​of that location.

[0098] The three-dimensional data model operation refers to constructing complex geometric scenes through a series of geometric operations, such as translation, rotation, stretching, and Boolean operations. The construction of three-dimensional models mainly includes geometric modeling, behavioral modeling, physical modeling, object characteristic modeling, and model segmentation. Among these, the creation and description of geometric modeling is the key to virtual scene modeling. Based on the original two-dimensional topological data structure, the topological relationships between four geometric elements (nodes, arcs, edges, and faces) and their topological relationships with four geometric targets (points, lines, faces, and volumes) are defined, and the topological relationships between points and volumes, lines and volumes, points and faces, and lines and faces are explicitly expressed.

[0099] As an example of the present invention, reference is made to Figure 8 The image shown is a schematic diagram of the feature map aggregation method for an example. The content described in this example includes:

[0100] S801-S803: These represent the region feature maps extracted from the keyframes, respectively.

[0101] S804: Represents the original map of the region obtained by aggregation operations and building a three-dimensional data model;

[0102] In the image shown, the irregular curves represent the extracted features, the arrows represent aggregation operations, and the set formed by S801-S803 is called the region feature map set.

[0103] The region feature map set formed by S801-S803 is just one example. The number of region feature maps in the region feature map set is obtained from key frame extraction.

[0104] The edges of the original image in the precise region can be irregular shapes with curves.

[0105] Step S5: Calculate based on the original map set of the precise region and the height information set to obtain the set of regional feature points. Mark the set of regional feature points on the original map of the precise region to obtain the original map set of precise regional features.

[0106] Specifically, based on the precise region original map set and the height information set, the height information from the relevant height information set is introduced into each precise region original map in the precise region original map set to calculate the region feature point set. The region feature point set is then marked on the precise region original map to obtain the precise region feature original map set.

[0107] In this embodiment of the invention, the calculation of the relevant height information refers to calculating the feature point set of each position by adding the height information of the position to the original image of the precise region in the original image set. The sum of the feature point sets obtained after calculating all positions in the region is called the region feature point set. The marking of the region feature point set onto the original image of the precise region refers to marking the feature points corresponding to each position on the original image of the precise region onto the extraction position according to the feature point calculation. The feature point sets corresponding to different positions are different.

[0108] The feature point extraction method involves grayscale conversion; Gamma correction is used to standardize the color space of the input image to adjust the image contrast, reduce the impact of local shadows and lighting changes, and suppress noise interference; the gradient of each pixel is calculated to capture contour information and further weaken lighting interference; the image is divided into small pixel matrices; the gradient histogram of each pixel matrix is ​​calculated to form the pixel features of each pixel matrix; several pixel matrices are combined into a larger pixel matrix, and the pixel features of all pixel matrices within the larger pixel matrix are concatenated to obtain the pixel features of the larger pixel matrix; the pixel features of all larger pixel matrices in the target image are concatenated to obtain the pixel features of the target image; this is the final feature vector that can be used for classification. Height information is added to the feature vector to obtain the position feature vector, which is used to obtain feature points.

[0109] Step S6: Perform boundary fusion operation on the original map set of precise regional features to obtain the original map of precise terrain.

[0110] Specifically, the boundary fusion operation is performed on the original map set of precise regional features to obtain the precise original terrain map, and the precise original terrain map is stored in the guidance system of the UAV.

[0111] In this embodiment of the invention, the boundary fusion operation of the original map set of precise regional features refers to sorting the original map set of precise regional features one by one according to the sorting order of the original map set of fuzzy regions, extracting the overlapping areas of the boundaries of the original map set of precise regional features according to the sequence, and performing fusion operation and boundary smoothing transition operation. The original map set of precise regional features is fused to form a complete map, which is called the precise topographic map.

[0112] The boundary smoothing transition operation refers to adjusting the faults that may occur after the boundary values ​​are merged, so that the original image of the precise regional features is smoothly linked together.

[0113] Step S7: Obtain the third captured image, atmospheric transparency value and light intensity value, calculate the first weight value and the second weight value using the atmospheric transparency value and light intensity value, and then compare them to obtain the maximum weight.

[0114] Specifically, the location of the drone is measured on-site to obtain a third captured image, atmospheric visibility value, and light intensity value. A first weight value is calculated based on the atmospheric visibility value and a preset visibility value. A second weight value is calculated based on the light intensity value and a preset light intensity value. The first weight value and the second weight value are compared to obtain the maximum weight. The maximum weight is used to select and switch the captured image type.

[0115] In this embodiment of the invention, the on-site measurement of the drone's location refers to the drone measuring its altitude and capturing panoramic images as its operational position changes. This is a prerequisite for accurate real-time positioning. The on-site measurement involves obtaining altitude information through a visual odometry system, capturing panoramic images (referred to as the third captured image) using the visual odometry system, measuring the light intensity at the real-time location using a light intensity measuring instrument, and obtaining atmospheric visibility by measuring atmospheric transparency in real time using an atmospheric detector. The preset visibility value is a visibility value set manually or obtained through training. The first weight value is the difference between the preset visibility value and the atmospheric visibility value. The deviation of this difference is calculated, and a corresponding threshold is matched based on the deviation. This threshold is called the first weight value. The preset light intensity value refers to the light intensity value obtained through manual setting or training. The second weight value refers to the difference obtained by comparing the preset light intensity value with the light intensity value. The deviation of the difference is calculated, and a corresponding threshold is matched according to the deviation. This threshold is called the second weight value. The maximum weight refers to the maximum value obtained by comparing the second weight value with the first weight value. Selecting and switching the shooting image type with the maximum weight means that when the maximum weight is the light intensity value but is lower than the preset light intensity value, the shooting image is converted from the default optical image to an infrared image. When the light intensity value is higher than the preset light intensity value, the default optical image is maintained. When the maximum weight is the atmospheric visibility value but is lower than the preset atmospheric visibility value, the default optical image is maintained. When the atmospheric visibility value is higher than the preset atmospheric visibility value, the shooting image is converted from the default optical image to an infrared image.

[0116] The preset light intensity value can be set to 0.5 lx or can be obtained through training. The preset atmospheric visibility value can be set to 300 meters or can be obtained through training.

[0117] In this embodiment of the invention, the atmospheric transparency value is used, for example, when there is heavy fog in the drone's operating area, to ensure the drone can operate normally, infrared images can be taken to obtain clear images in foggy weather; to maintain normal operation, the drone needs to detect the atmospheric transparency and adjust the captured images accordingly. When low visibility weather occurs, the captured images are switched to infrared images; for example, when there is a sandstorm in the drone's operating area, infrared images can be taken to obtain clear images in sandstorm weather; for example, when there is rain or snow in the drone's operating area, infrared images can be taken to obtain clear images in rain or snow weather.

[0118] In this embodiment of the invention, the light intensity value is used, for example, to convert the captured image to a clearer image when the light intensity gradually weakens during the alternation of day and night in the drone's operating area; for example, when the light intensity is weak due to overcast skies in the drone's operating area, resulting in unclear optical images, the image is converted to infrared images to ensure the drone can operate normally; for example, when a total solar eclipse occurs in the drone's operating area, causing a significant change in light intensity, infrared images are used to capture clear images during the total solar eclipse to ensure the drone can operate normally.

[0119] Step S8: Calculate and extract keyframes based on the third captured image to obtain the height information of the third captured image and the keyframes measured on-site;

[0120] Specifically, the height information of the third captured image is calculated based on the third captured image, and keyframes are extracted from the third captured image to obtain the on-site measurement keyframes.

[0121] In this embodiment of the invention, calculating the height information of the third captured image based on the third captured image refers to calculating the height of the image captured by the multi-view camera based on the visual odometry. Here, the captured image is called the third captured image. Extracting keyframes from the third captured image refers to performing a frame segmentation operation on the image to form an image frame set, feeding it into a convolution operation with a custom convolution kernel to filter images that meet the set sharpness requirements to obtain an initial keyframe set, and then performing deduplication processing on the initial keyframes that meet the set similarity requirements through a convolution operation to obtain the field measurement keyframe set.

[0122] The sharpness can be set to 0.90 or a preset value can be obtained through training. The similarity can be set to 0.4 or a preset value can be obtained through training.

[0123] Step S9: Measure keyframes in the field to obtain a panoramic image through calculation. Perform convolution operation on the panoramic image to obtain a real-time feature map set. Perform aggregation operation on the real-time feature map set to obtain a real-time terrain map.

[0124] Specifically, a panoramic image is obtained by calculating keyframes measured in the field and using the PP-MSVSR algorithm. The panoramic image is then used for convolution to obtain a real-time feature map set. Finally, the real-time feature map set is aggregated to obtain a real-time terrain map.

[0125] In this embodiment of the invention, the PP-MSVSR algorithm calculation refers to adjusting the pixels of the field measurement keyframes according to the PP-MSVSR algorithm and aligning them with the original accurate terrain image. The aligned field measurement keyframes are called panoramic images, making the feature maps extracted from the keyframes more accurate. The convolution operation using the panoramic image refers to extracting features from each keyframe in the keyframe set. The feature extraction method is to perform a convolution operation on the keyframes using a set convolution kernel to obtain the corresponding feature maps, which are named real-time feature maps. The set of feature maps is called the real-time feature map set. The aggregation operation of the real-time feature map set refers to using a depthwise convolution feature aggregation framework to aggregate the real-time features within the feature map set. Figure 1 Once an aggregation operation is performed, the resulting aggregation result is called a real-time topographic map.

[0126] As an example of the present invention, reference is made to Figure 8 The image shown is a schematic diagram of the feature map aggregation method for an example. The content described in this example includes:

[0127] S801-S803: These represent the real-time feature maps extracted from keyframes, respectively.

[0128] S804: Represents a real-time topographic map obtained through aggregation operations;

[0129] In the image shown, the irregular curves represent the extracted features, the arrows represent aggregation operations, and the set formed by S801-S803 is called the real-time feature map set.

[0130] The regional feature map set formed by S801-S803 is just one example. The number of regional feature maps in the real-time feature map set is obtained from key frame extraction.

[0131] Step S10: The height information of the third captured image is used to calculate the real-time topographic map, obtain real-time feature points, and mark the real-time feature points on the real-time topographic map to obtain the real-time feature topographic map.

[0132] Specifically, based on the height information of the third captured image, real-time terrain maps are calculated to obtain real-time feature points, which are then marked on the real-time terrain map to obtain a real-time feature map.

[0133] In this embodiment of the invention, the calculation of the real-time terrain map by introducing the height information of the third captured image means that the real-time terrain map is calculated based on the height information, and the resulting feature points are called real-time feature points. Marking the real-time feature points on the real-time terrain map means that the map formed at the newly marked position based on the extracted position of the real-time feature points on the real-time terrain map is called the real-time feature terrain map.

[0134] The feature point extraction method involves grayscale conversion; Gamma correction is used to standardize the color space of the input image to adjust the image contrast, reduce the impact of local shadows and lighting changes, and suppress noise interference; the gradient of each pixel is calculated to capture contour information and further weaken lighting interference; the image is divided into small pixel matrices; the gradient histogram of each pixel matrix is ​​calculated to form the pixel features of each pixel matrix; several pixel matrices are combined into a larger pixel matrix, and the pixel features of all pixel matrices within the larger pixel matrix are concatenated to obtain the pixel features of the larger pixel matrix; the pixel features of all larger pixel matrices in the target image are concatenated to obtain the pixel features of the target image; this is the final feature vector that can be used for classification. Height information is added to the feature vector to obtain the position feature vector, which is used to obtain feature points.

[0135] Figure 1 The uppercase A in Figure 2 In this context, the uppercase A indicates that the same uppercase A is used in a link. Figure 1 and Figure 2 The node.

[0136] Reference Figure 2 The above is a schematic diagram of another part of the steps of an image matching-based UAV navigation method according to the present invention. In this example, the image matching-based UAV navigation method includes:

[0137] Step S21: Match and filter the real-time feature map and the original precise map multiple times to obtain the highest matching degree. Compare the highest matching degree with the expected value set to obtain the region with the highest similarity. Reset this region as the current flight area of ​​the UAV and obtain the current precise position within the original precise map.

[0138] Specifically, the real-time feature map and the original precise map are matched multiple times using a preset matching order to obtain multiple sets of matching scores. The matching scores are then filtered to obtain the highest matching score. The highest matching score is compared with the expected value set to obtain the region corresponding to the matching score set with the highest similarity. This region is then reset as the current flight area of ​​the UAV. The current precise position of the UAV is obtained within the original precise map. The current precise positions that cannot be matched are sent back to the server for processing.

[0139] In this embodiment of the invention, the three preset matching orders are: (1) matching by gradually expanding the range according to the position of the precise terrain map corresponding to the GPS navigation positioning; (2) matching by matrix order of the precise terrain map in the absence of a network; (3) sorting the feature map matching order from high to low according to the feature point matching degree; the matching order (1) is a matching method based on GPS navigation for navigation operations, where the location of the GPS navigation positioning information is reconstructed on the precise terrain map, and the position of the UAV on the precise terrain map is determined by gradually expanding the range according to the location; the matching order (2) is a matching method based on the UAV working within the range of the precise terrain map without the support of other navigation systems. Figure 3 The matching method is to match the regions in the order shown. The matching order (3) is to sort the feature points of the first two matching methods from high to low and match them according to the sorting order.

[0140] In this embodiment of the invention, performing multiple matches refers to selecting whether to perform other matching methods and rematch based on the matching degree. The expected value set refers to setting a critical value for the matching degree at different stages of matching. This critical value is called the expected value, and the set of expected values ​​is called the expected value set. Comparing the highest matching degree with the expected value set means comparing the matching degree of the expected value set corresponding to the matching degree set with the highest matching degree value in the matching degree set. If the matching degree is lower, the matching method is switched; if it is higher, the position with the highest current matching degree is determined. Resetting the area as the current flight area of ​​the UAV means marking the position obtained from the matching degree through the matching degree of the expected value standard on the accurate terrain map as the current position of the UAV. If the position under GPS navigation deviates from the current position on the accurate terrain map, the position of GPS navigation is set to the current position on the accurate terrain map.

[0141] In this embodiment of the invention, reference is made to Figure 3 As shown, the step of performing multiple matches on the current position based on the matching degree includes:

[0142] Step S301: Perform the first matching between real-time feature points and regional feature points according to the matching order to obtain the first set of matching degrees;

[0143] Specifically, under the condition of the same height information, the real-time feature points and regional feature points are matched for the first time according to the matching order (1) or (2), and the maximum likelihood estimation algorithm is used to match the real-time feature points on the real-time topographic map with the regional feature points on the accurate topographic original map to obtain the first set of matching degree.

[0144] In this embodiment of the invention, "under the same height information" means comparing the feature points and feature maps at the height of the current position during operation with the corresponding height of the precise topographic map. The feature points and feature maps obtained at different heights are different. In order to accurately match the position of the current position on the precise topographic map, it is necessary to select matching feature points and feature maps that match the current height on the precise topographic map for matching. "The first matching of real-time feature points and regional feature points according to the matching order (1) or (2)" means that navigation operations can be performed whether there are other auxiliary navigation methods or no other navigation methods, only the matching order is adjusted. When there is auxiliary navigation, the matching order (1) is used. Line matching, when there is no auxiliary navigation, the matching order (2) is used for matching. The first matching refers to the first positioning matching when the UAV performs the first positioning matching for the current position. The maximum likelihood estimation algorithm performs similarity matching between the real-time feature points on the real-time terrain map and the regional feature points on the precise terrain map. This means comparing the real-time feature points calculated using the current position with the regional feature points at the same height on the precise terrain map one by one. Since there are more than one feature point at each position, the comparison of the feature points at each position will result in a similarity value. The set of similarity values ​​obtained by matching the same height position in the precise terrain map is called the matching degree set. Since it is the first matching, it is called the first set of matching degree sets.

[0145] Step S302: After matching the entire image, if there is no match in the first set of matching degrees that is higher than the first expected value, compare the highest match degree in the first set of matching degrees with the second expected value, and determine the matching point with the highest match degree as the current position;

[0146] Specifically, when the first matching set does not have a full map match with a matching degree higher than the first expected value during the first matching, the highest matching degree in the first matching set is obtained, and the matching point with the highest matching degree higher than the second expected value is determined as the current position on the accurate original topographic map.

[0147] In this embodiment of the invention, the first expected value refers to a threshold set artificially by the first matching or a threshold obtained through training, which is used as a standard line to determine whether a match has passed. The phrase "when the first matching is performed, the first set of matching degrees does not have a full map matching value higher than the first expected value" means that when the position matching degree on the accurate original topographic map is obtained, the matching is performed while comparing with the first expected value. After the matching is completed, the comparison is directly performed with the second expected value. The second expected value refers to a threshold set artificially by the highest value obtained after the first matching or a threshold obtained through training, which is used as a standard line to determine whether a match has passed.

[0148] Step S303: When the highest matching degree is lower than the second expected value, perform a second matching between the real-time feature map and the region feature map according to the matching order to obtain the second set of matching degrees;

[0149] Specifically, when the highest matching degree is lower than the second expected value, the real-time feature map and the regional feature map are matched for the second time according to the matching order (3), and the real-time feature map and the regional feature map are matched for similarity using the maximum likelihood estimation algorithm to obtain the second set of matching degree sets.

[0150] In this embodiment of the invention, the second matching of the feature map and the regional feature map according to the matching order (3) means that since the first matching with feature points failed, a second matching is required. The similarity matching of the feature map and the regional feature map using the maximum likelihood estimation algorithm means that the second matching uses the feature map calculated from the position before the point to compare with the regional feature maps at all positions of the same height on the original precise topographic map. Each regional feature map will get a similarity when compared with the feature map. The similarity set obtained after matching all positions of the same height on the original precise topographic map is called the matching degree set. Since it is the second matching, it is called the second set of matching degree sets.

[0151] Step S304: When the first matching set has a matching degree higher than the first expected value during the first matching, stop matching immediately and determine the stop matching point as the current position on the accurate original topographic map.

[0152] In this embodiment, the first matching refers to the first positioning matching performed by the UAV at the current location. The first expected value refers to the threshold set by the UAV in the expected value set during the first matching or the threshold obtained through training, which is used as the standard line to determine whether a matching is successful. The phrase "stop matching immediately when the first set of matching degrees is higher than the first expected value" means that when the position matching degree on the accurate terrain map is obtained, the matching is compared with the first expected value while matching. If the matching degree is higher than the first expected value, the matching is stopped. The phrase "determine the stop matching point as the current location on the accurate terrain map" means that the location with the highest matching degree is set as the current location of the UAV.

[0153] Step S305: Obtain the highest matching degree in the second set of matching degrees, and determine the matching point on the accurate original topographic map where the highest matching degree is higher than the third expected value as the current position.

[0154] In this embodiment, the third expected value refers to the threshold set artificially by the second matching in the expected value set or the standard line used to distinguish whether a match has passed by training.

[0155] Step S306: If the highest matching degree is lower than the third expected value, perform a third matching on the real-time feature points and regional feature points according to the matching order to obtain the third set of matching degrees;

[0156] Specifically, if the highest matching degree is lower than the third expected value, the UAV autonomously increases its flight altitude to the second altitude to obtain scattered real-time feature points on the real-time terrain map. According to the matching order (1) or (2), the real-time feature points and regional feature points are matched for the third time. The maximum likelihood estimation algorithm is used to match the real-time feature points on the real-time terrain map with the regional feature points on the accurate terrain map to obtain the third set of matching degrees.

[0157] In this embodiment, the third expected value refers to the threshold set by the artificial during the second matching or the threshold obtained through training, which is used as the standard line for determining whether a match has been passed. The drone autonomously raising its flight altitude to the second altitude means that, as shown in Figure (4), S401 refers to the general drone operation altitude, S402 refers to the second altitude, and S403 refers to the first altitude. When the first two matching attempts fail, it indicates that the area has undergone significant changes, such as an earthquake causing some houses to collapse and the feature points and feature maps to change drastically, a flood causing city streets to be submerged and the feature points and feature maps below to change drastically, or the current area being artificially modified for years due to navigation operations, causing the feature points and feature maps to change drastically. In such cases, the drone cannot successfully match at the general operation altitude and raises its altitude to the second altitude for the third matching. Because the second altitude is higher than the general drone operation altitude, the information collected is broader and more complete, making it easier to match successfully. The acquisition of scattered real-time feature points on the real-time terrain map refers to the acquisition of feature points at a higher position. The feature points of the region at the second altitude on the precise original topographic map are extracted and matched. The feature points that are successfully matched are scattered feature points. The third matching of real-time feature points and regional feature points according to the matching order (1) or (2) means that navigation can be performed whether there are other auxiliary navigation methods or no other navigation methods. Only the matching order is adjusted. When there is auxiliary navigation, the matching order (1) is used for matching. When there is no auxiliary navigation, the matching order (2) is used for matching. The similarity matching of real-time feature points on the real-time topographic map and regional feature points on the precise original topographic map using the maximum likelihood estimation algorithm means that the real-time feature points calculated at the current position are compared one by one with the regional feature points at the second altitude on the precise original topographic map. Since there are more than one feature point at each position, the comparison of feature points at each position will result in a similarity value. The set of similarity values ​​obtained by matching the same altitude position input from the precise original topographic map is called the matching degree set. Since it is the third matching, it is called the third set of matching degree sets.

[0158] As an example of the present invention, reference is made to Figure 4The image shown is a height comparison diagram of an example drone-captured image. The content described in this example includes:

[0159] S401: This indicates the normal operating altitude of the drone. The area within the elliptical dashed line below the drone is the normal shooting range of the drone.

[0160] S402: This refers to the altitude at which the drone is repositioned when it is unable to accurately locate its current position at its normal operating altitude. This altitude is called the second altitude.

[0161] S403: This refers to the height at which the work area is captured in a general overview, known as the first height.

[0162] There are two methods for obtaining the second altitude. The first method is simpler: manual setting. The second altitude is generally set between 50-100 meters, depending on the different operating areas. For example, when operating in orchards, the drone's operating altitude is generally low and the fruit trees are relatively uniform in height, so the second altitude should be set relatively low, generally around 30 meters. If the operating area is in an urban area, where the height of the area varies greatly and the height of the buildings is uneven, the second altitude needs to be set higher, generally around 100 meters. The second method is more complex. It uses data from a precise original terrain map and performs convolution training to obtain the most suitable second altitude. The second altitude obtained through this method varies depending on the terrain. For example, in a cliff terrain, the second altitude can be divided into two parts based on the cliff edge: the first part is the second altitude above the cliff, and the second part is the second altitude below the cliff. If the drone's operating area is a steep slope, the second altitude obtained through training can be tilted according to the slope of the terrain to achieve the most suitable second altitude.

[0163] The first altitude is higher than the second altitude, and the second altitude is definitely higher than the altitude at which the drone usually operates.

[0164] Step S307: If the highest matching degree is lower than the fourth expected value, perform four matching operations on the real-time feature map and the region feature map according to the matching order to obtain the fourth set of matching degrees;

[0165] Specifically, if the highest matching degree is lower than the fourth expected value, the real-time feature map and the regional feature map are matched four times according to the matching order (1) or (2), and the real-time feature map and the regional feature map are matched for similarity using the maximum likelihood estimation algorithm to obtain the fourth set of matching degrees.

[0166] In this embodiment, the fourth expected value refers to the threshold set by the third matching or the threshold obtained through training in the expected value set as the standard line for determining whether a match has been passed. The four-time matching of the feature map and the regional feature map according to the matching order (1) or (2) means that navigation can be performed whether there are other auxiliary navigation methods or no other navigation methods. Only the matching order is adjusted. When there is auxiliary navigation, the matching order (1) is used for matching. When there is no auxiliary navigation, the matching order (2) is used for matching. The similarity matching of the feature map and the regional feature map using the maximum likelihood estimation algorithm means that the fourth matching uses the feature map calculated from the point position to compare with the regional feature map of all positions at the second altitude on the precise topographic original map. Each regional feature map will get a similarity when compared with the feature map. The similarity set obtained after matching all positions at the second altitude on the precise topographic original map is called the matching degree set. Since it is the fourth matching, it is called the fourth matching degree set.

[0167] Step S308: Obtain the highest matching degree in the third set of matching degrees, and determine the matching point on the accurate original topographic map where the highest matching degree is higher than the fourth expected value as the current position.

[0168] In this embodiment, the fourth expected value refers to the threshold set artificially by the third matching in the expected value set or the threshold obtained through training, which is used as the standard line to distinguish whether a match has passed.

[0169] Step S309: If the high match degree is lower than the fifth expected value, send it back to the console for processing.

[0170] In this embodiment, the fifth expected value set is used to determine the standard line for whether a match is passed by either a threshold set manually by the fourth match or a threshold obtained through training. The "send back to console processing" means that if the UAV still cannot determine its current position after multiple matches, it needs to send the analysis information back to the console for manual processing.

[0171] Step S310: Obtain the highest matching degree in the fourth set of matching degrees, and determine the matching point on the accurate original topographic map where the highest matching degree is higher than the fifth expected value as the current position.

[0172] In this embodiment, the fifth expected value set is used as a standard line to determine whether a match has passed, either by a threshold set artificially during the fourth match or by a threshold derived through training.

[0173] The first expected value can be set to 0.95 or obtained through training; the second expected value can be set to 0.90 or obtained through training; the third expected value can be set to 0.80 or obtained through training; the fourth expected value can be set to 0.70 or obtained through training; and the fifth expected value can be set to 0.60 or obtained through training.

[0174] Step S22: Receive the original target location set on the precise terrain map, and generate a flight target navigation map based on the current precise location;

[0175] Specifically, it receives the flight path set on the precise original terrain map, generates a flight path navigation map based on the current precise location, and performs autonomous navigation flight.

[0176] In this embodiment of the invention, receiving the flight path set on the precise terrain map refers to the flight path set on the precise terrain map of the server. The server sends the flight path information to the UAV. The UAV receives the flight path information and generates flight path information on the UAV's precise terrain map based on the flight path information. Compared with navigation map plus navigation route, the amount of data transmitted by transmitting flight path information is greatly reduced, thereby speeding up the transmission efficiency and reducing the transmission difficulty. The autonomous navigation flight refers to first comparing the current position with the starting point of the flight path. When a deviation occurs, the UAV performs a return navigation operation by autonomously generating a flight navigation route based on the position before the point and the position of the starting point, and then performs autonomous navigation operation based on the flight path.

[0177] As an example of the present invention, reference is made to Figure 6 The diagram shown is a flight path navigation illustration for an example. In this example, the content includes:

[0178] S601: Represents the original, precise topographic map;

[0179] S602: Represents the flight path generated on the accurate original terrain map;

[0180] S603: Indicates the starting point of the flight path;

[0181] S604: This represents a regression flight navigation path generated based on the distance between the current precise location and the starting point of the flight path;

[0182] S605: Indicates the current precise location;

[0183] The S604 UAV can autonomously generate a navigation path S604 based on a precise original terrain map.

[0184] Step S23: Receive the flight path set on the precise terrain map, and generate a flight path navigation map based on the current precise location;

[0185] Specifically, it receives the original target location set on the precise terrain map, generates a flight target navigation map based on the current precise location, and conducts autonomous navigation flight.

[0186] In this embodiment of the invention, the original target location set on the precise topographic map refers to the original target location set on the precise topographic map of the server. The server sends the original target location information to the UAV. The UAV receives the original target location information and generates a precise target location on the UAV's precise topographic map based on the original target location information. By using the original target location information for transmission, the amount of data transmitted is greatly reduced compared with navigation map plus navigation route, thereby speeding up the transmission efficiency and reducing the transmission difficulty. The autonomous navigation flight refers to generating the optimal navigation path based on the current position and the position of the precise target location and performing autonomous navigation operation.

[0187] As an example of the present invention, reference is made to Figure 7 The diagram shown is a schematic representation of flight target navigation in an example. The content described in this example includes:

[0188] S701: Represents the original, precise topographic map;

[0189] S702: Indicates the original target location;

[0190] S703: Indicates a navigation route generated based on the original target location and the current precise location;

[0191] S704: Indicates the current precise location;

[0192] The S703 UAV can autonomously generate its own navigation path S703 based on a precise original topographic map.

[0193] Step S24: Generate a navigation path set. Analyze the navigation path set to generate an overlapping navigation area. Analyze the overlapping navigation area to obtain corrected feature points and eliminated feature points. Based on the corrected feature points and eliminated feature points, perform correction operations on the accurate original topographic map.

[0194] Specifically, the process involves analyzing the current flight area of ​​the UAV to generate a navigation path set; analyzing the navigation path set and generating an overlapping navigation area when the number of times an overlapping area appears in the navigation path set reaches an overlap threshold; analyzing the overlapping navigation area and counting the occurrences of each real-time feature point that belongs to the real-time feature point set but not to the regional feature point set, marking real-time feature points whose occurrences reach a first threshold as corrected feature points; counting the occurrences of each real-time feature point that belongs to the regional feature point set but not to the real-time feature point set, marking real-time feature points whose occurrences reach a second threshold as eliminated feature points; and using the corrected and eliminated feature points, refining the original accurate terrain map to obtain a more accurate original terrain map.

[0195] In this embodiment of the invention, the generated navigation path set refers to the set of historical paths used for navigation operations on the same precise topographic map. Each navigation operation stores the navigation path in the navigation path set for use in correcting the original precise topographic map. The navigation overlap region refers to the region traversed multiple times during navigation. The overlap threshold is a value set manually or trained to indicate a certain number of overlaps. The generated overlapping navigation region means that when a region reaches the required number of navigation overlaps, it is designated as an overlapping navigation region. This region is then processed to make it more suitable for navigation on the original precise topographic map. The first threshold is a value indicating the number of occurrences. The feature point set that does not exist in the regional feature point set refers to newly appearing feature points obtained by comparing the calculated feature point set of the current region with the regional feature point set of the region on the original precise topographic map. When these newly appearing feature points are stable and their stable occurrence count is higher than the first threshold, they are marked as corrected feature points. The second threshold is a value indicating the number of disappearances. The calculation... The occurrence count of each real-time feature point in the regional feature point set that does not exist in the real-time feature point set is defined as follows: real-time feature points whose occurrence count reaches a second threshold are marked as eliminated feature points. This refers to disappearing feature points obtained by comparing the calculated feature point set of the current region with the regional feature point set of the same region on the accurate topographic original map. When a feature point disappears stably, and the number of stable disappearances is higher than the second threshold, the feature point is marked as eliminated feature point. The first threshold and the second threshold are used to distinguish between corrected feature points and disappeared feature points. Their values ​​are generally equal, but different values ​​can be obtained based on training. The correction operation of the accurate topographic original map refers to searching for relevant feature maps based on the corrected feature points and replacing the feature maps with the corresponding regional feature maps to achieve adjustment operations on the accurate topographic original map. Similarly, searching for relevant feature maps based on the disappeared feature points and replacing the feature maps with the corresponding regional feature maps to achieve adjustment operations on the accurate topographic original map, making the accurate topographic original map more suitable for the current working area.

[0196] The number of overlaps can be set to 15 or obtained through training; the first threshold can be set to 10 or obtained through training; and the second threshold can be set to 10 or obtained through training.

[0197] The first threshold and the second threshold are unrelated; they are named to distinguish the two thresholds.

[0198] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0199] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0200] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. An image matching based navigation method for unmanned aerial vehicles, characterized in that, Applied to drones, the method includes the following steps: Step S1: Use visual odometry to take and measure the operating range of the UAV at the first altitude to obtain the first captured image. Extract keyframes from the first captured image to obtain the first keyframe set. Send the first keyframe set to the server so that the server can stitch the images together to obtain the blurred original image. Step S2: Divide and sort the blurred original image in matrix form to obtain the original image set of blurred regions; Step S3: Perform panoramic photography at multiple locations based on the geographical area range of each fuzzy region original image in the fuzzy region original image set to obtain a second captured image, where each location includes altitude and latitude and longitude. Extract keyframes from the second captured image to obtain a second keyframe set and an altitude information set. Step S4: Send the second keyframe set and height information set to the server so that the server can perform feature extraction based on the second keyframe set, obtain the regional feature map set, aggregate the regional feature map set to build a three-dimensional data model, thereby obtaining the accurate original regional map set; Step S5: Based on the precise region original map set and the height information set, calculate the relevant height information for each precise region original map in the precise region original map set to obtain the region feature point set. Mark the region feature point set on the precise region original map to obtain the precise region feature original map set. Step S6: Perform boundary fusion operation on the original map set of precise regional features to obtain the original map of precise terrain, and store the original map of precise terrain in the guidance system of the UAV. Step S7: Conduct on-site measurements of the drone's location to obtain the third captured image, atmospheric visibility value, and light intensity value. Calculate the first weight value based on the atmospheric visibility value and the preset visibility value. Calculate the second weight value based on the light intensity value and the preset light intensity value. Compare the first weight value and the second weight value to obtain the maximum weight. Use the maximum weight to select and switch the captured image type. Step S8: Calculate the height information of the third captured image based on the third captured image, extract key frames from the third captured image, and obtain the on-site measurement key frames. Step S9: Based on the keyframes measured in the field, a panoramic image is obtained by calculation using the PP-MSVSR algorithm. The panoramic image is then convolved to obtain a real-time feature map set. The real-time feature map set is then aggregated to obtain a real-time terrain map. Step S10: Calculate the real-time terrain map based on the height information of the third captured image to obtain real-time feature points, mark the real-time feature points on the real-time terrain map, and obtain the real-time feature terrain map. Step S11: The real-time feature map and the original precise map are matched multiple times using a preset matching order to obtain multiple sets of matching degree. The matching degree sets are filtered to obtain the highest matching degree. The highest matching degree is compared with the expected value set to obtain the region corresponding to the matching degree set with the highest similarity. This region is reset as the current flight area of ​​the UAV. The current precise position of the UAV is obtained in the original precise map. The current precise positions that cannot be matched are sent back to the server for processing. Step S12: Receive the flight path set on the precise terrain map, generate a flight path navigation map based on the current precise location, and perform autonomous navigation flight; or, receive the original target location set on the precise terrain map, generate a flight target navigation map based on the current precise location, and perform autonomous navigation flight. Step S13: Analyze the current flight area of ​​the UAV to generate a navigation path set; analyze the navigation path set, and when the number of times the navigation overlap area appears in the navigation path set reaches the overlap threshold, generate an overlapping navigation area; analyze the overlapping navigation area, count the number of times each real-time feature point belongs to the real-time feature point set but not to the regional feature point set, and mark the real-time feature points whose occurrence count reaches the first threshold as corrected feature points; count the number of times each real-time feature point belongs to the regional feature point set but not to the real-time feature point set, and mark the real-time feature points whose occurrence count reaches the second threshold as eliminated feature points; based on the corrected feature points and eliminated feature points, perform a correction operation on the original accurate terrain map to obtain a more accurate original terrain map; Step S13, which involves correcting and eliminating feature points to refine the original terrain map and obtain a more accurate original terrain map, includes the following steps: Searching based on the corrected feature points, obtaining the real-time feature map corresponding to the corrected feature points, and replacing the real-time feature map with the corresponding regional feature map, thereby realizing the correction and adjustment operation on the accurate original topographic map; The search is performed based on the disappearing feature points to obtain the real-time feature map corresponding to the disappearing feature points. The real-time feature map is then replaced with the corresponding regional feature map to perform correction and adjustment operations on the accurate original topographic map.

2. The method according to claim 1, characterized in that, Step S4, which involves sending the second keyframe set and height information set to the server so that the server can perform feature extraction based on the second keyframe set to obtain the region feature map, includes the following steps: The second keyframe set and altitude information set are sent to the server so that the server can highlight the feature regions of the keyframes in the second keyframe set based on the second keyframe set returned by the UAV, and use preset spatial weights and channel weights to extract features from different keyframes using convolution operations to obtain a set of regional feature maps.

3. The method according to claim 1, characterized in that, Step S4 involves aggregating the regional feature maps to create a 3D data model, thereby obtaining the accurate original regional map set. This process includes the following steps: A deep convolutional feature aggregation framework is used to aggregate regional feature maps and establish a three-dimensional data model, thereby obtaining an accurate original regional map set.

4. The method according to claim 1, characterized in that, Step S6 specifically includes the following steps: Extract the edge regions from the original precise region feature map set. Then, perform convolution operations on the overlapping or similar regions of the edge regions in the order of the original precise region feature map set to make the edges transition smoothly, thereby achieving the fusion operation and obtaining the accurate original topographic map.

5. The method according to claim 1, characterized in that, Step S9, which involves calculating the panoramic image based on key field measurements using the PP-MSVSR algorithm, includes the following steps: The PP-MSVSR algorithm is used to align the pixels of the field measurement keyframes with the pixels of the accurate original terrain image, thereby obtaining a panoramic image.

6. The method according to claim 1, characterized in that, Step S11 involves multiple matching operations between the real-time feature map and the precise original map using a preset matching order, which can be categorized into three types: (1) Matching is performed by gradually expanding the range based on the location of the original map corresponding to the GPS navigation positioning; (2) In the absence of a network, matching is performed using the matrix order of the accurate original topographic map; (3) Sort the real-time feature map matching order from high to low according to the feature point matching degree; Step S11 includes the following steps: Under the condition of the same height information, the real-time feature points and regional feature points are matched for the first time according to the matching order (1) or (2). The maximum likelihood estimation algorithm is used to match the real-time feature points on the real-time topographic map with the regional feature points on the precise topographic map to obtain the first set of matching degree. When the latest matching degree of the first matching degree set is higher than the first expected value during the first matching, the matching is stopped immediately, and the position of the stop matching point is determined on the accurate original topographic map as the current position; When the first matching does not have a full map match with a matching degree higher than the first expected value in the first matching set, the highest matching degree in the first matching set is obtained, and the matching point with the highest matching degree higher than the second expected value is determined as the current position on the accurate original topographic map. When the highest matching degree is lower than the second expected value, the real-time feature map and the regional feature map are matched for the second time according to the matching order (3), and the real-time feature map and the regional feature map are matched for similarity using the maximum likelihood estimation algorithm to obtain the second set of matching degree sets; Obtain the highest matching degree in the second set of matching degrees, and determine the matching point on the accurate original topographic map where the highest matching degree is higher than the third expected value as the current position; If the highest matching degree is lower than the third expected value, the UAV autonomously raises its flight altitude to the second altitude to obtain scattered real-time feature points on the real-time terrain map. According to the matching order (1) or (2), the real-time feature points and regional feature points are matched for the third time. The maximum likelihood estimation algorithm is used to match the real-time feature points on the real-time terrain map with the regional feature points on the accurate terrain map to obtain the third set of matching degree sets, where the first altitude is higher than the second altitude. Obtain the highest matching degree in the third set of matching degrees, and determine the matching point on the accurate original topographic map where the highest matching degree is higher than the fourth expected value as the current position; If the highest matching degree is lower than the fourth expected value, the real-time feature map and the regional feature map are matched four times according to the matching order (1) or (2), and the real-time feature map and the regional feature map are matched for similarity using the maximum likelihood estimation algorithm to obtain the fourth set of matching degree sets. Obtain the highest matching degree in the fourth set of matching degrees, and determine the matching point on the accurate original topographic map where the highest matching degree is higher than the fifth expected value as the current position; If the highest matching degree is lower than the fifth expected value, the first to fourth matching degree sets will be sent back to the server for processing, with the first, second, third, fourth, and fifth expected values ​​decreasing gradually.

7. The method according to claim 1, characterized in that, Step S12, which involves receiving the flight path set on the precise terrain map, generating a flight path navigation map based on the current precise location, and performing autonomous navigation flight, specifically includes the following steps: Receive the flight path set on the accurate original terrain map, and generate a flight path navigation map on the accurate original terrain map of the UAV based on the flight path; Construct a return flight path based on the current precise location and the starting point of the generated flight path navigation map, and fly to the starting point of the flight path navigation map according to the return flight path to conduct autonomous navigation flight.

8. The method according to claim 1, characterized in that, Step S12, which involves receiving the original target location set on the precise terrain map and generating a flight target navigation map based on the current precise location, includes the following steps: The system receives the original target location set on the precise terrain map, marks the precise target location on the precise terrain map of the UAV based on the original target location, generates a flight target navigation map based on the precise target location, and performs autonomous navigation flight.

Images