An unmanned aerial vehicle autonomous navigation method based on landscape matching

Abstract

The application provides a kind of unmanned aerial vehicle autonomous navigation method based on landscape matching, it is related to the technical field of unmanned aerial vehicle autonomous navigation, including: by collecting real-time and historical ground image, target position information, velocity information is measured using inertial sensor, the first position information is obtained by integrating velocity information processing, then match real-time and historical image to determine the most similar historical image and its position information, then compare two sets of position information to determine whether to update position, and finally based on position information and target position, a local path planning algorithm is used to generate a navigation route, to improve the accuracy of unmanned aerial vehicle positioning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle autonomous navigation, and in particular to an unmanned aerial vehicle autonomous navigation method based on landscape matching. BACKGROUND

[0002] In the rapid development and application of unmanned aerial vehicle technology, autonomous navigation technology has become the key to realizing efficient and accurate operation of unmanned aerial vehicles. However, traditional unmanned aerial vehicle autonomous navigation methods mainly rely on single technical means such as inertial navigation system (INS) or global positioning system (GPS). These methods can usually provide relatively accurate navigation information in open and signal good environments, but in complex environments (such as urban canyons, forests, etc.), due to factors such as signal shielding and multipath effect, the positioning accuracy will often decrease significantly, and even positioning failure may occur.

[0003] Even in a signal good environment, long-term reliance on inertial sensors for position calculation will cause the positioning accuracy to gradually decrease due to the drift of the sensors and the accumulation of integral errors. This accumulation of errors is particularly evident in long-time or long-distance flight missions, seriously affecting the accuracy of unmanned aerial vehicle positioning. SUMMARY

[0004] In order to improve the accuracy of unmanned aerial vehicle positioning, the present application provides an unmanned aerial vehicle autonomous navigation method based on landscape matching.

[0005] The present application provides an unmanned aerial vehicle autonomous navigation method based on landscape matching, which adopts the following technical solution:

[0006] An unmanned aerial vehicle autonomous navigation method based on landscape matching includes the following steps:

[0007] First acquisition: acquiring real-time ground images, historical ground images and target position information, and measuring speed information using an inertial sensor;

[0008] First calculation: integrating the speed information to obtain first position information;

[0009] Image matching: calculating the similarity of the real-time ground image and each historical ground image respectively, obtaining the historical ground image corresponding to the maximum similarity, and recording it as the first ground image;

[0010] Positioning: recording the historical position information carried in the first ground image as second position information;

[0011] First judgment: judging whether the difference between the first position information and the second position information is greater than a preset difference threshold, if yes, executing the step of position updating; if no, executing the step of first planning;

[0012] updating the second position information as the first position information;

[0013] first planning: generating a navigation route based on the first position information and target position information, using a local path planning algorithm.

[0014] The present application can obtain rich environmental data and motion state data by collecting real-time ground images, historical ground images and target position information, and using an inertial sensor to measure speed information. Subsequently, the present application can obtain a first position information based on speed accumulation by integrating the speed information. Subsequently, the present application can find the most similar historical image by calculating the similarity of the real-time ground image and each historical ground image, and obtain the historical position information carried thereby, and then use the historical position information carried thereby as second position information, thereby improving the positioning reliability by using the accuracy of the historical data. Subsequently, the present application can judge the accuracy of the current position information by comparing the difference between the first position information and the second position information, so as to timely discover and correct possible positioning errors. If it is judged that the position information needs to be updated, the second position information is updated as the first position information. Otherwise, a navigation route is generated based on the first position information and target position information, using a local path planning algorithm. The present application combines multiple positioning methods (inertial navigation, image matching), which can complement each other to improve the accuracy and reliability of positioning.

[0015] Optionally, after the step of performing the first calculation, before the step of performing the image matching, further comprising:

[0016] first modeling: constructing a visual bag-of-words model;

[0017] first segmentation: segmenting the historical ground image to obtain a plurality of first segmentation blocks; extracting features in the first segmentation blocks, denoted as first features, integrating the first features into a first feature vector, and integrating the first features into a first feature matrix;

[0018] first construction: inputting the first feature matrix into the visual bag-of-words model to obtain a first visual vector;

[0019] second segmentation: segmenting the real-time ground image to obtain a plurality of second segmentation blocks; extracting features in the second segmentation blocks, denoted as second features, and integrating the second features into a second feature matrix;

[0020] second construction: inputting the second feature matrix into the visual bag-of-words model to obtain a second visual vector;

[0021] In the step of image matching, the first visual vector is used instead of the historical ground image, and the second visual vector is used instead of the real-time ground image.

[0022] The application constructs a visual bag-of-words model, segments the historical ground image, extracts features in each segmented block, integrates the features into a first feature matrix, and inputs the first feature matrix into the visual bag-of-words model to obtain a first visual vector. This process can simplify complex image data into a compact vector representation, facilitating subsequent matching and comparison. Through segmentation and feature extraction, key information in the historical ground image can be more accurately captured, improving the accuracy of image matching. Subsequently, the application processes the real-time ground image in the same way as the historical ground image to obtain a second visual vector. In the image matching step, the first visual vector is used instead of the historical ground image, and the second visual vector is used instead of the real-time ground image, thereby improving the speed and accuracy of image matching. Since the visual vector representation is more compact and efficient than the original image data, using the visual vector for matching can reduce the amount of calculation and improve the matching speed. At the same time, since the visual vector can capture key features in the image, the matching result is also more accurate.

[0023] Optionally, after the step of performing the first segmentation, before the step of performing the first construction, further comprising:

[0024] Clustering: clustering all the first features using a spectral clustering algorithm to obtain a plurality of cluster centers;

[0025] Constructing a dictionary: constructing a new visual dictionary based on the cluster centers;

[0026] Model updating: updating the visual bag-of-words model using the new visual dictionary to obtain a new visual bag-of-words model.

[0027] The application clusters all the first features using a spectral clustering algorithm to obtain a plurality of cluster centers, which represent the main features in the data. Through clustering, the number of words in the visual dictionary can be reduced, and the processing efficiency of the visual bag-of-words model can be improved. At the same time, using the cluster centers as the words of the visual dictionary can more accurately represent the key features in the image. Subsequently, the application constructs a new visual dictionary based on the cluster centers, i.e. using the cluster centers as the words of the visual dictionary. In this way, each word represents a key feature in the image, and the visual dictionary contains all possible key features in the image. The new visual dictionary is more compact and efficient because it only contains key features in the image. At the same time, since the cluster centers are obtained by clustering the first features, the new visual dictionary can more accurately represent the features in the image. Subsequently, the application updates the visual bag-of-words model using the new visual dictionary, replacing the original visual dictionary with the new visual dictionary, and recalculating the visual vector of the image, so that the visual bag-of-words model can represent and compare images based on the new visual dictionary. By updating the visual bag-of-words model, it can more accurately represent the features in the image and improve the accuracy of image matching.

[0028] Optionally, after the step of performing clustering, before the step of constructing the dictionary, further comprising:

[0029] Second acquisition: determining the cluster where each cluster center is located;

[0030] Number determination: determining the number of cluster centers by barristers, including second calculation, third acquisition, iteration and fourth acquisition;

[0031] Second calculation: calculating the inter-cluster variance, the calculation model is as follows:

[0032] ;

[0033] Wherein, is the inter-cluster variance between the i th cluster and the j th cluster; is the weight of the i th cluster; is the average value of the first feature contained in the i th cluster; is the average value of all the first features contained in the i th cluster and the j th cluster; is the weight of the j th cluster; is the average value of the first feature contained in the j th cluster;

[0034] Third acquisition: recording the smallest inter-cluster variance as the first data;

[0035] Iteration: modifying the number of cluster centers, and iteratively performing the steps of clustering until the preset stopping condition is met;

[0036] Fourth acquisition: obtaining the first data after each execution of the step of third acquisition, and obtaining the smallest first data;

[0037] In the step of constructing the dictionary, the number of cluster centers corresponding to the smallest first data is used to construct the visual dictionary.

[0038] The application firstly determines the cluster where each cluster center is located, then determines the number of cluster centers by using the Great Law, and further improves the accuracy and stability of the clustering effect. By calculating the inter-cluster variance, the dispersion degree between clusters under different numbers of cluster centers can be quantified, then the smallest inter-cluster variance is recorded as the first data, then the number of cluster centers is modified, and the clustering is iteratively performed to the third acquisition step. By continuously modifying the number of cluster centers and re-executing the clustering, the inter-cluster variance under different numbers of cluster centers can be explored, and then the number of cluster centers that minimizes the inter-cluster variance is found. Then the first data after each execution of the third acquisition step is obtained, and the smallest first data is obtained. The data amount of the cluster center corresponding to the minimum first data is determined as the optimal number of cluster centers, which provides a basis for constructing a visual dictionary to improve the quality of the visual dictionary, and further improves the accuracy of the visual bag-of-words model.

[0039] Optionally, after the third acquisition step is performed, before the iteration step is performed, it further includes:

[0040] Second judgment: judging whether the first data is greater than a preset inter-cluster variance threshold, if yes, performing the iteration step; if no, performing the first construction step.

[0041] The application compares the currently calculated inter-cluster variance (i.e. the first data) with the preset inter-cluster variance threshold, thereby deciding whether to continue iteration to find a better number of cluster centers or to directly construct a visual dictionary. If the first data is already less than or equal to the preset inter-cluster variance threshold, it means that the current clustering effect is already good enough, and there is no need for additional iteration, thereby saving computing resources and time.

[0042] Optionally, after the image matching step is performed, before the positioning step is performed, it further includes:

[0043] Third judgment: judging whether the first ground image carries historical position information, if yes, performing the positioning step; if no, performing the target recognition step;

[0044] Target recognition: using image recognition technology to recognize target objects in the first ground image;

[0045] Target mapping: using an image matching algorithm to map the target objects with pre-existing satellite images inside the unmanned aerial vehicle to obtain a mapping result;

[0046] Fourth judgment: judging whether the mapping result meets the expectation, if yes, performing the repositioning step; if no, after a preset time interval, performing the first acquisition step;

[0047] Repositioning: the unmanned aerial vehicle flies to the target object, updates the position information of the target object as the second position information, and performs the first judgment step.

[0048] The application first determines whether the first ground image carries historical position information. If the image contains historical position information, positioning is directly performed, unnecessary target recognition and mapping steps are reduced, and time is saved. Otherwise, image recognition technology is used to recognize target objects in the first ground image, and an image matching algorithm is used to map the target objects with satellite images pre-existing in the unmanned aerial vehicle to obtain the accurate position of the target objects. Subsequently, the application determines whether the mapping result meets the expectation. If the mapping result does not meet the expectation, a step of reacquiring is triggered. After the unmanned aerial vehicle flies for a period of time, images are reacquired to position the unmanned aerial vehicle, thereby improving the autonomy of the unmanned aerial vehicle.

[0049] Optionally, after the step of performing the first planning, the method further comprises:

[0050] Second modeling: based on a reinforcement learning algorithm, integrating the running position of the unmanned aerial vehicle into a state space and integrating the action of the unmanned aerial vehicle into an action space to construct a Markov decision process model;

[0051] Setting reward: setting a reward function of the Markov decision process model, and a calculation model of the reward function is as follows:

[0052] ;

[0053] wherein R(·) is the reward function; is a reward for the direction of travel; is an obstacle avoidance reward;

[0054] Prediction: inputting the first position information into the Markov decision process model to obtain predicted position information;

[0055] First pruning: deleting a navigation route that does not contain the predicted position information.

[0056] The Markov decision process model is constructed, the running position and action of the unmanned aerial vehicle are integrated into a state space and an action space through a reinforcement learning algorithm, and a Markov decision process (MDP) model is constructed. The MDP model can make an optimal decision based on a current state and a reward function, and the autonomous navigation capability of the unmanned aerial vehicle is improved. Then, the reward function is set, the reward function only gives a negative reward or no reward, thereby introducing nonlinearity, making the flight route of the unmanned aerial vehicle include more randomness and uncertainty, so that the flight route is not easy to be captured or predicted. Then, the first position information is input into the Markov decision process model to obtain predicted position information. Through the predicted position information, the unmanned aerial vehicle can more accurately judge the surrounding environment, thereby improving the safety of flight. Then, the navigation route not containing the predicted position information is deleted. Through the deletion of unnecessary navigation routes, the flight direction of the unmanned aerial vehicle can be ensured to be towards the target position as much as possible. While increasing the randomness and uncertainty of the navigation route, the flight efficiency is improved as much as possible.

[0057] Optionally, after the step of performing the first pruning, further comprising:

[0058] Fifth acquisition: acquiring the remaining navigation routes, segmenting all the remaining navigation routes at a preset displacement size to obtain a plurality of navigation sub-routes and endpoint position information of the navigation sub-routes;

[0059] Second construction: constructing a tree structure according to the endpoint position information, wherein a parent node in the tree structure is an endpoint position information close to the first position information in the navigation sub-routes, and a child node is an endpoint position information far away from the first position information in the navigation sub-routes;

[0060] Third calculation: calculating a balance factor of the kth node The calculation model is as follows:

[0061] ;

[0062] Wherein, is the maximum height of the subtree of the kth node; is the minimum height of the subtree of the kth node;

[0063] Fifth judgment: judging whether the balance factor is greater than a preset balance threshold, if yes, performing the step of position replacement; if no, performing the step of second planning;

[0064] Position replacement: updating the endpoint position information corresponding to the k+1th node in the subtree with the maximum height to the predicted position information, and performing the step of first pruning;

[0065] The second planning: based on the predicted position information and the target position information, a new residual navigation route is generated by using an incremental path planning algorithm, and the fifth collecting step is executed until a preset stop condition is met.

[0066] The present application divides the residual navigation route by a preset displacement size to generate multiple navigation sub-routes and corresponding endpoint position information. Subsequently, the present application constructs a tree structure according to the endpoint position information, and uses the distance from the first position information as the basis for judging parent and child nodes. Subsequently, a balance factor is calculated to evaluate the balance of the tree structure, and then it is determined whether the balance factor is greater than a preset balance threshold. According to the comparison result of the balance factor and the preset balance threshold, it is determined whether to perform position replacement or second planning. Subsequently, the endpoint position information is updated. By updating the endpoint position information, the path can be optimized to be more in line with the actual situation. Conversely, a new residual navigation route is generated, so that the UAV can dynamically adjust the path planning according to the actual situation, improve the flexibility and adaptability of navigation, and the incremental path planning algorithm can adjust and optimize the existing path, improve the planning efficiency, and also generate more random and uncertain routes, further increasing the uncertainty of the UAV navigation route.

[0067] Optionally, each node of the tree structure includes multiple empty nodes.

[0068] By using the above technical solution, the present application reserves multiple empty nodes in the constructed tree structure. When a new route is obtained by using the incremental path planning algorithm and is added to the tree structure, it is not necessary to reconstruct the tree structure, thereby improving the calculation efficiency.

[0069] Optionally, the second constructing step further includes:

[0070] The sixth judgment: whether there is an empty node in the kth node of the tree structure, if yes, the node filling step is executed; if no, the second constructing step is executed.

[0071] Filling nodes: the endpoint position information of the new residual navigation route is filled into the empty nodes of the corresponding level according to a preset order.

[0072] The present application determines whether there is an empty node in each node of the tree structure. If there is, the endpoint position information of the new residual navigation route is filled into the empty nodes of the corresponding level, so as to make full use of these empty nodes. By filling the empty nodes, the path selection in the tree structure can be increased, thereby optimizing the path planning result.

[0073] In summary, the present application has at least one of the following beneficial technical effects:

[0074] 1.The application can obtain rich environmental data and motion state data by collecting real-time ground images, historical ground images and target position information, and using an inertial sensor to measure speed information. Subsequently, the application can obtain a first position information based on the accumulation of speed information by integrating the speed information. Subsequently, the application can find the most similar historical image by calculating the similarity of the real-time ground image and each historical ground image, and obtain the historical position information carried by the historical image, and then use the historical position information carried by the historical image as the second position information, thereby improving the reliability of positioning by using the accuracy of historical data.

[0075] 2.The application can judge the accuracy of the current position information by comparing the difference between the first position information and the second position information, so as to timely discover and correct possible positioning errors. If it is judged that the position information needs to be updated, the second position information is updated to the first position information. Otherwise, based on the first position information and the target position information, a local path planning algorithm is used to generate a navigation route. The application combines multiple positioning methods (inertial navigation, image matching), which can complement each other to improve the accuracy and reliability of positioning. BRIEF DESCRIPTION OF DRAWINGS

[0076] Figure 1 is a flowchart of embodiment 1 of the application;

[0077] Figure 2 is a flowchart of embodiment 2 of the application;

[0078] Figure 3 is a flowchart of S231 second acquisition and S232 quantity determination of embodiment 3 of the application;

[0079] Figure 4 is a flowchart of S31 third judgment and S32 confirmation of orientation of embodiment 3 of the application;

[0080] Figure 5 is a flowchart of embodiment 4 of the application. DETAILED DESCRIPTION

[0081] The application will be further described below in conjunction with Figures 1 to 5

[0082] Embodiment 1: This embodiment discloses an unmanned aerial vehicle autonomous navigation method based on landscape matching, referring to Figure 1 ​The method comprises: S1 first acquisition, S2 first calculation, S3 image matching, S4 positioning, S5 first judgment, S6 position updating, and S7 first planning. First, real-time and historical ground images, target position information, and speed information measured by an inertial sensor are acquired, and first position information is obtained by integral processing. Then, the real-time and historical images are matched to determine the most similar historical image and its position information. Subsequently, the two sets of position information are compared to determine whether to update the position. Finally, a local path planning algorithm is used to generate a navigation route based on the position information and the target position. The embodiment comprises the following steps:

[0083] S1 first acquisition, the unmanned aerial vehicle acquires data through the camera and sensor system carried thereon. The acquired data comprises:

[0084] The camera on the unmanned aerial vehicle is used to capture the ground scene in real time to obtain real-time ground images.

[0085] The unmanned aerial vehicle acquires historical ground images from a storage system. These historical ground images are used to match the real-time images and carry historical position information.

[0086] The unmanned aerial vehicle acquires target position information through a pre-set coordinate, a marker point on a map, or other forms of position information.

[0087] The inertial sensor (such as an accelerometer and a gyroscope) on the unmanned aerial vehicle measures the speed information of the unmanned aerial vehicle, including speed, angular velocity, and attitude information. These speed information is used to calculate the position of the unmanned aerial vehicle.

[0088] S2 first calculation, the speed information is integrated to obtain first position information.

[0089] The first position information acquired by the unmanned aerial vehicle can be calculated based on the initial position of the unmanned aerial vehicle. The direction change of the unmanned aerial vehicle is calculated by a DCM algorithm. Then, each direction change is combined with the linear speed to obtain a linear speed vector in three spatial dimensions. The displacement change in three spatial dimensions of the unmanned aerial vehicle is obtained by integrating the linear speed vector. The displacement change is added to the initial position to obtain the first position information.

[0090] In other embodiments, the position information at the previous moment or the previous few moments can also be used for calculation. This step will not be described in detail.

[0091] S3 image matching, the similarity of the real-time ground image and each historical ground image is calculated to obtain the historical ground image corresponding to the maximum similarity, which is recorded as the first ground image.

[0092] The way of calculating the similarity of the real-time ground image and each historical ground image can be:

[0093] The similarity between the two images is calculated using the Structural Similarity Index (SSIM). SSIM takes into account the brightness, contrast, and structure of the images. The value of SSIM ranges from 0 to 1, with a value closer to 1 indicating that the two images are more similar.

[0094] The similarity between the two images is calculated using the perceptual hashing algorithm. Perceptual hashing converts images into a fixed-length hash value to compare their similarity. The method first preprocesses the images (such as reducing the size, grayscale, calculating DCT, etc.), and then calculates the hash value of the image. By comparing the difference (such as Hamming distance) between the hash values of the two images, their similarity can be obtained.

[0095] The similarity between the two images is calculated using the cosine similarity. Cosine similarity evaluates the similarity between two vectors by calculating the cosine of the angle between them. In image similarity calculation, images can be converted into feature vectors (such as feature point descriptors extracted by SIFT, SURF, etc.), and then cosine similarity is used to compare the similarity of these feature vectors. The value of cosine similarity ranges from [-1, 1], with a value closer to 1 indicating that the two images are more similar.

[0096] S4 positioning, through data retrieval, obtains the historical position information carried in the first ground image stored in the database, and records the historical position information as the second position information.

[0097] S5 first judgment, judges whether the difference between the first position information and the second position information is greater than the preset difference threshold value, if yes, it means that the first position information is inaccurate, and S6 position update needs to be performed; if not, S7 first planning is executed.

[0098] S6 position update, updating the second position information as the first position information.

[0099] S7 first planning, based on the first position information and the target position information, using a local path planning algorithm to generate a navigation route.

[0100] The embodiment first captures the ground scene in real time by the camera and sensor system and acquires historical ground images, and simultaneously measures speed information by using the inertial sensor. Then, the speed information is integrated to obtain first position information based on the initial position of the UAV or the position at the previous moment. Next, the similarity between the real-time ground image and the historical ground image is calculated by using the structural similarity, perceptual hashing or cosine similarity method, the most similar historical ground image is found, and the historical position information carried by the historical ground image is acquired as second position information. After that, it is judged whether the difference between the first position information and the second position information exceeds a preset threshold, and if it exceeds, the position information is updated, thereby improving the accuracy of the UAV positioning.

[0101] Embodiment 2: Reference Figure 2 The difference between the embodiment and the embodiment 1 is that after the first calculation in S2 is performed, before the image matching in S3 is performed, the following steps are further included:

[0102] S21 first modeling, constructing a visual bag-of-words model by using the historical ground image.

[0103] S22 first segmentation, segmenting the historical ground image into a plurality of blocks, denoted as first segmented blocks.

[0104] Features are extracted from each first segmented block, which can be color, texture, shape or more complex local features. The features of all segmented blocks are integrated into a feature matrix (i.e. first feature matrix), wherein each row or column represents the features of a segmented block.

[0105] S23 clustering, clustering all first features by using a spectral clustering algorithm to obtain a plurality of cluster centers.

[0106] S24 constructing a dictionary, taking the cluster centers obtained by the spectral clustering as new visual words, and constructing a visual dictionary according to the new visual words.

[0107] S25 model updating, updating the visual bag-of-words model by using the new visual dictionary to obtain a new visual bag-of-words model.

[0108] S26 first constructing, inputting the first feature matrix into the new visual bag-of-words model in S25 model updating to obtain a first visual vector, wherein each element in the first visual vector represents the frequency or weight of the corresponding visual word in the historical ground image in the visual dictionary.

[0109] S27 second segmentation, segmenting the real-time ground image to obtain a plurality of second segmented blocks; extracting features in the second segmented blocks, denoted as second features, and integrating the second features into a second feature matrix.

[0110] S28 Second construction, input the second feature matrix into the visual bag-of-words model to obtain a second visual vector, each element in the second visual vector representing the frequency or weight of the corresponding vocabulary in the visual dictionary in the real-time ground image.

[0111] In the image matching of S3, the first visual vector is used instead of the historical ground image, and the second visual vector is used instead of the real-time ground image, and then the Euclidean distance between the first visual vector and each second visual vector is calculated.

[0112] In this embodiment, the historical ground image is segmented, features are extracted and integrated into a feature matrix, spectral clustering algorithm is used to cluster the features to construct a visual dictionary, and then the visual bag-of-words model is updated; subsequently, the feature matrices of the historical and real-time ground images are input into the model respectively, and are converted into corresponding visual vectors; in the image matching stage, these visual vectors are used instead of the original images for similarity calculation to realize efficient and accurate image comparison.

[0113] Embodiment 3: Refer to Figure 3 The difference between this embodiment and embodiment 2 is that after performing S23 clustering, before performing S24 dictionary construction, it further includes:

[0114] S231 Second acquisition, determine the cluster where each cluster center is located.

[0115] S232 Number determination, determine the number of cluster centers by using the Great Law, including S2321 Second calculation, S2322 Third acquisition, S2323 Iteration and S2324 Fourth acquisition.

[0116] S2321 Second calculation, calculate the inter-cluster variance, and the calculation model is as follows:

[0117] ;

[0118] Wherein, is the inter-cluster variance between the i th cluster and the j th cluster; is the weight of the i th cluster; is the average value of the first feature contained in the i th cluster; is the average value of all first features contained in the i th cluster and the j th cluster; is the weight of the j th cluster; is the average value of the first feature contained in the j th cluster.

[0119] S2322 Third acquisition, after calculating all inter-cluster variances, the smallest inter-cluster variance is recorded as the first data.

[0120] S2323 second judgment, judging whether the first data is greater than a preset inter-cluster variance threshold, if yes, it means that the current clustering number is not enough, and iteration is performed in S2324; if no, a dictionary is constructed in S24.

[0121] S2324 iteration, modifying the number of clustering centers, and performing S23 clustering to S2322 third collection iteratively until a preset stopping condition is met (such as reaching a maximum number of iterations, the inter-cluster variance no longer changes significantly, etc.).

[0122] S2325 fourth collection, in the iteration process, the embodiment records the minimum first data after each iteration, so as to select the optimal number of clustering centers after the iteration is completed.

[0123] In S24, the number of clustering centers corresponding to the minimum first data is used to construct a visual dictionary. This visual dictionary will be used to convert image features into visual vectors, thereby supporting subsequent image matching and retrieval tasks.

[0124] In the process of determining the number of clustering centers and constructing the visual dictionary, the embodiment first determines the cluster where each clustering center is located through the clustering algorithm, and then uses the majority law to find the optimal clustering scheme by calculating the inter-cluster variance and iteratively adjusting the number of clustering centers until the preset stopping condition is met. Finally, the number of clustering centers corresponding to the minimum time of the first data (i.e. the minimum inter-cluster variance) is used to construct a visual dictionary to optimize the visual bag-of-words model.

[0125] In other embodiments, with reference to Figure 4 After performing S3 image matching, before performing S4 positioning, it further includes:

[0126] S31 third judgment, judging whether the first ground image carries historical position information, if yes, performing S4 positioning; if no, performing S32 orientation confirmation.

[0127] S32 orientation confirmation, including S321 target identification, S322 target mapping, S323 fourth judgment and S324 repositioning.

[0128] S321 target identification, the unmanned aerial vehicle uses image recognition technology to identify target objects in the first ground image. These target objects can be buildings, road signs, natural landscapes, etc. Any feature that can be identified in a satellite image and whose position is known.

[0129] S322 Target mapping, the unmanned aerial vehicle uses an image matching algorithm to map the identified target objects with pre-existing satellite images stored in its internal memory. This mapping process aims to find the best match between the target objects in the first ground image and the corresponding target objects in the satellite image. The mapping result will provide relative information about the current location of the unmanned aerial vehicle relative to the known location in the satellite image.

[0130] S323 Fourth judgment, judge whether the mapping result meets the expectation, if yes, execute S324 relocation; if not, after a preset time interval, execute S1 first collection.

[0131] Whether the mapping result meets the expectation means that the unmanned aerial vehicle correctly identifies the target object, and the target object can be matched with the target object in the historical ground image.

[0132] S324 Relocation, the unmanned aerial vehicle flies to the target object, updates the position information of the target object as the second position information, and executes S5 first judgment.

[0133] Through the steps of target identification, target mapping and relocation, the unmanned aerial vehicle can update its position information using the features in the ground image.

[0134] Embodiment 4: Reference Figure 5 The difference between this embodiment and embodiment 1 is that after executing S7 first planning, the method further comprises:

[0135] S81 Second modeling, based on the reinforcement learning algorithm, a Markov decision process model is constructed, the process is as follows:

[0136] Constructing state space: the state space of the unmanned aerial vehicle includes its position coordinates, velocity vector, attitude information and perception data of the surrounding environment (such as the position and velocity of other objects), and the above state information collectively describes the complete state of the unmanned aerial vehicle at a certain time.

[0137] Constructing action space: the action space of the unmanned aerial vehicle includes forward, backward, left turn, right turn, ascending, descending and other possible operations, these actions are the actions that the unmanned aerial vehicle can take in a given state.

[0138] Setting transition probability: according to the current state and the action taken, the probability distribution of the next state of the unmanned aerial vehicle is determined.

[0139] Markov decision process model: combined with the state space, action space and transition probability, a Markov decision process model of the unmanned aerial vehicle can be constructed, which will be used to describe the decision-making process of the unmanned aerial vehicle in a random environment.

[0140] S82 Set reward, set the reward function of the Markov decision process model, the calculation model of the reward function is as follows:

[0141] ;

[0142] wherein R(·) is a reward function; is a travel direction reward; is an obstacle avoidance reward.

[0143] S83 prediction, inputting the first position information into a Markov decision process model to obtain predicted position information.

[0144] S84 first pruning, deleting a navigation route not containing the predicted position information according to the predicted position information.

[0145] S85 fifth collection, obtaining remaining navigation routes, segmenting all the remaining navigation routes at a preset displacement size to obtain a plurality of navigation sub-routes and endpoint position information of the navigation sub-routes.

[0146] S86 second construction, constructing a tree structure according to the endpoint position information, a parent node in the tree structure being an endpoint position information close to the first position information in the navigation sub-routes, and a child node being an endpoint position information far from the first position information in the navigation sub-routes.

[0147] S87 third calculation, calculating a balance factor of a kth node , and the calculation model is as follows:

[0148] ;

[0149] wherein, is a maximum height of a subtree of the kth node; is a minimum height of the subtree of the kth node.

[0150] S88 fifth judgment, judging whether the balance factor is greater than a preset balance threshold, if yes, performing S89 position replacement; and if no, performing S90 second planning.

[0151] S89 position replacement, updating endpoint position information corresponding to a k+1th node in a subtree with the maximum height to the predicted position information, and performing the first pruning.

[0152] S90 second planning, generating a new remaining navigation route based on the predicted position information and target position information by using an incremental path planning algorithm, performing the fifth collection of S85, and stopping until a preset stop condition is met.

[0153] The embodiment can realize optimization and prediction of the navigation route of the unmanned aerial vehicle by constructing a Markov decision process model and setting a suitable reward function. Meanwhile, in combination with calculation and judgment of the tree structure and the balance factor, the navigation route can be further balanced and optimized, and the autonomous navigation capability and the task execution efficiency of the unmanned aerial vehicle can be improved.

[0154] In other embodiments, each level of nodes of the tree structure includes a plurality of empty nodes. The S86 second construction further includes:

[0155] S91 sixth judgment, traversing the kth level of nodes of the tree structure, judging whether there is an empty node in the kth level of nodes of the tree structure, if yes, performing S92 filling nodes; if not, performing S86 second construction.

[0156] S92 filling nodes, filling the end position information of the new remaining navigation route into the empty nodes of the corresponding level according to a preset order.

[0157] The embodiment can effectively utilize the storage space of the tree structure to store more navigation information by checking and filling the empty nodes in the kth level of nodes, thereby improving the autonomous navigation capability and the task execution efficiency of the unmanned aerial vehicle.

[0158] The above are preferred embodiments of the present application, and are not intended to limit the protection scope of the present application, therefore: any equivalent changes made according to the structure, shape, principle of the present application shall be covered within the protection scope of the present application.

Claims

1. A method for autonomous navigation of unmanned aerial vehicles (UAVs) based on landscape matching, characterized in that, include: First acquisition: Acquire real-time ground images, historical ground images, and target location information, and use inertial sensors to measure velocity information; First calculation: Integrate the velocity information to obtain the first position information; Image matching: Calculate the similarity between the real-time ground image and each historical ground image, and obtain the historical ground image with the highest similarity, which is denoted as the first ground image; Positioning: The historical location information carried in the first ground image is recorded as the second location information; First judgment: Determine whether the difference between the first location information and the second location information is greater than a preset difference threshold. If so, execute the location update step. If not, proceed with the steps of the first plan; Location update: Update the second location information to the first location information; First planning: Based on the first location information and the target location information, a local path planning algorithm is used to generate a navigation route; After performing the first calculation step and before performing the image matching step, the process also includes: First modeling step: Constructing a visual bag-of-words model; First segmentation: Segment the historical ground image to obtain multiple first segmentation blocks; extract the features from the first segmentation blocks, denoted as first features, integrate the first features into a first feature vector, and integrate the first features into a first feature matrix; First construction: Input the first feature matrix into the visual bag-of-words model to obtain the first visual vector; Second segmentation: The real-time ground image is segmented to obtain multiple second segmentation blocks; features in the second segmentation blocks are extracted and denoted as second features, and the second features are integrated into a second feature matrix; The second construction involves inputting the second feature matrix into the visual bag-of-words model to obtain the second visual vector. In the image matching step, a first visual vector is used to replace historical ground images, and a second visual vector is used to replace real-time ground images.

2. The UAV autonomous navigation method based on landscape matching according to claim 1, characterized in that, After performing the first segmentation step and before performing the first construction step, the process also includes: Clustering: The spectral clustering algorithm is used to cluster all the first features to obtain multiple cluster centers; Building a dictionary: Constructing a new visual dictionary based on cluster centers; Model update: The visual bag-of-words model is updated with a new visual dictionary to obtain a new visual bag-of-words model.

3. The UAV autonomous navigation method based on landscape matching according to claim 2, characterized in that, The process, which follows the clustering step but precedes the dictionary construction step, also includes: Second data collection: Determine the cluster to which each cluster center belongs; Number determination: The number of cluster centers is determined using the big law method, including second calculation, third collection, iteration and fourth collection; The second calculation: calculate the inter-cluster variance, using the following model: ； in, Let be the inter-cluster variance between the i-th cluster and the j-th cluster; Let be the weight of the i-th cluster; It is the average value of the first feature contained in the i-th cluster; It is the average of all the first features contained in the i-th cluster and the j-th cluster; Let be the weight of the j-th cluster; The average value of the first feature contained in the j-th cluster; Third data collection: The smallest inter-cluster variance is recorded as the first data point; Iteration: Modify the number of cluster centers and iteratively execute the clustering steps up to the third collection until the preset stopping condition is met; Fourth data acquisition: Obtain the first data after each step of the third data acquisition, and obtain the smallest first data. In the dictionary construction step, the visual dictionary is constructed using the number of cluster centers corresponding to the minimum first data.

4. The UAV autonomous navigation method based on landscape matching according to claim 3, characterized in that, After performing the third acquisition step and before performing the iteration step, the following steps are also included: Second judgment: Determine whether the first data is greater than the preset inter-cluster variance threshold. If yes, then execute the iterative steps; otherwise, execute the first construction steps.

5. The UAV autonomous navigation method based on landscape matching according to claim 4, characterized in that, After the image matching step and before the localization step, the following steps are also included: The third step is to determine whether the first ground image carries historical location information. If so, the localization step is performed; otherwise, the target recognition step is performed. Target recognition: Using image recognition technology, identify target objects in the first ground image; Target mapping: An image matching algorithm is used to map the target object to a satellite image pre-stored inside the UAV to obtain the mapping result; Fourth judgment: Determine whether the mapping result meets expectations. If yes, execute the relocation step; if no, execute the first acquisition step after a preset interval. Relocation: The drone flies to the target object, updates the target object's position information to the second position information, and executes the first judgment step.

6. The UAV autonomous navigation method based on landscape matching according to any one of claims 1-5, characterized in that, After performing the first planning step, the method further includes: Second modeling: Based on reinforcement learning algorithms, the drone's operating position is integrated into a state space, and the drone's actions are integrated into an action space to construct a Markov decision process model. Setting the reward: Define the reward function for the Markov decision process model. The calculation model for the reward function is as follows: ； Where R(·) is the reward function; Rewards are given for the direction of travel; As a reward for obstacle avoidance; Prediction: Input the initial location information into the Markov decision process model to obtain the predicted location information; First deletion: Delete navigation routes that do not contain predicted location information.

7. The UAV autonomous navigation method based on landscape matching according to claim 6, characterized in that, After performing the first deletion step, the following steps are also included: Fifth data acquisition: Obtain the remaining navigation routes, divide all remaining navigation routes by a preset displacement size, and obtain multiple navigation sub-routes and the endpoint position information of the navigation sub-routes; Second construction: Construct a tree structure based on the endpoint position information. The parent node in the tree structure is the endpoint position information that is closest to the first position information in the navigation sub-route, and the child nodes are the endpoint position information that is far from the first position in the navigation sub-route. Third calculation: Calculate the balance factor of the k-th level node. The calculation model is as follows: ； in, Let be the maximum height of the subtree of the k-th level node; The minimum height of the subtree of the k-th level node; Fifth judgment: Determine whether the balance factor is greater than the preset balance threshold. If yes, then execute the position replacement step; if no, then execute the second planning step. Position replacement: Update the endpoint position information corresponding to the (k+1)th level node in the subtree with the maximum height to the predicted position information, and perform the first deletion step; Second planning: Based on the predicted location information and the target location information, an incremental path planning algorithm is used to generate a new remaining navigation route, and the fifth data collection step is executed until the preset stopping conditions are met.

8. The UAV autonomous navigation method based on landscape matching according to claim 7, characterized in that, Each level of the tree structure includes multiple empty nodes.

9. The UAV autonomous navigation method based on landscape matching according to claim 8, characterized in that, The second construction step also includes: Sixth determination: Determine whether there are empty nodes in the k-th level nodes of the tree structure. If yes, proceed with the node filling step; otherwise, proceed with the second construction step. Fill nodes: Fill the empty nodes at the corresponding level with the endpoint location information of the new remaining navigation routes in a preset order.

Images