Visual absolute positioning method and device for unmanned aerial vehicle

Abstract

The invention relates to the technical field of unmanned aerial vehicle visual positioning, and discloses an unmanned aerial vehicle visual absolute positioning method and device. The method comprises the steps of obtaining first feature data in a tile map, and constructing a feature database based on the first feature data; acquiring second feature data included in the aerial image of the unmanned aerial vehicle, matching the second feature data with the first feature data in the feature database, and determining a candidate first feature data set of which the similarity meets a preset threshold in the feature database; according to the candidate first feature data set, determining a center position coordinate of each candidate tile map in the corresponding candidate tile map set to obtain a center position coordinate set; performing particle filtering processing on the central position coordinate set to obtain an absolute positioning result of the unmanned aerial vehicle; the first feature data corresponding to the tile map is acquired and then stored, so that the demand of the unmanned aerial vehicle for the storage space is reduced, and the storage cost and the calculation cost can be greatly reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle visual positioning, and in particular to a method and device for unmanned aerial vehicle visual absolute positioning. Background Art

[0002] The airborne visual absolute positioning algorithm obtains the current latitude and longitude information of the UAV by matching aerial images with remote sensing images, thereby achieving absolute positioning of the UAV.

[0003] Existing airborne absolute positioning technology usually relies on the global satellite positioning system. When the satellite signal is poor or interfered, a new absolute positioning solution is needed. A feasible alternative is scene matching positioning, but this solution requires storing a large number of remote sensing image maps in the drone's onboard storage unit. There are problems such as high cost configuration of drone storage hardware due to excessive storage requirements and poor adaptability of the solution. A better solution is often needed. Summary of the invention

[0004] In view of this, the present invention provides a method and device for visual absolute positioning of an unmanned aerial vehicle, so as to solve the problem of high configuration cost caused by excessively large map storage space required in existing visual absolute positioning methods.

[0005] On one hand, the present invention provides a method for visual absolute positioning of a drone, the method comprising: using a feature extraction network to obtain first feature data included in a tile map, and constructing a feature database based on the first feature data; the tile map is obtained based on remote sensing image segmentation of an adaptation area;

[0006] Using the feature extraction network to obtain second feature data included in the aerial image of the drone, matching the second feature data with the first feature data in the feature database, and determining a set of candidate first feature data whose similarity meets a preset threshold in the feature database;

[0007] Determine the center position coordinates of each candidate tile map in the corresponding candidate tile map set according to the candidate first feature data set to obtain a center position coordinate set;

[0008] The center position coordinates in the center position coordinate set are processed by particle filtering to obtain the center position coordinates that meet the preset convergence conditions as the absolute positioning result of the UAV.

[0009] Another aspect of the present invention provides a method and device for visual absolute positioning of a drone, the device comprising: a feature extraction module, for acquiring first feature data included in a tile map by using a feature extraction network, and constructing a feature database based on the first feature data; the tile map is acquired based on remote sensing image segmentation of an adaptation area;

[0010] a feature matching module, configured to use the feature extraction network to obtain second feature data included in the aerial image of the drone, match the second feature data with the first feature data in the feature database, and determine in the feature database a set of candidate first feature data whose similarity meets a preset threshold;

[0011] A coordinate determination module, used to determine the center position coordinates of each candidate tile map in the corresponding candidate tile map set according to the candidate first feature data set, to obtain a center position coordinate set;

[0012] The filtering positioning module is used to perform particle filtering on the center position coordinates in the center position coordinate set to obtain the center position coordinates that meet the preset convergence conditions as the absolute positioning result of the UAV.

[0013] On the other hand, the present invention also provides a computer device, which includes: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor implements the above-mentioned drone visual absolute positioning method by executing the computer instructions.

[0014] On the other hand, the present invention further provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to implement the above-mentioned drone visual absolute positioning method.

[0015] On the other hand, the present invention also provides a computer program product, including computer instructions, which are used to enable a computer to execute the drone visual absolute positioning method of the above-mentioned first aspect or any corresponding embodiment thereof.

[0016] In this process, the corresponding tile map is obtained by segmenting the remote sensing map, and the first feature data corresponding to the tile map is further obtained by using the feature extraction network. The drone only needs to store the first feature data to match it with the second feature data of the aerial image to determine the positioning result. Compared with directly storing a large range of remote sensing images, the amount of first feature data extracted is smaller, thereby reducing the demand for storage space and significantly reducing storage and computing costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of a method for absolute positioning of a drone using visual means provided by an embodiment of the present invention;

[0019] Figure 2 It is a specific flow chart of a method for visual absolute positioning of a drone provided by an embodiment of the present invention;

[0020] Figure 3 It is a schematic diagram of a particle filter processing flow of a drone visual absolute positioning method provided by an embodiment of the present invention;

[0021] Figure 4 It is a structural schematic diagram of a method and device for visual absolute positioning of a drone provided by an embodiment of the present invention;

[0022] Figure 5 It is a structural schematic diagram of another unmanned aerial vehicle visual absolute positioning method device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The airborne visual absolute positioning algorithm obtains the current latitude and longitude information of the UAV by matching aerial images with remote sensing images, and realizes the absolute positioning of the UAV. Traditional algorithms usually focus on accurate positioning within a square kilometer, that is, positioning is achieved by matching the remote sensing images in the airborne database given the initial position of the UAV. In actual positioning, the initial position of the UAV is usually unknown, and it is usually necessary to search and locate in a large-scale adaptation area (for example, tens of thousands of square kilometers).

[0024] The prior art often has the following problems:

[0025] 1. It is necessary to store the remote sensing map of a large adaptation area in the onboard storage unit of the drone. The map storage space required by the drone is too large, resulting in high configuration costs for the onboard storage unit.

[0026] 2. In the feature extraction process of aerial images, specific semantic features are required in the images, otherwise feature data cannot be generated, resulting in limitations on the positioning and use scenarios of drones;

[0027] 3. During the calculation of the absolute positioning results of the drone, due to the limitations of the algorithm, there may be a problem of low positioning accuracy.

[0028] To solve at least one of the above problems, various embodiments of the present invention provide a method for visual absolute positioning of an unmanned aerial vehicle, including: using a feature extraction network to obtain first feature data included in a tile map, and constructing a feature database based on the first feature data; the tile map is obtained based on remote sensing image segmentation of an adaptation area; using a feature extraction network to obtain second feature data included in an aerial image of the unmanned aerial vehicle, matching the second feature data with the first feature data in the feature database, and determining a set of candidate first feature data in the feature database whose similarity meets a preset threshold; determining the center position coordinates of each candidate tile map in the corresponding candidate tile map set based on the candidate first feature data set, and obtaining a center position coordinate set; performing particle filtering on the center position coordinates in the center position coordinate set, and obtaining center position coordinates that meet preset convergence conditions as the absolute positioning result of the unmanned aerial vehicle.

[0029] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0030] According to an embodiment of the present invention, a method for visual absolute positioning of a drone is provided. Figure 1 FIG. 1 is a flow chart of a method for absolute positioning of a drone using visual means according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0031] Step S101, using a feature extraction network to obtain first feature data included in the tile map, and constructing a feature database based on the first feature data; the tile map is obtained based on the remote sensing image segmentation of the adaptation area;

[0032] Step S102, using a feature extraction network to obtain second feature data included in the aerial image of the drone, matching the second feature data with the first feature data in a feature database, and determining a set of candidate first feature data whose similarity meets a preset threshold in the feature database;

[0033] Step S103, determining the center position coordinates of each candidate tile map in the corresponding candidate tile map set according to the candidate first feature data set, and obtaining a center position coordinate set;

[0034] Step S104, performing particle filtering processing on the center position coordinates in the center position coordinate set, and obtaining the center position coordinates that meet the preset convergence conditions as the absolute positioning result of the UAV.

[0035] In a possible implementation, the feature extraction network may refer to a neural network based on deep learning, which may be used to extract first feature data corresponding to the tile map from the input tile map.

[0036] Here, the first feature data included in the tile map may be image features of the remote sensing image in the corresponding tile map; for example, the first feature image may include but is not limited to: edge, texture, shape, etc.

[0037] Furthermore, the tile map of the remote sensing image may refer to rectangular blocks of a fixed size obtained by dividing the map data of the remote sensing image according to a predetermined grid, and each tile map may contain image information of the remote sensing image in a corresponding area.

[0038] Furthermore, the feature database may be stored in an onboard storage unit of the drone to store the first feature data of the tile map.

[0039] In one possible implementation, an aerial image taken by a drone is obtained, a feature extraction network is used to obtain second feature data in the aerial image, the second feature data is matched and compared with the first feature data in a feature database, the similarity between the second feature data and the first feature data in the feature database is determined, and a set of candidate first feature data whose similarity meets a preset threshold is screened out.

[0040] Among them, the second feature data may refer to the image features corresponding to the aerial image; the form of similarity may include but is not limited to: Euclidean distance, cosine distance; the preset threshold satisfied by the similarity can be specifically defined according to actual needs, which will not be repeated here.

[0041] Furthermore, the determination of the first feature data and the second feature data does not require the presence of specific semantic features in the remote sensing image and the aerial image, thereby improving the ease of use of the absolute positioning of the drone; the semantic features may include but are not limited to: mountains, villages, and rivers.

[0042] In a possible implementation, the center position coordinates of each candidate tile map in the candidate tile map set may be longitude and latitude coordinates, and the center position coordinate set formed by the center position coordinates of each candidate tile map may be used to characterize a possible positioning position of the drone.

[0043] Furthermore, particle filtering processing is performed on the center position coordinate set, which can be implemented based on the following steps:

[0044] Initialize the particle set based on the distribution of each center position coordinate in the center position coordinate set;

[0045] Update the position of the particle set based on the UAV's motion model, and update the weights of the particles in the particle set according to the new aerial image matching results;

[0046] All particles are resampled according to the weight of each particle in the particle set, and particles that meet the preset convergence conditions are retained, so that the particles gradually converge to the final position, and the final position is used as the absolute positioning result of the drone.

[0047] Among them, particle filtering can refer to a Bayesian filtering method for estimating the state of a dynamic system (i.e., positioning position); a particle can refer to a hypothetical position estimate in a particle filtering algorithm; and the preset convergence conditions of a particle can include but are not limited to: particle variance and maximum number of particle iterations.

[0048] Through the above method, the adaptation area is divided into multiple tile maps, and the features of the tile maps are extracted and stored, avoiding the need for traditional methods to store complete large-scale remote sensing images and greatly reducing the storage space requirements; through the combination of feature extraction network and particle filtering, the positioning error of the UAV in a large-scale adaptation area is reduced, and the accuracy of the positioning results is improved.

[0049] In a specific embodiment, a feature extraction network is used to obtain first feature data included in a tile map, and a feature database is constructed based on the first feature data, which is implemented based on the following steps:

[0050] Acquire a remote sensing image of the adaptation area corresponding to the drone, and divide the remote sensing image into a first preset number of tile maps;

[0051] Using a feature extraction network to obtain a column vector corresponding to each tile map in a first preset number of tile maps;

[0052] Based on the column vectors respectively corresponding to the first preset number of tile maps, a first feature data set corresponding to the first preset number of tile maps is determined, and a feature database is constructed based on the first feature data set.

[0053] In a possible implementation, the adaptation area corresponding to the drone can be implemented based on the following steps:

[0054] The execution area information of the UAV is obtained, the area where the UAV is located is determined based on the execution area information, and an area of ​​a predetermined area including the area where the UAV is located is used as the adaptation area of ​​the UAV.

[0055] Here, the execution area information of the drone may refer to the predetermined range of the drone mission, and the execution area information may include but is not limited to: the mission starting point, the mission end point, and the area that may be covered in the mission; for example, when the drone is performing a forest fire detection mission, the forest may be the area where the drone is located.

[0056] Furthermore, the area including the predetermined area of ​​the area where the user is located may be an area of ​​10,000 square kilometers surrounding the area where the user is located; the predetermined area here may be specifically set based on factors such as the flight speed of the drone, navigation accuracy, and map data resolution, and no specific requirements are made here.

[0057] Exemplarily, dividing the remote sensing image into a first preset number of tile maps may refer to dividing the remote sensing image of the adaptation area into N continuous and overlapping tile maps; N may be a non-negative integer.

[0058] Here, dividing the remote sensing image into N continuous tile maps may mean ensuring that the N tile maps after division can completely cover the adaptation area. The continuity of the tile map can ensure the spatial consistency in the adaptation area and avoid erroneous boundary judgments during the processing process. Dividing the remote sensing image into N overlapping tile maps may mean that there are differences in the images of the same location in different tile maps to improve the fault tolerance during the processing process.

[0059] In a possible implementation, based on the column vectors corresponding to the first preset number of tile maps respectively, determining the first feature data set corresponding to the first preset number of tile maps, and constructing the feature database based on the first feature data set can be implemented based on the following steps:

[0060] Obtain a column vector v of M dimension corresponding to each tile map in the N tile maps, determine that the first feature data set is an N*M matrix, and construct a feature database based on the matrix.

[0061] Here, the above matrix can be expressed by the following formula:

[0062] V=[v 1 ,v 2 ,…,v N ] T

[0063] Through the above method, the storage space requirement of the drone is reduced by building a feature database, thereby reducing the hardware configuration cost of the drone; by building a structured feature database, when the drone is positioned, the aerial image and the first feature data in the feature database can be quickly matched, thereby improving the speed and efficiency of positioning; the method is also suitable for large-scale drone positioning tasks, and can quickly and accurately perform large-scale drone positioning based on the feature database.

[0064] In a specific embodiment, according to the candidate first feature data set, the center position coordinates of each candidate tile map in the corresponding candidate tile map set are determined to obtain the center position coordinate set, which is implemented based on the following steps:

[0065] Acquire an aerial image of the drone at the current moment, extract second feature data of the aerial image using a feature extraction network, match the second feature data with a set of candidate first feature data, and determine a distance from the second feature data to each first feature data in the set of candidate first feature data;

[0066] The distances are sorted in ascending order to obtain a center position coordinate set consisting of center position coordinates whose sorting results are less than a second preset number in the distances.

[0067] Specifically, a feature extraction network is used to extract the second feature data f of the aerial image, and the distance dist from the second feature data f to each first feature data in the candidate first feature data set is calculated, where dist can be an N*1 column vector; the N distances in the column vector dist are sorted in ascending order, and the center position coordinate set corresponding to the first K distances with the smallest distances in dist is selected; wherein K is the second preset number.

[0068] Furthermore, the distance may include but is not limited to: Euclidean distance and cosine distance; wherein, Euclidean distance may refer to a distance measurement method used to measure the straight-line distance between two points, and cosine distance may be a method of measuring the difference between two vectors by calculating the cosine value of the angle between the two vectors.

[0069] In a possible implementation, a center position coordinate set consisting of K center position coordinates may be used as an alternative positioning position of the drone.

[0070] Through the above method, the current geographic location coordinates of the drone can be inferred more accurately, thereby greatly reducing the positioning error; the center positions of multiple tile maps are selected as alternative positioning positions, which can enhance the robustness of the system in various complex environments.

[0071] In a specific embodiment, particle filtering is performed on the center position coordinates in the center position coordinate set to obtain the center position coordinates that meet the preset convergence conditions as the absolute positioning result of the UAV, including:

[0072] A mixed Gaussian distribution model is constructed using the center position coordinate set to determine initial weights of a third preset number of particles corresponding to the center position coordinates;

[0073] Determine predicted positions corresponding to the third preset number of particles at the current moment based on the position of each particle in the third preset number of particles at the previous moment;

[0074] Determine an update weight corresponding to each particle in the third preset number of particles based on the predicted positions of the third preset number of particles and a set of center position coordinates corresponding to the aerial image at the current moment;

[0075] Particle resampling is performed on a third preset number of particles based on the updated weights to obtain center position coordinates that meet a preset convergence condition.

[0076] In one possible implementation, a mixed Gaussian distribution model is constructed using a set of center position coordinates, based on the following steps:

[0077] Based on the K center position coordinates in the center position coordinate set at the current moment, a sampling process of the particle x is performed to obtain a third preset number of particles x, and an initial Gaussian mixture model is determined based on the particles x.

[0078] Specifically, the initial Gaussian mixture model associated with particle x can be expressed by the following formula:

[0079]

[0080] Among them, the possible multiple positioning locations of the drone can be described by probability density p(x); each Gaussian distribution N(x; μ k ,Σ k ) can correspond to the center position coordinate μ in a tile map k , through the covariance matrix Σ k The expansion degree of the positioning area is expressed in the form of k The weight of each positioning location area is represented to characterize the possibility of the positioning location area.

[0081] In the Gaussian distribution formula N(x; μ k ,Σ k ), the Gaussian distribution can be represented by the mean vector μ k With the covariance matrix Σ k Definition, used to characterize the distribution characteristics of a particle x within the range of the corresponding tile map; the mean vector can refer to the average position of the Gaussian distribution, and each element in the mean vector can represent the average value of the particle in each dimension; the covariance can represent the dependency between two random variables, and the covariance matrix can represent the expansion range of the data in each dimension and the correlation between the dimensions.

[0082] For example, the mean vector μ corresponding to the Gaussian distribution of particles k , which can be expressed as follows:

[0083] μ k =[lon k lat k ] T

[0084] Here, the mean vector μ kThe longitude and latitude coordinates of the K center position coordinates can be represented to indicate that the particles are evenly distributed near the K center position coordinates in the K tile maps.

[0085] Furthermore, the covariance matrix Σ corresponding to the Gaussian distribution of particles in the kth tile map k , which can be expressed as follows:

[0086]

[0087] here, It can represent the variance of particles in the kth tile map in the longitude direction. It can represent the variance of particles in the kth tile map in the latitude direction; in Gaussian distribution, by setting the particle in the kth tile map and The value of is set so that the corresponding k-th tile map is covered within a range of 3σ around the mean; where σ can represent the standard deviation, indicating the degree of dispersion of the data around the mean, and the range of 3σ around the mean can include 99.7% of the particles.

[0088] Furthermore, the weight π corresponding to the Gaussian distribution k , which can be expressed as follows:

[0089]

[0090] Among them, dist(k) can represent the distance between the first data feature of the kth tile map and the second data feature of the aerial image; when the distance is smaller, it means that the first data feature of the kth tile map is more similar to the second data feature of the aerial image, that is, the weight is greater.

[0091] In a possible implementation, based on the position of each particle in the third preset number of particles at the previous moment, the predicted positions corresponding to the third preset number of particles at the current moment are determined, which is implemented based on the following steps:

[0092] The position information of each particle in the third preset number of particles at the previous moment and the predicted motion model are obtained to determine the predicted positions corresponding to the third preset number of particles at the current moment.

[0093] Here, the position information of each particle at the first moment can be determined based on the initial Gaussian mixture model of the particle.

[0094] In a possible implementation, based on the predicted positions of the third preset number of particles and the center position coordinate set obtained by matching the tile map of the aerial image at the current moment, determining the update weight corresponding to each particle in the third preset number of particles includes:

[0095] When the predicted position of the particle at the current moment matches the center position coordinate set obtained by matching the tile map of the aerial image at the current moment, the weight value of the particle is increased; when the predicted position of the particle at the current moment does not match the center position coordinate set obtained by matching the tile map of the aerial image at the current moment, the weight value of the particle is reduced.

[0096] Furthermore, the weight update method may be to calculate the weight of the particle predicted position under a mixed Gaussian model formed by a set of center position coordinates obtained by matching the tile map with the aerial image at the current moment.

[0097] In a possible implementation, resampling the third preset number of particles based on the updated weights to obtain center position coordinates that meet a preset convergence condition includes:

[0098] According to the updated weight distribution, resampling is performed on the third preset number of particles to reduce the number of particles with small weights and increase the number of particles with large weights;

[0099] The particles are iteratively updated and resampled until the standard deviation of the updated particle position coordinate set is less than the set threshold. At this time, the particles meet the preset convergence conditions, and the mean of the particle position coordinate set is used as the absolute positioning result of the drone.

[0100] Through the above method, the final absolute positioning result of the drone is determined in the candidate tile map set based on particle filtering, making the absolute positioning of the drone more accurate; the combination of initial Gaussian distribution and particle filtering enables the drone to effectively cope with noise and uncertainty in the environment, thereby improving the robustness of absolute positioning.

[0101] In a specific embodiment, determining the predicted positions corresponding to the third preset number of particles at the current moment based on the position of each particle in the third preset number of particles at the previous moment includes:

[0102] Obtaining the position of the selected particle from the third preset number of particles at the last moment, and using the odometer to obtain the movement amount of the drone in a first time period; the first time period is the time interval from the last moment to the current moment;

[0103] The predicted position of the selected particle at the current moment is determined based on the position and the amount of motion.

[0104] Specifically, assuming that the position of the mth particle in the third preset number of particles at the previous moment is Determine the odometer movement from the previous moment to the current moment as u t , then the predicted position corresponding to the mth particle at the current moment is determined as

[0105] Here, the particle position at the previous moment and the drone motion in the time interval are added to obtain the predicted position of the particle at the current moment; the odometer motion is u t It is used to characterize the movement amount of the drone from the previous moment to the current moment, that is, the actual movement data of the drone.

[0106] Among them, odometer can be a technology that uses images or video sequences or inertial devices to estimate the position and attitude changes of the drone in space.

[0107] Through the above method, the relative motion estimation provided by the odometer allows the actual motion of the drone to be taken into account during the absolute positioning process, thereby improving the accuracy of positioning. The predicted position of each particle is determined based on the actual amount of motion rather than by random determination, which can make the positioning process of the drone more robust.

[0108] In a specific embodiment, the method further comprises:

[0109] When the average weight of the third preset number of particles is less than the set threshold for a period greater than the time threshold, it is determined that the particles are in an erroneous convergence state, the particles are reacquired for particle filtering, and the center position coordinates that meet the preset convergence conditions are obtained as the absolute positioning result of the UAV.

[0110] Specifically, when a particle updates its weight, the weights of all particles are monitored. If the updated weight of a particle is less than a set threshold for a period longer than a time threshold, the particle is determined to be in an erroneous convergence state, that is, the particle has deviated from the actual target position; the particle re-projection mechanism is triggered to regenerate a group of particles to reposition or increase the diversity of the algorithm, thereby increasing the probability of finding the correct target position.

[0111] Among them, a Gaussian mixture distribution is reconstructed with the particle position at the current moment, a group of particles are regenerated, the particle position is sampled according to the Gaussian distribution, the initial weight of the particle is determined, the regenerated particles are updated through particle filtering, and the center position coordinates that meet the preset convergence conditions are obtained as the absolute positioning result of the UAV.

[0112] Through the above method, particles can be re-cast when particle errors converge, thereby ensuring the accuracy of the final positioning result and increasing the robustness of the UAV in complex environments.

[0113] In a specific embodiment, the feature extraction network includes a feature extraction model and a feature aggregation model.

[0114] Here, the feature extraction model can be used to extract feature information from the input tile map or aerial image, and the feature aggregation model can be used to combine and integrate the feature information output from the feature extraction model to form a global feature description.

[0115] Through the above method, feature information in tile maps or aerial images is extracted through the feature extraction model, and then these features are fused and compressed through the feature aggregation model, so that the feature matching model can better understand and process the input features, thereby improving the accuracy in the absolute positioning process.

[0116] For example, Figure 2 FIG. 1 is a schematic diagram of a specific flow chart of a method for absolute positioning of a drone using visual means according to an embodiment of the present invention. Figure 2 As shown, the figure includes the following steps:

[0117] Step S201, obtaining a remote sensing tile map;

[0118] Here, a remote sensing image of the adaptation area corresponding to the drone is obtained, and the remote sensing image is divided into a first preset number of tile maps;

[0119] Step S202, determining first characteristic data;

[0120] Here, a feature extraction network is used to obtain first feature data included in the tile map;

[0121] Step S203, constructing a feature database;

[0122] Here, a feature database is constructed based on the first feature data;

[0123] Step S204, acquiring an aerial image sequence;

[0124] Here, an image sequence consisting of aerial images taken by a drone is obtained;

[0125] Step S205, obtaining second characteristic data;

[0126] Here, a feature extraction network is used to obtain second feature data included in the aerial image of the drone;

[0127] Step S206, feature matching;

[0128] Here, the second feature data is matched with the first feature data in the feature database, and a set of candidate first feature data whose similarity satisfies a preset threshold is determined in the feature database;

[0129] Step S207, determining the amount of motion between frames;

[0130] Here, the odometer is used to obtain the movement of the drone between two frames;

[0131] Step S208, particle filtering processing;

[0132] Here, the center position coordinates in the center position coordinate set are processed by particle filtering;

[0133] Step S209, obtaining the absolute positioning result;

[0134] Here, particle resampling is performed to obtain the center position coordinates that meet the preset convergence conditions as the absolute positioning result of the UAV.

[0135] Figure 3 FIG. 1 is a schematic diagram of a particle filter processing flow of a drone visual absolute positioning method provided by an embodiment of the present invention. Figure 3 As shown, the figure includes the following steps:

[0136] Step S301, performing particle sampling;

[0137] Here, a sampling process of particles x is performed based on K center position coordinates in the center position coordinate set at the current moment to obtain a third preset number of particles x;

[0138] Step S302, updating particle position;

[0139] Here, the position of the selected particle in the third preset number of particles at the previous moment is obtained, and the movement amount of the drone in the first time period is obtained by using an odometer, and the predicted position corresponding to the selected particle at the current moment is determined based on the position and the movement amount;

[0140] Step S303, updating particle weights;

[0141] Here, based on the predicted positions of the third preset number of particles and the matching result of the aerial image at the current moment, the update weight corresponding to each particle in the third preset number of particles is determined;

[0142] Step S304, determine whether the target is lost; if so, proceed to step S306, if not, proceed to step S305;

[0143] Here, when the average weight of the third preset number of particles is less than the set threshold for a period longer than the time threshold, it is determined that the particles are in an erroneous convergence state (i.e., the target is lost);

[0144] Step S305, particle resampling;

[0145] Here, based on the update weight, the third preset number of particles are resampled to obtain the updated center position coordinates of the particles;

[0146] Step S306, re-projection of particles;

[0147] Here, the particle re-casting mechanism is triggered to re-acquire particles for particle filtering processing until the center position coordinates that meet the preset convergence conditions are obtained as the absolute positioning result of the UAV;

[0148] Step S307, determine whether filtering is completed; if not, go to step S302, if so, end the process.

[0149] Figure 4 is a schematic diagram of the structure of a UAV visual absolute positioning device provided by an embodiment of the present invention, such as Figure 4 As shown, the device can be applied to intelligent electronic devices such as servers and computers; the device includes: a feature extraction module 401, a feature matching module 402, a candidate coordinate determination module 403 and a filtering positioning module 404; wherein,

[0150] The feature extraction module 401 is used to obtain first feature data included in the tile map by using a feature extraction network, and to construct a feature database based on the first feature data; the tile map is obtained based on the remote sensing image segmentation of the adaptation area;

[0151] A feature matching module 402 is used to obtain second feature data included in the aerial image of the drone using a feature extraction network, match the second feature data with the first feature data in the feature database, and determine a set of candidate first feature data whose similarity meets a preset threshold in the feature database;

[0152] The candidate coordinate determination module 403 is used to determine the center position coordinates of each candidate tile map in the corresponding candidate tile map set according to the candidate first feature data set, and obtain a center position coordinate set;

[0153] The filtering positioning module 404 is used to perform particle filtering on the center position coordinates in the center position coordinate set to obtain the center position coordinates that meet the preset convergence conditions as the absolute positioning result of the UAV.

[0154] In a specific embodiment, the feature extraction module 401 is used to obtain a remote sensing image of the drone corresponding to the adaptation area, and divide the remote sensing image into a first preset number of tile maps;

[0155] Using a feature extraction network to obtain a column vector corresponding to each tile map in a first preset number of tile maps;

[0156] Based on the column vectors respectively corresponding to the first preset number of tile maps, a first feature data set corresponding to the first preset number of tile maps is determined, and a feature database is constructed based on the first feature data set.

[0157] In a specific embodiment, the candidate coordinate determination module 403 is used to obtain an aerial image of the drone at the current moment, extract second feature data of the aerial image using a feature extraction network, match the second feature data with a set of candidate first feature data, and determine the distance from the second feature data to each first feature data in the set of candidate first feature data;

[0158] The distances are sorted in ascending order to obtain a center position coordinate set consisting of center position coordinates whose sorting results are less than a second preset number in the distances.

[0159] In a specific embodiment, the filtering and positioning module 404 is used to construct a mixed Gaussian distribution model using the center position coordinate set to determine the initial weights of a third preset number of particles corresponding to the center position coordinates;

[0160] Determine predicted positions corresponding to the third preset number of particles at the current moment based on the position of each particle in the third preset number of particles at the previous moment;

[0161] Determine an update weight corresponding to each particle in the third preset number of particles based on a set of center position coordinates obtained by matching the tile map of the third preset number of particles with the predicted positions of the third preset number of particles at the current moment;

[0162] Particle resampling is performed on a third preset number of particles based on the updated weights to obtain center position coordinates that meet a preset convergence condition.

[0163] In a specific embodiment, the filtering positioning module 404 is used to obtain the position of the selected particle from the third preset number of particles at the last moment, and to use the odometer to obtain the movement amount of the drone in a first time period; the first time period is the time interval from the last moment to the current moment;

[0164] The predicted position of the selected particle at the current moment is determined based on the position and the amount of motion.

[0165] In a specific embodiment, the device further comprises: a particle re-projection module 405; wherein,

[0166] The particle re-projection module 405 is used to determine that the particles are in an erroneous convergence state when the average weight of the third preset number of particles is less than the set threshold for a period greater than the time threshold, re-acquire the particles for particle filtering, and obtain the center position coordinates that meet the preset convergence conditions as the absolute positioning result of the drone.

[0167] In a specific embodiment, the feature extraction network includes a feature extraction model and a feature aggregation model.

[0168] It should be noted that: the drone visual absolute positioning device provided in the above embodiment only uses the division of the above program modules as an example to illustrate when implementing the corresponding drone visual absolute positioning method. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the above-described processing. Figure 1 The embodiments of the method shown belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0169] The embodiment of the present invention also provides a computer device having the above Figure 4 The UAV visual absolute positioning device shown.

[0170] See also Figure 5 , Figure 5 FIG. 1 is a schematic diagram of another UAV visual absolute positioning device provided by an embodiment of the present invention. Figure 5 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0171] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0172] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0173] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0174] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0175] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0176] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0177] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0178] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0179] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0180] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for absolute positioning of unmanned aerial vehicles using visual means, characterized in that: The method comprises: A feature extraction network is used to obtain first feature data included in a tile map, and a feature database is constructed based on the first feature data; the tile map is obtained based on remote sensing image segmentation of an adaptation area; Using the feature extraction network to obtain second feature data included in the aerial image of the drone, matching the second feature data with the first feature data in the feature database, and determining a set of candidate first feature data whose similarity meets a preset threshold in the feature database; Determine the center position coordinates of each candidate tile map in the corresponding candidate tile map set according to the candidate first feature data set to obtain a center position coordinate set; The center position coordinates in the center position coordinate set are processed by particle filtering to obtain the center position coordinates that meet the preset convergence conditions as the absolute positioning result of the UAV.

2. The method according to claim 1, characterized in that A feature extraction network is used to obtain first feature data included in the tile map, and a feature database is constructed based on the first feature data, which is implemented based on the following steps: Acquire a remote sensing image of the adaptation area corresponding to the drone, and divide the remote sensing image into a first preset number of tile maps; Using a feature extraction network to obtain a column vector corresponding to each tile map in the first preset number of tile maps; Based on the column vectors respectively corresponding to the first preset number of tile maps, a first feature data set corresponding to the first preset number of tile maps is determined, and a feature database is constructed based on the first feature data set.

3. The method according to claim 2, characterized in that According to the candidate first feature data set, the center position coordinates of each candidate tile map in the corresponding candidate tile map set are determined to obtain a center position coordinate set, which is implemented based on the following steps: Acquire an aerial image of the drone at the current moment, extract second feature data of the aerial image using a feature extraction network, match the second feature data with the candidate first feature data set, and determine a distance from the second feature data to each first feature data in the candidate first feature data set; The distances are sorted in ascending order to obtain a center position coordinate set consisting of center position coordinates whose sorting results are less than a second preset number among the distances.

4. The method according to claim 1, characterized in that The performing particle filtering on the center position coordinates in the center position coordinate set to obtain the center position coordinates satisfying a preset convergence condition as the absolute positioning result of the UAV includes: A mixed Gaussian distribution model is constructed using the center position coordinate set to determine initial weights of a third preset number of particles corresponding to the center position coordinates; Determine the predicted positions corresponding to the third preset number of particles at the current moment based on the position of each particle in the third preset number of particles at the previous moment; Determine an update weight corresponding to each particle in the third preset number of particles based on a set of center position coordinates of the third preset number of particles obtained by matching the tile map at the predicted position with the aerial image at the current moment; Particle resampling is performed on the third preset number of particles based on the update weights to obtain center position coordinates that meet a preset convergence condition.

5. The method according to claim 4, characterized in that The determining, based on the position of each particle in the third preset number of particles at the previous moment, the predicted positions corresponding to the third preset number of particles at the current moment respectively includes: Obtaining the position of a selected particle from the third preset number of particles at the last moment, and using an odometer to obtain the movement amount of the drone in a first time period; the first time period is the time interval from the last moment to the current moment; A predicted position of the selected particle corresponding to the current moment is determined based on the position and the movement amount.

6. The method according to claim 5, characterized in that The method further comprises: When the average weight of the third preset number of particles is less than the set threshold for a period greater than the time threshold, it is determined that the particles are in an erroneous convergence state, the particles are reacquired for particle filtering, and the center position coordinates that meet the preset convergence conditions are obtained as the absolute positioning result of the UAV.

7. The method according to claim 6, characterized in that The feature extraction network includes a feature extraction model and a feature aggregation model.

8. A visual absolute positioning device for unmanned aerial vehicles, characterized in that: The device comprises: A feature extraction module, used for acquiring first feature data included in a tile map by using a feature extraction network, and constructing a feature database based on the first feature data; the tile map is acquired based on the segmentation of a remote sensing image of an adaptation area; a feature matching module, configured to use the feature extraction network to obtain second feature data included in the aerial image of the drone, match the second feature data with the first feature data in the feature database, and determine in the feature database a set of candidate first feature data whose similarity meets a preset threshold; A candidate coordinate determination module, used to determine the center position coordinates of each candidate tile map in the corresponding candidate tile map set according to the candidate first feature data set, to obtain a center position coordinate set; The filtering positioning module is used to perform particle filtering on the center position coordinates in the center position coordinate set to obtain the center position coordinates that meet the preset convergence conditions as the absolute positioning result of the UAV.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the UAV visual absolute positioning method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the drone visual absolute positioning method according to any one of claims 1 to 7.

Images