A heterogeneous image matching method and system for unmanned aerial vehicles

Abstract

The present invention discloses a heterogeneous image matching method and system applied to a drone, and relates to the technical field of drone image matching. The specifically disclosed technical scheme constructs an image corresponding data set, establishes a corresponding relationship between drone image features and satellite image block features based on the corresponding relationship in the image corresponding data set, obtains an image feature corresponding data set, trains the image feature corresponding data set, obtains a heterogeneous feature matching model, uses the heterogeneous feature matching model to perform matching analysis on the drone image and the satellite image, and determines the corresponding block on the satellite image according to the analysis result. The present invention realizes the construction of a heterogeneous feature matching model through the above technical scheme, and realizes the matching of the drone image and the satellite image through the heterogeneous feature matching model, and has the advantages of high accuracy and good robustness.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle image matching, and in particular to a heterogeneous image matching method and system applied to unmanned aerial vehicles. Background Art

[0002] With the rapid development of drone technology, drones are increasingly used in various fields, among which video mapping systems have shown great potential in the fields of geographic information, agricultural monitoring, urban planning, etc. However, existing video mapping systems face a series of challenges in matching heterogeneous images (such as drone images and satellite images), which limits their widespread application in practical applications.

[0003] There are differences in resolution, phase, appearance, perspective, radiation, and orientation between heterogeneous images, which increase the difficulty of image matching. Traditional feature matching algorithms, such as SIFT (Scale Invariant Feature Transform) and ORB (Oriented FAST and Rotated BRIEF), often have difficulty obtaining sufficiently accurate matching points, especially in weak texture areas or when there is occlusion.

[0004] In summary, there is an urgent need for an accurate and effective UAV heterogeneous image matching method and system. Summary of the invention

[0005] The purpose of the present invention is to provide an accurate and effective UAV heterogeneous image matching method and system.

[0006] The present invention discloses a heterogeneous image matching method applied to a drone, comprising:

[0007] Establish the mapping relationship between drone images and satellite image blocks, and build the image corresponding dataset based on the mapping relationship;

[0008] The features of the drone image and satellite image blocks in the image corresponding dataset are intercepted, and based on the corresponding relationship in the image corresponding dataset, the corresponding relationship between the drone image features and the satellite image block features is established to obtain the image feature corresponding dataset;

[0009] The image feature corresponding data set is trained to obtain a heterogeneous feature matching model;

[0010] The heterogeneous feature matching model is used to perform matching analysis on UAV images and satellite images, and the corresponding blocks are determined on the satellite images based on the analysis results.

[0011] In some embodiments disclosed in the present invention, the heterogeneous image matching method further includes:

[0012] The matching of heterogeneous feature matching models is optimized by using multi-threaded optimization matching strategy and inertial interpolation strategy of motion model.

[0013] In some embodiments disclosed in the present invention, the method for optimizing the matching of heterogeneous feature matching models using the inertial difference strategy includes:

[0014] Collect the motion parameters of the drone, analyze them using the motion model, and determine the position change vector between adjacent interval frames;

[0015] The satellite image blocks matched between adjacent interval frames are determined, and based on the position change vectors between the adjacent interval frames, the satellite image blocks are matched to the intermediate frames between the adjacent interval frames.

[0016] In some embodiments disclosed in the present invention, a method for training a dataset corresponding to image features includes:

[0017] Label the drone image features and satellite image block features in the image feature corresponding data set, and divide the labeled image feature set into a training set, a test set, and a verification set;

[0018] Preprocess the image data in the training set, test set, and validation set, including image normalization, size adjustment, and data augmentation;

[0019] Build a CNN model architecture, including convolutional layers, pooling layers, and fully connected layers;

[0020] Select the loss function and the gradient descent optimization algorithm, and use the gradient descent optimization algorithm to reduce the loss function;

[0021] The training set is put into the CNN model architecture for training, and the test set is used to judge the generalization ability of the model during the training process, and the verification set is used to select and tune the model.

[0022] In some embodiments disclosed in the present invention, the heterogeneous image matching method further includes:

[0023] Based on the characteristics of the flight blocks, scene parameter configuration templates for different types of blocks are set, where different types of blocks include: urban built-up blocks, farmland blocks, grassland blocks, forest blocks and desert blocks;

[0024] Based on the type block where the drone is located, determine the parameter configuration template for the application, and adjust the model training or model output results based on the parameter configuration template.

[0025] In some embodiments disclosed in the present invention, the features captured from the drone image and satellite image blocks include: color features, texture features, edge features, and frequency domain features.

[0026] In some embodiments disclosed in the present invention, the heterogeneous image matching method further includes:

[0027] Determine the mapped area of ​​the drone image and the satellite image relative to the ground, and normalize the size of the drone image and the satellite image so that the mapped areas correspond to each other;

[0028] The satellite image is divided into grids, and the blocks corresponding to each grid of the satellite image are scanned and analyzed to determine the edge line distribution characteristics and color distribution characteristics of each grid block, and the edge line distribution characteristics and color distribution characteristics are characterized and assigned respectively to obtain a grid feature value group;

[0029] Acquire drone images in real time, scan and analyze drone image frames, and characterize and assign values ​​to edge distribution features and color distribution features to obtain a drone image feature value group;

[0030] Compare the drone image feature value groups of the continuous drone image frames with the grid feature value groups of several grid blocks. If there is a grid feature value group and a drone image feature value group of the drone image frame whose matching degree is greater than or equal to a preset value, associate the grid block corresponding to the grid feature value group with the drone image frame;

[0031] Based on the proximity of grid blocks corresponding to several consecutive frames of drone image frames, the mapping accuracy of drone images to satellite images is determined. If the mapping accuracy is greater than or equal to the preset value, the corresponding grid blocks are intercepted and combined, and analyzed together with the drone images in the heterogeneous feature matching model.

[0032] In some embodiments disclosed in the present invention, a method for calculating the degree of correspondence between a grid feature value group and a drone image feature value group of a drone image frame includes:

[0033] Determine edge lines in drone image frames or grid blocks whose length is greater than a preset value, and classify and mark the edge lines based on a number of preset length intervals;

[0034] Determine the dispersion of edge lines in different length intervals, combine the number of edge lines in different length intervals and the dispersion of edge lines in different length intervals, and associate them with the drone image frame or grid block;

[0035] Calculate the color variance in the drone image frame or grid block and associate the color variance with the drone image frame or grid block;

[0036] The number of edge lines in different length intervals, the dispersion of edge lines in different length intervals, and the color variance are combined to obtain a grid feature value group or a drone image feature value group;

[0037] Step S2034, constructing an edge line comparison operator based on the number and dispersion of edge lines in different length intervals, constructing a color comparison operator based on the color variance, determining the edge line attention weight and the color attention weight based on the type of the flight block to which the drone belongs, and combining the edge line comparison operator and the color comparison operator to calculate the degree of conformity;

[0038] The expression for calculating the degree of conformity is:

[0039] .

[0040] .

[0041] .

[0042] in, is the corresponding value of the degree of conformity, is the edge line attention weight, Focus weight for color, is the edge line comparison operator, is the color comparison operator, is the first edge line adjustment coefficient, is the second edge line adjustment coefficient, is the difference in the number of edge lines corresponding to the i-th length interval, is the dispersion difference of the edge line corresponding to the i-th length interval, Adjust the constant for the edge line, is the difference in color variance, Adjustment constant for color variance.

[0043] In some embodiments disclosed in the present invention, a method for determining the mapping accuracy of drone images to satellite images includes:

[0044] Determine the position of the grid blocks corresponding to the drone image frame relative to the satellite image, and dynamically randomly combine different grid blocks in pairs, and the number of combinations reaches a preset value;

[0045] The relative distance between the grid blocks in each pair of grid block combinations is determined, and the number of grid block combinations whose relative distance between the grid blocks is greater than or equal to a preset value is determined. Based on the preset combination number interval to which the number of combinations belongs, the mapping accuracy of the drone image to the satellite image is determined.

[0046] In some embodiments disclosed in the present invention, a heterogeneous image matching system applied to a drone is also disclosed, including:

[0047] The first module is used to establish the mapping relationship between drone images and satellite image blocks, and to construct the image corresponding dataset based on the mapping relationship;

[0048] The second module is used to perform feature extraction on the drone image and satellite image blocks in the image corresponding data set, and establish the correspondence between the drone image features and the satellite image block features based on the correspondence in the image corresponding data set to obtain the image feature corresponding data set;

[0049] The third module is used to train the image feature corresponding data set to obtain a heterogeneous feature matching model;

[0050] The fourth module is used to use the heterogeneous feature matching model to perform matching analysis on UAV images and satellite images, and determine the corresponding blocks on the satellite images based on the analysis results.

[0051] The present invention discloses a heterogeneous image matching method applied to a drone, and relates to the technical field of drone image matching. The specifically disclosed technical scheme constructs an image corresponding data set, establishes a corresponding relationship between drone image features and satellite image block features based on the corresponding relationship in the image corresponding data set, obtains an image feature corresponding data set, trains the image feature corresponding data set, obtains a heterogeneous feature matching model, uses the heterogeneous feature matching model to perform matching analysis on the drone image and the satellite image, and determines the corresponding block on the satellite image according to the analysis result. The present invention realizes the construction of a heterogeneous feature matching model through the above technical scheme, and realizes the matching of the drone image and the satellite image through the heterogeneous feature matching model, and has the advantages of high accuracy and good robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a method step diagram of a heterogeneous image matching method applied to unmanned aerial vehicles proposed by the present invention. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0054] The purpose of the present invention is to provide an accurate and effective UAV heterogeneous image matching method and system.

[0055] The present invention discloses a heterogeneous image matching method applied to unmanned aerial vehicles, see Figure 1 ,include:

[0056] Step S100: establishing a mapping relationship between drone images and satellite image blocks, and constructing an image corresponding data set based on the mapping relationship.

[0057] In this step, the mapping relationship between the drone image and the satellite image is established through some georeferencing or geometric transformation methods. This can include finding common feature points, performing coordinate transformation, or implementing it through geographic information system (GIS) tools; the establishment of this mapping relationship is the basis for subsequent steps, ensuring the geometric correspondence between the two images.

[0058] Step S200, feature extraction is performed on the drone image and satellite image blocks in the image corresponding data set, and based on the corresponding relationship in the image corresponding data set, a corresponding relationship between the drone image features and the satellite image block features is established to obtain an image feature corresponding data set.

[0059] In this step, based on the established mapping relationship, the features of the drone image and satellite image blocks are extracted from the image correspondence dataset; these features may include color information, texture information, shape information, etc. Then, based on the correspondence in the image correspondence dataset, the correspondence between the drone image features and the satellite image block features is established; the purpose of this step is to construct a dataset containing image feature correspondences.

[0060] In some embodiments disclosed in the present invention, the heterogeneous image matching method further includes:

[0061] Step S201 , determining the mapping areas of the drone image and the satellite image relative to the ground, and normalizing the size of the drone image and the satellite image in such a way that the mapping areas correspond to each other.

[0062] In this step, we first need to determine the mapping area of ​​the drone image and the satellite image on the ground. This mapping area is determined to ensure that the two have equal sizes in geographic space. Then, the drone image and the satellite image are normalized according to the equal mapping area. The purpose of this is to make them have consistent proportions and sizes, thereby reducing the scale differences in subsequent feature matching, while reducing the model's need to adapt to different scales, which helps to reduce resource consumption.

[0063] Step S202, gridding the satellite image, scanning and analyzing the blocks corresponding to each grid of the satellite image, determining the edge line distribution characteristics and color distribution characteristics of each grid block, and characterizing and assigning values ​​to the edge line distribution characteristics and color distribution characteristics respectively to obtain a grid feature value group.

[0064] Step S203, acquiring drone images in real time, scanning and analyzing the drone image frames, and characterizing and assigning values ​​to edge distribution features and color distribution features respectively to obtain a drone image feature value group.

[0065] In some embodiments disclosed in the present invention, a method for calculating the degree of correspondence between a grid feature value group and a drone image feature value group of a drone image frame includes:

[0066] Step S2031, determining edge lines in the drone image frame or grid block whose length is greater than a preset value, and classifying and marking the edge lines based on a plurality of preset length intervals;

[0067] Step S2032, determining the dispersion of edge lines in different length intervals, combining the number of edge lines in different length intervals and the dispersion of edge lines in different length intervals, and associating them with the drone image frame or grid block;

[0068] Step S2033, calculating the color variance in the drone image frame or the grid block, and associating the color variance with the drone image frame or the grid block;

[0069] Step S2034, combining the number of edge lines in different length intervals, the dispersion and color variance of edge lines in different length intervals to obtain a grid feature value group or a drone image feature value group;

[0070] Step S2035, constructing an edge line comparison operator based on the number and dispersion of edge lines in different length intervals, constructing a color comparison operator based on the color variance, determining the edge line attention weight and the color attention weight based on the type of the flight block to which the drone belongs, and combining the edge line comparison operator and the color comparison operator to calculate the degree of conformity;

[0071] The expression for calculating the degree of conformity is:

[0072] .

[0073] .

[0074] .

[0075] in, is the corresponding value of the degree of conformity, is the edge line attention weight, Focus weight for color, is the edge line comparison operator, is the color comparison operator, is the first edge line adjustment coefficient, is the second edge line adjustment coefficient, is the difference in the number of edge lines corresponding to the i-th length interval, is the dispersion difference of the edge line corresponding to the i-th length interval, Adjust the constant for the edge line, is the difference in color variance, Adjustment constant for color variance.

[0076] In step S204, the drone image feature value groups of continuous drone image frames are compared with the grid feature value groups of several grid blocks. If the degree of consistency between the grid feature value group and the drone image feature value group of the drone image frame is greater than or equal to a preset value, the grid block corresponding to the grid feature value group is associated with the drone image frame.

[0077] In this step, feature matching and association are performed by comparing the feature value groups of continuous drone image frames and several grid blocks; this process is carried out locally, reducing the computational complexity of global matching and thus reducing resource consumption.

[0078] Step S205, based on the proximity performance of the grid blocks corresponding to several consecutive frames of drone image frames, determine the mapping accuracy of the drone image to the satellite image. If the mapping accuracy is greater than or equal to a preset value, the corresponding grid blocks are intercepted and combined, and are put into the heterogeneous feature matching model for analysis together with the drone image.

[0079] The above steps all perform feature extraction and matching, as well as mapping accuracy evaluation, in a local area, which helps to reduce computational complexity, reduce the model's demand for resources, and improve overall efficiency.

[0080] In some embodiments disclosed in the present invention, a method for determining the mapping accuracy of drone images to satellite images includes:

[0081] Step S2051, determining the position of the grid block corresponding to the drone image frame relative to the satellite image, and dynamically and randomly combining different grid blocks in pairs, and the number of combinations reaches a preset value.

[0082] Step S2052, determining the relative distance between the grid blocks in each pair of grid block combinations, and determining the number of grid block combinations whose relative distance between the grid blocks is greater than or equal to a preset value, and determining the mapping accuracy of the drone image to the satellite image based on the preset combination number interval to which the combination number belongs.

[0083] Step S300: training the image feature corresponding data set to obtain a heterogeneous feature matching model.

[0084] In this step, the constructed image feature correspondence dataset is used to train the heterogeneous feature matching model. This involves using machine learning or deep learning algorithms, such as convolutional neural networks (CNNs), to learn the feature matching rules between drone imagery and satellite imagery. The training of the model enables it to accurately infer the correspondence between image features.

[0085] In some embodiments disclosed in the present invention, a method for training a dataset corresponding to image features includes:

[0086] Step S301 , label the drone image features and satellite image block features in the image feature corresponding data set, and divide the labeled image feature set into a training set, a test set, and a verification set.

[0087] In this step, the image feature corresponding data sets are labeled, and the drone image features and satellite image block features are marked as corresponding categories or labels respectively; then, the labeled image feature set is divided into training set, test set and verification set for subsequent model training and evaluation.

[0088] Step S302 , preprocessing the image data in the training set, the test set and the verification set, including image normalization, size adjustment and data enhancement.

[0089] In this step, image normalization is performed to normalize the pixel values ​​of the image so that they fall within a smaller range, which helps accelerate model convergence; resizing is performed to adjust the image to a uniform size for model input to ensure the consistency of input data; data enhancement is performed on the training set, such as random rotation, flipping, scaling, etc., to expand the training data and improve the generalization ability of the model.

[0090] Step S303, constructing a CNN model architecture, including a convolutional layer, a pooling layer, and a fully connected layer.

[0091] In this step, the architecture of the convolutional neural network (CNN) model is designed, including convolutional layers, pooling layers, and fully connected layers. Depending on the complexity of the problem and the characteristics of the dataset, the depth and width of the network are adjusted to ensure that the model can effectively extract features and perform classification.

[0092] Step S304, selecting a loss function and a gradient descent optimization algorithm, and using the gradient descent optimization algorithm to reduce the loss function.

[0093] In this step, an appropriate loss function is selected, which should match the nature of the problem; at the same time, a gradient descent optimization algorithm, such as Adam, SGD, etc., is selected to minimize the loss function and make the model converge gradually.

[0094] Step S305, the training set is put into the CNN model architecture for training, and the test set is used to determine the generalization ability of the model during the training process, and the verification set is used to select and tune the model.

[0095] In this step, the training set is put into the constructed CNN model for training, and the model parameters are optimized by the gradient descent algorithm. During the training process, the test set is used to evaluate the generalization ability of the model and judge the performance of the model on unseen data. At the same time, the verification set is used to select and tune the model to avoid overfitting or underfitting.

[0096] Step S400: Use a heterogeneous feature matching model to perform matching analysis on the drone image and the satellite image, and determine the corresponding block on the satellite image according to the analysis result.

[0097] In this step, the trained heterogeneous feature matching model is used to perform matching analysis on new drone images and satellite images. Through the model, the blocks corresponding to the drone images can be determined on the satellite images. This step provides key information for geo-positioning and map making of drone images in practical applications.

[0098] The core of the entire process of steps S100-S400 is to establish feature correspondences between images and implement automated matching analysis by training models. This method can improve efficiency when facing large-scale remote sensing data and provide reliable support for applications such as geographic information systems and map making.

[0099] In some embodiments disclosed in the present invention, the heterogeneous image matching method further includes:

[0100] Step S500, optimizing the matching of the heterogeneous feature matching model by using the multi-threaded optimization matching strategy and the inertial interpolation strategy of the motion model.

[0101] In this step, in the matching process of the heterogeneous feature matching model, a multi-threaded optimization matching strategy and an inertial interpolation strategy of the motion model are introduced to solve some common problems and improve the robustness and processing efficiency of the system; the following is a brief explanation of these two optimization strategies.

[0102] Multi-threaded optimization matching strategy: (1) Parallel processing: Use multi-threaded parallel processing of matching tasks to make full use of the computing resources of multi-core processors. This can speed up the feature matching process and improve the real-time performance and efficiency of the system. (2) Distributed computing: If the system is large, you can consider introducing distributed computing to distribute matching tasks to multiple computing nodes for processing. This helps to process large-scale data sets and improve the scalability of the overall system. (3) Task splitting and merging: Split the matching task into multiple small tasks and assign them to different threads for processing. Finally, merge the results of each thread to reduce the burden on each thread and improve the concurrency of the system.

[0103] Inertial interpolation strategy for motion models:

[0104] (1) Alleviate the problem of weak texture areas. When there are weak texture areas or some frames have no matching points, the inertial interpolation strategy of the motion model is used to interpolate some frames and predict the location of the matching points. This helps to alleviate the matching difficulties caused by weak texture areas. (2) Complete the matching points. When there are few or insufficient matching points, the interpolation of the motion model can be used to complete the missing matching points and improve the completeness of the matching. (3) Dynamically adjust the matching strategy. Dynamically adjust the matching strategy according to the texture characteristics and motion status of different regions. For complex dynamic scenes, introducing the inertial interpolation of the motion model can better adapt to different situations.

[0105] In some embodiments disclosed in the present invention, the method for optimizing the matching of heterogeneous feature matching models using the inertial difference strategy includes:

[0106] Step S501 , collecting the motion parameters of the drone, and analyzing the motion parameters using a motion model to determine the position change vector between adjacent interval frames.

[0107] Step S502 , determining satellite image blocks that are matched between adjacent interval frames, and matching satellite image blocks to intermediate frames between adjacent interval frames based on position change vectors between adjacent interval frames.

[0108] In some embodiments disclosed in the present invention, the heterogeneous image matching method further includes:

[0109] Step S600, based on the characteristics of the flight block, setting a scene parameter configuration template for different types of blocks, wherein the different types of blocks include: urban built-up blocks, farmland blocks, grassland blocks, forest blocks and desert blocks.

[0110] In this step, according to the characteristics of the flight block, the scene parameter configuration templates of different types of blocks are set; each type of block may have different geographical, environmental and visual characteristics, so the model parameters need to be configured in a targeted manner; these different types of blocks can include: (1) urban built-up blocks, dense buildings, roads, artificial structures, etc.; (2) farmland blocks, large tracts of cultivated land, rural scenery, crops; (3) grassland blocks, natural grassland, steppe areas; (4) forest blocks, dense trees, forest vegetation; (5) desert blocks, dry desert areas, sand dunes; each block type may have different parameter requirements for the feature matching model, such as sensitivity to features such as texture, color, and shape.

[0111] Step S700, based on the type block where the drone is located, determine the parameter configuration template to be applied, and adjust the model training or model output result based on the parameter configuration template.

[0112] In this step, based on the actual type of block the drone is in, select the corresponding template from the set scenario parameter configuration templates. This can be determined based on the geographical characteristics of the flight area, environmental conditions, and specific requirements of the flight mission. After selecting the appropriate parameter configuration template, adjust the model training or model output results to adapt to the special situation of the current block.

[0113] Among them, the parameters that can be adjusted include: (1) feature extraction parameters, adjusting the model's feature extraction layer parameters to make them more suitable for the characteristics of the current block; (2) classification layer parameters, adjusting the classification layer parameters according to the category characteristics of different blocks to improve the model's classification accuracy for specific categories; (3) data enhancement strategy, adjusting the data enhancement method according to the characteristics of the block to make the model more robust.

[0114] In some embodiments disclosed in the present invention, the features captured from the drone image and satellite image blocks include: color features, texture features, edge features, and frequency domain features.

[0115] In some embodiments disclosed in the present invention, a heterogeneous image matching system applied to a drone is also disclosed, including:

[0116] The first module is used to establish the mapping relationship between drone images and satellite image blocks, and to construct the image corresponding dataset based on the mapping relationship;

[0117] The second module is used to perform feature extraction on the drone image and satellite image blocks in the image corresponding data set, and establish the correspondence between the drone image features and the satellite image block features based on the correspondence in the image corresponding data set to obtain the image feature corresponding data set;

[0118] The third module is used to train the image feature corresponding data set to obtain a heterogeneous feature matching model;

[0119] The fourth module is used to use the heterogeneous feature matching model to perform matching analysis on UAV images and satellite images, and determine the corresponding blocks on the satellite images based on the analysis results.

[0120] The present invention discloses a heterogeneous image matching method applied to a drone, and relates to the technical field of drone image matching. The specifically disclosed technical scheme constructs an image corresponding data set, establishes a corresponding relationship between drone image features and satellite image block features based on the corresponding relationship in the image corresponding data set, obtains an image feature corresponding data set, trains the image feature corresponding data set, obtains a heterogeneous feature matching model, uses the heterogeneous feature matching model to perform matching analysis on the drone image and the satellite image, and determines the corresponding block on the satellite image according to the analysis result. The present invention realizes the construction of a heterogeneous feature matching model through the above technical scheme, and realizes the matching of the drone image and the satellite image through the heterogeneous feature matching model, and has the advantages of high accuracy and good robustness.

[0121] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A heterogeneous image matching method applied to unmanned aerial vehicles, characterized in that: include: Establish the mapping relationship between drone images and satellite image blocks, and build the image corresponding dataset based on the mapping relationship; The features of the drone image and satellite image blocks in the image corresponding dataset are intercepted, and based on the corresponding relationship in the image corresponding dataset, the corresponding relationship between the drone image features and the satellite image block features is established to obtain the image feature corresponding dataset; The image feature corresponding data set is trained to obtain a heterogeneous feature matching model; Use heterogeneous feature matching models to match and analyze UAV images and satellite images, and determine corresponding blocks on satellite images based on the analysis results; Also includes: Determine the mapped area of ​​the drone image and the satellite image relative to the ground, and normalize the size of the drone image and the satellite image so that the mapped areas correspond to each other; The satellite image is divided into grids, and the blocks corresponding to each grid of the satellite image are scanned and analyzed to determine the edge line distribution characteristics and color distribution characteristics of each grid block, and the edge line distribution characteristics and color distribution characteristics are characterized and assigned respectively to obtain a grid feature value group; Acquire drone images in real time, scan and analyze drone image frames, and characterize and assign values ​​to edge distribution features and color distribution features to obtain a drone image feature value group; Compare the drone image feature value groups of the continuous drone image frames with the grid feature value groups of several grid blocks. If there is a grid feature value group and a drone image feature value group of the drone image frame whose matching degree is greater than or equal to a preset value, associate the grid block corresponding to the grid feature value group with the drone image frame; Based on the proximity of grid blocks corresponding to several consecutive frames of drone image frames, the mapping accuracy of drone images to satellite images is determined. If the mapping accuracy is greater than or equal to the preset value, the corresponding grid blocks are intercepted and combined, and analyzed together with the drone images in the heterogeneous feature matching model.

2. The heterogeneous image matching method applied to a drone according to claim 1, characterized in that: Also includes: The matching of heterogeneous feature matching models is optimized by using multi-threaded optimization matching strategy and inertial interpolation strategy of motion model.

3. The heterogeneous image matching method applied to a drone according to claim 2 is characterized in that: Methods for optimizing the matching of heterogeneous feature matching models using the inertial difference strategy include: Collect the motion parameters of the drone, analyze them using the motion model, and determine the position change vector between adjacent interval frames; The satellite image blocks matched between adjacent interval frames are determined, and based on the position change vectors between the adjacent interval frames, the satellite image blocks are matched to the intermediate frames between the adjacent interval frames.

4. The heterogeneous image matching method applied to a drone according to claim 1, characterized in that: Methods for training image feature corresponding data sets include: Label the drone image features and satellite image block features in the image feature corresponding data set, and divide the labeled image feature set into a training set, a test set, and a verification set; Preprocess the image data in the training set, test set, and validation set, including image normalization, size adjustment, and data augmentation; Build a CNN model architecture, including convolutional layers, pooling layers, and fully connected layers; Select the loss function and the gradient descent optimization algorithm, and use the gradient descent optimization algorithm to reduce the loss function; The training set is put into the CNN model architecture for training, and the test set is used to judge the generalization ability of the model during the training process, and the verification set is used to select and tune the model.

5. The heterogeneous image matching method applied to a drone according to claim 1, characterized in that: Also includes: Based on the characteristics of the flight blocks, scene parameter configuration templates for different types of blocks are set, where different types of blocks include: urban built-up blocks, farmland blocks, grassland blocks, forest blocks and desert blocks; Based on the type block where the drone is located, determine the parameter configuration template for the application, and adjust the model training or model output results based on the parameter configuration template.

6. The heterogeneous image matching method applied to a drone according to claim 1, characterized in that: The features captured from drone image and satellite image blocks include: color features, texture features, edge features and frequency domain features.

7. The heterogeneous image matching method applied to a drone according to claim 1, characterized in that: The method for calculating the degree of correspondence between the grid feature value group and the drone image feature value group of the drone image frame includes: Determine edge lines in drone image frames or grid blocks whose length is greater than a preset value, and classify and mark the edge lines based on a number of preset length intervals; Determine the dispersion of edge lines in different length intervals, combine the number of edge lines in different length intervals and the dispersion of edge lines in different length intervals, and associate them with the drone image frame or grid block; Calculate the color variance in the drone image frame or grid block and associate the color variance with the drone image frame or grid block; The number of edge lines in different length intervals, the dispersion of edge lines in different length intervals, and the color variance are combined to obtain a grid feature value group or a drone image feature value group; An edge line comparison operator is constructed based on the number and dispersion of edge lines in different length intervals, and a color comparison operator is constructed based on color variance. The edge line attention weight and color attention weight are determined based on the type of flight block to which the drone belongs. The edge line comparison operator and the color comparison operator are combined to calculate the degree of match. The expression for calculating the degree of conformity is: ； ； ； in, is the corresponding value of the degree of conformity, is the edge attention weight, Focus weight for color, is the edge line comparison operator, is the color comparison operator, is the first edge line adjustment coefficient, is the second edge line adjustment coefficient, is the difference in the number of edge lines corresponding to the i-th length interval, is the dispersion difference of the edge line corresponding to the i-th length interval, Adjust the constant for the edge line, is the difference in color variance, Adjustment constant for color variance.

8. The heterogeneous image matching method applied to a drone according to claim 1, characterized in that: Methods for determining the accuracy of mapping drone imagery to satellite imagery include: Determine the position of the grid blocks corresponding to the drone image frame relative to the satellite image, and dynamically randomly combine different grid blocks in pairs, and the number of combinations reaches a preset value; The relative distance between the grid blocks in each pair of grid block combinations is determined, and the number of grid block combinations whose relative distance between the grid blocks is greater than or equal to a preset value is determined. Based on the preset combination number interval to which the number of combinations belongs, the mapping accuracy of the drone image to the satellite image is determined.

9. A heterogeneous image matching system applied to unmanned aerial vehicles, characterized in that: A heterogeneous image matching method for executing any one of claims 1 to 8, comprising: The first module is used to establish the mapping relationship between drone images and satellite image blocks, and to construct the image corresponding dataset based on the mapping relationship; The second module is used to perform feature extraction on the drone image and satellite image blocks in the image corresponding data set, and establish the correspondence between the drone image features and the satellite image block features based on the correspondence in the image corresponding data set to obtain the image feature corresponding data set; The third module is used to train the image feature corresponding data set to obtain a heterogeneous feature matching model; The fourth module is used to use the heterogeneous feature matching model to perform matching analysis on UAV images and satellite images, and determine the corresponding blocks on the satellite images based on the analysis results.

Images