Air-space-ground data matching method and system

Abstract

The invention provides an air-space-ground data matching method and system, and belongs to the field of data processing. The method comprises the steps that an air-space-ground positioning data set is constructed, and the air-space-ground positioning data set comprises an unmanned aerial vehicle view angle image, a ground view angle image and a satellite view angle image; and cross-view-angle target matching is carried out on the unmanned aerial vehicle view angle image, the ground view angle image and the satellite view angle image by using a view angle matching method based on interaction enhancement convergence, and matching of space-air-ground data is completed. According to the invention, the technical problems of lack of real and complex scenes and incomplete view angle coverage in the existing data set and the technical problems of low matching accuracy and unstable positioning caused by isolated image features and insufficient information fusion in the traditional space-air-ground view angle matching method are solved; therefore, the matching performance among the multi-view images and the stability and precision of the positioning system are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of data processing, and particularly relates to a space-air-ground data matching method and system. BACKGROUND

[0002] Cross-view geolocalization is an emerging visual positioning technology that breaks the dependence on traditional global navigation satellite system (GNSS) signals, and has important application value in complex environments such as urban canyons, tunnels, forest areas, or indoors where GNSS signals are weak or unavailable. This technology achieves accurate inference of the shooting location by comparing the visual similarity between the captured image and the reference image with geographic location information. Currently, computer vision-based geolocalization methods mostly rely on deep neural networks to model the spatial correspondence between images of different perspectives. Common strategies include extracting structured semantic features such as road structures, building outlines, and terrain textures from ground and satellite images, establishing semantic associations between perspectives, and combining methods such as region alignment, attention mechanisms, and feature contrast learning to improve the model's matching robustness under perspective differences and scale changes.

[0003] However, existing research generally faces two key technical bottlenecks. First, current mainstream cross-view positioning datasets are mostly composed of ground and satellite perspectives, and image sources mainly rely on platforms such as Google Street View and Google Earth, lacking real-world complex environment images, resulting in insufficient model generalization ability in actual scenarios. Moreover, these datasets generally do not consider the introduction of unmanned aerial vehicle (low-altitude) perspectives, limiting the research of air-ground-sky multi-platform joint positioning systems. With the development of unmanned aerial vehicle technology, unmanned aerial vehicles have become an important intermediary platform connecting ground and satellite perspectives. On the one hand, they can achieve air-ground collaborative perception, improving the spatial continuity of ground target recognition and tracking. On the other hand, they can also form vertical perspective pairing with satellite images, supporting high-precision unmanned aerial vehicle positioning and navigation. However, due to the cost and environmental complexity of unmanned aerial vehicle image acquisition, many current studies use virtual simulation to generate pseudo-unmanned aerial vehicle perspective images from satellite images, lacking the diversity and semantic consistency of real low-altitude images. At the same time, related research is mostly limited to dual-platform matching tasks, lacking unified modeling of ground, unmanned aerial vehicle, and satellite perspectives, and unable to meet the cross-perspective comprehensive perception and positioning needs in complex application scenarios.

[0004] Secondly, the existing deep learning matching method mainly focuses on one-way encoding and similarity measurement of images, and the semantic information between the perspectives is difficult to effectively interact and integrate, resulting in a split phenomenon in feature expression, which severely restricts the cross-angle matching performance, especially under complex conditions such as large terrain undulation, strong light change and extreme view angle difference. The lack of a perspective matching mechanism with feature interaction enhancement and aggregation capability has become a major obstacle to further improving the performance of cross-view geographic positioning. SUMMARY

[0005] In view of the above problems in the prior art, the space-air-ground data matching method and system provided by the present application solve the problems of lack of three-source real data in the existing cross-perspective target matching data set and large gap between cross-domain features of the same target in the existing cross-perspective target matching method.

[0006] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows: a space-air-ground data matching method, characterized in that it comprises the following steps: S1, constructing a space-air-ground positioning data set, wherein the space-air-ground positioning data set comprises unmanned aerial vehicle perspective images, ground perspective images and satellite perspective images; S2, based on the constructed space-air-ground positioning data set, using an interactive enhancement and aggregation based perspective matching method to perform cross-perspective target matching on the unmanned aerial vehicle perspective images, the ground perspective images and the satellite perspective images, and completing the matching of the space-air-ground data.

[0007] The present application has the beneficial effects that: the present application constructs a space-air-ground multi-source image matching data set covering three perspectives of ground, unmanned aerial vehicle and satellite, and proposes a unified feature extraction and alignment method based on interactive enhancement and aggregation, which realizes structure consistency modeling under multi-scale and multi-perspective. The method and system not only can significantly improve the accuracy and robustness of cross-platform image matching, but also provide a network structure clear, function complete and easy to extend solution for space-air-ground collaborative perception and high-precision spatial positioning in complex scenes, which has good practical value and promotion prospect, and can effectively solve the problems of lack of three-source joint data in the existing cross-perspective image matching method, and the technical difficulties of insufficient matching precision and poor generalization ability caused by focusing on single or double perspective matching task.

[0008] Further, the S2 comprises the following steps: S201, reading the unmanned aerial vehicle perspective images, the ground perspective images and the satellite perspective images and the corresponding geographic coordinate information from the constructed space-air-ground positioning data set, and obtaining multi-perspective cross-platform target data with geographic position labels; S202, according to the multi-perspective cross-platform target data, using affine transformation and projection transformation to expand data diversity and simulate different perspective scenes; S203, input the multi-view cross-platform target data into three backbone networks with the same structure respectively for feature extraction to obtain satellite image features, unmanned aerial vehicle image features and ground image features, wherein the satellite image features, the unmanned aerial vehicle image features and the ground image features are collectively referred to as target features; S204, the target features are divided into query image feature matrix and gallery feature matrix according to the simulated scene results, wherein, represents the number of query samples, represents the feature dimension, represents the number of gallery samples; S205, the loss functions of the three backbone networks are jointly optimized, and the final loss function is calculated by using the weighted average method, wherein the final loss function jointly constrains the three backbone networks with the same structure to map different platform images to the same feature space; S206, based on the image mapping results, the target features of the multi-view cross-platform target data are further mapped to the Gaussian sphere respectively to obtain query image feature matrix and gallery image matrix . S207, the features of the query image feature matrix and the gallery image matrix are extracted, the original similarity matrix of the query image feature matrix and the gallery image feature matrix is calculated. ; S208, based on the original similarity matrix , feature enhancement convergence is carried out between the query image features and the gallery features, and between the gallery features and the gallery features themselves to obtain enhanced query image features and gallery image features, and to calculate an enhanced similarity matrix . S209, the enhanced similarity matrix and the original similarity matrix are fused to obtain a final similarity matrix. S2010, using the final similarity matrix, target matching is performed on the query image to obtain the best matching target, and cross-view target matching is completed.

[0009] The beneficial effects of the above further scheme are: the application proposes a view matching method based on interactive enhancement convergence mechanism, which can effectively improve the feature correlation and matching robustness between different view images. The method has good universality and can be applied to various view matching frameworks, which has important significance for realizing stable and accurate positioning in complex environments.

[0010] Furthermore, the expression for the original similarity in S207 is as follows:

[0011] in, Indicates the original similarity. Indicates the first The query image and the first Similarity between images in the image library T This indicates the transpose operation.

[0012] Furthermore, step S208 includes the following steps: Obtain the K-nearest neighbor image database features for each query image, based on the original similarity matrix. For each query feature, sort them in descending order of similarity, and select the top... Index of library features ; Based on the selected previous Index of library features Extract the corresponding nearest neighbor image library feature matrix. For each query image feature Calculate the attention weights between the feature and the feature of the nearest graph library. ; Based on attention weights Enhance query features by weighted aggregation of features from neighboring graph databases. ; Information-enhanced query features Obtain the enhanced query feature matrix ; Calculate the cosine similarity matrix between image features in the image library. : ; Get each query image Nearest neighbor image database features, for the cosine similarity matrix For each query feature, sort them in descending order of similarity and select the top... Index of library features ; Based on the selected previous Index of library features Extract the corresponding nearest neighbor image library feature matrix. For each library feature Calculate the attention weights with neighboring features. ; Based on attention weights Nearest neighbor feature information is aggregated by weighting. The enhanced image library feature matrix is ​​obtained. ; Based on the enhanced query feature matrix Image library feature matrix Calculate the enhanced similarity matrix .

[0013] The beneficial effects of the above-mentioned further scheme are as follows: This invention introduces K-nearest neighbor features based on similarity ranking into the feature space and uses an attention mechanism to weighted aggregate them, thereby achieving bidirectional information enhancement of query features and image library features, effectively improving the discriminative and expressive capabilities of the features. This method not only integrates discriminative information from local neighborhoods, mitigating the impact of scale, angle, and illumination differences across viewpoints on matching accuracy, but also highlights more reliable nearest neighbor features through attention weights, suppressing the influence of interfering features. This allows the enhanced query features and image library features to achieve a more stable and consistent mapping in a unified feature space. Based on this, the final calculated similarity matrix more accurately reflects the semantic similarity between images across platforms, thus significantly improving the accuracy and robustness of cross-viewpoint target retrieval and matching.

[0014] Furthermore, the information-enhanced query features The expression is as follows: ; ; ; ; in, This represents the feature matrix of the library that is most similar to the query feature. k Nearest neighbor features, k This indicates the order of the first K feature sequences. K Indicates the range of nearest neighbor features. Represents the variable to be summed. T This indicates the transpose operation. Indicates the first k The original indices of the nearest neighbor features in the feature matrix of the image library. This represents the variable to be summed.

[0015] The beneficial effects of the above-mentioned further solutions are: the present invention enables the query features to fully absorb the common information of similar features in the database through data-driven adaptive weighted fusion of nearest neighbor features, strengthens the semantic association of features, and effectively improves the feature alignment accuracy when matching multi-source data from air, space, and ground. Furthermore, the nearest neighbor feature information The expression is as follows: ; ; ; ; in, k Indicates the preceding The sorting of the feature sequences Indicates the range of nearest neighbor features. Represents the variable to be summed. The first element in the feature matrix that is most similar to a feature in the library is the [feature name]. k Nearest neighbor features, Represents the variable to be summed. The first element in the feature matrix that is most similar to a feature in the library is the [feature name]. Nearest neighbor features, Indicates the first The original indices of the nearest neighbor features in the feature matrix of the image library. Indicates the first k The original indices of the nearest neighbor features in the feature matrix of the library.

[0016] The beneficial effects of the above-mentioned further solutions are as follows: This invention, through data-driven adaptive weighted fusion of nearest neighbor features, enables the image library features to absorb information from similar nearest neighbor features in the library, thereby strengthening the semantic expression of features. In the context of air-space-ground data matching, it can improve the accuracy of feature correlation mining. Faced with multi-source heterogeneous and noisy data, it enhances the robustness of feature matching by leveraging the data-driven adaptive nearest neighbor selection and weighting mechanism. Furthermore, the enhanced similarity matrix The expression is as follows: .

[0017] The beneficial effects of the above-mentioned further solutions are: the present invention obtains a new enhanced similarity matrix by enhancing the query feature matrix and enhancing the image library feature sentences. With the help of the neighbor similarity feature information incorporated in the previous feature enhancement process, the matrix elements can more accurately reflect the degree of correlation between air, space, and ground multi-source data, and enhance the distinguishability of effective feature matching.

[0018] Furthermore, the final similarity matrix is ​​expressed as follows: ; in, This represents the final similarity matrix. Indicates the fusion weight. Represents the enhanced similarity matrix. Indicates the original similarity.

[0019] The beneficial effect of the above-mentioned further solutions is that the present invention introduces fusion weights. The enhanced similarity matrix is obtained by linear fusion with the original similarity matrix The final similarity matrix is obtained by linear fusion . Not only does it retain the basic matching information contained in the original similarity matrix , ensuring that the data remains associated and is not lost, but it also injects precise association information enhanced by the enhanced similarity matrix of the neighborhood features, improving the matching discrimination. At the same time, the fusion weight supports flexible regulation and can be adjusted according to the characteristics of space-air-ground data (such as the degree of multi-source heterogeneity, noise level, etc.).

[0020] The present application provides a space-air-ground data matching system, comprising: An interface interaction control module for building a graphical user interface and receiving user operation instructions; An application controller module for generating a space-air-ground data matching task after receiving the user's matching instructions, and transmitting the image path and parameters to the backend matching module; A backend matching module for automatically selecting the corresponding feature extraction backbone network according to the image path, parameters, and view type of the image, and performing image feature extraction and cross-view matching operations based on the space-air-ground data matching task; A result export module for recording and outputting the matching results; A main program module for starting and managing the space-air-ground data matching system, wherein the graphical user interface environment is built and initialized during the initialization phase of the space-air-ground data matching system.

[0021] The present application has the following advantages: the present application constructs a space-air-ground multi-source image matching data set covering three views of ground, unmanned aerial vehicle and satellite, and proposes a unified feature extraction and alignment method based on interactive enhancement convergence, realizing structure consistency modeling under multi-scale and multi-view. This method and system not only significantly improve the accuracy and robustness of cross-platform image matching, but also provide a network structure clear, complete function, easy to expand solution for space-air-ground collaborative perception and high-precision spatial positioning in complex scenes, with good practical value and promotion prospect, which can effectively solve the problem of lack of three-source joint data in existing cross-view image matching methods, and the technical difficulties of insufficient matching accuracy and poor generalization ability caused by focusing on single or double view matching tasks. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 The method flowchart of the present application.

[0023] Figure 2 The system structure schematic diagram of the present application. DETAILED DESCRIPTION

[0024] The specific embodiments of the present application are described below to enable those skilled in the art to understand the present application, but it should be clear that the present application is not limited to the scope of the specific embodiments, and for those skilled in the art, it is obvious that various changes are within the spirit and scope of the present application defined and determined by the appended claims, and all the inventions utilizing the concept of the present application are within the scope of protection.

[0025] Embodiment 1 To solve the problems in the background art, it is urgent to construct a high-quality image matching positioning dataset covering real complex scenes, and containing three types of views of ground, unmanned aerial vehicle and satellite, and to propose a matching method with inter-view feature interaction and convergence mechanism, so as to comprehensively improve the performance of cross-platform visual positioning system in terms of precision, robustness and generalization ability. This has important theoretical value and practical significance for promoting the development of air-ground-sky multi-source perception fusion positioning technology, such as Figure 1 As shown in the figure, an air-space-ground data matching method is implemented as follows: S1, constructing an air-space-ground positioning dataset, wherein the air-space-ground positioning dataset includes unmanned aerial vehicle view images, ground view images and satellite view images; In this embodiment, first, the geographical position of the target area is obtained based on the satellite image, and the satellite image is cropped, with an image size of 512x512 and the target area centered. The unmanned aerial vehicle image is collected according to the satellite image position, and the unmanned aerial vehicle image is collected in a three-round flight mode, with 20 pictures taken each time, and the flight height is 50, 60 and 70 m, a total of 60 pictures, and the size of the unmanned aerial vehicle picture is uniformly scaled to 1920x1080. The ground view is collected according to the satellite image position, and a camera is used to take 20 pictures around the target on the ground, with the size uniformly scaled to 1920x1080.

[0026] In this embodiment, the collected image data is labeled with geographical position as anchor point, and three types of images are constructed into one-to-one image groups, each group of three views corresponding to the same geographical target area, and the same target area of three platform data is identified with a uniform serial number, specifically: With geographical position as anchor point, three types of images are constructed into one-to-one image groups, each group of three views corresponding to the same geographical target area. The collected image data is aligned in coordinates and registered in view, and the images from three types of platforms are constructed into one-to-one image groups with geographical position as anchor point, each group of three view images corresponding to the same geographical target area, realizing spatial consistency pairing across views.

[0027] In this embodiment, the collected images are preprocessed, including image cropping, size unification, color normalization, illumination correction and the like, to meet the input specification requirements of the subsequent deep learning model.

[0028] In this embodiment, the GPS information is combined with the geographic reference system to label the geographic coordinates, shooting time and the like of each group of images, and a mapping relationship between the image group and the geographic label is established. The data set is organized and identified, a uniform naming rule is used for sample numbering, the metadata such as image source, shooting time and geographic coordinates are clearly labeled, and a convenient indexing mode and grouping strategy are provided for subsequent algorithm training.

[0029] Through the above steps, the space-air-ground positioning data set constructed has multi-view, multi-scale and high-quality image matching samples, can widely support the double-view matching tasks between unmanned aerial vehicle and ground image, unmanned aerial vehicle and satellite image, ground and satellite image, and the multi-view target recognition and positioning algorithm test of three-source joint, and provides basic support for space-air-ground collaborative perception and spatial positioning research.

[0030] The present application constructs a real image matching and positioning data set covering multi-view of air, ground and space, and the data set contains high-quality image labeling information. For each target building, 20 ground view images, 60 unmanned aerial vehicle view images, corresponding satellite images and geographic position information are collected, the data is rich and accurate, and fully meets the research needs of complex view matching and positioning tasks.

[0031] S2, based on the constructed space-air-ground positioning data set, a view matching method based on interactive enhancement convergence is used to perform cross-view target matching on the unmanned aerial vehicle view image, the ground view image and the satellite view image, and the matching of the space-air-ground data is completed. In this embodiment, the present application proposes a view matching method based on interactive enhancement convergence, which is used to realize cross-view target matching between ground view, unmanned aerial vehicle view and satellite view images, and improve the image alignment and spatial position association ability between different platforms, and the implementation method is as follows: S201, read the unmanned aerial vehicle view image, the ground view image and the satellite view image and the corresponding geographic coordinate information from the constructed space-air-ground positioning data set, and obtain multi-view cross-platform target data with geographic position label; S202, according to the multi-view cross-platform target data, affine transformation and projection transformation are used to expand the data diversity and simulate different view scenes; S203, input the multi-view cross-platform target data into three backbone networks with the same structure respectively for feature extraction, to obtain satellite image features, unmanned aerial vehicle image features and ground image features, wherein the satellite image features, the unmanned aerial vehicle image features and the ground image features are collectively referred to as target features. S204, dividing the target features into a query image feature matrix and a gallery feature matrix according to the simulated scene result and a gallery feature matrix , wherein, represents the number of query samples, represents the feature dimension, represents the number of gallery samples; In a specific task, the target features are further divided into a query image feature matrix and a gallery feature matrix according to the retrieval scene and a gallery feature matrix For example, in a "ground-to-satellite" retrieval task, the ground image features are organized into a query matrix , and the satellite image features are organized into a gallery matrix ; S205, jointly optimizing the loss functions of the three backbone networks, and calculating the final loss function by using a weighted average method, wherein the final loss function jointly constrains the three backbone networks with the same structure to map different platform images to the same feature space; In this embodiment, the instance loss is used to constrain the features of the three source images before normalization, and the cross-entropy loss and the contrast loss function are used to constrain the normalized local features, and the final loss is calculated by using a weighted sum method. For example, the instance loss of the satellite image can be expressed as: ; , wherein, represents the instance loss of the satellite image, represents the satellite view image, represents the satellite view image feature extraction network, represents the weight of the last layer of the classification layer.

[0032] The final weighted sum loss function is:

[0033] , wherein, represents the final weighted sum loss function, , respectively represent the instance loss of the unmanned aerial vehicle and the ground view image.

[0034] S206, based on the image mapping result, further mapping the target features of the multi-view cross-platform target data respectively to obtain a query image feature matrix and a gallery image matrix ; In this embodiment, the features of multi-view cross-platform data are further mapped, and batch normalization and forward propagation layers are used to further map them onto a Gaussian sphere. Specifically: Both the query image feature matrix and the library image feature matrix are L2 normalized.

[0035] To query the image feature matrix For example, for each query image feature First, calculate its L2 norm: ; Then, normalization was performed: ; in, To minimize the value, avoid dividing by zero.

[0036] This yields the normalized query image feature matrix. .

[0037] Similarly, for the image feature matrix of the image library After calculating the L2 norm of the vectors and then normalizing them, we can obtain the normalized image matrix of the image library: .

[0038] S207. Extract the feature matrix of the query image. Image matrix of the image library Calculate the feature matrix of the query image based on its characteristics. Image feature matrix of the image library The original similarity matrix ; In this embodiment, cross-modal interaction enhancement is performed on query features and image library features. First, the image feature matrix is ​​calculated. Image feature matrix of the image library Calculate the original similarity, and then use cosine similarity to calculate the similarity between each query feature and all library features: ; in, Indicates the original similarity. Indicates the first The query image and the first Similarity between images in the image library T This indicates the transpose operation.

[0039] S208, Based on the original similarity matrix Feature enhancement convergence is performed between query image features and library features, as well as between library features and themselves, to obtain enhanced query image features and library image features, and then the enhanced similarity matrix is ​​calculated. Specifically, it is as follows: Obtain the K-nearest neighbor image database features for each query image, based on the original similarity matrix. For each query feature, sort them in descending order of similarity, and select the top... Index of library features ; Based on the selected previous Index of library features Extract the corresponding nearest neighbor image library feature matrix. For each query image feature Calculate the attention weights between the feature and the feature of the nearest graph library. ; Based on attention weights Enhance query features by weighted aggregation of features from neighboring graph databases. ; Information-enhanced query features Obtain the enhanced query feature matrix ; Calculate the cosine similarity matrix between image features in the image library. ; Get each query image Nearest neighbor image database features, for the cosine similarity matrix For each query feature, sort them in descending order of similarity and select the top... Index of library features ; Based on the selected previous Index of library features Extract the corresponding nearest neighbor image library feature matrix. For each library feature Calculate the attention weights with neighboring features. ; Based on attention weights Nearest neighbor feature information is aggregated by weighting. The enhanced image library feature matrix is ​​obtained. ; Based on the enhanced query feature matrix Image library feature matrix Calculate the enhanced similarity matrix .

[0040] In this embodiment, the information obtained for each query image is... Nearest neighbor image database features, for the original similarity matrix For each row (each query feature), sort them in descending order of similarity, and select the top... Index of library features: .

[0041] extract the corresponding neighbor gallery feature matrix .

[0042] For each query image feature , compute its attention weight with neighbor gallery features: ; where denotes the attention weight of the th query feature to the th neighbor gallery feature.

[0043] Then, enhance the query feature by weighted aggregation of neighbor gallery features' information: ; Finally, obtain the enhanced query feature matrix: .

[0044] Subsequently, do self-interaction enhancement for gallery features in the same modality. First, compute the cosine similarity between gallery features: ; where denotes the similarity between the th gallery image and the th gallery image.

[0045] Then, obtain the neighbor gallery features for each query image. For each row (each query feature) of , sort the first gallery features in descending order of similarity, and select the index of the gallery features: ; where denotes the k th nearest feature in the gallery feature matrix that is most similar to the query feature, k denotes the ranking in the first K feature sequence, K denotes the neighbor feature range, denotes the summation variable, T denotes the transpose operation, denotes the original sequence number of the k th nearest feature in the gallery feature matrix, denotes the summation variable.

[0046] extract the corresponding neighbor gallery feature matrix , for each gallery feature , compute the attention weight with neighbor features:

[0047] Weighted aggregated neighborhood feature information , to obtain an enhanced gallery feature matrix .

[0048] wherein, k denotes the order in the first feature sequence, denotes the neighborhood feature range, denotes the summation variable, denotes the first k neighbor feature in the gallery feature matrix that is most similar to the gallery feature, denotes the summation variable, denotes the first neighbor feature in the gallery feature matrix that is most similar to the gallery feature, denotes the original sequence number of the first neighbor feature in the gallery feature matrix, denotes the original sequence number of the first k neighbor feature in the gallery feature matrix.

[0049] obtaining an enhanced query feature matrix and a gallery feature matrix , and then calculating the similarity of the two enhanced matrices .

[0050] S209, fusing the enhanced similarity matrix and the original similarity matrix to obtain a final similarity matrix; In this embodiment, the enhanced feature and the original feature are weighted to obtain the final similarity matrix, and the target matching of the query image is performed to obtain the best matching target, which is specifically: fuse the enhanced matrix similarity and the original similarity , set the fusion weight as , and use it to balance the enhanced feature similarity (local structure information) and the original feature similarity (global feature information), thereby obtaining the final similarity . wherein, denotes the final similarity matrix, denotes the fusion weight, denotes the enhanced similarity matrix, denotes the original similarity.

[0051] perform Top-K ordering according to the final similarity , and match to obtain the The candidate matches construct a Rank List, and each element in the Rank List contains an aerial image feature vector and its similarity to the ground image feature.

[0052] A threshold is set for screening the matching pairs in the Rank List. When the similarity of the first element in the Rank List is greater than the threshold , the matching pair is considered to be valid. Finally, the accuracy of the matching is further verified by checking the consistency of the matching pair in different images.

[0053] S2010, target matching is performed on the query image by using the final similarity matrix, and the best matching target is obtained, and the cross-view target matching is completed.

[0054] The view matching method based on interactive enhancement convergence provided by the application can effectively adapt to the scale difference and content difference between multi-source heterogeneous views, realize high-robustness and high-precision image matching and geographical position alignment in a complex scene, and provide key support for space-air-ground collaborative perception and three-source fusion positioning system.

[0055] Embodiment 2 As shown in Figure 2 , the application provides an air-space-ground data matching system for executing the air-space-ground data matching method described in embodiment 1, comprising: An interface interaction control module for constructing a graphical user interface and receiving user operation instructions; An application controller module for generating an air-space-ground data matching task after the user's matching instructions, and transmitting image paths and parameters to a backend matching module; A backend matching module for automatically selecting a corresponding feature extraction backbone network according to the image paths, parameters and view types of the images, and performing image feature extraction and cross-view matching operations based on the air-space-ground data matching task; A result export module for recording and outputting the matching results; A main program module for starting and managing the air-space-ground data matching system, wherein the graphical user interface environment is built and initialized in the initialization stage of the air-space-ground data matching system.

[0056] In this embodiment, as shown in Figure 2 , the system includes an image uploading module M10, an image feature extraction module M20 and a feature search and matching module M30, wherein: The interface interaction control module M10 is configured to build a graphical user interface and receive operation instructions from the user. The module is provided with interactive controls such as uploading images, starting matching, clearing results, and exiting the system. The user can import an image file to be queried through the interface, and the supported image types include JPEG and PNG formats. After uploading the image, the system will check the format and integrity of the image to ensure the validity of the input data. The module is also used to receive and display the matching results, including the matching image and its corresponding latitude and longitude coordinate information, to facilitate the user to obtain the positioning results.

[0057] The application controller module M20 serves as the scheduling and process control center of the system, and is configured to verify the user behavior and coordinate the execution of each functional module. After receiving the matching request from the user, the controller first determines whether there is an uploaded image, and generates a matching task according to the set task parameters (such as matching accuracy level: Top-1, Top-5, Top-10, etc.). Then, the controller transmits the image path and parameters to the backend matching module M30, and listens to the processing state. After the matching is completed, the controller synchronously transmits the results to the interface interaction control module M10 for visualization display, and sends the related data to the result export module M40 to complete the recording operation.

[0058] The backend matching module M30 is configured to perform image feature extraction and cross-view matching operations. The module receives the image path transmitted by the controller, and automatically selects the corresponding feature extraction branch network according to the view type of the image (such as ground image, unmanned aerial vehicle image, or satellite image). After the input image is processed by the network, a high-dimensional feature vector is generated, and then compared with the stored multi-source image features in the image library. According to the preset matching accuracy, the paths of several most similar images and their corresponding target geographical coordinates (longitude and latitude) are returned. The module uses a trained deep neural network model, which can support later iteration and update to improve the matching accuracy.

[0059] The result export module M40 is configured to record and output the matching results in a standardized manner. The module receives the matching result information from the controller, and saves the content in a preset format, including the target image label, latitude and longitude coordinates, processing time, and generation timestamp. The recorded results can be output as a log file, a CSV text, or a database structured data to meet the later query, comparison, and traceability requirements. The module supports automatic recording mechanism and has stable data export capability.

[0060] The main program module M50 is used for overall startup and resource management of the system. During the system initialization phase, the main program module M50 completes the construction of the graphical user interface environment, and uniformly initializes and configures each functional module, and enters the event loop. During the system running process, the main program continuously maintains the communication and running state between each module, and ensures the overall coordination and stability of the system. After receiving the user's exit operation, the main program module M50 calls the resource cleaning function, releases the system resources, terminates the process and closes the interface, and completes the safe shutdown of the system.

[0061] The connection relationship of the above modules is as follows: After the system is started, the main program module M50 first initializes the entire system environment, and sequentially loads and activates the interface interaction control module M10, the application controller module, the backend matching module M30 and the result export module M40, to provide support for the running of each functional module. During the system running period, the main program module M50 continuously maintains the event loop, is responsible for the unified control of resource management and program exit, and ensures the stable running of the system.

[0062] The user operates through the interface interaction control module M10, including uploading images, initiating matching tasks, etc. When the user uploads images, the interface module transmits the image path and related information to the application controller module. After receiving the user's request, the controller module first verifies the legality of the operation, confirms whether the image is valid, and sets the matching parameters (such as matching accuracy level) according to the task configuration. Then, the controller module packages the image path and parameters into a matching task, and sends it to the backend matching module M30.

[0063] After receiving the task, the backend matching module M30 automatically selects the corresponding feature extraction network structure according to the source type of the image (such as ground image, unmanned aerial vehicle image or satellite image), pre-processes and extracts features from the image, compares the feature vector with the multi-source image features in the image library, and finally returns the path of the most similar image and its corresponding latitude and longitude coordinate information. The matching result is returned to the application controller module.

[0064] After receiving the matching result, the controller module sends the result to two directions: on the one hand, the image path and geographic coordinates are transmitted to the interface interaction control module M10, which is used to display the matching image and the corresponding positioning information in the front-end interface; on the other hand, the result is transmitted to the result export module M40, which completes the formatting processing and log storage of the matching record.

[0065] The result export module M40 receives the result data from the controller, extracts the target label, latitude and longitude, processing time, timestamp, etc., and records them into the log file or structured data table according to the preset standard, to realize the archiving and traceable management of the result.

[0066] Through the technical means, the problems of lacking three-source joint data in the existing cross-view image matching method and the technical difficulties of insufficient matching precision and poor generalization ability caused by only focusing on single or double-view matching task can be effectively solved. The application constructs an air-space-ground multi-source image matching dataset covering three views of ground, unmanned aerial vehicle and satellite, and proposes a unified feature extraction and alignment method based on interactive enhancement convergence, realizing structure consistency modeling under multi-scale and multi-view. The method and system can not only significantly improve the accuracy and robustness of cross-platform image matching, but also provide a clear network structure, complete function and easy-to-expand solution for air-space-ground collaborative perception and high-precision spatial positioning in complex scenes, with good practical value and promotion prospect.

Claims

1. A method for matching air-space-ground data, characterized in that, Includes the following steps: S1. Construct an air-to-ground positioning dataset, which includes UAV view images, ground view images, and satellite view images; S2. Based on the constructed air-space-ground positioning dataset, a viewpoint matching method based on interactive enhancement convergence is used to perform cross-viewpoint target matching on UAV viewpoint images, ground viewpoint images, and satellite viewpoint images, thus completing the matching of air-space-ground data.

2. The air-space-ground data matching method according to claim 1, characterized in that, S2 includes the following steps: S201. Read UAV perspective images, ground perspective images, and satellite perspective images, along with their corresponding geographic coordinate information, from the constructed air-space-ground positioning dataset to obtain multi-view cross-platform target data with geographic location labels. S202. Based on multi-view cross-platform target data, affine transformation and projection transformation are used to expand the data diversity and simulate different viewpoint scenarios. S203. Input the multi-view cross-platform target data into three backbone networks with the same structure for feature extraction to obtain satellite image features, UAV image features and ground image features. Among them, satellite image features, UAV image features and ground image features are collectively referred to as target features. S204. Divide the target features into query image feature matrices according to the simulated scene results. Image library feature matrix ,in, Indicates the number of samples queried. Representing feature dimension, Indicates the number of samples in the image library; S205. Jointly optimize the loss functions of the three backbone networks and calculate the final loss function using a weighted average method. The final loss function jointly constrains the three backbone networks with the same structure to map images from different platforms to the same feature space. S206. Based on the image mapping results, the target features of the multi-view cross-platform target data are further mapped onto a Gaussian sphere to obtain the query image feature matrix. Image matrix of the image library ; S207. Extract the feature matrix of the query image. Image matrix of the image library Calculate the feature matrix of the query image based on its characteristics. Image feature matrix of the image library The original similarity matrix ; S208, Based on the original similarity matrix Feature enhancement convergence is performed between query image features and library features, as well as between library features and themselves, to obtain enhanced query image features and library image features, and then the enhanced similarity matrix is ​​calculated. ; S209, Enhance the similarity matrix Compared with the original similarity matrix The similarity matrix is ​​obtained by performing a fusion process. S2010. Using the final similarity matrix, target matching is performed on the query image to obtain the best matching target, thus completing cross-view target matching.

3. The air-space-ground data matching method according to claim 2, characterized in that, The expression for the original similarity in S207 is as follows: in, Indicates the original similarity. Indicates the first The query image and the first Similarity between images in the image library T This indicates the transpose operation.

4. The air-space-ground data matching method according to claim 2, characterized in that, S208 includes the following steps: Obtain the K-nearest neighbor image database features for each query image, based on the original similarity matrix. For each query feature, sort them in descending order of similarity, and select the top... Index of library features ; Based on the selected previous Index of library features Extract the corresponding nearest neighbor image library feature matrix. For each query image feature Calculate the attention weights between the feature and the feature of the nearest graph library. ; Based on attention weights Enhance query features by weighted aggregation of features from neighboring graph databases. ; Information-enhanced query features Obtain the enhanced query feature matrix ; Calculate the cosine similarity matrix between image features in the image library. : ； Get each query image Nearest neighbor image database features, for the cosine similarity matrix For each query feature, sort them in descending order of similarity and select the top... Index of library features ; Based on the selected previous Index of library features Extract the corresponding nearest neighbor image library feature matrix. For each library feature Calculate the attention weights with neighboring features. ; Based on attention weights Nearest neighbor feature information is aggregated by weighting. The enhanced image library feature matrix is ​​obtained. ; Based on the enhanced query feature matrix and image library feature matrix Calculate the enhanced similarity matrix .

5. The air-space-ground data matching method according to claim 4, characterized in that, The information enhancement query feature The expression is as follows: ； ； ； ； in, This represents the feature matrix of the library that is most similar to the query feature. k Nearest neighbor features, k This indicates the order of the first K feature sequences. K Indicates the range of nearest neighbor features. Represents the variable to be summed. T This indicates the transpose operation. Indicates the first k The original indices of the nearest neighbor features in the feature matrix of the image library. This represents the variable to be summed.

6. The air-space-ground data matching method according to claim 4, characterized in that, The nearest neighbor feature information The expression is as follows: ； ； ； ； in, k Indicates the preceding The sorting of the feature sequences Indicates the range of nearest neighbor features. Represents the variable to be summed. The first element in the feature matrix that is most similar to a feature in the library is the [feature name]. k Nearest neighbor features, Represents the variable to be summed. The first element in the feature matrix that is most similar to a feature in the library is the [feature name]. Nearest neighbor features, Indicates the first The original indices of the nearest neighbor features in the feature matrix of the image library. Indicates the first k The original indices of the nearest neighbor features in the feature matrix of the library.

7. The air-space-ground data matching method according to claim 4, characterized in that, The enhanced similarity matrix The expression is as follows: 。 8. The air-space-ground data matching method according to claim 4, characterized in that, The expression for the final similarity matrix is ​​as follows: ； in, This represents the final similarity matrix. Indicates the fusion weight. Represents the enhanced similarity matrix. Indicates the original similarity.

9. A space-air-ground data matching system, used to execute the space-air-ground data matching method according to any one of claims 1-8, characterized in that, include: The interface interaction control module is used to build a graphical user interface and receive user operation commands; The application controller module is used to generate air-space-ground data matching tasks according to the user's matching instructions, and to pass the image path and parameters to the backend matching module. The backend matching module is used to automatically select the corresponding feature extraction backbone network based on the image path, parameters and image viewpoint type, and perform image feature extraction and cross-viewpoint matching operations based on the air-space-ground data matching task. The results export module is used to record and output the matching results; The main program module is used to start and manage the air-space-ground data matching system. In the initialization phase of the air-space-ground data matching system, the graphical user interface environment is built and initialized.

Images