A lightweight cross-view positioning method and system based on quantized knowledge distillation and semantic enhancement matching

Abstract

The application discloses a kind of light cross-view positioning method and system based on quantification knowledge distillation and semantic enhancement matching, comprising: unmanned aerial vehicle aerial image is used as query image, similarity matching is carried out between query image and pre-constructed geographic labeled satellite image database, and the highest similarity matching satellite image is obtained;Respectively extract the texture feature and semantic feature of query image and matching satellite image, and respectively generate corresponding semantic perception descriptor;Based on this, the texture feature similarity matrix and semantic feature similarity matrix between query image and matching satellite image are calculated respectively;Two similarity matrices are multiplied at element level, and the final similarity matrix is obtained, so that the pixel-level key point matching pair between query image and matching satellite image is determined;According to pixel-level key point matching pair and the geographic coordinates corresponding to matching satellite image, the geographic position coordinates of selected point in query image are obtained.The application realizes model light-weight deployment while ensuring high positioning accuracy.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, computer vision, and intelligent positioning technology, and more specifically to a lightweight cross-view positioning method and system based on quantized knowledge distillation and semantic enhancement matching. Background Technology

[0002] Cross-view visual positioning is a key component of UAV autonomous navigation systems. Its technical process is usually divided into two levels: the first level is the coarse positioning stage, which uses visual position recognition technology to compare the aerial images collected by the UAV in real time with a pre-built geographic labeled satellite image database to filter out candidate areas with similar spatial locations; the second level is the fine positioning stage, which establishes sub-pixel-level feature correspondence between aerial images and candidate satellite images and calculates the precise geographic coordinates of the UAV by combining geometric constraints.

[0003] Cross-view visual localization technology is a current research hotspot. Traditional visual location recognition methods mainly rely on global descriptor retrieval and local feature matching. Methods based on the bag-of-words model and Fisher vectors achieve scene encoding by aggregating hand-designed local features. Subsequently, the introduction of convolutional neural networks has promoted the development of end-to-end feature learning. Typical examples include NetVLAD and its improved version Patch-NetVLAD, which map convolutional features to fixed-dimensional global descriptors through differentiable vector aggregation layers. Subsequently, methods based on the Transformer architecture have further enhanced feature representation capabilities. Among them, the DINOv2 network (Self-Distillation with No Labels version 2) obtains powerful visual feature representations through self-supervised learning, and the BoQ network (Bag of Learnable Queries) proposes an aggregation strategy based on learnable queries. In recent years, TeTRA-VPR has innovatively combined ternary quantization technology with the Transformer architecture. By constraining network weights to three discrete values ​​{-1, 0, 1}, it replaces matrix multiplication with sparse addition, significantly reducing model memory consumption and inference latency while maintaining high retrieval accuracy, laying the foundation for deploying visual location recognition systems on resource-constrained platforms. In the field of local feature matching, research has evolved from manually designed descriptors to data-driven learning. Traditional methods rely on scale-space extremum detection and gradient orientation histograms to construct geometrically invariant feature representations, while deep learning-based methods have achieved performance breakthroughs by jointly optimizing keypoint detection and descriptor extraction. To address the matching challenges in complex scenes, introducing high-level semantic information to assist low-level texture matching has become an important research direction. However, existing fusion strategies mostly remain at the level of simple feature concatenation and have not yet established an effective cross-modal interaction mechanism.

[0004] However, existing technologies still have the following shortcomings in cross-view positioning scenarios: First, the inherent view gap between aerial oblique views and satellite orthophotos causes drastic changes in scene appearance, and existing models are mostly optimized based on ground-level horizontal views, resulting in severely insufficient generalization ability in cross-view scenarios. Second, highly similar regularized structures are common in artificial environments, and without an effective semantic differentiation mechanism, false matches are easily introduced. Existing methods typically fuse texture and semantic information through simple stitching, failing to adaptively adjust the fusion ratio based on local features, thus failing to fully leverage complementary advantages. Third, high-precision positioning schemes often rely on network structures with large parameters and high computational complexity, as well as high-dimensional feature representations, leading to high memory consumption and large inference latency, making them difficult to deploy on small UAV platforms with limited computing resources. Finally, when directly transferring models trained on ground-view to cross-view tasks, extreme scale changes and domain shifts often lead to training instability and a significant decrease in accuracy. Full parameter fine-tuning is not only costly to train but also prone to overfitting, necessitating an efficient and lightweight parameter adaptation mechanism to achieve robust transfer.

[0005] Therefore, how to achieve lightweight model deployment while ensuring high positioning accuracy, and provide reliable technical support for intelligent navigation in GPS-denied environments, is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] In view of the above problems, the present invention proposes a lightweight cross-view localization method and system based on quantized knowledge distillation and semantic enhancement matching to overcome or at least partially solve the above problems.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] In a first aspect, the present invention provides a lightweight cross-view localization method based on quantized knowledge distillation and semantic enhancement matching, comprising the following steps: S1. Obtain aerial images of the target UAV and use them as query images. Utilize a cross-domain progressive transfer learning strategy based on quantized knowledge distillation to perform similarity matching between the query images and a pre-built geographic labeled satellite image database. Select the satellite image with the highest similarity as the matching satellite image for the query images. S2. Extract the texture features and semantic features of the query image and the matching satellite image respectively; adaptively fuse the texture features and semantic features of the query image and the matching satellite image respectively to generate semantic-aware descriptors of the query image and the matching satellite image respectively; S3. Based on the semantic-aware descriptors of the query image and the matching satellite image, calculate the texture feature similarity matrix and the semantic feature similarity matrix between the query image and the matching satellite image, respectively; multiply the texture feature similarity matrix and the semantic feature similarity matrix element by element to obtain the final similarity matrix; S4. Based on the final similarity matrix, determine the pixel-level key point matching pairs between the query image and the matching satellite image; according to the pixel-level key point matching pairs and the geographic coordinates corresponding to the matching satellite image, obtain the geographic location coordinates of the selected point in the query image.

[0009] Furthermore, in S1, the cross-domain progressive transfer learning strategy based on quantized knowledge distillation uses a ternary quantized Transformer network as the backbone network, and during the training process, the weights of the backbone network are constrained to three discrete values: {-1, 0, 1}.

[0010] Furthermore, when training the backbone network, the weight parameters of the backbone network are frozen, and only the lightweight drone adapter and feature aggregation layer connected to the backbone network are trained.

[0011] Furthermore, during the training of the backbone network, the input image pairs of the backbone network are processed based on data augmentation strategies; the data augmentation strategies include at least one of random affine transformation, perspective transformation, color jitter, and random erasure.

[0012] Furthermore, the total loss function of the backbone network during training includes a joint optimization knowledge distillation loss function and a triplet metric learning loss function.

[0013] Furthermore, the lightweight drone adapter employs a network module that includes a bottleneck structure and residual connections.

[0014] Further, in S2, the texture features and semantic features of the query image and the matching satellite image are respectively subjected to the following operations to generate their respective semantically aware descriptors: The texture features and the semantic features are respectively mapped to the same feature dimension space through a linear projection layer; In the feature dimension space, texture features are used as the first query vector, a weighted mixture of semantic features and texture features is used as the first key vector, and semantic features are used as the first value vector; multi-layer cross-attention calculation is performed on the first query vector, the first key vector, and the first value vector to output a refined texture descriptor; In the feature dimension space, semantic features are used as the second query vector, and texture features are used as the second key vector and the second value vector. Multi-layer cross-attention calculation is performed on the second query vector, the second key vector and the second value vector to output a refined semantic descriptor. The refined texture descriptor and the refined semantic descriptor constitute the semantic-aware descriptor.

[0015] Furthermore, in the process of generating the refined texture descriptor, an adaptive gating coefficient is introduced; the adaptive gating coefficient is obtained in the following way: The texture features of the current layer are input into the gated projection layer, and the gated scalar corresponding to each key point is generated by the Sigmoid activation function; the gated scalar is used as the adaptive gate coefficient.

[0016] Secondly, this invention provides a lightweight cross-view localization system based on quantized knowledge distillation and semantic enhancement matching, applying the aforementioned method; the system includes: The image acquisition module is used to acquire aerial images of the target drone and use them as query images; The cross-domain retrieval module is used to perform similarity matching between the query image and a pre-built geographic labeled satellite image database using a cross-domain progressive transfer learning strategy based on quantized knowledge distillation, and to use the satellite image with the highest similarity as the matching satellite image of the query image. The feature extraction and fusion module is used to extract the texture features and semantic features of the query image and the matching satellite image respectively; and adaptively fuse the texture features and semantic features of the query image and the matching satellite image respectively to generate semantic-aware descriptors of the query image and the matching satellite image respectively. The similarity calculation module is used to calculate the texture feature similarity matrix and the semantic feature similarity matrix between the query image and the matching satellite image based on the semantic-aware descriptors of the query image and the matching satellite image, respectively; and to perform element-wise multiplication of the texture feature similarity matrix and the semantic feature similarity matrix to obtain the final similarity matrix. The positioning and solving module is used to determine pixel-level key point matching pairs between the query image and the matching satellite image based on the final similarity matrix; and to obtain the geographical coordinates of selected points in the query image based on the pixel-level key point matching pairs and the geographical coordinates corresponding to the matching satellite image.

[0017] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a lightweight cross-view localization method and system based on quantized knowledge distillation and semantic enhancement matching, which has the following beneficial effects: This invention utilizes a cross-domain progressive transfer learning strategy based on quantized knowledge distillation, which effectively bridges the significant differences between UAV and satellite perspectives. This ensures that the correct candidate regions can be retrieved quickly and accurately from massive satellite images even in complex environments, thus solving the fundamental problem of unreliable cross-perspective retrieval.

[0018] This invention adaptively fuses low-level texture features with high-level semantic features of an image to generate a more discriminative semantically perceptual descriptor, enhancing its ability to distinguish repetitive structures and similar appearances. Furthermore, by using a semantic similarity matrix to impose element-level constraints on texture matching results, a large number of erroneous matching points that do not conform to semantic consistency can be intelligently filtered out, fundamentally reducing the false matching rate.

[0019] This invention outputs pixel-level keypoint matching pairs and their precise geographic coordinates, achieving high-precision positioning and providing reliable technical support for UAVs to complete practical tasks such as autonomous navigation and target calibration in GPS-denied environments. This method represents a significant advancement in accuracy, robustness, and practicality. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the lightweight cross-view localization method based on quantized knowledge distillation and semantic enhancement matching provided in an embodiment of the present invention.

[0022] Figure 2 This is a schematic diagram of the framework of a lightweight cross-view localization method based on quantized knowledge distillation and semantic enhancement matching provided in an embodiment of the present invention.

[0023] Figure 3 This is a schematic diagram of cross-domain transfer learning provided in an embodiment of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] This invention discloses a lightweight cross-view localization method based on quantized knowledge distillation and semantic enhancement matching, such as... Figure 1 As shown, it includes the following steps: S1. Obtain aerial images of the target UAV and use them as query images. Utilize a cross-domain progressive transfer learning strategy based on quantized knowledge distillation to perform similarity matching between the query images and a pre-built geographic labeled satellite image database. Select the satellite image with the highest similarity as the matching satellite image for the query image. S2. Extract the texture features and semantic features of the query image and the matching satellite image respectively; adaptively fuse the texture features and semantic features of the query image and the matching satellite image respectively to generate semantic-aware descriptors of the query image and the matching satellite image respectively. S3. Based on the semantic-aware descriptors of the query image and the matching satellite image, calculate the texture feature similarity matrix and the semantic feature similarity matrix between the query image and the matching satellite image respectively; multiply the texture feature similarity matrix and the semantic feature similarity matrix element by element to obtain the final similarity matrix; S4. Based on the final similarity matrix, determine the pixel-level key point matching pairs between the query image and the matching satellite image; based on the pixel-level key point matching pairs and the geographic coordinates corresponding to the matching satellite image, obtain the geographic coordinates of the selected points in the query image.

[0026] This invention proposes a lightweight cross-view positioning method based on quantized knowledge distillation and semantic enhancement matching, which can be widely applied to various scenarios in GPS-denied or interfered environments. In emergency rescue scenarios, it can provide accurate spatial reference information for fire reconnaissance drones and search and rescue drones, supporting rapid post-disaster response and personnel search. In infrastructure inspection scenarios, it can automate the recording of inspection trajectories and the identification of abnormal locations for facilities such as power lines, oil pipelines, bridges, and tunnels. In surveying and mapping scenarios, it can provide stable and reliable positioning support for geological exploration, agricultural and forestry monitoring, and other tasks. In operational environments, this method can adapt to the positioning needs of drones in various complex environments such as urban low-altitude areas, mountain valleys, and offshore platforms. Ultimately, this invention achieves pixel-level accurate visual positioning, outputting the satellite image geographic coordinates corresponding to key points in drone aerial images, providing high-precision latitude and longitude location information for the aforementioned application scenarios.

[0027] like Figure 2 As shown, the lightweight cross-perspective localization method based on quantized knowledge distillation and semantic enhancement matching proposed in this invention includes two core stages: a global retrieval process and a semantic enhancement fine matching framework. The global retrieval process is embodied in step S1, and the semantic enhancement fine matching framework is embodied in steps S2-S4. The following sections will provide a detailed explanation of each step.

[0028] In step S1 above, to address the problem that quantization models are difficult to train directly end-to-end and pre-trained knowledge is difficult to transfer to cross-view scenarios, this embodiment of the invention designs a cross-domain progressive transfer learning strategy. Specifically, the target UAV aerial image (i.e., UAV-captured image) is acquired and used as the query image. Using the cross-domain progressive transfer learning strategy based on quantization knowledge distillation, the query image is matched with a pre-built geographic labeled satellite image database for similarity. The global search results (labeled as 1, 2, 3...) are output in order of similarity, and the satellite image with the highest similarity is used as the matching satellite image of the query image. like Figure 3 As shown, the cross-domain progressive transfer learning strategy based on quantized knowledge distillation uses a ternary quantized Transformer network as the backbone network, and trains the backbone network in the following manner: (1) Data augmentation: The input UAV aerial images and satellite images are processed based on data augmentation strategies. The data augmentation strategies include at least one of random affine transformation, perspective transformation, color jitter and random erasure. Among them, random affine transformation and perspective transformation are used to simulate different flight altitudes and camera attitudes; color jitter is used to cope with changes in lighting; random erasure is used to simulate occlusion.

[0029] (2) Training weights: When training the backbone network, the weight parameters of the backbone network are frozen, and only the lightweight UAV adapter and feature aggregation layer connected to the backbone network are trained. For example Figure 3 The blue solid boxes represent the frozen part, which retains its pre-trained weights and does not participate in training; the orange dashed boxes represent the unfrozen part, whose parameters are optimized and updated during training.

[0030] The aforementioned lightweight UAV adapter is specifically a UAV adapter head, comprising a network module with a bottleneck structure and residual connections. It achieves domain adaptation from ground-based visual location recognition to cross-view UAV scenarios through a progressive influence coefficient and residual connections, introducing only a small number of trainable parameters. The progressive influence coefficient is a learnable scalar parameter λ, initially set to a small value to minimize the impact of the adapter output on the original global features of the frozen backbone network output. It gradually increases as training progresses, allowing the model to smoothly transition from ground-domain feature representation to cross-view UAV domain feature representation, avoiding catastrophic forgetting. The residual connections add the UAV adapter output to the original global features, resulting in a summed feature f. adapted Represented as: f adapted =f original +λ·Adapter(f original ) Among them, f originalTo freeze the original global features of the backbone network output, Adapter(·) is the transformed output of the lightweight bottleneck network, and λ is the asymptotic influence coefficient.

[0031] The aforementioned feature aggregation layer is used to aggregate multiple tokens output by the backbone network into a compact global descriptor for image similarity retrieval.

[0032] (3) Quantization scheduling: During the training process, the gradual degree of quantization of the backbone network weights is controlled. The weights of the backbone network are constrained to three discrete values ​​{-1, 0, 1} using a gradual strategy. Sparse addition replaces matrix multiplication, which significantly reduces the memory usage of the model and greatly improves the inference speed. It achieves the unity of lightweight and high precision and is suitable for real-time deployment on resource-constrained UAV platforms.

[0033] (4) Loss function: The total loss function of the backbone network during training includes the joint optimization knowledge distillation loss function and the triplet metric learning loss function; The distillation loss function described above is expressed as:

[0034] The knowledge distillation loss function is used to ensure that the output features of the student network (trainable part) are consistent with those of the teacher network (frozen part); where, Indicates the size of the training batch; Indicates the first The network output feature vector for each sample of students; Indicates the first The teacher network output feature vector for each sample.

[0035] The triplet metric learning loss function described above is expressed as follows:

[0036] The triplet metric learning loss function is used to bring image descriptors from the same geographic location closer together and descriptors from different locations further apart; where, Indicates the total number of training samples; A global descriptor representing the anchor sample; A global descriptor representing a positive sample; A global descriptor representing negative samples. Represents the Euclidean distance function; The margin hyperparameter represents the triplet loss; Positive value operation; This invention employs a cross-domain transfer learning strategy that freezes the pre-trained backbone network and trains only a lightweight UAV adapter head and feature aggregation layer. It combines knowledge distillation loss and triplet loss for joint optimization, achieving efficient domain adaptation from ground visual location recognition to UAV-satellite cross-view scenarios with very few trainable parameters. The retrieval accuracy is significantly improved compared to existing methods.

[0037] In step S2 above, as Figure 2 As shown, both the query image and the matching satellite image are input into the dual-branch feature extraction module (DBFE). The texture features of the query image and the matching satellite image are extracted by the XFeat network (AcceleratedFeatures for Lightweight Image Matching). The semantic features of the query image and the matching satellite image are extracted by the DINOv2 network (Self-Distillation with No Labels version 2). Subsequently, addressing the problem that existing methods simply splice texture and semantic features and cannot adaptively adjust the fusion ratio, this invention designs a feature fusion mechanism based on cross-attention. Specifically, the texture and semantic features of the query image and the matching satellite image are adaptively fused to generate semantically aware descriptors for the query image and the matching satellite image, respectively; specifically: (1) Adaptively fuse the texture features and semantic features of the query image to generate a semantically aware descriptor for the query image: like Figure 2 As shown, the texture and semantic features of the query image are input into the Cross-Attention Feature Fusion (CAFF) module, which contains three processing stages: projection embedding, texture thinning branch, and semantic thinning branch. 1) Projective embedding: The texture features of the query image extracted via the XFeat network and the semantic features of the query image extracted via the DINOv2 network are respectively mapped to the same feature dimension space through a learnable linear projection layer. K × D );in K For the number of key points, D For feature dimensions.

[0038] In the same feature dimension space, subsequent texture refinement branch and semantic refinement branch operations are performed.

[0039] 2) Texture Refinement Branch: The texture refinement branch is achieved through... L The iterative refinement process of the layer-crossing attention network integrates semantic information with texture features. Each layer's processing includes: ① Initialization: Use the texture features of the query image as input to the current layer; ② Adaptive Gating: Content-aware adaptive gating coefficients are calculated separately. These coefficients are obtained as follows: The texture features of the current layer (i.e., the initialized input of the current layer) are input to the gating projection layer, and a gating scalar corresponding to each keypoint is generated through a Sigmoid activation function; the gating scalar is then used as the adaptive gating coefficient; expressed as:

[0040] in, Indicates the first Adaptive gating coefficients for layered cross-attention networks; Indicates the first Texture features of layered cross-attention networks; Indicates the first Learnable gated projection matrices within layers; This represents the sigmoid function; the adaptive gating coefficient. Determine the degree to which each key point depends on semantic and texture information.

[0041] ③ Cross-attention mechanism: Three elements for constructing an attention mechanism: First query vector Q: The texture features of the query image are used as the first query vector, represented as follows: ,in, Indicates the first The query matrix of a layered cross-attention network; Indicates the first The query projection weight matrix of the layered cross-attention network; The first key vector K: The semantic and texture features of the query image are weighted and mixed, and the weighted mixture result is used as the first key vector, represented as follows: ,in, Indicates the first The key matrix of a multilayer cross-attention network; Indicates the first Semantic features of layered cross-attention networks; First value vector V: The semantic features of the query image are used as the first value vector, represented as follows: ,in, Indicates the first The value matrix of the layered cross-attention network; After performing cross-attention calculation on the first query vector, first key vector, and first value vector at the current layer, a residual connection is made with the cross-attention calculation result of the previous layer to output the updated texture feature representation of the current layer:

[0042] in, Indicates the first Texture features of multi-layer cross-attention networks; after multiple layers ( L After iterative updates via cross-attention in the Lth layer (the last layer), a refined texture descriptor is output. Specifically, this refined texture descriptor is the texture feature output after iterative updates via the Lth layer (the last layer) of the cross-attention network, i.e., the refined texture descriptor. ; In this process, through adaptive gating mechanism and multi-layer cross-attention iteration, deep fusion of texture features and semantic features is achieved, enabling each key point to autonomously determine the degree of dependence on semantic and texture information based on the quality of local features.

[0043] 3) Semantic Refinement Branch: The semantic refinement branch adopts a symmetrical structure, fusing texture information into semantic features through an L-layer cross-attention iterative refinement process. Similar to the texture refinement branch mentioned above, the three elements constructing this cross-attention mechanism are: Second query vector: The semantic features are used as the second query vector, represented as follows: ;in, This represents the second query matrix of the i-th layer cross-attention network; This represents the second query projection weight matrix of the i-th layer cross-attention network; This represents the semantic features of the (i-1)th layer of the cross-attention network; Second key vector and second value vector: Both use texture features as the second key vector and second value vector, represented as follows: ;in, This represents the texture features of the (i-1)th layer of the cross-attention network. It should be noted that, unlike the texture thinning branch, the semantic thinning branch does not introduce adaptive gating coefficients when generating the second key vector and the second value vector; instead, it directly uses the texture features as the key and value vectors. After performing cross-attention calculation on the second query vector, second key vector, and second value vector at the current layer, a residual connection is made with the cross-attention calculation result of the previous layer to output the updated semantic features of the current layer, represented as:

[0044] in, Let represent the semantic features of the i-th layer of the cross-attention network; after multiple (L layers) of cross-attention iterations, the semantic features output by the L-th layer (i.e., the last layer) are determined as the refined semantic descriptor, i.e., the refined semantic descriptor. .

[0045] The refined texture descriptor and refined semantic descriptor mentioned above constitute the semantic-aware descriptor of the query image, and are respectively represented as the refined texture descriptor. Refine semantic descriptors .

[0046] The texture refinement branch and semantic refinement branch described above have a symmetrical structure. In the texture refinement branch, texture features are the main component (i.e., the first query vector comes from texture features), and semantic information is queried and fused using key vectors and value vectors constructed from semantic features, thereby refining the texture features. In the semantic refinement branch, semantic features are the main component (i.e., the second query vector comes from semantic features), and texture information is queried and fused using key vectors and value vectors constructed from texture features, thereby refining the semantic features. The texture refinement branch and the semantic refinement branch reinforce each other. The only difference between them is that the texture refinement branch introduces an adaptive gating coefficient to weight and mix semantic and texture information when constructing the key vector, while the semantic refinement branch does not introduce gating and directly uses texture features as key and value vectors.

[0047] (2) Adaptively fuse the texture and semantic features of the matching satellite image to generate a semantically aware descriptor for the matching satellite image; this process can refer to the above process for generating a semantically aware descriptor for the query image. This includes: 1) The texture features and semantic features of the matching satellite images are mapped to the same feature dimension space through a linear projection layer; 2) In the feature dimension space, texture features are used as the first query vector, a weighted mixture of semantic features and texture features is used as the first key vector, and semantic features are used as the first value vector; multi-layer cross-attention calculation is performed on the first query vector, the first key vector, and the first value vector to output a refined texture descriptor; 3) In the feature dimension space, semantic features are used as the second query vector, and texture features are used as the second key vector and the second value vector. Multi-level cross-attention calculation is performed on the second query vector, the second key vector and the second value vector to output a refined semantic descriptor. The refined texture descriptor and refined semantic descriptor are used to construct the semantic-aware descriptor for matching satellite images. The calculation method for the refined texture descriptor and refined semantic descriptor corresponding to the matching satellite image is the same as that for the query image branch, i.e., they are the texture features output from the last layer after multiple layers of cross-attention iteration. and semantic features .

[0048] In step S3 above, based on the semantic-aware descriptors of the query image and the matching satellite image, the texture feature similarity matrix between the query image and the matching satellite image is calculated respectively. and semantic feature similarity matrix Specifically, let the refined texture descriptor of the query image be... Query the refined semantic descriptor of the image The refined texture descriptor matching the satellite image is The refined semantic descriptor for matching satellite images is Then the texture feature similarity matrix and semantic feature similarity matrix They are defined as follows:

[0049]

[0050] Among them, the texture feature similarity matrix The (p,q)th element in the semantic feature similarity matrix represents the texture similarity between the p-th keypoint in the query image and the q-th keypoint in the satellite image; The (p,q)th element in the matrix represents the semantic similarity between corresponding keypoint pairs; both matrices have been normalized, and their values ​​range from [...]. 1,1].

[0051] Next, the texture feature similarity matrix Similarity matrix with semantic features Element-wise multiplication yields the final similarity matrix. The method employs element-wise multiplication to ensure that matching candidates are retained only when both texture similarity and semantic similarity are high, thereby effectively filtering out erroneous matches across semantic categories.

[0052] In step S4 above, based on the final similarity matrix, mutual nearest neighbor (MNN) matching is used to determine pixel-level keypoint matching pairs between the query image and the matching satellite image. Based on the pixel-level keypoint matching pairs and the corresponding geographic coordinates of the matching satellite image, the geographic coordinates of the selected points in the query image are obtained. Mutual nearest neighbor (MNN) matching ensures that only the nearest neighbor matching point pairs are retained, further improving the reliability of the matching and forming a double constraint with the aforementioned "element-level multiplication strategy."

[0053] In summary, the lightweight cross-view localization method based on quantized knowledge distillation and semantic enhancement matching disclosed in this invention has a modular two-stage framework design. The global retrieval module and the fine matching module can be independently optimized and upgraded. It exhibits excellent generalization ability under various terrain environments and lighting conditions, laying the foundation for practical deployment and application on embedded UAV platforms.

[0054] Based on the same inventive concept, embodiments of the present invention also provide a lightweight cross-view localization system based on quantized knowledge distillation and semantic enhancement matching, the system comprising: The image acquisition module is used to acquire aerial images of the target drone and use them as query images; The cross-domain retrieval module utilizes a cross-domain progressive transfer learning strategy based on quantized knowledge distillation to perform similarity matching between the query image and a pre-built geographic labeled satellite image database, and selects the satellite image with the highest similarity as the matching satellite image for the query image. The feature extraction and fusion module is used to extract the texture features and semantic features of the query image and the matching satellite image respectively; and adaptively fuse the texture features and semantic features of the query image and the matching satellite image to generate semantic-aware descriptors of the query image and the matching satellite image respectively. The similarity calculation module is used to calculate the texture feature similarity matrix and semantic feature similarity matrix between the query image and the matching satellite image based on the semantically perceived descriptors of the query image and the matching satellite image, respectively; and then multiply the texture feature similarity matrix and the semantic feature similarity matrix element by element to obtain the final similarity matrix. The localization and resolution module is used to determine pixel-level keypoint matching pairs between the query image and the matching satellite image based on the final similarity matrix; and to obtain the geographical coordinates of the selected point in the query image based on the pixel-level keypoint matching pairs and the corresponding geographical coordinates of the matching satellite image.

[0055] Since the principle behind the problem solved by this system is similar to the aforementioned lightweight cross-view localization method based on quantized knowledge distillation and semantic enhancement matching, the implementation of this system can be found in the implementation of the aforementioned method, and the repetitions will not be repeated.

[0056] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0057] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A lightweight cross-view localization method based on quantized knowledge distillation and semantic enhancement matching, characterized in that, Includes the following steps: S1. Obtain aerial images of the target UAV and use them as query images. Utilize a cross-domain progressive transfer learning strategy based on quantized knowledge distillation to perform similarity matching between the query images and a pre-built geographic labeled satellite image database. Select the satellite image with the highest similarity as the matching satellite image for the query images. S2. Extract the texture features and semantic features of the query image and the matching satellite image respectively; The texture features and semantic features of the query image and the matching satellite image are adaptively fused to generate semantic-aware descriptors for the query image and the matching satellite image, respectively. S3. Based on the semantic-aware descriptors of the query image and the matching satellite image, calculate the texture feature similarity matrix and the semantic feature similarity matrix between the query image and the matching satellite image, respectively; The texture feature similarity matrix and the semantic feature similarity matrix are multiplied element-wise to obtain the final similarity matrix; S4. Based on the final similarity matrix, determine the pixel-level key point matching pairs between the query image and the matching satellite image; Based on the pixel-level keypoint matching pairs and the geographic coordinates corresponding to the matching satellite image, the geographic coordinates of the selected point in the query image are obtained.

2. The lightweight cross-view localization method based on quantized knowledge distillation and semantic enhancement matching as described in claim 1, characterized in that, In S1, the cross-domain progressive transfer learning strategy based on quantized knowledge distillation uses a ternary quantized Transformer network as the backbone network, and during training, the weights of the backbone network are constrained to three discrete values: {-1, 0, 1}.

3. The lightweight cross-view localization method based on quantized knowledge distillation and semantic enhancement matching as described in claim 2, characterized in that, When training the backbone network, the weight parameters of the backbone network are frozen, and only the lightweight drone adapter and feature aggregation layer connected to the backbone network are trained.

4. The lightweight cross-view localization method based on quantized knowledge distillation and semantic enhancement matching as described in claim 2, characterized in that, During the training of the backbone network, the input image pairs of the backbone network are processed based on data augmentation strategies; the data augmentation strategies include at least one of random affine transformation, perspective transformation, color jitter, and random erasure.

5. The lightweight cross-view localization method based on quantized knowledge distillation and semantic enhancement matching as described in claim 2, characterized in that, The total loss function of the backbone network during training includes the joint optimization knowledge distillation loss function and the triplet metric learning loss function.

6. The lightweight cross-view localization method based on quantized knowledge distillation and semantic enhancement matching as described in claim 3, characterized in that, The lightweight drone adapter employs a network module that includes a bottleneck structure and residual connections.

7. The lightweight cross-view localization method based on quantized knowledge distillation and semantic enhancement matching as described in claim 1, characterized in that, In step S2, the following operations are performed on the texture features and semantic features of the query image and the matching satellite image, respectively, to generate semantic-aware descriptors for the query image and the matching satellite image: The texture features and the semantic features are respectively mapped to the same feature dimension space through a linear projection layer; In the feature dimension space, texture features are used as the first query vector, a weighted mixture of semantic features and texture features is used as the first key vector, and semantic features are used as the first value vector; multi-layer cross-attention calculation is performed on the first query vector, the first key vector, and the first value vector to output a refined texture descriptor; In the feature dimension space, semantic features are used as the second query vector, and texture features are used as the second key vector and the second value vector. Multi-layer cross-attention calculation is performed on the second query vector, the second key vector and the second value vector to output a refined semantic descriptor. The refined texture descriptor and the refined semantic descriptor constitute the semantic-aware descriptor.

8. The lightweight cross-view localization method based on quantized knowledge distillation and semantic enhancement matching as described in claim 7, characterized in that, In the process of generating the refined texture descriptor, an adaptive gating coefficient is introduced; the adaptive gating coefficient is obtained in the following way: The texture features of the current layer are input into the gated projection layer, and the gated scalar corresponding to each key point is generated by the Sigmoid activation function; the gated scalar is used as the adaptive gate coefficient.

9. A lightweight cross-view localization system based on quantized knowledge distillation and semantic enhancement matching, characterized in that, The system comprises: applying the method of any one of claims 1-8; The image acquisition module is used to acquire aerial images of the target drone and use them as query images; The cross-domain retrieval module is used to perform similarity matching between the query image and a pre-built geographic labeled satellite image database using a cross-domain progressive transfer learning strategy based on quantized knowledge distillation, and to use the satellite image with the highest similarity as the matching satellite image of the query image. The feature extraction and fusion module is used to extract the texture features and semantic features of the query image and the matching satellite image respectively; and adaptively fuse the texture features and semantic features of the query image and the matching satellite image respectively to generate semantic-aware descriptors of the query image and the matching satellite image respectively. The similarity calculation module is used to calculate the texture feature similarity matrix and the semantic feature similarity matrix between the query image and the matching satellite image based on the semantic-aware descriptors of the query image and the matching satellite image, respectively; and to perform element-wise multiplication of the texture feature similarity matrix and the semantic feature similarity matrix to obtain the final similarity matrix. The positioning and solving module is used to determine pixel-level key point matching pairs between the query image and the matching satellite image based on the final similarity matrix; and to obtain the geographical coordinates of selected points in the query image based on the pixel-level key point matching pairs and the geographical coordinates corresponding to the matching satellite image.

Images