Query-based domain-independent visual place recognition method and device
By constructing a domain-independent visual location recognition model, using style transfer technology to generate domain-labeled data, and combining two-layer adversarial learning and triplet supervision strategies, the robustness problem of visual location recognition in cross-domain changes is solved, and high-precision visual location recognition is achieved.
Patent Information
- Application Number
- CN202610767269.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-29
- Publication Date
- 2026-07-21
AI Technical Summary
Existing technologies struggle to achieve robust visual location recognition when faced with changes in unknown domains, and existing methods have limited cross-domain generalization capabilities.
A domain-independent visual location recognition model is constructed. Style transfer technology is used to generate domain-labeled data. Domain-invariant features are learned through a two-layer adversarial learning framework and a triplet supervision strategy, including a backbone network, BoQ blocks and linear projection layers. The model is optimized by combining adversarial loss and triplet loss based on query features and image features.
It achieves state-of-the-art localization accuracy on multiple cross-domain visual location recognition benchmark datasets, possesses excellent domain generalization ability, and improves the robustness and accuracy of visual location recognition.
Smart Images
Figure CN122435290A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a query-based domain-independent visual location recognition method and apparatus. Background Technology
[0002] Visual location recognition is a key technology in autonomous mobile robots and self-driving systems. It determines the current location by matching the current visual observation with a database of geotagged images. Visual location recognition faces three main challenges: perspective change, perceptual confusion, and domain change. Among these, domain change (e.g., seasonal changes, weather changes, day-night cycles) can cause the same location to appear visually different at different times, posing a significant challenge to achieving robust visual location recognition.
[0003] Existing methods for achieving domain-independent visual location recognition can be broadly categorized into two types. The first type trains models on large-scale, diverse datasets, implicitly expecting the models to learn features robust to domain changes. However, these methods lack explicit domain supervision, making it difficult to guarantee that the models truly learn domain-independent invariant features. The second type employs domain adaptation techniques, adjusting the model for specific target domains (such as nighttime or winter) or using generative models for data augmentation. These methods typically require access to data from the target domain or additional generative models, have limited generalization ability, and experience significant performance degradation when faced with unknown or unseen domain changes.
[0004] Therefore, a technical solution is urgently needed to solve the above problems. Summary of the Invention
[0005] Therefore, it is necessary to provide a query-based domain-independent visual location recognition method and apparatus to address the aforementioned technical problems.
[0006] A query-based domain-independent visual location identification method, the method comprising the following steps:
[0007] A domain-independent visual location recognition model is constructed, which consists of a backbone network, multiple cascaded BoQ blocks, and a linear projection layer; each BoQ block contains an encoder and a set of learnable query vectors. Based on a large-scale visual location recognition dataset, an augmented training set is constructed using style transfer techniques. The augmented training set contains images with domain labels and original images without domain labels. Given a batch of augmented training set images, the domain-independent visual location recognition model generates query features, image features, and global descriptors based on the currently input images. For images with domain labels, query features and image features are passed through a gradient inversion layer and then input into a shared domain discriminator, enabling the domain-independent visual location recognition model to learn domain-independent features and calculate the adversarial loss at the query feature level and the adversarial loss at the image feature level. Query combinations are obtained based on query features, and triple indexes corresponding to global descriptors are obtained; reliable query combinations are filtered and the most difficult negative samples and the most easily distinguishable positive sample combinations are mined, and triple loss is calculated. The total loss is constructed by combining the multi-similarity loss of the global descriptor, the triplet loss, the adversarial loss at the query feature level, and the adversarial loss at the image feature level. The total loss is optimized until convergence, resulting in the trained domain-independent visual location recognition model. Visual location recognition is performed using a trained domain-independent visual location recognition model.
[0008] In one embodiment, an augmented training set is constructed using style transfer techniques based on a large-scale visual location recognition dataset, including: Style transfer techniques are used to augment large-scale visual location recognition datasets, generating various synthetic domain data with domain labels; Images are uniformly and randomly drawn from the original domain and multiple synthetic domains to form an augmented training set with a data volume comparable to that of a large-scale visual location recognition dataset.
[0009] In one embodiment, a batch of augmented training set images is input, and the domain-independent visual location recognition model generates query features, image features, and a global descriptor based on the currently input images, including: Input a batch of augmented training set images, which are then processed by the backbone network to obtain... N indivual d 3D image features, image features are sequentially passed through L One BoQ block; In the l In each BoQ block, the input image features After processing by the encoder, the updated image features are obtained. In the BoQ block M After a learnable query vector is updated through a self-attention mechanism, it is compared with... Perform cross-attention to generate query features; The query features output from all BoQ blocks are concatenated along the sequence dimension and then linearly weighted through a linear projection layer to obtain the weighted query features. ,Will After flattening, L2 normalization is performed to obtain the global descriptor.
[0010] In one embodiment, for a domain-labeled image, the query features and image features are passed through a gradient inversion layer and then input into a shared neighborhood discriminator, enabling the domain-independent visual location recognition model to learn domain-independent features, including: For images with domain labels, the query features are input into the shared neighborhood discriminator after passing through a gradient inversion layer; after the image features are reshaped into a two-dimensional feature map, they are input into the neighborhood feature extractor after passing through a gradient inversion layer to extract neighborhood feature vectors, and then input into the shared neighborhood discriminator. The gradient inversion layer performs an identity mapping during forward propagation and multiplies the gradient by a negative coefficient during backward propagation. The shared neighborhood discriminator optimizes classification ability by minimizing cross-entropy loss, and the gradient inversion layer causes the BoQ block to be updated in the direction of increasing classification loss, resulting in neighborhood-invariant features.
[0011] In one embodiment, query combinations are obtained based on query features, and the triple index corresponding to the global descriptor is obtained; reliable query combinations are filtered and the most difficult negative samples and the most easily distinguishable positive sample combinations are mined, including: Decompose the weighted query features into Each of the unnormalized vectors is L2 normalized to obtain the query combination; An online hard case mining strategy is used to obtain triple indexes from global descriptors and corresponding location labels:
[0012] in, Strategies for mining difficult cases online; For global descriptors; Label the location; The index set for anchor point samples; Positive sample index for anchor points; Negative sample index for the anchor point; For anchor points The i Query combinations anchor point The positive sample with the highest similarity among all positive samples. i Each query combination is considered a positive sample query combination, and the similarity of the positive samples is calculated according to the following formula:
[0013] in, Positive sample similarity; anchor point The corresponding positive sample index set; For the dot product operation of vectors, ; anchor point The iA combination of queries; anchor point Positive samples j The first in i A combination of queries; From the anchor point All negative samples The global descriptor most similar to the anchor point is selected from the options. These samples constitute a difficult negative sample pool. For anchor points The i Query combinations With a difficult negative sample pool The negative sample with the highest similarity i Each query combination is used as a negative sample query combination, and the similarity of the negative samples is calculated according to the following formula:
[0014] in, Negative sample similarity; Select the positive samples with the highest similarity H Calculate the triplet loss for each query combination.
[0015] In one embodiment, the triplet loss is calculated according to the following formula:
[0016] in, For triplet loss; This represents the number of anchor points in the batch. This represents the marginal value of the triplet loss.
[0017] In one embodiment, the total loss is:
[0018] in, Total loss; For multi-similarity loss of global descriptors; To query the adversarial loss at the feature level; Adversarial loss at the image feature level; The weight hyperparameters for the triplet loss; To query the weight hyperparameters of the adversarial loss at the feature level; represents the weight hyperparameters of the adversarial loss at the image feature level.
[0019] In one embodiment, the adversarial loss of the query feature hierarchy is calculated according to the following formula:
[0020] in, For the first Query characteristics; For parameters Gradient inversion layer; For real-world domain labels; The cross-entropy loss function; For shared domain discriminators; The adversarial loss at the image feature level is calculated according to the following formula:
[0021] in, This is the feature vector of the domain.
[0022] In one embodiment, the shared neighborhood discriminator includes two hidden layers and a ReLU activation function.
[0023] A query-based domain-independent visual location recognition device, the device comprising: The model building module is used to build a domain-independent visual location recognition model, which consists of a backbone network, multiple cascaded BoQ blocks, and a linear projection layer; each BoQ block contains an encoder and a set of learnable query vectors. The training set construction module is used to construct an augmented training set based on a large-scale visual location recognition dataset using style transfer techniques. The augmented training set contains images with domain labels and original images without domain labels. The model input module is used to input a batch of augmented training set images. The domain-independent visual location recognition model generates query features, image features, and global descriptors based on the currently input images. The two-layer adversarial training module is used to input query features and image features into a shared domain discriminator after passing them through a gradient inversion layer for domain-labeled images. This enables the domain-independent visual location recognition model to learn domain-independent features and calculates the adversarial loss at the query feature level and the adversarial loss at the image feature level. The triplet supervision module is used to obtain query combinations based on query features and obtain the triplet index corresponding to the global descriptor; it filters reliable query combinations and mines the most difficult negative sample and the most easily distinguishable positive sample combination, and calculates the triplet loss. The loss training module is used to construct the total loss by combining the multi-similarity loss of the global descriptor, the triplet loss, the adversarial loss at the query feature level, and the adversarial loss at the image feature level. The total loss is optimized until convergence, and the trained domain-independent visual location recognition model is obtained. The visual location recognition module is used to perform visual location recognition using a trained domain-independent visual location recognition model.
[0024] The aforementioned query-based domain-independent visual location recognition method and apparatus generate domain-labeled data through style transfer technology and construct an augmented training set, thereby providing explicit supervision signals for subsequent training. This enables the domain-independent visual location recognition model to more effectively remove domain information. A two-level adversarial learning framework is introduced, where query features and image features are input into a shared domain discriminator after passing through a gradient inversion layer. Adversarial training is then used to force the model to lose domain-specific information at both feature levels, allowing the domain-independent visual location recognition model to learn domain-independent features. By selecting reliable query combinations and mining the most difficult negative sample and the most easily distinguishable positive sample combinations, a triplet loss is calculated, and a query combination-based triplet supervision strategy is implemented, thereby enhancing the discriminative power of domain-invariant features.
[0025] This invention achieves optimal positioning accuracy on multiple cross-domain visual location recognition benchmark datasets and has excellent domain generalization ability. Attached Figure Description
[0026] Figure 1 This is a flowchart illustrating a query-based domain-independent visual location recognition method in one embodiment; Figure 2 This is a schematic diagram of the structure of a domain-independent visual location recognition model in one embodiment, wherein (a) is a schematic diagram of the basic framework and (b) is a schematic diagram of a two-level adversarial learning framework; Figure 3 This is a structural block diagram of a query-based domain-independent visual location recognition device in one embodiment. Detailed Implementation
[0027] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0028] In one embodiment, such as Figure 1 , Figure 2 As shown, a query-based domain-independent visual location recognition method is provided, the method comprising the following steps: Step 201: Construct a domain-independent visual location recognition model, which consists of a backbone network, multiple cascaded BoQ blocks, and a linear projection layer; each BoQ block contains an encoder and a set of learnable query vectors.
[0029] It should be noted that BoQ (Bag-of-Queries) is a learnable query feature aggregation architecture. The constructed domain-independent visual location recognition model extracts domain-independent and highly discriminative global descriptors from the input image.
[0030] Step 202: Based on a large-scale visual location recognition dataset, construct an augmented training set using style transfer techniques. The augmented training set contains images with domain labels and original images without domain labels.
[0031] Step 203: Input a batch of augmented training set images. The domain-independent visual location recognition model generates query features, image features, and global descriptors based on the currently input images.
[0032] Step 204: For images with domain labels, the query features and image features are passed through a gradient inversion layer and then input into a shared domain discriminator, so that the domain-independent visual location recognition model learns domain-independent features, and the adversarial loss at the query feature level and the adversarial loss at the image feature level are calculated.
[0033] Step 205: Obtain query combinations based on query features and obtain the triple index corresponding to the global descriptor; filter reliable query combinations and mine the most difficult negative sample and the most easily distinguishable positive sample combination, and calculate the triple loss.
[0034] Step 206: Combine the multi-similarity loss of the global descriptor, the triplet loss, the adversarial loss at the query feature level, and the adversarial loss at the image feature level to construct the total loss. Optimize the total loss until convergence to obtain the trained domain-independent visual location recognition model.
[0035] Step 207: Perform visual location recognition using the trained domain-independent visual location recognition model.
[0036] In the aforementioned query-based domain-independent visual location recognition method, domain-labeled data is generated through style transfer technology, and an augmented training set is constructed to provide explicit supervision signals for subsequent training, enabling the domain-independent visual location recognition model to more effectively remove domain information. A two-level adversarial learning framework is introduced, in which query features and image features are input into a shared domain discriminator after passing through a gradient inversion layer. Then, adversarial training is used to force the model to lose domain-specific information at both feature levels, enabling the domain-independent visual location recognition model to learn domain-independent features. By selecting reliable query combinations and mining the most difficult negative sample and the most easily distinguishable positive sample combinations, triple loss is calculated, and a query combination-based triple supervision strategy is implemented to enhance the discriminativeness of domain-invariant features.
[0037] In one embodiment, an augmented training set is constructed using style transfer techniques based on a large-scale visual location recognition dataset, including: Style transfer techniques are used to augment large-scale visual location recognition datasets, generating various synthetic domain data with domain labels; Images are uniformly and randomly drawn from the original domain and multiple synthetic domains to form an augmented training set with a data volume comparable to that of a large-scale visual location recognition dataset.
[0038] It should be noted that most of the images in the augmented training set are domain-labeled images.
[0039] In this embodiment, a variety of synthetic domain data with domain labels are generated through style transfer technology, thereby constructing an augmented training set containing domain-labeled images and original domain images, providing supervision signals for subsequent two-level adversarial learning, enabling the model to remove domain information more effectively.
[0040] Specifically, in one embodiment, six synthetic domain data (fog, rain, snow, wind, night, and clear sky) were generated using style transfer technology. Images were uniformly and randomly extracted from the original domain and the six synthetic domains to form an augmented training set with a data volume comparable to a large-scale visual location recognition dataset.
[0041] In one embodiment, a batch of augmented training set images is input, and the domain-independent visual location recognition model generates query features, image features, and a global descriptor based on the currently input images, including: Input a batch of augmented training set images, which are then processed by the backbone network to obtain... N indivual d 3D image features, image features are sequentially passed through L One BoQ block; Specifically, the backbone network can be DINOv2-B followed by 3×3 convolutional layers for dimensionality reduction; the input image... (in, For the height of the image, The image width is used to obtain a set of local image feature sequences through a backbone network. The backbone network extracts features from the last 4 layers of DINOv2-B and concatenates them, then performs dimensionality reduction via convolution to obtain... N indivual d 3D image features .
[0042] In the l ( In ) BoQ blocks, the input image features After processing by the encoder, the updated image features are obtained. ( in, (for standard transformer structure), in the BoQ block M After a learnable query vector is updated through a self-attention mechanism, it is compared with... Perform cross-attention to generate query features; Specifically, in the BoQ blockM A learnable query vector The system integrates information from each other through a self-attention mechanism, thereby updating the information. ,in, For bullish attention operations; subsequently, with Perform cross-attention, aggregate image content, and generate query features. .
[0043] The query features output from all BoQ blocks are concatenated along the sequence dimension and then linearly weighted through a linear projection layer to obtain the weighted query features. ,Will After flattening, L2 normalization is performed to obtain the global descriptor.
[0044] Specifically, the query features output by all BoQ blocks are concatenated along the sequence dimension to obtain... After linear weighting through a linear projection layer, the weighted query features are obtained. ,Will After flattening, L2 normalization is performed to obtain the global descriptor.
[0045] In one embodiment, for a domain-labeled image, the query features and image features are passed through a gradient inversion layer and then input into a shared neighborhood discriminator, enabling the domain-independent visual location recognition model to learn domain-independent features, including: For images with domain labels, the query features are input into the shared neighborhood discriminator after passing through a gradient inversion layer; after the image features are reshaped into a two-dimensional feature map, they are input into the neighborhood feature extractor after passing through a gradient inversion layer to extract neighborhood feature vectors, and then input into the shared neighborhood discriminator. The gradient inversion layer performs an identity mapping during forward propagation and multiplies the gradient by a negative coefficient during backward propagation. The shared neighborhood discriminator optimizes classification ability by minimizing cross-entropy loss, and the gradient inversion layer causes the BoQ block to be updated in the direction of increasing classification loss, resulting in neighborhood-invariant features.
[0046] It should be noted that the goal of the shared domain discriminator is to accurately predict the domain to which the input features belong, and to optimize its classification ability by minimizing the cross-entropy loss.
[0047] In this embodiment, a two-layer adversarial learning framework is used to apply domain discrimination constraints at the query feature level and the image feature level, respectively, forcing the model to learn domain-invariant feature representations at both key levels, thereby making the global descriptor output by the model domain-independent.
[0048] Specifically, the shared neighborhood discriminator comprises two hidden layers and a ReLU activation function, and its structure is formalized as follows:
[0049] in, For dimension d The input feature vector; and For learnable weights and biases; The output is a classification logits for multiple synthesis domains. and These are the outputs of the first hidden layer and the second hidden layer, respectively.
[0050] Image features Reconstructed into a two-dimensional feature map Then, it is input into the neighborhood feature extractor through the gradient inversion layer. Extracting neighborhood feature vectors Domain Feature Extractor Composed of a small convolutional network, it aims to extract compact neighborhood feature representations from image features. Its specific computation process is as follows:
[0051] in, A convolutional layer with a kernel size of 3×3 that preserves spatial dimensions; This is an average pooling layer with a kernel size and step size of 2. For globally adaptive average pooling, the spatial dimension is compressed to 1×1; This is the output of the first convolutional layer; This is the output of the average pooling layer; This is the output of the second convolutional layer.
[0052] By extracting the domain feature vector The input is a shared-domain discriminator, which outputs classification logits for multiple synthetic domains and optimizes them using cross-entropy loss.
[0053] By employing a two-level adversarial training framework, the shared domain discriminator and domain feature extractor are trained to accurately identify the domain to which an image belongs. On the other hand, due to gradient inversion, the underlying image features are driven to evolve in directions that the discriminator cannot correctly classify, thus achieving domain invariance at the image feature level. This two-level adversarial constraint makes the model's domain-invariant learning more thorough, effectively improving its overall generalization ability.
[0054] In one embodiment, query combinations are obtained based on query features, and the triple index corresponding to the global descriptor is obtained; reliable query combinations are filtered and the most difficult negative samples and the most easily distinguishable positive sample combinations are mined, including: Decompose the weighted query features into Each of the unnormalized vectors is L2 normalized to obtain the query combination; An online hard case mining strategy is used to obtain triple indexes from global descriptors and corresponding location labels:
[0055] in, Strategies for mining difficult cases online; For global descriptors; Label the location; The index set for anchor point samples; Positive sample index for anchor points; Negative sample index for the anchor point; For anchor points The i Query combinations anchor point The positive sample with the highest similarity among all positive samples. i Each query combination is considered a positive sample query combination, and the similarity of the positive samples is calculated according to the following formula:
[0056] in, Positive sample similarity; anchor point The corresponding positive sample index set; For the dot product operation of vectors, ; anchor point The i A combination of queries; anchor point Positive samples j The first in i A combination of queries; From the anchor point All negative samples The global descriptor most similar to the anchor point is selected from the options. These samples constitute a difficult negative sample pool. For anchor points The i Query combinations With a difficult negative sample pool The negative sample with the highest similarity i Each query combination is used as a negative sample query combination, and the similarity of the negative samples is calculated according to the following formula:
[0057] in, Negative sample similarity; Select the positive samples with the highest similarityH Calculate the triplet loss for each query combination.
[0058] It should be noted that the similarity of positive samples is the anchor point. The first of all positive samples i Query combinations and anchors The i Query combinations Similarity; negative sample similarity is for the hard negative sample pool The i Query combinations and anchors The i Query combinations The similarity is calculated. Each query combination can be viewed as a weighted aggregation of the original query features, which together constitute a multi-view description of the image content. Considering that not all query combinations have equal discriminative power for the current anchor point, a maximum similarity strategy is used to select the top positive samples with the highest similarity. H The triplet loss is calculated for each query combination, thus focusing supervision on reliable query combinations.
[0059] In this embodiment, a triplet supervision strategy based on query combination is designed, which further enhances the discriminativeness of domain-invariant features by focusing on the most reliable and most difficult query combinations.
[0060] In one embodiment, the triplet loss is calculated according to the following formula:
[0061] in, For triplet loss; This represents the number of anchor points in the batch. This represents the marginal value of the triplet loss.
[0062] In this embodiment, the designed triplet loss ensures that the model can focus on the most informative query combinations in each anchor image. By widening the distance between these reliable combinations and the most similar positive / negative combinations, it encourages each domain-invariant query combination to learn more discriminative visual features, thereby ensuring that the model can focus on the most informative query combinations in each anchor image. This fine-grained supervision indirectly improves the discriminative performance of the final global descriptor, thus achieving better results in cross-domain visual location recognition tasks.
[0063] In one embodiment, the total loss is:
[0064] in, Total loss; For multi-similarity loss of global descriptors; To query the adversarial loss at the feature level; Adversarial loss at the image feature level; The weight hyperparameters for the triplet loss; To query the weight hyperparameters of the adversarial loss at the feature level; represents the weight hyperparameters of the adversarial loss at the image feature level.
[0065] In this embodiment, by jointly optimizing the total loss, the model can maintain or even enhance the discriminative ability of these features across different locations while learning domain-invariant features, ultimately obtaining a robust and highly discriminative global image descriptor.
[0066] In one embodiment, the adversarial loss of the query feature hierarchy is calculated according to the following formula:
[0067] in, For the first Query characteristics; For parameters Gradient inversion layer; For real-world domain labels; The cross-entropy loss function; For shared domain discriminators; The adversarial loss at the image feature level is calculated according to the following formula:
[0068] in, This is the feature vector of the domain.
[0069] To verify the effectiveness of this invention, evaluations were conducted on several publicly available datasets with significant domain variations, including Nordland (seasonal variations), Tokyo24 / 7 (day-night variations), and SVOX (weather and light variations). The evaluation criterion adopted was Recall@N, widely used in visual location recognition tasks, defined as the percentage of queries with at least one correct match among the top N search results, abbreviated as R@N.
[0070] The method described in this invention is compared with state-of-the-art visual location recognition methods (such as SALAD, BoQ, ImAge, EDTformer, etc.). The results are shown in Table 1. On the Nordland dataset, the Recall@1 of the method described in this invention reaches 93.5%, significantly higher than the comparison methods. On the Tokyo24 / 7 dataset, the Recall@1 of the method described in this invention reaches 97.5%, also at the optimal level. Under different weather and lighting conditions on the SVOX dataset, the method described in this invention achieves the highest Recall@1 in almost all scenarios. The experimental results demonstrate that the method described in this invention has domain-independent characteristics and achieves state-of-the-art performance in cross-domain visual location recognition tasks.
[0071] Table 1. Test results of different methods on different datasets.
[0072] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.
[0073] In one embodiment, such as Figure 3 As shown, a query-based domain-independent visual location recognition device is provided, the device comprising: The model building module 901 is used to build a domain-independent visual location recognition model, which consists of a backbone network, multiple cascaded BoQ blocks and linear projection layers; the BoQ block contains an encoder and a set of learnable query vectors.
[0074] Training set construction module 902 is used to construct an augmented training set based on a large-scale visual location recognition dataset using style transfer techniques. The augmented training set contains domain-labeled images and original unlabeled images.
[0075] The model input module 903 is used to input a batch of augmented training set images. The domain-independent visual location recognition model generates query features, image features, and global descriptors based on the currently input images.
[0076] The two-layer adversarial training module 904 is used to input query features and image features into a shared domain discriminator after passing them through a gradient inversion layer for domain-labeled images. This enables the domain-independent visual location recognition model to learn domain-independent features and calculates the adversarial loss at the query feature level and the adversarial loss at the image feature level.
[0077] The triplet supervision module 905 is used to obtain query combinations based on query features and obtain the triplet index corresponding to the global descriptor; to filter reliable query combinations and mine the most difficult negative sample and the most easily distinguishable positive sample combination, and to calculate the triplet loss.
[0078] The loss training module 906 is used to construct the total loss by combining the multi-similarity loss of the global descriptor, the triplet loss, the adversarial loss at the query feature level, and the adversarial loss at the image feature level. The total loss is optimized until convergence, resulting in the trained domain-independent visual location recognition model.
[0079] The visual location recognition module 907 is used to perform visual location recognition using a trained domain-independent visual location recognition model.
[0080] Specific limitations regarding the query-based domain-independent visual location recognition device can be found in the limitations of the query-based domain-independent visual location recognition method described above, and will not be repeated here. Each module in the aforementioned query-based domain-independent visual location recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0081] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0082] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A query-based domain-independent visual location recognition method, characterized in that, The method includes the following steps: A domain-independent visual location recognition model is constructed, which consists of a backbone network, multiple cascaded BoQ blocks, and a linear projection layer; each BoQ block contains an encoder and a set of learnable query vectors. Based on a large-scale visual location recognition dataset, an augmented training set is constructed using style transfer techniques. The augmented training set contains images with domain labels and original images without domain labels. Given a batch of augmented training set images, the domain-independent visual location recognition model generates query features, image features, and global descriptors based on the currently input images. For images with domain labels, query features and image features are passed through a gradient inversion layer and then input into a shared domain discriminator, enabling the domain-independent visual location recognition model to learn domain-independent features and calculate the adversarial loss at the query feature level and the adversarial loss at the image feature level. Query combinations are obtained based on query features, and triple indexes corresponding to global descriptors are obtained; reliable query combinations are filtered and the most difficult negative samples and the most easily distinguishable positive sample combinations are mined, and triple loss is calculated. The total loss is constructed by combining the multi-similarity loss of the global descriptor, the triplet loss, the adversarial loss at the query feature level, and the adversarial loss at the image feature level. The total loss is optimized until convergence, resulting in the trained domain-independent visual location recognition model. Visual location recognition is performed using a trained domain-independent visual location recognition model.
2. The query-based domain-independent visual location recognition method according to claim 1, characterized in that, Based on a large-scale visual location recognition dataset, an augmented training set was constructed using style transfer techniques, including: Style transfer techniques are used to augment large-scale visual location recognition datasets, generating various synthetic domain data with domain labels; Images are uniformly and randomly drawn from the original domain and multiple synthetic domains to form an augmented training set with a data volume comparable to that of a large-scale visual location recognition dataset.
3. The query-based domain-independent visual location recognition method according to claim 1, characterized in that, Given a batch of augmented training set images, the domain-independent visual location recognition model generates query features, image features, and a global descriptor based on the currently input images, including: Input a batch of augmented training set images, which are then processed by the backbone network to obtain... N indivual d 3D image features, image features are sequentially passed through L One BoQ block; In the l In each BoQ block, the input image features After processing by the encoder, the updated image features are obtained. In the BoQ block M After a learnable query vector is updated through a self-attention mechanism, it is compared with... Perform cross-attention to generate query features; The query features output from all BoQ blocks are concatenated along the sequence dimension and then linearly weighted through a linear projection layer to obtain the weighted query features. ,Will After flattening, L2 normalization is performed to obtain the global descriptor.
4. The query-based domain-independent visual location recognition method according to claim 1, characterized in that, For domain-labeled images, the query features and image features are passed through a gradient inversion layer and then input into a shared neighborhood discriminator, enabling the domain-independent visual location recognition model to learn domain-independent features, including: For images with domain labels, the query features are input into the shared neighborhood discriminator after passing through a gradient inversion layer; after the image features are reshaped into a two-dimensional feature map, they are input into the neighborhood feature extractor after passing through a gradient inversion layer to extract neighborhood feature vectors, and then input into the shared neighborhood discriminator. The gradient inversion layer performs an identity mapping during forward propagation and multiplies the gradient by a negative coefficient during backward propagation. The shared neighborhood discriminator optimizes classification ability by minimizing cross-entropy loss, and the gradient inversion layer causes the BoQ block to be updated in the direction of increasing classification loss, resulting in neighborhood-invariant features.
5. The query-based domain-independent visual location recognition method according to claim 3, characterized in that, Query combinations are obtained based on query features, and the triple index corresponding to the global descriptor is obtained; reliable query combinations are filtered, and the most difficult negative sample and the most easily distinguishable positive sample combinations are mined, including: Decompose the weighted query features into Each of the unnormalized vectors is L2 normalized to obtain the query combination; An online hard case mining strategy is used to obtain triple indexes from global descriptors and corresponding location labels: in, Strategies for mining difficult cases online; For global descriptors; Label the location; The index set for anchor point samples; Positive sample index for anchor points; Negative sample index for the anchor point; For anchor points The i Query combinations anchor point The positive sample with the highest similarity among all positive samples. i Each query combination is considered a positive sample query combination, and the similarity of the positive samples is calculated according to the following formula: in, Positive sample similarity; anchor point The corresponding positive sample index set; For the dot product operation of vectors, ; anchor point The i A combination of queries; anchor point Positive samples j The first in i A combination of queries; From the anchor point All negative samples The global descriptor most similar to the anchor point is selected from the options. These samples constitute a difficult negative sample pool. For anchor points The i Query combinations With a difficult negative sample pool The negative sample with the highest similarity i Each query combination is used as a negative sample query combination, and the similarity of the negative samples is calculated according to the following formula: in, Negative sample similarity; Select the positive samples with the highest similarity H Calculate the triplet loss for each query combination.
6. The query-based domain-independent visual location recognition method according to claim 5, characterized in that, The triplet loss is calculated according to the following formula: in, For triplet loss; This represents the number of anchor points in the batch. This represents the marginal value of the triplet loss.
7. The query-based domain-independent visual location recognition method according to claim 6, characterized in that, The total loss is: in, Total loss; For multi-similarity loss of global descriptors; To query the adversarial loss at the feature level; Adversarial loss at the image feature level; The weight hyperparameters for the triplet loss; To query the weight hyperparameters of the adversarial loss at the feature level; represents the weight hyperparameters of the adversarial loss at the image feature level.
8. The query-based domain-independent visual location recognition method according to claim 7, characterized in that, The adversarial loss at the query feature level is calculated according to the following formula: in, For the first Query characteristics; For parameters Gradient inversion layer; For real-world domain labels; The cross-entropy loss function; For shared domain discriminators; The adversarial loss at the image feature level is calculated according to the following formula: in, This is the feature vector of the domain.
9. The query-based domain-independent visual location recognition method according to claim 1, characterized in that, The shared domain discriminator comprises two hidden layers and a ReLU activation function.
10. A query-based domain-independent visual location recognition device, characterized in that, The device includes: The model building module is used to build a domain-independent visual location recognition model, which consists of a backbone network, multiple cascaded BoQ blocks, and a linear projection layer; each BoQ block contains an encoder and a set of learnable query vectors. The training set construction module is used to construct an augmented training set based on a large-scale visual location recognition dataset using style transfer techniques. The augmented training set contains images with domain labels and original images without domain labels. The model input module is used to input a batch of augmented training set images. The domain-independent visual location recognition model generates query features, image features, and global descriptors based on the currently input images. The two-layer adversarial training module is used to input query features and image features into a shared domain discriminator after passing them through a gradient inversion layer for domain-labeled images. This enables the domain-independent visual location recognition model to learn domain-independent features and calculates the adversarial loss at the query feature level and the adversarial loss at the image feature level. The triplet supervision module is used to obtain query combinations based on query features and obtain the triplet index corresponding to the global descriptor; it filters reliable query combinations and mines the most difficult negative sample and the most easily distinguishable positive sample combination, and calculates the triplet loss. The loss training module is used to construct the total loss by combining the multi-similarity loss of the global descriptor, the triplet loss, the adversarial loss at the query feature level, and the adversarial loss at the image feature level. The total loss is optimized until convergence, and the trained domain-independent visual location recognition model is obtained. The visual location recognition module is used to perform visual location recognition using a trained domain-independent visual location recognition model.