Vision position recognition method and device of robot

Abstract

The application relates to the technical field of robots, and provides a visual position recognition method and device of a robot, which comprises the following steps: inputting a query image into a global feature extraction network to obtain an image global retrieval descriptor; and matching the image global retrieval descriptor with each database image descriptor pre-stored in an environment map to obtain a visual position recognition result of the robot. The application generates an inhibition weight of a local feature based on the correlation between the local features, thereby weighting and adjusting the distribution response of the local feature with respect to each preset clustering center by using the inhibition weight, obtaining a weighted and adjusted distribution response, generating an image global retrieval descriptor by residual aggregation of the local feature based on the weighted and adjusted distribution response, increasing the proportion of place discriminative features in the image global retrieval descriptor, and inhibiting non-discriminative repeated background features, so that the representation and differentiation ability of a specific place is enhanced, and the accuracy of the visual position recognition result of the robot is ensured.

Description

Technical Field

[0001] This invention relates to the field of robotics, and in particular to a method and apparatus for visual position recognition of a robot. Background Technology

[0002] During autonomous robot movement, positional information is fundamental for path planning, motion control, and task execution. Due to the cumulative error of visual odometry, continuous positioning results may drift over time, leading to an unstable judgment of the robot's own position.

[0003] To assist in determining the robot's location when the reliability of continuous localization decreases or when relocalization is required, the environmental image currently acquired by the robot can be converted into an image representation suitable for retrieval. This representation can then be compared with a database image representation pre-stored in the environmental map to find the database image that is most similar to the currently acquired image. The robot's current location can then be determined based on the marked position of this database image on the map.

[0004] Current research typically models robot visual location recognition as an image retrieval problem. While this approach enhances image representation capabilities, the contributions of different regions to the global retrieval descriptor often differ during feature aggregation. For less discriminative local features such as repetitive textures and common background regions, retaining a high contribution during aggregation can lead to excessive accumulation of these non-discriminatory features in the global retrieval descriptor, thus weakening the descriptor's ability to represent specific locations. Existing methods suppress excessive accumulation of repetitive structures within a single image during Vector of Locally Aggregated Descriptors (VLAD) aggregation through intra-image self-similarity, but these methods struggle to sufficiently reduce the impact of common background regions between different images on the global image retrieval descriptor. Summary of the Invention

[0005] This invention provides a visual position recognition method and apparatus for robots, which solves the problem that the existing technology can only suppress the excessive accumulation of repetitive structures within a single image in VLAD aggregation through self-similarity relationships within the image, but cannot effectively reduce the influence of common background areas between different images on the global image retrieval descriptor, resulting in the global image retrieval descriptor still containing a large amount of non-discriminative information and insufficient location representation ability.

[0006] This invention provides a visual position recognition method for a robot, comprising the following steps.

[0007] Use the environmental image currently collected by the robot as the query image; The query image is input into a global feature extraction network to obtain the global image retrieval descriptor output by the global feature extraction network; Based on the global image retrieval descriptor and the image descriptors of each database pre-stored in the environment map, the visual position recognition result of the robot is determined; The global feature extraction network includes a local feature extraction module, a repetitive feature suppression module, and a gated local aggregation descriptor vector module; The local feature extraction module is used to extract local features of the query image; the duplicate feature suppression module is used to generate suppression weights for the local features based on the correlation between the local features; the gated local aggregation descriptor vector module is used to determine the allocation response of the local features relative to each preset cluster center, and to perform weighted adjustment on the allocation response based on the suppression weights to obtain a weighted allocation response, and to perform residual aggregation on the local features based on the weighted allocation response to generate the global image retrieval descriptor.

[0008] According to the visual position recognition method for robots provided by the present invention, the global feature extraction network is obtained by iteratively executing the following steps until a preset iteration termination condition is met: Obtain a training sample set, which includes location identifiers for multiple locations and multiple training images corresponding to each location identifier. Divide multiple training images labeled with the same location identifier into a set of images for the same location. Location samples are constructed using the location identifier as a unit, and each location sample includes multiple images randomly sampled from the image set of the location corresponding to the same location identifier; each training batch includes multiple location samples; the training images of the same training batch are input into an initial global feature extraction network to obtain the predicted global retrieval descriptor output by the initial global feature extraction network; the initial global feature extraction network includes an initial local feature extraction module, an initial duplicate feature suppression module, an initial gated local aggregation descriptor vector module, and a cross-image duplicate feature suppression branch; Based on the initial local feature extraction module, the predicted local features of each training image in the training batch are extracted, and based on the initial repetition feature suppression module, the single-frame suppression weights corresponding to each training image are generated. Based on the cross-image repetition feature suppression branch, cross-image suppression weights are determined to characterize the intensity of common background features among different training images within the training batch; The predicted global retrieval descriptor is obtained by weighting and aggregating the predicted local features based on the fusion suppression weights by the initial gated local aggregation descriptor vector module; the fusion suppression weights are obtained by element-wise multiplication of the single-frame suppression weights and the cross-image suppression weights. Based on the predicted global retrieval descriptor and the location identifier, a multi-similarity loss is determined, and the model parameters of the initial global feature extraction network are jointly updated based on the multi-similarity loss.

[0009] According to a visual position recognition method for a robot provided by the present invention, the step of determining cross-image suppression weights based on the cross-image repetition feature suppression branch to characterize the intensity of common background features among different training images in the training batch includes: Spatial adaptive average pooling is performed on the predicted local features to obtain low-resolution local features of each training image. Based on the low-resolution local features, the average appearance features of all images in the training batch are determined, and the similarity matrix between the low-resolution local features and the average appearance features is determined. Based on the statistical distribution of the similarity matrix, the global mean parameter and global standard deviation parameter of the similarity matrix are updated. Then, based on the updated global mean parameter, global standard deviation parameter and the similarity matrix, a standardized mapping and nonlinear activation process are performed to obtain the cross-image suppression weights.

[0010] According to a visual location recognition method for a robot provided by the present invention, the step of determining a multi-similarity loss based on the predicted global retrieval descriptor and the location identifier includes: Using the predicted global retrieval descriptor as the query feature, and based on the location identifiers corresponding to each training image in the training batch, candidate positive samples belonging to the same location identifier as the query feature and candidate negative samples belonging to different location identifiers as the query feature are determined. Based on the multi-similarity sample mining strategy, positive sample pairs are selected from the candidate positive samples, and negative sample pairs are selected from the candidate negative samples; The multi-similarity loss is determined based on the first feature similarity between the predicted global retrieval descriptor and the positive sample pair, and the second feature similarity between the predicted global retrieval descriptor and the negative sample pair.

[0011] According to a visual position recognition method for a robot provided by the present invention, the step of generating suppression weights for the local features based on the correlation between the local features includes: A self-similarity matrix is ​​generated based on the correlation coefficient between any two spatial location features in the local features. The self-similar matrix is ​​linearly mapped based on preset learnable weights, and mean-normalization and exponential transformation are performed on the linearly mapped matrix to obtain the suppression weight of each spatial location feature.

[0012] According to a visual position recognition method for a robot provided by the present invention, the step of determining the allocation response of the local features relative to each preset cluster center, and weighting the allocation response based on the suppression weight to obtain the weighted allocation response, includes: Determine the initial clustering assignment response of the local features relative to each preset cluster center; The suppression weights are normalized and used as a gating control signal. The gating control signal is then multiplied element-wise with the initial clustering allocation response to obtain the weighted and adjusted allocation response.

[0013] According to a method for visual position recognition of a robot provided by the present invention, determining the visual position recognition result of the robot based on the global image retrieval descriptor and pre-stored image descriptors in the environment map includes: The image global retrieval descriptor is dimensionality compressed to obtain a dimensionality-reduced global retrieval descriptor; Determine the Euclidean distance between the dimensionality-reduced global retrieval descriptor and each of the database image descriptors; Based on the Euclidean distance, the database images corresponding to each database image descriptor are sorted, and the position information recorded in the first sorted database image is determined as the visual position recognition result.

[0014] According to a visual position recognition method for a robot provided by the present invention, the step of extracting local features of the query image includes: The query image is input into a preset visual backbone network for feature mapping to obtain the initial feature tensor output by the visual backbone network. The initial feature tensor is rearranged in dimensions, and the rearranged features are normalized along the channel dimension to obtain the local features.

[0015] According to the present invention, a visual position recognition method for a robot, wherein the step of performing residual aggregation on the local features based on the weighted and adjusted allocation response to generate the global image retrieval descriptor includes: Determine the feature residuals between the local features and each of the preset cluster centers; Based on the weighted adjustment of the allocation response, the feature residuals are weighted and summed to obtain the local aggregation vector corresponding to each preset cluster center; The global image retrieval descriptor is determined based on the local aggregation vectors corresponding to each preset cluster center.

[0016] The present invention also provides a visual position recognition device for a robot, comprising the following modules: The acquisition unit is used to acquire the environmental image currently collected by the robot as the query image; The input unit is used to input the query image into the global feature extraction network to obtain the global image retrieval descriptor output by the global feature extraction network. The matching unit is used to determine the visual position recognition result of the robot based on the global image retrieval descriptor and the image descriptors of each database pre-stored in the environment map. The global feature extraction network includes a local feature extraction module, a repetitive feature suppression module, and a gated local aggregation descriptor vector module; The local feature extraction module is used to extract local features of the query image; the duplicate feature suppression module is used to generate suppression weights for the local features based on the correlation between the local features; the gated local aggregation descriptor vector module is used to determine the allocation response of the local features relative to each preset cluster center, and to perform weighted adjustment on the allocation response based on the suppression weights to obtain a weighted allocation response, and to perform residual aggregation on the local features based on the weighted allocation response to generate the global image retrieval descriptor.

[0017] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the visual position recognition method of the robot as described above.

[0018] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the visual position recognition method for a robot as described above.

[0019] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the visual position recognition method for a robot as described above.

[0020] The visual position recognition method and apparatus for robots provided by this invention generate suppression weights for local features based on the correlation between local features. These suppression weights are then used to weight and adjust the allocation responses of local features relative to each preset cluster center, resulting in a weighted allocation response. This weighted allocation response is then used to perform residual aggregation on the local features, generating a global image retrieval descriptor. This increases the proportion of location-discriminative features in the global image retrieval descriptor while suppressing non-discriminative repetitive background features, thereby enhancing the representation and differentiation capabilities of specific locations. This avoids the problem of excessive accumulation of weakly discriminative local features such as repetitive textures and common background areas in the global image retrieval descriptor, ensuring the accuracy of the robot's visual position recognition results. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating the visual position recognition method for robots provided by the present invention.

[0023] Figure 2 This is a flowchart illustrating the training steps of the global feature extraction network provided by the present invention.

[0024] Figure 3 This is a schematic diagram of the process for determining multiple similarity loss provided by the present invention.

[0025] Figure 4 This is a schematic diagram of the structure of the visual position recognition device for robots provided by the present invention.

[0026] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0028] This invention provides a visual position recognition method for robots, which can be applied to control systems such as autonomous navigation, path planning, or relocation of robots. Figure 1 This is a flowchart illustrating the visual position recognition method for robots provided by the present invention, as shown below. Figure 1 As shown, the method includes the following: Step 110: Obtain the environmental image currently collected by the robot as the query image.

[0029] Specifically, firstly, the environmental image currently being captured by the robot can be used as the query image. This environmental image reflects the visual characteristics of the robot's surroundings. In practice, various visual sensors mounted on the robot, such as monocular cameras, binocular cameras, RGB-D cameras, or panoramic cameras, can be used to capture real-time images of the surrounding environment, which are then used as the query image for performing this visual position recognition task.

[0030] Step 120: Input the query image into the global feature extraction network to obtain the global image retrieval descriptor output by the global feature extraction network; The global feature extraction network includes a local feature extraction module, a repetitive feature suppression module, and a gated local aggregation descriptor vector module; The local feature extraction module is used to extract local features of the query image; the duplicate feature suppression module is used to generate suppression weights for the local features based on the correlation between the local features; the gated local aggregation descriptor vector module is used to determine the allocation response of the local features relative to each preset cluster center, and to perform weighted adjustment on the allocation response based on the suppression weights to obtain a weighted allocation response, and to perform residual aggregation on the local features based on the weighted allocation response to generate the global image retrieval descriptor.

[0031] Specifically, after obtaining the query image, the query image can be input into the global feature extraction network to obtain the global image retrieval descriptor output by the global feature extraction network.

[0032] Here, the global image retrieval descriptor refers to the highly representative and discriminative features of the query image at a global scale.

[0033] Specifically, to overcome the deficiency in existing technologies where features from different regions contribute indiscriminately to the global retrieval descriptor, the global feature extraction network can include a local feature extraction module, a repetition feature suppression module, and a gated local aggregation descriptor vector module.

[0034] Here, the local feature extraction module is used to extract local features from the query image. Local features reflect detailed information about different spatial locations or image patches in the query image, such as shape, texture, color, or deep semantic features. The local feature extraction module scans and maps the query image through a deep network structure, transforming the query image into local features that can characterize the attributes of each local region of the image.

[0035] Here, the repetition feature suppression module generates suppression weights for local features based on the correlation between them. The correlation between local features reflects the similarity between features in different local regions within the query image. By calculating this correlation within the network, repetitive structures within the image can be automatically identified, such as large areas of sky, road surfaces, and repetitive building facades—non-discriminatory background features.

[0036] Here, the suppression weight is used to reflect the degree of suppression or retention of different local features in the subsequent feature aggregation process. For local features that are highly correlated and are judged as redundant or repetitive regions, the repetitive feature suppression module will generate a smaller suppression weight for the local feature to suppress it; while for local features that are unique and have strong discriminative power, such as the edge of a specific landmark building, a larger weight will be generated for the local feature to retain it.

[0037] Here, the gated local aggregation descriptor vector module is used to determine the allocation response of local features relative to each preset cluster center, and to adjust the allocation response based on the suppression weight to obtain the weighted allocation response. Based on the weighted allocation response, the local features are residual aggregated to generate the global image retrieval descriptor.

[0038] Here, the preset cluster centers are preferably learnable cluster centers, which can be understood as a set of feature space clustering benchmarks learned during network training, used to characterize the typical distribution pattern of local features in the feature space. The assignment response reflects the probability or degree of soft assignment of each local feature to each preset cluster center.

[0039] This application ingeniously introduces a gating mechanism, namely, weighting the allocation response based on suppression weights. The suppression weights output by the repetitive feature suppression module are used as a gating control signal and combined with the initial allocation response. Through this adjustment, the allocation response of local features identified as repetitive features is significantly weakened, thereby reducing their proportion in the global features. Subsequently, residual aggregation is performed on the local features based on the weighted allocation response. That is, the feature residuals between the local features and their corresponding preset cluster centers are calculated and weighted and accumulated according to the adjusted allocation response. Finally, a global image retrieval descriptor with stronger discriminative power and free from common background interference is output.

[0040] Step 130: Based on the global image retrieval descriptor and the image descriptors of each database pre-stored in the environment map, determine the visual position recognition result of the robot.

[0041] Specifically, after obtaining the global image retrieval descriptor, the global image retrieval descriptor can be matched with the pre-stored image descriptors in the environment map to obtain the robot's visual position recognition result.

[0042] The environment map includes pre-built and stored database images of multiple known geographical locations, as well as database image descriptors corresponding to the database images after global feature extraction.

[0043] Here, the visual position recognition result of the robot is determined based on the global image retrieval descriptor and the pre-stored database image descriptors in the environment map. This can be achieved by calculating the similarity or spatial distance between the global image retrieval descriptor and each database image descriptor in the environment map to measure the matching degree, and determining the current position of the robot based on the geographical coordinates associated with the database image descriptor with the highest matching degree, thus obtaining the visual position recognition result.

[0044] The method provided in this invention generates suppression weights for local features based on the correlation between local features. These suppression weights are then used to weight and adjust the allocation responses of local features relative to each preset cluster center, resulting in a weighted allocation response. This weighted allocation response is then used to perform residual aggregation on the local features, generating a global image retrieval descriptor. This increases the proportion of location-discriminative features in the global image retrieval descriptor while suppressing non-discriminative repetitive background features, thereby enhancing the representation and differentiation capabilities of specific locations. This avoids the problem of excessive accumulation of weakly discriminative local features such as repetitive textures and common background areas in the global image retrieval descriptor, ensuring the accuracy of the robot's visual location recognition results.

[0045] Based on the above embodiments, the global feature extraction network is obtained by iteratively executing the following steps until a preset iteration termination condition is met: Step 21: Obtain a training sample set, which includes location identifiers for multiple locations and multiple training images corresponding to each location identifier. Divide the multiple training images labeled with the same location identifier into a set of images for the same location. Step 22: Construct location samples based on the location identifiers. Each location sample includes multiple images randomly sampled from the image set corresponding to the same location identifier. Each training batch includes multiple location samples. Input the training images of the same training batch into the initial global feature extraction network to obtain the predicted global retrieval descriptor output by the initial global feature extraction network. The initial global feature extraction network includes an initial local feature extraction module, an initial duplicate feature suppression module, an initial gated local aggregation descriptor vector module, and a cross-image duplicate feature suppression branch. Step 23: Extract the predicted local features of each training image in the training batch based on the initial local feature extraction module, and generate the single-frame suppression weights corresponding to each training image based on the initial repetition feature suppression module. Step 24: Determine cross-image suppression weights based on the cross-image repetition feature suppression branch to characterize the intensity of common background features among different training images in the training batch; Step 25: The initial gated local aggregation descriptor vector module performs weighted aggregation on the predicted local features based on the fusion suppression weight to obtain the predicted global retrieval descriptor; the fusion suppression weight is obtained by element-wise multiplication of the single-frame suppression weight and the cross-image suppression weight. Step 26: Determine the multi-similarity loss based on the predicted global retrieval descriptor and the location identifier, and jointly update the model parameters of the initial global feature extraction network based on the multi-similarity loss.

[0046] Specifically, Figure 2 This is a flowchart illustrating the training steps of the global feature extraction network provided by the present invention, as shown below. Figure 2 As shown, firstly, a training sample set is obtained, which includes location identifiers for multiple locations and multiple training images corresponding to each location identifier. Multiple training images labeled with the same location identifier are then divided into a set of images for the same location.

[0047] Here, the training sample set refers to a large-scale image dataset used by the global feature extraction network to learn the visual features of the environment. Location identifiers refer to the real-world geographic location labels or category IDs corresponding to each training image. Multiple training images refer to image samples collected from different perspectives or at the same location identifier.

[0048] Here, the dimension of the training image input is . ,in and These are the height and width of the training image, respectively. and The preferred value is one that is divisible by 14.

[0049] Secondly, location samples are constructed based on location identifiers. Each location sample includes multiple images randomly sampled from the image set of the location corresponding to the same location identifier. Each training batch includes multiple location samples, and the training images of the same training batch are input into the initial global feature extraction network to obtain the predicted global retrieval descriptor output by the initial global feature extraction network.

[0050] In this context, a training batch refers to a subset of images processed simultaneously during a single forward and backward propagation of the global feature extraction network. During training, samples are constructed on a location-based basis. For each location, four training images are randomly sampled from the corresponding image set, and these four images are configured with the same location. When using batch training, each batch preferably contains 60 location sample images, with each location sample comprising four randomly sampled images; therefore, each batch contains a total of 240 training images.

[0051] The initial global feature extraction network includes an initial local feature extraction module, an initial repetitive feature suppression module, an initial gated local aggregation descriptor vector module, and a cross-image repetitive feature suppression branch.

[0052] It should be noted that the cross-image repetition feature suppression branch, as an auxiliary constraint structure in the training phase, only works in the training phase and does not participate in the execution in the testing (inference) phase. It is used to guide the initial global feature extraction network to reduce the response intensity of common backgrounds between different images in the training batch.

[0053] Here, the predicted global retrieval descriptor is used to reflect the global feature representation output by the initial global feature extraction network under the current network parameters.

[0054] Furthermore, the predicted local features of each training image in the training batch can be extracted based on the initial local feature extraction module, and the single-frame suppression weights corresponding to each training image can be generated based on the initial repetition feature suppression module.

[0055] In this context, predicted local features refer to the local spatial and semantic features extracted by the global feature extraction network from the current training image during the initial training phase. Here, single-frame suppression weights refer to the weight matrix calculated using only the internal feature self-similarity of the current single training image to suppress internal repetitive textures.

[0056] Meanwhile, cross-image suppression weights are determined based on the cross-image repetition feature suppression branch to characterize the intensity of common background features among different training images within a training batch.

[0057] The common background feature intensity reflects the salience of non-discriminatory features that coexist across multiple images within a training batch, such as large areas of the same sky or similar road surfaces. Cross-image suppression weights refer to the suppression coefficients generated by the initial global feature extraction network for these common background regions appearing across images, based on the global perspective of the entire batch.

[0058] Then, the initial gated local aggregation descriptor vector module can perform weighted aggregation of the predicted local features based on the fusion suppression weight to obtain the predicted global retrieval descriptor. The fusion suppression weight is obtained by multiplying the single-frame suppression weight and the cross-image suppression weight element by element.

[0059] Here, the fusion suppression weights are used to reflect the dual modulating information of suppressing repetitive textures within a single image and suppressing common backgrounds across images. Multiplying these two weights element-wise ensures that local features with low repetition levels within a single frame and uncommon appearance relative to the batch average receive a relatively higher response during the aggregation process, ultimately generating a predicted global retrieval descriptor.

[0060] Finally, the multi-similarity loss can be determined based on the predicted global retrieval descriptor and location identifier, and the model parameters of the initial global feature extraction network can be jointly updated based on the multi-similarity loss to finally obtain the global feature extraction network.

[0061] Here, the multi-similarity loss is used to optimize the distribution of predicted global retrieval descriptors in the metric space, thereby narrowing the distance between descriptors corresponding to images with the same location identifier and increasing the distance between descriptors corresponding to images with different locations. By calculating the multi-similarity loss, the AdamW optimizer is used to jointly update the model parameters of all modules in the initial global feature extraction network through backpropagation. The preferred number of training epochs is 8 generations; the initial learning rate of the initial local feature extraction module is preferably set to 6 × 10⁻⁶. -5 The initial learning rate for the remaining modules and branches is preferably set to 1.2 × 10⁻⁶. -4 The preferred learning rate scheduling strategy is a cosine decay strategy that includes a warm-up phase, with the number of warm-up steps set to 500 and the minimum learning rate ratio for the cosine decay phase set to 0.05.

[0062] The preset iteration termination condition can be that the performance index of the initial global feature extraction network on the validation set no longer improves, or that the value of the multiple similarity loss function tends to stabilize and no longer decreases after several consecutive iterations, or that the preset maximum number of training rounds is reached. This embodiment of the invention does not specifically limit this.

[0063] The method provided in this embodiment of the invention adds a cross-image repetitive feature suppression branch parallel to the initial repetitive feature suppression module for joint training during the training phase of the initial global feature extraction network. This can effectively suppress the influence of common common background regions between different images within the training batch, guide the initial global feature extraction network to focus on discriminative features strongly correlated with specific locations, and fundamentally enhance the robustness and representational ability of the image global retrieval descriptor output by the trained global feature extraction network.

[0064] Based on the above embodiments, step 24 includes: Step 241: Perform spatial adaptive average pooling on the predicted local features to obtain low-resolution local features of each training image. Step 242: Based on the low-resolution local features, determine the average appearance features of all images in the training batch, and determine the similarity matrix between the low-resolution local features and the average appearance features; Step 243: Based on the statistical distribution of the similarity matrix, update the global mean parameter and global standard deviation parameter of the similarity matrix, and perform standardized mapping and nonlinear activation processing based on the updated global mean parameter, global standard deviation parameter and the similarity matrix to obtain the cross-image suppression weight.

[0065] Specifically, firstly, spatial adaptive average pooling can be performed on the predicted local features to obtain low-resolution local features of each training image.

[0066] Low-resolution local features refer to the basic features that retain macroscopic semantics but remove subtle redundant information after downsampling the predicted local features. Specifically, the cross-image repetition feature suppression branch serves as an auxiliary constraint during the training phase, guiding the repetition feature suppression module to reduce the response intensity of common features between different images within the training batch, i.e., the intensity of common background features; during the testing phase, the cross-image repetition feature suppression branch does not participate in the inference process.

[0067] Here, the cross-image repetition feature suppression branch includes the first adaptive average pooling operation Pool_1, the fifth-dimensional rearrangement operation Reshape_5, the first-layer normalization operation LayerNorm_1, and the fourth-layer L2 normalization operation L2Norm_4, etc. This suppresses local features... R 3. Input Pool_1 (output space size is 8×8) to obtain the first initial low-resolution local features. LR 1. The dimensions are 768×8×8. Reshape_5 will... LR 1. Rearrangement into second initial low-resolution local features LR 2. The dimensions are 64×768. LR 2. After passing through LayerNorm_1 and L2Norm_4 with normalization dimensions of 768, low-resolution local features are obtained. LR 3. The dimensions are 64×768.

[0068] Then, based on the low-resolution local features, the average appearance features of all images in the training batch can be determined, and the similarity matrix between the low-resolution local features and the average appearance features can be determined.

[0069] Here, the average appearance feature refers to the mean center representation of the features of all training images in the current training batch; the similarity matrix is ​​used to reflect the closeness between the low-resolution local features of a single training image and the average appearance of the entire batch.

[0070] Specifically, the second averaging operation Mean_2 is used to process low-resolution local features. LR 3. Calculate the average in the first dimension to obtain the first batch of intra-image representations. BF 1. The dimensions are 768; BF The second batch of in-line image representations is obtained after processing by the fifth L2 normalization operation, L2Norm_5. BF 2. The size is 768; the third averaging operation Mean_3 is used to... BF 2. The average appearance feature is obtained by averaging the image representations within the training batch. BM The size is 768. Low-resolution local features. LR 3 and average appearance characteristics BM After processing using the second matrix multiplication operation MatMul_2, the first batch of commonness matrices is obtained. C 1. The size is 64, and the similarity matrix is ​​obtained through linear mapping. C 2: C 2 = -2 + 2· C 1. The size is 64.

[0071] Finally, based on the statistical distribution of the similarity matrix, the global mean and global standard deviation parameters of the similarity matrix can be updated. Then, based on the updated global mean, global standard deviation parameters, and similarity matrix, standardized mapping and nonlinear activation processing can be performed to obtain cross-image suppression weights.

[0072] Among them, the global mean parameter and the global standard deviation parameter are used to reflect the global statistical information of similarity accumulated across multiple training batches, preventing drastic fluctuations in the data distribution within a single batch.

[0073] Here, the similarity matrix C 2. After performing the fourth averaging operation (Mean_4) and the first standard deviation calculation operation (Std_1), the similarity matrix is ​​obtained. C mean of 2 and standard deviation Using the first normalization operation Normalize_1 to C 2. After standardization, the commonness matrix of the third batch is obtained. C 3. The size is 64, and the formula is as follows: ; in, This represents the commonity matrix within the third batch. Represents the similarity matrix. This represents the mean of the similarity matrix. This represents the standard deviation of the similarity matrix.

[0074] Subsequently, the cross-batch exponential moving average operation EMA_1 is introduced for... Perform the fifth averaging operation, Mean_5, to obtain... mean At the same time, Perform the second standard deviation calculation operation Std_2 to obtain Standard deviation .Will and the global mean parameter implemented by the register_buffer function RM glo The updated global mean parameter is obtained by performing a weighted summation operation (WeightSum_1) with a ratio of 0.1:0.9. and global standard deviation parameter RStd glo (Initially set to 1) The updated global standard deviation parameter is obtained by weighting and summing the values ​​in a ratio of 0.1:0.9 using WeightSum_1. In the second standardization operation Normalize_2, the global mean parameter is used... RM glo and global standard deviation parameter RStd glo right Perform a standardized mapping to obtain the fourth batch in-group commonity matrix. The size is 64.

[0075] With the fourth learnable weight initially set to 1 LW Multiply by 4 and combine with the fifth learnable weight initially set to 0. LW The sum of 5 is processed by a non-linear activation method, such as the second Sigmoid operation (Sigmoid_2), to obtain the fifth batch in-batch commonity matrix. The size is 64. (The rest of the text appears to be incomplete and requires further context.) After processing with the sixth-dimensional rearrangement operation Reshape_6, the commonality matrix within the sixth batch is obtained. The dimensions are 1×8×8. The commonity matrix within the seventh batch was obtained by upsampling using the first upsampling operation Interp_1 in a bilinear interpolation mode. The dimensions are 1×(H / 14)×(W / 14), and the cross-image suppression weights are obtained by performing a seventh-dimensional rearrangement operation, Reshape_7. WB 2. The size is L.

[0076] The method provided in this invention standardizes the similarity matrix by using spatial adaptive pooling and momentum update of global statistical parameters. This enables more stable and adaptive quantization of the intensity of common background features across images, greatly avoiding interference from fluctuations in batch data distribution during training on the calculation of suppression weights and improving the stability of global feature extraction network optimization.

[0077] Based on the above embodiments, step 26, which involves determining the multiple similarity loss based on the predicted global retrieval descriptor and the location identifier, includes: Step 261: Using the predicted global retrieval descriptor as the query feature, and based on the location identifiers corresponding to each training image in the training batch, determine the candidate positive samples that belong to the same location identifier as the query feature, and the candidate negative samples that belong to different location identifiers as the query feature. Step 262: Based on the multi-similarity sample mining strategy, positive sample pairs are selected from the candidate positive samples, and negative sample pairs are selected from the candidate negative samples; Step 263: Determine the multi-similarity loss based on the first feature similarity between the predicted global retrieval descriptor and the positive sample pair, and the second feature similarity between the predicted global retrieval descriptor and the negative sample pair.

[0078] Specifically, Figure 3 This is a schematic diagram of the process for determining multiple similarity loss provided by the present invention, as shown below. Figure 3 As shown, firstly, the predicted global retrieval descriptor is used as the query feature. Based on the location identifiers corresponding to each training image in the training batch, candidate positive samples that belong to the same location identifier as the query feature and candidate negative samples that belong to different location identifiers as the query feature are determined.

[0079] Then, based on the multi-similarity sample mining strategy, positive sample pairs participating in loss calculation are selected from candidate positive samples, and negative sample pairs participating in loss calculation are selected from candidate negative samples.

[0080] In this context, positive sample pairs refer to matching pairs composed of descriptors extracted from two training images with the same location identifier, representing similar features of the same location; negative sample pairs refer to incorrect matching pairs composed of descriptors extracted from training images with different location identifiers, representing irrelevant features of different locations. The multi-similarity sample mining strategy refers to a strategy that, during the contrastive learning training process, dynamically selects difficult samples that contribute more to model training based on feature similarity distribution for candidate positive and candidate negative samples corresponding to the same query feature, in order to construct high-quality positive and negative sample pairs.

[0081] Finally, the multi-similarity loss can be determined based on the first feature similarity between the predicted global retrieval descriptor and positive sample pairs, and the second feature similarity between the predicted global retrieval descriptor and negative sample pairs.

[0082] The first feature similarity is used to reflect the feature inner product score between positive sample pairs; the second feature similarity is used to reflect the feature inner product score between negative sample pairs.

[0083] Here, the formula for the multi-similarity loss is as follows: ; in, Indicates multiple similarity loss; ,as well as Both represent feature similarity functions; Indicates the similarity of the first feature. Indicates the similarity of the second feature; This indicates a prediction of the global search descriptor. Indicates a positive sample pair; Indicates negative sample pairs; Indicates query characteristics, Representation and Query Features Positive samples from the same location that constitute a positive sample pair Representation and Query Features Negative samples from different locations that constitute negative sample pairs; The positive sample weight coefficient is preferably set to 1.0. The negative sample weighting coefficient is preferably set to 50. The similarity benchmark value is preferably set to 0.0; This indicates the predicted number of global search descriptors.

[0084] The method provided in this invention determines multiple similarity loss based on predicting the first feature similarity between the global retrieval descriptor and positive sample pairs, and predicting the second feature similarity between the global retrieval descriptor and negative sample pairs. This enables the global feature extraction network to effectively shorten the distance between positive sample pairs and widen the distance between negative sample pairs in the feature space, resulting in a better metric space distribution for the generated descriptors.

[0085] Based on the above embodiments, generating the suppression weights of the local features based on the correlation between the local features includes: Step 31: Generate a self-similarity matrix based on the correlation coefficient between any two spatial location features in the local features; Step 32: Perform linear mapping on the self-similar matrix based on preset learnable weights, and perform meanization and exponential transformation on the linearly mapped matrix to obtain the suppression weight of each spatial location feature.

[0086] Specifically, the repetition feature suppression module generates a self-similarity matrix based on the correlation coefficient between any two spatial location features in the local features.

[0087] Among them, spatial location features refer to the channel vector representation of a specific spatial coordinate position in the query image within the local features.

[0088] Here, the correlation coefficient is used to reflect the size of the vector inner product between different spatial location features within the same query image; the self-similarity matrix is ​​a two-dimensional measure matrix of the similarity between all local location features within the query image.

[0089] Specifically, for a local feature of size 768×L, after swapping the first and second dimensions of the local feature, performing a matrix multiplication operation with the original local feature yields a self-similarity matrix of size L×L. Sim 1. Where L is the total number of spatial locations, L = (H / 14) × (W / 14). Each element in the self-similarity matrix represents the correlation coefficient between any two spatial location features.

[0090] Then, the self-similar matrix can be linearly mapped based on the preset learnable weights, and mean-normalization and exponential transformation can be performed on the linearly mapped matrix to obtain the suppression weights of each spatial location feature.

[0091] Specifically, the self-similarity matrix Sim Multiplying 1 by the coefficient 2 and then subtracting 2 yields the second self-similarity matrix. Sim 2. Then, the second self-similar matrix... Sim 2 and the preset first learnable weights LW Multiply by 1 and add the second learnable weight LW 2. Perform a linear mapping to obtain the matrix after the linear mapping. Sim 3. The size is L×L. After processing with the Sigmoid function, a mean-based operation (average calculation) is performed on the second dimension to obtain the mean-based result of size L. Sim 4; Finally, the third learnable weight LW 3 represents the power exponent, which performs an exponential transformation on the mean-based result to generate the final suppression weight.

[0092] The method provided in this invention calculates the spatial self-similarity of local features and performs a nonlinear exponential transformation, giving a lower retention weight to repeating structures within an image, thus effectively reducing the interference of repeating textures within a single frame image.

[0093] Based on the above embodiments, determining the allocation response of the local features relative to each preset cluster center, and weighting the allocation response based on the suppression weights to obtain the weighted allocation response, includes: Step 41: Determine the initial clustering assignment response of the local features relative to each preset clustering center; Step 42: Normalize the suppression weights and use them as a gating control signal. Multiply the gating control signal element-wise with the initial clustering allocation response to obtain the weighted and adjusted allocation response.

[0094] Specifically, the gated local aggregation descriptor vector module determines the initial clustering assignment response of local features relative to each preset cluster center.

[0095] Specifically, by inputting local features into a preset convolution operation, the initial clustering assignment response corresponding to K preset cluster centers can be directly obtained. Preferably, the convolution kernel is 1×1, and the number of output channels is the number of cluster centers K, where K is preferably 64.

[0096] Then, the suppression weights can be normalized and used as the gating control signal. The gating control signal is then multiplied element-wise with the initial clustering allocation response to obtain the weighted allocation response.

[0097] Specifically, the suppression weights are first averaged, and this average is used as the divisor to perform element-wise division with the suppression weights themselves, followed by a Softmax operation for normalization. Then, a new dimension is added to the first dimension and replicated K times to obtain a gated control signal of size K×L. Element-wise multiplication is then performed between the gated control signal and the initial clustering allocation response after dimension rearrangement to obtain the weighted and adjusted allocation response.

[0098] The method provided in this invention uses the normalized suppression weight as a gating control signal to directly intervene in the response intensity of cluster assignment, thereby achieving soft dynamic weighting of the contribution of local features under different cluster centers, reducing the allocation ratio of duplicate features, and improving the cluster purity of important discriminative features.

[0099] Based on the above embodiments, step 130 includes: Step 130-1: Perform dimensionality compression on the image global retrieval descriptor to obtain a dimensionality-reduced global retrieval descriptor; Step 130-2: Determine the Euclidean distance between the dimensionality-reduced global retrieval descriptor and each of the database image descriptors; Step 130-3: Based on the Euclidean distance, sort the database images corresponding to each database image descriptor, and determine the position information recorded in the first sorted database image as the visual position recognition result.

[0100] Specifically, firstly, the global image retrieval descriptor can be dimensionally compressed to obtain a reduced-dimensional global retrieval descriptor.

[0101] In this embodiment, the query image is obtained by acquiring the environmental image of the robot's surroundings at the current moment through a vision sensor installed on the robot itself. I q The input is fed into a pre-trained global feature extraction network, which outputs a global image retrieval descriptor. Subsequently, after dimensionality compression using Principal Component Analysis (PCA), a dimensionality-reduced global retrieval descriptor is obtained. D q .

[0102] Similarly, the database images in the environmental map { DB 1, DB 2, ..., DB z Input a pre-trained global feature extraction network, output a global retrieval descriptor for the database images. fd 1, fd 2,..., fd z}, and after PCA dimensionality reduction, the pre-stored image descriptors of each database are obtained { Dd 1, Dd 2,..., Dd z},in z This represents the number of database images in the environmental map.

[0103] Furthermore, the Euclidean distance between the dimensionality-reduced global retrieval descriptor and the image descriptors of each database can be determined.

[0104] Finally, based on Euclidean distance, the database images corresponding to each database image descriptor can be sorted, and the location information recorded by the top-ranked database image can be determined as the visual location recognition result.

[0105] In this embodiment, the dimensionality-reduced global retrieval descriptor of the query image and the image descriptors of each database are calculated. Dd t The Euclidean distance between the database images is calculated, where 1 ≤ t ≤ z. Then, the database image corresponding to the database image descriptor with the smallest Euclidean distance is selected as the first matching element, and the pose of the database image recorded in the environment map is used as the visual location recognition result of the query image.

[0106] The method provided in this invention significantly reduces the computational complexity and storage space overhead in the feature matching process by compressing the dimensionality of the high-dimensional image global retrieval descriptor before performing distance metric matching, while ensuring the accuracy of location recognition, thereby significantly improving the real-time performance of robot online localization.

[0107] Based on the above embodiments, the extraction of local features of the query image includes: Step 51: Input the query image into a preset visual backbone network for feature mapping to obtain the initial feature tensor output by the visual backbone network; Step 52: Perform dimensional rearrangement on the initial feature tensor, and normalize the rearranged features along the channel dimension to obtain the local features.

[0108] Specifically, the local feature extraction module inputs the query image into a preset visual backbone network for feature mapping, and obtains the initial feature tensor output by the visual backbone network.

[0109] The local feature extraction module includes a visual backbone network, a first-dimensional rearrangement operation `Reshape_1`, and a first L2 normalization operation `L2Norm_1`. `Reshape_1` is obtained using the `reshape` operation from the `torch` library. `L2Norm_1` is used to normalize the input vector along the channel dimension using the L2 norm. For a query image input of size 3×H×W, after processing by the visual backbone network, an initial feature tensor is obtained. R 1. The dimensions are L×768, where L=(H / 14)×(W / 14).

[0110] Then, the initial feature tensor can be dimensionally rearranged, and the rearranged features can be normalized along the channel dimension to obtain local features. Specifically, Reshape_1 reshapes the initial feature tensor... R 1. Perform dimensional rearrangement to obtain the second local feature. R 2. The dimensions are 768 × (H / 14) × (W / 14). The second local feature... R 2. Input L2Norm_1 to obtain local features. R 3. The dimensions are 768×(H / 14)×(W / 14).

[0111] Based on the above embodiments, the step of performing residual aggregation on the local features based on the weighted and adjusted allocation response to generate the image global retrieval descriptor includes: Step 61: Determine the feature residuals between the local features and each of the preset cluster centers; Step 62: Based on the weighted and adjusted allocation response, the feature residuals are weighted and summed to obtain the local aggregation vector corresponding to each preset cluster center; Step 63: Determine the global image retrieval descriptor based on the local aggregation vectors corresponding to each preset cluster center.

[0112] Specifically, firstly, the suppression weights output by the repetition feature suppression module are... WB 1 is directly used as the first repetition suppression weight. WU 1. The dimension is L, where L = (H / 14) × (W / 14). WU 1. The result obtained after the preset averaging operation Mean_1 is used as the divisor, and... WU After processing by the division operation Div_1, the second repetition suppression weight is obtained by processing by Softmax_1. WU 2. The size is L.

[0113] Next, in WU Add a new dimension to the first dimension of 2, and repeat K times (K is the number of cluster centers, preferably 64), to obtain the third repetition suppression weight. WU 3. The dimensions are K×L.

[0114] At the same time, local features R 3. Perform convolution operation Conv_1 to obtain the first cluster assignment response. A 1. The dimensions are K×(H / 14)×(W / 14). A 1. The initial clustering assignment response is obtained by reshaping the data in 3 dimensions. A 2. The dimensions are K×L.

[0115] Subsequently, the third repetition suppression weight was adjusted. WU 3. Response to initial cluster assignment A 2. After performing element-wise multiplication using the Multi_1 operation, the weighted adjusted allocation response is obtained. A 3. The dimensions are K×L.

[0116] Finally, the weighted adjusted allocation response A 3. Local features R 3 and each preset cluster center Clu 1. Input them together into the residual aggregation module VLAD_Core_1 of the gated local aggregation descriptor vector module. Inside VLAD_Core_1, the feature residuals between local features and each preset cluster center are determined, and based on the weighted adjustment of the allocation response, the feature residuals are weighted and summed to obtain the local aggregation vector corresponding to each preset cluster center. AF 1. The dimensions are K×768.

[0117] Finally, the global image retrieval descriptor can be determined based on the local aggregation vectors corresponding to each preset cluster center. In specific implementation, the local aggregation vectors corresponding to each preset cluster center... AF 1. After further processing with L2Norm_2 (norm 2 normalization) and Reshape_4 (dimensional rearrangement and flattening), the second aggregated feature is obtained. AF 2. The size is 768K. This second aggregation feature AF 2. After further processing by the L2Norm_3 operation, the final image global retrieval descriptor is obtained.

[0118] Since K is preferably 64, the size of the output global image retrieval descriptor is 768K, or 49152.

[0119] The method provided in this invention aggregates feature residuals based on weighted adjustment of the allocation response, enabling the final generated global image retrieval descriptor to highly condense the core discriminative local details in the image and filter out useless background interference, thereby achieving an extremely excellent scene feature representation effect.

[0120] The visual position recognition device for robots provided by the present invention will be described below. The visual position recognition device for robots described below can be referred to in correspondence with the visual position recognition method for robots described above.

[0121] Based on any of the above embodiments, the present invention provides a visual position recognition device for a robot. Figure 4 This is a schematic diagram of the structure of the visual position recognition device for robots provided by the present invention, as shown below. Figure 4 As shown, the device includes: The acquisition unit 410 is used to acquire the environmental image currently collected by the robot as the query image; Input unit 420 is used to input the query image into a global feature extraction network to obtain an image global retrieval descriptor output by the global feature extraction network; The matching unit 430 is used to determine the visual position recognition result of the robot based on the global image retrieval descriptor and the image descriptors of each database pre-stored in the environment map. The global feature extraction network includes a local feature extraction module, a repetitive feature suppression module, and a gated local aggregation descriptor vector module; The local feature extraction module is used to extract local features of the query image; the duplicate feature suppression module is used to generate suppression weights for the local features based on the correlation between the local features; the gated local aggregation descriptor vector module is used to determine the allocation response of the local features relative to each preset cluster center, and to perform weighted adjustment on the allocation response based on the suppression weights to obtain a weighted allocation response, and to perform residual aggregation on the local features based on the weighted allocation response to generate the global image retrieval descriptor.

[0122] The apparatus provided in this invention generates suppression weights for local features based on the correlation between local features. These suppression weights are then used to weight and adjust the allocation responses of local features relative to each preset cluster center, resulting in a weighted allocation response. This weighted allocation response is then used to perform residual aggregation on the local features, generating a global image retrieval descriptor. This increases the proportion of location-discriminative features in the global image retrieval descriptor while suppressing non-discriminative repetitive background features, thereby enhancing the representation and differentiation capabilities of specific locations. This avoids the excessive accumulation of weakly discriminative local features such as repetitive textures and common background areas in the global image retrieval descriptor, ensuring the accuracy of the robot's visual location recognition results.

[0123] Based on any of the above embodiments, a training unit is further included, wherein the training unit specifically includes: A sample acquisition unit is used to acquire a training sample set, which includes location identifiers for multiple locations and multiple training images corresponding to each location identifier. Multiple training images labeled with the same location identifier are divided into a set of images for the same location. An input network unit is used to construct location samples based on the location identifier. Each location sample includes multiple images randomly sampled from the image set of the location corresponding to the same location identifier. Each training batch includes multiple location samples. The training images of the same training batch are input into an initial global feature extraction network to obtain the predicted global retrieval descriptor output by the initial global feature extraction network. The initial global feature extraction network includes an initial local feature extraction module, an initial duplicate feature suppression module, an initial gated local aggregation descriptor vector module, and a cross-image duplicate feature suppression branch. A single-frame suppression weight unit is determined, which is used to extract the predicted local features of each training image in the training batch based on the initial local feature extraction module, and generate the single-frame suppression weight corresponding to each training image based on the initial repetition feature suppression module. A cross-image suppression weight unit is determined, which is used to determine cross-image suppression weights based on the cross-image repetitive feature suppression branch to characterize the intensity of common background features between different training images in the training batch; The weighted aggregation unit is used to perform weighted aggregation of the predicted local features based on the fusion suppression weight by the initial gated local aggregation descriptor vector module to obtain the predicted global retrieval descriptor; the fusion suppression weight is obtained by multiplying the single-frame suppression weight and the cross-image suppression weight element-wise; A multi-similarity loss unit is determined to determine the multi-similarity loss based on the predicted global retrieval descriptor and the location identifier, and to jointly update the model parameters of the initial global feature extraction network based on the multi-similarity loss.

[0124] Based on any of the above embodiments, the determination of the cross-image suppression weight unit is specifically used for: Spatial adaptive average pooling is performed on the predicted local features to obtain low-resolution local features of each training image. Based on the low-resolution local features, the average appearance features of all images in the training batch are determined, and the similarity matrix between the low-resolution local features and the average appearance features is determined. Based on the statistical distribution of the similarity matrix, the global mean parameter and global standard deviation parameter of the similarity matrix are updated. Then, based on the updated global mean parameter, global standard deviation parameter and the similarity matrix, a standardized mapping and nonlinear activation process are performed to obtain the cross-image suppression weights.

[0125] Based on any of the above embodiments, the determination of the multiple similarity loss unit is specifically used for: Using the predicted global retrieval descriptor as the query feature, and based on the location identifiers corresponding to each training image in the training batch, candidate positive samples belonging to the same location identifier as the query feature and candidate negative samples belonging to different location identifiers as the query feature are determined. Based on the multi-similarity sample mining strategy, positive sample pairs are selected from the candidate positive samples, and negative sample pairs are selected from the candidate negative samples; The multi-similarity loss is determined based on the first feature similarity between the predicted global retrieval descriptor and the positive sample pair, and the second feature similarity between the predicted global retrieval descriptor and the negative sample pair.

[0126] Based on any of the above embodiments, the repetition feature suppression module is specifically used for: A self-similarity matrix is ​​generated based on the correlation coefficient between any two spatial location features in the local features. The self-similar matrix is ​​linearly mapped based on preset learnable weights, and mean-normalization and exponential transformation are performed on the linearly mapped matrix to obtain the suppression weight of each spatial location feature.

[0127] Based on any of the above embodiments, the gated local aggregation descriptor vector module is specifically used for: Determine the initial clustering assignment response of the local features relative to each preset cluster center; The suppression weights are normalized and used as a gating control signal. The gating control signal is then multiplied element-wise with the initial clustering allocation response to obtain the weighted and adjusted allocation response.

[0128] Based on any of the above embodiments, the matching unit 430 is specifically used for: The image global retrieval descriptor is dimensionality compressed to obtain a dimensionality-reduced global retrieval descriptor; Determine the Euclidean distance between the dimensionality-reduced global retrieval descriptor and each of the database image descriptors; Based on the Euclidean distance, the database images corresponding to each database image descriptor are sorted, and the position information recorded in the first sorted database image is determined as the visual position recognition result.

[0129] Based on any of the above embodiments, the local feature extraction module is specifically used for: The query image is input into a preset visual backbone network for feature mapping to obtain the initial feature tensor output by the visual backbone network. The initial feature tensor is rearranged in dimensions, and the rearranged features are normalized along the channel dimension to obtain the local features.

[0130] Based on any of the above embodiments, the gated local aggregation descriptor vector module is specifically used for: Determine the feature residuals between the local features and each of the preset cluster centers; Based on the weighted adjustment of the allocation response, the feature residuals are weighted and summed to obtain the local aggregation vector corresponding to each preset cluster center; The global image retrieval descriptor is determined based on the local aggregation vectors corresponding to each preset cluster center.

[0131] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communications bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other through the communications bus 540. The processor 510 can call logic instructions in the memory 530 to execute a visual position recognition method for the robot. This method includes: acquiring a currently collected environmental image as a query image; inputting the query image into a global feature extraction network to obtain an image global retrieval descriptor output by the global feature extraction network; determining the robot's visual position recognition result based on the image global retrieval descriptor and pre-stored database image descriptors in the environmental map; the global feature extraction network includes a local feature extraction module, a repetition feature suppression module, and a gated local aggregation descriptor vector module; the local feature extraction module is used to extract local features from the query image; the repetition feature suppression module is used to generate suppression weights for the local features based on the correlation between the local features; the gated local aggregation descriptor vector module is used to determine the allocation response of the local features relative to each preset cluster center, and to perform weighted adjustment on the allocation response based on the suppression weights to obtain a weighted allocation response; and to perform residual aggregation on the local features based on the weighted allocation response to generate the image global retrieval descriptor.

[0132] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0133] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the robot visual position recognition method provided by the above methods. The method includes: acquiring an environmental image currently collected by the robot as a query image; inputting the query image into a global feature extraction network to obtain an image global retrieval descriptor output by the global feature extraction network; and determining the robot's visual position recognition result based on the image global retrieval descriptor and pre-stored database image descriptors in the environmental map. The global feature extraction network includes a local feature extraction module, a repetition feature suppression module, and a gated local aggregation descriptor vector module. The local feature extraction module is used to extract local features of the query image. The repetition feature suppression module is used to generate suppression weights for the local features based on the correlation between the local features. The gated local aggregation descriptor vector module is used to determine the allocation response of the local features relative to each preset cluster center, and to perform weighted adjustment on the allocation response based on the suppression weights to obtain a weighted allocation response. Based on the weighted allocation response, residual aggregation is performed on the local features to generate the global image retrieval descriptor.

[0134] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a visual position recognition method for a robot provided by the methods described above. This method includes: acquiring a currently collected environmental image of the robot as a query image; inputting the query image into a global feature extraction network to obtain an image global retrieval descriptor output by the global feature extraction network; determining the visual position recognition result of the robot based on the image global retrieval descriptor and pre-stored database image descriptors in the environmental map; the global feature extraction network includes a local feature extraction module, a repetition feature suppression module, and a gated local aggregation descriptor vector module; the local feature extraction module is used to extract local features from the query image; the repetition feature suppression module is used to generate suppression weights for the local features based on the correlation between the local features; the gated local aggregation descriptor vector module is used to determine the allocation response of the local features relative to each preset cluster center, and to perform weighted adjustment on the allocation response based on the suppression weights to obtain a weighted allocation response; and to perform residual aggregation on the local features based on the weighted allocation response to generate the image global retrieval descriptor.

[0135] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0136] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for visual position recognition of a robot, characterized in that, include: Use the environmental image currently collected by the robot as the query image; The query image is input into a global feature extraction network to obtain the global image retrieval descriptor output by the global feature extraction network; Based on the global image retrieval descriptor and the image descriptors of each database pre-stored in the environment map, the visual position recognition result of the robot is determined; The global feature extraction network includes a local feature extraction module, a repetitive feature suppression module, and a gated local aggregation descriptor vector module; The local feature extraction module is used to extract local features of the query image; the duplicate feature suppression module is used to generate suppression weights for the local features based on the correlation between the local features; the gated local aggregation descriptor vector module is used to determine the allocation response of the local features relative to each preset cluster center, and to perform weighted adjustment on the allocation response based on the suppression weights to obtain a weighted allocation response, and to perform residual aggregation on the local features based on the weighted allocation response to generate the global image retrieval descriptor.

2. The visual position recognition method for a robot according to claim 1, characterized in that, The global feature extraction network is obtained by iteratively executing the following steps until a preset iteration termination condition is met: Obtain a training sample set, which includes location identifiers for multiple locations and multiple training images corresponding to each location identifier. Divide multiple training images labeled with the same location identifier into a set of images for the same location. Location samples are constructed using the location identifier as a unit, and each location sample includes multiple images randomly sampled from the image set of the location corresponding to the same location identifier; each training batch includes multiple location samples; the training images of the same training batch are input into an initial global feature extraction network to obtain the predicted global retrieval descriptor output by the initial global feature extraction network; the initial global feature extraction network includes an initial local feature extraction module, an initial duplicate feature suppression module, an initial gated local aggregation descriptor vector module, and a cross-image duplicate feature suppression branch; Based on the initial local feature extraction module, the predicted local features of each training image in the training batch are extracted, and based on the initial repetition feature suppression module, the single-frame suppression weights corresponding to each training image are generated. Based on the cross-image repetition feature suppression branch, cross-image suppression weights are determined to characterize the intensity of common background features among different training images within the training batch; The predicted global retrieval descriptor is obtained by weighting and aggregating the predicted local features based on the fusion suppression weights by the initial gated local aggregation descriptor vector module; the fusion suppression weights are obtained by element-wise multiplication of the single-frame suppression weights and the cross-image suppression weights. Based on the predicted global retrieval descriptor and the location identifier, a multi-similarity loss is determined, and the model parameters of the initial global feature extraction network are jointly updated based on the multi-similarity loss.

3. The visual position recognition method for a robot according to claim 2, characterized in that, The step of determining the cross-image suppression weights based on the cross-image repetition feature suppression branch to characterize the intensity of common background features among different training images within the training batch includes: Spatial adaptive average pooling is performed on the predicted local features to obtain low-resolution local features of each training image. Based on the low-resolution local features, the average appearance features of all images in the training batch are determined, and the similarity matrix between the low-resolution local features and the average appearance features is determined. Based on the statistical distribution of the similarity matrix, the global mean parameter and global standard deviation parameter of the similarity matrix are updated. Then, based on the updated global mean parameter, global standard deviation parameter and the similarity matrix, a standardized mapping and nonlinear activation process are performed to obtain the cross-image suppression weights.

4. The visual position recognition method for a robot according to claim 2, characterized in that, The determination of multiple similarity loss based on the predicted global retrieval descriptor and the location identifier includes: Using the predicted global retrieval descriptor as the query feature, and based on the location identifiers corresponding to each training image in the training batch, candidate positive samples belonging to the same location identifier as the query feature and candidate negative samples belonging to different location identifiers as the query feature are determined. Based on the multi-similarity sample mining strategy, positive sample pairs are selected from the candidate positive samples, and negative sample pairs are selected from the candidate negative samples; The multi-similarity loss is determined based on the first feature similarity between the predicted global retrieval descriptor and the positive sample pair, and the second feature similarity between the predicted global retrieval descriptor and the negative sample pair.

5. The visual position recognition method for a robot according to any one of claims 1 to 4, characterized in that, The step of generating suppression weights for the local features based on the correlation between the local features includes: A self-similarity matrix is ​​generated based on the correlation coefficient between any two spatial location features in the local features. The self-similar matrix is ​​linearly mapped based on preset learnable weights, and mean-normalization and exponential transformation are performed on the linearly mapped matrix to obtain the suppression weight of each spatial location feature.

6. The visual position recognition method for a robot according to any one of claims 1 to 4, characterized in that, The step of determining the allocation response of the local features relative to each preset cluster center, and weighting the allocation response based on the suppression weight to obtain the weighted allocation response, includes: Determine the initial clustering assignment response of the local features relative to each preset cluster center; The suppression weights are normalized and used as a gating control signal. The gating control signal is then multiplied element-wise with the initial clustering allocation response to obtain the weighted and adjusted allocation response.

7. The visual position recognition method for a robot according to any one of claims 1 to 4, characterized in that, The step of determining the robot's visual position recognition result based on the global image retrieval descriptor and pre-stored database image descriptors in the environment map includes: The image global retrieval descriptor is dimensionality compressed to obtain a dimensionality-reduced global retrieval descriptor; Determine the Euclidean distance between the dimensionality-reduced global retrieval descriptor and each of the database image descriptors; Based on the Euclidean distance, the database images corresponding to each database image descriptor are sorted, and the position information recorded in the first sorted database image is determined as the visual position recognition result.

8. The visual position recognition method for a robot according to any one of claims 1 to 4, characterized in that, The extraction of local features from the query image includes: The query image is input into a preset visual backbone network for feature mapping to obtain the initial feature tensor output by the visual backbone network. The initial feature tensor is rearranged in dimensions, and the rearranged features are normalized along the channel dimension to obtain the local features.

9. The visual position recognition method for a robot according to any one of claims 1 to 4, characterized in that, The step of performing residual aggregation on the local features based on the weighted and adjusted allocation response to generate the global image retrieval descriptor includes: Determine the feature residuals between the local features and each of the preset cluster centers; Based on the weighted adjustment of the allocation response, the feature residuals are weighted and summed to obtain the local aggregation vector corresponding to each preset cluster center; The global image retrieval descriptor is determined based on the local aggregation vectors corresponding to each preset cluster center.

10. A visual position recognition device for a robot, characterized in that, include: The acquisition unit is used to acquire the environmental image currently collected by the robot as the query image; The input unit is used to input the query image into the global feature extraction network to obtain the global image retrieval descriptor output by the global feature extraction network. The matching unit is used to determine the visual position recognition result of the robot based on the global image retrieval descriptor and the image descriptors of each database pre-stored in the environment map. The global feature extraction network includes a local feature extraction module, a repetitive feature suppression module, and a gated local aggregation descriptor vector module; The local feature extraction module is used to extract local features of the query image; the duplicate feature suppression module is used to generate suppression weights for the local features based on the correlation between the local features; the gated local aggregation descriptor vector module is used to determine the allocation response of the local features relative to each preset cluster center, and to perform weighted adjustment on the allocation response based on the suppression weights to obtain a weighted allocation response, and to perform residual aggregation on the local features based on the weighted allocation response to generate the global image retrieval descriptor.

Images