Loop detection method and device based on weighted local features and global description features

Abstract

The application provides a loop detection method and device based on weighted local features and global description features, which inputs a current frame image into a pre-trained feature extraction model to obtain global description features and weighted local features, and then searches for candidate loop images according to the global description features of the current frame image and the global description features of each historical frame image in a graph database, performs feature matching on the weighted local features of the candidate loop images and the current frame image, checks the geometric consistency between the candidate loop images and the current frame image according to the matching features to obtain a geometric consistency score, and finally determines a final loop image according to the geometric consistency score to realize loop detection. The method can effectively identify static salient regions, reduce the interference of dynamic objects and non-discriminative regions on loop detection, significantly improve the accuracy of loop detection, and is especially suitable for complex environments containing dynamic objects and non-discriminative regions.

Description

Technical Field

[0001] The present application relates to the technical field of loop closure detection, and in particular to a loop closure detection method and device based on weighted local features and global description features. Background Art

[0002] In the field of autonomous driving, loop closure detection (LCD) is a key technology to maintain high-precision positioning and Figure 1 However, due to the interference of viewpoint changes, appearance changes and dynamic objects, traditional loop closure detection methods face many challenges.

[0003] Traditional Bag-of-Visual-Words (BoVW)-based methods rely on handcrafted feature descriptors such as SURF and SIFT, which have limited performance in complex environments. In recent years, deep learning-based convolutional neural networks have been widely used in loop closure detection. However, most existing methods lack effective strategies to suppress interference from dynamic objects and non-discriminative regions. Furthermore, images from adjacent moments can interfere with loop closure detection. Summary of the Invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, the present application aims to provide a loop detection method and device based on weighted local features and global description features to solve the problem of low loop detection accuracy caused by dynamic objects and non-discriminative areas in the related art, and to improve the accuracy of loop detection.

[0005] The present invention provides a loop detection method based on weighted local features and global description features, which includes:

[0006] Inputting the current frame image into a pre-trained feature extraction model to obtain global description features and weighted local features of the current frame image;

[0007] Determine, based on the global description features of the current frame image and the global description features of each historical frame image in the graph database, a plurality of candidate loop images corresponding to the current frame image, wherein the graph database is used to store the global description features of the historical frame images popped out by a first-in-first-out queue, and the first-in-first-out queue is used to maintain m adjacent historical frame images before the current frame;

[0008] For each of the candidate loop closure images, perform feature matching on weighted local features of the candidate loop closure image and the current frame image, and determine a geometric consistency score corresponding to the candidate loop closure image based on the matched feature pairs, wherein the geometric consistency score is used to reflect the geometric consistency between the candidate loop closure image and the current frame image;

[0009] Based on the geometric consistency scores corresponding to the candidate loop closure images, a final loop closure image is determined.

[0010] Optionally, the current frame image is input into a pre-trained feature extraction model to obtain global description features and weighted local features of the current frame image, including:

[0011] Inputting the current frame image into a pre-trained feature extraction model, wherein the feature extraction model includes a backbone network, a polarized self-attention module, a first dimensionality reduction module, an aggregation module, and a second dimensionality reduction module;

[0012] Extracting initial local features from the current frame image based on the backbone network;

[0013] Performing weighted processing on each feature point in the initial local features according to the polarized self-attention module, and performing dimensionality reduction on the result output by the polarized self-attention module according to the first dimensionality reduction module to obtain weighted local features;

[0014] The initial local features are aggregated into compact features according to the aggregation module, and the dimension of the result output by the aggregation module is reduced according to the second dimension reduction module to obtain global description features.

[0015] Optionally, the polarized self-attention module includes a channel-domain self-attention unit and a spatial-domain self-attention unit, and weighted processing is performed on each feature point in the initial local features according to the polarized self-attention module, including:

[0016] Performing channel-domain selection on the initial local features according to the channel-domain self-attention unit to obtain channel-domain weighted features;

[0017] The channel domain weighted features are subjected to spatial domain selection according to the spatial domain self-attention unit to obtain spatial channel domain weighted features.

[0018] Optionally, the channel domain self-attention unit includes a channel importance analysis unit, a channel content selection unit, a first matrix dot product unit, a channel dimension increase unit, a first activation function unit, and a channel domain multiplication unit. Channel domain selection is performed on the initial local features according to the channel domain self-attention unit to obtain channel domain weighted features, including:

[0019] Analyzing the importance of each channel in the initial local features according to the channel importance analysis unit to obtain a channel attention weight;

[0020] Determining the channel content selected in the initial local feature according to the channel content selection unit to obtain a channel selected feature;

[0021] Performing a dot product operation on the channel attention weight and the channel selected feature based on the first matrix dot product unit, and performing channel dimension increasing processing on the dot product operation result through the channel dimension increasing unit;

[0022] According to the first activation function unit, the channel dimensionality increase result is limited to a set range to obtain the channel domain feature weight, and the channel domain feature weight is multiplied by the initial local feature through the channel domain multiplication unit to obtain the channel domain weighted feature.

[0023] Optionally, the spatial domain self-attention unit includes a spatial criticality analysis unit, a spatial content retention unit, a second matrix dot product unit, a matrix dimension change unit, a second activation function unit, and a spatial domain multiplication unit. The spatial domain self-attention unit performs spatial domain selection on the channel domain weighted features to obtain spatial channel domain weighted features, including:

[0024] Analyzing the criticality of each spatial position in the channel-domain weighted features according to the spatial criticality analysis unit to obtain a spatial attention weight;

[0025] retaining spatial position information in the channel domain weighted features according to the spatial content retaining unit to obtain a spatially retained feature;

[0026] Performing a dot product operation on the spatial attention weight and the spatial retention feature based on the second matrix dot product unit, and performing matrix dimension conversion processing on the dot product operation result through the matrix dimension conversion unit;

[0027] According to the second activation function unit, the matrix dimension change result is limited to a set range to obtain the spatial domain feature weight, and the spatial domain feature weight is multiplied by the channel domain weighted feature through the spatial domain multiplication unit to obtain the spatial channel domain weighted feature.

[0028] Optionally, determining a plurality of candidate loop images corresponding to the current frame image based on the global description features of the current frame image and the global description features of each historical frame image in the graph database includes:

[0029] Determining similarities between the current frame image and each of the historical frame images based on a global description feature of the current frame image and a global description feature of each of the historical frame images in the graph database;

[0030] A preset number of historical frame images are selected from the graph database in descending order of similarity as candidate loop images.

[0031] Optionally, determining the similarity between the current frame image and each of the historical frame images based on the global description features of the current frame image and the global description features of each of the historical frame images in the graph database includes:

[0032] The top layer of the graph database is used as the current layer, and the global description feature of the current frame image is used as the current node. The distance between the current node and each node in the current layer is calculated, wherein the graph database stores each node in a hierarchical manner, and the node represents the global description feature of the historical frame image. The higher the layer number, the fewer the number of nodes. The distance represents similarity.

[0033] Determining a search start node among the nodes in the current layer based on the distance, and determining an intermediate position based on the distance between the nodes in the current layer;

[0034] Taking the next layer of the current layer as a new current layer, and determining the search cutoff node of the current layer according to the intermediate position;

[0035] Starting from the search start node of the current layer, calculating the distance between the current node and each node in the current layer until calculating the distance between the current node and the search end node;

[0036] Return to the step of determining a search start node among the nodes of the current layer based on the distance until the bottom layer of the graph database is reached.

[0037] Optionally, determining a geometric consistency score corresponding to the candidate loop closure image based on the matching feature pair includes:

[0038] Determining the number of matching feature pairs and the spatial offset of each matching feature pair in the horizontal and vertical directions;

[0039] A geometric consistency score is determined according to the number of the matching feature pairs and the spatial offset of each matching feature pair in the horizontal and vertical directions.

[0040] Optionally, after determining the final loop closure image based on the geometric consistency scores corresponding to the candidate loop closure images, the method further includes:

[0041] Determine whether the corresponding final loop image is detected in multiple consecutive moments before or after the current frame;

[0042] If not, the final loop image corresponding to the current frame image is deleted.

[0043] An embodiment of the present application further provides an electronic device, comprising:

[0044] processor and memory;

[0045] The processor is used to execute the steps of the loop detection method based on weighted local features and global description features provided in any embodiment of the present application by calling the program or instructions stored in the memory.

[0046] An embodiment of the present application also provides a computer-readable storage medium, which stores a program or instruction, and the program or instruction enables a computer to execute the steps of the loop detection method based on weighted local features and global description features provided in any embodiment of the present application.

[0047] In summary, the present application proposes a loop detection method based on weighted local features and global description features. The method inputs the current frame image into a pre-trained feature extraction model to obtain the global description features and weighted local features of the current frame image. Then, based on the global description features of the current frame image and the global description features of each historical frame image in the graph database, multiple candidate loop images corresponding to the current frame image are searched in the graph database. After the candidate loop images are screened out, for each candidate loop image, feature matching is performed on the weighted local features of the candidate loop image and the current frame image. The geometric consistency between the candidate loop image and the current frame image is verified based on the matching features to obtain a geometric consistency score. Finally, the final loop image is determined according to the geometric consistency scores corresponding to each candidate loop image, and loop detection is realized. This method effectively identifies static salient areas through weighted local features, which can reduce the interference of dynamic objects and non-discriminative areas on loop detection. Moreover, through the close coupling of weighted local features and global description features, the accuracy of loop detection is significantly improved, especially for complex environments containing dynamic objects and non-discriminative areas. In addition, this method maintains the adjacent m historical frame images before the current frame through a first-in-first-out queue, which can avoid the historical frame images adjacent to the current frame from participating in the candidate loop frame search, thereby avoiding false positive loops caused by interference from adjacent images and further improving the accuracy of loop detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the specific implementation methods or the description of the prior art. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0049] Figure 1 This is a flow chart of a loop detection method based on weighted local features and global description features provided in an embodiment of the present application;

[0050] Figure 2is a schematic diagram of a feature extraction model provided in an embodiment of the present application;

[0051] Figure 3 is a schematic diagram of a polarized self-attention module provided in an embodiment of the present application;

[0052] Figure 4 This is a schematic diagram of a search for a graph database provided in an embodiment of the present application;

[0053] Figure 5 This is a schematic diagram of a loop detection process provided by an embodiment of the present application;

[0054] Figure 6 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the invention are shown in the accompanying drawings.

[0056] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0057] As mentioned in the background technology, in response to the problems in the prior art, this application proposes a loop detection method based on weighted local features and global description features. Figure 1 This is a flow chart of a loop detection method based on weighted local features and global description features provided in an embodiment of the present application. Figure 1 The loop detection method based on weighted local features and global description features specifically includes:

[0058] S110 , inputting the current frame image into a pre-trained feature extraction model to obtain global description features and weighted local features of the current frame image.

[0059] The feature extraction model may be a neural network model for extracting global descriptive features and weighted local features from an image. The feature extraction model may be trained using sample image data.

[0060] In an embodiment of the present application, the global description feature may be a feature that describes the global characteristics of the image, and the weighted local feature may be a feature that describes the local characteristics of the image and weights the local feature points to emphasize static significant features and reduce interference from dynamic objects and non-recognizable areas.

[0061] Specifically, the feature extraction model can be composed of a backbone network, a structure for extracting global descriptive features, and a structure for extracting weighted local features. The backbone network initially extracts local features from the current frame image, and then the corresponding global descriptive features and weighted local features are obtained through the structure for extracting global descriptive features and the structure for extracting weighted local features, respectively.

[0062] In some embodiments, the current frame image is input into a pre-trained feature extraction model to obtain global descriptive features and weighted local features of the current frame image, including the following steps:

[0063] Step 11: Input the current frame image into a pre-trained feature extraction model, wherein the feature extraction model includes a backbone network, a polarized self-attention module, a first dimensionality reduction module, an aggregation module, and a second dimensionality reduction module;

[0064] Step 12: extracting initial local features from the current frame image based on the backbone network;

[0065] Step 13: weighting each feature point in the initial local features according to the polarized self-attention module, and reducing the dimension of the output of the polarized self-attention module according to the first dimensionality reduction module to obtain weighted local features;

[0066] Step 14: Aggregate the initial local features into compact features according to the aggregation module, and reduce the dimension of the output of the aggregation module according to the second dimensionality reduction module to obtain global description features.

[0067] Among them, in step 11, the current frame image can be input into a pre-trained feature extraction model, first passing through the backbone network therein, and then the output of the backbone network passes through the polarization self-attention module and the aggregation module respectively, the output of the polarization self-attention module passes through the first dimensionality reduction module to obtain weighted local features, and the output of the aggregation module passes through the second dimensionality reduction module to obtain global description features.

[0068] In an embodiment of the present application, the backbone network can be composed of multiple convolutional blocks, such as the first 12 convolutional blocks of MobileNetV3. Specifically, each convolutional block can be composed of an inverse residual structure, that is, a "dimensionality increase-depthwise separable convolution-dimensionality reduction" design.

[0069] In step 12, after the current frame image enters the backbone network, it first enters the first convolution block. For each convolution block, 1×1 point-by-point convolution can be used to increase the dimension of the input to increase the number of feature channels. Then, 3×3 depth-separable convolution can be used to extract features, and the output can be sent to the squeeze and excitation structure to strengthen the channel expression. Finally, 1×1 point-by-point convolution can be used to reduce the dimension, and residual connections are established at both ends to obtain the output of the convolution block. Of course, after the current frame image is processed by multiple convolution blocks in sequence, the last convolution block outputs the initial local features. .

[0070] It should be noted that the use of the lightweight MobileNetV3 backbone network to encode the initial local features of the current frame image exponentially reduces the number of parameters while maintaining accuracy compared to other heavyweight convolutional neural networks (such as ResNet and VGG). Furthermore, compared to other lightweight convolutional neural networks (such as NasNet and ShuffleNet), this achieves higher accuracy.

[0071] Furthermore, the initial local features output by the backbone network are fed into the polarized self-attention module and the aggregation module. In step 13, the polarized self-attention module considers that not all dense local features are suitable for loop closure detection tasks. Therefore, it can introduce a polarized self-attention mechanism to guide feature selection, weighting the initial local features to improve the channel and spatial expression of the features, making them more suitable for loop closure detection in dynamic and non-identifiable scenarios.

[0072] After the polarized self-attention module performs weighted processing on each feature point in the initial local features, the output of the polarized self-attention module can also be reduced in dimensionality through a first dimensionality reduction module, such as PCA (Principal Component Analysis), to project high-dimensional data into a low-dimensional space through linear transformation, thereby reducing feature complexity and subsequent calculations to obtain weighted local features.

[0073] Furthermore, in step 14, the aggregation module may aggregate the initial local features into compact features. For example, each feature point in the initial local features may be assigned to each cluster center through a soft assignment function.

[0074] For example, the aggregation module can use NetVLAD (Net Vector of Locally Aggregated Descriptors). First, the initial local features can be clustered to obtain cluster centers. Then, the residual vector between each feature point in the initial local features and its nearest cluster center is calculated, and these residuals are aggregated.

[0075] After aggregating the initial local features into compact features, the second dimensionality reduction module can also perform dimensionality reduction on the output of the aggregation module. This linear transformation projects the high-dimensional data into a low-dimensional space, reducing feature complexity and subsequent computational effort to obtain global descriptive features. In addition to dimensionality reduction, the second dimensionality reduction module can also perform whitening on the reduced results to accelerate model convergence and improve dimensionality reduction, ultimately yielding the final global descriptive features.

[0076] Figure 2 is a schematic diagram of a feature extraction model provided in an embodiment of the present application, such as Figure 2 As shown in the figure, after the current frame image enters the backbone network, the initial local features can be obtained. The initial local features enter the polarization self-attention module and the aggregation module respectively. The polarization self-attention module weights the initial local features (such as weighting through the channel domain self-attention unit, and then weighting through the spatial domain self-attention unit), and then performs PCA dimensionality reduction to obtain weighted local features; the aggregation module can aggregate the initial local features and obtain global description features through PCA dimensionality reduction.

[0077] Through steps 11 to 14 above, a lightweight MobileNetV3 network can be used, which reduces the number of parameters and computation time and improves the feature extraction speed. In addition, the polarized self-attention module can effectively identify static salient areas and reduce the interference of dynamic objects and non-discriminative areas on loop detection. This can significantly improve the accuracy of loop detection, especially in complex environments.

[0078] In a specific embodiment, the polarized self-attention module includes a channel domain self-attention unit and a spatial domain self-attention unit, and weighted processing is performed on each feature point in the initial local feature according to the polarized self-attention module, including the following steps:

[0079] Step 131: Perform channel-domain selection on the initial local features according to the channel-domain self-attention unit to obtain channel-domain weighted features;

[0080] Step 132: Perform spatial domain selection on the channel domain weighted features according to the spatial domain self-attention unit to obtain spatial channel domain weighted features.

[0081] The polarized self-attention module can be composed of a channel-domain self-attention unit and a spatial-domain self-attention unit, which are connected in series.

[0082] In step 131, the channel-domain self-attention unit may perform channel-domain selection on the initial local features, that is, determine the feature weights of the channel domain to obtain channel-domain weighted features.

[0083] Furthermore, considering that the feature points in the static area require more global context information than the feature points in the dynamic area, fine-grained spatial domain feature selection is required after coarse-grained channel domain selection. The spatial domain self-attention unit is used to perform spatial domain selection on the channel domain weighted features, that is, the spatial channel domain weighted features are obtained through the guidance of spatial domain self-attention.

[0084] In an embodiment of the present application, through the channel domain self-attention unit and the spatial domain self-attention unit, coarse-grained channel domain feature selection and fine-grained spatial domain feature selection are performed in turn, and features guided by channel and spatial domain self-attention can be obtained, which can effectively identify static salient areas in the image and reduce the weights of dynamic objects and non-discriminative areas.

[0085] In one example, the channel domain self-attention unit includes a channel importance analysis unit, a channel content selection unit, a first matrix dot product unit, a channel dimension increase unit, a first activation function unit, and a channel domain multiplication unit. The initial local features are selected in the channel domain according to the channel domain self-attention unit to obtain channel domain weighted features, including the following steps:

[0086] Step 1311: Analyze the importance of each channel in the initial local features using the channel importance analysis unit to obtain a channel attention weight;

[0087] Step 1312: determining the channel content selected in the initial local features according to the channel content selection unit to obtain a channel selected feature;

[0088] Step 1313: Perform a dot product operation on the channel attention weight and the channel selected feature based on the first matrix dot product unit, and perform channel dimension increasing processing on the dot product operation result through the channel dimension increasing unit;

[0089] Step 1314: According to the first activation function unit, the channel dimensionality increase result is limited to a set range to obtain the channel domain feature weight, and the channel domain feature weight is multiplied by the initial local feature through the channel domain multiplication unit to obtain the channel domain weighted feature.

[0090] The channel importance analysis unit can be used to analyze important channels and can be composed of a 1×1 convolution, a matrix dimensioning unit, and an activation function (such as Softmax). The channel content selection unit can be used to select channel content and can be composed of a 1×1 convolution and a matrix dimensioning unit. The channel dimensioning unit can be used to increase the dimension of weight channels and can be composed of a 1×1 convolution and a layer normalization unit.

[0091] Specifically, the initial local features can be fed into the channel importance analysis unit and the channel content selection unit, respectively. In step 1311, the channel importance analysis unit can perform a 1×1 convolution on the initial local features, transforming the channel dimension from C to C / 2 after the convolution. The channel dimension is then increased to C by the matrix dimension conversion unit, and the dynamic range of the output result of the matrix dimension conversion unit is enhanced by Softmax to obtain the channel attention weight.

[0092] In step 1312, the channel content selection unit may perform 1×1 convolution on the initial local features, and the channel dimension after the convolution is transformed from C to C / 2. The channel dimension is maintained at C / 2 through the matrix dimension conversion unit to obtain the channel selected features.

[0093] Furthermore, in step 1313, the channel attention weight and the channel selected feature can be subjected to a dot product operation, i.e., matrix multiplication, through the first matrix dot product unit, and then the channel dimension increasing unit performs channel dimension increasing processing on the dot product operation result through a 1×1 convolution and layer normalization unit to increase the channel dimension to C.

[0094] Finally, in step 1314, the channel dimensionality increase result can be limited to a set range through the first activation function unit (such as the Sigmod function) to limit the feature weight to the range of 0~1, thereby obtaining the channel domain feature weight; and then, through the channel domain multiplication unit, the channel domain feature weight is multiplied by the initial local feature to obtain the channel domain weighted feature.

[0095] Figure 3 Schematic diagram of a polarized self-attention module provided in an embodiment of the present application. Figure 3 As shown in the figure, the initial local features can first enter the channel domain self-attention unit. In the channel domain self-attention unit, the 1×1 convolution and matrix dimension change at the top are the channel content selection unit, and the 1×1 convolution, matrix dimension change and activation function (Softmax function) at the bottom are the channel importance analysis unit. Represents the first matrix dot product unit, located at The subsequent 1×1 convolution and layer normalization units are channel dimension-raising units, and the activation function after the channel dimension-raising unit is the first activation function unit (Sigmod function). Represents channel-domain multiplication unit.

[0096] refer to Figure 3The initial local features can first undergo 1×1 convolution and matrix dimension transformation in the channel content selection unit to obtain channel-selected features, and then undergo 1×1 convolution, matrix dimension transformation, and activation function in the channel importance analysis unit to obtain channel attention weights. Furthermore, the channel attention weights and channel-selected features undergo matrix multiplication and enter the channel dimension enhancement unit. They undergo 1×1 convolution and layer normalization in the channel dimension enhancement unit for channel dimension enhancement, and then pass through the first activation function unit (Sigmod function) to obtain channel-domain feature weights. The channel-domain multiplication unit performs matrix multiplication on the channel-domain feature weights and the initial local features to obtain channel-domain weighted features. The channel-domain weighted features subsequently enter the spatial-domain self-attention unit.

[0097] For example, the output of the channel-domain self-attention unit can be expressed as:

[0098] ,

[0099] Where, is the channel domain weighted feature output by the channel domain self-attention unit, is the initial local feature, The dimension representing the initial local features is , is the number of channels, and are the height (i.e., the number of pixel rows) and width (i.e., the number of pixel columns), is the channel-domain multiplication operator. is the channel domain feature weight, It can be calculated by the following formula:

[0100] ;

[0101] Where, 、 Represents a 1×1 convolution of the initial local features. and Reshape operator representing tensor, i.e. matrix dimension change, is the dot product operation of the matrix, Represents 1×1 convolution and layer normalization of the result of the dot product operation in the brackets. Represents the Softmax operator, which uses the Softmax function to perform nonlinear transformation on the matrix dimension change result in the brackets. Represents the Sigmod function, that is, the Sigmod function is used to process the layer normalization results in the brackets.

[0102] In the above steps 1311-1314, the initial local features pass through the channel importance analysis unit and can be used to calculate the attention weight. After passing through the channel content selection unit, they can be used for subsequent weighted aggregation feature values ​​(i.e., reconstructing the features after multiplying with the attention weight), which can reduce information interference and improve attention resolution. Compared with using the same two processing branches to obtain the same channel attention weights and channel selected features, it can avoid the attention map from being too averaged and losing the ability to distinguish differences. In an embodiment of the present application, separately processing the channel attention weights and channel selected features can enable the channel importance analysis unit to focus on extracting attention information, and the channel content selection unit to focus on retaining input information. The two complement each other and enhance the expression ability, control ability and flexibility of the polarized self-attention mechanism.

[0103] In one example, the spatial domain self-attention unit includes a spatial criticality analysis unit, a spatial content retention unit, a second matrix dot product unit, a matrix dimension change unit, a second activation function unit, and a spatial domain multiplication unit. The spatial domain self-attention unit performs spatial domain selection on the channel domain weighted features to obtain spatial channel domain weighted features, including the following steps:

[0104] Step 1321: Analyze the criticality of each spatial position in the channel-domain weighted features using the spatial criticality analysis unit to obtain a spatial attention weight;

[0105] Step 1322: retaining spatial position information in the channel domain weighted features according to the spatial content retaining unit to obtain spatially retained features;

[0106] Step 1323: Perform a dot product operation on the spatial attention weight and the spatial retention feature based on the second matrix dot product unit, and perform matrix dimension conversion processing on the dot product operation result through the matrix dimension conversion unit;

[0107] Step 1324: According to the second activation function unit, the matrix dimension transformation result is limited to a set range to obtain the spatial domain feature weight, and the spatial domain feature weight is multiplied by the channel domain weighted feature through the spatial domain multiplication unit to obtain the spatial channel domain weighted feature.

[0108] The spatial criticality analysis unit can be used to analyze key spatial locations and can be composed of a 1×1 convolution, a global pooling unit, a matrix re-dimensioning unit, and an activation function (Softmax). The spatial content preservation unit can be used to preserve the spatial location information in the input and can be composed of a 1×1 convolution and a matrix re-dimensioning unit.

[0109] Specifically, the channel-domain weighted features output by the channel-domain self-attention unit can be respectively input into the spatial key analysis unit and the spatial content retention unit in the spatial key analysis unit. In step 1321, the spatial key analysis unit can perform a 1×1 convolution on the channel-domain weighted features. The channel dimension after the convolution is transformed from C to C / 2. While ensuring that the spatial retention features are at a higher resolution, global pooling is used to compress the dimensionality of the features after the 1×1 convolution and aggregate spatial information. After the internal matrix dimension change, Softmax is used to enhance the features to obtain the spatial attention weights.

[0110] In step 1322, the spatial content retention unit can perform 1×1 convolution on the channel domain weighted features. The channel dimension after convolution is transformed from C to C / 2. The channel dimension is maintained at C / 2 through the internal matrix dimension change unit to obtain the spatial retention feature, so that the spatial retention feature is maintained at a higher resolution.

[0111] Furthermore, in step 1323, a dot product operation, i.e., matrix multiplication, can be performed on the spatial attention weight and the spatial retention feature through a second matrix dot product unit, and then the dot product operation result can be matrix-dimensionalized through a matrix dimensioning unit, so that the channel dimension of the spatial channel domain weighted feature finally outputted is C.

[0112] Finally, in step 1324, the second activation function unit (Sigmod function) can be used to limit the matrix dimension change result to a set range, so as to limit the feature weight to the range of 0~1, and obtain the spatial domain feature weight; and then, through the spatial domain multiplication unit, the spatial domain feature weight is multiplied by the channel domain weighted feature to obtain the spatial channel domain weighted feature.

[0113] like Figure 3 As shown in the figure, the channel domain weighted features can enter the spatial domain self-attention unit. In the spatial domain self-attention unit, the 1×1 convolution and matrix dimension change at the top are spatial content retention units, and the 1×1 convolution, global pooling, matrix dimension change and activation function (Softmax function) at the bottom are spatial criticality analysis units. Represents the second matrix dot product unit, located at The subsequent matrix dimension change is the matrix dimension change unit, and the activation function after the matrix dimension change unit is the second activation function unit (Sigmod function). Represents the spatial domain multiplication unit.

[0114] refer to Figure 3The channel-domain weighted features can first undergo 1×1 convolution and matrix dimension transformation in the spatial content preservation unit to obtain spatial preservation features. Then, they undergo 1×1 convolution, global pooling, matrix dimension transformation, and activation function in the spatial criticality analysis unit to obtain spatial attention weights. Furthermore, after the spatial attention weights are multiplied by the spatial preservation features, they enter the matrix dimension transformation unit and the second activation function unit in sequence to obtain spatial domain feature weights. Finally, the spatial domain feature weights are multiplied by the channel-domain weighted features through the spatial domain multiplication unit to obtain spatial channel domain weighted features.

[0115] For example, the output of the spatial channel domain weighted feature can be expressed as:

[0116] , ;

[0117] Where, is the spatial channel domain weighted feature output by the spatial domain self-attention unit, is the channel domain weighted feature output by the channel domain self-attention unit, The dimension representing the weighted features in the channel domain is , is the spatial domain multiplication operator. is the spatial domain feature weight, It can be calculated by the following formula:

[0118] ;

[0119] Where, 、 Represents a 1×1 convolution of the channel domain weighted features, and 、 Reshape operator representing tensor, i.e. matrix dimension change, Represents the global average pooling of the convolution results in the brackets. is the dot product operation of the matrix, It represents the use of Softmax function to perform nonlinear transformation on the dot product operation results within the expansion. Represents the Sigmod function, that is, the Sigmod function is used to process the matrix dimension change result in the brackets.

[0120] In the above steps 1321-1324, the channel domain weighted features pass through the spatial criticality analysis unit and can be used to calculate the attention weights. After passing through the spatial content retention unit, they can be used for subsequent weighted aggregation feature values, which can reduce information interference and improve attention resolution. Compared with using the same two processing branches to obtain the same spatial attention weights and spatial retention features, it can avoid the attention map from being too averaged and losing the ability to distinguish differences. In an embodiment of the present application, processing the spatial attention weights and spatial retention features separately can enable the spatial criticality analysis unit to focus on extracting key spatial positions, and the spatial content retention unit to focus on retaining spatial position information with high resolution. The two complement each other and enhance the expressiveness, controllability and flexibility of the polarized self-attention mechanism.

[0121] S120 , determining a plurality of candidate loop images corresponding to the current frame image based on the global description features of the current frame image and the global description features of each historical frame image in the graph database.

[0122] The graph database is used to store the global description features of the historical frame images popped out by the first-in-first-out queue, and the first-in-first-out queue is used to maintain the adjacent m historical frame images before the current frame.

[0123] In an embodiment of the present application, the first-in-first-out queue can always maintain the m historical frame images before the current frame. When the current frame is updated, the first-in-first-out queue can pop out the historical frame images according to the first-in-first-out rule to ensure that the number of stored images is fixed at m.

[0124] Specifically, the first-in-first-out queue can be understood as maintaining historical frame images within a preset time window (m). When the current frame is updated, the previous current frame image will enter the first-in-first-out queue as a historical frame image. The number of images maintained by the first-in-first-out queue is fixed to m (a positive integer). If the queue length is greater than m after inserting a new image, the first-in-first-out queue will pop out the historical frame image that entered first, and the popped-out historical frame image will enter the graph database.

[0125] For example, taking the current time as t, the first-in-first-out queue can maintain the historical frame images from (tm) to (t-1), and the historical frame images before (t-1) will be popped out of the first-in-first-out queue to the graph database in turn.

[0126] The historical frame images entered into the graph database can participate in the candidate loop image search for the current frame. The graph database can store the global description features of the historical frame images in the form of nodes and edges. Nodes represent the global description features of the historical frame images, and edges represent the similarity between global description features, facilitating the subsequent candidate loop image search in the graph database.

[0127] It should be noted that, in the embodiment of the present application, a first-in-first-out queue is used to dynamically maintain historical frame images within a preset time window, and images that exceed the preset time window are popped out to be stored in the graph database. The purpose is: considering that the process of searching for candidate loop images through global description features is a coarse-grained search process, there is a strong similarity between images at adjacent moments, which may lead to the occurrence of false positives. In order to avoid such situations, by allowing historical frame images stored in the graph database to participate in the candidate loop image search of the current frame, the purpose of avoiding interference from images adjacent to the current frame can be achieved, thereby ensuring the accuracy of loop detection. At the same time, by maintaining historical frame images that do not participate in the search through a fixed-length first-in-first-out queue, and being able to dynamically update the queue and the graph database as time flows, compared to storing all historical frame images and corresponding time points in the graph database, selecting some images by time points to participate in the loop search can greatly improve the efficiency of the loop search and avoid search errors.

[0128] In an embodiment of the present application, multiple candidate loop images similar to the current frame image can be retrieved based on the similarity between the global description features of the current frame image and the global description features of each historical frame image in the graph database.

[0129] In some embodiments, determining multiple candidate loop images corresponding to the current frame image based on the global description features of the current frame image and the global description features of each historical frame image in the graph database includes:

[0130] Based on the global description features of the current frame image and the global description features of each historical frame image in the graph database, the similarity between the current frame image and each historical frame image is determined; in descending order of similarity, a preset number of historical frame images are selected from the graph database as candidate loop images.

[0131] Among them, for the historical frame images in the graph database, the cosine similarity of the global description features of the current frame image and the historical frame images can be calculated as shown in the following formula:

[0132] ;

[0133] Where, is the cosine similarity of the global description features of the current frame image q and the historical frame image r, is the global description feature of the current frame image q, is the global description feature of the historical frame image r.

[0134] Furthermore, a preset number of historical frame images can be selected from the graph database in descending order of similarity, i.e., in descending order of cosine similarity, as candidate loop images. The preset number can be a pre-set number of images for the loop frame search, which can be set according to actual needs.

[0135] In an embodiment of the present application, in order to improve the efficiency of loop search, the graph database can be set to a hierarchical storage structure. The higher the number of layers, the fewer the corresponding nodes. The search is performed in each layer from top to bottom, and local nodes are selected in the next layer for search based on the results of the search in the previous layer. This avoids calculating the similarity of the global description features of all historical frame images in the graph database and the current frame image, greatly improving the search efficiency of candidate loop images.

[0136] In a specific embodiment, based on the global description features of the current frame image and the global description features of each historical frame image in the graph database, determining the similarity between the current frame image and each historical frame image includes the following steps:

[0137] Step 21: Take the top layer of the graph database as the current layer, take the global description feature of the current frame image as the current node, and calculate the distance between the current node and each node in the current layer;

[0138] Step 22: determining a search start node among the nodes in the current layer based on the distance, and determining an intermediate position based on the distance between the nodes in the current layer;

[0139] Step 23: The next layer of the current layer is used as the new current layer, and the search cutoff node of the current layer is determined according to the middle position;

[0140] Step 24: Starting from the search start node of the current layer, calculate the distance between the current node and each node in the current layer until the distance between the current node and the search end node is calculated;

[0141] Step 25: Return to the step of determining the search start node among the nodes in the current layer based on the distance until the bottom layer of the graph database is reached.

[0142] Among them, the graph database uses a hierarchical manner to store each node. The node represents the global description feature of the historical frame image. The higher the number of layers, the fewer the corresponding nodes, and the distance represents similarity.

[0143] For example, the global descriptive features of all historical frame images stored in the graph database can be used as nodes, and the similarities between the nodes can be calculated to obtain the edges between the nodes. Furthermore, all nodes and the edges between nodes can be stored in the bottom layer. Then, from all the nodes in the bottom layer, some sparse nodes (i.e., nodes with edge lengths greater than a threshold) can be filtered out and stored in the layer above the bottom layer. Then, in the newly constructed layer, some sparse nodes can be filtered out and stored in the layer above. This process is repeated until the top layer of the graph database is constructed.

[0144] Specifically, in step 21, the top layer can be used as the current layer, and the global description features of the current frame image can be inserted into the graph database as the current node, and the distance between the current node and each node in the current layer can be calculated, that is, the edge between the current node and each node in the current layer can be calculated.

[0145] Furthermore, in step 22, based on the calculated distance, the node closest to the current node can be selected as the search start node among the nodes in the current layer, and the middle position between the nodes can be taken based on the distance between the nodes in the current layer.

[0146] Furthermore, in steps 23-25, the layer below the current layer can be used as the new current layer, and the node in the current layer closest to the middle position can be used as the search end node. In the current layer, starting from the search start node and along the direction of the search end node, the distance between each node in the current layer and the current node is calculated until the distance between the current node and the search end node is obtained. At this time, the process returns to step 22 to continue searching the distance in the previous layer until the bottom layer of the graph database is reached.

[0147] For example, Figure 4 This is a search diagram of a graph database provided in an embodiment of the present application, such as Figure 4 As shown, taking the graph database containing 3 layers as an example, namely Layer0, Layer1, and Layer2, the historical frame images popped out from the first-in-first-out queue will be inserted into the graph database.

[0148] refer to Figure 4, the gray node represents the current node, that is, the global description feature of the current frame image, and the black node represents the node stored in each layer. First, start the search from Layer 2 and calculate the distance between the current node and nodes 0 and 1 in Layer 2. Since node 1 is closer to the current node, node 1 can be used as the search starting node of Layer 1; then, in Layer 1, node 3, which is closest to the middle position between nodes 0 and 1, can be used as the search end node. Starting from node 1 to node 3, the distance between each node and the current node is calculated. Since node 3 is closer to the current node, node 3 can be used as the search starting node of Layer 0; then, in Layer 0, node 4, which is closest to the middle position between nodes 2 and 3, can be used as the search end node. Starting from node 3 to node 4, the distance between each node and the current node is calculated.

[0149] Through the above steps 21 to 25, the similarity of the global description features of some historical frame images in the graph database and the current frame image can be obtained by searching layer by layer, avoiding calculation of all historical frame images in the graph database. While ensuring the search accuracy of candidate loop images, the search efficiency can be greatly improved, and calculation of historical frame images that are significantly different from the current frame image can be avoided.

[0150] S130: For each candidate loop image, perform feature matching on the weighted local features of the candidate loop image and the current frame image, and determine a geometric consistency score corresponding to the candidate loop image based on the matched feature pairs.

[0151] The geometric consistency score is used to reflect the geometric consistency between the candidate loop image and the current frame image. After obtaining multiple candidate loop images, for each candidate loop image, the candidate loop image and the current frame image can be regarded as a loop image pair, and then the spatial geometric consistency check can be performed on the loop image pair.

[0152] Specifically, feature matching can be performed first on the weighted local features of the candidate loop image and the current frame image. For example, the weighted local features of the candidate loop image and the current frame image can be calculated using a mutual nearest neighbor method to determine the matching feature points in the candidate loop image and the current frame image, and the matching feature points are used as matching feature pairs.

[0153] Furthermore, considering that in scenes with changing perspectives, regional features have overall movement consistency, the spatial geometric consistency between the candidate loop image and the current frame image can be verified based on all matching feature pairs. By evaluating the geometric consistency between the candidate loop image and the current frame image, a geometric consistency score is obtained to determine whether there is a large spatial offset between the candidate loop image and the current frame image.

[0154] In some embodiments, determining a geometric consistency score corresponding to a candidate loop closure image based on the matching feature pairs includes:

[0155] Determine the number of matching feature pairs and the spatial offsets of each matching feature pair in the horizontal and vertical directions; and determine a geometric consistency score based on the number of matching feature pairs and the spatial offsets of each matching feature pair in the horizontal and vertical directions.

[0156] The spatial offsets of the two feature points in each matching feature pair in the horizontal and vertical directions can be calculated, and then the geometric consistency score can be determined by combining the number of matching feature pairs and the spatial offsets of each matching feature pair.

[0157] ;

[0158] Where, is the geometric consistency score, is the number of matching feature pairs, is the distance between matching feature pairs, , indicating the index of the matching feature pair. , , respectively represent the deviation geometry of the matching feature pairs in the horizontal and vertical directions; 、 Represent matching feature pairs ( ) in the horizontal direction, 、 Represent matching feature pairs ( ) in the vertical direction. Represents matching feature pairs ( ) Deviation geometry in horizontal and vertical directions. Represents the maximum deviation values ​​of the matching feature pairs in the horizontal and vertical directions respectively. They represent the average deviation of the matching feature pairs in the horizontal direction and the average deviation in the vertical direction, respectively, and can be calculated by solving the average value.

[0159] This implementation allows for evaluating spatial offsets caused by changes in viewpoint based on matched feature pairs. Compared to common random sampling consensus methods, this approach eliminates the need for time-consuming intrinsic matrix calculations and efficiently completes geometric verification by performing simple interpolation operations on pixel coordinates (i.e., feature points). When matching feature pairs are obtained using mutual nearest neighbor matching, this approach provides a simple and efficient geometric verification without sacrificing recall.

[0160] S140: Determine a final loop closure image based on the geometric consistency scores corresponding to the candidate loop closure images.

[0161] Specifically, after obtaining the geometric consistency score, candidate loop images with geometric consistency scores less than a preset score threshold can be eliminated, and among the remaining candidate loop images, the candidate loop image with the highest geometric consistency score is selected as the final loop image.

[0162] In the embodiment of the present application, considering that loop detection usually has temporal consistency, that is, after a loop is detected at the current moment, the loop can often continue to be detected in adjacent frames such as the previous frame or the next frame, therefore, after performing a geometric consistency check, a temporal consistency check can also be performed.

[0163] In some embodiments, after determining the final loop closure image based on the geometric consistency scores corresponding to the candidate loop closure images, the method further includes:

[0164] Determine whether the corresponding final loop image is detected in multiple consecutive moments before or after the current frame; if not, delete the final loop image corresponding to the current frame image.

[0165] Specifically, it can be determined whether the corresponding final loop image is detected in multiple consecutive moments before or after the current frame. If so, it means that the time consistency check is passed and the final loop image corresponding to the current frame image can be output. If not, it means that the time consistency check is not passed and the final loop image corresponding to the current frame image can be deleted, that is, no loop is detected in the current frame.

[0166] Through the above implementation, after the final loop closure image is obtained according to the geometric consistency check, a temporal consistency check can be further performed, thereby further improving the accuracy of loop closure detection.

[0167] For example, Figure 5 This is a schematic diagram of a loop detection process provided by an embodiment of the present application, such as Figure 5 As shown in the figure, the current frame image can first enter the feature extraction model to obtain global description features and weighted local features, and then use the global description features to perform a fast search for similar vectors based on the graph database to obtain the top n candidate loop images, and then use the weighted local features to perform spatial offset geometry verification and temporal consistency verification to obtain the final result and determine whether a loop occurs in the current frame.

[0168] The loop detection method based on weighted local features and global description features provided in the embodiment of the present application inputs the current frame image into a pre-trained feature extraction model to obtain the global description features and weighted local features of the current frame image, and then searches for multiple candidate loop images corresponding to the current frame image in the graph database based on the global description features of the current frame image and the global description features of each historical frame image in the graph database. After the candidate loop images are screened out, for each candidate loop image, feature matching is performed on the weighted local features of the candidate loop image and the current frame image, and the geometric consistency between the candidate loop image and the current frame image is verified based on the matching features to obtain a geometric consistency score. According to the geometric consistency scores corresponding to each candidate loop image, the final loop image is determined to realize loop detection. This method effectively identifies static salient areas through weighted local features, which can reduce the interference of dynamic objects and non-discriminative areas on loop detection. Moreover, through the close coupling of weighted local features and global description features, the accuracy of loop detection is significantly improved, especially for complex environments containing dynamic objects and non-discriminative areas. In addition, this method maintains the adjacent m historical frame images before the current frame through a first-in-first-out queue, which can avoid the historical frame images adjacent to the current frame from participating in the candidate loop frame search, thereby avoiding false positive loops caused by interference from adjacent images and further improving the accuracy of loop detection.

[0169] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 6 As shown, the electronic device 400 includes one or more processors 401 and a memory 402 .

[0170] The processor 401 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0171] The memory 402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 401 may execute the program instructions to implement the loop detection method based on weighted local features and global description features of any embodiment of the present application described above and / or other desired functions. Various contents such as initial external parameters, thresholds, etc. may also be stored in the computer-readable storage medium.

[0172] In one example, electronic device 400 may further include an input device 403 and an output device 404, which are interconnected via a bus system and / or other connection mechanisms (not shown). Input device 403 may include, for example, a keyboard, a mouse, etc. Output device 404 may output various information to the outside, including warning information, braking force, etc. Output device 404 may include, for example, a display, a speaker, a printer, a communication network, and remote output devices connected thereto.

[0173] Of course, to simplify, Figure 6 Only some of the components related to the present application in the electronic device 400 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, the electronic device 400 may further include any other appropriate components according to specific application scenarios.

[0174] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the loop detection method based on weighted local features and global description features provided in any embodiment of the present application.

[0175] The computer program product may be written in any combination of one or more programming languages ​​to implement the program code for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0176] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, causes the processor to execute the steps of the loop detection method based on weighted local features and global description features provided in any embodiment of the present application.

[0177] The computer-readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0178] It should be noted that the terms used in this application are only for describing specific embodiments and are not intended to limit the scope of this application. As shown in the specification and claims of this application, unless the context clearly indicates an exception, the words "one", "an", "a kind of" and / or "the" do not specifically refer to the singular and may also include the plural. The terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of further restrictions, the elements defined by the sentence "comprise a..." do not exclude the presence of other identical elements in the process, method or device comprising the elements.

[0179] It should also be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on this application. Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", etc. should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be a communication between the internal parts of two elements. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0180] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. The above is only the preferred implementation method of this application. It should be pointed out that due to the limitations of textual expression, there are objectively infinite specific structures. For ordinary technicians in this technical field, without departing from the principles of this application, they can also make several improvements, modifications or changes, and can also combine the above technical features in an appropriate manner; these improvements, modifications, changes or combinations, or the direct application of the inventive concept and technical solution to other occasions without improvement, should be regarded as the scope of protection of this application.

Claims

1. A loop detection method based on weighted local features and global description features, characterized in that: The method comprises: Inputting the current frame image into a pre-trained feature extraction model, wherein the feature extraction model includes a backbone network, a polarized self-attention module, a first dimensionality reduction module, an aggregation module, and a second dimensionality reduction module; Extracting initial local features from the current frame image based on the backbone network; Performing weighted processing on each feature point in the initial local features according to the polarized self-attention module, and performing dimensionality reduction on the result output by the polarized self-attention module according to the first dimensionality reduction module to obtain weighted local features; Aggregating the initial local features into compact features according to the aggregation module, and reducing the dimension of the result output by the aggregation module according to the second dimensionality reduction module to obtain a global description feature; Based on the global description features of the current frame image and the global description features of each historical frame image in the graph database, multiple candidate loop images corresponding to the current frame image are determined, wherein the graph database is used to store the global description features of the historical frame images popped out by the first-in-first-out queue, and the first-in-first-out queue is used to maintain the m adjacent historical frame images before the current frame; the graph database has a hierarchical storage structure, and the higher the layer, the fewer the number of nodes corresponding to it. The search is performed in each layer from top to bottom, and local nodes are selected in the next layer for search based on the search results of the previous layer; For each of the candidate loop closure images, perform feature matching on weighted local features of the candidate loop closure image and the current frame image, and determine a geometric consistency score corresponding to the candidate loop closure image based on the matched feature pairs, wherein the geometric consistency score is used to reflect the geometric consistency between the candidate loop closure image and the current frame image; Based on the geometric consistency scores corresponding to the candidate loop closure images, a final loop closure image is determined.

2. The method according to claim 1, characterized in that The polarized self-attention module includes a channel domain self-attention unit and a spatial domain self-attention unit, and performs weighted processing on each feature point in the initial local feature according to the polarized self-attention module, including: Performing channel-domain selection on the initial local features according to the channel-domain self-attention unit to obtain channel-domain weighted features; The channel domain weighted features are subjected to spatial domain selection according to the spatial domain self-attention unit to obtain spatial channel domain weighted features.

3. The method according to claim 2, characterized in that The channel-domain self-attention unit includes a channel importance analysis unit, a channel content selection unit, a first matrix dot product unit, a channel dimension increase unit, a first activation function unit, and a channel-domain multiplication unit. Channel-domain selection is performed on the initial local features according to the channel-domain self-attention unit to obtain channel-domain weighted features, including: Analyzing the importance of each channel in the initial local features according to the channel importance analysis unit to obtain a channel attention weight; Determining the channel content selected in the initial local feature according to the channel content selection unit to obtain a channel selected feature; Performing a dot product operation on the channel attention weight and the channel selected feature based on the first matrix dot product unit, and performing channel dimension increasing processing on the dot product operation result through the channel dimension increasing unit; According to the first activation function unit, the channel dimensionality increase result is limited to a set range to obtain the channel domain feature weight, and the channel domain feature weight is multiplied by the initial local feature through the channel domain multiplication unit to obtain the channel domain weighted feature.

4. The method according to claim 2, characterized in that The spatial domain self-attention unit includes a spatial criticality analysis unit, a spatial content retention unit, a second matrix dot product unit, a matrix dimension change unit, a second activation function unit, and a spatial domain multiplication unit. The spatial domain self-attention unit performs spatial domain selection on the channel domain weighted features to obtain spatial channel domain weighted features, including: Analyzing the criticality of each spatial position in the channel-domain weighted features according to the spatial criticality analysis unit to obtain a spatial attention weight; retaining spatial position information in the channel domain weighted features according to the spatial content retaining unit to obtain a spatially retained feature; Performing a dot product operation on the spatial attention weight and the spatial retention feature based on the second matrix dot product unit, and performing matrix dimension conversion processing on the dot product operation result through the matrix dimension conversion unit; According to the second activation function unit, the matrix dimension change result is limited to a set range to obtain the spatial domain feature weight, and the spatial domain feature weight is multiplied by the channel domain weighted feature through the spatial domain multiplication unit to obtain the spatial channel domain weighted feature.

5. The method according to claim 1, wherein Determining a plurality of candidate loop images corresponding to the current frame image based on the global description features of the current frame image and the global description features of each historical frame image in the graph database includes: Determining similarities between the current frame image and each of the historical frame images based on a global description feature of the current frame image and a global description feature of each of the historical frame images in the graph database; A preset number of historical frame images are selected from the graph database in descending order of similarity as candidate loop images.

6. The method according to claim 5, characterized in that Determining the similarity between the current frame image and each of the historical frame images based on the global description features of the current frame image and the global description features of each of the historical frame images in the graph database includes: The top layer of the graph database is used as the current layer, and the global description feature of the current frame image is used as the current node. The distance between the current node and each node in the current layer is calculated, wherein the graph database stores each node in a hierarchical manner, and the node represents the global description feature of the historical frame image. The higher the layer number, the fewer the number of nodes. The distance represents similarity. Determining a search start node among the nodes in the current layer based on the distance, and determining an intermediate position based on the distance between the nodes in the current layer; Taking the next layer of the current layer as a new current layer, and determining the search cutoff node of the current layer according to the intermediate position; Starting from the search start node of the current layer, calculating the distance between the current node and each node in the current layer until calculating the distance between the current node and the search end node; Return to the step of determining a search start node among the nodes of the current layer based on the distance until the bottom layer of the graph database is reached.

7. The method according to claim 1, characterized in that Determining a geometric consistency score corresponding to the candidate loop closure image based on the matching feature pair includes: Determining the number of matching feature pairs and the spatial offset of each matching feature pair in the horizontal and vertical directions; A geometric consistency score is determined according to the number of the matching feature pairs and the spatial offset of each matching feature pair in the horizontal and vertical directions.

8. The method according to claim 1, characterized in that After determining the final loop closure image based on the geometric consistency scores corresponding to the candidate loop closure images, the method further includes: Determine whether the corresponding final loop image is detected in multiple consecutive moments before or after the current frame; If not, the final loop image corresponding to the current frame image is deleted.

9. An electronic device, characterized in that: The electronic device comprises: processor and memory; The processor is used to execute the steps of the loop detection method based on weighted local features and global description features as described in any one of claims 1 to 8 by calling the program or instructions stored in the memory.

Images