Feature extraction method of feature point extraction network based on fusion of double attention in SLAM system
By introducing a feature point extraction network with dual attention in the visual SLAM system, using convolutional neural network and attention mechanism, the problems of insufficient accuracy and low stability of the existing feature point extraction method are solved, and the characteristics of the SLAM system are improved Point extraction accuracy and stability thus improve the accuracy of position estimation.
Patent Information
- Application Number
- CN202410974116.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-19
AI Technical Summary
The existing feature point extraction method based on deep learning has problems of insufficient accuracy and low stability in visual SLAM systems, which affects the image matching accuracy and the accuracy of robot positioning.
A fusion dual attention feature point extraction network based on convolutional neural network is adopted. Through the feature extraction module, dual attention module, detection module and description module, combined with position attention and channel attention mechanism, a weighted feature map is generated and binary probability is performed. The generation of graphs and dense feature maps improves the robustness of feature extraction.
It significantly improves the accuracy and stability of feature point extraction, improves the accuracy and robustness of pose estimation of SLAM systems, and enhances the accuracy of image matching.
Smart Images

Figure CN118982676A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a feature extraction method of a feature point extraction network, in particular to a feature extraction method of a feature point extraction network based on fusion of dual attention in a SLAM system, belonging to the technical field of target detection. Background Art
[0002] Simultaneous Localization and Mapping (SLAM) refers to the process of building an environmental model and estimating the robot's own motion when it moves in an unknown environment. Visual SLAM is a classic SLAM method in the SLAM system. In the early days, visual SLAM was not widely used due to its large amount of computation and poor real-time performance. However, with the advancement of computer technology and graphics processing units (GPUs), visual SLAM has overcome these shortcomings and gradually become the mainstream of autonomous positioning and environmental perception.
[0003] Feature extraction is an important step in visual SLAM. The feature points of an image map out the correspondence between different images. Accurately extracting feature points can improve the matching accuracy of the image. The processing methods of image feature points are mainly divided into traditional methods and methods based on deep learning. With the widespread application of computer vision and the increasingly complex environment faced by image processing, the robustness of traditional feature point extraction methods has gradually deteriorated. With the development of deep learning, especially the introduction of convolutional neural networks, feature point extraction methods based on deep learning have made significant progress. These methods can learn robust feature representations from large-scale data sets through end-to-end training. However, the existing feature point extraction methods based on deep learning have insufficient feature extraction network accuracy and low stability. Summary of the invention
[0004] Purpose of the invention: The purpose of the present invention is to provide a feature extraction method based on a feature point extraction network fused with dual attention in a SLAM system, which can improve the accuracy and stability of the feature extraction network.
[0005] Technical solution: The feature extraction method based on the feature point extraction network integrating dual attention in a SLAM system of the present invention comprises:
[0006] (1) Establishing a feature point extraction network integrating dual attention and integrating it into the existing visual SLAM framework; the feature point extraction network is a convolutional neural network, including a feature extraction module, a dual attention module, a detection module and a description module;
[0007] (2) Convert the image to be processed into a grayscale image and output a feature map through a feature extraction module;
[0008] (3) passing the feature map through a dual attention module to obtain a weighted feature map;
[0009] (4) passing the weighted feature map through the convolutional layer in the detection module and the convolutional layer in the description module to generate a binary probability map and a dense feature map respectively;
[0010] (5) The probability map is converted to the original resolution by pixel reorganization through the detection module to obtain the coordinates of the key points;
[0011] (6) Obtain the corresponding descriptor from the description module according to the key point coordinates;
[0012] (7) The descriptor is converted into a binary descriptor through the binarization layer of the description module to complete feature extraction.
[0013] Furthermore, the feature extraction module of the feature point extraction network is composed of six convolutional layers, including two single-layer convolutional layers Conv1 and Conv2, and two double-layer convolutional layers Conv3 and Conv4, the convolution kernel size of the single-layer convolutional layer is 4*4, the step size is 2, and the padding is 1, the convolution kernel size of the front convolutional layer of the double-layer convolutional layer is 3*3, the step size is 1, and the padding is 1, the convolution kernel size of the rear convolutional layer of the double-layer convolutional layer is 4*4, the step size is 2, and the padding is 1, each convolutional layer uses the ELU activation function for nonlinear transformation, and adopts a 6-layer convolution structure;
[0014] The dual attention module of the feature point extraction network includes a position attention module and a channel attention module;
[0015] The description module includes two convolutional layers ConvF_1 and ConvF_2, wherein ConvF_1 uses a convolution kernel of size 3x3, the number of output channels is 256, the step size is 1 for convolution, and the padding is 1; the ConvF_2 uses a convolution kernel of size 1x1, the number of output channels is 256, the step size is 1 for convolution, and no padding is performed; the generated descriptor is ELU activated and normalized by the L2 norm.
[0016] Furthermore, the number of channels of the feature map output by step (2) is 64, 128 or 256.
[0017] Furthermore, the step (3) is specifically as follows: modeling the dependency between the position and channel dimensions of the feature map through a position attention module and a channel attention module respectively; the position attention module is used to determine the spatial dependency between any two positions, and the channel attention module is used to capture the channel dependency between any two channel maps, and update them using the weights of all channel maps; the outputs of the position attention module and the channel attention module are fused to enhance the feature representation and obtain a weighted feature map.
[0018] Furthermore, the specific implementation method of the position attention module is to add context information to local features to enhance feature representation; assuming that the feature map entering the position attention module is A1, the feature map A1 is respectively passed through the convolution layer to obtain three feature maps B, G and D, and the shapes are all C*H*W, C is the number of channels, H is the height, and W is the width; the feature map B is reshaped to N*C, and the feature map G and the feature map D are reshaped to C*N, where the number of pixels N=H*W; G is multiplied by the transposed matrix of B, and the position attention map S is obtained by the activation function Softmax; the feature map D is multiplied by the transposed matrix of S, and then multiplied by the scale coefficient α, and finally added to A1 to obtain the final output position weighted feature map E1; the calculation formulas of the position attention map S and the feature map E1 are as follows:
[0019]
[0020]
[0021] In the formula, S ba represents the dependence of position a on position b, α is the scale coefficient, B a is the element of feature graph B at position a, G b is the element of feature graph G at position b, D a is the element of feature graph D at position a, A1 b is the element of feature graph A1 at position b, E1 b is the element of feature graph E1 at position b, exp() is the exponential function with base e;
[0022] The specific implementation method of the channel attention module is to improve the feature representation by establishing a mutual dependence relationship between channels; assuming that the feature map entering the channel attention module is A2, the channel attention module reshapes the feature map A2 into C*N, multiplies it by the reshaped transposed matrix, and obtains the channel attention map X through the Softmax layer; A2 performs a matrix multiplication operation on X and then multiplies it by the scale coefficient β, and performs an element-by-element addition operation with A2, and outputs the channel weighted feature map E2. The calculation process of the channel attention map X and the channel weighted feature map E2 is as follows:
[0023]
[0024]
[0025] Where, X nm represents the dependence between the mth channel and the nth channel; β is the scale factor, A2 m is the element of feature map A2 in the mth channel, A2 n is the element of feature map A2 in the nth channel, E2n is the element of the feature map E2 in the nth channel.
[0026] Furthermore, the binary probability map described in step (4) is used to predict key points, and the dense feature map is used to generate descriptors.
[0027] Furthermore, the step (5) is specifically as follows: predicting the key point by outputting a binary probability map, wherein each pixel value in the probability map is composed of 1 and 0, wherein a pixel value of 1 represents that the pixel position is a key point, and a pixel value of 0 represents that the pixel position is not a key point; and introducing a weighted cross entropy method into the loss function, wherein the loss function is defined as:
[0028]
[0029]
[0030] Where o represents the probability map at the original resolution; a1 and α2 are weight coefficients used to balance the number of positive and negative samples; x cur is the position of the feature point in the current image frame, is x cur The positive sample in the next image frame, x i is the position of the i-th feature point in the image frame, is the i-th feature point x i Label; o(x i ) is the feature point x i The probability of existence; the coordinates of the feature points of the current image frame samples required for key point learning are obtained by detecting Shi-Tomasi corner points, and the training samples of the next image frame are obtained by mapping the samples detected in the current image frame to the next image frame.
[0031] Furthermore, the triple loss function of the descriptor in step (6) is defined as:
[0032]
[0033] d(x cur , x tar )||b cur (x cur )-b tar (x tar )||2
[0034] Among them, d represents the square of the Hamming distance, and p is the cutoff distance margin, which is used to represent and The threshold of the distance difference, b(x) is the mapping function, and its value is ±1. is the position of the feature point in the current image frame; yes Negative samples in the next image frame are obtained through comprehensive negative sample mining method; yes The positive sample in the next image frame is obtained by the real inference trajectory of the camera in the standard dataset:
[0035]
[0036] Where R gt and t gt They are the rotation matrix and translation vector in the real reasoning trajectory, and π represents the feature points in the 2D image according to their 2D coordinates and the depth value z i The process of mapping into 3D space.
[0037] Furthermore, the forward calculation and reverse propagation of the binarization layer described in step (7) are defined as follows:
[0038] Forward operation:
[0039]
[0040] Backward Propagation:
[0041]
[0042] Where f is a dense feature map, b is the binary version of f, and sign is the sign function with the output being ±1, 1 |f|≤1 Used to eliminate the single characteristic response gradient of f whose absolute value is greater than 1;
[0043] STE randomly discretized neurons are used for gradient calculation and back propagation, and b(x) is approximated as:
[0044] Htanh(x)=Clip(x,-1,1)=max(-1,min(1-x))
[0045] This makes the back propagation differentiable.
[0046] Furthermore, the visual SLAM framework is ORB-SLAM2, which includes a system initialization module, a tracking module, a local map construction module, a closed-loop detection module and a global map optimization module; the integration of the feature point extraction network into the visual SLAM framework is specifically to replace the feature extraction link in the tracking module with the feature point extraction network.
[0047] Beneficial effects: Compared with the existing feature point extraction network, the present invention improves the structure of the feature point extraction network and adds a dual attention mechanism to the feature point extraction network, thereby improving the accuracy and stability of the feature points extracted by the feature point extraction network, and further applies the improved network to the SLAM system, thereby improving the pose estimation accuracy and robustness of the SLAM system. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 is a method flow chart of an embodiment of the present invention;
[0049] Figure 2 This is a structural diagram of a dual attention feature point extraction network fused with an embodiment of the present invention;
[0050] Figure 3 is a structural diagram of a dual attention mechanism module according to an embodiment of the present invention;
[0051] Figure 4 It is a flow chart of a visual SLAM system integrating a dual attention feature point extraction network according to an embodiment of the present invention. DETAILED DESCRIPTION
[0052] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0053] As attached Figure 1 As shown, the feature extraction method based on the feature point extraction network fused with dual attention in the SLAM system of this embodiment includes:
[0054] (1) Establishing a feature point extraction network integrating dual attention and integrating it into the existing visual SLAM framework; the feature point extraction network is a convolutional neural network, including a feature extraction module, a dual attention module, a detection module and a description module;
[0055] (2) Convert the image to be processed into a grayscale image and output a feature map through a feature extraction module;
[0056] (3) passing the feature map through a dual attention module to obtain a weighted feature map;
[0057] (4) passing the weighted feature map through the convolutional layer in the detection module and the convolutional layer in the description module to generate a binary probability map and a dense feature map respectively;
[0058] (5) The probability map is converted to the original resolution by pixel reorganization through the detection module to obtain the coordinates of the key points;
[0059] (6) Obtain the corresponding descriptor from the description module according to the key point coordinates;
[0060] (7) The descriptor is converted into a binary descriptor through the binarization layer of the description module to complete feature extraction.
[0061] Specifically, as attached Figure 2 As shown, in step (2), a single grayscale image is input into the feature extraction module to obtain a feature map. The feature extraction module consists of six convolutional layers. Conv1 and Conv2 are single-layer convolutional layers, with a convolution kernel size of 4*4, a step size of 2, and a padding of 1. Conv3 and Conv4 are double-layer convolutions, with a convolution kernel size of 3*3, a step size of 1, and a padding of 1 for the front convolutional layer. The convolution kernel size of the rear convolutional layer is 4*4, a step size of 2, and a padding of 1. Each convolutional layer uses the ELU activation function for nonlinear transformation. In order to ensure the efficiency of the network operation, a 6-layer convolution structure is adopted, and the number of channels of the convolutional layer is set to 1-32-64-128-128-256-256. The input image is I, and a convolution operation is performed on the input image to obtain a multi-scale feature map P1. The input image is I 320×240×1 , the resolution is 320*240, the number of channels is 1, and the input image is convolved to obtain a multi-scale feature map P1 is passed through the dual attention module to obtain the weighted feature map P2.
[0062] As attached Figure 3 As shown in the figure, the module structure diagram of the dual attention mechanism of the present invention is composed of a position attention module and a channel attention module. In step (3), in order to enhance the global features of the feature map, the spatial and channel dependencies of the feature map P1 are modeled by the position attention module and the channel attention module respectively. The position attention module determines the spatial dependency between any two positions, and the channel attention module captures the channel dependency between any two channel maps and updates them using the weights of all channel maps. The outputs of the two attention modules are fused to enhance the feature representation as a weighted feature map The resolution is 40*30 and the number of channels is 256.
[0063] The spatial attention module adds contextual information to local features to enhance their representation. Feature map A1 is passed through the convolutional layer to obtain three feature maps B, G, and D, with a shape of C*H*W. Feature map B is reshaped to N*C, and feature map G and feature map D are reshaped to C*N, where N=H*W is the number of pixels. Multiply the transposed matrix of G and B, and then use the activation function Softmax to obtain the spatial attention map S. Matrix D is multiplied by the transposed matrix of S, multiplied by the scale factor α, and finally added to A1 to obtain the final output spatial weighted feature map E1. The calculation formulas for spatial attention map S and feature map E1 are as follows:
[0064]
[0065] Where Sba Indicates the correlation of position a to position b. The larger the value, the greater the similarity. α is the scale factor. B a is the element of feature graph B at position a, G b is the element of feature graph G at position b, D a is the element of feature graph D at position a, A1 b is the element of feature graph A1 at position b.
[0066] The channel attention module improves feature representation by establishing mutual dependencies between channels. Unlike the position attention module, the channel attention module reshapes the feature map A2 into C*N, multiplies it by the reshaped transposed matrix, and passes it through the Softmax layer to obtain the channel attention map X. Then, X is matrix multiplied by A2 and then multiplied by the scale factor β, and finally element-by-element addition is performed with A to output the channel weighted feature map E2. The calculation process of the channel attention map X and the channel weighted feature map E2 is as follows:
[0067]
[0068] Where, X nm represents the dependence between the mth channel and the nth channel; β is the scale factor, A2 m is the element of feature map A2 in the mth channel, A2 n is the element of feature map A2 in the nth channel, E2 n is the element of the feature map E2 in the nth channel.
[0069] In step (4), the feature map P2 is converted into a binary probability map by the convolution layer in the detection module. Furthermore, in step (5), each pixel value in the probability map is composed of 1 and 0, where the pixel value 1 represents that the pixel position is a key point, and the pixel value 0 represents that it is not. The probability map is converted to the original resolution by pixel reorganization in this module, and the key point coordinates are output. In order to balance the positive and negative samples, the weighted cross entropy method is introduced in the loss function, and the loss function is defined as:
[0070]
[0071] Where o represents the probability map at the original resolution. α1 and α2 are weight coefficients used to balance the number of positive and negative samples. cur is the position of the feature point in the current image frame, is x cur The positive sample in the next image frame, x i is the position of the i-th feature point in the image frame, is x i The label of o(x i ) is the position x iThe probability of the existence of feature points. The coordinates of the feature points of the current image frame samples required for key point learning are obtained by detecting Shi-Tomasi corner points, and the training samples of the next image frame are obtained by mapping the samples detected in the current image frame to the next image frame.
[0072] In step (4), the feature map P2 is output as a dense feature map through a convolutional layer in the description module. Furthermore, in step (6), the corresponding descriptor is output in combination with the key point coordinates, and a binary descriptor is output through a binary activation layer.
[0073] The description module has two convolutional layers. ConvF_1 uses a 3x3 kernel with 256 output channels, stride 1, and padding of 1. ConvF_2 uses a 1x1 kernel with 256 output channels, stride 1, and no padding. The generated descriptors are ELU activated and then normalized by the L2 norm.
[0074] The triple loss function of the descriptor is defined as:
[0075]
[0076] d(x cur , x tar )||b cur (x cur )-b tar (x tar )||2 (8)
[0077] Among them, d represents the square of the Hamming distance, and p is the cutoff distance margin, which is used to represent and The threshold of the distance difference, b(x) is the mapping function, and its value is ±1. is the position of the feature point in the current image frame; yes Negative samples in the next image frame are obtained through comprehensive negative sample mining method; yes The positive sample in the next image frame is obtained by the real inference trajectory of the camera in the standard dataset:
[0078]
[0079] Where R gt and t gt They are the rotation matrix and translation vector in the real reasoning trajectory, and π represents the feature points in the 2D image according to their 2D coordinates and the depth value z i The process of mapping into 3D space.
[0080] Negative samples are obtained through the Exhaustive Negative Sample Mining algorithm.
[0081] The forward computation and backward propagation of the binary activation layer described in step (7) are defined as follows:
[0082]
[0083] Where f is a low-resolution dense feature map and b is the binary version of f. sign is a sign function with an output of ±1. |f|≤1 Eliminate the single feature response gradient of f whose absolute value is greater than 1. Since b(x) is a function whose gradient is 0 everywhere, it is not suitable for back propagation. Therefore, STE (straight-through estimator, STE) is used to randomly discretize neurons for gradient calculation and back propagation, that is, b(x) is approximated as:
[0084] Htanh(x)=Clip(x,-1,1)=max(-1,min(1-x)) (12)
[0085] This makes the back propagation differentiable.
[0086] As attached Figure 4 As shown in FIG. 1 , it is a flowchart of the visual SLAM system of this embodiment that integrates the dual attention feature point extraction network. The visual SLAM framework used is ORB-SLAM2. PCA-GCNv2 is used to replace the feature extractor of the ORB-SLAM2 front end, and a visual SLAM algorithm (PCAG-SLAM) that integrates the dual attention neural network is proposed. Figure 3 As shown in the figure, PCAG-SLAM consists of three threads: tracking, mapping, and loop detection. After the input frame, the tracking thread extracts the feature points of the input frame through PCA-GCNv2, performs initial pose estimation based on feature matching, estimates the pose of the key frame through local pose optimization, and finally determines whether the current frame meets the key frame conditions. After the key frame is inserted, the mapping thread starts working, deletes the map points in the redundant key frame and creates new map points, uses local BA optimization, and finally deletes the redundant key frame. In the loop detection thread, a closed-loop candidate key frame group of the current frame is created, and the loop of the current frame is obtained by calculating the sim3 similarity transformation. The map points of the loop are matched and fused with the current frame, and finally the essential map is optimized to optimize the pose and map points of all key frames in the essential map.
[0087] Combined with the above embodiments, it is further proved that the present invention has a better feature point extraction effect by integrating the dual attention feature point extraction network. The visual SLAM algorithm integrating the dual attention feature point extraction network also improves the accuracy and robustness of posture estimation.
[0088] The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiment. Any equivalent modifications or changes made by ordinary technicians in this field based on the contents disclosed by the present invention should be included in the protection scope recorded in the claims.
Claims
1. A feature extraction method based on a feature point extraction network fused with dual attention in a SLAM system, characterized in that: include: (1) Establishing a feature point extraction network integrating dual attention and integrating it into the existing visual SLAM framework; the feature point extraction network is a convolutional neural network, including a feature extraction module, a dual attention module, a detection module and a description module; (2) Convert the image to be processed into a grayscale image and output a feature map through a feature extraction module; (3) passing the feature map through a dual attention module to obtain a weighted feature map; (4) passing the weighted feature map through the convolutional layer in the detection module and the convolutional layer in the description module to generate a binary probability map and a dense feature map respectively; (5) The probability map is converted to the original resolution by pixel reorganization through the detection module to obtain the coordinates of the key points; (6) Obtain the corresponding descriptor from the description module according to the key point coordinates; (7) The descriptor is converted into a binary descriptor through the binarization layer of the description module to complete feature extraction.
2. The feature extraction method based on the feature point extraction network integrating dual attention in the SLAM system according to claim 1, characterized in that: The feature extraction module of the feature point extraction network is composed of six convolutional layers, including two single-layer convolutional layers Conv1 and Conv2, and two double-layer convolutional layers Conv3 and Conv4. The convolution kernel size of the single-layer convolutional layer is 4*4, the step size is 2, and the padding is 1. The convolution kernel size of the front convolutional layer of the double-layer convolutional layer is 3*3, the step size is 1, and the padding is 1. The convolution kernel size of the rear convolutional layer of the double-layer convolutional layer is 4*4, the step size is 2, and the padding is 1. Each convolutional layer uses the ELU activation function for nonlinear transformation, and adopts a 6-layer convolution structure; The dual attention module of the feature point extraction network includes a position attention module and a channel attention module; The description module includes two convolutional layers ConvF_1 and ConvF_2, wherein ConvF_1 uses a convolution kernel of size 3x3, the number of output channels is 256, the step size is 1 for convolution, and the padding is 1; the ConvF_2 uses a convolution kernel of size 1x1, the number of output channels is 256, the step size is 1 for convolution, and no padding is performed; the generated descriptor is ELU activated and normalized by the L2 norm.
3. The feature extraction method based on the feature point extraction network integrating dual attention in the SLAM system according to claim 1 is characterized in that: The number of channels of the feature map output by step (2) is 64, 128 or 256.
4. The feature extraction method based on the feature point extraction network integrating dual attention in the SLAM system according to claim 2 is characterized in that: The step (3) is specifically as follows: modeling the dependency between the position and channel dimensions of the feature map through a position attention module and a channel attention module respectively; the position attention module is used to determine the spatial dependency between any two positions, and the channel attention module is used to capture the channel dependency between any two channel maps, and update them using the weights of all channel maps; the outputs of the position attention module and the channel attention module are fused to enhance the feature representation and obtain a weighted feature map.
5. The feature extraction method based on the feature point extraction network fused with dual attention in the SLAM system according to claim 4, characterized in that: The specific implementation method of the position attention module is to add context information to local features to enhance feature representation; assume that the feature map entering the position attention module is A1, and the feature map A1 is respectively passed through the convolution layer to obtain three feature maps B, G and D, all of which have a shape of C*H*W, where C is the number of channels, H is the height, and W is the width; the feature map B is reshaped to N*C, and the feature maps G and D are reshaped to C*N, where the number of pixels N=H*W; multiply the transposed matrix of G and B, and obtain the position attention map S through the activation function Softmax; multiply the transposed matrix of the feature map D and S, and then multiply it by the scale coefficient α, and finally add it to A1 to obtain the final output position weighted feature map E1; the calculation formulas of the position attention map S and the feature map E1 are as follows: In the formula, S ba represents the dependence of position a on position b, α is the scale coefficient, B a is the element of feature graph B at position a, G b is the element of feature graph G at position b, D a is the element of feature graph D at position a, A1 b is the element of feature graph A1 at position b, E1 b is the element of feature graph E1 at position b, exp() is the exponential function with base e; The specific implementation method of the channel attention module is to improve the feature representation by establishing a mutual dependence relationship between channels; assuming that the feature map entering the channel attention module is A2, the channel attention module reshapes the feature map A2 into C*N, multiplies it by the reshaped transposed matrix, and obtains the channel attention map X through the Softmax layer; A2 performs a matrix multiplication operation on X and then multiplies it by the scale coefficient β, and performs an element-by-element addition operation with A2, and outputs the channel weighted feature map E2. The calculation process of the channel attention map X and the channel weighted feature map E2 is as follows: Where, X nm Indicates the dependency between the mth channel and the nth channel; β is the scale factor, A2 m is the element of feature map A2 in the mth channel, A2 n is the element of feature map A2 in the nth channel, E2 n is the element of the feature map E2 in the nth channel.
6. The feature extraction method based on the feature point extraction network integrating dual attention in the SLAM system according to claim 1 is characterized in that: The binary probability map described in step (4) is used to predict key points, and the dense feature map is used to generate descriptors.
7. The feature extraction method based on the feature point extraction network integrating dual attention in the SLAM system according to claim 2 is characterized in that: The step (5) is specifically as follows: predicting the key point by outputting a binary probability map, wherein each pixel value in the probability map is composed of 1 and 0, wherein a pixel value of 1 represents that the pixel position is a key point, and a pixel value of 0 represents that the pixel position is not a key point; introducing a weighted cross entropy method into the loss function, wherein the loss function is defined as: Where o represents the probability map at the original resolution; α1 and α2 are weight coefficients used to balance the number of positive and negative samples; x cur is the position of the feature point in the current image frame, is x cur The positive sample in the next image frame, x i is the position of the i-th feature point in the image frame, is the i-th feature point x i Label; o(x i ) is the feature point x i The probability of existence; the coordinates of the feature points of the current image frame samples required for key point learning are obtained by detecting Shi-Tomasi corner points, and the training samples of the next image frame are obtained by mapping the samples detected in the current image frame to the next image frame.
8. The feature extraction method based on the feature point extraction network integrating dual attention in the SLAM system according to claim 7 is characterized in that: The triple loss function of the descriptor in step (6) is defined as: d(x cur ,x tar )=||b cur (x cur )-b tar (x tar )||2 Among them, d represents the square of the Hamming distance, and p is the cutoff distance margin, which is used to represent and The threshold of the distance difference, b(x) is the mapping function, and its value is ±1. is the position of the feature point in the current image frame; yes Negative samples in the next image frame are obtained through comprehensive negative sample mining method; yes The positive sample in the next image frame is obtained by the real inference trajectory of the camera in the standard dataset: Where R gt and t gt They are the rotation matrix and translation vector in the real reasoning trajectory, and π represents the feature points in the 2D image according to their 2D coordinates and the depth value z i The process of mapping into 3D space.
9. The feature extraction method based on the feature point extraction network integrating dual attention in the SLAM system according to claim 1, characterized in that: The forward calculation and reverse propagation of the binarization layer described in step (7) are defined as follows: Forward operation: Backward Propagation: Where f is a dense feature map, b is the binary version of f, and sign is the sign function with the output being ±1, 1 |f|≤1 Used to eliminate the single characteristic response gradient of f whose absolute value is greater than 1; STE randomly discretized neurons are used for gradient calculation and back propagation, and b(x) is approximated as: Htanh(x)=Clip(x,-1,1)=max(-1,min(1-x)) This makes the back propagation differentiable.
10. The feature extraction method based on the feature point extraction network fused with dual attention in the SLAM system according to claim 1 is characterized in that: The visual SLAM framework is ORB-SLAM2, which includes a system initialization module, a tracking module, a local map construction module, a closed-loop detection module and a global map optimization module; the integration of the feature point extraction network into the visual SLAM framework specifically replaces the feature extraction link in the tracking module with the feature point extraction network.
Citation Information
Patent Citations
Visual detection method for pantograph-catenary arcing of electrified railway
CN112766195A
Cited By
Real-time three-dimensional modeling method and system based on 5G flight control and medium
CN119478297A
A real-time three-dimensional modeling method, system and medium based on 5G flight control
CN119478297B
