Improved GCNv2 feature point extraction method based on deep learning

Abstract

The invention discloses an improved GCNv2 feature point extraction method based on deep learning, and relates to the technical field of computer vision and robot self-positioning mapping. In order to solve the defect of insufficient robustness of feature point extraction in a dynamic and multi-scale complex scene in the prior art, the invention provides the following scheme: multi-scale feature input is realized by acquiring image frames continuously acquired by a camera and constructing a multi-layer image pyramid; inputting the multi-scale image into a GCNv2 feature extraction network to generate a feature probability graph and a binary descriptor so as to obtain a high-precision feature point set; performing range tree-based spatial adaptive non-maximum suppression processing on the feature point set to realize feature point distribution homogenization; and inputting the optimized feature point set into a visual SLAM system to carry out feature matching, pose estimation and map optimization so as to realize high-precision pose calculation and map updating. The method is suitable for robot vision positioning and three-dimensional mapping work in a complex dynamic environment, and the precision and robustness of the system can be remarkably improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of computer vision and robot autonomous positioning mapping, and particularly relates to an improved GCNv2 feature point extraction based on deep learning. BACKGROUND

[0002] In the technical field of computer vision and robot autonomous positioning mapping (SLAM), a visual-based SLAM system is one of the core methods for realizing environment perception and pose estimation. Traditional visual SLAM systems usually rely on manually designed feature point extraction algorithms such as SIFT, SURF, ORB, etc. These algorithms analyze the image gray gradient, corner and local region texture to realize feature point detection and matching. However, in actual complex scenes, the traditional method has significant limitations. When the environment changes in light, scale, texture sparseness or dynamic object interference, the hand-crafted feature extraction algorithm often fails to maintain stable feature point detection effect, resulting in a decrease in tracking accuracy and mapping robustness of the system.

[0003] In recent years, with the development of deep learning technology, researchers have begun to try to introduce a convolutional neural network (CNN) into the feature extraction process, such as the SuperPoint, D2-Net, GCN, etc. methods, through end-to-end network learning to realize more robust feature point detection and description. These methods have significantly improved the accuracy and stability of feature extraction in static environments. However, in dynamic and complex scenes, deep learning methods still face problems such as uneven feature point distribution, large dynamic object interference and insufficient computational efficiency. Taking GCNv2 as an example, although it introduces a lightweight structure and a binary descriptor to improve the feature point matching efficiency, it still has problems such as feature point concentration, local redundancy and excessive response in dynamic object regions in multi-scale change scenes. In addition, the traditional non-maximum suppression (NMS) algorithm usually uses a fixed window to screen features, which cannot adaptively adjust the suppression range according to the image content, resulting in uneven spatial distribution of feature points and affecting the accuracy and stability of pose estimation.

[0004] In summary, the existing technology has the defects of insufficient robustness of feature point extraction in dynamic and multi-scale complex scenes, uneven distribution of feature points, and difficulty of adaptive optimization of traditional NMS algorithms. SUMMARY

[0005] To solve the defects of insufficient robustness of feature point extraction in dynamic and multi-scale complex scenes, uneven distribution of feature points and difficulty of adaptive optimization of traditional NMS algorithms in the existing technology, the technical scheme provided by the application is as follows: An improved GCNv2 feature point extraction method based on deep learning, comprising: the step of acquiring image frames continuously collected by a camera and constructing a multi-layer image pyramid, sequentially performing Gaussian blur and down-sampling processing on the original image to form a multi-scale image sequence, and outputting a multi-scale image set; the step of inputting the multi-scale image set into a GCNv2 feature extraction network, generating a feature probability map and a feature map through an encoding and decoding process, determining the feature point position according to the probability value after restoring to the original resolution through up-sampling, and generating a binary descriptor by combining the feature map and the binary network layer output, and outputting a feature point set containing feature point coordinates and descriptor information; the step of performing a range tree-based spatial adaptive non-maximum suppression processing on the feature point set, sorting the feature points according to the response intensity and constructing a range tree structure, determining the neighborhood range through dynamic adjustment of the search radius and binary search, removing the low-response feature points in the neighborhood, and outputting a uniformly distributed and high-quality feature point set; the step of inputting the uniformly screened feature point set into a visual SLAM system framework, performing feature matching, pose estimation and graph optimization, and realizing pose calculation and map updating through the corresponding relationship between the feature points in consecutive frames.

[0006] Further, in a preferred embodiment, the multi-layer image pyramid includes eight layers of image structures, and each layer of image is generated from the previous layer of image through Gaussian blur and scale down-sampling.

[0007] Further, in a preferred embodiment, the GCNv2 feature extraction network generates a feature probability map and a feature map through a lightweight convolutional encoding and decoding structure, and utilizes multi-layer convolutional feature fusion in the up-sampling process to improve the accuracy and robustness of feature point detection.

[0008] Further, in a preferred embodiment, the binary network layer is used to map the continuous numerical features in the feature map into binary feature descriptors, and generates a binary descriptor with a length of 256 bits after normalization processing.

[0009] Further, in a preferred embodiment, the range tree-based spatial adaptive non-maximum suppression processing includes sorting the feature points according to the response intensity, establishing a balanced binary range tree structure, dynamically adjusting the search radius, and performing neighborhood rejection operation.

[0010] Further, in a preferred embodiment, the visual SLAM system framework keeps the local mapping, loop detection and closed-loop optimization modules unchanged, and embeds the improved GCNv2 feature point extraction module into the front end of the system.

[0011] Based on the same inventive concept, the application also provides an improved GCNv2 feature point extraction device based on deep learning, comprising: The module for acquiring image frames continuously collected by a camera and constructing a multi-layer image pyramid, sequentially performing Gaussian blur and down-sampling processing on the original image to form a multi-scale image sequence, and outputting a multi-scale image set; The module for inputting the multi-scale image set into a GCNv2 feature extraction network, generating a feature probability map and a feature map through an encoding and decoding process, restoring the feature probability map to the original resolution through up-sampling, determining the feature point positions according to the probability values, and generating a binary descriptor by combining the feature map and the binary network layer output, and outputting a feature point set containing feature point coordinates and descriptor information; The module for performing a range tree-based spatial adaptive non-maximum suppression processing on the feature point set, sorting the response intensities and constructing a range tree structure, determining the neighborhood range through dynamic adjustment of the search radius and binary search, removing the low-response feature points in the neighborhood, and outputting a uniformly distributed and high-quality feature point set; The module for inputting the uniformly screened feature point set into a visual SLAM system framework, performing feature matching, pose estimation and graph optimization, and realizing pose calculation and map updating through the corresponding relationship between the feature points in consecutive frames.

[0012] Based on the same inventive concept, the present application also provides a computer storage medium for storing a computer program, which, when read by a computer, causes the computer to execute the method.

[0013] Based on the same inventive concept, the present application also provides a computer comprising a processor and a storage medium, which, when the processor reads the computer program stored in the storage medium, causes the computer to execute the method.

[0014] Based on the same inventive concept, the present application also provides a computer program product as a computer program, which, when executed, realizes the method.

[0015] Compared with the prior art, the technical solution provided by the present application has the following advantages: The present application introduces a multi-layer image pyramid structure, which effectively improves the scale invariance and detection robustness of the feature points. By performing multi-scale down-sampling and blur processing on the same frame image at different resolutions, the system can stably extract feature points in the case of significant scale difference between near and far objects, thereby avoiding the problem of feature point matching failure in the scale change scene of traditional ORB algorithm. Experimental results show that the feature points extracted by the multi-layer pyramid can maintain a high matching success rate between near and far targets, significantly improving the stability of the SLAM system in large scene pose estimation.

[0016] The scheme uses GCNv2 deep learning network for feature extraction, instead of traditional artificial feature detector, which significantly enhances the accuracy and robustness of feature point detection and description. The GCNv2 network obtains a probability map with semantic features and a 256-bit binary descriptor through end-to-end training, so that the feature points can maintain high discriminability and high repeatability in scenes with varying illumination, occlusion or sparse texture. Compared with traditional ORB features, the feature points extracted by GCNv2 have lower mismatch rate when dynamic objects exist, significantly reducing the pose drift problem caused by dynamic feature interference, thereby improving the accuracy and robustness of the entire SLAM system.

[0017] The scheme adopts a range tree-based spatial adaptive non-maximum suppression (NMS) algorithm to realize uniform distribution of feature points. The range tree structure can quickly search the neighborhood of feature points in multi-dimensional space, dynamically adjust the suppression range according to the response intensity, so that the selected feature points have high response values and can maintain uniform distribution on the image. Compared with the quadtree distribution algorithm used in ORB-SLAM2, this method has more accurate control of feature point density in local regions, avoiding the problems of feature point concentration or sparsity. Experimental results show that the NMS method based on range tree improves the uniformity of feature points by about 20% on average on the TUM dataset, significantly improving the tracking stability and mapping consistency of the system in dynamic complex scenes. Figure One

[0018] In addition, the improved GCNv2 feature point extraction module is combined with the ORB-SLAM2 framework to form an improved visual SLAM system that combines deep feature perception and traditional graph optimization structure. Based on maintaining the original local mapping and loop detection mechanism, the improved module provides higher quality feature input, significantly improving the trajectory accuracy and running robustness of the system in dynamic environments. Compared with the original ORB-SLAM2, the absolute trajectory error and relative pose error of the scheme are significantly reduced on the TUM RGB-D dataset, verifying that the integrated strategy significantly improves the accuracy and stability while maintaining real-time performance.

[0019] It is suitable for high-precision SLAM systems for robot visual positioning and three-dimensional mapping in complex dynamic environments. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 Flow chart for combining deep learning feature points with traditional visual SLAM; Figure 2 Comparison diagram of feature point extraction module; Figure 3 Image pyramid diagram; Figure 4 GCNv2 feature extraction network diagram; ​Figure 5 The feature point extraction effect comparison schematic diagram is shown in Figure 1. Among them, (a) is the ORB feature point extraction effect, (b) is the improved GCNv2 feature point extraction effect. Figure 6 The selected five data set images are shown in Figure 2. Figure 7 The image segmentation schematic diagram is shown in Figure 3. DETAILED DESCRIPTION

[0021] In order to make the advantages and benefits of the technical solutions provided by the present application more clear, the technical solutions provided by the present application will be further described in detail in combination with the drawings, and the specific embodiments are as follows: Embodiment one, the embodiment provides an improved GCNv2 feature point extraction method based on deep learning, comprising: The steps of acquiring the image frames continuously collected by the camera and constructing a multi-layer image pyramid, sequentially performing Gaussian blur and down-sampling processing on the original image to form a multi-scale image sequence, and outputting a multi-scale image set; The steps of inputting the multi-scale image set into the GCNv2 feature extraction network, generating a feature probability map and a feature map through the encoding and decoding process, restoring to the original resolution after upsampling, determining the feature point position according to the probability value and combining the feature map and the binary network layer output to generate a binary descriptor, and outputting a feature point set containing feature point coordinates and descriptor information; The steps of performing a range tree-based spatial adaptive non-maximum suppression processing on the feature point set, sorting and constructing a range tree structure according to the response intensity, determining the neighborhood range through dynamic adjustment of the search radius and binary search, removing the low-response feature points in the neighborhood, and outputting a uniformly distributed and high-quality feature point set; The steps of inputting the uniformly screened feature point set into the visual SLAM system framework, performing feature matching, pose estimation and graph optimization, and realizing pose calculation and map updating through the corresponding relationship between the feature points in the continuous frames.

[0022] The multi-layer image pyramid includes eight layers of image structures, and each layer of image is generated from the previous layer of image through Gaussian blur and scale down-sampling.

[0023] The GCNv2 feature extraction network generates a feature probability map and a feature map through a lightweight convolutional encoding and decoding structure, and utilizes multi-layer convolutional feature fusion in the upsampling process to improve the precision and robustness of feature point detection.

[0024] The binary network layer is used to map the continuous numerical features in the feature map into binary feature descriptors, and generates a binary descriptor with a length of 256 bits after normalization processing.

[0025] The range tree-based spatial adaptive non-maximum suppression processing includes sorting feature points according to response intensity, establishing a balanced binary range tree structure, dynamically adjusting a search radius, and a neighborhood rejection operation.

[0026] The visual SLAM system framework keeps the local mapping, loop closure detection, and loop closure optimization modules unchanged, and embeds the improved GCNv2 feature point extraction module in the front end of the system.

[0027] An improved GCNv2 feature point extraction device based on deep learning is also provided, comprising: A module for acquiring image frames continuously collected by a camera and constructing a multi-layer image pyramid, sequentially performing Gaussian blur and down-sampling processing on the original image to form a multi-scale image sequence, and outputting a multi-scale image set; A module for inputting the multi-scale image set into a GCNv2 feature extraction network, generating a feature probability map and a feature map through an encoding and decoding process, determining feature point positions according to probability values after up-sampling to restore the original resolution, and combining the feature map and a binary network layer output to generate a binary descriptor, and outputting a feature point set containing feature point coordinates and descriptor information; A module for performing range tree-based spatial adaptive non-maximum suppression processing on the feature point set, sorting according to response intensity and constructing a range tree structure, determining a neighborhood range through dynamic adjustment of a search radius and binary search, removing low-response feature points in the neighborhood, and outputting a uniformly distributed and high-quality feature point set; A module for inputting the uniformly screened feature point set into a visual SLAM system framework, performing feature matching, pose estimation, and graph optimization, and realizing pose calculation and map updating through the corresponding relationship between consecutive frame features.

[0028] A computer storage medium for storing a computer program is also provided, and when the computer program is read by a computer, the computer executes the method.

[0029] A computer comprising a processor and a storage medium is also provided, and when the processor reads a computer program stored in the storage medium, the computer executes the method.

[0030] A computer program product is also provided as a computer program, which, when executed, implements the method.

[0031] Embodiment Two, this embodiment is a further detailed description of the technical solution provided in Embodiment One, specifically: The application discloses an improved GCNv2 feature point extraction method based on deep learning and an implementation method thereof in a visual SLAM system.

[0032] Firstly, input processing is performed on continuous image frames collected by a camera. Since the depth of objects in a scene changes during the movement of the camera, the scale of the same object in different frames in the image is different. In order to ensure that the feature points in different scales can still be stably detected, an eight-layer image pyramid is constructed. Specifically, the original image is taken as the first layer, and image sequences with decreasing resolutions are generated layer by layer through Gaussian blur and downsampling operations. Each layer of image retains the overall structural information of the original image, while gradually reducing the resolution, so that different levels can extract global features of distant targets and detailed features of nearby objects. The multi-scale image set output by the image pyramid forms the input of the subsequent feature extraction network, providing scale invariance and multi-level feature information support for the system.

[0033] Secondly, the images of the multi-layer image pyramid are sequentially input into the improved GCNv2 feature extraction network. The network learns the global and local feature correlation of the image through deep convolution structure by referring to the SuperPoint structure, generates a low-resolution feature probability map and a feature map through encoding and decoding processing of the input image. Unlike the traditional ORB method based on gray gradient and corner response, GCNv2 can maintain the stability of feature point detection under the conditions of illumination change, motion blur and partial occlusion. The feature probability map output by the network is restored to the original resolution through upsampling, and the probability value of each pixel is used to determine the position of the potential feature point. On this basis, the binary descriptor corresponding to each feature point is extracted through matching with the feature vector in the corresponding region of the feature map and combining the output of the binary network layer. After normalization and sign function transformation processing, the descriptor generates a 256-bit binary vector, which is used for subsequent efficient matching. Compared with the traditional floating-point descriptor, the binary descriptor significantly reduces the computational complexity while maintaining the accuracy of similarity judgment, so that the system can complete large-scale feature matching under the condition of ensuring real-time. The output result of this step is a feature point set containing feature point coordinates, response intensity and descriptor information.

[0034] Subsequently, the feature point set is subjected to a range tree-based spatial adaptive non-maximum suppression processing. Traditional non-maximum suppression usually uses a fixed window to locally screen feature points, which can easily cause uneven distribution of feature points in different resolution images or regions with significant texture changes. Therefore, the method introduces a range tree data structure to achieve dynamic and uniform screening of feature points through spatial layering and fast query mechanism. The algorithm first arranges the feature points in descending order of response intensity and stores them in the range tree. Then, the search boundary is initialized according to the image size and the desired number of feature points, and the appropriate spatial range is determined through binary search. When a high-response feature point is encountered, the neighborhood feature points with a radius of the current search range are searched in the range tree centered on the point, and the lower-response points in the neighborhood are removed from the candidate set. This process dynamically balances the feature point density and distribution uniformity, making the final output feature points more evenly distributed across the image and avoiding excessive clustering in dynamic objects or rich texture areas. Compared with the quadtree uniformization strategy used by ORB-SLAM2, the method significantly improves the balance and global consistency of local feature point distribution.

[0035] Next, the feature point set optimized by the range tree NMS is used as input to replace the traditional ORB feature module and access the ORB-SLAM2 system framework. In this framework, the high-quality feature points generated by the improved module are used for feature matching, pose estimation, and key frame management. The system estimates the relative pose between adjacent frames using matched feature points, and performs graph optimization and loop detection in the global range. Due to the more uniform and high-precision description of the input feature points, the stability and accuracy of the pose estimation results are improved, especially in scenes with moving objects, the mismatch rate is significantly reduced, thereby avoiding system drift and repositioning failure.

[0036] Finally, the method is experimentally verified on a typical dynamic dataset. The experimental platform uses an Intel i7 processor and a GTX1050 GPU, and the test dataset is the TUM RGB-D public dataset. By comparing with the original ORB-SLAM2 system, the root mean square values of absolute trajectory error and relative pose error are calculated. The results show that the average error of the proposed scheme is significantly reduced in multiple scene conditions, the feature point distribution uniformity is improved by about 20%, and stable operation is maintained under conditions of light changes, occlusions, and dynamic object interference. The experimental results verify the practicality and robustness of the method in dynamic and complex environments.

[0037] In summary, the embodiment improves the uniformity of feature point detection, the accuracy of feature matching, and the stability of pose estimation through the cooperation of multi-scale feature extraction, deep learning network description, and spatial adaptive NMS. It has good real-time performance and generalization ability, and can be widely used in robot navigation, autonomous driving, augmented reality, intelligent monitoring, and other high-precision positioning and mapping tasks.

[0038] Embodiment three, in combination Figures 1-7 To illustrate the embodiment, the above-mentioned technical solutions are further described in detail through specific embodiments. An improved GCNv2 feature point extraction algorithm based on deep learning is proposed, and it is combined with the ORB-SLAM2 system. This algorithm combines image pyramid, GCNv2 deep learning network, and spatial adaptive non-maximum suppression (NMS) strategy based on range tree, which can more accurately extract feature points in multi-scale changes and complex environments, and improve the performance of visual SLAM system in complex scenes. The embodiment is verified on static and dynamic datasets, the main purpose is to show the effect and advantage after replacing the feature point extraction module. By applying the improved GCNv2 feature point extraction algorithm to the ORB-SLAM2 system, replacing its original ORB feature point extraction module, while keeping the local mapping and loop detection modules in ORB-SLAM2, an improved visual SLAM framework is formed, as shown in Figure 1 Through comparative experiments, the improvement of the improved system in feature extraction accuracy and robustness is verified.

[0039] Improved GCNv2 feature point extraction The overall process of the improved GCNv2 feature point extraction algorithm. This algorithm mainly improves the performance and robustness of traditional feature point extraction methods through three key steps. First, for each input image, an eight-layer image pyramid is constructed. Image pyramid not only can extract feature points with scale invariance, but also can increase the total number of detected feature points, thereby significantly improving the robustness of feature extraction. Then, the GCNv2 deep learning network is used to extract features from each layer of the image pyramid, obtaining the feature point probability map and the feature vector containing feature descriptor information. Finally, the non-maximum suppression (NMS) algorithm based on range tree is used to uniformly process the extracted feature points, so as to obtain effective feature points with higher feature point response value and uniform distribution. Figure 2 The comparison process of ORB-SLAM2 and the feature point extraction module proposed in the embodiment is shown. Through this improved feature point extraction algorithm, the embodiment not only effectively improves the robustness of the visual SLAM system in complex scenes such as dynamic scenes, but also enhances the effectiveness of feature points.

[0040] Image pyramid The motion of the camera causes the visual SLAM scene to constantly change, and the projection of objects in the image changes as the camera position changes, especially the change in depth information. When an object is closer to the camera, there is more detailed information in the image; when an object is farther away from the camera, the details in the image become blurred. Therefore, the motion of the camera can affect the detection and matching of feature points, especially in cases of large scale changes. In order to reduce this impact and ensure accurate feature matching between adjacent image frames, using the image pyramid method has become an effective strategy.

[0041] An image pyramid is constructed by multiple levels of images, and each level of image is obtained by downsampling and Gaussian blur processing of the original image at different scales. The structure of the image pyramid is shown in Figure 3 The resolution of each level of image gradually decreases, so that feature points can be extracted at different scales. Lower resolution images can help extract feature points of objects far from the camera, while higher resolution images can help detect details of close-range objects.

[0042] The main purpose of using an image pyramid is to enhance the scale invariance of feature points. This makes the image pyramid widely used in feature extraction and matching algorithms. The commonly used image pyramid structure is the Gaussian pyramid, in which each level of image is obtained by downsampling and Gaussian blur processing of the previous level of image. The Gaussian pyramid creates multiple layers of images with different resolutions by gradually reducing the image resolution, so that it can capture multi-scale features in the image. Through the construction of the image pyramid, feature points can be detected at multiple scales, greatly enhancing the robustness of feature point extraction. Combined with the GCNv2 feature extraction network, this method can effectively process features at different scales, improving the stability and accuracy of the visual SLAM system in dynamic and complex environments.

[0043] GCNv2 feature extraction network In the design of a feature point-based SLAM system, the selection of a feature extractor is one of the most important design choices. Although the widely used ORB-based SLAM performs well in standard scenes, it often fails to track in dynamic or other difficult scenes. In contrast, the SLAM using the GCNv2 network as a key point detector can better handle such scenes.

[0044] The initial GCN network is based on ResNet-50 and bidirectional recurrent network, however, this implementation requires powerful computing platform support because its computation is quite large. Inspired by SuperPoint which only detects single frame, GCNv2 feature extraction network is improved to only perform low resolution prediction on single frame, replace the ResNet-50 network structure used to extract feature points with SuperPoint network, and remove the recurrent neural network for predicting key point positions. At the same time, in order to greatly improve the efficiency while keeping the accuracy from declining significantly, GCNv2 retains the triple loss function based on real relative pose.

[0045] As shown in Figure 4 , after the input image is forward propagated, a low resolution probability map and a feature map containing multiple channels are generated. Then, through upsampling operation, the pixel probability is mapped to the original resolution. In the upsampled probability map, the probability value of each pixel is used to determine the exact position of the feature point in the probability map. Finally, according to the position of the feature point, the corresponding value in the feature map and the output of the binary network layer are combined to extract the binary descriptor corresponding to each feature point.

[0046] The descriptor network layer also uses encoding and decoding layers, and in order to accelerate feature matching, the features output by the shared network layer are converted into binary descriptors. In order to generate binary descriptors in the same format as ORB, GCNv2 adds a binary activation layer before output, and through normalization processing, binary 256-bit descriptors with high matching and robustness are obtained. The binary activation layer is essentially a nonlinear sign function, as follows:

[0047]

[0048] GCNv2 uses a similar triple loss function as GCN, instead of using the L2 distance between the coordinates of two feature points in the image to measure the difference between the feature vectors, GCNv2 uses the Hamming distance between the binary descriptors of the feature points as the distance measure between the feature vectors. This method is more suitable for the binary descriptors generated in GCNv2, allowing more efficient and accurate evaluation of the similarity between features. The triple loss function of GCNv2 is as follows:

[0049]

[0050] wherein, represents the Hamming distance between the binary descriptors corresponding to the key points.

[0051] Uniformization of NMS feature points based on range tree In dynamic SLAM systems, the distribution of feature points is of great significance. Dynamic objects usually have rich textures, making it easy to attract a large number of feature point detections. However, excessive reliance on feature points in these dynamic areas can lead to inaccurate pose estimation. To ensure high-precision pose estimation, feature points of dynamic objects must be effectively removed, but sometimes the remaining static feature points are not enough to support accurate pose estimation. In addition, dynamic objects may temporarily block static regions that were originally rich in texture, making it difficult to effectively utilize high-quality static feature points, thereby affecting the stability of the SLAM system and the accuracy of pose estimation. Therefore, ensuring the uniform distribution of feature points on the image is crucial, which is important for providing comprehensive environmental information, avoiding excessive reliance on volatile dynamic features, improving tracking robustness, and reducing computational burden.

[0052] Traditional keypoint extraction methods are usually based on local response, which often leads to excessive concentration of feature points in certain areas. Non-maximum suppression algorithms attempt to reduce this concentration phenomenon by retaining the strongest feature points. However, traditional NMS methods cannot guarantee the uniform distribution of feature points throughout the image. In addition, fixed-size NMS windows often struggle to adapt to different image resolutions or local feature variations. ORB-SLAM2 employs a quadtree structure to achieve feature point uniformization, although this method effectively avoids global feature point aggregation, feature points may still be overly concentrated in certain local areas.

[0053] To overcome these shortcomings, this study proposes a range tree-based spatial adaptive non-maximum suppression (NMS) algorithm for uniformization of GCNv2 features. Range trees are a data structure for efficiently processing multi-dimensional spatial data, belonging to a type of balanced binary tree (such as AVL trees or red-black trees). Their advantage lies in their ability to quickly retrieve all points within a specific spatial range, enabling efficient spatial range queries. In this algorithm, range trees are used to quickly identify and suppress low-quality key points near high-quality key points, ensuring that selected key points not only have high quality but also are uniformly distributed on the image. The main flow of the algorithm is as follows: (1) Sort the input feature point set by response strength from large to small; (2) Based on the sorted feature point set construct a range tree; (3) According to the image size and the desired number of feature points initialize the binary search boundaries; (4) Perform the following binary search steps to determine the appropriate search range : (a) Initialize the output set of feature points ; (b) Initialize the index set , mark all feature points as optional state; (c) Traverse each feature point in turn : If : i. Add the feature point to the index set ; ii. Query all feature points within a radius of centered at from the range tree; ; iii. Remove the feature points in from the index set ; (d) If the difference between the current number of feature points and the desired number of feature points satisfies the termination condition , end the binary search and output the set ; (5) If the above termination condition is not met, adjust the search range and continue step (4).

[0054] Through the above process, the algorithm can effectively ensure the uniform distribution of feature points and ensure the high quality of selected feature points. The binary search dynamically determines the search range , accurately controls the number of feature points, so that the feature points are neither sparse nor dense. This algorithm not only optimizes the distribution of feature points, but also significantly improves the computing efficiency, which can ensure the stability and efficiency of the system when dealing with complex scenes while maintaining high feature point quality.

[0055] Introduction of experimental environment and data set The proposed visual odometry algorithm is evaluated in dynamic environments through experiments. The experiment is conducted on a notebook computer with an Intel i7 CPU, 16GB of memory, and a GTX 1050 GPU, with Ubuntu 18.04 as the operating system.

[0056] To objectively evaluate the performance of the proposed algorithm, the experiments used the TUMRGB-D dataset, widely used in dynamic SLAM testing. Released in 2012 by the Computer Vision Group at the Technical University of Munich (TUM), this dataset is one of the commonly used benchmark datasets for evaluating visual odometry and SLAM algorithms. The TUM RGB-D dataset contains 39 different sequences, designed to provide a standardized testing environment for visual odometry and SLAM systems. Images were captured using a Microsoft Kinect depth camera at a frame rate of 30Hz and a resolution of 640×480 pixels. Each frame contains a color image and a corresponding depth image. Furthermore, the TUM RGB-D dataset provides ground truth trajectories of camera motion, obtained through a high-precision motion capture system using eight high-speed tracking cameras. By comparing these trajectories with the ground truth trajectories, the localization accuracy and robustness of the visual SLAM algorithm can be quantitatively evaluated. The dataset also includes automated tools for trajectory comparison and error measurement, facilitating systematic performance evaluation.

[0057] The TUM RGB-D dataset is categorized into multiple classes based on different use cases, including office environments and warehouses, covering visual SLAM tasks in various typical application scenarios. These scenarios are suitable for testing the performance of SLAM algorithms under different conditions, such as changes in lighting and the presence of dynamic objects. By comparing the trajectories generated by the SLAM system on this dataset with given real trajectories, the actual performance of the proposed algorithm in dynamic environments can be accurately evaluated, especially its effectiveness in feature point extraction, dynamic object handling, and pose estimation.

[0058] SLAM system performance evaluation indicators To quantitatively evaluate the differences in accuracy of tracking performance among different algorithms, the experiment used two error metrics: absolute trajectory error (APE) and relative pose error (RPE).

[0059] Absolute trajectory error (APE) is used to evaluate the difference between the experimentally estimated trajectory and the true trajectory for each frame. i The absolute trajectory error of a frame can be expressed as:

[0060] in, The true trajectory is in the th position. pose of each frame The trajectory estimated by the experiment is in the th... Pose of frame. The most commonly used evaluation method is to calculate the average value of the Euclidean distance between the estimated trajectory and the true trajectory, and further obtain the root mean square error (RMSE) statistics of APE under all frames, the specific calculation process is as follows:

[0061] Wherein, represents the total number of frames of the pose trajectory, and the pose representation method of Lie group and Lie algebra mapping is adopted. In addition to the root mean square error, the ATE index can also be evaluated by calculating the standard deviation (S.D.) to evaluate the absolute error level between the pose trajectories.

[0062] Relative pose error (RPE) is used to measure the attitude error between the estimated trajectory and the true trajectory of two adjacent frames in the SLAM algorithm. The first i The relative pose error of frame can be expressed as:

[0063] Wherein, is a fixed time interval, and the default value 1s is adopted in the embodiment. The relative pose error generally includes translation and rotation error parts, and the two error parts can be calculated respectively. The calculation expression of the root mean square error statistics of RPE considering only the translation error is:

[0064] In addition, the root mean square error statistics of RPE considering the rotation error can also be calculated separately. In addition to the root mean square error, the standard deviation (S.D.) of RPE can also be calculated to evaluate the relative error level between the pose trajectories.

[0065] In order to further reflect the effectiveness of the algorithm of the embodiment, the promotion amplitude of the algorithm needs to be calculated, which is defined as follows:

[0066] Wherein, represents the promotion amplitude value, represents the value of the algorithm or system of the embodiment, represents the value of the comparative algorithm or system.

[0067] Feature point distribution experiment and analysis Figure 5The qualitative comparison of the traditional ORB feature extraction and the GCNv2 feature extraction based on deep learning proposed in this section is shown in the figure. It can be seen from the comparison that the ORB feature extraction method can extract a large number of feature points in a scene with clear texture, but the distribution of these feature points shows obvious redundancy and aggregation. Especially when there are moving objects in the image (such as the person in the figure), the ORB feature points will concentrate on these dynamic objects. This phenomenon is a major challenge for SLAM systems, because the presence of moving objects can cause matching errors of feature points, leading to a decrease in positioning accuracy, or even system failure. If relying on traditional ORB feature points, it is likely that a large number of invalid feature points will emerge due to the influence of dynamic objects, while the effective static feature points are not enough for high-precision pose estimation. In contrast, the GCNv2 feature point extraction method based on deep learning is obviously superior to the traditional ORB feature extraction algorithm. As can be seen from the image, the distribution of GCNv2 feature points is more uniform, and the coverage of feature points in the entire image is more extensive and reasonable.

[0068] To further quantify the improved GCNv2 feature point uniform distribution performance, the present embodiment refers to the evaluation method of image feature point distribution uniformity, adopts the evaluation method of Zhu Haifeng et al., and selects 5 images in the TUM dataset for calculation, as shown in Figure 6 The present embodiment compares this method with the original ORB-SLAM2 feature extraction method. In this evaluation method, the image is divided into horizontal, vertical, left-up to right-down, left-down to right-up, and center to periphery directions, as shown in FIG. 7. After dividing the image in the above directions, the number of feature points in the 10 regions is counted, and then a description vector is generated. The standard deviation of the vector is defined as the uniformity of the feature point distribution on the image , and the calculation method is as follows:

[0069] In the formula, is the number of feature points in each region, is the average of the 10 feature point quantities.

[0070] To ensure the comparability of the uniformity results, it is necessary to ensure that the number of feature points extracted by ORB-SLAM2 and the method of the present embodiment is the same. Therefore, the parameter of the feature point extractor in this experiment is set to 1000. After calculating the feature point distribution uniformity of each image using the above formula, the present embodiment records the results in Table 1 and performs quantitative analysis.

[0071] As shown in Table 1, the method of the embodiment improves the average degree of feature point distribution uniformity by 20.13% compared with the traditional algorithm. This shows that the method of the embodiment is superior to ORB-SLAM2 in the uniformization ability of feature extraction, and the feature points extracted by the method of the embodiment can more comprehensively express image information.

[0072]

[0073] The above further describes the technical solutions provided by the present application through several specific embodiments, in order to highlight the advantages and benefits of the technical solutions provided by the present application. However, the above several specific embodiments are not used as a limitation on the present application, and any reasonable modifications and improvements, combinations and equivalent replacements of the embodiments, etc. based on the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. An improved GCNv2 feature point extraction method based on deep learning, characterized in that, include: The steps are: to acquire image frames continuously captured by the camera and construct a multi-layer image pyramid; to perform Gaussian blur and downsampling on the original images in sequence to form a multi-scale image sequence; and to output a multi-scale image set. The multi-scale image set is input into the GCNv2 feature extraction network. Through the encoding and decoding process, a feature probability map and a feature map are generated. After upsampling to restore the original resolution, the location of the feature points is determined according to the probability value. The binary descriptor is generated by combining the feature map with the output of the binarization network layer. The step of outputting a feature point set containing feature point coordinates and descriptor information is as follows: The steps are as follows: performing spatial adaptive nonmaximum suppression processing based on range tree on the feature point set, sorting by response intensity and constructing a range tree structure, determining the neighborhood range by dynamically adjusting the search radius and binary search, removing low-response feature points in the neighborhood, and outputting a uniformly distributed and high-quality feature point set. The set of feature points that has been homogenized and filtered is input into the visual SLAM system framework for feature matching, pose estimation and graph optimization. The pose calculation and map update are achieved through the feature correspondence between consecutive frames.

2. The improved GCNv2 feature point extraction method based on deep learning according to claim 1, characterized in that, The multi-layer image pyramid consists of eight layers of image structure, with each layer generated by Gaussian blurring and scale downsampling of the image above it.

3. The improved GCNv2 feature point extraction method based on deep learning according to claim 1, characterized in that, The GCNv2 feature extraction network generates feature probability maps and feature maps through a lightweight convolutional encoding and decoding structure, and improves the accuracy and robustness of feature point detection by using multi-layer convolutional feature fusion during the upsampling process.

4. The improved GCNv2 feature point extraction method based on deep learning according to claim 1, characterized in that, The binarization network layer is used to map continuous numerical features in the feature map into binary feature descriptors, which are then normalized to generate a 256-bit binary descriptor.

5. The improved GCNv2 feature point extraction method based on deep learning according to claim 1, characterized in that, Spatial adaptive nonmaximum suppression based on range tree includes sorting feature points by response intensity, establishing a balanced binary range tree structure, dynamically adjusting the search radius, and performing neighborhood removal operations.

6. The improved GCNv2 feature point extraction method based on deep learning according to claim 1, characterized in that, The visual SLAM system framework retains the local mapping, loop closure detection, and loop closure optimization modules, while embedding the improved GCNv2 feature point extraction module into the system front end.

7. An improved GCNv2 feature point extraction device based on deep learning, characterized in that, include: This module acquires continuously captured image frames from the camera and constructs a multi-layer image pyramid. It then sequentially performs Gaussian blur and downsampling on the original images to form a multi-scale image sequence and outputs a multi-scale image set. The multi-scale image set is input into the GCNv2 feature extraction network. Through the encoding and decoding process, feature probability maps and feature maps are generated. After upsampling to restore the original resolution, the location of feature points is determined according to the probability value. The feature map and the output of the binarization network layer are combined to generate binary descriptors. The module outputs a feature point set containing feature point coordinates and descriptor information. The module performs spatial adaptive nonmaximum suppression processing based on range tree on the feature point set, sorts the feature points by response intensity and constructs a range tree structure, determines the neighborhood range by dynamically adjusting the search radius and binary search, removes low-response feature points in the neighborhood, and outputs a uniformly distributed and high-quality feature point set. The set of feature points that has been homogenized and filtered is input into the visual SLAM system framework for feature matching, pose estimation and graph optimization. The module realizes pose calculation and map update through the feature correspondence between consecutive frames.

8. A computer storage medium for storing computer programs, characterized in that, When the computer program is read by the computer, the computer executes the method of claim 1.

9. A computer, comprising a processor and a storage medium, characterized in that, When the processor reads the computer program stored in the storage medium, the computer executes the method of claim 1.

10. A computer program product, as a computer program, is characterized by: When the computer program is executed, it implements the method of claim 1.

Images