Ground mobile robot passable area identification method based on 3D convolution and attention mechanism
Through the 3D convolution and attention mechanism method, the challenge of ground agents identifying passable areas in complex environments is solved, and the navigation information generation is achieved with high accuracy and reliability, suitable for dangerous operating environments and reducing computing resources Consumption.
Patent Information
- Application Number
- CN202411199116.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-11-19
AI Technical Summary
It is a challenge to accurately identify accessible areas in complex, uneven and rough environments, and prior art has difficulty effectively handling sparse lidar data, resulting in inaccuracies and potential dangers in navigation decisions.
The passable area recognition method of ground mobile robots using 3D convolution and attention mechanisms is used to process lidar point cloud data through voxelization, introduce residual modules and attention mechanisms, optimize sparse convolution networks, extract key feature information, and generate Map of accessible areas with high accuracy and reliability.
Improves the agent's perception ability of the environment and the accuracy of navigation information, and can quickly generate passable area maps in complex environments. It is suitable for search, rescue and bomb disposal tasks, reduces computing resource consumption, and enhances the efficiency of the algorithm. .
Smart Images

Figure CN118982813A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of ground intelligent machine vision, and in particular to a method for identifying traversable areas of a ground mobile robot using a 3D convolution and attention mechanism. Background Art
[0002] The autonomous navigation capability of intelligent agents is the key to their safe movement in complex environments. Autonomous navigation research is mainly divided into two directions: path planning and simultaneous localization and mapping (SLAM). Path planning ensures that the intelligent agent can find a safe and reliable path to the destination, while SLAM focuses on identifying the traversable area around the intelligent agent to provide a basis for navigation decisions. Although there have been a lot of research in the field of SLAM, most of these studies have focused on urban roads or indoor environments. Ground intelligent agents are widely used in dangerous environments such as search and rescue, bomb disposal, and extraterrestrial exploration. The ground in these environments is often uneven and extremely rough, so misidentification of traversable areas may have a serious impact on the navigation decisions of the intelligent agent and may even lead to fatal accidents. Therefore, it is crucial for intelligent agent systems to be able to accurately and robustly identify traversable areas. In order to improve the performance and robustness of intelligent agents in these environments, it is key to obtain as much environmental information as possible. 3D point cloud data collected by laser radar (LiDAR) provides more natural geometric information than 2D images, which helps to perceive the environment more accurately. In addition, agents are often required to operate continuously over a period of time, and changes in the environment are a factor that cannot be ignored. The system can use past temporal information as clues and constraints to improve overall performance. However, perceiving whether the terrain is passable from sparse LiDAR data is a challenge because complex off-road terrain may contain features such as rapidly changing surfaces, dense vegetation, and hanging branches. Therefore, ground agents must be able to infer the geometric and semantic content of their surroundings to determine which areas are passable and which are not, which is a problem that has not yet been fully solved.
[0003] The traversable area identification problem is formulated as a semantic terrain classification problem, where semantic and geometric features unify various semantic objects into a single traversable semantic body. From a semantic perspective, objects such as large rocks and tree trunks are not traversable, while gravel, grass, and low bushes are traversable by off-road agents. From a geometric perspective, hanging obstacles can be ignored, and objects of the same semantic class may have different traversability depending on obstacle height (e.g., tall bushes vs. short bushes). Therefore, semantic classes are grouped according to their traversability, while allowing their traversability level to be adjusted based on their specific geometry. Summary of the invention
[0004] The purpose of the present invention is to provide a traversable area recognition method for a ground mobile robot with a 3D convolution and attention mechanism, which is suitable for autonomous navigation of ground intelligent agents. The method uses 3D convolution and attention mechanisms to process three-dimensional point cloud information obtained by laser radar scanning. First, the acquired point cloud data is voxelized, and then the voxelized point cloud is recognized by an optimized sparse bird's-eye view network. A residual module is introduced between the submanifold convolution layer and the sparse convolution layer of the three-dimensional sparse convolution, and an attention mechanism module is added at the end of the point cloud feature extraction to improve the network's attention to key feature information. A traversable area recognition method for a ground intelligent agent with a 3D convolution and attention mechanism is designed to obtain a more accurate traversable area for the intelligent agent. The traversable area map generated by the algorithm has high accuracy and reliability, and is suitable for complex, uneven and rough dangerous environments, such as search, rescue and bomb disposal tasks. By extracting the features of useful information in the sparse bird's-eye view network of the three-dimensional laser radar point cloud information, the traversable area map generated can obtain a better recognition effect than the bird's-eye view network, providing accurate navigation information for the ground intelligent agent.
[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is: a method for identifying a traversable area for a ground mobile robot based on a 3D convolution and attention mechanism, the steps of which are:
[0006] Step 1: Build a traversable data set:
[0007] New traversal cost semantic labels are generated from the RELLIS-3D outdoor dataset and SemanticKITTI highway dataset of the lidar dataset, and the traversable area is divided into different risk levels. To address the problem that the edge data of the lidar point cloud is too sparse, the point cloud of each frame in the original dataset of RELLIS-3D and SemanticKITTI and its past and future t-frame scans are aggregated, and the aggregated point cloud is used to generate a complete and dense semantic label of traversability from the past to the future. In particular, the aggregated point cloud data is only used for label generation and is not used as input for training and prediction; based on dense labels, the traversability category is divided into three levels of traversability labels: no danger, low risk, and high risk, and semantic classes with similar costs are mapped to these three levels of traversability labels; traversability is considered to be different positions in the same dimension. Traversability is the same dimension in different dimensions.
[0008] Experiments are conducted on the RELLIS-3D outdoor dataset and the SemanticKITTI road dataset.
[0009] The RELLIS-3D dataset consists of sequences acquired by five sensors, using a ground mobile off-road vehicle to collect data in different scenarios, including runways, tarmacs, terrain, woodlands, bushes, pastures, and lakes; the sensors include an Ouster OS1 LiDAR, a 64-channel, 2048 horizontal resolution, 10Hz, 45° vertical field of view LiDAR, a Velodyne Ultra Puck, a 32-channel, 10Hz, 40° vertical field of view LiDAR, a Nerian Karmin2+Nerian SceneScan, a 10Hz 3D stereo camera, a Basler acA1920-50gc, 16mm / F18 EDMUND lens, image resolution 1920x1200, 10Hz color camera, GPS / IMU: Vectornav VN300 Dual AntennaGNSS / INS, 300Hz GPS, 100Hz IMU inertial navigation system, two computers, one for motion control of the agent and the other for data acquisition and sensor processing;
[0010] SemanticKITTI provides 11 sequences based on the KITTI Vision benchmark odometry dataset, collected by a standard car equipped with two color and two PointGrey Flea2, 10Hz, grayscale cameras with image resolution 1392×512, a Velodyne HDL-64E 3D laser scanner with 10Hz, 6410Hz, 64 lasers, 100m range, a GPS / IMU positioning unit with RTK correction signals, and a computer-run real-time database. The data scenes include urban traffic, residential areas, highways, and rural roads.
[0011] Step 2: Establish network architecture:
[0012] First, the input 3D lidar point cloud is voxelized into a sparse voxel grid, where each voxel can contain multiple points and has 4-dimensional features. Among them: x, y, z, r are four-dimensional coordinate axes, and n is the number of voxels; the sparse voxel grid is fed into a 3D sparse convolutional layer with a residual module to extract features, where only the z dimension is compressed; an attention mechanism module is applied to the extracted 2D feature map, and then a convolutional gated recurrent unit is used to aggregate the temporal information of the previous scan, and finally sent to the classifier for classification and filling in the blank areas not detected by LiDAR.
[0013] Step 3: Introduce residual sparse convolution:
[0014] In sparse convolution, the residual module is applied after the submanifold convolution with the same input and output dimensions, and the output feature map F is calculated as follows:
[0015] F=F'+Spc(F')
[0016] Where F' is the output feature map of the previous sparse convolution block, and Spc represents a series of operations in the current sparse convolution block. In 3D sparse convolution, submanifold sparse convolution only convolves the active sites in the input features to maintain sparsity and reduce the computational burden. Regular sparse convolution can effectively expand the receptive field, similar to 2D convolution. The residual module can enhance the representation ability of deep networks and solve the gradient vanishing problem. Although the use of ReLU activation function makes CNN immune to the gradient vanishing problem to a certain extent, its effect is still limited. The main reason for using ReLU activation function is that its derivative calculation is simple.
[0017] Step 4: Add attention mechanism:
[0018] The convolutional bottleneck attention model derives a one-dimensional channel attention map M c and a two-dimensional spatial attention map M based on F s , M c Both average pooling and max pooling are used to determine the meaningful features in F.
[0019] The spatial information of F is aggregated through average pooling and maximum pooling operations to generate and These two descriptors are then fed into a multilayer perceptron and calculated as follows:
[0020]
[0021] Where s represents the sigmoid function, W 1 and W 0 is the weight of the MLP;
[0022] With M c In comparison, M s Focusing on the location of useful information, the two complement each other. s The calculation formula is:
[0023]
[0024] Where s represents the sigmoid function, f 7×7 Represents a 2D convolution operation with a kernel size of 7×7.
[0025] Step 5: Generate a traversable area map:
[0026] 5.1) Point cloud data preprocessing: The original 3D LiDAR point cloud data obtained by the LiDAR scan is converted into a three-dimensional coordinate system, and preprocessing operations such as denoising and filtering are performed. The preprocessed point cloud data is voxelized to divide the continuous three-dimensional space into discrete voxels;
[0027] 5.2) Feature extraction: The voxelized point cloud data is input into the sparse voxel network architecture in step 2, where each voxel contains multiple points and has three-dimensional features. The attributes of each voxel are calculated. Then the sparse voxel network is fed into a series of 3D sparse convolutional layers with residual modules according to step 3 to extract features and compress the third dimension;
[0028] 5.3) Application of attention mechanism: Apply the attention mechanism module in step 4 to the extracted two-dimensional feature map. The convolution bottleneck attention model can generate a one-dimensional channel attention map and a two-dimensional spatial attention map, focusing on the key channels and spatial positions in the feature map respectively. Through the attention mechanism, the network can pay more attention to the key features in the point cloud data;
[0029] 5.4) Temporal information fusion: The convolutional gated recurrent unit is used to aggregate the temporal information of the previous scan and fuse it with the feature map of the current frame. Temporal information fusion helps the network understand changes in the environment;
[0030] 5.5) Classification and blank area filling: The fused feature map is input into the classifier to classify each voxel to determine whether it is a passable area. The classifier divides the voxels into different categories according to the definition of the passable area and the risk level. The network prediction results are used to fill the blank areas not detected by the lidar to generate a complete passable area map;
[0031] 5.6) Map visualization: The generated traversable area map is visualized, the traversable areas are represented by different colors, and superimposed on the original point cloud data. Finally, the classifier label is output and the point cloud data is classified into three levels.
[0032] The beneficial effects created by the present invention are as follows: the present invention relates to a method for identifying traversable areas of ground agents based on 3D convolution and attention mechanisms. By processing the three-dimensional point cloud data obtained by laser radar scanning, the present invention can more accurately capture the geometric features of the ground, thereby improving the agent's ability to perceive the environment. 3D point cloud data provides richer depth information, allowing the agent to understand its surrounding environment more realistically. Introducing an attention mechanism module in a 3D convolutional network enables the network to pay more attention to key features in point cloud data, such as changes in terrain, edges of obstacles, etc. This mechanism helps the network extract more useful information from sparse laser radar data, thereby improving the accuracy of traversable area identification. By adding a residual module between sparse convolutional layers, the present invention enhances the robustness of the network. The residual module can effectively alleviate the gradient vanishing problem of the network during training, so that the network can better learn complex feature representations, and maintain high recognition performance even when the ground is uneven or extremely rough. The method of the present invention can process laser radar data in real time and quickly generate a traversable area map, providing instant navigation information for ground agents. This is particularly important for intelligent agents that need to make decisions quickly in complex and dynamic environments. The traversable area identification method of the present invention is not only applicable to dangerous operating environments such as search, rescue, and bomb disposal, but also to broader fields such as extraterrestrial exploration. Its high accuracy and reliability enable ground-based intelligent agents to perform tasks safely and effectively in a variety of different complex environments. Through the optimization of sparse bird's-eye view networks and the introduction of attention mechanisms, the present invention reduces the consumption of computing resources and improves the efficiency of the algorithm. This is an important advantage for intelligent agent platforms with limited resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 The data collection vehicle and its settings for the RELLIS-3D and KITTI datasets;
[0034] Figure 2 The encoding part of the sparse voxel network architecture is shown in Figure 2.
[0035] Figure 3 Figure 2 is the decoding part of the sparse voxel network architecture;
[0036] Figure 4 is the residual sparse convolution graph;
[0037] Figure 5 The confusion matrix diagram of the two networks on different data sets. DETAILED DESCRIPTION
[0038] A traversable area recognition method for a ground mobile robot based on 3D convolution and attention mechanism, the steps are as follows:
[0039] Step 1: Build a traversable data set:
[0040] New traversal cost semantic labels are generated from the RELLIS-3D outdoor dataset and SemanticKITTI highway dataset of the lidar dataset, and the traversable area is divided into different risk levels. To address the problem that the edge data of the lidar point cloud is too sparse, the point cloud of each frame in the original dataset of RELLIS-3D and SemanticKITTI and its past and future t-frame scans are aggregated, and the aggregated point cloud is used to generate a complete and dense semantic label of traversability from the past to the future. In particular, the aggregated point cloud data is only used for label generation and is not used as input for training and prediction; based on dense labels, the traversability category is divided into three levels of traversability labels: no danger, low risk, and high risk, and semantic classes with similar costs are mapped to these three levels of traversability labels; traversability is considered to be different positions in the same dimension. Traversability is the same dimension in different dimensions.
[0041] Experiments are conducted on the RELLIS-3D outdoor dataset and the SemanticKITTI road dataset.
[0042] The RELLIS-3D dataset consists of sequences acquired by five sensors, using a ground mobile off-road vehicle to collect data in different scenarios, including runways, tarmacs, terrain, woodlands, bushes, pastures, and lakes; the sensors include an Ouster OS1 LiDAR, a 64-channel, 2048 horizontal resolution, 10Hz, 45° vertical field of view LiDAR, a Velodyne Ultra Puck, a 32-channel, 10Hz, 40° vertical field of view LiDAR, a Nerian Karmin2+Nerian SceneScan, a 10Hz 3D stereo camera, a Basler acA1920-50gc, 16mm / F18 EDMUND lens, image resolution 1920x1200, 10Hz color camera, GPS / IMU: Vectornav VN300 Dual AntennaGNSS / INS, 300Hz GPS, 100Hz IMU inertial navigation system, two computers, one for motion control of the agent and the other for data acquisition and sensor processing;
[0043] SemanticKITTI provides 11 sequences based on the KITTI Vision benchmark odometry dataset, collected by a standard car equipped with two color and two PointGrey Flea2, 10Hz, grayscale cameras with image resolution 1392×512, a Velodyne HDL-64E 3D laser scanner with 10Hz, 6410Hz, 64 lasers, 100m range, a GPS / IMU positioning unit with RTK correction signals, and a computer-run real-time database. The data scenes include urban traffic, residential areas, highways, and rural roads.
[0044] Step 2: Establish network architecture:
[0045] First, the input 3D lidar point cloud is voxelized into a sparse voxel grid, where each voxel can contain multiple points and has 4-dimensional features. Among them: x, y, z, r are four-dimensional coordinate axes, and n is the number of voxels; the sparse voxel grid is fed into a 3D sparse convolutional layer with a residual module to extract features, where only the z dimension is compressed; an attention mechanism module is applied to the extracted 2D feature map, and then a convolutional gated recurrent unit is used to aggregate the temporal information of the previous scan, and finally sent to the classifier for classification and filling in the blank areas not detected by LiDAR.
[0046] Step 3: Introduce residual sparse convolution:
[0047] In sparse convolution, the residual module is applied after the submanifold convolution with the same input and output dimensions, and the output feature map F is calculated as follows:
[0048] F=F'+Spc(F')
[0049] Where F' is the output feature map of the previous sparse convolution block, and Spc represents a series of operations in the current sparse convolution block. In 3D sparse convolution, submanifold sparse convolution only convolves the active sites in the input features to maintain sparsity and reduce the computational burden. Regular sparse convolution can effectively expand the receptive field, similar to 2D convolution. The residual module can enhance the representation ability of deep networks and solve the gradient vanishing problem. Although the use of ReLU activation function makes CNN immune to the gradient vanishing problem to a certain extent, its effect is still limited. The main reason for using ReLU activation function is that its derivative calculation is simple.
[0050] Step 4: Add attention mechanism:
[0051] The convolutional bottleneck attention model derives a one-dimensional channel attention map M c and the 2D spatial attention map M based on F s , Mc Both average pooling and max pooling are used to determine the meaningful features in F.
[0052] The spatial information of F is aggregated through average pooling and maximum pooling operations to generate and These two descriptors are then fed into a multilayer perceptron and calculated as follows:
[0053]
[0054] Where s represents the sigmoid function, W 1 and W 0 is the weight of the MLP;
[0055] With M c In comparison, M s Focusing on the location of useful information, the two complement each other. s The calculation formula is:
[0056]
[0057] Where s represents the sigmoid function, f 7×7 Represents a 2D convolution operation with a kernel size of 7×7.
[0058] Step 5: Generate a traversable area map:
[0059] 5.1) Point cloud data preprocessing: The original 3D LiDAR point cloud data obtained by the LiDAR scan is converted into a three-dimensional coordinate system, and preprocessing operations such as denoising and filtering are performed. The preprocessed point cloud data is voxelized to divide the continuous three-dimensional space into discrete voxels;
[0060] 5.2) Feature extraction: The voxelized point cloud data is input into the sparse voxel network architecture in step 2, where each voxel contains multiple points and has three-dimensional features. The attributes of each voxel are calculated. Then the sparse voxel network is fed into a series of 3D sparse convolutional layers with residual modules according to step 3 to extract features and compress the third dimension;
[0061] 5.3) Application of attention mechanism: Apply the attention mechanism module in step 4 to the extracted two-dimensional feature map. The convolution bottleneck attention model can generate a one-dimensional channel attention map and a two-dimensional spatial attention map, focusing on the key channels and spatial positions in the feature map respectively. Through the attention mechanism, the network can pay more attention to the key features in the point cloud data;
[0062] 5.4) Temporal information fusion: The convolutional gated recurrent unit is used to aggregate the temporal information of the previous scan and fuse it with the feature map of the current frame. Temporal information fusion helps the network understand changes in the environment;
[0063] 5.5) Classification and blank area filling: The fused feature map is input into the classifier to classify each voxel to determine whether it is a passable area. The classifier divides the voxels into different categories according to the definition of the passable area and the risk level. The network prediction results are used to fill the blank areas not detected by the lidar to generate a complete passable area map;
[0064] 5.6) Map visualization: The generated traversable area map is visualized, the traversable areas are represented by different colors, and superimposed on the original point cloud data. Finally, the classifier label is output and the point cloud data is classified into three levels.
[0065] Embodiment 1:
[0066] The method of the present invention was used for training and testing on two public datasets. The point cloud of the dataset was voxelized into a sparse voxel network, which was fed into a series of 3D sparse convolutional layers with residual modules to extract features and compress the third dimension. Then, the attention mechanism module was applied to the extracted two-dimensional feature map, and the convolutional gated recurrent unit was used to aggregate the temporal information of the previous scan, and finally it was sent to the classifier for classification and filling the blank areas not detected by LiDAR. The mean intersection over union (mIoU) was used as the evaluation indicator of the model prediction results for the classification results, and the following results were obtained.
[0067] Step S1: Results on the RELLIS-3D dataset;
[0068] In the left half of Table 1, we can see that a 3D convolution and attention mechanism for ground-based agent traversable area recognition has a 3.2% higher mIoU than BEVNet on the RELLIS-3D outdoor dataset (all values listed in the table are percentages). The intersection over union (IoU) of the free category is improved by 6.4%, and the classification performance of the low-cost category and the fatal category is also improved to varying degrees. In this experimental setting, the semantic class that is mapped to the medium-cost class the most is the shrub, which has fuzzy boundaries and cluttered locations in the dataset. It is speculated that the attention module does not correctly assign weights to these fuzzy edge features when extracting features from the LiDAR point cloud.
[0069] Step S2: Results on the SemanticKITTI dataset;
[0070] The right half of Table 1 shows the experimental results on the SemanticKITTI dataset, which consists of road scenes. Compared with the RELLIS-3D outdoor dataset, the classes in SemanticKITTI are mostly more regular, but some areas contain more small objects. As can be seen from the table, the proposed algorithm improves by about 1 percentage point in IoU and the lethal category Lethal. However, there is a 1.1% drop in the low-cost category Low-cost, which includes many low-grass areas in this scheme. The navigation characteristics of the low-cost class are very similar to the road characteristics in the free class, indicating that the attention mechanism does not effectively distinguish the edge features of these similar classes.
[0071] Table 1 IoU of different methods on two datasets
[0072]
[0073] Step S3: Generate a confusion matrix to compare the performance of the two networks on different data sets;
[0074] The prediction results for each pixel in all test samples are evaluated, such as Figure 5 As shown in , each row of the matrix represents the proportion of pixels in the corresponding category of the row that are predicted to be the corresponding category of each column. It should be noted that during training, areas not detected by the lidar are set to the "unknown" class to improve the visual consistency of the prediction results. Therefore, in the test results, in addition to these three semantic categories, some pixels are predicted to be "unknown". The "unknown" class is not very meaningful for the evaluation of network performance, so it is not included in the result statistics.
[0075] The present invention improves network performance by designing a residual sparse convolution module and incorporating an attention mechanism. The residual structure is applied to 3D sparse convolution to improve the stability of feature extraction. The attention mechanism is used to highlight the key features of the current frame. The effectiveness of this method in improving network performance is demonstrated through comparative experiments.
Claims
1. A traversable area recognition method for a ground mobile robot based on 3D convolution and attention mechanism, characterized in that: The steps are: Step 1: Build a traversable data set: Generate new traversal cost semantic labels from the RELLIS-3D outdoor dataset and SemanticKITTI highway dataset of the lidar dataset, divide the traversable area into different risk levels, and in order to solve the problem that the edge data of the lidar point cloud is too sparse, aggregate the point cloud of each frame in the original dataset of RELLIS-3D and SemanticKITTI and its past and future t-frame scans, and use the aggregated point cloud to generate complete and dense traversability semantic labels from the past to the future. In particular, the aggregated point cloud data is only used for label generation and is not used as input for training and prediction; based on the dense labels, divide the traversability category into three levels of traversability labels: no danger, low danger, and high danger, and map the semantic classes with similar costs to these three levels of traversability labels; traversability is considered to be different positions in the same dimension. Traversability is the same dimension in different dimensions; Step 2: Establish network architecture: First, the input 3D lidar point cloud is voxelized into a sparse voxel grid, where each voxel can contain multiple points and has 4-dimensional features. Where: x, y, z, r are four-dimensional coordinate axes, and n is the number of voxels; the sparse voxel grid is fed into a 3D sparse convolutional layer with a residual module to extract features, where only the z dimension is compressed; an attention mechanism module is applied to the extracted 2D feature map, and then a convolutional gated recurrent unit is used to aggregate the temporal information of the previous scan, and finally sent to the classifier for classification and filling in the blank areas not detected by LiDAR; Step 3: Introduce residual sparse convolution: In sparse convolution, the residual module is applied after the submanifold convolution with the same input and output dimensions, and the output feature map F is calculated as follows: F=F'+Spc(F') Where F' is the output feature map of the previous sparse convolution block, and Spc represents a series of operations in the current sparse convolution block. In three-dimensional sparse convolution, submanifold sparse convolution only convolves the active sites in the input features to maintain sparsity and reduce the computational burden. Regular sparse convolution can effectively expand the receptive field, similar to 2D convolution. The residual module can enhance the representation ability of deep networks and solve the problem of gradient disappearance. Although the use of ReLU activation function makes CNN immune to the influence of gradient disappearance to a certain extent, its effect is still limited. The main reason for using ReLU activation function is that its derivative calculation is simple; Step 4: Add attention mechanism: The convolutional bottleneck attention model derives a one-dimensional channel attention map M c and the 2D spatial attention map M based on F s , M c Both average pooling and max pooling are used to determine the meaningful features in F. The spatial information of F is aggregated through average pooling and maximum pooling operations to generate and These two descriptors are then fed into a multilayer perceptron and calculated as follows: Where s represents the sigmoid function, W1 and W0 are the weights of MLP; With M c In comparison, M s Focusing on the location of useful information, the two complement each other. s The calculation formula is: Where s represents the sigmoid function, f 7×7 represents a 2D convolution operation with a kernel size of 7×7; Step 5: Generate a traversable area map: 5.1) Point cloud data preprocessing: The original 3D LiDAR point cloud data obtained by the LiDAR scan is converted into a three-dimensional coordinate system, and preprocessing operations such as denoising and filtering are performed. The preprocessed point cloud data is voxelized to divide the continuous three-dimensional space into discrete voxels; 5.2) Feature extraction: The voxelized point cloud data is input into the sparse voxel network architecture in step 2, where each voxel contains multiple points and has three-dimensional features. The attributes of each voxel are calculated. Then the sparse voxel network is fed into a series of 3D sparse convolutional layers with residual modules according to step 3 to extract features and compress the third dimension; 5.3) Application of attention mechanism: Apply the attention mechanism module in step 4 to the extracted two-dimensional feature map. The convolution bottleneck attention model can generate a one-dimensional channel attention map and a two-dimensional spatial attention map, focusing on the key channels and spatial positions in the feature map respectively. Through the attention mechanism, the network can pay more attention to the key features in the point cloud data; 5.4) Temporal information fusion: The convolutional gated recurrent unit is used to aggregate the temporal information of the previous scan and fuse it with the feature map of the current frame. Temporal information fusion helps the network understand changes in the environment; 5.5) Classification and blank area filling: The fused feature map is input into the classifier to classify each voxel to determine whether it is a passable area. The classifier divides the voxels into different categories according to the definition of the passable area and the risk level. The network prediction results are used to fill the blank areas not detected by the lidar to generate a complete passable area map; 5.6) Map visualization: The generated traversable area map is visualized, the traversable areas are represented by different colors, and superimposed on the original point cloud data. Finally, the classifier label is output and the point cloud data is classified into three levels.
2. According to claim 1, a traversable area recognition method for a ground mobile robot based on a 3D convolution and attention mechanism is characterized by: In the step 1: Experiments were conducted on the RELLIS-3D outdoor dataset and the SemanticKITTI road dataset. The RELLIS-3D dataset consists of sequences acquired by five sensors, using a ground mobile off-road vehicle to collect data in different scenarios, including runways, tarmacs, terrain, woodlands, bushes, pastures, and lakes; the sensors include an Ouster OS1 LiDAR, 64 channels, 2048 horizontal resolution, 10Hz, 45° vertical field of view, a Velodyne UltraPuck, 32 channels, 10Hz, 40° vertical field of view, a Nerian Karmin2 + Nerian SceneScan, 10Hz 3D stereo camera, a Basler acA1920-50gc, 16mm / F18 EDMUND lens, image resolution 1920x1200, 10Hz color camera, GPS / IMU: Vectornav VN300 Dual Antenna GNSS / INS, 300HzGPS, 100Hz IMU inertial navigation system, two computers, one for motion control of the agent and the other for data acquisition and sensor processing; SemanticKITTI provides 11 sequences based on the KITTI Vision benchmark odometry dataset, collected by a standard car equipped with two color and two PointGrey Flea2, 10Hz, grayscale cameras with image resolution 1392×512, a Velodyne HDL-64E 3D laser scanner with 10Hz, 6410Hz, 64 lasers, 100m range, a GPS / IMU positioning unit with RTK correction signals, and a computer-run real-time database. The data scenes include urban traffic, residential areas, highways, and rural roads.
Citation Information
Cited By
Mobile robot visual navigation method based on voxel space attention mechanism
CN119559515A
Road understanding method and device under conditions of satellite denial and no prior map
CN120160615A
