Passable map construction method for unstructured environment

Abstract

The invention discloses a passable map construction method for an unstructured environment, and the method comprises the steps: projecting points in a laser radar coordinate system into a pixel coordinate system, associating point cloud data with semantic information, and obtaining a single-frame semantic point cloud under the pixel coordinate system; assigning a passable attribute, an impassable attribute or an unknown attribute to the point cloud according to the semantic category; in single-frame point clouds, for the point clouds with passable attributes, improved Gaussian process regression is adopted to carry out secondary refinement, and for the point clouds with unknown attributes, geometric features of the point clouds are analyzed to judge whether the area is passable; performing positioning by using the original single-frame point cloud to obtain a vehicle pose, and continuously inserting the single-frame semantic point cloud into a global point cloud map; and converting the global point cloud map into a three-dimensional grid map, updating the three-dimensional grid map, and finally mapping the three-dimensional grid map to a plane to form a two-dimensional occupation map. According to the method, the problem of difficulty in mapping caused by a complicated unstructured environment can be solved.

Description

Technical Field

[0001] The present invention belongs to the field of unmanned vehicle perception and mapping, and specifically relates to a method for constructing a traversable map for a vehicle in an outdoor unstructured environment. Background Art

[0002] In today's world, with the rapid development of automation and intelligent technologies, environmental perception technology is the primary step for robots and autonomous vehicles to perceive their surroundings in complex environments. This is particularly true in unstructured scenarios such as field exploration and disaster relief. Platforms must construct real-time traversability maps that incorporate terrain attributes and semantic information to support subsequent path planning. However, current research on road traversability analysis primarily focuses on urban roads with flat surfaces and clear road boundaries. Unstructured environments, characterized by diverse road attributes, varying levels of undulation, and less distinct road boundaries, pose significant challenges to traversability analysis. Cameras can provide high-resolution semantic information, but they primarily rely on deep learning models for semantic segmentation. However, deep learning models rely on training data for their performance, and may fail to correctly classify unseen objects such as rocks or scattered cardboard boxes. This leads to misclassification and makes it difficult to determine the true spatial distribution and traversability of objects.

[0003] Jing et al. fused radar camera information for pose estimation and 3D reconstruction. The camera was used to perform semantic segmentation of the environment, assigning semantic information to point clouds through spatial mapping, and optimizing the semantic map with the help of conditional random fields (CRFs). However, this method is computationally intensive and cannot meet real-time requirements. Chen et al. proposed an adaptive point cloud transformation based on height differences and a cascaded feature fusion structure, which achieved relatively good detection results on the KITTI dataset. However, it was mainly targeted at urban environments and no real-vehicle experiments were conducted. Guan extracted terrain features from RGB images and 3D point clouds and integrated these features into a global map. The terrain was represented as an elevation grid map and updated in real time. Each grid cell stored the average height value of the latest point cloud data, as well as slope, step height, and semantic information.

[0004] Although researchers have proposed many methods for detecting drivable areas based on sensor information fusion in the past, the construction of drivable maps in unstructured environments still faces the following problems:

[0005] (1) Semantic segmentation models cannot identify unknown objects in unstructured environments, cannot determine their categories, and cannot determine their passability;

[0006] (2) Unstructured environments are complex and changeable. The traditional DeepLabV3+ semantic segmentation network has problems such as insufficient segmentation, rough edges, and insufficient real-time performance.

[0007] (3) It is impossible to balance the accuracy and real-time performance of real-time image semantic segmentation, and it is also impossible to ensure the mapping efficiency of semantic information and three-dimensional maps, which affects the real-time construction of high-precision three-dimensional semantic maps. Summary of the Invention

[0008] In view of the shortcomings in the prior art, the present invention provides a method for constructing a traversable map for an unstructured environment.

[0009] The present invention achieves the above technical objectives through the following technical means.

[0010] A traversable map construction method for unstructured environments:

[0011] Build an improved DeepLabV3+ semantic segmentation network to perform semantic segmentation on images transmitted by the camera;

[0012] Project the points in the LiDAR coordinate system to the camera's pixel coordinate system, and use the nearest neighbor search algorithm to associate the point cloud data with semantic information to obtain a single-frame semantic point cloud in the pixel coordinate system. Then, assign the point cloud a passable attribute, an impassable attribute, or an unknown attribute based on the semantic category.

[0013] In a single-frame point cloud, for point clouds with passable attributes, an improved Gaussian process regression is used for secondary refinement to eliminate point clouds with visual recognition errors. For point clouds with unknown attributes, the geometric features of the point cloud are analyzed to determine whether the area is passable.

[0014] Combined with the Lego-LOAM algorithm, the original single-frame point cloud is used for positioning to obtain the vehicle pose, and the single-frame semantic point cloud is continuously inserted into the global point cloud map;

[0015] The global point cloud map is converted into a three-dimensional grid map. For grid updates, the Bayesian update method is used; for grid semantic updates, the maximum probability semantic update method is used. Then, mapping rules are established to map the three-dimensional grid map onto a plane to form a two-dimensional occupancy map.

[0016] Furthermore, the point cloud is assigned a passable attribute, an impassable attribute, or an unknown attribute according to the semantic category: the point cloud with semantic information of buildings, shrubs, people, and background is assigned an impassable attribute, and the point cloud with semantic information of grass, paved roads, and dirt roads is assigned a passable attribute. For objects that cannot be recognized by the improved DeepLabV3+ semantic segmentation network, an unknown attribute is assigned.

[0017] Furthermore, the specific process of the secondary refinement is as follows:

[0018] Voxel grid filtering is used to downsample the point cloud with passable attributes;

[0019] The road area is divided into several sub-areas Sector (s1, s2, ..., sj) and parallelized Gaussian process regression modeling is performed;

[0020] Select ground seed points in each sub-region Sector (s1, s2, ..., sj);

[0021] Use seed points to build an initial Gaussian process regression model;

[0022] Add other non-seed points in the sub-region that meet the constraints to the initial Gaussian process regression model, and re-optimize the Gaussian process regression model, and iterate continuously until no new points are added;

[0023] The non-seed points in the sub-region that meet the constraints are classified as the passable area point cloud, while the points not accepted by the Gaussian process regression model are classified as the inpassable area point cloud.

[0024] Furthermore, the seed point selection criteria include:

[0025] For height features, point clouds with a height value z close to the radar installation height h0 are preferred, that is, satisfying |z+h0|≤Δh, where Δh represents the height tolerance threshold;

[0026] Distance feature, select the point cloud that is closer to the vehicle's current position d, that is, satisfying d≤d_max, where d_max represents the maximum distance threshold.

[0027] Furthermore, the constraints include:

[0028] Variance Constraint: Var k <Model th , among which Var k Indicates the variance of the point in the height z direction, Model th is the preset threshold.

[0029] Furthermore, the constraints include:

[0030] Distance constraint: Among them, Mean k is the mean value of the model in the z direction, δ is the noise parameter, Dist th is the distance threshold, and z represents the height value of the non-seed point.

[0031] Furthermore, the constraints include:

[0032] Normal vectors are consistent: |cos(θ2)|<|cos(θ zmax )| and|cos(θ y )|>|cos(θ ymax)|, where cos(θ z ) represents the cosine value of the angle between the normal vector of the point and the Z axis of the vehicle's horizontal coordinate system, cos(θ y ) represents the cosine value of the angle between the normal vector of the point and the Y axis of the horizontal coordinate system of the vehicle body, cos(θ zmax ) represents the maximum cosine threshold of the angle between the normal vector of the point and the Z axis, cos(θ ymax ) represents the cosine threshold of the maximum angle between the normal vector of the point and the Y axis.

[0033] Furthermore, the improved DeepLabV3+ semantic segmentation network adopts an encoder-decoder structure. The encoder part uses a lightweight MobileNetV3 as the backbone network to replace the traditional Xception network. The decoder part adopts a multi-level feature fusion strategy. The features extracted by the MobileNetV3 backbone network are adjusted in number of channels and then spliced ​​with the high-level features processed by the CBAM attention module.

[0034] Furthermore, the three-dimensional grid map is mapped onto a plane, specifically by screening grids with a height of -1m to 1.5m and a traffic attribute of an impassable area for mapping.

[0035] The beneficial effects of the present invention are:

[0036] (1) This paper proposes a real-time semantic segmentation method based on lightweight improvement. The original backbone network Xception is changed to MobileNetV3 through the DeepLabV3+ network, and the CBAM attention module is embedded. While maintaining the lightweight of the model, the segmentation accuracy is improved to meet the real-time requirements of three-dimensional semantic mapping; by establishing semantic-geometric consistency constraints, the accuracy of three-dimensional semantic map construction is effectively guaranteed.

[0037] (2) To address the problem of semantic segmentation misjudgment, the present invention first generates candidate regions based on the semantic segmentation results, and then uses Gaussian process regression to perform probabilistic modeling on the semantic point cloud of the candidate regions to eliminate abnormal regions;

[0038] (3) Since the semantic segmentation model is highly dependent on the “white list” during training, for unrecognizable areas, the present invention integrates the geometric features of the point cloud to perform accessibility reasoning on the area. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the technical solution.

[0040] Figure 1 Schematic diagram of the framework of the system for constructing a traversable map in an unstructured environment according to the present invention;

[0041] Figure 2 This is a schematic diagram of the improved DeepLabV3+ network structure described in the present invention;

[0042] Figure 3 Schematic diagram of the Gaussian process regression secondary refinement process of the present invention;

[0043] Figure 4 This is a partially enlarged view of the normal vector of the unknown object point cloud according to the present invention;

[0044] FIG5( a ) is a diagram of the actual scene of scene 1 according to the present invention;

[0045] FIG5( b ) is a three-dimensional accessibility grid map of scene 1 according to the present invention;

[0046] Figure 5(c) is a two-dimensional traversable map of scene 1 according to the present invention;

[0047] FIG6( a ) is a partial magnified image of scene 1 RGB according to the present invention;

[0048] FIG6( b ) is a partially enlarged view of semantic segmentation in scene 1 according to the present invention;

[0049] FIG7( a ) is a diagram of the actual scene of scenario 2 according to the present invention;

[0050] FIG7( b ) is a three-dimensional accessibility grid map of scene 2 according to the present invention;

[0051] FIG7( c ) is a two-dimensional traversable map of scene 1 according to the present invention. DETAILED DESCRIPTION

[0052] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the protection scope of the present invention is not limited thereto.

[0053] A framework diagram of a traversable map construction method for unstructured environments is shown in the figure below: Figure 1 As shown in the figure. First, an improved DeepLabV3+ semantic segmentation model is designed, using MobileNetV3 as a lightweight backbone network. The CBAM attention mechanism is introduced in the decoder, and the boundary feature extraction capability is improved by optimizing the parameters of the ASPP module. Secondly, a semantic point cloud is generated based on the semantic segmentation results to achieve a rough extraction of traversable areas. The potential traversable areas are then secondary optimized using Gaussian process regression. For areas that the visual semantic segmentation model fails to identify, the geometric features of the point cloud are used to supplement the traversability judgment.

[0054] The specific steps include:

[0055] S1: Perform semantic segmentation on images transmitted by the camera by building an improved DeepLabV3+ semantic segmentation network model;

[0056] S2: After semantic segmentation is completed, the points in the LiDAR coordinate system are projected into the pixel coordinate system based on the coordinate relationship between the LiDAR and the camera. A nearest neighbor search algorithm is then used to associate the point cloud data with the semantic information, resulting in a single-frame semantic point cloud in the image pixel coordinate system. The point cloud is then assigned a passable, impassable, or unknown attribute based on the semantic category.

[0057] S3: In a single-frame point cloud, for point clouds with traversable attributes, an improved Gaussian process regression is used to perform secondary refinement to eliminate point clouds with visual recognition errors;

[0058] S4: In a single-frame point cloud, for point clouds with unknown attributes, the geometric features of the point cloud are analyzed to determine whether the area is passable;

[0059] S5: Combined with the Lego-LOAM algorithm, the original point cloud is used for positioning to obtain the vehicle pose, and the single-frame semantic point cloud is continuously inserted into the global point cloud map;

[0060] S6: After obtaining the global point cloud map, it is converted into a three-dimensional grid map. For grid updates, the Bayesian update method is used to update the occupancy probability of the grid map based on the vehicle's current posture state and observation values ​​and the existing grid map probability at the previous moment. For grid semantic updates, the maximum probability semantic update algorithm is used. Finally, by establishing specific mapping rules, the three-dimensional grid map is mapped to a plane to form a two-dimensional occupancy map.

[0061] In step S1, the present invention proposes an improved DeepLabV3+ semantic segmentation network architecture based on deep learning, which achieves a better balance between computational efficiency and segmentation accuracy through multi-level optimization design. The improved DeepLabV3+ semantic segmentation network structure is as follows: Figure 2 As shown in the figure, the network adopts an encoder-decoder structure, and uses a lightweight MobileNetV3 as the backbone network in the encoder part to replace the traditional Xception network, which significantly reduces the computational complexity of the model.

[0062] In the encoder, the input image first passes through the four feature extraction stages of MobileNetV3, achieving downsampling of 1 / 2, 1 / 4, 1 / 8, and 1 / 16, respectively. The 1 / 2 and 1 / 4 downsampling stages primarily extract low-level edge and texture features, which retain rich spatial detail information; the 1 / 8 and 1 / 16 downsampling stages focus on extracting high-level semantic features, increasing the receptive field to understand more complex scene content. Furthermore, the dilated convolutional structure is retained in the last two downsampling stages (1 / 16 and 1 / 8). This design effectively expands the receptive field without increasing the number of parameters, helping to capture a wider range of contextual information. At the end of the encoder, the atrous spatial pyramid pooling (ASPP) structure is retained. This structure captures multi-scale contextual information through four parallel branches (including three dilated convolutions with different dilation rates and a global average pooling). This design is particularly beneficial for processing objects of different sizes in the scene.

[0063] In the decoder, this architecture employs a multi-layer feature fusion strategy. Features extracted from the MobileNetV3 backbone network first undergo 1×1 convolution to adjust the number of channels before being concatenated with high-level features processed by the CBAM attention module. The ASPP module uses convolutions with varying dilation rates to capture features at different scales. However, when the dilation rate is too high, some image details and edge information can be lost, leading to missed detections, unclear edges, and inaccurate edges in the drivable area segmentation task. Incorporating the CBAM module in the decoder effectively avoids these issues. The CBAM module consists of two submodules: the Channel Attention Module and the Spatial Attention Module. The Channel Attention Module first evaluates the importance of each channel in the feature map and captures inter-channel dependencies through global average and max pooling. The Spatial Attention Module further emphasizes important regions in the spatial dimension. The collaborative work of these two modules significantly improves the network's ability to represent key features. The concatenated features undergo 3×3 convolution for feature fusion and refinement, and are finally restored to the original resolution through 4x upsampling. This improved DeepLabV3+ semantic segmentation network structure significantly reduces the computational complexity through the collaborative optimization of deep separable convolution and attention mechanism, while ensuring the segmentation accuracy, especially for small-scale targets, and avoiding missed detections and unclear and inaccurate edges.

[0064] To train the improved DeepLabV3+ semantic segmentation network, vehicle-mounted cameras were used to capture video streams from unstructured environments under varying viewpoints, weather conditions, and lighting intensities. Images were captured at fixed frame intervals. To ensure a sufficient number of images, open-source datasets of unstructured environments were also introduced to supplement the dataset. Objects in the captured images were annotated using Labelme software, and the annotated images were randomly divided into training, validation, and test sets in an 8:1:1 ratio. During the training phase, the improved DeepLabV3+ semantic segmentation network was trained using the PyTorch deep learning framework, using the cross-entropy loss function and the Dice loss function to measure the model's prediction output. After each training cycle, the model's performance was evaluated using the validation set. Based on the validation set evaluation results, hyperparameters were adjusted and optimized to further improve model performance. The trained neural network model then assigned semantic labels and label probabilities to each pixel in the image data.

[0065] In step S2, to impart semantic information about road attributes to the point cloud, filter out point clouds outside the camera's field of view, and reduce the amount of subsequent point cloud input, the image segmentation results need to be converted into semantic point clouds based on the sensor coordinate system transformation relationship, and the point clouds within the traversable area are roughly extracted through semantic mapping. The relative pose of the lidar and camera can be expressed as a rotation matrix R and a translation vector T. According to equation (1), the points in the lidar coordinate system can be projected into the camera coordinate system. Then, a nearest neighbor search algorithm is used to associate the point cloud data with the semantic information, thereby obtaining a semantic point cloud in the image pixel coordinate system.

[0066]

[0067] Among them, (u, v) is the image coordinate system, (x, y, z) is the world coordinate system, (f u , f v , u0, v0) are the intrinsic parameters of the camera, and (R, T) are the extrinsic parameters of the camera and radar.

[0068] Inconsistent sampling frequencies between sensors result in misalignment of information sampled from different sensors at the same moment, impacting data fusion accuracy. Therefore, sensor time synchronization is necessary. Compared to cameras, lidar has a lower sampling frequency and longer cycle time, making it a suitable soft synchronization benchmark, complementing frames with high-frequency image and IMU data. A neighboring time interval δ is set, and each frame in the point cloud data is iterated over to obtain the timestamp t of the current frame's point cloud data. Within the semantically segmented image, the nearest sampling point with a timestamp within the range t±δ is searched. Once the nearest sampling point is found, the point cloud data, image data, and IMU data are marked as synchronized, completing processing for the current frame. If no matching image or IMU data is found within the range t±δ, the time interval δ is increased by n times the original value, and the search for the nearest sampling point is repeated until matching image and IMU data is found or the preset maximum search range is reached. This adaptive method effectively addresses temporal skew while maintaining computational efficiency through bounded iterative optimization.

[0069] After the semantically segmented image undergoes spatial mapping and temporal synchronization, a single-frame semantic point cloud is generated, and attributes are assigned based on its semantic information. Specifically, point clouds containing semantic information about buildings, shrubs, people, and backgrounds are assigned an inaccessible attribute; point clouds containing semantic information about grass, paved roads, and dirt roads are assigned a traversable attribute; and objects that cannot be identified by the improved DeepLabV3+ semantic segmentation network model are assigned an unknown attribute.

[0070] In step S3, although visual semantic segmentation can identify most of the drivable area, some low vegetation or small obstacles near the road boundary may be mistakenly classified as drivable areas. Therefore, based on the above problems, a secondary refinement of the drivable area based on Gaussian process regression is proposed. In this research framework, the core goal of Gaussian process regression modeling is to learn the spatial distribution characteristics of known road point cloud data, establish a spatial probability model, and predict the probability distribution of the point cloud belonging to the road area. This process can be formally expressed as formula (2):

[0071] f(x)~GP(m(x),k(x,x')) (2)

[0072] Gaussian process regression describes the distribution of data by defining a mean function m(x) and a covariance function k(x,x'). In this study, the radial basis function is used as the covariance function of Gaussian process regression. The specific form of the radial basis function is shown in formula (3):

[0073]

[0074] Among them, σ f is the signal variance, which controls the amplitude of the function; is a length scale parameter that controls the smoothness of the function; ||x-x' || is the Euclidean distance between points x and x'.

[0075] The specific process of the secondary refinement is as follows: firstly, the point cloud with the attribute of passable is subjected to down-sampling processing by using voxel grid filtering, and the data size is significantly reduced by aggregating the point cloud in a specific size space into a representative point; secondly, the road area is divided into a plurality of sub-regions Sector (s1, s2,..., sj), and parallelized Gaussian process regression modeling is performed, which reduces the dimension of the covariance matrix and reduces the computational complexity from global to local. Then, the ground seed points are selected in each sub-region Sector (s1, s2,..., sj), and the selection of the seed points is mainly based on the following two key features: (1) height feature, the point cloud with the height value z close to the installation height h0 of the radar (|z+h0|≤Δh) is preferentially selected, and the point cloud usually corresponds to the road surface; (2) distance feature, based on the observation characteristics of the sensor, the point cloud with a distance d close to the current position of the vehicle (d≤d_max) is selected, and the data has higher measurement accuracy and lower misclassification probability; wherein, Δh represents a height tolerance threshold for controlling the allowed deviation range of the selected seed points, and d_max represents a maximum distance threshold. By establishing double constraint conditions, the method can effectively select the seed point set P seed ={p1,p2,...,pn} that is most representative of the road area, providing high-precision initial input for subsequent Gaussian process regression modeling, and significantly improving the robustness and computational efficiency of road area recognition.

[0076] Based on formula (2), the initial Gaussian process regression model f k (x) is constructed using the seed points. The other non-seed points in the sub-region are evaluated to determine whether they belong to the passable region or the impassable region. The evaluation is based on the following three constraint conditions:

[0077] (1) Variance constraint

[0078] Var k <Model th (4)

[0079] wherein, Var k represents the variance of the point in the height z direction, and Model th is a preset threshold. The constraint is used to determine whether the point is close to the distribution of the Gaussian process regression model.

[0080] (2) Distance constraint

[0081]

[0082] wherein, Mean kis the mean value of the model in the z direction, δ is the noise parameter, Dist th is the distance threshold, and z represents the height value of the non-seed point. This constraint is used to determine whether the point is within the confidence interval of the model.

[0083] On a flat road surface, the height variation is small, and the model variance Var k is also small. Therefore, Model th is set to be small to improve the recognition accuracy of the model for road point clouds. A smaller Model th value can more strictly filter out point clouds that meet the road model, thereby reducing misjudgments. On a complex road surface, the height variation is large, and the model variance Var k is also large. Therefore, Model th is set to be large to adapt to the large height variation. A larger Model th value can more leniently accept some point clouds with large height variations, thereby reducing the situation of misjudging as non-road points. Similarly, the setting of Dist th value is also the same. The type of the road surface on which the vehicle travels is obtained through visual semantic segmentation, and Model th , Dist th are assigned values for different road surfaces, as shown in Table 1.

[0084] Table 1 Size of Model th and Dist th for different road surfaces

[0085]

[0086] (3) Consistent normal vector

[0087] |cos(θ z )| < |cos(θ zmax )| and |cos(θ y )| > |cos(θ ymax )| (6)

[0088] where cos(θ z ) and cos(θ y ) represent the cosine values of the angle between the normal vector of the point and the Z axis and the Y axis (vehicle body horizontal coordinate system), respectively, cos(θ zmax ) represents the maximum cosine threshold of the angle between the normal vector of the point and the Z axis, and cos(θ ymax ) represents the maximum cosine threshold of the angle between the normal vector of the point and the Y axis. This constraint is used to exclude points that meet the distribution but do not belong to the road (such as vegetation).

[0089] The non-seed points that meet the above constraints are added to the initial Gaussian process regression model, and the model is re-optimized. This process is iterated continuously until no new points are added to the model. After optimization, the points in the sub-region that meet the constraints are classified as passable area point clouds, while the points that are not accepted by the model are classified as inaccessible area point clouds. The voxel downsampling and sub-region division strategy significantly reduces the computing time, making this method suitable for real-time applications. The incremental optimization process can gradually adjust the model, improve the modeling accuracy of the road point cloud, and effectively distinguish between roads and obstacles. The algorithm flow is as follows: Figure 3 shown.

[0090] In step S4, for areas not visually identified, the geometric features of the point cloud corresponding to the unknown area are extracted to determine whether the area is traversable. The local normal vector, roughness index, and tilt angle are calculated to comprehensively describe the potential geometric information associated with each point in the point cloud, which can be used to analyze complex scenes such as ramps and gullies. Furthermore, the estimation of the local descriptor of the uneven point not only considers the geometric properties of the points within the neighborhood, but also indirectly considers the geometric properties of the points in the neighborhood of each point, improving the accuracy and robustness of traversability detection.

[0091] First, select a point P in the point cloud with unknown attributes q , whose neighborhood is represented by P q，m (m is the number of points in the neighborhood), and the neighborhood centroid is calculated using formula (7).

[0092]

[0093] Then, the covariance matrix of its neighborhood is constructed using formula (8):

[0094]

[0095] Finally, the covariance matrix C is decomposed to obtain the minimum eigenvalue, and the eigenvector corresponding to the minimum eigenvalue is point P q Through its neighborhood P q，m Construct the normal vector, Figure 4 It is the point cloud normal vector map for visual recognition of unknown objects. Define point P q The local descriptor is U(P q ,P q,m )={n q,x ,n q,y ,n q,z ,l q,m}, where the definitions of the elements are as shown in formula (9) and formula (10):

[0096]

[0097] Where n kis the normal vector of each point, which can be obtained by equations (7) and (8); x, y, and z are the unit vectors of the three axes of the vehicle's horizontal coordinate system; n q,x 、n q,y 、n q , respectively represent P q Through its neighborhood P q，m Construct the components of the normal vector on the three axes (X, Y, Z) of the vehicle's horizontal coordinate system. q The sum of the normal vectors of the points in the neighborhood is n q,m , provides global orientation information of the local surface; and l q,m Then the neighborhood P is evaluated q，m The roughness of the surface can be interpreted as the local roughness index.

[0098] Based on the local descriptor of the point cloud, the local features of the point cloud are further extracted q,z 、f q,y .f q,z 、f q,y The specific explanation is as follows:

[0099] f q,z :Represented as point P q The angle between the normal vector of the point and its neighborhood points and the projection on the OXY plane of the vehicle's horizontal coordinate system and the Z axis represents the longitudinal slope.

[0100]

[0101] f q,y :Represented as point P q The angle between the normal vector of the point and its neighborhood points and the projection on the OXZ plane of the vehicle's horizontal coordinate system and the Z axis represents the slope of the cross slope.

[0102]

[0103] Get l q,m 、f q,z and f q,y After three parameters, point P is determined by establishing specific constraints. q The traffic attributes and constraints are shown in formula (13).

[0104] l q,m <l max And f q,z <-α1,f q,z >α2 and |f q,z |>β max (13)

[0105] Among them, l max is the maximum roughness index threshold, α1 is the maximum longitudinal slope upslope angle, α2 is the maximum longitudinal slope downslope angle, βmax is the maximum cross slope angle.

[0106] In step S5, Lego-LOAM receives the raw laser point cloud (not semantically segmented) to realize vehicle pose estimation; Lego-LOAM is a lightweight laser radar positioning and mapping algorithm, which compensates for point cloud motion distortion by fusing IMU data, and segments ground and non-ground point clouds to extract edge and plane features. A two-step optimization strategy is adopted: first, the pitch angle, roll angle and vertical displacement are optimized using ground points, and then the horizontal displacement and yaw angle are optimized through non-ground points, while combining the initial attitude information provided by the IMU to accelerate convergence. High-precision pose estimation is realized through inter-frame matching and local map optimization, and the IMU data is also used for motion prediction and denoising. The key innovation of the Lego-LOAM algorithm lies in feature classification, step-by-step ICP and IMU fusion, which significantly reduces the computational complexity while ensuring accuracy, realizing real-time positioning in complex environments. The semantic point cloud (from S2-S4) is projected from the radar coordinate system to the global coordinate system through the pose transformation matrix output by Lego-LOAM.

[0107] In step S6, the grid occupancy probability is updated. At any data collection time t, the occupancy probability value of the i-th grid is updated according to Bayes' theorem, as shown in equation (14):

[0108] L(m i |z 1:t )=L(m i |z t )-L(m i )+L(m i |z 1:t-1 ) (14)

[0109] In the formula, m i ∈{0,1} represents the binary occupancy state of grid i; z 1:t represents the observation of the environment from the initial time to the current time t; the expression represented by the symbol L is shown in equation (15):

[0110]

[0111] where p(x) represents the probability of the grid being occupied;

[0112] By setting the threshold value of L(m i |z 1:t ), it can be determined whether the grid is occupied. For grids with occupancy probability below the threshold value, it is considered that the grid is not occupied, and its semantic category and passability probability do not need to be maintained.

[0113] In a dynamic environment, each grid may be observed with different semantic labels at different times. In order to accurately determine the final semantic label of the grid, it is necessary to fuse the observation data at multiple times. In order to improve the computational efficiency, the maximum probability semantic update algorithm is adopted. Assume that the sensor reading at time t is z t , then the semantic label of grid v from time l to t is Label probability The current measured value and status The calculation is as shown in formula (16):

[0114]

[0115] Among them, the parameter ε∈(0,1) is set to 0.9. By comparing the label probability of the time interval [l, tl] and the current time t, the semantic label Update to the corresponding attribute with a larger probability value (passable and impassable), as shown in formula (17).

[0116]

[0117] After obtaining the 3D grid map of the point cloud, it is projected onto a 2D plane and converted into a 2D occupancy map that is more suitable for path planning.

[0118] If you choose to project all grids onto a two-dimensional plane, all point clouds such as the ground in the passable area and high-altitude suspended objects will be converted into obstacles on the two-dimensional plane. Therefore, grids with a height of -1m to 1.5m and a passable attribute of inaccessible areas are selected for projection.

[0119] The method proposed by the present invention was tested, and the experimental results are as follows Figure 5(a) 、 5(b) ,5(c),7(a),7(b),7(c);5(a) and 7(a) are two actual scene diagrams of this experiment; Figure 5(b) 、 7(b) In the figure, the green grid represents the passable area, and the red grid represents the obstacle (i.e., the impassable area); Figure 5(c) 、 7(c)Fig. 5(a) is a schematic diagram of a scene in which a paper box is introduced to simulate an unknown object in an unstructured environment. The semantic segmentation network fails to detect the category of the paper box due to the lack of training samples of this category. However, by analyzing the geometric features of the point cloud in this region, it is determined to be an "unpassable" region according to the constraint condition (formula (13)), as shown in the yellow circle in Fig. 5(b). Compared with the real situation, the geometric feature analysis method can effectively detect this kind of unknown obstacle, proving the effectiveness of analyzing the geometric features to supplement the segmentation of passable regions.

[0120] Although the semantic segmentation network performs well in most scenarios, its dependence on texture features may lead to misjudgment in long-distance or low-texture areas. To verify the effectiveness of the method of the present application in visual error recognition, as shown in Fig. 6(a) and Fig. 6(b), in a typical off-road scene, the sparse branches in the distance are misclassified as grassland by the semantic segmentation network due to their similar color to the grassland, and are then divided into passable regions. By using the Gaussian process regression of the present application to further subdivide this region, the vertical structural features of the region are successfully identified, and it is then corrected to the "unpassable" category.

[0121] The above embodiments are preferred embodiments of the present application, but the present application is not limited to the above embodiments, and any obvious improvements, replacements or modifications made by those skilled in the art without departing from the essential content of the present application shall fall within the protection scope of the present application.

Claims

1. A method for constructing a traversable map for an unstructured environment, characterized by: Build an improved DeepLabV3+ semantic segmentation network to perform semantic segmentation on images transmitted by the camera; Project the points in the LiDAR coordinate system to the camera's pixel coordinate system, and use the nearest neighbor search algorithm to associate the point cloud data with semantic information to obtain a single-frame semantic point cloud in the pixel coordinate system. Then, assign the point cloud a passable attribute, an impassable attribute, or an unknown attribute based on the semantic category. In a single-frame point cloud, for point clouds with passable attributes, an improved Gaussian process regression is used for secondary refinement to eliminate point clouds with visual recognition errors. For point clouds with unknown attributes, the geometric features of the point cloud are analyzed to determine whether the area is passable. Combined with the Lego-LOAM algorithm, the original point cloud is used for positioning to obtain the vehicle pose, and the single-frame semantic point cloud is continuously inserted into the global point cloud map; The global point cloud map is converted into a three-dimensional grid map. For grid update, the Bayesian update method is adopted. For grid semantic update, the maximum probability semantic update method is adopted. Then, the three-dimensional grid map is mapped onto a plane to form a two-dimensional occupancy map.

2. The method for constructing a traversable map for an unstructured environment according to claim 1, wherein: The point cloud is assigned a passable attribute, an impassable attribute, or an unknown attribute according to the semantic category: the point cloud with semantic information of buildings, shrubs, people, and background is assigned an impassable attribute, and the point cloud with semantic information of grass, paved roads, and dirt roads is assigned a passable attribute. For objects that cannot be recognized by the improved DeepLabV3+ semantic segmentation network, an unknown attribute is assigned.

3. The method for constructing a traversable map for an unstructured environment according to claim 2, wherein: The specific process of the secondary refinement is as follows: Voxel grid filtering is used to downsample the point cloud with passable attributes; The road area is divided into several sub-areas Sector (s1, s2, ..., sj) and parallelized Gaussian process regression modeling is performed; Select ground seed points in each sub-region Sector (s1, s2, ..., sj); Use seed points to build an initial Gaussian process regression model; Add other non-seed points in the sub-region that meet the constraints to the initial Gaussian process regression model, and re-optimize the Gaussian process regression model, and iterate continuously until no new points are added; The non-seed points in the sub-region that meet the constraints are classified as the passable area point cloud, while the points not accepted by the Gaussian process regression model are classified as the inpassable area point cloud.

4. The method for constructing a traversable map for an unstructured environment according to claim 3, wherein: The selection criteria of the seed points include: For height features, point clouds with height values ​​z close to the radar installation height h0 are preferred, that is, satisfying |z+h0|≤Δh, where Δh represents the height tolerance threshold.

5. The method for constructing a traversable map for an unstructured environment according to claim 4, wherein: The selection criteria of the seed points also include: Distance feature, select the point cloud that is closer to the vehicle's current position d, that is, satisfying d≤d_max, where d_max represents the maximum distance threshold.

6. The method for constructing a traversable map for an unstructured environment according to claim 3, wherein: The constraints include: Variance Constraint: Var k <Model th , among which Var k Indicates the variance of the point in the height z direction, Model th is the preset threshold.

7. The method for constructing a traversable map for an unstructured environment according to claim 6, wherein: The constraints include: Distance constraint: Among them, Mean k is the mean value of the model in the z direction, δ is the noise parameter, Dist th is the distance threshold, and z represents the height value of the non-seed point.

8. The method for constructing a traversable map for an unstructured environment according to claim 7, wherein: The constraints include: Normal vectors are consistent: |cos(θ z )|<|cos(θ zmax )| and|cos(θ y )|>|cos(θ ymax )|, where cos(θ z ) represents the cosine value of the angle between the normal vector of the point and the Z axis of the vehicle's horizontal coordinate system, cos(θ y ) represents the cosine value of the angle between the normal vector of the point and the Y axis of the horizontal coordinate system of the vehicle body, cos(θ zmax ) represents the maximum cosine threshold of the angle between the normal vector of the point and the Z axis, cos(θ ymax ) represents the cosine threshold of the maximum angle between the normal vector of the point and the Y axis.

9. The method for constructing a traversable map for an unstructured environment according to claim 1, wherein: The improved DeepLabV3+ semantic segmentation network adopts an encoder-decoder structure. The encoder part uses a lightweight MobileNetV3 as the backbone network to replace the traditional Xception network. The decoder part adopts a multi-level feature fusion strategy. The features extracted by the MobileNetV3 backbone network are adjusted in number of channels and then spliced ​​with the high-level features processed by the CBAM attention module.

10. The method for constructing a traversable map for an unstructured environment according to claim 1, wherein: The three-dimensional grid map is mapped onto a plane, specifically by selecting grids with a height of -1m to 1.5m and a traffic attribute of an impassable area for mapping.

Images