Road boundary polygonal outline construction method and system

Abstract

The present invention relates to a method and system for constructing a polygonal outline of a road boundary. The method comprises: fusing point cloud data of a current frame and a plurality of adjacent historical frames in a vehicle body coordinate system of the current frame to obtain fused point cloud data within a preset range; obtaining a probability feature map based on a pre-trained road boundary semantic segmentation model; obtaining a category feature map based on the probability feature map; obtaining a pixel set of each semantic category based on the category feature map; judging whether the pixel set of a bifurcation boundary is an empty set; if so, fitting a first road left boundary curve and a first road right boundary curve, and obtaining a closed polygonal outline; if not, dividing the preset range into a straight road segment, a bifurcated left road segment, and a bifurcated right road segment, and obtaining a closed polygonal outline of each road segment or obtaining a closed polygonal outline of one of the bifurcated left road segment and the bifurcated right road segment and the straight road segment.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud data processing, and more particularly to a method and system for constructing a road boundary polygonal outline. Background Art

[0002] The rapid development of autonomous driving technology in recent years has not only brought new driving experiences but also significantly reduced traffic accidents, casualties, and economic losses. LiDAR is a key sensor for environmental perception in autonomous driving systems, and point cloud-based object detection is essential for LiDAR to perceive environmental information.

[0003] In autonomous driving, accurate obstacle detection plays a crucial role in ensuring vehicle safety. Existing deep learning-based 3D object detection algorithms can only identify predefined categories of objects, requiring the detection of missed obstacles using traditional point cloud clustering algorithms. Unrelated point clouds outside the drivable area, such as trees, guardrails, and medians, can interfere with the execution of point cloud clustering algorithms. Therefore, pre-filtering background point clouds using road boundary polygons is crucial for point cloud object clustering.

[0004] Currently, mainstream assisted driving systems for highways, elevated roads, and expressways filter point clouds based on prior road information provided by high-definition maps. However, factors such as high map production costs, slow map updates, and strict map industry regulations significantly limit the applicability of high-definition maps in autonomous driving systems. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for constructing a road boundary polygon outline, which constructs a closed road boundary polygon outline based on deep learning and laser point cloud, thereby filtering the background point cloud.

[0006] Based on the above purpose, the present invention provides a method for constructing a road boundary polygonal outline, comprising the steps of:

[0007] S100: Acquire point cloud data of a current frame and multiple adjacent historical frames of a vehicle's laser radar in a vehicle body coordinate system of the current frame, and fuse the point cloud data of the current frame and the multiple adjacent historical frames in the vehicle body coordinate system of the current frame to obtain fused point cloud data within a preset range;

[0008] S200: Inputting the fused point cloud data within a preset range into a pre-trained road boundary semantic segmentation model to obtain a probabilistic feature map; the probabilistic feature map includes a plurality of pixels and a probability of each pixel belonging to a preset semantic category, each preset semantic category including background, left road boundary, right road boundary, and bifurcation boundary, and each pixel of the probabilistic feature map is associated with each laser point of the fused point cloud data;

[0009] S300: Determine the preset semantic category of each pixel according to the probability that each pixel in the probability feature map belongs to each preset semantic category, and obtain a category feature map;

[0010] S400: Classifying each pixel of the category feature map to obtain pixel sets of each preset semantic category;

[0011] S500: Determine whether the pixel set at the bifurcation boundary is an empty set; if so, execute steps S600-S700; otherwise, execute steps S800-S900;

[0012] S600: performing curve fitting on each pixel in the pixel set at the left boundary of the road to obtain a first road left boundary curve, and performing curve fitting on each pixel in the pixel set at the right boundary of the road to obtain a first road right boundary curve;

[0013] S700: Taking a plurality of first road left boundary points and a plurality of first road right boundary points at preset intervals within a preset range according to the first road left boundary line and the first road right boundary line, and obtaining a closed polygonal contour based on the plurality of first road left boundary points and the plurality of first road right boundary points;

[0014] S800: Curve fitting is performed on each pixel in the pixel set at the left boundary of the road to obtain a second left boundary line of the road, curve fitting is performed on each pixel in the pixel set at the right boundary of the road to obtain a second right boundary line of the road, and curve fitting is performed on each pixel in the pixel set at the bifurcation boundary to obtain a bifurcation left boundary line and a bifurcation right boundary line;

[0015] S900: Divide the preset range into a straight road segment, a forked left road segment and a forked right road segment according to the pixel set of the fork boundary that is closest to the vehicle, and obtain the closed polygonal outline of each road segment or obtain the closed polygonal outline of one of the forked left road segment and the forked right road segment and the straight road according to the second road left boundary line, the second road right boundary line, the forked left boundary line and the forked right boundary line.

[0016] Furthermore, step S100 specifically includes:

[0017] S110: Obtaining point cloud data of the current frame and multiple adjacent historical frames, and converting the point cloud data of the current frame and multiple adjacent historical frames from the lidar coordinate system to the vehicle body coordinate system corresponding to each frame based on the external parameters of the lidar;

[0018] S120: converting the point cloud data of each adjacent historical frame from the body coordinate system of each adjacent historical frame to the world coordinate system according to the pose of the body coordinate system of each adjacent historical frame in the world coordinate system;

[0019] S130: According to the pose of the vehicle body coordinate system of the current frame in the world coordinate system, the point cloud data of each adjacent historical frame is converted from the world coordinate system to the vehicle body coordinate system of the current frame;

[0020] S140: Fusing the point cloud data of the current frame and each adjacent historical frame in the vehicle body coordinate system of the current frame to obtain fused point cloud data;

[0021] S150: Filtering the fused point cloud data according to its spatial position to obtain fused point cloud data within a preset range.

[0022] Furthermore, the road boundary semantic segmentation model includes a column feature network, a 2D convolution backbone network and a semantic segmentation prediction head. The column feature network is used to map the fused point cloud data within a preset range to each point cloud body column, and extract the features of the point cloud data within each body column to obtain a pseudo feature map; the 2D convolution backbone network is used to receive the pseudo feature map and output a deep feature map; the semantic segmentation prediction head is used to output a probability feature map.

[0023] Furthermore, the road boundary semantic segmentation model is trained by the following method:

[0024] Acquire a training sample, wherein the training sample is fused point cloud data within a preset range obtained by fusing multiple frames of point cloud data;

[0025] The training samples are converted into a bird's-eye view image. The left and right road boundaries and the bifurcation boundary are then marked on the bird's-eye view image with polylines. The probability that each pixel on the bird's-eye view image belongs to each preset semantic category is obtained as the label of the training sample.

[0026] The preset road boundary semantic segmentation model is trained using the training samples and the labels of the training samples to obtain a trained road boundary semantic segmentation model.

[0027] Furthermore, step S300 specifically includes:

[0028] For each pixel in the probability feature map, if the maximum value of the probabilities that the pixel belongs to each preset semantic category is greater than or equal to a preset probability threshold, it is determined that the pixel belongs to the preset semantic category corresponding to the maximum probability value; otherwise, the preset semantic category of the pixel is background.

[0029] Furthermore, before classifying the category feature map, the method further includes:

[0030] Denoising the category feature map to obtain a denoised category feature map; and / or

[0031] Connected domain detection is performed on the denoised category feature map, and the preset semantic categories of all pixels in the connected domain whose number of pixels is less than a preset threshold are set as background, so as to obtain a category feature map after connected domain detection.

[0032] Furthermore, obtaining a closed polygonal outline according to the plurality of first road left boundary points and the plurality of first road right boundary points specifically includes:

[0033] Connecting a plurality of first road left boundary points and a plurality of first road right boundary points in sequence to obtain a closed polygonal outline; or

[0034] A plurality of first road left boundary valid points are selected from a plurality of first road left boundary points, such that an angle between two line segments formed by connecting any one of the first road left boundary valid points and two adjacent first road left boundary valid points is greater than a preset value; and a plurality of first road right boundary valid points are selected from a plurality of first road right boundary valid points, such that an angle between two line segments formed by connecting any one of the first road right boundary valid points and two adjacent first road right boundary valid points is greater than a preset value; and then the plurality of first road left boundary valid points are sequentially connected with the plurality of first road right boundary valid points to obtain a closed polygonal outline.

[0035] Furthermore, obtaining a closed polygonal outline of each road section according to the second road left boundary line, the second road right boundary line, the bifurcation left boundary line, and the bifurcation right boundary line specifically includes:

[0036] For each road section, a plurality of boundary points of the road section are obtained at preset intervals based on the second road left boundary line, the second road right boundary line and / or the bifurcation left boundary line and the bifurcation right boundary line, and a closed polygonal outline of the road section is obtained based on the plurality of boundary points of the road section;

[0037] The plurality of boundary points of the road section include a plurality of left boundary points and a plurality of right boundary points. The closed polygonal outline of the road section is obtained according to the plurality of boundary points of the road section, specifically including:

[0038] connecting the plurality of left boundary points and the plurality of right boundary points of the road section in sequence to form a closed polygonal outline of the road section; or

[0039] A plurality of valid left boundary points are selected from the plurality of left boundary points of the section of road, so that the angle between two line segments formed by connecting any one valid left boundary point and two adjacent left boundary points is greater than a preset value; and a plurality of valid right boundary points are selected from the plurality of right boundary points of the section of road, so that the angle between two line segments formed by connecting any one valid right boundary point and two adjacent right boundary points is greater than a preset value; and then the plurality of valid left boundary points and the plurality of valid right boundary points are sequentially connected to form a closed polygonal outline of the section of road.

[0040] Furthermore, obtaining a closed polygonal outline of one of the bifurcated left section road and the bifurcated right section road and the straight road according to the second road left boundary line, the second road right boundary line, the bifurcated left boundary line, and the bifurcated right boundary line specifically includes:

[0041] If the y-axis coordinate of the pixel closest to the vehicle in the pixel set at the bifurcation boundary is greater than 0, it is determined that the vehicle will enter the right bifurcation road, and a closed polygonal outline of the straight road and the right bifurcation road is obtained based on the left boundary line of the second road, the right boundary line of the second road, and the right boundary line of the bifurcation;

[0042] If the y-axis coordinate of the pixel closest to the vehicle in the pixel set of the bifurcation boundary is less than 0, it is determined that the vehicle will enter the left section of the bifurcation road, and the closed polygonal outline of the straight road and the left section of the bifurcation road is obtained based on the left boundary line of the second road, the right boundary line of the second road and the bifurcation left boundary line.

[0043] Another aspect of the present invention provides a road boundary polygonal outline construction system, comprising:

[0044] An acquisition module is used to acquire point cloud data of a current frame and multiple adjacent historical frames of the vehicle's laser radar in the vehicle body coordinate system of the current frame, and fuse the point cloud data of the current frame and multiple adjacent historical frames in the vehicle body coordinate system of the current frame to obtain fused point cloud data within a preset range;

[0045] A prediction module is configured to input the fused point cloud data within a preset range into a pre-trained road boundary semantic segmentation model to obtain a probabilistic feature map; the probabilistic feature map includes a plurality of pixels and the probability that each pixel belongs to a preset semantic category, each of which includes background, left road boundary, right road boundary, and bifurcation boundary, and each pixel of the probabilistic feature map is correlated with each laser point of the fused point cloud data;

[0046] a determination module, configured to determine the preset semantic category of each pixel according to the probability that each pixel in the probability feature map belongs to each preset semantic category, and obtain a category feature map;

[0047] A classification module, configured to classify each pixel of the category feature map to obtain pixel sets of each semantic category;

[0048] A judgment module, used to judge whether the pixel set of the bifurcation boundary is an empty set;

[0049] a fitting module configured to, when the judgment result of the judgment module is yes, perform curve fitting on each pixel in the pixel set of the left boundary of the road to obtain a first road left boundary curve, and perform curve fitting on each pixel in the pixel set of the right boundary of the road to obtain a first road right boundary curve; and, when the judgment result of the judgment module is no, perform curve fitting on each pixel in the pixel set of the left boundary of the road to obtain a second road left boundary line, perform curve fitting on each pixel in the pixel set of the right boundary of the road to obtain a second road right boundary line, and perform curve fitting on each pixel in the pixel set of the bifurcation boundary to obtain a bifurcation left boundary line and a bifurcation right boundary line;

[0050] The contour construction module is used to, when the judgment result of the judgment module is yes, select multiple first road left boundary points and first road right boundary points within a preset range at preset intervals based on the first road left boundary line and the first road right boundary line, and obtain a closed polygonal contour based on the multiple first road left boundary points and the multiple first road right boundary points; and when the judgment result of the judgment module is no, divide the preset range into a straight road segment, a forked left road segment and a forked right road segment based on the pixels closest to the vehicle in the pixel set of the fork boundary, obtain a closed polygonal contour of each road segment, or obtain a closed polygonal contour of one of the forked left road segment and the forked right road segment and the straight road.

[0051] The present invention's method and system for constructing polygonal road boundary outlines uses a deep learning network model to perform semantic segmentation on point cloud data, obtaining road boundary points. The system then fits road boundary lines based on these points and constructs a closed polygonal outline. This method and system can filter background point clouds to eliminate their influence on point cloud object clustering. The system boasts high precision, robustness, wide applicability, and low environmental requirements, addressing the high cost of high-precision maps and the vulnerability of visual images to weather and lighting conditions in existing technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A flowchart of a method for constructing a road boundary polygonal outline according to an embodiment of the present invention;

[0053] Figure 2 A bird's-eye view of point cloud data of a method for constructing a road boundary polygonal outline according to an embodiment of the present invention;

[0054] Figure 3 A bird's-eye view of a road after marking the road boundary according to the method for constructing a road boundary polygonal outline according to an embodiment of the present invention;

[0055] Figure 4 A structural block diagram of a road boundary polygonal outline construction system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The preferred embodiments of the present invention are given below in conjunction with the accompanying drawings and described in detail.

[0057] like Figure 1 As shown, an embodiment of the present invention provides a method for constructing a road boundary polygonal outline, comprising the steps of:

[0058] S100: Obtain point cloud data of a current frame and multiple adjacent historical frames of the vehicle's laser radar in the vehicle body coordinate system of the current frame, and fuse the point cloud data of the current frame and multiple adjacent historical frames in the vehicle body coordinate system of the current frame to obtain fused point cloud data within a preset range.

[0059] LiDAR is mounted on a vehicle (usually on the roof). As the vehicle drives, it scans the road to be inspected, generating point cloud data. Each frame of point cloud data includes the timestamp of that frame, the 3D coordinates of each laser point in that frame's point cloud data, and the reflection intensity. The 3D coordinates of each laser point are defined in the LiDAR coordinate system.

[0060] Step S100 specifically includes:

[0061] S110: Obtain point cloud data of the current frame and multiple adjacent historical frames, and convert the point cloud data of the current frame and multiple adjacent historical frames from the lidar coordinate system to the vehicle body coordinate system corresponding to each frame based on the external parameters of the lidar.

[0062] The body coordinate system is based on the center of the vehicle's rear axle as its origin. The forward direction of the vehicle is the positive x-axis, the left side of the vehicle is the positive y-axis, and the upward direction of the vehicle is the positive z-axis. Because the body coordinate system is based on the vehicle itself, its position in the world coordinate system changes constantly as the vehicle moves.

[0063] In some embodiments, the external parameters of the laser radar can be calculated by a sensor calibration algorithm, which includes: placing a cardboard box in front of the vehicle, measuring the 3D position coordinates of the corner points of the carton in the vehicle body coordinate system by a tape measure or a laser measuring instrument, which are called reference points. At the same time, the 3D coordinate positions of the corner points of the carton in the point cloud data are obtained, which are called points to be aligned. Obviously, the reference point and the point to be aligned corresponding to the same corner point form a pair. After a good spatial transformation, the overlap between the reference point and the point to be aligned should be as high as possible. In this way, an error function is established, and the accurate external parameters, namely the rotation matrix and the translation vector, can be solved by iterative closest point alignment algorithm (ICP). In other embodiments, the external parameters of the laser radar can also be calculated by automatically extracting feature points in the point cloud and image through an online calibration algorithm. The online calibration algorithm is an open source algorithm and will not be described here.

[0064] Since the point cloud data will be converted to the vehicle body coordinate system, this embodiment has no restriction on the pitch angle of the laser radar installation.

[0065] In some embodiments, after obtaining the point cloud data of the current frame and the adjacent historical frames, preprocessing can be performed first, including point cloud motion distortion correction, point cloud filtering, etc., wherein the point cloud motion distortion correction is to correct the Cartesian coordinates of each laser point by considering the motion of the vehicle, and the information used for correction comes from the IMU (six-axis sensor) or some kind of odometer, such as a visual inertial odometer; the point cloud filtering is composed of a sequential combination of a 3D voxel grid filter and a random downsampling filter, which can be adjusted, activated and deactivated separately to remove noise and out-of-range points, control the amount of data and reduce the computational load.

[0066] S120: According to the posture of the vehicle body coordinate system of each adjacent historical frame in the world coordinate system, the point cloud data of each adjacent historical frame is converted from the vehicle body coordinate system of each adjacent historical frame to the world coordinate system.

[0067] The origin of the world coordinate system is the position of the vehicle at the time of ignition. It is a coordinate system that describes absolute coordinates. During the vehicle's driving process, the world coordinate system always remains fixed, while the vehicle's position in the world coordinate system will change. The positioning module on the vehicle calculates the position of the vehicle body coordinate system in the world coordinate system at a certain moment in real time based on information such as the IMU (six-axis sensor) and GPS. This position information is stored in the form of a positioning information sequence. Therefore, for each historical frame of point cloud data, the timestamp of the historical frame is obtained, and the vehicle posture at that moment is queried from the positioning information sequence based on the timestamp. Then, based on the vehicle posture, the point cloud data of the historical frame is converted from the vehicle body coordinate system to the world coordinate system.

[0068] The number of adjacent historical frames is related to the frame rate of the lidar. Usually, point cloud data before the preset time of the current moment is obtained for fusion. For example, if the preset time is 0.5 seconds and the frame rate of the lidar is 10Hz, then the point cloud data of the five historical frames before the current frame need to be obtained.

[0069] S130: According to the pose of the vehicle body coordinate system of the current frame in the world coordinate system, the point cloud data of each adjacent historical frame is converted from the world coordinate system to the vehicle body coordinate system of the current frame.

[0070] The timestamp can be obtained from the point cloud data of the current frame, and then the vehicle posture at that moment can be queried from the positioning information sequence based on the timestamp. The point cloud data of each historical frame in the world coordinate system can be converted to the vehicle body coordinate system of the current frame based on the inverse transformation matrix of the vehicle posture.

[0071] S140: Fusing the point cloud data of the current frame and each adjacent historical frame in the vehicle body coordinate system of the current frame to obtain fused point cloud data.

[0072] Since the point cloud data of the current frame and the point cloud data of each adjacent historical frame are in the same coordinate system, they can be fused, that is, the point cloud data of multiple frames are merged into one frame to obtain the fused point cloud data. The fused point cloud data includes the timestamp of the current frame, the 3D coordinates of each laser point in the vehicle body coordinate system of the current frame, the reflection intensity and the timestamp difference. The timestamp difference is obtained by subtracting the timestamps of each adjacent historical frame from the timestamp of the current frame, which represents the time interval between each laser point in the adjacent historical frames and the current frame; for example, the timestamp difference of all laser points in the current frame is 0, while the timestamp difference of each laser point in the adjacent historical frames is a value greater than 0 (for example, 0.1 seconds).

[0073] Since the point cloud data of the current frame and multiple adjacent historical frames are fused, the point cloud can be made dense enough. Within these frames, the vehicle is always moving and can scan the surrounding environment at different positions. If a target point is blocked in the current frame but not in the previous historical frames, the target point will appear in the fused point cloud data, thus overcoming the problem of missing local point cloud information due to vehicle occlusion.

[0074] S150: Filtering the fused point cloud data according to its spatial position to obtain fused point cloud data within a preset range.

[0075] Specifically, filtering is to set a preset range, such as [x min ,x max ,y min ,y max ,z min ,z max ], then remove the point cloud data outside the preset range and only retain the point cloud data within the preset range. In an exemplary embodiment, the preset range is 60 meters in front of the origin of the vehicle body coordinate system of the current frame and 15 meters to the left and right, that is, x min =0,x max =60m,y min =-15m,y max =15m, z min =-2m,z max =4m.

[0076] S200: Input the fused point cloud data within a preset range into a pre-trained road boundary semantic segmentation model to obtain a probability feature map output by the road boundary semantic segmentation model; the probability feature map includes multiple pixels and the probability of each pixel belonging to each preset semantic category, each preset semantic category includes background, left road boundary, right road boundary and bifurcation boundary, and each pixel of the probability feature map is correlated with each laser point of the fused point cloud data.

[0077] The road boundary semantic segmentation model is an improvement on the existing PointPillars network. The existing PointPillars network consists of three parts: a Pillar Feature Net, a feature encoding network that converts point clouds into virtual images; a 2D Convolutional Convolutional Network backbone that processes these virtual images into high-dimensional representations; and an SSD detection head that regresses 3D object bounding boxes. The PointPillars network accepts point clouds as input and predicts 3D bounding boxes for objects such as cars, pedestrians, and two-wheeled vehicles. In order to be suitable for semantic segmentation tasks, three modifications are made to the PointPillars network in this embodiment, including: 1) the input features are increased from the original 9 to 14, including the coordinates x, y, z of each laser point in the point cloud data, the reflection intensity r, the center of mass coordinates xc, yc, zc of the laser point in the body column, the offset Δxp, Δyp of the laser point to the geometric center of the body column, the timestamp difference dt, the total number of laser points n in the body column where the laser point is located, and the offset Δxc, Δyc, Δzc of the laser point to the center of mass coordinates of the body column. In this way, the input information used is richer; 2) a smaller downsampling step size is used. The existing 2D convolutional backbone network contains two sub-networks: one is a top-down network that produces feature maps with smaller and smaller spatial resolutions, and the other network performs upsampling and channel feature splicing combination of the top-down feature maps. The network consists of 3 blocks, each of which halves the resolution of the input feature map. Therefore, the resolutions of the feature maps output by these 3 blocks are [H / 2, W / 2], [H / 4, W / 4], and [H / 8, W / 8], respectively, and the resolution of the feature map after the second network is spliced ​​and combined is [H / 2, W / 2]. In this embodiment, the step size of the first block is modified to 1, so that the resolutions of the feature maps output by these 3 blocks are [H, W], [H / 2, W / 2], and [H / 4, W / 4], respectively, and the resolution of the feature map after the second network is spliced ​​and combined is [H, W]. The advantage of this is that the model can generate full-resolution semantic predictions with higher prediction accuracy. 3) The existing detection head for 3D object detection tasks is modified to a semantic segmentation prediction head in the field of image semantic segmentation, which is used to output pixel-level segmentation prediction results.

[0078] Specifically, the road boundary semantic segmentation model first divides the bird's-eye view plane into multiple grids evenly, maps the fused point cloud data within a preset range to each grid to form a point cloud body column, which is a point cloud collection with infinite spatial range in the z direction; then, the body column feature network is used to extract the features of the point cloud data in each body column and perform maximum pooling aggregation to form a pseudo feature map (Pseudo Feature Map). Image); after the pseudo feature map is input into a 2D convolution backbone network composed of multiple layers of 2D convolution, a deep feature map is obtained. The deep feature map is input into the semantic segmentation prediction head and processed by the softmax function to output a probability feature map. The probability feature map includes the coordinates of each pixel (the coordinates in the vehicle body coordinate system of the current frame) and the probability that the pixel belongs to each preset semantic category. Each preset semantic category includes background, left road boundary, right road boundary and bifurcation boundary. The coordinates of the pixel are correlated with the coordinates of each laser point in the point cloud data, that is, the pixel corresponds to the grid one-to-one. The coordinates of the pixel are the coordinates of the center of the corresponding grid in the vehicle body coordinate system of the current frame, and the probability that the pixel belongs to each preset semantic category is the probability that all laser points in the grid belong to each semantic category. The bifurcation boundary refers to the boundary of a bifurcated road. For example, a "Y"-shaped bifurcation road has two "V"-shaped boundaries, including the left bifurcation boundary and the right bifurcation boundary.

[0079] In some embodiments, the road boundary semantic segmentation model is trained using the following method:

[0080] S210: Acquire a training sample, wherein the training sample is fused point cloud data within a preset range obtained by fusing multiple frames of point cloud data.

[0081] S220: Convert the training sample into a bird's-eye view image, then mark the left boundary of the road, the right boundary of the road, and the bifurcation boundary with broken lines on the bird's-eye view image, and obtain the probability that each pixel on the bird's-eye view image belongs to each semantic category as a label for the training sample.

[0082] The bird's-eye view refers to a view obtained by observing the point cloud from a bird's-eye view. Assuming the spatial resolution is 0.15m / pixel, the preset range is x = [0m, 60m], y = [-15m, 15m], and z = [-2m, 4m], then the size of the bird's-eye view is 200*400, which has 200*400 = 80,000 pixels. Each pixel corresponds to a 0.15m*0.15m grid. The coordinates of the pixel are the coordinates of the center of the grid in the vehicle body coordinate system of the current frame. Then, each laser point of the point cloud data is projected onto the bird's-eye view, as shown in the following example. Figure 2As shown in the figure, laser points falling within the same grid are associated with the pixel corresponding to the grid, that is, all laser points within the grid belong to the pixel, and the semantic category of the pixel is the semantic category of all laser points within the grid. In application, all pixels with laser points can be marked in red, and pixels without laser points can be marked in white, thus obtaining a color bird's-eye view image that is easier to observe. The road boundaries can be directly observed from the bird's-eye view image, and the left and right road boundaries and bifurcation boundaries can be directly marked with broken lines, as shown in the figure below. Figure 3 As shown in ; Since the polyline is composed of pixels, the coordinates of the pixels on the polyline can be known. The semantic category of the pixels on the left boundary of the road is the left boundary of the road, so the probability of belonging to the left boundary of the road is 1, and the probability of belonging to the other three semantic categories is 0. Similarly, the coordinates and probabilities of the pixels on the right boundary of the road and the bifurcation boundary can be obtained. The pixels outside the polyline are all background, so the probability of them belonging to the background is 1, and the probability of belonging to the other categories is 0; in this way, the coordinates of each pixel on the bird's-eye view and the probability of belonging to each semantic category, that is, the label of the training sample, can be obtained.

[0083] S230: Using the training samples and the labels of the training samples, a preset road boundary semantic segmentation model is trained to obtain a trained road boundary semantic segmentation model.

[0084] As described above, by inputting the training samples into the preset road boundary semantic segmentation model, a predicted probability feature map can be obtained, that is, the coordinates of each pixel and the predicted value of its probability belonging to each semantic category. The loss value between the predicted value of the probability of each pixel and the probability of the corresponding pixel in the label is the loss value of the pixel. The average of the loss values ​​of all pixels is taken as the loss value of the training sample. The road boundary semantic segmentation model is trained with the goal of minimizing the loss value of the training sample, and a trained road boundary semantic segmentation model can be obtained.

[0085] The following example illustrates the forward reasoning process of the road boundary semantic segmentation model:

[0086] 1. Point cloud voxelization:

[0087] Assuming the preset x-coordinate range is [0m, 60m], the y-coordinate range is [-15m, 15m], and the z-coordinate range is [-2m, 4m]. The preset range is divided into voxels, each with a length of 0.15m, a width of 0.15m, and a height of 6m. This results in 60 / 0.15 = 400 voxels on the x-axis, 30 / 0.15 = 200 voxels on the y-axis, and 6 / 6 = 1 voxel on the z-axis. This voxel-based space creates a 200-row, 400-column grid. Each laser point in the point cloud data is then assigned to a voxel based on its x and y coordinates. The position of each voxel on the grid, i.e., the row and column indices, is recorded.

[0088] 2. Body column feature network:

[0089] The body column feature network traverses each body column and encodes a feature vector of length m (for example, m = 64) for the body column. Specifically, k laser points located in the body column are extracted (for example, k = 40, if the number exceeds 40, 40 laser points are randomly selected and retained; if the number is less than 40, the number is expanded to 40 by filling with zero values). 14 feature values ​​are calculated for each laser point, including the x, y, and z coordinates of the laser point in the vehicle body coordinate system of the current frame, the reflection intensity r of the laser point, the timestamp difference dt of the laser point (the timestamp difference between the frame where the laser point is located and the current frame), the total number n of laser points in the body column where the laser point is located, and the centroid coordinates (x, y, and z) of the laser point in the body column where the laser point is located. c ,y c ,z c ), the offset of the laser point relative to the center of mass coordinates (Δx c ,Δy c ,Δz c ), the offset of the laser point relative to the geometric center of the body column (Δx p ,Δy p ). Thus, a 40*14 matrix is ​​obtained, where 40 represents 40 laser points and 14 represents 14 eigenvalues ​​for each laser point. This 40*14 matrix is ​​input to a fully connected neural network layer (the perception layer, such as a PointNet network) to obtain a 40*64 matrix, where 40 represents the 40 laser points and 64 represents the encoded length of the eigenvector for each point. Finally, a MaxPooling operation is performed to obtain a 1*64 eigenvector, which serves as the eigenvector representation of the body column.

[0090] 3. Pseudo feature map construction

[0091] Using the scatter operator, the feature vector of each body column is indexed by its coordinates in the grid image and placed back into the corresponding position in the grid image, resulting in a 200*400*64 "pseudo-feature map." For empty body columns (i.e., those without any laser points), the channel data in the "pseudo-feature map" is all zero.

[0092] 4. Semantic Segmentation Network

[0093] The semantic segmentation network uses this 200*400*64 "pseudo feature map" as input and uses a 2D convolutional neural network (FPN) (Feature Pyramid Network) as the backbone network to encode and decode the feature map. The convolutional layer downsampling step size of the backbone network is adjusted to ensure a total downsampling step size of 1 for the semantic segmentation network. Finally, a 200*400*4 predicted probability feature map is output. Specifically, for each pixel in the predicted probability feature map, the probability of four preset semantic categories (including background, left road boundary, right road boundary, and bifurcation boundary) is predicted.

[0094] S300: Determine the preset semantic category of each pixel according to the probability that each pixel in the probability feature map belongs to each preset semantic category, and obtain a category feature map.

[0095] The semantic category of each pixel can be determined based on the probability that each pixel in the probability feature map belongs to each preset semantic category. In some embodiments, the semantic category with the highest probability can be selected as the semantic category of the pixel. For example, if the probabilities of a pixel belonging to the background, the left road boundary, the right road boundary, and the bifurcation boundary are 0.8, 0.1, 0.05, and 0.05, respectively, then the semantic category of the pixel is determined to be background.

[0096] In some other embodiments, a probability threshold score_threshold can be preset, usually 0.9; then each pixel in the probability feature map is traversed to obtain the maximum value socre_k and the corresponding channel index of the probability that the pixel belongs to each preset semantic category. If the maximum value socre_k is less than score_threshold, the semantic category of the pixel is determined to be background, and its semantic category index is marked as 0. If the maximum value socre_k is greater than or equal to score_threshold, the pixel is determined to belong to the semantic category corresponding to the maximum probability value, and the semantic category index of the pixel is marked as k. The semantic category index value of each pixel position is stored in a single-channel image, and a category feature map label_map with a size of 200*400*1 can be obtained, where 200 means that the image has 200 rows, corresponding to the number of body pillars in the y-axis direction of the vehicle, and 400 means that the category feature map has 400 columns, corresponding to the number of body pillars in the x-axis direction of the vehicle. Obviously, one body pillar corresponds to one pixel.

[0097] In some embodiments, the category feature map may be preprocessed first, and then the preprocessed category feature map may be used for subsequent steps. Preprocessing includes denoising and connected domain detection. Specifically, the image morphological opening operation (first corrosion and then expansion) may be used to remove spots and burr-like noise on the category feature map label_map; then connected domain detection is performed on label_map, and the total number of connected domains, the number of pixels in each connected domain, and the connected domain labels corresponding to the pixels are calculated. The semantic category indexes of all pixels in a connected domain whose number of pixels is less than a preset threshold are reset to 0, i.e., their semantic categories are redefined as background.

[0098] S400: Classify each pixel of the category feature map to obtain pixel sets of each preset semantic category.

[0099] Since the category feature map includes the coordinates of each pixel and the semantic category, each pixel point can be classified to obtain the pixel sets of background, left road boundary, right road boundary and bifurcation boundary respectively.

[0100] In an exemplary embodiment, each column of the label_map can be traversed in sequence with a step size of 1 along the x-axis direction of the category feature map label_map, and the pixels in the column are respectively classified into the pixel sets of pixels of each semantic category. The specific classification method is:

[0101] For each pixel, 1) if the semantic category index of the pixel is 1 (i.e., the semantic category is the left boundary of the road) and the semantic category index of the right adjacent pixel of the pixel is 0 (i.e., the semantic category is background), then the pixel is classified into the pixel set of the left boundary of the road; 2) if the semantic category index of the pixel is 2 (i.e., the semantic category is the right boundary of the road) and the semantic category index of the left adjacent pixel of the pixel is 0, then the pixel is classified into the pixel set of the right boundary of the road; 3) if the semantic category index of the pixel is equal to 3 (i.e., the semantic category is bifurcation boundary) and the semantic category index of the left adjacent pixel of the pixel is equal to 0, then the pixel is classified into the pixel set of the bifurcation boundary and belongs to the left boundary of the bifurcation boundary (i.e., bifurcation left boundary); 4) if the semantic category index of the pixel is equal to 3 and the semantic category index of the right adjacent pixel of the pixel is equal to 0, then the pixel is classified into the pixel set of the bifurcation boundary and belongs to the right boundary of the bifurcation boundary (i.e., bifurcation right boundary). That is, the pixel set at the bifurcation boundary also includes two subsets, namely, the pixel set at the bifurcation left boundary and the pixel set at the bifurcation right boundary.

[0102] S500: Determine whether the pixel set at the bifurcation boundary is an empty set; if so, execute steps S600-S700; otherwise, execute steps S800-S900.

[0103] Since the preset range may or may not have a fork in the road, if there is a fork in the road, then the pixel set of the fork boundary is not an empty set, otherwise the pixel set of the fork boundary is an empty set. Therefore, it is necessary to first determine whether there is a fork in the preset range by determining whether the pixel set of the fork boundary is an empty set.

[0104] S600: performing curve fitting on each pixel in the pixel set at the left boundary of the road to obtain a first left boundary line of the road, and performing curve fitting on each pixel in the pixel set at the right boundary of the road to obtain a first right boundary line of the road.

[0105] If there is no fork in the preset range, it means there is only one road. Then a curve y=f can be fitted based on the x-coordinate and y-coordinate of each pixel in the pixel set at the left edge of the road. L1 (x), that is, the left boundary line of the first road; a curve y=f can be fitted based on the x-coordinate and y-coordinate of each pixel in the pixel set of the right boundary of the road. R1 (x), which is the right boundary line of the first road.

[0106] In some embodiments, the curve to be fitted may be fitted using a cubic curve function y=d+c*x+b*x^2+a*x^3 and a RANSAC (random sampling consensus) algorithm to obtain an optimal curve.

[0107] S700: Taking a plurality of first road left boundary points and a plurality of first road right boundary points within a preset range at preset intervals according to the first road left boundary line and the first road right boundary line, and obtaining a closed polygonal outline according to the plurality of first road left boundary points and the plurality of first road right boundary points.

[0108] In some embodiments, the preset interval can be 1m, that is, an x ​​value is taken every 1m on the x-axis, and then substituted into the first road left boundary line and the first road right boundary line respectively to obtain the corresponding y value. Each x value and its corresponding y value of the first road left boundary line constitute a first road left boundary point, and each x value and its corresponding y value of the first road right boundary line constitute a first road right boundary point. For example, if the x coordinate range of the preset range is [0m, 60m], then the obtained first road left boundary point is [0, f L1 (0)]、[1,f L1 (1)], ...[60,f L1 (60)], the right boundary point of the first road is [0,f R1 (0)]、[1,f R1 (1)], ...[60,f R1 (60)], connect the left boundary points of the first road in sequence, connect the right boundary points of the first road in sequence, and then [0,f L1 (0)] and [0,f R1(0)], connect [60,f L1 (60)] and [60,f R1 (60)] can be connected to obtain a closed polygonal contour, which is the polygonal contour of the road boundary within the preset range.

[0109] In some embodiments, multiple first road left boundary points and first road right boundary points may be downsampled before being connected. Specifically, for any first road left boundary point p1, the two first road left boundary points before and after the first road left boundary point are p0 and p2. p1p0 and p1p2 are connected, and the angle between p1p0 and p1p2 is calculated. If the angle is greater than a preset value (e.g., 2°), the first road left boundary point p1 is selected as a valid first road left boundary point; otherwise, it is an invalid first road left boundary point. Similarly, multiple first road right boundary valid points can be selected from the multiple first road right boundary points. All invalid first road left boundary points and invalid first road right boundary points are discarded, leaving only valid first road left boundary points and valid first road right boundary points. The multiple first road left boundary valid points and multiple first road right boundary valid points are then sequentially connected to form a closed polygonal outline. Since invalid points are discarded, the angle between adjacent line segments can be ensured to be greater than the preset value, and the number of line segments (i.e., edges) of the polygonal outline can be reduced.

[0110] S800: Curve fitting is performed on each pixel in the pixel set at the left boundary of the road to obtain a second left boundary line of the road, and curve fitting is performed on each pixel in the pixel set at the right boundary of the road to obtain a second right boundary line of the road, and curve fitting is performed on each pixel in the pixel set at the bifurcation boundary to obtain a bifurcation left boundary line and a bifurcation right boundary line.

[0111] If the pixel set of the bifurcation boundary is not an empty set, it means that there is a bifurcation road in the preset range. Then, it is necessary to perform curve fitting on each pixel in the pixel set of the left boundary of the road, the right boundary of the road, and the bifurcation boundary, so as to obtain the second road left boundary line f L2 (x), second road right boundary line f R2 (x), bifurcation boundary line, since the pixel set of the bifurcation boundary includes two subsets: the pixel set of the bifurcation left boundary and the pixel set of the bifurcation right boundary, the bifurcation left boundary line f can be obtained by curve fitting each pixel of the bifurcation left boundary pixel set. ML (x), the right boundary line of the bifurcation is obtained by curve fitting each pixel of the pixel set of the right boundary of the bifurcation. MR (x) The specific fitting method may be the same as that in step S700 and will not be described in detail here.

[0112] S900: Divide the preset range into a straight road segment, a forked left road segment and a forked right road segment according to the pixel set of the fork boundary that is closest to the vehicle, and obtain the closed polygonal outline of each road segment or obtain the closed polygonal outline of one of the forked left road segment and the forked right road segment and the straight road according to the second road left boundary line, the second road right boundary line, the forked left boundary line and the forked right boundary line.

[0113] The pixel closest to the vehicle is the pixel with the smallest x coordinate, assuming x ms , the y coordinate of the pixel is y ms , so the preset range can be divided into three segments, where the x-coordinate range of the straight line segment is [0,x ms ], the x-coordinate range of the left and right boundaries of the bifurcation is [x ms ,x max ], where x max is the maximum value of the x coordinate in the preset range. Assume that x max = 60m, then the x-coordinate range of the left and right boundaries of the bifurcation is [x ms ,60m].

[0114] For a straight road segment, assuming the preset interval is 1m, take multiple x coordinates (0,1,2…x ms ), respectively substitute the above x coordinates into the curve equation of the left boundary line of the second road to obtain the corresponding y coordinate values ​​f L2 (x), thus obtaining multiple straight line road left boundary points [0, f L2 (0)], [1, f L2 (1)], [2, f L2 (2)]…[x ms , f L2 (x ms )]; Similarly, by substituting the above x coordinate into the curve equation of the second road right boundary line, multiple straight line segment road right boundary points [0, f R2 (0)], [1, f R2 (1)], [2, f R2 (2)]…[x ms , f R2 (x ms )]; connect the left boundary points of the above-mentioned multiple straight line segments in sequence, connect the right boundary points of the above-mentioned multiple straight line segments in sequence, and [0, f L2 (0)] and [0, f R2 (0)] connection and [x ms , f L2 (x ms )] and [x ms , f R2 (x ms)] can be connected to obtain the closed polygonal outline of the straight line segment road.

[0115] For the left section of the bifurcated road, multiple x coordinates (x ms ,x ms +1,…,60), and respectively substitute the above x coordinates into the curve equation of the left boundary line of the second road to obtain the corresponding y coordinate values ​​f L2 (x), thus obtaining multiple left boundary points of the bifurcated left section road [x ms , f L2 (x ms )]、[x ms +1,f L2 (x ms +1)]…[60,f L2 (60)]; Similarly, by substituting the above x coordinate into the equation of the left boundary line of the bifurcation, we can obtain multiple right boundary points of the left section of the bifurcation road [x ms , f ML (x ms )]、[x ms +1,f ML (x ms +1)]…[60,f ML (60)], by sequentially connecting multiple left boundary points of the bifurcated left section road and multiple right boundary points of the bifurcated left section road, a closed polygonal outline of the bifurcated left section road can be obtained.

[0116] For the right section of the bifurcated road, multiple x coordinates (x ms ,x ms +1,…,60), and respectively substitute the above x coordinates into the curve equation of the right boundary line of the second road to obtain the corresponding y coordinate values ​​f R2 (x), thus obtaining multiple right boundary points of the right section of the bifurcated road [x ms , f R2 (x ms )]、[x ms +1,f R2 (x ms +1)]…[60,f R2 (60)]; Similarly, by substituting the above x coordinate into the equation of the right boundary line of the bifurcation, we can obtain multiple left boundary points of the right section of the bifurcation road [x ms , f MR (x ms )]、[x ms +1,f MR (x ms +1)]…[60,f MR (60)], by sequentially connecting multiple left boundary points of the forked right section road and multiple right boundary points of the forked right section road, a closed polygonal outline of the forked right section road can be obtained.

[0117] In some embodiments, before connecting the boundary points of each road segment, they may be downsampled first, and then connected after selecting valid points. The downsampling method is the same as in step S700 and will not be repeated here.

[0118] In some embodiments, it is also possible to ms To determine whether the vehicle is going to the forked left road or the forked right road, if y ms If y is greater than 0, it means the vehicle will enter the right section of the forked road. ms If the value is less than 0, it means that the vehicle will enter the left fork road. After the judgment is completed, only the straight line segment and the polygonal outline of the left or right fork road to be entered can be obtained, without obtaining the outline of the other fork road, to save time.

[0119] The road boundary polygonal outline construction method of the embodiment of the present invention performs semantic segmentation on point cloud data based on a deep learning network model to obtain road boundary points. It then fits road boundary lines based on these road boundary points and constructs a closed polygonal outline. This method can filter background point clouds to eliminate their influence on point cloud object clustering. The road boundary polygonal outline construction system of the embodiment of the present invention offers high precision, high robustness, wide applicability, and low environmental requirements. It addresses the high cost of high-precision maps in existing technologies, as well as the vulnerability of visual images to weather and lighting.

[0120] like Figure 4 As shown, another embodiment of the present invention provides a road boundary polygon outline construction system, comprising:

[0121] An acquisition module 11 is configured to acquire point cloud data of a current frame and multiple adjacent historical frames of a vehicle's laser radar in the vehicle body coordinate system of the current frame, and fuse the point cloud data of the current frame and the multiple adjacent historical frames in the vehicle body coordinate system of the current frame to obtain fused point cloud data within a preset range;

[0122] A prediction module 12 is configured to input the fused point cloud data within a preset range into a pre-trained road boundary semantic segmentation model to obtain a probabilistic feature map; the probabilistic feature map includes pixel coordinates and the probability that each pixel belongs to a preset semantic category, each of which includes background, left road boundary, right road boundary, and bifurcation boundary. Each pixel in the probabilistic feature map is associated with each laser point in the fused point cloud data;

[0123] A determination module 13 is configured to determine the semantic category of each pixel according to the probability that each pixel in the probability feature map belongs to each preset semantic category, thereby obtaining a category feature map;

[0124] A classification module 14 is used to classify each pixel of the category feature map to obtain pixel sets of each semantic category;

[0125] A judging module 15 is used to judge whether the pixel set at the bifurcation boundary is an empty set;

[0126] a fitting module 16 configured to, when the judgment result of the judgment module is yes, perform curve fitting on each pixel in the pixel set at the left boundary of the road to obtain a first road left boundary curve, and perform curve fitting on each pixel in the pixel set at the right boundary of the road to obtain a first road right boundary curve; and, when the judgment result of the judgment module is no, perform curve fitting on each pixel in the pixel set at the left boundary of the road to obtain a second road left boundary line, perform curve fitting on each pixel in the pixel set at the right boundary of the road to obtain a second road right boundary line, and perform curve fitting on each pixel in the pixel set at the bifurcation boundary to obtain a bifurcation left boundary line and a bifurcation right boundary line;

[0127] The contour construction module 17 is used to, when the judgment result of the judgment module is yes, select multiple first road left boundary points and first road right boundary points within a preset range at preset intervals based on the first road left boundary line and the first road right boundary line, and obtain a closed polygonal contour based on the multiple first road left boundary points and the multiple first road right boundary points; and when the judgment result of the judgment module is no, divide the preset range into a straight road segment, a forked left road segment, and a forked right road segment based on the pixels closest to the vehicle in the pixel set of the fork boundary, and obtain a closed polygonal contour of each road segment or obtain a closed polygonal contour of one of the forked left road segment and the forked right road segment and the straight road.

[0128] The specific implementation methods of the above modules are the same as the road boundary polygon outline construction method in the previous embodiment, and will not be repeated here.

[0129] The road boundary polygonal outline construction system of the embodiment of the present invention performs semantic segmentation on point cloud data based on a deep learning network model to obtain road boundary points. It then fits road boundary lines based on these road boundary points and constructs a closed polygonal outline. This system can filter background point clouds to eliminate their influence on point cloud object clustering. The road boundary polygonal outline construction system of the embodiment of the present invention offers high precision, high robustness, wide applicability, and low environmental requirements. It can address the high cost of high-precision maps in existing technologies and the susceptibility of visual images to weather and lighting.

[0130] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of the present invention. Various modifications are possible. In other words, any simple, equivalent changes and modifications made in accordance with the claims and description of the present invention are within the scope of protection of the patent claims. Anything not fully described in this invention constitutes conventional technology.

Claims

1. A method for constructing a road boundary polygonal outline, characterized in that: Including steps: S100: Acquire point cloud data of a current frame and multiple adjacent historical frames of a vehicle's laser radar in a vehicle body coordinate system of the current frame, and fuse the point cloud data of the current frame and the multiple adjacent historical frames in the vehicle body coordinate system of the current frame to obtain fused point cloud data within a preset range; S200: Inputting the fused point cloud data within a preset range into a pre-trained road boundary semantic segmentation model to obtain a probability feature map; The probability feature map includes a plurality of pixels and the probability of each pixel belonging to each preset semantic category, each preset semantic category including background, left road boundary, right road boundary and bifurcation boundary, and each pixel of the probability feature map is associated with each laser point of the fused point cloud data; S300: Determine the preset semantic category of each pixel according to the probability that each pixel in the probability feature map belongs to each preset semantic category, and obtain a category feature map; S400: Classifying each pixel of the category feature map to obtain pixel sets of each preset semantic category; S5 00: Determine whether the pixel set at the bifurcation boundary is an empty set; If yes, execute steps S600-S700; Otherwise, execute steps S800-S900; S600: performing curve fitting on each pixel in the pixel set at the left boundary of the road to obtain a first road left boundary curve, and performing curve fitting on each pixel in the pixel set at the right boundary of the road to obtain a first road right boundary curve; S700: Taking a plurality of first road left boundary points and a plurality of first road right boundary points at preset intervals within a preset range according to the first road left boundary line and the first road right boundary line, and obtaining a closed polygonal contour based on the plurality of first road left boundary points and the plurality of first road right boundary points; S800: Curve fitting is performed on each pixel in the pixel set at the left boundary of the road to obtain a second left boundary line of the road, curve fitting is performed on each pixel in the pixel set at the right boundary of the road to obtain a second right boundary line of the road, and curve fitting is performed on each pixel in the pixel set at the bifurcation boundary to obtain a bifurcation left boundary line and a bifurcation right boundary line; S900: Divide the preset range into a straight road segment, a forked left road segment and a forked right road segment according to the pixel set of the fork boundary that is closest to the vehicle, and obtain the closed polygonal outline of each road segment or obtain the closed polygonal outline of one of the forked left road segment and the forked right road segment and the straight road according to the second road left boundary line, the second road right boundary line, the forked left boundary line and the forked right boundary line.

2. The method for constructing a road boundary polygonal outline according to claim 1, characterized in that: Step S100 specifically includes: S110: Obtaining point cloud data of the current frame and multiple adjacent historical frames, and converting the point cloud data of the current frame and multiple adjacent historical frames from the lidar coordinate system to the vehicle body coordinate system corresponding to each frame based on the external parameters of the lidar; S120: converting the point cloud data of each adjacent historical frame from the body coordinate system of each adjacent historical frame to the world coordinate system according to the pose of the body coordinate system of each adjacent historical frame in the world coordinate system; S130: According to the pose of the vehicle body coordinate system of the current frame in the world coordinate system, the point cloud data of each adjacent historical frame is converted from the world coordinate system to the vehicle body coordinate system of the current frame; S140: Fusing the point cloud data of the current frame and each adjacent historical frame in the vehicle body coordinate system of the current frame to obtain fused point cloud data; S150: Filtering the fused point cloud data according to its spatial position to obtain fused point cloud data within a preset range.

3. The method for constructing a road boundary polygonal outline according to claim 1, wherein: The road boundary semantic segmentation model includes a column feature network, a 2D convolutional backbone network and a semantic segmentation prediction head. The column feature network is used to map the fused point cloud data within a preset range to each point cloud body column and extract the features of the point cloud data within each body column to obtain a pseudo feature map; the 2D convolutional backbone network is used to receive the pseudo feature map and output a deep feature map; and the semantic segmentation prediction head is used to output a probability feature map.

4. The method for constructing a road boundary polygonal outline according to claim 1, wherein: The road boundary semantic segmentation model is trained by the following method: Acquire a training sample, wherein the training sample is fused point cloud data within a preset range obtained by fusing multiple frames of point cloud data; The training samples are converted into a bird's-eye view image. The left and right road boundaries and the bifurcation boundary are then marked on the bird's-eye view image with polylines. The probability that each pixel on the bird's-eye view image belongs to each preset semantic category is obtained as the label of the training sample. The preset road boundary semantic segmentation model is trained using the training samples and the labels of the training samples to obtain a trained road boundary semantic segmentation model.

5. The method for constructing a road boundary polygonal outline according to claim 1, wherein: Step S300 specifically includes: For each pixel in the probability feature map, if the maximum value of the probabilities that the pixel belongs to each preset semantic category is greater than or equal to a preset probability threshold, it is determined that the pixel belongs to the preset semantic category corresponding to the maximum probability value; otherwise, the preset semantic category of the pixel is background.

6. The method for constructing a road boundary polygonal outline according to claim 1, wherein: Before classifying the category feature map, the method further includes: Denoising the category feature map to obtain a denoised category feature map; and / or Connected domain detection is performed on the denoised category feature map, and the preset semantic categories of all pixels in the connected domain whose number of pixels is less than a preset threshold are set as background, so as to obtain a category feature map after connected domain detection.

7. The method for constructing a road boundary polygonal outline according to claim 1, wherein: Obtaining a closed polygonal outline according to a plurality of first road left boundary points and a plurality of first road right boundary points specifically includes: Connecting a plurality of first road left boundary points and a plurality of first road right boundary points in sequence to obtain a closed polygonal outline; or A plurality of first road left boundary valid points are selected from a plurality of first road left boundary points, such that an angle between two line segments formed by connecting any one of the first road left boundary valid points and two adjacent first road left boundary valid points is greater than a preset value; and a plurality of first road right boundary valid points are selected from a plurality of first road right boundary valid points, such that an angle between two line segments formed by connecting any one of the first road right boundary valid points and two adjacent first road right boundary valid points is greater than a preset value; and then the plurality of first road left boundary valid points are sequentially connected with the plurality of first road right boundary valid points to obtain a closed polygonal outline.

8. The method for constructing a road boundary polygonal outline according to claim 1, wherein: Obtaining a closed polygonal outline of each road segment based on the second road left boundary line, the second road right boundary line, the bifurcation left boundary line, and the bifurcation right boundary line, specifically including: For each road section, a plurality of boundary points of the road section are obtained at preset intervals based on the second road left boundary line, the second road right boundary line and / or the bifurcation left boundary line and the bifurcation right boundary line, and a closed polygonal outline of the road section is obtained based on the plurality of boundary points of the road section; The plurality of boundary points of the road section include a plurality of left boundary points and a plurality of right boundary points. The closed polygonal outline of the road section is obtained according to the plurality of boundary points of the road section, specifically including: connecting the plurality of left boundary points and the plurality of right boundary points of the road section in sequence to form a closed polygonal outline of the road section; or A plurality of valid left boundary points are selected from the plurality of left boundary points of the section of road, so that the angle between two line segments formed by connecting any one valid left boundary point and two adjacent left boundary points is greater than a preset value; and a plurality of valid right boundary points are selected from the plurality of right boundary points of the section of road, so that the angle between two line segments formed by connecting any one valid right boundary point and two adjacent right boundary points is greater than a preset value; and then the plurality of valid left boundary points and the plurality of valid right boundary points are sequentially connected to form a closed polygonal outline of the section of road.

9. The method for constructing a road boundary polygonal outline according to claim 1, wherein: Acquiring a closed polygonal outline of one of the bifurcated left section road and the bifurcated right section road and the straight road according to the second road left boundary line, the second road right boundary line, the bifurcated left boundary line, and the bifurcated right boundary line specifically includes: If the y-axis coordinate of the pixel closest to the vehicle in the pixel set at the bifurcation boundary is greater than 0, it is determined that the vehicle will enter the right bifurcation road, and a closed polygonal outline of the straight road and the right bifurcation road is obtained based on the left boundary line of the second road, the right boundary line of the second road, and the right boundary line of the bifurcation; If the y-axis coordinate of the pixel closest to the vehicle in the pixel set of the bifurcation boundary is less than 0, it is determined that the vehicle will enter the left section of the bifurcation road, and the closed polygonal outline of the straight road and the left section of the bifurcation road is obtained based on the left boundary line of the second road, the right boundary line of the second road and the bifurcation left boundary line.

10. A road boundary polygonal outline construction system, characterized in that: include: An acquisition module is used to acquire point cloud data of a current frame and multiple adjacent historical frames of the vehicle's laser radar in the vehicle body coordinate system of the current frame, and fuse the point cloud data of the current frame and multiple adjacent historical frames in the vehicle body coordinate system of the current frame to obtain fused point cloud data within a preset range; The prediction module is used to input the fused point cloud data within a preset range into a pre-trained road boundary semantic segmentation model to obtain a probabilistic feature map; The probability feature map includes a plurality of pixels and a probability of each pixel belonging to each preset semantic category, wherein the preset semantic categories include background, left road boundary, right road boundary, and bifurcation boundary, and each pixel of the probability feature map is correlated with each laser point of the fused point cloud data; a determination module, configured to determine the preset semantic category of each pixel according to the probability that each pixel in the probability feature map belongs to each preset semantic category, and obtain a category feature map; A classification module, configured to classify each pixel of the category feature map to obtain pixel sets of each preset semantic category; A judgment module, used to judge whether the pixel set of the bifurcation boundary is an empty set; a fitting module configured to, when the judgment result of the judgment module is yes, perform curve fitting on each pixel in the pixel set of the left boundary of the road to obtain a first road left boundary curve, and perform curve fitting on each pixel in the pixel set of the right boundary of the road to obtain a first road right boundary curve; and, when the judgment result of the judgment module is no, perform curve fitting on each pixel in the pixel set of the left boundary of the road to obtain a second road left boundary line, perform curve fitting on each pixel in the pixel set of the right boundary of the road to obtain a second road right boundary line, and perform curve fitting on each pixel in the pixel set of the bifurcation boundary to obtain a bifurcation left boundary line and a bifurcation right boundary line; The contour construction module is used to, when the judgment result of the judgment module is yes, select multiple first road left boundary points and first road right boundary points within a preset range at preset intervals based on the first road left boundary line and the first road right boundary line, and obtain a closed polygonal contour based on the multiple first road left boundary points and the multiple first road right boundary points; and when the judgment result of the judgment module is no, divide the preset range into a straight road segment, a forked left road segment and a forked right road segment based on the pixels closest to the vehicle in the pixel set of the fork boundary, obtain a closed polygonal contour of each road segment, or obtain a closed polygonal contour of one of the forked left road segment and the forked right road segment and the straight road.

Images