Point cloud semantic segmentation method based on mechanical laser radar and distance view

Abstract

The invention discloses a point cloud semantic segmentation method based on a mechanical laser radar and a distance view, and belongs to the field of automatic driving and the technical field of laser radars. The implementation method comprises the following steps: acquiring environment data by using a mechanical laser radar, converting acquired 3D point cloud data from a Cartesian coordinate system into a spherical coordinate system for representation, and generating an original 2D multi-channel distance view; copying and taking out a transverse left side 1 / 2N part and a right side 1 / 2N part of the original 2D multi-channel distance view, splicing the left side copied part on the right side of the original 2D multi-channel distance view, splicing the right side copied part on the left side, and forming an improved distance view of which the transverse resolution is increased by 1 / N compared with that of the original size; and inputting the improved distance view into a 2D image full-convolution semantic segmentation network for segmentation, performing cutting operation on a semantic segmentation result, performing back projection to a three-dimensional point cloud space, performing KNN clustering post-processing, correcting a semantic label of each point by using consensus voting of the nearest K point, and obtaining an effective and reliable high-precision point cloud semantic segmentation result.

Description

Technical Field

[0001] The present invention relates to a point cloud semantic segmentation method based on a mechanical laser radar and a range view, and belongs to the field of autonomous driving and laser radar technology. Background Art

[0002] In the stage of autonomous driving environmental perception, LiDAR, as a key environmental perception sensor, can accurately measure the distance, speed and shape of objects, providing three-dimensional perception capabilities for autonomous driving vehicles. As one of the core tasks of this stage, LiDAR point cloud semantic segmentation has important application value. By assigning semantic labels to each point in the point cloud data collected by LiDAR, such as cars, pedestrians, cyclists, roads and buildings, it is used to distinguish between passable areas and obstacles in the environment, thereby achieving in-depth understanding and accurate perception of driving scenes, which is crucial for achieving high-precision and high-reliability autonomous driving.

[0003] At present, the point cloud semantic segmentation method based on solid-state laser radar has the problem of blind spot perception. Although solid-state laser radar has the advantages of compact structure and high stability, its horizontal field of view is usually limited to 80° to 120°. This physical limitation makes this method have blind spots in perception. The point cloud semantic segmentation method based on mechanical laser radar relies on mechanical rotating parts (such as motors and turntables) to achieve 360-degree scanning and collection of environmental data. Although there is no blind spot in perception, the disordered and unstructured characteristics of the collected point cloud data make the point-based or voxel-based methods have bottlenecks in terms of computational efficiency and computational cost, making it difficult to take into account the requirements of real-time and accuracy of autonomous driving.

[0004] The projection-based method usually uses a spherical front view projection method to convert disordered 3D point cloud data into a structured range view (RV), and uses two-dimensional image technology to process it, while having good segmentation accuracy and efficiency. However, in this process, due to the spatial distribution characteristics of the point cloud, the continuous point cloud data in the area directly behind the LiDAR will appear vertically disconnected and distributed on both sides of the range view in the projected range view. The resulting discontinuity will seriously affect the accuracy and reliability of the point cloud segmentation in the area behind the LiDAR, so that the autonomous driving vehicle may not be able to correctly perceive the obstacles directly behind it during driving, increasing the risk of collision. Summary of the invention

[0005] The problem of insufficient perception and recognition accuracy of rear obstacles by autonomous vehicles during driving is solved, thereby avoiding safety hazards in autonomous vehicles during driving. The purpose of the present invention is to provide a point cloud semantic segmentation method based on mechanical laser radar and distance view, which first uses mechanical laser radar to collect environmental data, converts the collected 3D point cloud data from Cartesian coordinate system to spherical coordinate system, and then generates an original 2D multi-channel distance view; the 1 / 2N left and 1 / 2N right parts of the original 2D multi-channel distance view are copied and taken out, and the left copy part is spliced ​​on the right side of the original 2D multi-channel distance view, and the right copy part is spliced ​​on the left side, forming an improved distance view with a 1 / N increase in lateral resolution compared with the original size, and edge consistency supervision information is introduced to deal with the information breakage problem of the lateral edge area of ​​the original 2D multi-channel distance view; the improved distance view is input into the 2D image full convolution semantic segmentation network for segmentation, the semantic segmentation result is back-projected to the three-dimensional point cloud space after cropping, and KNN clustering post-processing is performed, that is, the semantic label of each point is corrected by using the consensus voting of the nearest K points, and an effective and reliable high-precision point cloud semantic segmentation result is obtained.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] The present invention discloses a point cloud semantic segmentation method based on mechanical laser radar and range view, comprising the following steps:

[0008] S1. Use a mechanical laser radar to sample the environment and obtain 3D point cloud data in the Cartesian coordinate system generated by the laser radar.

[0009] S2, using the spherical front view projection method, transforms the 3D point cloud data collected by S1 from the Cartesian coordinate system to the spherical coordinate system, and then generates the original 2D multi-channel distance view.

[0010] S21. Use spherical front view projection to convert the 3D point cloud data in the Cartesian coordinate system generated by the laser radar into a spherical coordinate system representation, as shown in expression (1):

[0011]

[0012] Among them, φ i (∈(-π,π]) is the point p on the laser scanning line i The azimuth angle in the spherical coordinate system; θ i (∈(f down , f up ]) is the point p in the laser scanning point cloud i The pitch angle in spherical coordinates is expressed as, f up With f downRespectively represent the maximum upper and lower pitch angle values, and there is a maximum upper pitch angle f up >0 and maximum downward pitch angle f down <0;r i Indicates the radial distance of the point in spherical coordinates; x i ,y i , z i They are respectively the coordinates of the point in the x-axis, y-axis, and z-axis of the Cartesian coordinate system with the laser radar as the origin.

[0013] S22. The spherical coordinates obtained in S21 are transformed into distance view pixel coordinates by coordinate origin transformation and normalized scaling: the origin is translated to the upper left corner of the image, and the image is scaled according to the laser radar parameters used (azimuth resolution, number of lines, etc.), as shown in expression (2):

[0014]

[0015] Where W and H represent the horizontal resolution and vertical resolution of the two-dimensional range view respectively; u i represents point p on the laser scan line i Corresponding to the horizontal pixel coordinates of the projected two-dimensional distance view; i represents point p on the laser scan line i Corresponding to the horizontal pixel coordinate of the projected 2D distance view.

[0016] S23, using formula (1) (2) to transform all points in the 3D point cloud data generated by S1 into the distance view coordinate system, obtain the pixel coordinates of the 3D point cloud data in the distance view coordinate system, record the mapping relationship between the two, and fill in the corresponding 3D point cloud data information for each pixel coordinate in the distance view to obtain the X channel distance view, that is, the original 2D multi-channel distance view, as the two-dimensional dense representation of the 3D point cloud data:

[0017]

[0018] in, Represents the pixel coordinate corresponding to the point cloud point p i X attributes, namely x i ,y i , z i ,i i , r i Among them, X; in addition, since projection often has a one-to-many phenomenon, that is, the same coordinate position on the distance image can be obtained by projecting multiple point cloud points, the point cloud point with the closest distance is selected.

[0019] S3. Using bilateral compensation stitching, copy and take out the 1 / 2N left and 1 / 2N right parts of the original 2D multi-channel distance view, and splice the left copied part to the right side of the original 2D multi-channel distance view, and splice the right copied part to the left side of the original 2D multi-channel distance view, to form an improved distance view with a lateral resolution increased by 1 / N compared to the original 2D multi-channel distance view; introduce edge consistency supervision information to deal with the problem of information breakage in the lateral edge area of ​​the original 2D multi-channel distance view.

[0020] S31, extract the left edge area of ​​the original 2D multi-channel distance view, copy and take out the leftmost 1 / 2N part of the horizontal axis direction of the original 2D multi-channel distance view obtained in S2, that is, take out the part with the coordinate index range of horizontal [0, W / 2N] and vertical [0, H] on the original 2D multi-channel distance view, and record it as R left :

[0021]

[0022] S32, extract the right edge area of ​​the original 2D multi-channel distance view, copy and take out the rightmost 1 / 2N part of the horizontal axis direction of the original 2D multi-channel distance view obtained in S2, that is, take out the part with the coordinate index range of horizontal [WW / 2N, W] and vertical [0, H] on the original 2D multi-channel distance view, and record it as R right :

[0023]

[0024] S33, using bilateral compensation stitching to form an improved distance view: the R extracted from S31 and S32 left and R right They are spliced ​​on the right and left sides of the original 2D multi-channel distance view respectively, and the improved distance view is shown in expression (6):

[0025] R e =[R right RR left ] (6)

[0026] By introducing bilateral compensation splicing, a “bridge” is provided for discontinuous regions, so that discontinuous regions can be connected more naturally semantically. The complete representation of the distance view is improved as shown in expression (7):

[0027]

[0028] S4. Input the improved distance view obtained in S3 into the neural network, and perform semantic segmentation on it using the 2D image fully convolutional semantic segmentation network to obtain a semantic mask of the improved distance view.

[0029] S41, the improved distance view R obtained in step S3 e Input the designed encoder-decoder fully convolutional segmentation network, first improve the distance view R e Use convolution operation to obtain the feature map F of C channel I Next, the encoder is used to transform the feature map F I Four downsampling operations are performed, with the maximum downsampling multiple being 32 times, from which four feature maps of different sizes are obtained, which are called F1, F2, F3, and F4 in the order of acquisition.

[0030] S42, the feature map F4 finally obtained by the encoder is input into the decoder part, and four upsampling operations are performed on it by deconvolution. After each upsampling operation is completed, the feature maps of the corresponding sizes obtained by S41 (in order, F3, F2, F1) are added to restore the high-frequency edge information lost due to the downsampling operation, and the final output is the same as the improved distance view R e Multi-channel output feature map F of the same size O .

[0031] S43, the multi-channel output feature map F obtained in step S42 O Input the classification head, first use the convolution operation to generate an output tensor with a shape of [S, H, (N+1)W / N], and then normalize the output tensor through the Softmax classifier to map it to a category probability distribution with a shape of [S, H, (N+1)W / N], and obtain the confidence that each pixel in the improved distance view belongs to a certain semantic category. The value range is transformed to between 0 and 1, and the sum of the semantic confidences of the S categories is 1; take the category with the maximum confidence in the S dimension as the semantic label of the point, thereby obtaining the semantic mask M of the improved distance view e , which is the two-dimensional distance view semantic segmentation result.

[0032] S5. Crop the improved distance view semantic mask obtained in S4 to obtain the semantic mask of the original 2D multi-channel distance view, and back-project it into the three-dimensional point cloud space, perform KNN clustering, find the K nearest points for each point, use the semantic labels of these neighboring points to correct the semantic type of the point, combine local and global information to enhance the correlation between points in the local neighborhood, and obtain high-precision point cloud semantic segmentation results.

[0033] S51, improving the distance view semantic mask M obtained in step S43 e Cut and transform the semantic mask M e The portion corresponding to step S33 spliced ​​on the left and right sides of the improved distance view is cut off and removed to obtain a semantic category mask M corresponding to the original 2D multi-channel distance view R in step S2;

[0034] S52. Back-project the semantic category mask M obtained in S51 to the three-dimensional point cloud space: Use the mapping relationship between the 3D point cloud data and the distance view pixel coordinates obtained in step S2 to assign semantic labels to all lidar points corresponding to each distance view pixel coordinate, thereby obtaining a coarse semantic segmentation result of the 3D point cloud data.

[0035] S53, perform post-processing of KNN clustering: find the K nearest points for each point in the 3D point cloud data, and use the semantic labels of these K points to correct the semantic type of the point, that is, count the semantic labels of these K nearest neighbor points, and designate the label with the most occurrences as the semantic label of the current point, so as to obtain a high-precision point cloud semantic segmentation result.

[0036] Beneficial effects:

[0037] 1. The present invention discloses a method for semantic segmentation of point cloud based on mechanical laser radar and distance view, which uses deep learning to semantically segment laser radar data. The point cloud data generated by the laser radar is preprocessed, the three-dimensional point cloud information is mapped to the two-dimensional distance view, and the two-dimensional image processing technology is used to reduce the computational complexity. While preserving more dense and complete point cloud information, a large amount of computational overhead generated by the three-dimensional convolution segmentation process is avoided. At the same time, it has good segmentation accuracy and efficiency, and has good adaptability to automatic driving systems with high real-time requirements.

[0038] 2. The present invention discloses a point cloud semantic segmentation method based on mechanical laser radar and range view, which effectively solves the discontinuity problem of point cloud data in the area directly behind the spherical front view projection laser radar after projection by improving the stitching operation of the distance view. By introducing edge consistency supervision information, the network can better understand the structure and continuity of the point cloud data when processing the stitched distance view, reducing the segmentation error caused by environmental changes or sensor noise, and enabling the neural network to more accurately understand the continuity of the point cloud data, thereby significantly improving the accuracy of point cloud segmentation.

[0039] 3. The present invention discloses a point cloud semantic segmentation method based on mechanical laser radar and range view, which can process the point cloud data of the area directly behind the laser radar more smoothly and naturally by back-projecting the semantic labels into the three-dimensional point cloud space and then performing KNN clustering post-processing operations. The KNN clustering algorithm queries the information of the K nearest neighbor points of each point and determines the final label of the point based on the predicted labels of these neighbor points, thereby effectively performing fine classification processing on the point cloud data and improving the overall capability of point cloud data processing. In particular, KNN clustering shows its applicability when processing complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1The present invention is a flowchart of a point cloud semantic segmentation method based on mechanical laser radar and range view.

[0041] Figure 2 The present invention provides a flow chart for obtaining the original 2D multi-channel distance view through the segmentation method of the present invention.

[0042] Figure 3 The present invention provides a segmentation method to obtain an improved distance view process.

[0043] Figure 4 This is a neural network model diagram of the segmentation method described in the present invention.

[0044] Figure 5 The original 2D multi-channel distance view semantic mask is obtained by the segmentation method of the present invention.

[0045] Figure 6 To compare the original point cloud obtained by the segmentation method of the present invention with the point cloud semantic segmentation result. DETAILED DESCRIPTION

[0046] In order to illustrate the technical problems solved by the present invention and the beneficial effects thereof, the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0047] Reference Figure 1 , this embodiment discloses a point cloud semantic segmentation method based on mechanical laser radar and range view, and the specific implementation steps are:

[0048] S1. Use a mechanical laser radar to sample the environment and obtain 3D point cloud data in the Cartesian coordinate system generated by the laser radar.

[0049] S11, start the laser radar to collect data, and assume that a frame of 3D point cloud data generated by the laser radar is in represents the real number domain space, n represents the number of points generated by one frame of laser radar point cloud data; (x i ,y i , z i ,i i )| i∈n is the i-th row of data in the matrix P, representing point p in a frame of point cloud data i , x i ,y i , z i ,i i are the coordinates of the point on the x-axis, y-axis, and z-axis in the Cartesian coordinate system with the laser radar as the origin, and the reflection intensity of the corresponding point;

[0050] S12. Collect 3D point cloud data generated by the mechanical laser radar scanning the 360-degree environment during the driving process of the autonomous driving vehicle, and obtain the laser radar point cloud set {P1, P2, P3, P4...}.

[0051] S2, using the spherical front view projection method, convert each frame of 3D point cloud data in the lidar point cloud set collected by S12 from the Cartesian coordinate system to the spherical coordinate system, and then generate the original 2D multi-channel distance view, such as Figure 2 shown.

[0052] S21. Use spherical front view projection to convert the 3D point cloud data in the Cartesian coordinate system generated by the laser radar into a spherical coordinate system representation, as shown in expression (1):

[0053]

[0054] Among them, φ i (∈(-π,π]) is the point p on the laser scanning line i The azimuth angle in the spherical coordinate system; θ i (∈(f down , f up ]) is the point p in the laser scanning point cloud i The pitch angle in spherical coordinates is expressed as, f up With f down Respectively represent the maximum upper and lower pitch angle values, and there is a maximum upper pitch angle f up >0 and maximum downward pitch angle f down <0;r i Indicates the radial distance of the point in spherical coordinates; x i ,y i , z i They are respectively the coordinates of the point in the x-axis, y-axis, and z-axis of the Cartesian coordinate system with the laser radar as the origin.

[0055] S22. The spherical coordinates obtained in S21 are transformed into distance view pixel coordinates by coordinate origin transformation and normalized scaling: the origin is translated to the upper left corner of the image, and the image is scaled according to the parameters of the Velodyne Lidar HDL-64E laser radar used (the azimuth resolution range is 0.08° to 0.35°, and the number of laser lines is 64), which is specifically shown in expression (2):

[0056]

[0057] Wherein, W and H represent the horizontal resolution and vertical resolution of the two-dimensional range view, respectively, and in the embodiment of the present invention, they are 1024 and 64 respectively; i represents point p on the laser scan line iCorresponding to the horizontal pixel coordinates of the projected two-dimensional distance view; i represents point p on the laser scan line i Corresponding to the horizontal pixel coordinate of the projected 2D distance view.

[0058] S23, using formula (1) (2) to transform all points in the 3D point cloud data generated by S1 into the distance view coordinate system, obtain the pixel coordinates of the 3D point cloud data in the distance view coordinate system, record the mapping relationship between the two, and fill in the corresponding 3D point cloud data information for each pixel coordinate in the distance view to obtain a 5-channel distance view, that is, the original 2D multi-channel distance view, as a two-dimensional dense representation of the 3D point cloud data:

[0059]

[0060] in, Represents the pixel coordinate corresponding to the point cloud point p i The five properties of x i ,y i , z i ,i i , r i In addition, since projection often presents a one-to-many phenomenon, that is, the same coordinate position on the distance image can be obtained by projecting multiple point cloud points, the point cloud point with the closest distance is selected.

[0061] S3. Using bilateral compensation stitching, copy and take out the 1 / 32 left and 1 / 32 right parts of the original 2D multi-channel distance view, and splice the left copied part to the right side of the original 2D multi-channel distance view, and splice the right copied part to the left side of the original 2D multi-channel distance view, thereby forming an improved distance view with a lateral resolution increased by 1 / 16 compared with the original 2D multi-channel distance view, thereby introducing edge consistency supervision information to deal with the problem of information breakage in the lateral edge area of ​​the original 2D multi-channel distance view.

[0062] S31, extract the left edge area of ​​the original 2D multi-channel distance view, copy and take out the leftmost 1 / 32 part of the horizontal axis direction of the original 2D multi-channel distance view obtained in S2, that is, take out the part with the coordinate index range of horizontal [0, W / 32] and vertical [0, H] on the original 2D multi-channel distance view, and record it as R left :

[0063]

[0064] S32, extract the right edge area of ​​the original 2D multi-channel distance view, copy and take out the 1 / 32 part on the right side of the horizontal axis direction of the original 2D multi-channel distance view obtained in S2, that is, take out the part with the coordinate index range of horizontal [WW / 32, W] and vertical [0, H] on the original 2D multi-channel distance view, and record it as Rright :

[0065]

[0066] S33, using bilateral compensation stitching to extract R from S31 and S32 left and R right The improved distance view is formed by splicing the right and left sides of the original 2D multi-channel distance view. The lateral resolution of the improved distance view is increased by 1 / 16 (the left and right sides are increased by 1 / 32, a total of 2 / 32, i.e. 1 / 16) compared with the lateral width of the original 2D multi-channel distance view. This operation introduces more consistency supervision information in the edge area of ​​the original 2D multi-channel distance view. The process is as follows Figure 3 As shown, the improved distance view is shown in expression (6):

[0067] R e =[R right RR left ] (6)

[0068] By introducing bilateral compensation splicing, it is equivalent to providing a "bridge" for the discontinuous regions, so that the discontinuous regions can be connected more naturally in semantics. Its complete representation is shown in expression (7):

[0069]

[0070] S4, the improved distance view R obtained in S3 e The input is a neural network, and a 2D image fully convolutional semantic segmentation network is used to perform semantic segmentation on it, thereby obtaining a semantic mask for the improved distance view.

[0071] S41, the improved distance view R obtained in step S3 e Input the designed encoder-decoder fully convolutional segmentation network, first improve the distance view R e Use convolution operation to obtain 32-channel feature map F I Next, the encoder is used to transform the feature map F I Four downsampling operations are performed, with the maximum downsampling multiple being 32 times, from which four feature maps of different sizes are obtained, which are called F1, F2, F3, and F4 in the order of acquisition. In the embodiment of the present invention, the number of dual residual block structures in encoders 1, 2, 3, and 4 is 2, 8, 8, and 4, respectively.

[0072] S42, the feature map F4 finally obtained by the encoder is input into the decoder part, and four upsampling operations are performed on it by deconvolution. After each upsampling operation is completed, the feature maps of the corresponding sizes obtained by S41 (in order, F3, F2, F1) are added to restore the high-frequency edge information lost due to the downsampling operation, and the final output is the same as the improved distance view R e 32-channel output feature map F of the same size O .

[0073] S43, the 32-channel output feature map F obtained in step S42 O Input the classification head, first use the convolution operation to generate an output tensor with a shape of [19, 64, 1044], and then normalize the output tensor through the Softmax classifier to map it to a category probability distribution with a shape of [19, 64, 1044], and obtain the confidence that each pixel in the improved distance view belongs to a certain semantic category. The value range is transformed to between 0 and 1, and the sum of the semantic confidences of S categories is 1; take the category with the maximum confidence in the S dimension as the semantic label of the point, thereby obtaining the semantic mask M of the improved distance view e , which is the two-dimensional distance view semantic segmentation result.

[0074] S5. Crop the improved distance view semantic mask obtained in S4 to obtain the semantic mask of the original 2D multi-channel distance view, and back-project it into the three-dimensional point cloud space, perform KNN clustering, find the K nearest points for each point, use the semantic labels of these neighboring points to correct the semantic type of the point, combine local and global information to enhance the correlation between points in the local neighborhood, and obtain high-precision point cloud semantic segmentation results.

[0075] S51, improving the distance view semantic mask M obtained in step S43 e The 1 / 32 parts of the left and right sides of the improved distance view corresponding to step S33 are cut off to obtain the semantic category mask M corresponding to the original 2D multi-channel distance view R of step S2. The cutting process is as follows: Figure 5 shown.

[0076] S52. Back-project the semantic category mask M obtained in S51 to the three-dimensional point cloud space: Use the mapping relationship between the 3D point cloud data and the distance view pixel coordinates obtained in step S2 to assign semantic labels to all lidar points corresponding to each distance view pixel coordinate, thereby obtaining a coarse semantic segmentation result of the 3D point cloud data.

[0077] S53, perform post-processing of KNN clustering: find the nearest K points for each point in the 3D point cloud data, and use the semantic labels of these K points to correct the semantic type of the point, that is, count the semantic labels of the K nearest neighbor points, and designate the label with the most occurrences as the semantic label of the current point, thereby obtaining a high-precision point cloud semantic segmentation result. In the embodiment of the present invention, the K value of the KNN clustering method is set to 5, and the distance calculation method is the Euclidean distance.

[0078] Through the LiDAR point cloud semantic segmentation method of this embodiment, the mechanical LiDAR is used to collect environmental data for semantic segmentation testing. The segmentation results are as follows: Figure 6 As shown in the figure. The upper part is the visualization of the original LiDAR point cloud data, and the lower part is the visualization point cloud segmentation result processed by the segmentation method and endowed with semantics. It can be clearly seen from the figure that the LiDAR point cloud data processed by this method achieves accurate segmentation effect.

[0079] The specific description above further illustrates the purpose, technical solutions and beneficial effects of the invention in detail. It should be understood that the above is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A point cloud semantic segmentation method based on mechanical laser radar and range view, characterized by: The following steps are included: S1. Use a mechanical laser radar to sample the environment and obtain 3D point cloud data in a Cartesian coordinate system generated by the laser radar; S2, using the spherical front view projection method, transforms the 3D point cloud data collected by S1 from the Cartesian coordinate system to the spherical coordinate system, and then generates the original 2D multi-channel distance view; S3. Using bilateral compensation stitching, copy and take out the 1 / 2N left and 1 / 2N right parts of the original 2D multi-channel distance view, and splice the left copied part to the right side of the original 2D multi-channel distance view, and splice the right copied part to the left side of the original 2D multi-channel distance view, to form an improved distance view with a 1 / N increase in lateral resolution compared to the original 2D multi-channel distance view; introduce edge consistency supervision information to deal with the problem of information breakage in the lateral edge area of ​​the original 2D multi-channel distance view; S4, inputting the improved distance view obtained in S3 into the neural network, and performing semantic segmentation on it using the 2D image fully convolutional semantic segmentation network, thereby obtaining a semantic mask of the improved distance view; S5. Crop the improved distance view semantic mask obtained in S4 to obtain the semantic mask of the original 2D multi-channel distance view, and back-project it into the three-dimensional point cloud space, perform KNN clustering, find the K nearest points for each point, use the semantic labels of these neighboring points to correct the semantic type of the point, combine local and global information to enhance the correlation between points in the local neighborhood, and obtain high-precision point cloud semantic segmentation results.

2. The method for semantic segmentation of point cloud based on mechanical laser radar and range view according to claim 1, characterized in that: The implementation method of step S2 is: S21. Use spherical front view projection to convert the 3D point cloud data in the Cartesian coordinate system generated by the laser radar into a spherical coordinate system representation, as shown in expression (1): Among them, φ i (∈(-π,π]) is the point p on the laser scanning line i The azimuth angle in the spherical coordinate system; θ i (∈(f down , f up ]) is the point p in the laser scanning point cloud i The pitch angle in spherical coordinates is expressed as, f up With f down Respectively represent the maximum upper and lower pitch angle values, and there is a maximum upper pitch angle f up >0 and maximum downward pitch angle f down <0;r i Indicates the radial distance of the point in spherical coordinates; x i ,y i , z i are the coordinates of the point in the x-axis, y-axis, and z-axis of the Cartesian coordinate system with the laser radar as the origin; S22. The spherical coordinates obtained in S21 are transformed into distance view pixel coordinates by coordinate origin transformation and normalized scaling: the origin is translated to the upper left corner of the image, and the image is scaled according to the lidar parameters used, as shown in expression (2): Where W and H represent the horizontal resolution and vertical resolution of the two-dimensional range view respectively; u i represents point p on the laser scan line i Corresponding to the horizontal pixel coordinate of the projected two-dimensional distance view; v i represents point p on the laser scan line i Corresponding to the horizontal pixel coordinates of the projected two-dimensional distance view; The laser radar parameters include azimuth resolution and number of lines; S23, using formula (1) (2) to transform all points in the 3D point cloud data generated by S1 into the distance view coordinate system, obtain the pixel coordinates of the 3D point cloud data in the distance view coordinate system, record the mapping relationship between the two, and fill in the corresponding 3D point cloud data information for each pixel coordinate in the distance view to obtain the X channel distance view, that is, the original 2D multi-channel distance view, as the two-dimensional dense representation of the 3D point cloud data: in, Represents the pixel coordinate corresponding to the point cloud point p i X attributes, namely x i ,y i , z i ,i i , r i Among them, X; in addition, the same coordinate position on the distance image is obtained by projecting multiple point cloud points, and the point cloud point with the closest distance is selected.

3. The method for semantic segmentation of point cloud based on mechanical laser radar and range view according to claim 2, characterized in that: The implementation method of step S3 is: S31, extract the left edge area of ​​the original 2D multi-channel distance view, copy and take out the leftmost 1 / 2N part of the horizontal axis direction of the original 2D multi-channel distance view obtained in S2, that is, take out the part with the coordinate index range of horizontal [0, W / 2N] and vertical [0, H] on the original 2D multi-channel distance view, and record it as R left : S32, extract the right edge area of ​​the original 2D multi-channel distance view, copy and take out the rightmost 1 / 2N part of the horizontal axis direction of the original 2D multi-channel distance view obtained in S2, that is, take out the part with the coordinate index range of horizontal [WW / 2N, W] and vertical [0, H] on the original 2D multi-channel distance view, and record it as R right : S33, using bilateral compensation stitching to form an improved distance view: the R extracted from S31 and S32 left and R right They are spliced ​​on the right and left sides of the original 2D multi-channel distance view respectively, and the improved distance view is shown in expression (6): R e =[R right R R left ] (6) By introducing bilateral compensation splicing, a "bridge" is provided for discontinuous areas, so that discontinuous areas can be connected more naturally semantically, and the complete representation of the distance view is improved as shown in expression (7):

4. The method for semantic segmentation of point cloud based on mechanical laser radar and range view according to claim 3, characterized in that: The implementation method of step S4 is: S41, the improved distance view R obtained in step S3 e Input the designed encoder-decoder fully convolutional segmentation network, first improve the distance view R e Use convolution operation to obtain the feature map F of C channel I , use the encoder to transform the feature map F I Four downsampling operations, with the maximum downsampling multiple being 32 times, from which four feature maps of different sizes are obtained, which are called F1, F2, F3, and F4 in the order of acquisition; S42, input the feature map F4 finally obtained by the encoder into the decoder part, and perform four upsampling operations on it by deconvolution. After each upsampling operation, the feature maps of the corresponding sizes obtained by S41 are added in the order of F3, F2, and F1 to restore the high-frequency edge information lost due to the downsampling operation, and the output is the same as the improved distance view R e Multi-channel output feature map F of the same size O ; S43, the multi-channel output feature map F obtained in step S42 O Input the classification head, first use the convolution operation to generate an output tensor with a shape of [S, H, (N+1)W / N], and then normalize the output tensor through the Softmax classifier to map it to a category probability distribution with a shape of [S, H, (N+1)W / N], and obtain the confidence that each pixel in the improved distance view belongs to a certain semantic category. The value range is transformed to between 0 and 1, and the sum of the semantic confidences of the S categories is 1; take the category with the maximum confidence in the S dimension as the semantic label of the point, thereby obtaining the semantic mask M of the improved distance view e , which is the two-dimensional distance view semantic segmentation result.

5. The method for semantic segmentation of point cloud based on mechanical laser radar and range view according to claim 4, characterized in that: The implementation method of step S5 is: S51, improving the distance view semantic mask M obtained in step S43 e Cut and transform the semantic mask M e The portion corresponding to step S33 spliced ​​on the left and right sides of the improved distance view is cut off and removed to obtain a semantic category mask M corresponding to the original 2D multi-channel distance view R in step S2; S52, back-projecting the semantic category mask M obtained in S51 to the three-dimensional point cloud space: using the mapping relationship between the 3D point cloud data and the distance view pixel coordinates obtained in step S2, assigning semantic labels to all the laser radar points corresponding to each distance view pixel coordinate, thereby obtaining a coarse semantic segmentation result of the 3D point cloud data; S53, perform post-processing of KNN clustering: find the K nearest points for each point in the 3D point cloud data, and use the semantic labels of these K points to correct the semantic type of the point, that is, count the semantic labels of these K nearest neighbor points, and designate the label with the most occurrences as the semantic label of the current point, so as to obtain a high-precision point cloud semantic segmentation result.

Images