Ground detection method and system based on laser point cloud, and computer-readable medium

Through semantic segmentation neural network and rasterized processing technology, the problem that existing ground detection methods rely on manual map labeling is solved, and high-accuracy detection of ground point cloud points in vehicle driving scenarios is achieved, reducing ground point cloud obstacles Interference in object judgment.

WO2024244438A1PCT designated stage expired Publication Date: 2024-12-05HOZON NEW ENERGY AUTOMOBILE CO LTD

Patent Information

Application Number
PCT/CN2023/141745
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-06-02
Filing Date
2023-12-26
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

The existing ground detection methods highly rely on manual high-precision maps and cannot fully cover all scenarios of vehicle driving, resulting in low accuracy in ground point and cloud point detection in some driving scenarios, affecting obstacle judgment.

Method used

The laser point cloud points are classified based on semantic segmentation neural network model, point cloud points larger than the preset threshold are removed, and ground point cloud points are screened through rasterization processing and fitting plane calculations to improve detection accuracy.

Benefits of technology

No need to manually mark the map, it is suitable for most scenarios of vehicle driving, effectively screening out actual ground points and cloud points, improving the accuracy of ground detection and reducing interference to target detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2023141745_05122024_PF_FP_ABST
    Figure CN2023141745_05122024_PF_FP_ABST
Patent Text Reader

Abstract

The present invention relates to a ground detection method and system based on a laser point cloud, and a computer-readable medium. The method comprises: step S1: inputting point cloud points into a semantic segmentation neural network model, and the semantic segmentation neural network model outputting categories of the point cloud points, wherein the categories comprise ground and non-ground; step S2: using, as point cloud points to be processed, the point cloud points of which the category is ground, and removing point cloud points to be processed which are greater than a first preset threshold, to obtain residual candidate point cloud points; step S3: rasterizing the candidate point cloud points to obtain coordinates of the candidate point cloud points in a grid, wherein the grid comprises a plurality of first sub-grids; step S4: traversing the first sub-grids, and calculating a fitting plane equation of each first sub-grid; and step S5: calculating a first distance between each candidate point cloud point and a fitting plane corresponding to the fitting plane equation of each first sub-grid, and using, as ground detection point cloud points, the candidate point cloud points of which the first distance is less than a second preset threshold. The present invention can improve the accuracy of detection of ground point cloud points.
Need to check novelty before this filing date? Find Prior Art

Description

Ground detection method, system and computer-readable medium based on laser point cloud Technical Field

[0001] The present invention mainly relates to the field of laser radar point cloud detection technology, and specifically relates to a ground detection method, system and computer-readable medium based on laser point cloud. Background Art

[0002] In the field of autonomous driving, vehicles are often equipped with LiDAR to perceive their surroundings. LiDAR has the characteristics of high detection resolution, strong anti-interference ability, rich information acquisition, and the ability to operate around the clock. During the autonomous driving process, the vehicle needs to detect targets in the surrounding environment. For example, when obstacles such as pedestrians, stones, mounds of earth, and fences are detected, the vehicle needs to take appropriate evasive measures. When performing target detection based on LiDAR's laser point cloud, part of the field of view of the vehicle's LiDAR may cover the ground, and the ground point cloud will affect the judgment of the obstacle size. The ground point cloud is invalid sensor information for target detection tasks. Therefore, it is necessary to filter out ground point cloud points according to the ground detection method and then remove the ground point cloud points to prevent the ground point cloud points from interfering with target detection.

[0003] Existing ground detection methods typically perform ground fitting based on manually annotated high-precision maps. These manually annotated high-precision maps include various road elements such as road boundaries. Ground detection methods perform ground fitting on laser point cloud points within the marked road boundaries to filter out ground point cloud points. Existing ground detection methods are highly dependent on manually annotated high-precision maps, which cannot fully cover all vehicle driving scenarios. Consequently, ground point cloud detection accuracy is low in some vehicle driving scenarios.

[0004] Summary of the Invention

[0005] The technical problem to be solved by this application is to provide a ground detection method, system and computer-readable medium based on laser point cloud, which can improve the accuracy of ground point cloud point detection and is suitable for most scenarios of vehicle driving.

[0006] The technical solution adopted by the present application to solve the above-mentioned technical problems is a ground detection method based on laser point cloud, wherein the laser point cloud includes at least one point cloud point, and the ground detection method includes: step S1: inputting each point cloud point into a semantic segmentation neural network model, and the semantic segmentation neural network model outputs the category of each point cloud point, wherein the categories include ground and non-ground; step S2: taking each point cloud point classified as ground as a point cloud point to be processed, removing the point cloud points to be processed that are greater than a first preset threshold from the point cloud points to be processed, and obtaining the remaining at least one candidate point cloud point; step S3: rasterizing each candidate point cloud point to obtain the coordinates of each candidate point cloud point in the grid, wherein the grid includes multiple first sub-grids; step S4: traversing each first sub-grid and calculating the fitting plane equation of each first sub-grid; and step S5: calculating the first distance between each candidate point cloud point and the fitting plane corresponding to the fitting plane equation of each first sub-grid, and taking the candidate point cloud points whose first distance is less than the second preset threshold as ground detection point cloud points.

[0007] In one embodiment of the present application, in step S4, the step of calculating the fitting plane equation of each first sub-grid includes: step S4a: randomly selecting multiple non-collinear candidate point cloud points in the current first sub-grid, and constructing a candidate plane equation based on the coordinates of the multiple candidate point cloud points; step S4b: calculating the inlier rate in the current first sub-grid based on the candidate plane equation; step S4c: judging whether to use the candidate plane equation as the fitting plane equation of the current first sub-grid based on the inlier rate in the current first sub-grid.

[0008] In one embodiment of the present application, in step S4 , while traversing each first sub-grid, steps S4 a to S4 c are repeated.

[0009] In one embodiment of the present application, the first sub-grid includes multiple second sub-grids; step S4a further includes: randomly selecting three non-collinear candidate point cloud points in any second sub-grid of the current first sub-grid, and constructing a candidate plane equation based on the coordinates of the three candidate point cloud points.

[0010] In one embodiment of the present application, the first sub-grid includes a low-resolution grid, and the second sub-grid includes a high-resolution grid.

[0011] In one embodiment of the present application, in step S4b, the step of calculating the inlier rate in the current first sub-grid based on the candidate plane equation includes: step S4ba: calculating the second distance between each candidate point cloud point in the current first sub-grid and the fitting plane corresponding to the candidate plane equation; step S4bb: determining whether the second distance is less than a third preset threshold value, and if so, recording the candidate point cloud point corresponding to the second distance as an inlier; if not, recording the candidate point cloud point corresponding to the second distance as an outlier; step S4bc: calculating the ratio of the number of inliers to the number of candidate point cloud points in the current first sub-grid, and obtaining the inlier rate in the current first sub-grid.

[0012] In one embodiment of the present application, in step S4c, the step of determining whether to use the candidate plane equation as the fitting plane equation of the current first sub-grid based on the inlier rate in the current first sub-grid includes: determining whether the inlier rate in the current first sub-grid is greater than a fourth preset threshold; if so, using the candidate plane equation as the fitting plane equation of the current first sub-grid; if not, repeating steps S4a to S4c and recording the current number of iterations; if the current number of iterations is greater than or equal to the fifth preset threshold, ending the traversal of the current first sub-grid.

[0013] In one embodiment of the present application, after step S4, each first sub-grid corresponds to a fitting plane equation; in step S5, the first distance between each candidate point cloud point and the fitting plane corresponding to the fitting plane equation of each first sub-grid is calculated using the following formula:

[0014] Among them, H i,j represents the first distance between the candidate point cloud point included in the i-th row and j-th column grid and the fitting plane corresponding to the current fitting plane equation, W m,n Represents the grid weight of the mth row and nth column. K represents the neighborhood of the current grid. The neighborhood of the current grid includes the number of other first subgrids around the current grid. i, j, m, n, and K are all integers.

[0015] In one embodiment of the present application, a semantic segmentation neural network model is constructed based on a convolutional neural network, and the semantic segmentation neural network model includes a bird's-eye view module and a range view module; in step S1, each point cloud point is input into the semantic segmentation neural network model, and the step of the semantic segmentation neural network model outputting the category of each point cloud point includes: step S1a: inputting each point cloud point into the bird's-eye view module and the range view module respectively to obtain a bird's-eye view feature vector and a range view feature vector; step S1b: using multiple layers of convolutional layers to perform dimensionality reduction processing on the bird's-eye view feature vector and the range view feature vector respectively to obtain a bird's-eye view high-order feature vector and a range view high-order feature vector; step S1c: using multiple layers of deconvolution layers to perform dimensionality increase processing on the bird's-eye view high-order feature vector and the range view high-order feature vector respectively to obtain a bird's-eye view increased dimensional feature vector and a range view increased dimensional feature vector; step S1d: weighted fusion of the bird's-eye view increased dimensional feature vector and the range view increased dimensional feature vector to obtain a fused feature vector, and outputting the category of each point cloud point according to the fused feature vector.

[0016] In one embodiment of the present application, the non-ground items in the categories include: one or more of: vehicles, pedestrians, cyclists, green plants, and curbs.

[0017] In order to solve the above technical problems, the present application also proposes a ground detection system based on laser point cloud, comprising: a memory for storing instructions executable by a processor; a processor for executing instructions to implement the above ground detection method based on laser point cloud.

[0018] In order to solve the above technical problems, the present application also proposes a computer-readable medium storing computer program code, which implements the above-mentioned ground detection method based on laser point cloud when executed by a processor.

[0019] The technical solution of the present application performs semantic segmentation on the point cloud points according to the semantic segmentation neural network model to obtain point cloud points of the ground category and point cloud points of the non-ground category. The semantic segmentation neural network model can adaptively identify the point cloud point category, which can improve the versatility of laser point cloud classification. After removing the point cloud points greater than the first preset threshold from the point cloud points of the ground category, the present application uses the remaining point cloud points as candidate point cloud points. After rasterizing the candidate point cloud points, it is convenient to further screen the required candidate point cloud points in the grid. By calculating the first distance between each candidate point cloud point and the fitting plane corresponding to the fitting plane equation of each first sub-grid, the candidate point cloud points with the first distance less than the second preset threshold are used as ground detection point cloud points, which can effectively screen out the actual ground point cloud points, improve the accuracy of ground point cloud point detection, and subsequently eliminate the screened ground detection point cloud points, thereby avoiding the interference of ground point cloud points on the target detection task. The present application does not rely on manually annotated high-precision maps and is applicable to most scenarios of vehicle driving.

[0020] Summary of the Figures

[0021] In order to make the above-mentioned objects, features and advantages of the present application more clearly understood, the specific embodiments of the present application are described in detail below with reference to the accompanying drawings, wherein:

[0022] FIG1 is an exemplary flow chart of a ground detection method based on laser point cloud according to an embodiment of the present application;

[0023] FIG2 is an exemplary schematic diagram of a semantic segmentation neural network model according to an embodiment of the present application;

[0024] FIG3 is an exemplary schematic diagram of a point cloud point within a grid in one embodiment of the present application;

[0025] FIG4 is a system block diagram of a ground detection system based on laser point cloud according to an embodiment of the present application.

[0026] Preferred embodiments of the present invention

[0027] In order to make the above-mentioned objectives, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below with reference to the accompanying drawings.

[0028] In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0029] As used in this application and the claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not intended to refer to the singular and may include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps or elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include additional steps or elements.

[0030] Flowcharts are used in this application to illustrate the operations performed by systems according to embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the various steps may be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0031] The present application proposes a ground detection method based on laser point cloud, which can be applied to vehicles equipped with laser radar to provide corresponding information for autonomous decision-making planning of vehicle automatic driving. The ground detection method based on laser point cloud of the present application can be run on the vehicle side, for example, in the controller of the vehicle, and can also be run on the cloud platform. When the ground detection method based on laser point cloud is run on the cloud platform, the vehicle-side data and the cloud platform data interact through a wireless network. Exemplarily, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an interconnected cloud, multiple clouds, etc. or any combination thereof. The present application does not limit the operating environment of the ground detection method based on laser point cloud.

[0032] FIG1 is an exemplary flow chart of a ground detection method based on a laser point cloud according to an embodiment of the present application. Referring to FIG1 , the ground detection method based on a laser point cloud according to this embodiment includes the following steps:

[0033] Step S1: Input each point cloud point into the semantic segmentation neural network model, and the semantic segmentation neural network model outputs the category of each point cloud point, where the categories include ground and non-ground.

[0034] Step S2: taking each point cloud point of the ground category as a point cloud point to be processed, removing the point cloud points to be processed that are greater than a first preset threshold from the point cloud points to be processed, and obtaining at least one remaining candidate point cloud point.

[0035] Step S3: rasterize each candidate point cloud point to obtain the coordinates of each candidate point cloud point in the grid, wherein the grid includes a plurality of first sub-grids.

[0036] Step S4: traverse each first sub-grid and calculate the fitting plane equation of each first sub-grid.

[0037] Step S5: Calculate the first distance between each candidate point cloud point and the fitting plane corresponding to the fitting plane equation of each first sub-grid, and use the candidate point cloud points whose first distance is less than the second preset threshold as ground detection point cloud points.

[0038] The above steps S1 to S5 are described in detail below:

[0039] In step S1, each point cloud point is input into a semantic segmentation neural network model, and the semantic segmentation neural network model outputs a category for each point cloud point, where the categories include ground and non-ground. In some embodiments, the non-ground in the category includes: one or more categories of vehicles, pedestrians, cyclists, green plants, and roadsides, and the ground in the category may include lane lines. Exemplarily, the point cloud point category output by the semantic segmentation neural network model is equivalent to semantic information, and the present application further processes the point cloud point data based on this semantic information.

[0040] FIG2 is an exemplary schematic diagram of a semantic segmentation neural network model according to an embodiment of the present application. Referring to FIG2 , in some embodiments, the semantic segmentation neural network model is constructed based on a convolutional neural network. The semantic segmentation neural network model includes a bird's-eye view module 221 and a range view module 222. In step S1, each point cloud point 210 is input into the semantic segmentation neural network model. The steps of outputting the category 260 of each point cloud point by the semantic segmentation neural network model include:

[0041] Step S1a: Input each point cloud point 210 into the bird's-eye view module 221 and the range view module 222 respectively to obtain a bird's-eye view feature vector and a range view feature vector.

[0042] Step S1b: Use multi-layer convolutional layers 231 and 232 to perform dimensionality reduction processing on the bird's-eye view feature vector and the range view feature vector respectively to obtain the bird's-eye view high-order feature vector and the range view high-order feature vector.

[0043] Step S1c: Use multi-layer deconvolution layers 241 and 242 to perform dimensionality upscaling on the bird's-eye view high-order feature vector and the range view high-order feature vector, respectively, to obtain the bird's-eye view upscaling feature vector and the range view upscaling feature vector.

[0044] Step S1d: In the feature vector fusion module 250 , the bird's-eye view up-dimensional feature vector and the range view up-dimensional feature vector are weightedly fused to obtain a fused feature vector, and the category 260 of each point cloud point is output according to the fused feature vector.

[0045] Continuing to refer to FIG2 , exemplarily, the semantic segmentation neural network model of the present application processes the point cloud point 210 through the bird's-eye view module 221 and the range view module 222 to obtain a multi-perspective point cloud feature vector, which can optimize the overall performance of the semantic segmentation neural network model. In the aforementioned step S1b, the multi-layer convolution layer 231 and the multi-layer convolution layer 232 may include the same convolution structure or different convolution structures, and the multi-layer convolution layer 231 and the multi-layer convolution layer 232 may synchronously process the bird's-eye view feature vector output by the bird's-eye view module 221 and the range view feature vector output by the range view module 222. After the processing of the aforementioned step S1b, the high-order features of the point cloud points can be extracted based on the local correlation of the feature vectors, which is equivalent to encoding the original point cloud point features. In the aforementioned step S1c, the multi-layer deconvolution layer 241 and the multi-layer deconvolution layer 242 may include the same deconvolution structure or different deconvolution structures, and the multi-layer deconvolution layer 241 and the multi-layer deconvolution layer 242 may synchronously process the bird's-eye view high-order feature vector output by the multi-layer convolution layer 231 and the range view high-order feature vector output by the multi-layer convolution layer 232. After the processing of the aforementioned step S1c, it is equivalent to decoding the high-order features and mapping them back to the original point cloud features. After the processing of the aforementioned step S1d, the point cloud segmentation results under different perspectives can be obtained based on the projection relationship between the point cloud points. The semantic segmentation neural network model of the present application can adaptively identify point cloud point categories, which can improve the versatility of laser point cloud classification.

[0046] In step S2, each point cloud point classified as ground is used as a point cloud point to be processed, and the point cloud points to be processed that are greater than a first preset threshold are removed from the point cloud points to obtain at least one remaining candidate point cloud point. For example, the present application removes some point cloud points from the point cloud points to be processed and uses the remaining point cloud points to be processed as candidate point cloud points. These candidate point cloud points retain more ground information, and subsequent further processing of these candidate point cloud points can improve the accuracy of ground point cloud point detection.

[0047] FIG3 is an exemplary schematic diagram of point cloud points within a grid in an embodiment of the present application. Referring to FIG3 , the small black dots in FIG3 represent candidate point cloud points 310, and the grid marked by the dashed box represents the first subgrid 330. The embodiment shown in FIG3 includes 5 rows and 6 columns, for a total of 30 first subgrids 330. The present application does not limit the number of first subgrids. In step S3, each candidate point cloud point 310 is rasterized to obtain the coordinates of each candidate point cloud point 310 within a grid 320, wherein the grid 320 includes multiple first subgrids 330. Exemplarily, the process of rasterizing the candidate point cloud points 310 includes initially generating an empty grid, converting the original three-dimensional coordinate information of each candidate point cloud point 310 into coordinate information corresponding to the grid 320, and assigning each candidate point cloud point 310 to a grid 320 based on the converted coordinate information, which is equivalent to establishing an index for each candidate point cloud point 310.

[0048] Continuing with FIG3 , in step S4, each first sub-grid 330 is traversed and the fitting plane equation of each first sub-grid 330 is calculated. Exemplarily, step S4 is equivalent to performing random consistency sampling on the grid 320, and subsequently performing ground fitting processing based on the fitting plane equation of each first sub-grid 330.

[0049] In some embodiments, in step S4, the step of calculating the fitting plane equation for each first subgrid 330 includes the following: Step S4a: Randomly selecting multiple non-collinear candidate point cloud points 310 in the current first subgrid 330, and constructing a candidate plane equation based on the coordinates of the multiple candidate point cloud points 310. Step S4b: Calculating the inlier rate in the current first subgrid 330 based on the candidate plane equation. Step S4c: Determining whether to use the candidate plane equation as the fitting plane equation for the current first subgrid 330 based on the inlier rate in the current first subgrid 330. In some embodiments, in step S4, steps S4a to S4c are repeated while traversing each first subgrid 330.

[0050] Continuing to refer to FIG3 , in some embodiments, the first sub-grid 330 includes a plurality of second sub-grids 331. For example, in the embodiment shown in FIG3 , each first sub-grid 330 includes nine second sub-grids 331. The present application does not limit the number of second sub-grids. Step S4a further includes: randomly selecting three non-collinear candidate point cloud points 310 from any number of second sub-grids 331 of the current first sub-grid 330, and constructing a candidate plane equation based on the coordinates of the three candidate point cloud points 310. In some embodiments, the present application uses a multi-resolution rasterization processing method when rasterizing each candidate point cloud point 310, so that the first sub-grid 330 includes a low-resolution grid and the second sub-grid 331 includes a high-resolution grid.

[0051] In some embodiments, in step S4b, the step of calculating the inlier rate in the current first subgrid 330 based on the candidate plane equation includes: step S4ba: calculating the second distance between each candidate point cloud point 310 in the current first subgrid 330 and the fitted plane corresponding to the candidate plane equation. Step S4bb: determining whether the second distance is less than a third preset threshold; if so, recording the candidate point cloud point 310 corresponding to the second distance as an inlier; if not, recording the candidate point cloud point 310 corresponding to the second distance as an outlier. Step S4bc: calculating the ratio of the number of inliers in the current first subgrid 330 to the number of candidate point cloud points 310 to obtain the inlier rate in the current first subgrid 330.

[0052] Continuing with reference to FIG. 3 , in some embodiments, in step S4c, the step of determining whether to use the candidate plane equation as the fitting plane equation of the current first subgrid 330 based on the inlier rate in the current first subgrid 330 includes: determining whether the inlier rate in the current first subgrid 330 is greater than a fourth preset threshold; if so, using the candidate plane equation as the fitting plane equation of the current first subgrid 330; if not, repeating steps S4a to S4c and recording the current number of iterations, and ending the traversal of the current first subgrid 330 if the current number of iterations is greater than or equal to a fifth preset threshold. Exemplarily, the complete random consistency sampling process includes steps S4a to S4c, and when the inlier rate in the current first subgrid 330 is greater than the fourth preset threshold, or when the current number of iterations is greater than or equal to the fifth preset threshold, the random consistency sampling is terminated, thereby obtaining a fitting plane equation that meets the conditions.

[0053] In step S5, a first distance is calculated between each candidate point cloud point 310 in the grid 320 and the fitting plane corresponding to the fitting plane equation of each first sub-grid 330, and the candidate point cloud point 310 whose first distance is less than a second preset threshold is used as a ground detection point cloud point. In some embodiments, after the aforementioned step S4, each first sub-grid 330 corresponds to a fitting plane equation; in step S5, the first distance between each candidate point cloud point 310 and the fitting plane corresponding to the fitting plane equation of each first sub-grid 330 is calculated using the following formula (1):

[0054] Among them, H i,j represents the first distance between the candidate point cloud point 310 included in the i-th row and j-th column grid and the fitting plane corresponding to the current fitting plane equation, W m,n represents the grid weight of the mth row and nth column, K represents the neighborhood of the current grid, and the neighborhood of the current grid includes the number of other first sub-grids 330 around the current grid. i, j, m, n, and K are all integers.

[0055] For example, the first sub-grid 330 is equivalent to a low-resolution grid. In the process of performing ground fitting processing on the coarse-grained first sub-grid 330 to calculate its fitting plane equation, it is necessary to consider the consistency between the plane equation of the current first sub-grid 330 and the plane equations of other adjacent first sub-grids 330. Therefore, all the fitting plane equations are optimized as a whole through the above formula (1), which can make the ground fitting processing process smooth as a whole and improve the accuracy of ground point cloud point detection.

[0056] The technical solution of the present application performs semantic segmentation on the point cloud points according to the semantic segmentation neural network model to obtain point cloud points of the ground category and point cloud points of the non-ground category. The semantic segmentation neural network model can adaptively identify the point cloud point category, which can improve the versatility of laser point cloud classification. After removing the point cloud points greater than a first preset threshold from the point cloud points of the ground category, the present application uses the remaining point cloud points as candidate point cloud points 310. After rasterizing the candidate point cloud points 310, it is convenient to further screen the required candidate point cloud points 310 in the grid 320. By calculating the first distance between each candidate point cloud point 310 and the fitting plane corresponding to the fitting plane equation of each first sub-grid 330, the candidate point cloud points 310 with the first distance less than the second preset threshold are used as ground detection point cloud points. The actual ground point cloud points can be effectively screened out, which can improve the accuracy of ground point cloud point detection. The screened ground detection point cloud points can be subsequently eliminated, thereby avoiding the interference of ground point cloud points on the target detection task. The present application does not rely on manually annotated high-precision maps and is applicable to most scenarios of vehicle driving.

[0057] The present application also includes a laser point cloud-based ground detection system, comprising a memory and a processor, wherein the memory is used to store instructions executable by the processor; and the processor is used to execute the instructions to implement the laser point cloud-based ground detection method described above.

[0058] FIG4 is a system block diagram of a laser point cloud-based ground detection system according to an embodiment of the present application. Referring to FIG4 , the laser point cloud-based ground detection system 400 may include a vehicle system 410 and a server computer 420. The vehicle system 410 wirelessly exchanges data with the server computer 420 via a communication network 430. The vehicle system 410 includes a vehicle communication port 411, a vehicle processor 412, a read-only memory (ROM) 413, and a random access memory (RAM) 414. In some embodiments, the vehicle processor 412 may be composed of one or more processors. The vehicle communication port 411 enables data communication between the vehicle system 410 and the outside world. The vehicle system 410 may also include various forms of program storage units and data storage units, such as the read-only memory (ROM) 413 and the random access memory (RAM) 414, capable of storing various data files used for processing and / or communication by the vehicle system 410, as well as possible program instructions executed by the vehicle processor 412. The vehicle processor 412 executes these instructions to implement the main components of the method. The results processed by the vehicle processor 412 can be transmitted to the user device through the vehicle communication port 411 and displayed on the user interface.

[0059] The above-mentioned ground detection method based on laser point cloud can be implemented as a processor program, stored in the program storage unit of the vehicle system 410, and can be loaded into the vehicle processor 412 for execution to implement the ground detection method based on laser point cloud of the present application.

[0060] The present application also includes a computer-readable medium storing computer program code, which, when executed by a processor, implements the above-mentioned ground detection method based on laser point cloud.

[0061] When the ground detection method based on laser point cloud is implemented as a computer program, it can also be stored in a computer-readable storage medium as an article of manufacture. For example, a computer-readable storage medium may include, but is not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips), optical disks (e.g., compact disks (CDs), digital versatile disks (DVDs)), smart cards, and flash memory devices (e.g., electrically erasable programmable read-only memories (EPROMs), cards, sticks, key drives). In addition, the various storage media described herein can represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" may include, but is not limited to, wireless channels and various other media (and / or storage media) that can store, contain, and / or carry code and / or instructions and / or data.

[0062] It should be understood that the embodiments described above are merely illustrative. The embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For hardware implementation, the processor may be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, and / or other electronic units designed to perform the functions described herein, or a combination thereof.

[0063] Some aspects of the present application can be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software can be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". The processor can be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors or combinations thereof. In addition, various aspects of the present application may be expressed as computer products located in one or more computer-readable media, which include computer-readable program code. For example, computer-readable media may include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, tapes...), optical disks (e.g., compact disks CDs, digital versatile disks DVDs...), smart cards, and flash memory devices (e.g., cards, sticks, key drives...).

[0064] A computer-readable medium may include a propagated data signal embodying computer program code, for example, in baseband or as part of a carrier wave. The propagated signal may be in a variety of forms, including electromagnetic, optical, etc., or a suitable combination thereof. A computer-readable medium may be any computer-readable medium other than a computer-readable storage medium that can be connected to an instruction execution system, apparatus, or device to communicate, propagate, or transmit the program for use. The program code on the computer-readable medium may be transmitted via any suitable medium, including radio, cable, fiber optic cable, radio frequency signal, or similar medium, or any combination of the above.

[0065] The basic concepts have been described above. It will be apparent to those skilled in the art that the above disclosures are merely examples and do not limit the present application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and revisions to the present application. Such modifications, improvements, and revisions are suggested in the present application and remain within the spirit and scope of the exemplary embodiments of the present application.

[0066] At the same time, this application uses specific terms to describe the embodiments of this application. For example, "one embodiment," "an embodiment," and / or "some embodiments" refer to a certain feature, structure, or characteristic related to at least one embodiment of this application. Therefore, it should be emphasized and noted that "one embodiment," "an embodiment," or "an alternative embodiment" mentioned twice or multiple times in different locations in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this application may be appropriately combined.

[0067] In some embodiments, numbers are used to describe the quantity of components and attributes. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise stated, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the description and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of the present application are approximate values, in specific embodiments, the settings of such numerical values ​​are as accurate as possible within the feasible range.

Claims

1. A ground detection method based on laser point cloud, wherein the laser point cloud comprises at least one point cloud point, characterized in that: include: Step S1: input each point cloud point into a semantic segmentation neural network model, and the semantic segmentation neural network model outputs the category of each point cloud point, wherein the category includes ground and non-ground; Step S2: taking each point cloud point of the ground category as a point cloud point to be processed, removing the point cloud points to be processed that are greater than a first preset threshold from the point cloud points to be processed, and obtaining the remaining at least one candidate point cloud point; Step S3: rasterizing each candidate point cloud point to obtain the coordinates of each candidate point cloud point in the grid, wherein the grid includes a plurality of first sub-grids; Step S4: traverse each first sub-grid and calculate the fitting plane equation of each first sub-grid; and Step S5: Calculate the first distance between each candidate point cloud point and the fitting plane corresponding to the fitting plane equation of each first sub-grid, and take the candidate point cloud points whose first distance is less than a second preset threshold as ground detection point cloud points.

2. The ground detection method according to claim 1, characterized in that: In step S4, the step of calculating the fitting plane equation of each first sub-grid includes: Step S4a: randomly selecting a plurality of non-collinear candidate point cloud points in the current first sub-grid, and constructing a candidate plane equation according to the coordinates of the plurality of candidate point cloud points; Step S4b: calculating the inlier rate in the current first sub-grid according to the candidate plane equation; Step S4c: judging whether to use the candidate plane equation as the fitting plane equation of the current first sub-grid according to the inlier rate in the current first sub-grid.

3. The ground detection method according to claim 2, characterized in that: In step S4, in the process of traversing each first sub-grid, steps S4a to S4c are repeated.

4. The ground detection method according to claim 2, characterized in that: The first sub-grid includes multiple second sub-grids; the step S4a further includes: randomly selecting three non-collinear candidate point cloud points in any second sub-grid of the current first sub-grid, and constructing a candidate plane equation according to the coordinates of the three candidate point cloud points.

5. The ground detection method according to claim 4, characterized in that: The first sub-grid includes a low-resolution grid, and the second sub-grid includes a high-resolution grid.

6. The ground detection method according to claim 2, characterized in that: In the step S4b, the step of calculating the inlier rate in the current first sub-grid according to the candidate plane equation includes: Step S4ba: calculating a second distance between each candidate point cloud point in the current first sub-grid and the fitting plane corresponding to the candidate plane equation; Step S4bb: determining whether the second distance is less than a third preset threshold value, if so, recording the candidate point cloud point corresponding to the second distance as an inner point; if not, recording the candidate point cloud point corresponding to the second distance as an outer point; Step S4bc: Calculate the ratio of the number of the inliers to the number of the candidate point cloud points in the current first sub-grid to obtain the inlier rate in the current first sub-grid.

7. The ground detection method according to claim 2 or 6, characterized in that: In step S4c, the step of judging whether to use the candidate plane equation as the fitting plane equation of the current first sub-grid according to the inlier rate in the current first sub-grid includes: Determining whether the inlier rate in the current first sub-grid is greater than a fourth preset threshold; If yes, the candidate plane equation is used as the fitting plane equation of the current first sub-grid; If not, repeat the steps S4a to S4c and record the current number of iterations. If the current number of iterations is greater than or equal to the fifth preset threshold, end traversing the current first sub-grid.

8. The ground detection method according to claim 1, characterized in that: After step S4, each first sub-grid corresponds to a fitting plane equation; in step S5, the first distance between each candidate point cloud point and the fitting plane corresponding to the fitting plane equation of each first sub-grid is calculated using the following formula: Among them, H i,j represents the first distance between the candidate point cloud point included in the i-th row and j-th column grid and the fitting plane corresponding to the current fitting plane equation, W m,n represents the grid weight of the mth row and the nth column, K represents the neighborhood of the current grid, and the neighborhood of the current grid includes the number of other first sub-grids around the current grid. i, j, m, n, and K are all integers.

9. The ground detection method according to claim 1, characterized in that: The semantic segmentation neural network model is constructed based on a convolutional neural network, and the semantic segmentation neural network model includes a bird's-eye view module and a range view module; in step S1, each point cloud point is input into the semantic segmentation neural network model, and the semantic segmentation neural network model outputs the category of each point cloud point, including: Step S1a: input each point cloud point into the bird's-eye view module and the range view module respectively to obtain a bird's-eye view feature vector and a range view feature vector; Step S1b: using a multi-layer convolutional layer to perform dimensionality reduction processing on the bird's-eye view feature vector and the range view feature vector respectively, to obtain a bird's-eye view high-order feature vector and a range view high-order feature vector; Step S1c: using a multi-layer deconvolution layer to perform dimension-upgrading processing on the bird's-eye view high-order feature vector and the range view high-order feature vector respectively, to obtain a bird's-eye view dimension-upgrading feature vector and a range view dimension-upgrading feature vector; Step S1d: weightedly fuse the bird's-eye view up-dimensional feature vector and the range view up-dimensional feature vector to obtain a fused feature vector, and output the category of each point cloud point according to the fused feature vector.

10. The ground detection method according to claim 1, characterized in that: The non-ground in the categories include: one or more of vehicles, pedestrians, cyclists, green plants, and curbs.

11. A ground detection system based on laser point cloud, characterized in that: include: a memory for storing instructions executable by a processor; A processor, configured to execute the instructions to implement the ground detection method as described in any one of claims 1-10.

12. A computer readable medium storing computer program code, characterized in that: When the computer program code is executed by a processor, the ground detection method according to any one of claims 1 to 10 is implemented.

Citation Information

Patent Citations

  • Clustering method fusing point cloud semantic category and distance

    CN114648654A

  • Unmanned mine car ground detection method and device

    CN115564920A

  • Road three-dimensional map construction method based on road edge detection

    CN116152459A

  • Ground detection method and system based on laser point cloud and computer readable medium

    CN116704461A

  • Identifying objects using LiDAR

    GB2608201A

Cited By

  • Ground segmentation method and system based on adaptive grid division and intensity information

    CN119359746A

  • Point cloud-based ground line generation method and device and intelligent driving equipment

    CN121982131A

  • A ground line generation method and device based on a point cloud and an intelligent driving device

    CN121982131B