A road traversable area extraction method and extraction system

Abstract

The present invention provides a method and system for extracting traversable areas of a road surface. The extraction method includes: extracting a point cloud road surface based on point cloud data; segmenting the traversable area of ​​the road surface in the image data based on a deep learning semantic segmentation model; inverse perspective transforming the segmented image traversable area into the three-dimensional space of the point cloud data based on posture information; extracting a point cloud traversable area from the point cloud road surface based on the image traversable area in the three-dimensional space of the point cloud data that is inverse perspective transformed; dividing the point cloud traversable area into a safety level, and vectorizing the boundaries of areas of different levels. The present invention improves the efficiency of extracting traversable areas during driving of autonomous vehicles by means of image deep learning technology. It extracts traversable areas from point cloud data based on image data and three-dimensional point cloud data. Compared with single data sources and traditional extraction methods, it has better scene generalization capabilities and is also of certain reference value for map updates.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving environment perception, and more specifically, to a method and system for extracting drivable areas on a road surface. Background Art

[0002] High-precision maps for autonomous driving are an essential component of autonomous vehicles, providing valuable support for vehicle positioning, safe driving, route planning, and energy conservation. However, in real-world autonomous driving scenarios, to ensure maximum safety, environmental perception requires the fusion of multi-source data. Therefore, in addition to the foundational support of high-precision maps (HDMAPs), the strengths of individual sensors must be leveraged to improve the reliability of environmental perception information. Real-time extraction of traversable areas is particularly crucial. These areas constitute safety zones during vehicle operation, and all real-time vehicle operations are based on this extraction. Furthermore, real-time extraction of traversable areas provides reference information for map updates. To achieve this goal, cutting-edge image deep learning technologies are required to ensure the accuracy of perception results in real-time and efficient perception. Summary of the Invention

[0003] The present invention aims to solve the technical problems existing in the prior art and provides a method and system for extracting traversable areas of a road surface.

[0004] According to a first aspect of the present invention, a method for extracting traversable areas of a road surface is provided, comprising: extracting a point cloud road surface based on point cloud data; segmenting the traversable area of ​​the road surface in image data based on a deep learning semantic segmentation model; inverse perspective transforming the segmented image traversable area into the three-dimensional space of the point cloud data based on posture information; extracting a point cloud traversable area from the point cloud road surface based on the image traversable area in the three-dimensional space of the point cloud data based on the inverse perspective transformation; dividing the point cloud traversable area into safety levels, and performing boundary vectorization on areas of different levels.

[0005] On the basis of the above technical solution, the present invention can also make the following improvements.

[0006] Optionally, extracting a point cloud road surface based on point cloud data includes:

[0007] Each frame of point cloud data is divided into blocks with equal spacing along the heading direction to obtain multiple point cloud data blocks, each of which contains multiple point cloud points;

[0008] For any point cloud data block, a ground seed point is selected, and based on the ground seed point, multiple initial ground points are obtained;

[0009] Based on the ground seed points and multiple initial ground points, plane fitting is performed to obtain the initial fitting plane;

[0010] Updating the ground point based on the distance between each point cloud point in any point cloud data block and the initial fitting plane;

[0011] Based on the updated ground points, plane fitting is performed again. After multiple iterations of plane fitting, the ground points finally updated are the point cloud road surface extraction results.

[0012] Optionally, each frame of point cloud data corresponds to a GPS trajectory point, and for any point cloud data block, a ground seed point is selected, and multiple initial ground points are obtained based on the ground seed point, including:

[0013] Convert the GPS track points and each frame of point cloud data to the point cloud XOY plane;

[0014] For all point cloud points in any point cloud data block, calculate the distance between each point cloud point and the GPS track point, and use the point cloud point with the smallest distance to the GPS track point as the ground seed point;

[0015] Taking the ground seed point as the center point, searching for point cloud points within a preset radius in any point cloud data block, and taking the ground seed point and the point cloud points within the preset radius as initial ground points.

[0016] Optionally, updating the ground point based on the distance between each point cloud point in any point cloud data block and the initial fitting plane includes:

[0017] The distance value from each point cloud point except the initial ground point in any point cloud data block to the initial fitting plane is calculated, and the point cloud points with distance values ​​less than a preset distance threshold are included in the ground points to update the ground points.

[0018] Optionally, re-performing plane fitting based on the updated ground points, and after multiple iterations of plane fitting, finally updating the ground points to obtain the point cloud road surface extraction results, including:

[0019] Re-fitting the plane based on the updated ground points, and again updating the ground points based on the distance between each point cloud point in any point cloud data block and the re-fitted plane;

[0020] The plane is re-iterated until the distance between each point cloud point in any point cloud data block and the re-fitted plane is greater than a preset distance threshold, and the finally obtained ground point is used as the point cloud road surface extraction result.

[0021] Optionally, obtain pose information as follows:

[0022] Based on the initialization pose information, the landmark elements in the high-precision map HD MAP are converted to the image coordinate system;

[0023] In the image coordinate system, the landmark elements in the HD MAP are matched with the same landmark elements semantically segmented from the image data to obtain the matching results;

[0024] Based on the matching results, GPS information and IMU data, the initialization pose information is optimized;

[0025] Correspondingly, based on the optimized pose information, the segmented image passable area is inversely transformed into the three-dimensional space of the point cloud data.

[0026] Optionally, the extracting of the point cloud traversable area from the point cloud road surface based on the image traversable area in the three-dimensional space of the point cloud data based on the inverse perspective transformation includes:

[0027] The image passable area converted from the inverse perspective to the three-dimensional space of the point cloud data is matched with the point cloud road surface to obtain the matched point cloud passable area in the point cloud road surface.

[0028] Optionally, the different levels of areas include an absolute safety zone, a relative safety zone, and a non-relative safety zone. The security level classification of the point cloud passable area and the boundary vectorization of the different levels of areas include:

[0029] When the HD MAP road surface range is smaller than the point cloud traversable area, the absolute safety zone is the HD MAP road surface range, and the relative safety zone is the point cloud traversable area minus the HD MAP road surface range;

[0030] When the HD MAP road surface range is larger than the point cloud traversable area, the absolute safety zone is the point cloud traversable area, and the non-relative safety zone is the area other than the HD MAP road surface range minus the point cloud traversable area.

[0031] Vectorize the left and right boundaries of different levels of safety areas.

[0032] According to a second aspect of the present invention, a road traversable area extraction system is provided, comprising:

[0033] A first extraction module is used to extract a point cloud road surface based on the point cloud data;

[0034] The segmentation module is used to segment the drivable area of ​​the road in the image data based on the deep learning semantic segmentation model;

[0035] A transformation module is used to perform an inverse perspective transformation of the segmented image passable area into the three-dimensional space of the point cloud data based on the pose information;

[0036] a second extraction module for extracting a point cloud traversable area from the point cloud road surface based on an image traversable area converted from an inverse perspective transformation to a three-dimensional space of the point cloud data;

[0037] The vectorization module is used to divide the passable area of ​​the point cloud into safety levels and to vectorize the boundaries of areas of different levels.

[0038] According to a third aspect of the present invention, an electronic device is provided, comprising a memory and a processor, wherein the processor is configured to implement the steps of a method for extracting a traversable area of ​​a road surface when executing a computer management program stored in the memory.

[0039] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer management program is stored. When the computer management program is executed by a processor, the steps of the method for extracting a traversable area on a road are implemented.

[0040] The present invention provides a method and system for extracting drivable areas on roads. Using high-precision 3D point clouds, forward-looking wide-angle images, GPS trajectories, IMUs, and odometry information as data sources, the system leverages the unique strengths of each sensor. By integrating multi-source data with semantic segmentation results from image deep learning techniques, the system extracts smoothed vectorized points of drivable areas from point cloud data. This system utilizes deep image learning techniques to improve the efficiency of drivable area extraction for autonomous vehicles. Compared to single-source extraction methods and traditional methods, the system offers superior scene generalization capabilities and is valuable for map updates. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A flow chart of a method for extracting traversable areas of a road provided by the present invention;

[0042] Figure 2 This is the overall flow chart of the road drivable area extraction method;

[0043] Figure 3 A schematic diagram of the structure of a road traversable area extraction system provided by the present invention;

[0044] Figure 4 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention;

[0045] Figure 5 A schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0046] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0047] In response to the current situation where a single data source cannot extract drivable areas with sufficient accuracy in real-world scenarios of autonomous driving, this invention uses high-precision three-dimensional point clouds, forward-looking wide-angle images, GPS trajectories, IMU, and odometer information as data sources, leverages the unique advantages of each sensor, and uses the semantic segmentation results of image deep learning technology to fuse multi-source data to extract smooth vectorized points of drivable areas from point cloud data.

[0048] Example 1

[0049] A method for extracting traversable areas on a road, see Figure 1 The method for extracting the traversable area of ​​a road surface includes: S1, extracting a point cloud road surface based on point cloud data.

[0050] The road point cloud data used in the present invention is point cloud data acquired by a multi-line laser radar, and the image is wide-angle image data. Based on the acquired point cloud data, the specific steps for extracting a point cloud road surface include: dividing each frame of point cloud data into blocks with equal spacing along the heading direction to obtain multiple point cloud data blocks, each of which contains multiple point cloud points; selecting a ground seed point for each point cloud data block, and obtaining multiple initial ground points based on the ground seed point; performing plane fitting based on the ground seed point and the multiple initial ground points to obtain an initial fitting plane; updating the ground point based on the distance between each point cloud point in the point cloud data block and the initial fitting plane; and re-performing plane fitting based on the updated ground point. After multiple iterations of plane fitting, the updated ground point is the point cloud road surface extraction result.

[0051] As an embodiment, each frame of point cloud data corresponds to a GPS track point, and for any point cloud data block, a ground seed point is selected, and based on the ground seed point, multiple initial ground points are obtained, including: converting the GPS track point and each frame of point cloud data to the point cloud XOY plane; for all point cloud points in any point cloud data block, calculating the distance between each point cloud point and the GPS track point, and taking the point cloud point with the smallest distance to the GPS track point as the ground seed point; with the ground seed point as the center point, searching for point cloud points within a preset radius in any point cloud data block, and taking the ground seed point and the point cloud points within the preset radius as the initial ground points.

[0052] As an embodiment, the updating of ground points based on the distance between each point cloud point in any point cloud data block and the initial fitting plane includes: calculating the distance value from each point cloud point in any point cloud data block except the initial ground point to the initial fitting plane, and incorporating point cloud points whose distance values ​​are less than a preset distance threshold into the ground points to update the ground points.

[0053] As an embodiment, the plane fitting is re-performed based on the updated ground points. After multiple iterations of plane fitting, the finally updated ground points are the point cloud road surface extraction results, including: re-fitting the plane based on the updated ground points, and again updating the ground points based on the distance between each point cloud point in any point cloud data block and the re-fitted plane; re-iteratively fitting the plane until the distance between each point cloud point in any point cloud data block and the re-fitted plane is greater than a preset distance threshold, and the finally obtained ground points are used as the point cloud road surface extraction results.

[0054] Specifically, after obtaining point cloud data of a road using a LiDAR sensor, road surface information needs to be extracted from it. This road surface information extracted from the point cloud data is hereinafter referred to as the point cloud road surface. The specific method for extracting the point cloud road surface from the point cloud data is as follows: to address incomplete road surface extraction due to slope issues, each frame of the point cloud is segmented into evenly spaced blocks along the heading direction, resulting in a number of point cloud data blocks, each of which corresponds to a GPS track point. Because the GPS track points and the point cloud data blocks are located in different coordinate systems, both the GPS track points and the point cloud data are converted to the point cloud XY plane. In the point cloud XY plane, for each of the multiple point cloud points in each point cloud data block, the distance from each point cloud point to the GPS track point is calculated. The point cloud point with the smallest distance to the GPS track point is selected as the ground seed point. Then, with the ground seed point as the center, all point cloud points within the point cloud block whose radial distance to the ground seed point is less than a preset distance are searched as ground points. A plane fitting is performed using the ground seed point and the selected ground points to obtain an initial fitting plane.

[0055] Then, for each other point cloud point in the point cloud data block, calculate the distance value from each point cloud point to the initial fitting plane, and include the point cloud points with distance values ​​less than the preset distance value into the ground points to update the ground points. Based on the updated ground points, perform plane fitting again, update more ground points, and iterate the plane fitting multiple times until the distance values ​​of all point cloud points in the point cloud data block in the fitting plane are greater than the preset distance value. The iteration is completed, and the ground points determined at this time are used as the point cloud road surface extraction results.

[0056] S2, based on the deep learning semantic segmentation model, segments the drivable area of ​​the road in the image data.

[0057] It's understandable that for image data, selecting an appropriate deep learning semantic segmentation model can segment the image data into traversable areas and obtain the traversable areas. Compared to point cloud deep learning technology, image deep learning technology is more mature, and its inference results are more robust and accurate, which makes up for the weak generalization of point cloud road surface coarse extraction.

[0058] S3, based on the pose information, the segmented image passable area is inversely transformed into the three-dimensional space of the point cloud data.

[0059] As an embodiment, the posture information is obtained in the following manner: based on the initialized posture information, the marker elements in the high-precision map HDMAP are converted to the image coordinate system; in the image coordinate system, the marker elements in the HD MAP are matched with the same marker elements semantically segmented from the image data to obtain a matching result; based on the matching result and the GPS information and IMU data, the initialized posture information is optimized; accordingly, based on the optimized posture information, the segmented image passable area is inversely transformed into the three-dimensional space of the point cloud data.

[0060] Specifically, before the vehicle drives, an initial posture information is determined. When the vehicle drives, the initial posture information needs to be continuously updated and optimized. Specifically, based on the initial posture information, the landmarks in the high-precision map HD MAP, such as traffic signs and poles, are converted to the image coordinate system. In the image coordinate system, the landmark elements in the HD MAP are matched with the same landmark elements semantically segmented from the image data to obtain a matching result.

[0061] The initial pose information is optimized based on the matching results and the vehicle's GPS information and IMU data. There are two situations when optimizing the initial pose: (1) When there are no poles, signboards, or even lane line elements, or only parallel road markings, the pose is updated and optimized based on the GPS information and IMU; in scenarios where there are no valid image semantic segmentation results, the GPS and IMU are used to ensure pose accuracy. (2) When the GPS signal is lost, the pose is updated and optimized using the IMU and odometer information; in the absence of valid GPS information, the estimated value calculated by the IMU and odometer information is used to supplement the driving trajectory, and the image optimization is performed after the GPS signal is restored.

[0062] Based on the optimized pose information, the traversable area of ​​the image is inversely transformed into the three-dimensional space of the point cloud data.

[0063] S4, extracting a point cloud traversable area from the point cloud road surface based on the image traversable area converted from the three-dimensional space of the point cloud data by inverse perspective transformation.

[0064] It can be understood that in step S3, the image passable area is inversely transformed into the three-dimensional space of the point cloud data, the image passable area is matched in the point cloud road surface, and the passable area is extracted from the point cloud road surface, which is hereinafter referred to as the point cloud passable area.

[0065] Using the optimized pose information, the image's traversable area is inversely transformed into 3D space and matched against the point cloud road surface. While ensuring accuracy, the actual road surface extracted from the point cloud is supplemented to address missing road surfaces in the point cloud caused by low point density. The result of this inverse perspective transformation is crucial in this step, determining the effectiveness of the inverse calculation of the image's 2D traversable area back into 3D space and compensating for any unnecessary planes or missing areas in the rough extraction of the point cloud road surface.

[0066] S5, dividing the passable area of ​​the point cloud into safety levels, and vectorizing the boundaries of areas of different levels.

[0067] As an embodiment, the different levels of areas include an absolute safety zone, a relative safety zone, and a non-relative safety zone. The point cloud traversable area is divided into safety levels and the boundaries of the different levels of areas are vectorized. This includes: when the HD MAP road surface range is smaller than the point cloud traversable area, the absolute safety zone is the HD MAP road surface range, and the relative safety zone is the area of ​​the point cloud traversable area minus the HD MAP road surface range; when the HD MAP road surface range is larger than the point cloud traversable area, the absolute safety zone is the point cloud traversable area, and the non-relative safety zone is the area other than the HD MAP road surface range minus the point cloud traversable area; and left and right boundaries of the different levels of safety areas are vectorized.

[0068] It's understandable that in the above steps, the point cloud traversable area is extracted from the point cloud road surface and divided into two safety levels. Specifically, the point cloud traversable area is divided into two levels, based on two situations: 1. When the HD MAP road surface area is smaller than the point cloud traversable area, the area containing the HD MAP road surface is the absolute safety zone, and the remaining area after subtracting the HD MAP road surface from the point cloud traversable area is the relative safety zone.

[0069] 2. When the HD MAP road surface area is larger than the point cloud traversable area, the point cloud traversable area is considered an absolute safety zone, while the HD MAP road surface area minus the point cloud traversable area is considered a relative safety zone. Finally, the safety zones of different levels are smoothed and their left and right boundaries are vectorized.

[0070] Example 2

[0071] A method for extracting traversable areas on a road, see Figure 2The method for extracting traversable road areas includes: for road point cloud data collected by a multi-line lidar sensor, segmenting each frame of point cloud data into multiple point cloud data blocks, and extracting the point cloud road surface from the point cloud data blocks. For wide-angle image data, semantic segmentation is performed on the image data to extract the traversable area from the image data. Based on the pose information, the traversable area is inversely transformed into the three-dimensional space of the point cloud data. The pose information is calculated based on GPS information, IMU data, and odometry information.

[0072] In the three-dimensional spatial coordinate system of the point cloud data, the image's passable area is matched with the point cloud road surface, and the passable area in the point cloud road surface (called the point cloud passable area) is extracted and output. Finally, the point cloud passable area is divided into multiple areas according to the safety level, and the boundaries of the multiple areas are vectorized.

[0073] Example 3

[0074] A road traversable area extraction system, see Figure 3 The road traversable area extraction system includes a first extraction module 301, a segmentation module 302, a transformation module 303, a second extraction module 304 and a vectorization module 305, wherein:

[0075] The first extraction module 301 is used to extract the point cloud road surface based on the point cloud data; the segmentation module 302 is used to segment the road surface passable area in the image data based on the deep learning semantic segmentation model; the transformation module 303 is used to perform an inverse perspective transformation of the segmented image passable area to the three-dimensional space of the point cloud data based on the posture information; the second extraction module 304 is used to extract the point cloud passable area from the point cloud road surface based on the image passable area that is inversely transformed to the three-dimensional space of the point cloud data; the vectorization module 305 is used to divide the point cloud passable area into safety levels and perform boundary vectorization on areas of different levels.

[0076] It can be understood that the road traversable area extraction system provided by the present invention corresponds to the road traversable area extraction method provided in the aforementioned embodiments. The relevant technical features of the road traversable area extraction system can refer to the relevant technical features of the road traversable area extraction method, which will not be repeated here.

[0077] Example 4

[0078] See also Figure 4 , Figure 4 Schematic diagram of an embodiment of an electronic device provided by an embodiment of the present invention. Figure 4As shown, an embodiment of the present invention provides an electronic device 400, including a memory 410, a processor 420 and a computer program 411 stored in the memory 410 and executable on the processor 420. When the processor 420 executes the computer program 411, the following steps are implemented: extracting a point cloud road surface based on point cloud data; segmenting a road passable area in the image data based on a deep learning semantic segmentation model; inversely transforming the segmented image passable area into the three-dimensional space of the point cloud data based on posture information; extracting a point cloud passable area from the point cloud road surface based on the image passable area in the three-dimensional space of the point cloud data based on the inverse perspective transformation; dividing the point cloud passable area into a security level, and vectorizing the boundaries of areas of different levels.

[0079] Example 5

[0080] See also Figure 5 , Figure 5 Schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention. Figure 5 As shown, this embodiment provides a computer-readable storage medium 500, on which a computer program 511 is stored. When the computer program 511 is executed by a processor, the following steps are implemented: extracting a point cloud road surface based on point cloud data; segmenting a road surface passable area in the image data based on a deep learning semantic segmentation model; based on posture information, inverse perspective transforming the segmented image passable area into the three-dimensional space of the point cloud data; extracting a point cloud passable area from the point cloud road surface based on the image passable area in the three-dimensional space of the point cloud data based on the inverse perspective transformation; dividing the point cloud passable area into a safety level, and vectorizing the boundaries of areas of different levels.

[0081] Embodiments of the present invention provide a method and system for extracting drivable areas on a road surface. These methods utilize high-precision three-dimensional point clouds, forward-looking wide-angle images, GPS trajectories, IMUs, and odometry information as data sources. Leveraging the unique strengths of each sensor, they leverage semantic segmentation results from deep image learning techniques, fuse multi-source data, and extract smoothed vectorized points of drivable areas from point cloud data. This deep image learning technology improves the efficiency of drivable area extraction for autonomous vehicles. Compared to single-source extraction methods and traditional methods, the system offers superior scene generalization capabilities and is valuable for map updates.

[0082] It should be noted that, in the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0083] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0084] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0085] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0086] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0087] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0088] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for extracting traversable areas of a road surface, characterized in that: include: Extract point cloud road surface based on point cloud data; Segment the drivable area of ​​the road in the image data based on the deep learning semantic segmentation model; Based on the pose information, the segmented image passable area is inversely transformed into the three-dimensional space of the point cloud data; Extracting a point cloud traversable area from the point cloud road surface based on an image traversable area in three-dimensional space converted from inverse perspective to point cloud data; Dividing the passable area of ​​the point cloud into safety levels and vectorizing the boundaries of areas of different levels; The point cloud road surface extraction method based on point cloud data includes: dividing each frame of point cloud data into blocks with equal spacing in heading direction to obtain multiple point cloud data blocks, each point cloud data block contains multiple point cloud points; for any point cloud data block, selecting a ground seed point, and obtaining multiple initial ground points based on the ground seed point; performing plane fitting based on the ground seed point and the multiple initial ground points to obtain an initial fitting plane; updating the ground point based on the distance between each point cloud point in any point cloud data block and the initial fitting plane; re-performing plane fitting based on the updated ground point, and after multiple iterations of plane fitting, the ground point finally updated is the point cloud road surface extraction result; each frame of point cloud data corresponds to a GPS track point, and for any point cloud data block, selecting a ground seed point, and obtaining multiple initial ground points based on the ground seed point, includes: converting the GPS track point and each frame of point cloud data to the point cloud XOY plane; for all point cloud points in any point cloud data block, calculating the distance between each point cloud point and the GPS track point, and taking the point cloud point with the smallest distance to the GPS track point as the ground seed point; Taking the ground seed point as the center point, searching for point cloud points within a preset radius in any point cloud data block, and taking the ground seed point and the point cloud points within the preset radius as initial ground points.

2. The method for extracting traversable areas of a road surface according to claim 1, characterized in that: The updating of ground points based on the distance between each point cloud point in any point cloud data block and the initial fitting plane includes: calculating the distance value from each point cloud point in any point cloud data block except the initial ground point to the initial fitting plane, and incorporating point cloud points with distance values ​​less than a preset distance threshold into the ground points to update the ground points.

3. The method for extracting traversable areas of a road surface according to claim 1 or 2, characterized in that: The plane is re-fitted based on the updated ground points. After multiple iterations of plane fitting, the finally updated ground points are the point cloud road surface extraction results, including: re-fitting the plane based on the updated ground points, and again updating the ground points based on the distance between each point cloud point in any point cloud data block and the re-fitted plane; re-iteratively fitting the plane until the distance between each point cloud point in any point cloud data block and the re-fitted plane is greater than a preset distance threshold, and the finally obtained ground points are used as the point cloud road surface extraction results.

4. The method for extracting traversable areas of a road surface according to claim 1, characterized in that: The pose information is obtained by: Based on the initial pose information, the marker elements in the high-precision map HD MAP are converted to the image coordinate system; in the image coordinate system, the marker elements in the HD MAP are matched with the same marker elements semantically segmented from the image data to obtain the matching results; Based on the matching results, GPS information, and IMU data, the initial pose information is optimized. Accordingly, based on the optimized pose information, the segmented image passable area is inversely transformed into the three-dimensional space of the point cloud data.

5. The method for extracting traversable areas of a road surface according to claim 1, characterized in that: The method extracts the point cloud traversable area from the point cloud road surface based on the image traversable area in the three-dimensional space transformed to the point cloud data, including: matching the image traversable area in the three-dimensional space transformed to the point cloud data to the point cloud road surface, and obtaining the matched point cloud traversable area in the point cloud road surface.

6. The method for extracting traversable areas of a road surface according to claim 1, characterized in that: The different levels of areas include absolute safety areas, relative safety areas, and areas without relative safety areas. The point cloud traversable area is divided into safety levels and the boundaries of areas of different levels are vectorized, including: when the HD MAP road surface range is smaller than the point cloud traversable area, the absolute safety area is the HD MAP road surface range, and the relative safety area is the area of ​​the point cloud traversable area minus the HD MAP road surface range; when the HD MAP road surface range is larger than the point cloud traversable area, the absolute safety area is the point cloud traversable area, and the area without relative safety area is other areas of the HD MAP road surface range minus the point cloud traversable area; and left and right boundaries of safety areas of different levels are vectorized.

7. A road traversable area extraction system, characterized in that: include: A first extraction module is used to extract a point cloud road surface based on the point cloud data; The segmentation module is used to segment the drivable area of ​​the road in the image data based on the deep learning semantic segmentation model; A transformation module is used to perform an inverse perspective transformation of the segmented image passable area into the three-dimensional space of the point cloud data based on the pose information; a second extraction module for extracting a point cloud traversable area from the point cloud road surface based on an image traversable area converted from an inverse perspective transformation to a three-dimensional space of the point cloud data; A vectorization module is used to divide the passable area of ​​the point cloud into safety levels and to vectorize the boundaries of areas of different levels; the extraction of point cloud road surface based on point cloud data includes: dividing each frame of point cloud data into blocks with equal spacing in heading direction to obtain multiple point cloud data blocks, each point cloud data block contains multiple point cloud points; for any point cloud data block, selecting a ground seed point, and obtaining multiple initial ground points based on the ground seed point; performing plane fitting based on the ground seed point and the multiple initial ground points to obtain an initial fitting plane; updating the ground point based on the distance between each point cloud point in any point cloud data block and the initial fitting plane; re-performing plane fitting based on the updated ground point, and after multiple iterations of plane fitting, the ground point finally updated is the point cloud road surface extraction result; each frame of point cloud data corresponds to a GPS track point, and for any point cloud data block, selecting a ground seed point, and obtaining multiple initial ground points based on the ground seed point, includes: converting the GPS track point and each frame of point cloud data to the point cloud XOY plane; For all point cloud points in any point cloud data block, calculate the distance between each point cloud point and the GPS track point, and use the point cloud point with the smallest distance to the GPS track point as the ground seed point; with the ground seed point as the center point, search for point cloud points within a preset radius in any point cloud data block, and use the ground seed point and the point cloud points within the preset radius as the initial ground points.

8. A computer-readable storage medium, characterized in that A computer management program is stored thereon, and when the computer management program is executed by the processor, the steps of the method for extracting the traversable area of ​​the road surface as described in any one of claims 1 to 6 are implemented.

Images