Drivable area detection method, computer equipment, storage medium and vehicle

Abstract

The present invention relates to the field of vehicle technology, and specifically provides a drivable area detection method, computer equipment, storage medium and vehicle, aiming to solve the problem of accurately determining the drivable area. To this end, the method of the present invention includes obtaining a three-dimensional point cloud in the vehicle driving environment; using a ground height estimation model based on a convolutional neural network and estimating the ground height of the current environment based on the three-dimensional point cloud in the current environment; determining non-ground point clouds that do not belong to the ground in the three-dimensional point cloud based on the ground height; performing obstacle detection on the non-ground point cloud to obtain one or more obstacles; and determining the drivable area in the vehicle driving environment based on the position of the obstacle. Through the above method, non-ground point clouds can be quickly and accurately screened out from a large number of three-dimensional point clouds, thereby improving the efficiency and accuracy of determining drivable areas using non-ground point clouds.

Description

Technical Field

[0001] The present invention relates to the field of vehicle technology, and specifically provides a drivable area detection method, computer equipment, storage medium and vehicle. Background Art

[0002] At present, conventional vehicle drivable area detection methods mainly include methods using depth camera detection and methods using three-dimensional point cloud detection. The method using depth camera detection mainly detects the distance between the vehicle and the surrounding obstacles through the depth camera set on the vehicle, and determines the drivable area of ​​the vehicle according to the detected distance. This method is limited by the depth perception ability of the depth camera, and it is difficult to detect the accurate drivable area. The method using three-dimensional point cloud detection mainly performs obstacle detection on the three-dimensional point cloud in the vehicle driving environment to determine whether there are obstacles in the driving environment and the distance between the obstacles and the vehicle, and determines the drivable area of ​​the vehicle according to the distance. In the method using three-dimensional point cloud detection, in order to ensure the accuracy of obstacle detection, it is necessary to first filter out the ground point cloud belonging to the ground in the three-dimensional point cloud, and then perform obstacle detection on the remaining three-dimensional point cloud. At present, the conventional ground point cloud filtering method mainly manually sets the filtering rules, and then determines which three-dimensional point clouds are ground point clouds one by one according to the set filtering rules and filters them out. Since the number of three-dimensional point clouds is usually large, this method takes a lot of time to complete the filtering of ground point clouds. In addition, this method completely relies on the set filtering rules and cannot filter out the ground point clouds that are not covered by the filtering rules, and has poor generalization ability.

[0003] Accordingly, the art needs a new technical solution to solve the above problems. Summary of the invention

[0004] In order to overcome the above-mentioned defects, the present invention is proposed to provide a drivable area detection method, computer equipment, storage medium and vehicle that solve or at least partially solve the technical problem of how to quickly and accurately determine non-ground point clouds that do not belong to the ground in a three-dimensional point cloud, and then accurately determine the drivable area based on the non-ground point clouds.

[0005] In a first aspect, the present invention provides a method for detecting a drivable area, the method comprising:

[0006] Obtaining a three-dimensional point cloud in the vehicle driving environment;

[0007] The ground height estimation model based on convolutional neural network is used to estimate the ground height of the current environment according to the three-dimensional point cloud in the current environment;

[0008] Determine a non-ground point cloud that does not belong to the ground in the three-dimensional point cloud according to the ground height;

[0009] Performing obstacle detection on the non-ground point cloud to obtain one or more obstacles;

[0010] Determining a drivable area in the vehicle driving environment according to the position of the obstacle;

[0011] Wherein, the method of estimating the ground height of the current environment by using a ground height estimation model based on a convolutional neural network and according to a three-dimensional point cloud in the current environment comprises at least the following steps:

[0012] The ground height estimation model is used to perform rasterization processing on the point cloud space of the three-dimensional point cloud to form multiple three-dimensional point cloud grids, and three-dimensional convolution features of multiple downsampling scales and two-dimensional convolution features of multiple downsampling scales are respectively extracted from the three-dimensional point cloud in each of the point cloud grids, and the three-dimensional convolution features and the two-dimensional convolution features are feature fused to obtain point cloud grid features, and the ground height of the ground area where each point cloud grid is located is estimated according to the point cloud grid features of each point cloud grid.

[0013] In a technical solution of the above-mentioned drivable area detection method, the step of "extracting three-dimensional convolution features of multiple downsampling scales and two-dimensional convolution features of multiple downsampling scales from each three-dimensional point cloud in the point cloud grid" specifically includes:

[0014] Using three-dimensional convolutions of multiple downsampling scales to perform convolution operations on the three-dimensional point clouds in each of the point cloud grids, respectively, to obtain three-dimensional convolution features of multiple downsampling scales for each of the point cloud grids;

[0015] The three-dimensional convolution features are converted into initial two-dimensional convolution features, and two-dimensional convolutions of multiple downsampling scales are used to perform convolution operations on each of the initial two-dimensional convolution features respectively, so as to obtain final two-dimensional convolution features of multiple downsampling scales of each of the point cloud grids.

[0016] In a technical solution of the above-mentioned drivable area detection method, the step of “fusing the three-dimensional convolution feature with the two-dimensional convolution feature to obtain a point cloud grid feature” specifically includes:

[0017] Convert all three-dimensional convolution features of each downsampling scale into three-dimensional convolution features of a preset sampling scale, perform convolution operations on each of the three-dimensional convolution features of the preset sampling scale using a convolution with a convolution kernel size of 1×1×1, add and densify the three-dimensional convolution features after the convolution operation to form a new three-dimensional convolution feature, and convert the new three-dimensional convolution feature into a first two-dimensional convolution feature;

[0018] Convert all the two-dimensional convolution features of each downsampling scale into the two-dimensional convolution features of the preset sampling scale, perform convolution operation on each of the two-dimensional convolution features of the preset sampling scale respectively by using convolution with a convolution kernel size of 1×1, and add and densify the two-dimensional convolution features after the convolution operation to form a second two-dimensional convolution feature;

[0019] A convolution operation with a convolution kernel size of 1×1 is respectively performed on the first two-dimensional convolution feature and the second two-dimensional convolution feature, and the first two-dimensional convolution feature and the second two-dimensional convolution feature subjected to the convolution operation are added to obtain a point cloud grid feature.

[0020] In a technical solution of the above-mentioned drivable area detection method, the method further includes training the ground height estimation model in the following manner, so as to estimate the ground height of the current environment using the trained ground height estimation model:

[0021] Inputting the three-dimensional point cloud training sample into the ground height estimation model, obtaining the point cloud grid features of the training sample extracted by the ground height estimation model from the three-dimensional point cloud training sample;

[0022] The ground height estimation model and the preset model for point cloud processing according to the ground height are respectively trained according to the point cloud grid features of the training samples, and the model training is stopped when both the ground height estimation model and the preset model for point cloud processing according to the ground height meet convergence conditions.

[0023] In a technical solution of the above-mentioned drivable area detection method, the preset model for performing point cloud processing according to the ground height includes at least a ground point cloud segmentation model and / or a target detection model for performing target detection on a three-dimensional point cloud. The method further includes training the ground height estimation model according to the point cloud grid features of the training samples through the following steps:

[0024] Using a square loss function, a smooth loss function and a height consistency loss function, the ground height estimation model is trained according to the point cloud grid features of the training samples;

[0025] The square loss function is a loss function that uses the Euclidean distance between the first ground height annotation value of the three-dimensional point cloud training sample and the ground height estimation value output by the ground height estimation model as the loss value; the first ground height annotation value is the ground height annotation value in the sample label used when the three-dimensional point cloud training sample is used to perform model training on the ground height estimation model;

[0026] The smoothing loss function is a loss function that uses the difference between the ground height estimation values ​​of adjacent three-dimensional point cloud training samples output by the ground height estimation model as the loss value;

[0027] The height consistency loss function is a loss function that takes the Manhattan distance between the second ground height annotation value of the three-dimensional point cloud training sample and the ground height estimation value output by the ground height estimation model as the loss value; the second ground height annotation value is the ground height annotation value in the sample label used when the ground point cloud segmentation model is trained using the three-dimensional point cloud training sample.

[0028] In a technical solution of the above-mentioned drivable area detection method, the step of "determining a non-ground point cloud that does not belong to the ground in the three-dimensional point cloud according to the ground height" specifically includes:

[0029] Determine the ground height of the ground area where the point cloud grid to which the current three-dimensional point cloud belongs is located;

[0030] Adding a preset compensation amount to the ground height to form a new ground height;

[0031] Determine whether the point cloud height of the current three-dimensional point cloud is less than the new ground height;

[0032] If yes, the current 3D point cloud is a ground point cloud belonging to the ground;

[0033] If not, the current three-dimensional point cloud is a non-ground point cloud that does not belong to the ground.

[0034] In a technical solution of the above-mentioned drivable area detection method, the step of "determining the drivable area in the vehicle driving environment according to the position of the obstacle" specifically includes:

[0035] Establishing a polar coordinate system corresponding to the three-dimensional coordinate system of the three-dimensional point cloud, generating a polar coordinate grid map according to a preset number of sector grids, and projecting the three-dimensional point cloud onto the polar coordinate grid map;

[0036] Determine the projection position of each obstacle according to the projection position of the three-dimensional point cloud corresponding to each obstacle on the polar coordinate grid map;

[0037] Calculate the distance between the vehicle and the nearest obstacle in each sector grid;

[0038] A drivable area in the vehicle driving environment is determined according to the distance.

[0039] In a second aspect, a computer device is provided, which includes a processor and a storage device, wherein the storage device is suitable for storing multiple program codes, and the program codes are suitable for being loaded and run by the processor to execute the drivable area detection method described in any one of the technical solutions of the above-mentioned drivable area detection method.

[0040] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, wherein the program codes are suitable for being loaded and run by a processor to execute the drivable area detection method described in any one of the technical solutions of the above-mentioned drivable area detection method.

[0041] In a fourth aspect, a vehicle is provided, the vehicle comprising the computer device described in the above technical solution.

[0042] The above one or more technical solutions of the present invention have at least one or more of the following beneficial effects:

[0043] In the technical solution of the present invention, a three-dimensional point cloud in a vehicle driving environment can be obtained, and a ground height estimation model based on a convolutional neural network is used to estimate the ground height of the current environment according to the three-dimensional point cloud in the current environment; non-ground point clouds that do not belong to the ground in the three-dimensional point cloud are determined according to the ground height, and obstacle detection is performed on the non-ground point cloud to obtain one or more obstacles, and finally the drivable area in the vehicle driving environment is determined according to the position of the obstacle. Among them, using a ground height estimation model based on a convolutional neural network and estimating the ground height of the current environment according to the three-dimensional point cloud in the current environment at least includes the following steps: using the ground height estimation model to rasterize the point cloud space of the three-dimensional point cloud to form multiple three-dimensional point cloud grids, extracting three-dimensional convolution features of multiple downsampling scales and two-dimensional convolution features of multiple downsampling scales from the three-dimensional point cloud in each point cloud grid, performing feature fusion on the three-dimensional convolution features and the two-dimensional convolution features to obtain point cloud grid features, and estimating the ground height of the ground area where each point cloud grid is located according to the point cloud grid features of each point cloud grid.

[0044] The ground height determined based on the ground height estimation model can quickly filter out ground point clouds and non-ground point clouds from a large number of three-dimensional point clouds, overcoming the problem of low efficiency in the prior art when determining which three-dimensional point clouds are ground point clouds one by one according to the set filtering rules. In addition, by extracting three-dimensional convolution features and two-dimensional convolution features of multiple downsampling scales from the three-dimensional point cloud and performing feature fusion to form point cloud grid features, and then estimating the ground height based on the point cloud grid features, it can be applied to different landform environments. Regardless of the landform environment, by extracting features of multiple downsampling scales, it can ensure that effective ground features are extracted, so that the ground height can be accurately estimated, overcoming the problem of the prior art that when determining the ground point cloud according to the manually set filtering rules, it completely relies on the set filtering rules, and cannot determine the ground point cloud not covered by the filtering rules, and has poor generalization ability.

[0045] Furthermore, in some technical solutions of the present invention, in order to obtain accurate point cloud grid features to further improve the accuracy of determining ground point clouds and non-ground point clouds, when training the ground height estimation model, the three-dimensional point cloud training samples can be input into the ground height estimation model to obtain the point cloud grid features extracted by the ground height estimation model from the three-dimensional point cloud training samples, and then the ground height estimation model and the preset model for point cloud processing according to the ground height (hereinafter referred to as the "point cloud processing model") are trained according to the point cloud grid features, and the model training is stopped when both the ground height estimation model and the point cloud processing model meet the convergence conditions.

[0046] Model training for the ground height estimation model can optimize the feature extraction parameters of the ground height estimation model, and model training for the point cloud processing model can also optimize the feature extraction parameters of the ground height estimation model. Finally, under the coordinated training of the ground height estimation model and the point cloud processing model, the feature extraction parameters of the ground height estimation model can be better optimized, thereby improving the accuracy of the ground height estimation model in extracting point cloud raster features, obtaining accurate point cloud raster features, and further improving the accuracy of determining ground point clouds and non-ground point clouds. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The disclosure of the present invention will become more easily understood with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present invention. Among them:

[0048] Figure 1 is a schematic diagram of a polar coordinate grid diagram according to an embodiment of the present invention;

[0049] Figure 2 is a schematic flow chart of main steps of a method for detecting a drivable area according to an embodiment of the present invention;

[0050] Figure 3 It is a flowchart of main steps of a method for estimating the ground height of a current environment using a ground height estimation model based on a convolutional neural network and based on a three-dimensional point cloud in the current environment according to an embodiment of the present invention;

[0051] Figure 4 It is a flowchart diagram of main steps of a method for determining a drivable area in a vehicle driving environment according to the position of an obstacle according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] Some embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0053] In the description of the present invention, "processor" may include hardware, software or a combination of the two. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware or a combination of the two. Computer-readable storage media include any suitable medium that can store program codes, such as a magnetic disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B or A and B. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and / or B" and may include only A, only B or A and B. The singular terms "one" and "the" may also include plural forms.

[0054] Here we first explain some terms involved in the present invention.

[0055] Polar coordinates refers to a coordinate system consisting of a pole O, a polar axis Ox and a polar diameter in a plane. The polar axis Ox is a ray drawn from the pole O. The polar diameter refers to the distance from a point P in the plane to the pole O. Point P can be expressed as P(ρ, θ), where ρ represents the polar diameter of point P, and θ represents the polar angle of point P. The polar angle is the angle from the polar axis Ox to the line segment OP in a counterclockwise direction. In the embodiment of the present invention, the pole O of the polar coordinate system and the origin of the three-dimensional coordinate system of the three-dimensional point cloud can be the same point, which can be a point on the vehicle.

[0056] Polar coordinate grid diagram refers to a polar coordinate representation diagram formed by gridding a plane with a certain polar angle range in the polar coordinate system. The plane can be divided into multiple grids through grid division. Figure 1As shown, in the embodiment of the present invention, the method of grid division is to divide the plane into a plurality of sector-shaped grids starting from the polar axis Ox in a counterclockwise direction with the pole O as the starting point, and each grid corresponds to a certain polar angle range. For example, the polar angle range of the plane is 120°, and the number of grids formed by the division is 192, then the polar angle range corresponding to each grid is 0.625°. Rotating in the counterclockwise direction, the polar angle range corresponding to the first grid is 0°-0.625°, and the polar angle range corresponding to the second grid is 0.625°-1.25°, and so on and so forth to obtain the angles corresponding to other grids.

[0057] The following is a description of an embodiment of a method for detecting a drivable area according to the present invention.

[0058] See attached Figure 2 , Figure 2 FIG. 1 is a flow chart showing the main steps of a method for detecting a drivable area according to an embodiment of the present invention. Figure 2 As shown, the drivable area detection method in the embodiment of the present invention mainly includes the following steps S101 to S105.

[0059] Step S101: Acquire a three-dimensional point cloud in the vehicle driving environment.

[0060] A three-dimensional point cloud refers to three-dimensional data determined based on an echo signal reflected back to the vehicle after an environmental point in the vehicle's driving environment receives an electromagnetic wave sent by the vehicle to the environmental point, and the three-dimensional data includes the coordinates of the environmental point in a three-dimensional coordinate system. In an embodiment of the present invention, the vehicle can send electromagnetic waves to environmental points in the driving environment through radar, etc., wherein the radar includes but is not limited to millimeter-wave radar and laser radar. In a preferred embodiment, the radar can be a laser radar.

[0061] Step S102: using a ground height estimation model based on a convolutional neural network and according to the three-dimensional point cloud in the current environment, estimating the ground height of the current environment.

[0062] See attached Figure 3 In this embodiment of the present invention, the ground height of the current environment can be estimated through the following steps S1021 to S1024.

[0063] Step S1021: using a ground height estimation model to perform rasterization processing on the point cloud space of the three-dimensional point cloud to form a plurality of three-dimensional point cloud grids.

[0064] Point cloud space refers to the three-dimensional space formed by the point cloud coordinate system of the three-dimensional point cloud. The point cloud space is divided according to the three dimensions of the x, y and z axes in the point cloud coordinate system to form multiple three-dimensional point cloud grids. A point cloud grid is a voxel (Volume Pixel) in the point cloud space.

[0065] Step S1022: extracting three-dimensional convolution features of multiple downsampling scales and two-dimensional convolution features of multiple downsampling scales for the three-dimensional point cloud in each point cloud grid.

[0066] In this embodiment, a 3D convolution operation can be performed on a 3D point cloud using a plurality of different downsampling scales to obtain 3D convolution features of a plurality of downsampling scales; at the same time, a 2D convolution operation can be performed on a 3D point cloud using a plurality of different downsampling scales to obtain 2D convolution features of a plurality of downsampling scales. For example, in some embodiments, 3D convolution features of 2x, 4x, and 8x downsampling scales can be obtained, and 2D convolution features of 8x, 16x, and 32x downsampling scales can be obtained at the same time.

[0067] Further, in some embodiments, after using three-dimensional convolutions with multiple downsampling scales to perform convolution operations on the three-dimensional point clouds in each point cloud grid, and obtaining three-dimensional convolution features with multiple downsampling scales for each point cloud grid, the three-dimensional convolution features can be converted into initial two-dimensional convolution features, and then two-dimensional convolutions with multiple downsampling scales can be used to perform convolution operations on each initial two-dimensional convolution feature, so as to obtain the final two-dimensional convolution features with multiple downsampling scales for each point cloud grid.

[0068] The feature tensor of a three-dimensional convolution feature is four-dimensional, which can be expressed as X*Y*Z*C, where C represents the feature length or the number of channels. The feature tensor of a two-dimensional convolution feature is three-dimensional, which can be expressed as X*Y*C, where C represents the feature length or the number of channels. When converting a three-dimensional convolution feature into an initial two-dimensional convolution feature, the Z-dimensional feature and the C-dimensional feature in the three-dimensional convolution feature can be concatenated to form a new feature tensor X*Y*(Z*C), and the initial two-dimensional convolution feature can be obtained.

[0069] Step S1023: performing feature fusion on the three-dimensional convolution feature and the two-dimensional convolution feature to obtain the point cloud grid feature.

[0070] In some implementations, the three-dimensional convolution feature and the two-dimensional convolution feature may be fused to obtain the point cloud grid feature through the following steps 11 to 13.

[0071] Step 11: Convert all the three-dimensional convolution features of each downsampling scale into three-dimensional convolution features of a preset sampling scale, perform convolution operations on the three-dimensional convolution features of each preset sampling scale using a convolution kernel size of 1×1×1, add and densify the three-dimensional convolution features after the convolution operation to form a new three-dimensional convolution feature, and convert the new three-dimensional convolution feature into a first two-dimensional convolution feature.

[0072] Using a convolution with a convolution kernel size of 1×1×1 to perform a convolution operation on each three-dimensional convolution feature of a preset sampling scale can make the number of channels of all three-dimensional convolution features the same.

[0073] In some embodiments, after obtaining three-dimensional convolution features with 2 times, 4 times and 8 times downsampling scales, all of these three-dimensional convolution features can be converted into three-dimensional convolution features with 4 times downsampling scale. Specifically, the three-dimensional convolution features with 4 times downsampling scale remain unchanged, the three-dimensional convolution features with 2 times downsampling scale are downsampled again by 2 times the sampling scale to obtain three-dimensional convolution features with 4 times the downsampling scale, and the three-dimensional convolution features with 8 times downsampling scale are upsampled again by 2 times the sampling scale to obtain three-dimensional convolution features with 4 times the downsampling scale. A convolution operation with a convolution kernel size of 1×1×1 is performed on the three three-dimensional convolution features with 4 times the downsampling scale, so that the number of channels of the three three-dimensional convolution features is the same, and finally the three three-dimensional convolution features are added and densified to obtain new three-dimensional convolution features.

[0074] It should be noted that, in this embodiment, the conventional convolution feature addition and densification method in the field of convolution technology can be used to add and densify the three three-dimensional convolution features, and the embodiment of the present invention does not specifically limit this.

[0075] Step 12: Convert all the two-dimensional convolution features of each downsampling scale into two-dimensional convolution features of a preset sampling scale, use a convolution with a convolution kernel size of 1×1 to perform a convolution operation on each two-dimensional convolution feature of the preset sampling scale, and add and densify the two-dimensional convolution features after the convolution operation to form a second two-dimensional convolution feature.

[0076] Using a convolution with a convolution kernel size of 1×1 to perform a convolution operation on each two-dimensional convolution feature of a preset sampling scale can make the number of channels of all two-dimensional convolution features the same.

[0077] In some embodiments, after obtaining two-dimensional convolution features with 8 times, 16 times, and 32 times downsampling scales, all of these two-dimensional convolution features can be converted into two-dimensional convolution features with 4 times downsampling scales. Specifically, the two-dimensional convolution features with 8 times downsampling scales are again upsampled by 2 times the sampling scale to obtain two-dimensional convolution features with 4 times the downsampling scale, the two-dimensional convolution features with 16 times the downsampling scales are again upsampled by 4 times the sampling scale to obtain two-dimensional convolution features with 4 times the downsampling scale, and the two-dimensional convolution features with 32 times the downsampling scales are again upsampled by 8 times the sampling scale to obtain two-dimensional convolution features with 4 times the downsampling scale. Convolution operations are performed on the three two-dimensional convolution features with 4 times the downsampling scales respectively using a convolution with a convolution kernel size of 1×1, so that the number of channels of the three two-dimensional convolution features is the same, and finally the three two-dimensional convolution features are added and densified to obtain new two-dimensional convolution features.

[0078] It should be noted that in this embodiment, the conventional convolution feature addition and densification method in the field of convolution technology can be used to add and densify the three two-dimensional convolution features, and the embodiment of the present invention does not specifically limit this.

[0079] Step 13: Use convolution with a convolution kernel size of 1×1 to perform convolution operations on the first two-dimensional convolution features and the second two-dimensional convolution features respectively, add the first two-dimensional convolution features and the second two-dimensional convolution features after the convolution operation to obtain the point cloud grid features.

[0080] Using a convolution with a convolution kernel size of 1×1 to perform convolution operations on the first two-dimensional convolution feature and the second two-dimensional convolution feature respectively can make the number of channels of the first two-dimensional convolution feature and the second two-dimensional convolution feature the same.

[0081] It should be noted that, in this embodiment, the conventional convolution feature addition method in the field of convolution technology can be used to add the first two-dimensional convolution feature and the second two-dimensional convolution feature, and the embodiment of the present invention does not specifically limit this.

[0082] Step S1024: Estimate the ground height of the ground area where each point cloud grid is located according to the point cloud grid features of each point cloud grid.

[0083] By extracting three-dimensional convolution features and two-dimensional convolution features of multiple downsampling scales from the three-dimensional point cloud and performing feature fusion to form point cloud raster features, and then estimating the ground height based on the point cloud raster features, the approach can be applied to different landform environments. No matter what kind of landform environment it is, by extracting features of multiple downsampling scales, it is possible to ensure that effective ground features are extracted, thereby enabling the ground height to be accurately estimated.

[0084] The above is a detailed description of step S102 , and the following will continue to describe steps S103 to S105 .

[0085] Step S103: determining non-ground point clouds that do not belong to the ground in the three-dimensional point cloud according to the ground height.

[0086] The coordinate of the z-axis in the point cloud coordinate system represents the point cloud height of the 3D point cloud. By comparing the ground height with the point cloud height, it is possible to filter out which 3D point clouds are ground point clouds belonging to the ground and which 3D point clouds are non-ground point clouds that do not belong to the ground. Specifically, if the point cloud height is less than or equal to the ground height, the 3D point cloud is a ground point cloud; if the point cloud height is greater than the ground height, the 3D point cloud is a non-ground point cloud.

[0087] Step S104: performing obstacle detection on the non-ground point cloud to obtain one or more obstacles.

[0088] In this embodiment, the non-ground point cloud can be clustered according to the point cloud spacing between the three-dimensional point clouds to obtain one or more point cloud clusters, each of which represents an obstacle. According to the coordinates of the point cloud in the point cloud cluster, the position, shape, height and other information of the obstacle can be determined.

[0089] It should be noted that in this embodiment, conventional methods in the field of obstacle detection technology can be used to obtain information such as the position, shape, height, etc. of each obstacle using the coordinates of the point cloud in the point cloud cluster, and the embodiment of the present invention does not specifically limit this.

[0090] Step S105: Determine a drivable area in the vehicle driving environment according to the location of the obstacle.

[0091] After determining the location of the obstacle, the drivable area in the vehicle driving environment can be determined according to the distance between the vehicle and the obstacle. For example, if the distance between the vehicle and the obstacle is 10 meters, the drivable area can be within 10 meters from the vehicle as the starting point.

[0092] Based on the methods described in steps S101 to S105 above, non-ground point clouds can be quickly and accurately screened out from a large number of three-dimensional point clouds, thereby improving the efficiency and accuracy of determining the drivable area using non-ground point clouds.

[0093] The above steps S103 and S105 are further explained below.

[0094] In some implementations of the above step S103, in order to avoid interference with the ground height caused by factors such as inaccurate three-dimensional point cloud acquisition, the ground height can be compensated first, and then the non-ground point cloud can be determined based on the compensated ground height. Specifically, in this implementation, the non-ground point cloud that does not belong to the ground in the three-dimensional point cloud can be determined through the following steps S1031 to S1033.

[0095] Step S1031: Determine the ground height of the ground area where the point cloud grid to which the current three-dimensional point cloud belongs is located.

[0096] In this embodiment, when the point cloud space of the three-dimensional point cloud is rasterized to form multiple three-dimensional point cloud grids, a grid index can be set for each three-dimensional point cloud. By querying the grid index, it can be determined which point cloud grid the three-dimensional point cloud belongs to.

[0097] After determining which point cloud grid the three-dimensional point cloud belongs to, the x-axis and y-axis coordinates of the point cloud grid in the point cloud coordinate system are obtained, and the plane area determined by the x-axis and y-axis coordinates is the ground area where the point cloud grid is located.

[0098] Step S1032: adding a preset compensation amount to the ground height to form a new ground height.

[0099] Those skilled in the art can flexibly set the specific value of the preset compensation amount according to actual needs. In some preferred implementations, the preset compensation amount can be set to a value greater than zero.

[0100] Step S1033: Determine whether the point cloud height of the current three-dimensional point cloud is less than the new ground height; if so, the current three-dimensional point cloud is a ground point cloud belonging to the ground; if not, the current three-dimensional point cloud is a non-ground point cloud that does not belong to the ground.

[0101] Through steps S1031 to S1033, the ground point cloud and the non-ground point cloud in the three-dimensional point cloud can be accurately screened out.

[0102] The above is a further description of step S103 , and the following is a further description of step S105 .

[0103] In some implementations of the above step S105, Figure 4 As shown, in order to accurately determine the drivable area in the vehicle driving environment, the drivable area in the vehicle driving environment may be determined through the following steps S1051 to S1054.

[0104] Step S1051: establishing a polar coordinate system corresponding to the three-dimensional coordinate system of the three-dimensional point cloud, generating a polar coordinate grid map according to a preset number of sector grids, and projecting the three-dimensional point cloud onto the polar coordinate grid map.

[0105] The polar coordinate system established with the origin of the three-dimensional coordinate system as the pole is the polar coordinate system corresponding to the three-dimensional coordinate system.

[0106] After projecting the three-dimensional point cloud onto the polar coordinate grid map, the coordinates of the projection point of each three-dimensional point cloud in the polar coordinate grid map can be obtained according to the coordinate conversion relationship between the polar coordinate system and the three-dimensional coordinate system. According to the coordinates of the projection point and the position information of each grid, the projection point falling into each grid can be obtained.

[0107] Step S1052: Determine the projection position of each obstacle according to the projection position of the three-dimensional point cloud corresponding to each obstacle on the polar coordinate grid map.

[0108] The projection position of the three-dimensional point cloud on the polar coordinate grid can be obtained according to the position of the projection point corresponding to the three-dimensional point cloud, and the projection position of the obstacle can be determined according to the projection positions of all the three-dimensional point clouds contained in the obstacle.

[0109] Step S1053: Calculate the distance between the vehicle and the nearest obstacle in each sector grid.

[0110] Step S1054: Determine a drivable area in the vehicle driving environment according to the distance.

[0111] Specifically, the vehicle is taken as the starting point, and the range within the above distance is taken as the drivable area.

[0112] The above is a further explanation of step S105.

[0113] The following is an explanation of the model training method of the ground height estimation model in an embodiment of the present invention.

[0114] In the embodiment of the present invention, the ground height estimation model can be trained through the following steps S201 to S202, so that the trained ground height estimation model can be used to estimate the ground height of the current environment in step S102.

[0115] Step S201: input the three-dimensional point cloud training samples into the ground height estimation model, and obtain the point cloud grid features of the training samples extracted by the ground height estimation model from the three-dimensional point cloud training samples.

[0116] The method for acquiring the point cloud grid features of the training samples is the same as the method described in steps S1021 to S1023 in the aforementioned method embodiment, and will not be described in detail here.

[0117] Step S202: Perform model training on the ground height estimation model and the preset model for point cloud processing based on the ground height according to the point cloud grid features of the training samples, and stop model training when both the ground height estimation model and the preset model for point cloud processing based on the ground height meet convergence conditions.

[0118] The convergence conditions include but are not limited to: the number of iterations of the model training reaches a preset number threshold, and the loss value of the loss function used in the model training is less than a preset loss value threshold.

[0119] It should be noted that, in this embodiment, a conventional model training method can be used to train the ground height estimation model using a preset loss function. For example, a training sample is input into the ground height estimation model, a loss value of the loss function is calculated by forward propagation, a parameter gradient of the model parameter is calculated according to the loss value, and the model parameter is updated according to the parameter gradient by back propagation, thereby completing an iterative training.

[0120] Model training for the ground height estimation model can optimize the feature extraction parameters of the ground height estimation model, and model training for the point cloud processing model can also optimize the feature extraction parameters of the ground height estimation model. Finally, under the coordinated training of the ground height estimation model and the point cloud processing model, the feature extraction parameters of the ground height estimation model can be better optimized, thereby improving the accuracy of the ground height estimation model in extracting point cloud raster features, obtaining accurate point cloud raster features, and further improving the accuracy of determining ground point clouds and non-ground point clouds.

[0121] In an embodiment of the present invention, a conventional model for performing point cloud processing according to ground height in the field of point cloud processing technology can be selected to perform collaborative training on a ground height estimation model. In some implementations, the preset model for performing point cloud processing according to ground height includes at least a ground point cloud segmentation model and / or a target detection model for performing target detection on a three-dimensional point cloud.

[0122] The following is an explanation of the loss function used in model training of the ground height estimation model.

[0123] When the preset model for processing point clouds according to ground height includes a ground point cloud segmentation model, the square loss function, the smooth loss function, and the height consistency loss function can be used simultaneously to train the ground height estimation model according to the point cloud grid features of the training samples. In addition, when the preset model for processing point clouds according to ground height includes both a ground point cloud segmentation model and a target detection model for target detection on a three-dimensional point cloud, the square loss function, the smooth loss function, and the height consistency loss function can also be used simultaneously to train the ground height estimation model according to the point cloud grid features of the training samples.

[0124] When the preset model for point cloud processing based on ground height includes a target detection model for target detection on three-dimensional point clouds, but does not include a ground point cloud segmentation model, the square loss function and the smooth loss function can be used at the same time to train the ground height estimation model according to the point cloud raster features of the training samples.

[0125] The square loss function, smooth loss function and highly consistent loss function are explained below.

[0126] 1. Square loss function

[0127] The square loss function is a loss function that uses the Euclidean distance between the first ground height annotation value of the 3D point cloud training sample and the ground height estimation value output by the ground height estimation model as the loss value. The smaller the Euclidean distance between the first ground height annotation value and the ground height estimation value, the closer the ground height estimation value is to the first ground height annotation value, and the more accurate the ground height estimation value is.

[0128] The first ground height annotation value is a ground height annotation value in a sample label used when a ground height estimation model is trained using a three-dimensional point cloud training sample.

[0129] In order to improve the labeling efficiency in the embodiment of the present invention, an information labeling tool such as Patchwork may be used to automatically label the ground height labeling values ​​of a large number of three-dimensional point cloud training samples.

[0130] 2. Smooth loss function

[0131] The smoothing loss function is a loss function that uses the difference between the ground height estimation values ​​of adjacent 3D point cloud training samples output by the ground height estimation model as the loss value.

[0132] The smaller the difference between the ground height estimation values ​​of adjacent 3D point cloud training samples, the smoother the ground height estimation value output by the ground height estimation model is and the closer it is to the actual ground height.

[0133] 3. Highly consistent loss function

[0134] The height consistency loss function is a loss function that uses the Manhattan distance between the second ground height annotation value of the 3D point cloud training sample and the ground height estimation value output by the ground height estimation model as the loss value. The smaller the Manhattan distance between the second ground height annotation value and the ground height estimation value, the closer the ground height estimation value is to the second ground height annotation value, and the more accurate the ground height estimation value is.

[0135] The second ground height annotation value is a ground height annotation value in a sample label used when a ground point cloud segmentation model is trained using a three-dimensional point cloud training sample.

[0136] In this embodiment, the second ground height annotation value can be obtained by the following method:

[0137] First, the ground segmentation model of the image using the two-dimensional image is used to segment the ground to obtain multiple two-dimensional ground points. Then, according to the coordinate conversion relationship between the two-dimensional image and the three-dimensional point cloud, each two-dimensional ground point is converted to the point cloud coordinate system of the three-dimensional point cloud to obtain the corresponding ground point cloud. Finally, the coordinate of the z-axis of the ground point cloud in the point cloud coordinate system is used as the second ground height annotation value of the ground point cloud.

[0138] The above is a detailed description of the model training method for the ground height estimation model.

[0139] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effects of the present invention, different steps do not have to be performed in such an order. They can be performed simultaneously (in parallel) or in other orders. These changes are within the scope of protection of the present invention.

[0140] It is understood by those skilled in the art that the present invention implements all or part of the processes in the method of the above embodiment, and can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium that can carry the computer program code. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0141] Furthermore, the present invention also provides a computer device. In one embodiment of a computer device according to the present invention, the computer device includes a processor and a storage device, the storage device can be configured to store a program for executing the drivable area detection method of the above method embodiment, and the processor can be configured to execute the program in the storage device, which includes but is not limited to the program for executing the drivable area detection method of the above method embodiment. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention. The computer device can be a device formed by various electronic devices.

[0142] Furthermore, the present invention also provides a computer-readable storage medium. In a computer-readable storage medium embodiment according to the present invention, the computer-readable storage medium can be configured to store a program for executing the drivable area detection method of the above-mentioned method embodiment, and the program can be loaded and run by the processor to implement the above-mentioned drivable area detection method. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention. The computer-readable storage medium can be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present invention is a non-temporary computer-readable storage medium.

[0143] Furthermore, the present invention also provides a vehicle. In an embodiment of a vehicle according to the present invention, the vehicle may include the computer device described in the above-mentioned computer device embodiment. In this embodiment, the vehicle may be an autonomous driving vehicle, an unmanned vehicle, or other vehicles. In addition, according to the type of power source, the vehicle in this embodiment may be a fuel vehicle, an electric vehicle, a hybrid vehicle that mixes electric energy and fuel, or a vehicle that uses other new energy sources.

[0144] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A method for detecting a drivable area, characterized in that: The method comprises: Obtaining a three-dimensional point cloud in the vehicle driving environment; The ground height estimation model based on convolutional neural network is used to estimate the ground height of the current environment according to the three-dimensional point cloud in the current environment; Determine a non-ground point cloud that does not belong to the ground in the three-dimensional point cloud according to the ground height; Performing obstacle detection on the non-ground point cloud to obtain one or more obstacles; Determining a drivable area in the vehicle driving environment according to the position of the obstacle; Wherein, the method of estimating the ground height of the current environment by using a ground height estimation model based on a convolutional neural network and according to a three-dimensional point cloud in the current environment comprises at least the following steps: The ground height estimation model is used to perform rasterization processing on the point cloud space of the three-dimensional point cloud to form multiple three-dimensional point cloud grids, and three-dimensional convolution features of multiple downsampling scales and two-dimensional convolution features of multiple downsampling scales are respectively extracted from the three-dimensional point cloud in each of the point cloud grids, and the three-dimensional convolution features and the two-dimensional convolution features are feature fused to obtain point cloud grid features, and the ground height of the ground area where each point cloud grid is located is estimated according to the point cloud grid features of each point cloud grid.

2. The drivable area detection method according to claim 1, characterized in that: The step of “respectively extracting three-dimensional convolution features of multiple downsampling scales and two-dimensional convolution features of multiple downsampling scales from each three-dimensional point cloud in the point cloud grid” specifically includes: Using three-dimensional convolutions of multiple downsampling scales to perform convolution operations on the three-dimensional point clouds in each of the point cloud grids, respectively, to obtain three-dimensional convolution features of multiple downsampling scales for each of the point cloud grids; The three-dimensional convolution features are converted into initial two-dimensional convolution features, and two-dimensional convolutions of multiple downsampling scales are used to perform convolution operations on each of the initial two-dimensional convolution features respectively, so as to obtain final two-dimensional convolution features of multiple downsampling scales of each of the point cloud grids.

3. The drivable area detection method according to claim 1, characterized in that: The step of “performing feature fusion on the three-dimensional convolution feature and the two-dimensional convolution feature to obtain a point cloud grid feature” specifically includes: Convert all three-dimensional convolution features of each downsampling scale into three-dimensional convolution features of a preset sampling scale, perform convolution operations on each of the three-dimensional convolution features of the preset sampling scale using a convolution with a convolution kernel size of 1×1×1, add and densify the three-dimensional convolution features after the convolution operation to form a new three-dimensional convolution feature, and convert the new three-dimensional convolution feature into a first two-dimensional convolution feature; Convert all the two-dimensional convolution features of each downsampling scale into the two-dimensional convolution features of the preset sampling scale, perform convolution operation on each of the two-dimensional convolution features of the preset sampling scale respectively by using convolution with a convolution kernel size of 1×1, and add and densify the two-dimensional convolution features after the convolution operation to form a second two-dimensional convolution feature; A convolution operation with a convolution kernel size of 1×1 is respectively performed on the first two-dimensional convolution feature and the second two-dimensional convolution feature, and the first two-dimensional convolution feature and the second two-dimensional convolution feature subjected to the convolution operation are added to obtain a point cloud grid feature.

4. The drivable area detection method according to any one of claims 1 to 3, characterized in that: The method further includes training the ground height estimation model in the following manner, so as to estimate the ground height of the current environment using the trained ground height estimation model: Inputting the three-dimensional point cloud training sample into the ground height estimation model, obtaining the point cloud grid features of the training sample extracted by the ground height estimation model from the three-dimensional point cloud training sample; The ground height estimation model and the preset model for point cloud processing according to the ground height are respectively trained according to the point cloud grid features of the training samples, and the model training is stopped when both the ground height estimation model and the preset model for point cloud processing according to the ground height meet convergence conditions.

5. The drivable area detection method according to claim 4, characterized in that: The preset model for processing point clouds according to ground height at least includes a ground point cloud segmentation model and / or a target detection model for detecting targets on three-dimensional point clouds. The method further includes training the ground height estimation model according to the point cloud grid features of the training samples through the following steps: Using a square loss function, a smooth loss function and a height consistency loss function, the ground height estimation model is trained according to the point cloud grid features of the training samples; The square loss function is a loss function that uses the Euclidean distance between the first ground height annotation value of the three-dimensional point cloud training sample and the ground height estimation value output by the ground height estimation model as the loss value; the first ground height annotation value is the ground height annotation value in the sample label used when the three-dimensional point cloud training sample is used to perform model training on the ground height estimation model; The smoothing loss function is a loss function that uses the difference between the ground height estimation values ​​of adjacent three-dimensional point cloud training samples output by the ground height estimation model as the loss value; The height consistency loss function is a loss function that takes the Manhattan distance between the second ground height annotation value of the three-dimensional point cloud training sample and the ground height estimation value output by the ground height estimation model as the loss value; the second ground height annotation value is the ground height annotation value in the sample label used when the ground point cloud segmentation model is trained using the three-dimensional point cloud training sample.

6. The method for detecting a drivable area according to claim 1, characterized in that: The step of "determining a non-ground point cloud that does not belong to the ground in the three-dimensional point cloud according to the ground height" specifically includes: Determine the ground height of the ground area where the point cloud grid to which the current three-dimensional point cloud belongs is located; Adding a preset compensation amount to the ground height to form a new ground height; Determine whether the point cloud height of the current three-dimensional point cloud is less than the new ground height; If yes, the current 3D point cloud is a ground point cloud belonging to the ground; If not, the current three-dimensional point cloud is a non-ground point cloud that does not belong to the ground.

7. The drivable area detection method according to claim 1, characterized in that: The step of "determining the drivable area in the vehicle driving environment according to the position of the obstacle" specifically includes: Establishing a polar coordinate system corresponding to the three-dimensional coordinate system of the three-dimensional point cloud, generating a polar coordinate grid map according to a preset number of sector grids, and projecting the three-dimensional point cloud onto the polar coordinate grid map; Determine the projection position of each obstacle according to the projection position of the three-dimensional point cloud corresponding to each obstacle on the polar coordinate grid map; Calculate the distance between the vehicle and the nearest obstacle in each sector grid; A drivable area in the vehicle driving environment is determined according to the distance.

8. A computer device comprising a processor and a storage device, wherein the storage device is suitable for storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by the processor to execute the drivable area detection method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the drivable area detection method according to any one of claims 1 to 7.

10. A vehicle, characterized in that: The vehicle comprises the computer device of claim 8.

Images