Obstacle detection method and device and computer equipment

Abstract

The invention relates to an obstacle detection method and device and computer equipment, and the method comprises the steps: obtaining the point cloud data of a laser radar; inputting the point cloud data into an obstacle detection model to obtain an obstacle detection result; inputting the point cloud data into a noise segmentation model to obtain a noise segmentation result; and under a unified view, checking the obstacle detection result and the noise segmentation result, and judging whether each obstacle in the obstacle detection result is detected by mistake or not. Through the obstacle detection method and device, the problems of large calculation amount and poor obstacle detection effect in related technologies are solved, filtering of obstacle false detection is realized, and the obstacle detection effect is improved.

Description

[0001] This application is a divisional application of the application with the application date of November 18, 2024, application number 202411640597.6, and invention title "Obstacle Detection Method, Device and Computer Equipment". Technical Field

[0002] This application relates to the technical field of point cloud data processing, and particularly to an obstacle detection method, device and computer equipment. Background Art

[0003] In autonomous driving, the environmental perception system is the greatest guarantee for the driving safety of vehicles, providing basic support for high-level tasks such as subsequent vehicle path planning and behavior decision-making. Among them, there are point cloud noise points in the point cloud data based on lidar. The main sources of point cloud noise points are two aspects: on the one hand, the noise points caused by the surface of the target object. For example, the properties of the surface material of the target object (the surface material with high reflectivity causes the point cloud reflection energy to be too strong, making the target object larger than its actual size), roughness (the uneven surface causes the emission angle of the point cloud to change), etc.; on the other hand, the noise points caused by the external scanning environment, such as particulate matter such as rain, snow, fog, and dust blocking the point cloud and unable to reflect the point cloud back, etc. The traditional solution is to use a filtering algorithm to remove the point cloud noise points in the point cloud data, and then detect obstacles based on the denoised point cloud data to improve the detection accuracy. However, the computational complexity of this solution is very large, and its final detection effect also depends on the technical level of technicians, resulting in poor obstacle detection effect.

[0004] In view of the problems of large computational complexity and poor obstacle detection effect in the related art, no effective solution has been proposed yet. Summary of the Invention

[0005] In this embodiment, an obstacle detection method, device and computer equipment are provided to solve the problems of large computational complexity and poor obstacle detection effect in the related art.

[0006] In the first aspect, in this embodiment, an obstacle detection method is provided, including:

[0007] Obtaining the point cloud data of the lidar;

[0008] Inputting the point cloud data into an obstacle detection model to obtain an obstacle detection result;

[0009] Inputting the point cloud data into a noise segmentation model to obtain a noise segmentation result;

[0010] In a unified view, verifying the obstacle detection result and the noise segmentation result, and determining whether each obstacle in the obstacle detection result is a false detection.

[0011] In some of these embodiments, under the unified view, verifying the obstacle detection result with the noise segmentation result to determine whether each obstacle in the obstacle detection result is a false detection includes:

[0012] When the noise segmentation result is in the feature view and the obstacle detection result is in the bird's-eye view;

[0013] Converting the noise segmentation result into a grid map in the bird's-eye view to obtain a noise segmentation map;

[0014] Under the bird's-eye view, verifying the obstacle detection result with the noise segmentation map to determine whether each obstacle in the obstacle detection result is a false detection.

[0015] In some of these embodiments, verifying the obstacle detection result with the noise segmentation map to determine whether each obstacle in the obstacle detection result is a false detection includes:

[0016] Verifying the obstacle detection result with the noise segmentation map to obtain the noise point cloud ratio of each obstacle in the obstacle detection result;

[0017] Comparing the noise point cloud ratio of the obstacle with a preset ratio threshold, and determining whether the obstacle is a false detection according to the comparison result.

[0018] In some of these embodiments, under the unified view, verifying the obstacle detection result with the noise segmentation result to determine whether each obstacle in the obstacle detection result is a false detection includes:

[0019] When the noise segmentation result is in the feature view and the obstacle detection result is in the bird's-eye view;

[0020] Converting the obstacle detection result into a grid map in the feature view to obtain an obstacle detection map;

[0021] Under the feature view, verifying the obstacle detection map with the noise segmentation result to determine whether each obstacle in the obstacle detection result is a false detection.

[0022] In some of these embodiments, converting the obstacle detection result into a grid map in the feature view to obtain an obstacle detection map includes:

[0023] Based on preset projection parameters, projecting the point cloud data corresponding to all two-dimensional points in each grid of the obstacle detection result into the grid map in the feature view to obtain the obstacle detection map.

[0024] In some of these embodiments, verifying the obstacle detection map with the noise segmentation result to determine whether each obstacle in the obstacle detection result is a false detection includes:

[0025] Verifying the obstacle detection map with the noise segmentation result to obtain the noise point cloud ratio of each obstacle in the obstacle detection result;

[0026] Comparing the noise point cloud ratio of the obstacle with a preset ratio threshold, and determining whether the obstacle is a false detection according to the comparison result.

[0027] In some of these embodiments, inputting the point cloud data into a noise segmentation model to obtain a noise segmentation result includes:

[0028] Converting the point cloud data into a feature map in the feature view;

[0029] Inputting the feature map with the point cloud data into the noise segmentation model to obtain a noise segmentation result in the feature view.

[0030] In some of these embodiments, converting the point cloud data into a feature map in the feature view includes:

[0031] Determining whether multiple points in the point cloud data are converted to the same pixel point in the feature map;

[0032] If multiple points in the point cloud data are converted to the same pixel point in the feature map, select a target point from the points to be converted into the feature map; the target point is the point closest to the origin of the vehicle body coordinate system among the points; the feature map is under the vehicle body coordinate system.

[0033] In some of these embodiments, in the unified view, verifying the obstacle detection result with the noise segmentation result to determine whether each obstacle in the obstacle detection result is a false detection includes:

[0034] When both the noise segmentation result and the obstacle detection result are in the feature view, in the feature view, verifying the obstacle detection result with the noise segmentation result to determine whether each obstacle in the obstacle detection result is a false detection;

[0035] Or, when both the noise segmentation result and the obstacle detection result are in the bird's-eye view, in the bird's-eye view, verifying the obstacle detection result with the noise segmentation result to determine whether each obstacle in the obstacle detection result is a false detection.

[0036] Second, in this embodiment, an obstacle detection device is provided, including:

[0037] An acquisition module, configured to acquire point cloud data of a lidar;

[0038] A first processing module, configured to input the point cloud data into an obstacle detection model to obtain an obstacle detection result;

[0039] A second processing module, configured to input the point cloud data into a noise segmentation model to obtain a noise segmentation result;

[0040] A verification module, configured to verify the obstacle detection result and the noise segmentation result under a unified view, and determine whether each obstacle in the obstacle detection result is a false detection.

[0041] In a third aspect, in this embodiment, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the obstacle detection method described in the first aspect above is implemented.

[0042] Compared with the related art, in the obstacle detection method, device, and computer device provided in this embodiment, by acquiring the point cloud data of the lidar; inputting the point cloud data into an obstacle detection model to obtain an obstacle detection result; inputting the point cloud data into a noise segmentation model to obtain a noise segmentation result; verifying the obstacle detection result and the noise segmentation result under a unified view, and determining whether each obstacle in the obstacle detection result is a false detection, the problems of large computational amount and poor obstacle detection effect in the related art are solved, the filtering of false detections of obstacles is realized, and the obstacle detection effect is improved.

[0043] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more concise and understandable. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0045] Figure 1 is a hardware structure block diagram of a terminal device for the obstacle detection method provided in an embodiment of the present application;

[0046] Figure 2 is a flowchart of the obstacle detection method provided in an embodiment of the present application;

[0047] Figure 3 is a flowchart of an embodiment of step S240;

[0048] Figure 4 is a flowchart of another embodiment of step S240;

[0049] Figure 5 is a flowchart of step S243;

[0050] Figure 6 is a flowchart of step S245;

[0051] Figure 7 is a flowchart of step S230;

[0052] Figure 8 is a structural block diagram of an obstacle detection device provided by an embodiment of the present application.

[0053] In the figure: 102, a processor; 104, a memory; 106, a transmission device; 108, an input / output device; 210, an acquisition module; 220, a first processing module; 230, a second processing module; 240, a verification module. Detailed implementation manners

[0054] For a clearer understanding of the purpose, technical solution, and advantages of the present application, the present application will be described and illustrated below with reference to the accompanying drawings and embodiments.

[0055] Unless otherwise defined, the technical terms or scientific terms involved in the present application shall have the general meaning understood by those with ordinary skills in the technical field to which the present application belongs. In the present application, words such as "a", "one", "a kind of", "the", "these", etc. do not indicate a limitation in quantity, and they can be singular or plural. The terms "including", "comprising", "having" and any variants thereof involved in the present application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device including a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent in these processes, methods, products, or devices. The terms "connected", "coupled", etc. involved in the present application do not limit to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in the present application refers to two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships may exist. For example, "A and / or B" may represent: A exists alone, A and B exist simultaneously, and B exists alone. Usually, the character " / " indicates that the objects associated before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in the present application only distinguish similar objects and do not represent a specific order for the objects.

[0056] The method embodiments provided in this embodiment can be executed on a terminal, a computer, or a similar computing device. For example, when running on a terminal, Figure 1 is a block diagram of the hardware structure of the terminal for the obstacle detection method in this embodiment. As Figure 1 shown, the terminal may include one or more ( Figure 1 only one is shown in the figure) processors 102 and a memory 104 for storing data. Among them, the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a field programmable gate array FPGA. The above terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above terminal. For example, the terminal may further include more or fewer components than Figure 1 shown in the figure, or have a different configuration from Figure 1 shown in the figure.

[0057] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to the obstacle detection method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely set relative to the processor 102, and these remote memories can be connected to the terminal through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0058] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one instance, the transmission device 106 includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (abbreviated as RF) module, which is used to communicate with the Internet wirelessly.

[0059] In this embodiment, an obstacle detection method is provided. Figure 2 is a flowchart of the obstacle detection method in this embodiment. As Figure 2 shown, the process includes the following steps:

[0060] Step S210: Obtain the point cloud data of the lidar;

[0061] Step S220: Input the point cloud data into the obstacle detection model to obtain the obstacle detection result;

[0062] Step S230: Input the point cloud data into the noise segmentation model to obtain the noise segmentation result;

[0063] Step S240: Under the unified view, verify the obstacle detection result and the noise segmentation result, and determine whether each obstacle in the obstacle detection result is a false detection.

[0064] It should be noted that both the obstacle detection model and the noise segmentation model are pre-trained and complete. The networks of the obstacle detection model and the noise segmentation model include, but are not limited to, convolutional neural network (CNN), fully connected neural network (FCN), generative adversarial network (GAN), residual network (ResNet), etc. By using relevant training samples and training methods to train the corresponding networks, the obstacle detection model and the noise segmentation model can be obtained. For example: For the noise segmentation model: If during training, the training sample is the point cloud data and the output is the noise segmentation result in the feature view (Range View, RV); RangeView can also be called the range view, interval view, or field of view range view, etc. Then when the noise segmentation model is applied, the input is the point cloud data and the output is the noise segmentation result in the feature view. If during training, the training sample is the feature map in the feature view and the output is the noise segmentation result in the feature view. Then when the noise segmentation model is applied, the input is the feature map in the feature view (the point cloud data needs to be converted into the feature map in the feature view), and the output is the noise segmentation result in the feature view. In this embodiment, the view can also be the bird's eye view (Bird's Eye View, BEV), point view (Point View, PV), etc. There is no limitation on the training samples and training methods.

[0065] Among them, the lidar is installed on the vehicle. During the use of the vehicle, the point cloud data of the lidar can be obtained in real time. Then, the point cloud data is respectively input into the obstacle detection model and the noise segmentation model to obtain the corresponding obstacle detection result and noise segmentation result. Among them, the obstacle detection result includes the detected obstacle and the point cloud information corresponding to the obstacle. If there are multiple obstacles in the obstacle detection result, each obstacle has its corresponding point cloud information. Finally, under the unified view, the obstacle detection result is verified with the noise segmentation result. At this time, the view where the obstacle detection result and the noise segmentation result are located can be determined first. If the obstacle detection result and the noise segmentation result are in the same view, then the obstacle detection result can be directly verified with the noise segmentation result to determine whether each obstacle in the obstacle detection result is a false detection. If the obstacle detection result and the noise segmentation result are in different views, then the obstacle detection result can be converted to the view of the noise segmentation result, or the noise segmentation result can be converted to the view of the obstacle detection result, or the obstacle detection result and the noise segmentation result can be converted to a third-party view, and then the obstacle detection result is verified with the noise segmentation result to determine whether each obstacle in the obstacle detection result is a false detection.

[0066] Since there are point cloud noises in the point cloud data collected by the lidar, it will affect the accuracy of obstacle detection. Therefore, in the traditional solution: a filtering algorithm is used to remove the point cloud noises in the point cloud data, and then obstacles are detected based on the denoised point cloud data to improve the detection accuracy. However, the computational complexity of this solution is very high, and its final detection effect also depends on the technical level of technicians, resulting in poor obstacle detection effect. In this embodiment, by obtaining the point cloud data of the lidar; inputting the point cloud data into the obstacle detection model to obtain the obstacle detection result; inputting the point cloud data into the noise segmentation model to obtain the noise segmentation result; under the unified view, verifying the obstacle detection result with the noise segmentation result to determine whether each obstacle in the obstacle detection result is a false detection, the problems of large computational complexity and poor obstacle detection effect in the related technology are solved. The output result of the noise segmentation model is used to verify the output result of the obstacle detection model, realizing the filtering of false detections of obstacles and improving the detection effect of obstacles.

[0067] The above steps will be described in detail below:

[0068] It should be noted that the specific implementation of step S240 is also different when the views where the noise segmentation result and the obstacle detection result are located are different.

[0069] For the case where the noise segmentation result is in the feature view and the obstacle detection result is in the bird's-eye view:

[0070] In some of the embodiments, such asFigure 3 As shown, in the unified view in step S240, verifying the obstacle detection result and the noise segmentation result to determine whether each obstacle in the obstacle detection result is a false detection includes the following steps:

[0071] Step S241, when the noise segmentation result is in the feature view and the obstacle detection result is in the bird's-eye view;

[0072] Step S242, converting the noise segmentation result into a grid map in the bird's-eye view to obtain a noise segmentation map;

[0073] Step S243, in the bird's-eye view, verifying the obstacle detection result and the noise segmentation map to determine whether each obstacle in the obstacle detection result is a false detection;

[0074] Among them, the steps from step S241 to step S243 can be considered as uniformly converting to the bird's-eye view for verification to determine whether each obstacle in the obstacle detection result is a false detection.

[0075] Since the noise segmentation result is used to verify whether the obstacles in the obstacle detection result are false detections, and since the point cloud information corresponding to the obstacles is in the bird's-eye view (the bird's-eye view is constructed in the vehicle body coordinate system), in order to better verify the obstacles in the obstacle detection result in the bird's-eye view with the noise segmentation result in the feature view, the noise segmentation information in the noise segmentation result is projected into a grid map in the bird's-eye view with a preset size using a relevant conversion algorithm to obtain a noise segmentation map; the size of the grid map is the same as the size of the map where the obstacles are located in the obstacle detection result.

[0076] For example: The size of the grid map can be 800×320, in its bird's-eye view. This grid map represents the range of [-20m, 140m] in the x direction and [-32m, 32m] in the y direction in the vehicle body coordinate system, and the distance unit is 0.2m. During the conversion process, multiple pixels (RV pixels) of different noise segmentation results will be converted into pixels in the same grid map in the bird's-eye view; since noise is a strong feature, as long as one of these RV pixels is noise, the corresponding pixel in the bird's-eye view is also a noise pixel, and if all of these RV pixels are non-noise pixels, the corresponding pixel in the bird's-eye view is also a non-noise pixel. For the judgment of whether it is noise, it can be: The input feature value of each pixel in the noise segmentation map corresponds to the x, y, z position information in the vehicle body coordinate system (bird's-eye view); thus, it can be determined which pixel in the grid map in the bird's-eye view of the obstacle detection result this pixel corresponds to. Thereby, it can be determined whether this point is noise. The above is only an example, and of course, the specific parameters of the grid map are not limited.

[0077] In this embodiment, while reducing the computational amount, it is possible to convert the obstacle detection result and the noise segmentation result to the bird's-eye view for verification to determine whether each obstacle in the obstacle detection result is a false detection.

[0078] In some of these embodiments, as Figure 4 shown, in the unified view in step S240, verifying the obstacle detection result and the noise segmentation result to determine whether each obstacle in the obstacle detection result is a false detection includes the following steps:

[0079] Step S244, when the noise segmentation result is in the feature view and the obstacle detection result is in the bird's-eye view;

[0080] Step S245, converting the obstacle detection result to a grid map in the feature view to obtain an obstacle detection map;

[0081] Step S246, in the feature view, verifying the obstacle detection map and the noise segmentation result to determine whether each obstacle in the obstacle detection result is a false detection.

[0082] Among them, the steps from step S244 to step S246 can be considered as uniformly converting to the feature view for verification to determine whether each obstacle in the obstacle detection result is a false detection.

[0083] Since the noise segmentation result is used to verify whether the obstacles in the obstacle detection result are false detections, and since the point cloud information corresponding to the obstacles is in the bird's-eye view, in order to better verify the obstacles in the obstacle detection result in the bird's-eye view with the noise segmentation result in the feature view, a relevant conversion algorithm is used to convert the obstacle detection result to a grid map in the feature view with a preset size to obtain an obstacle detection map; the size of the grid map is the same as the size of the map where the obstacles in the obstacle detection result are located.

[0084] In this embodiment, while reducing the computational amount, it is possible to convert the obstacle detection result and the noise segmentation result to the feature view for verification to determine whether each obstacle in the obstacle detection result is a false detection.

[0085] In other embodiments, a similar method as above can also be used to convert the obstacle detection result and the noise segmentation result to a third-party view, and then verify the obstacle detection result and the noise segmentation result to determine whether each obstacle in the obstacle detection result is a false detection, which will not be repeated here.

[0086] In some of these embodiments, as Figure 5As shown, verifying the obstacle detection result with the noise segmentation map in step S243 to determine whether each obstacle in the obstacle detection result is a false detection includes the following steps:

[0087] Step S2431: Verify the obstacle detection result with the noise segmentation map to obtain the noise point cloud ratio of each obstacle in the obstacle detection result;

[0088] Step S2432: Compare the noise point cloud ratio of the obstacle with a preset ratio threshold, and determine whether the obstacle is a false detection according to the comparison result.

[0089] Specifically, the obstacle detection result includes obstacles and point cloud information; each obstacle has corresponding point cloud information. The above implementation process is as follows:

[0090] First, determine whether the corresponding pixel point of each point cloud information in the detection result in the noise segmentation map is a noise pixel; if it is a noise pixel, mark the point cloud information as a noise point cloud; second, traverse all the point cloud information in the obstacle detection result, and use the number of marked noise point clouds as the evaluation condition to obtain the noise point cloud ratio of each obstacle in the obstacle detection result; third, traverse each obstacle to determine whether the noise point cloud ratio of the obstacle exceeds the preset ratio threshold; when the noise point cloud ratio of the obstacle exceeds the preset ratio threshold, remove the corresponding obstacle; when the noise point cloud ratio of the obstacle does not exceed the preset ratio threshold, retain the corresponding obstacle. In the above judgment process, the ratio threshold can also be adjusted according to the size of the obstacle; if the obstacle is in the range below ten meters, the ratio threshold can be 70%; if the obstacle is in the range above ten meters, the ratio threshold can be 30%.

[0091] Through this embodiment, the filtering effect of obstacle false detection is improved, and the accuracy of obstacle detection is guaranteed.

[0092] In other embodiments, in order to improve the filtering effect of obstacle false detection and guarantee the accuracy of obstacle detection, multiple detections can be performed and then comprehensively evaluated. For example: steps S2431 to S2432 are for the detection result of one frame of point cloud data, then the detection results of multiple frames of point cloud data can be continuously detected, and if the obstacle is judged to be a false detection in two consecutive detection results, then it will be finally confirmed.

[0093] In some of the embodiments, as Figure 6 shown, converting the obstacle detection result into a grid map in the feature view in step S245 to obtain an obstacle detection map includes the following steps:

[0094] Step S2451: Based on the preset projection parameters, project the point cloud data corresponding to all two-dimensional points in each grid of the obstacle detection result onto the grid map under the feature view to obtain the obstacle detection map.

[0095] Specifically, a relevant conversion algorithm is used to convert the obstacle detection result into the grid map under the feature view to obtain the detected grid map. Among them, the projection parameters are preset and are the internal and external parameters of the point cloud data acquisition device. Since the bird's-eye view is a view from above looking at an object or scene and has no height information, while the feature view is a view from a horizontal perspective and has height information. Therefore, the point cloud data (three-dimensional points) corresponding to all two-dimensional points in the grid is used to provide the height information required by the feature view, thereby completing the conversion of all two-dimensional points in the obstacle detection result. The implementation process can be: combine the point cloud data (three-dimensional points) corresponding to all two-dimensional points in each grid of the obstacle detection result with the projection parameters and project them onto the grid map under the feature view; traverse each grid in each obstacle detection result in this way to obtain the obstacle detection map.

[0096] Through this embodiment, the missing height information is provided by the point cloud data, improving the accuracy of the data in the obstacle detection map.

[0097] In some of these embodiments, the step of verifying the obstacle detection map with the noise segmentation result in step S246 to determine whether each obstacle in the obstacle detection result is a false detection includes the following steps:

[0098] Step S2461: Verify the obstacle detection map with the noise segmentation result to obtain the noise point cloud ratio of each obstacle in the obstacle detection result;

[0099] Step S2462: Compare the noise point cloud ratio of the obstacle with a preset ratio threshold, and determine whether the obstacle is a false detection according to the comparison result.

[0100] The above process is as follows: Determine whether the point cloud information corresponding to each pixel point in the obstacle detection map and the pixel point corresponding to the noise segmentation result is a noise pixel; if it is a noise pixel, mark the point cloud information as noise point cloud; secondly, traverse all the point cloud information, and use the number of marked noise point clouds as the evaluation condition to obtain the noise point cloud ratio of each obstacle in the obstacle detection result; thirdly, traverse each obstacle and judge whether the noise point cloud ratio of the obstacle exceeds the preset ratio threshold; when the noise point cloud ratio of the obstacle exceeds the preset ratio threshold, remove the corresponding obstacle; when the noise point cloud ratio of the obstacle does not exceed the preset ratio threshold, retain the corresponding obstacle. In the above judgment process, the ratio threshold can also be adjusted according to the size of the obstacle; if the obstacle is in a range below ten meters, the ratio threshold can be 70%; if the obstacle is in a range above ten meters, the ratio threshold can be 30%.

[0101] Through this embodiment, by using the characteristics of the feature view, obstacles at different distances can be identified to further improve the filtering effect of obstacle misdetection and ensure the accuracy of obstacle detection.

[0102] For the case where both the noise segmentation result and the obstacle detection result are in the feature view, or both the noise segmentation result and the obstacle detection result are in the bird's-eye view:

[0103] In some of these embodiments, in step S240, under the unified view, the obstacle detection result is verified with the noise segmentation result to determine whether each obstacle in the obstacle detection result is a misdetection, including the following steps:

[0104] When both the noise segmentation result and the obstacle detection result are in the feature view, in the feature view, the obstacle detection result is verified with the noise segmentation result to determine whether each obstacle in the obstacle detection result is a misdetection;

[0105] Or, when both the noise segmentation result and the obstacle detection result are in the bird's-eye view, in the bird's-eye view, the obstacle detection result is verified with the noise segmentation result to determine whether each obstacle in the obstacle detection result is a misdetection.

[0106] The specific implementation of this embodiment can refer to the above related process and will not be repeated here.

[0107] In some of these embodiments, in step S220, inputting the point cloud data into the obstacle detection model to obtain the obstacle detection result includes the following steps:

[0108] Step S221, preprocess the point cloud data;

[0109] Step S222: Input the preprocessed point cloud data into the obstacle detection model to obtain the obstacle detection result.

[0110] Specifically, the preprocessing includes but is not limited to filtering and denoising. Among them, in the scenario of a vehicle driving on a road, the filtering process uses a road edge detection algorithm to filter the point cloud data on both sides of the road, which can effectively reduce the number of irrelevant point clouds, thereby obtaining effective point cloud data. Since the acquisition of point cloud data is affected by factors such as the accuracy of lidar, it is necessary to denoise the point cloud data.

[0111] Through this embodiment, inputting the preprocessed point cloud data into the obstacle detection model to obtain the obstacle detection result can effectively improve the recognition accuracy of the obstacle detection model.

[0112] In some of these embodiments, as Figure 7 shown, inputting the point cloud data into the noise segmentation model in step S230 to obtain the noise segmentation result includes the following steps:

[0113] Step S231: Convert the point cloud data into a feature map in the feature view;

[0114] Step S232: Input the feature map with the point cloud data into the noise segmentation model to obtain the noise segmentation result in the feature view.

[0115] Specifically, all the point cloud data is converted into a feature map (RV feature map) in the RV view. The size of this feature map is 640×128, its vertical size is [-64°, 64°], its horizontal size is [-12.8°, 12.8°], and the resolution is 0.2. The above is just an example, and of course, the specific parameters of the feature map are not limited.

[0116] Input the above feature map into the noise segmentation model to obtain the noise segmentation result in the feature view; each pixel value in this noise segmentation result is a value within the range of [0, 1], and this value represents the possibility that the pixel is noise, with the possibility of 1 being the greatest and the possibility of 0 being the smallest. If the predicted value of this pixel is greater than the preset noise threshold, then this pixel is a noise pixel; otherwise, this pixel is a non-noise pixel.

[0117] Among them, the view where the feature map is located is related to the view of the training samples of the noise segmentation model; for example: if the training samples of the noise segmentation model are in the RV view, then all the point cloud data is converted into a feature map in the RV view. In other embodiments, if the training samples of the noise segmentation model are in other views, then the point cloud data will be correspondingly converted into the corresponding view, and this is not limited.

[0118] Among them, the training samples can be RV feature maps, and each feature point of the RV feature map includes at least three feature information (x, y, z). Of course, the more feature information of the feature point, the better the noise segmentation effect of the noise segmentation model; in other embodiments, the feature information of the feature point can also include reflection intensity information, depth information, etc.; among them, the depth information is If each feature point includes the above five feature information, then the RV feature map can be expressed as 640×128×5. Since the feature map with point cloud data corresponds to the RV feature map of the training sample, the feature map with point cloud data can also be expressed as 640×128×5.

[0119] Through this embodiment, the point cloud data is converted into the feature map under the feature view, and the effective and stable data is retained, so that the output of the noise segmentation model is more generalizable.

[0120] In some of these embodiments, converting the point cloud data into the feature map in step S231 includes the following steps:

[0121] Determine whether multiple points in the point cloud data are converted to the same pixel point of the feature map;

[0122] If multiple points in the point cloud data are converted to the same pixel point of the feature map, select a target point from the points and convert it into the feature map; the target point is the point closest to the origin of the vehicle body coordinate system among the points; the feature map is under the vehicle body coordinate system.

[0123] Specifically, during the conversion of the point cloud data into the feature map, multiple different points will be converted into the same pixel in the feature map, which is determined by geometric properties. In order to preferentially detect the obstacle closest to the vehicle body, the point closest to the origin of the vehicle body coordinate system is selected as the target point and converted into the feature map. In other embodiments, these points can also be fused into a target point and then converted into the feature map, which can improve the robustness of the detection.

[0124] It should be noted that the steps shown in the above process or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0125] In this embodiment, an obstacle detection device is also provided. This device is used to implement the above-mentioned embodiments and preferred implementation manners, and those that have been described will not be repeated here. The following terms such as "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0126] Figure 8 is the structural block diagram of the obstacle detection device in this embodiment. As Figure 8 shown, the device includes:

[0127] An acquisition module 210, configured to acquire the point cloud data of the lidar;

[0128] A first processing module 220, configured to input the point cloud data into an obstacle detection model to obtain an obstacle detection result;

[0129] A second processing module 230, configured to input the point cloud data into a noise segmentation model to obtain a noise segmentation result;

[0130] A verification module 240, configured to verify the obstacle detection result and the noise segmentation result under a unified view to determine whether each obstacle in the obstacle detection result is a false detection.

[0131] Through the above device, the problems of large computational amount and poor obstacle detection effect in the related art are solved, the filtering of false detections of obstacles is realized, and the detection effect of obstacles is improved.

[0132] In some of these embodiments, the noise segmentation result is in the feature view, and the obstacle detection result is in the bird's-eye view; the verification module 240 is further configured to convert the noise segmentation result into a grid map in the bird's-eye view to obtain a noise segmentation map;

[0133] Verify the obstacle detection result and the noise segmentation map to determine whether each obstacle in the obstacle detection result is a false detection;

[0134] Or, convert the obstacle detection result into a grid map in the feature view to obtain an obstacle detection map;

[0135] Verify the obstacle detection map and the noise segmentation result to determine whether each obstacle in the obstacle detection result is a false detection.

[0136] In some of these embodiments, the verification module 240 is further configured to verify the obstacle detection result and the noise segmentation map to obtain the noise point cloud ratio of each obstacle in the obstacle detection result;

[0137] Compare the noise point cloud ratio of the obstacle with a preset ratio threshold, and determine whether the obstacle is a false detection according to the comparison result.

[0138] In some embodiments, the verification module 240 is further configured to project the point cloud data corresponding to all two-dimensional points in each grid in the obstacle detection result onto the grid map in the feature view based on the preset projection parameters, to obtain an obstacle detection map.

[0139] In some embodiments, the verification module 240 is further configured to verify the obstacle detection map with the noise segmentation result to obtain the noise point cloud ratio of each obstacle in the obstacle detection result;

[0140] Compare the noise point cloud ratio of the obstacle with a preset ratio threshold, and determine whether the obstacle is a false detection according to the comparison result.

[0141] In some embodiments, the second processing module 230 is further configured to convert the point cloud data into a feature map in the feature view;

[0142] Input the feature map with the point cloud data into the noise segmentation model to obtain the noise segmentation result in the feature view.

[0143] In some embodiments, the second processing module 230 is further configured to determine whether multiple points in the point cloud data are converted to the same pixel point in the feature map;

[0144] If multiple points in the point cloud data are converted to the same pixel point in the feature map, select a target point from the points and convert it into the feature map; the target point is the point closest to the origin of the vehicle body coordinate system among the points; the feature map is in the vehicle body coordinate system.

[0145] In some embodiments, both the noise segmentation result and the obstacle detection result are in the feature view, or, both the noise segmentation result and the obstacle detection result are in the bird's-eye view;

[0146] The verification module 240 is further configured to verify the obstacle detection result with the noise segmentation result to determine whether each obstacle in the obstacle detection result is a false detection.

[0147] It should be noted that the above-mentioned various modules can be functional modules or program modules, and can be implemented either by software or by hardware. For the modules implemented by hardware, the above-mentioned various modules can be located in the same processor; or the above-mentioned various modules can also be located in different processors in any combined form.

[0148] In this embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0149] Optionally, the above computer device may further include a transmission device and an input / output device. Among them, the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0150] Optionally, in this embodiment, the above processor may be configured to execute the following steps through a computer program:

[0151] S1, obtain the point cloud data of the lidar;

[0152] S2, input the point cloud data into the obstacle detection model to obtain the obstacle detection result;

[0153] S3, input the point cloud data into the noise segmentation model to obtain the noise segmentation result;

[0154] S4, under a unified view, verify the obstacle detection result and the noise segmentation result, and determine whether each obstacle in the obstacle detection result is a false detection.

[0155] It should be noted that the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and will not be repeated in this embodiment.

[0156] In addition, in combination with the obstacle detection method provided in the above embodiments, a storage medium may also be provided in this embodiment to implement it. A computer program is stored on the storage medium; when the computer program is executed by a processor, any one of the obstacle detection methods in the above embodiments is implemented.

[0157] It should be understood that the specific embodiments described here are only used to explain this application, rather than to limit it. According to the embodiments provided in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the protection scope of the present application.

[0158] Obviously, the drawings are only some examples or embodiments of the present application. For those of ordinary skill in the art, the present application can also be applied to other similar situations according to these drawings without creative work. In addition, it can be understood that although the work done during the development process may be complex and time-consuming, for those of ordinary skill in the art, some design, manufacturing, or production changes based on the technical content disclosed in the present application are only conventional technical means and should not be regarded as insufficient disclosure of the present application.

[0159] As used herein, the term "embodiment" means that the specific features, structures, or characteristics described in connection with an embodiment may be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily mean the same embodiment, nor does it mean independence or alternative to other embodiments that are mutually exclusive. Those of ordinary skill in the art can clearly or implicitly understand that the embodiments described in the present application can be combined with other embodiments without conflict.

[0160] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of patent protection. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. An obstacle detection method, characterized in that: include: Get the point cloud data of LiDAR; Inputting the point cloud data into an obstacle detection model to obtain an obstacle detection result; Inputting the point cloud data into a noise segmentation model to obtain a noise segmentation result; Under a unified view, verifying the obstacle detection result with the noise segmentation result to determine whether each obstacle in the obstacle detection result is a false detection; The point cloud data is input into the noise segmentation model to obtain the noise segmentation result, including: Converting the point cloud data into a feature map under a feature view; The feature map with the point cloud data is input into a noise segmentation model to obtain a noise segmentation result under a feature view.

2. The obstacle detection method according to claim 1, characterized in that: Converting the point cloud data into a feature map under a feature view includes: Determine whether multiple points in the point cloud data are converted to the same pixel point of the feature map; If multiple points in the point cloud data are converted to the same pixel point of the feature map, a target point is selected from the points and converted to the feature map; the target point is the point among the points that is closest to the original location of the vehicle body coordinate system; the feature map is in the vehicle body coordinate system.

3. An obstacle detection device, characterized in that: include: Acquisition module, used to obtain point cloud data of laser radar; A first processing module, used for inputting the point cloud data into an obstacle detection model to obtain an obstacle detection result; A second processing module is used to input the point cloud data into a noise segmentation model to obtain a noise segmentation result; The second processing module is further used to convert the point cloud data into a feature map under a feature view; Inputting the feature map with the point cloud data into a noise segmentation model to obtain a noise segmentation result under a feature view; The verification module is used to verify the obstacle detection result and the noise segmentation result in a unified view to determine whether each obstacle in the obstacle detection result is a false detection.

4. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the steps of the obstacle detection method according to any one of claims 1 to 2.

Images