Obstacle identification method and device, electronic equipment and vehicle

Abstract

The present application relates to a kind of obstacle identification method, device, electronic equipment and vehicle, obtain the point cloud data of the point cloud device of vehicle acquisition;Determine the emission law of laser beam emitted by point cloud device, point cloud data is divided into the point cloud data of visible area and the point cloud data of invisible area;Filter the point cloud data of invisible area in point cloud data and obtain first filtered point cloud data;First filtered point cloud data is input into bird's eye view perception model, and first output result output by bird's eye view perception model is obtained;According to first output result, filter noise in first filtered point cloud data and obtain second filtered point cloud data;First filtered point cloud data is input into point cloud denoising model, and second output result output by point cloud denoising model is obtained;According to second filtered point cloud data, obstacle model and second output result, determine the obstacle identification result corresponding to point cloud data.The present application embodiment can cope with complex mixed noise scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving environment perception, in particular to an obstacle identification method and device, an electronic device and a vehicle. BACKGROUND

[0002] In an intelligent driving system, a laser radar is a core sensor for obtaining three-dimensional information (environment information) of an environment around a vehicle, and point cloud data generated by the laser radar can accurately reflect distance, shape and position of an obstacle and the like. However, in actual application, the point cloud data often contains noises of multiple noise types, which affects the accuracy and robustness of environment information obtained by the intelligent driving system.

[0003] Currently, mainstream denoising technologies are designed only for a single noise type, and cannot filter noises of multiple noise types in the point cloud data, so as to be difficult to cope with a complex mixed noise scene, and further to cause that intelligent driving cannot be accurately performed based on the point cloud data, and personal safety of a user of the vehicle cannot be ensured. SUMMARY

[0004] One of the purposes of the present application is to provide an obstacle identification method to solve the problem that in the related art, noises of multiple noise types in the point cloud data cannot be filtered, so as to be difficult to cope with a complex mixed noise scene, and further to cause that intelligent driving cannot be accurately performed based on the point cloud data, and personal safety of a user of the vehicle cannot be ensured; the second purpose is to provide an obstacle identification device; the third purpose is to provide an electronic device; and the fourth purpose is to provide a vehicle.

[0005] In order to achieve the above purposes, the technical solutions adopted by the present application are as follows:

[0006] An obstacle identification method, the method comprising:

[0007] acquiring point cloud data collected by a point cloud device of a vehicle;

[0008] determining a transmission rule of a laser beam transmitted by the point cloud device when the point cloud data is collected, dividing the point cloud data into point cloud data of a visible region and point cloud data of an invisible region; the point cloud data of the visible region is point cloud data covered by the laser beam and not blocked by an obstacle, and the point cloud data of the invisible region is point cloud data other than the point cloud data of the visible region in the point cloud data; filtering the point cloud data of the invisible region in the point cloud data to obtain first filtered point cloud data;

[0009] input the first filtered point cloud data into a bird's eye view perception model to obtain a first output result output by the bird's eye view perception model; the first output result comprises at least a point cloud segmentation result corresponding to the first filtered point cloud data under different perspectives, obstacle semantic category information, and an obstacle model; the point cloud segmentation result comprises a first semantic category, and the first semantic category is used to represent that the first filtered point cloud data is noise or an obstacle; the obstacle semantic category information is used to represent that the first filtered point cloud data is an obstacle;

[0010] filtering noise in the first filtered point cloud data according to the first output result to obtain second filtered point cloud data;

[0011] input the second filtered point cloud data into a point cloud denoising model to obtain a second output result output by the point cloud denoising model; the second output result comprises second filtered point cloud data segmented according to a set size and a second semantic category corresponding to the segmented second filtered point cloud data; the second semantic category is used to represent that the second filtered point cloud data is noise or an obstacle;

[0012] determining an obstacle recognition result corresponding to the point cloud data according to the second filtered point cloud data, the obstacle model, and the second output result.

[0013] An obstacle recognition device, the device comprising:

[0014] a point cloud data acquisition module, configured to acquire point cloud data collected by a point cloud device of a vehicle;

[0015] a first point cloud data filtering module, configured to determine a transmission rule of a laser beam emitted by the point cloud device when collecting the point cloud data, divide the point cloud data into point cloud data of a visible region and point cloud data of an invisible region; the point cloud data of the visible region is point cloud data covered by the laser beam and not blocked by an obstacle, and the point cloud data of the invisible region is point cloud data other than the point cloud data of the visible region; and filter the point cloud data of the invisible region to obtain first filtered point cloud data;

[0016] a first output result output module, configured to input the first filtered point cloud data into a bird's eye view perception model to obtain a first output result output by the bird's eye view perception model; the first output result comprises at least a point cloud segmentation result corresponding to the first filtered point cloud data under different perspectives, obstacle semantic category information, and an obstacle model; the point cloud segmentation result comprises a first semantic category, and the first semantic category is used to represent that the first filtered point cloud data is noise or an obstacle; and the obstacle semantic category information is used to represent that the first filtered point cloud data is an obstacle.

[0017] a second point cloud data filtering module configured to filter noise in the first filtered point cloud data according to the first output result to obtain second filtered point cloud data;

[0018] a second output result output module configured to input the second filtered point cloud data into a point cloud denoising model to obtain a second output result output by the point cloud denoising model; the second output result includes second filtered point cloud data obtained by segmenting the second filtered point cloud data according to a set size and a second semantic category corresponding to the segmented second filtered point cloud data; the second semantic category is used to represent whether the second filtered point cloud data is noise or an obstacle;

[0019] an obstacle identification module configured to determine an obstacle identification result corresponding to the point cloud data according to the second filtered point cloud data, the obstacle model and the second output result.

[0020] An electronic device, comprising: a processor; a memory for storing processor-executable instructions;

[0021] The processor is configured to execute the instructions to implement the obstacle identification method described above.

[0022] A computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the mobile terminal, the mobile terminal can execute the obstacle identification method described above.

[0023] A vehicle, comprising the electronic device described above.

[0024] The beneficial effects of the present application are:

[0025] In the embodiment of the present application, first, the point cloud data collected by the point cloud device of the vehicle is obtained, the noise in the point cloud data can be filtered according to the spatial distribution of the point cloud data to obtain first filtered point cloud data, that is, the emission rule of the laser beam emitted by the point cloud device when collecting the point cloud data is determined, and the point cloud data is divided into point cloud data in the visible region and point cloud data in the invisible region; the point cloud data in the visible region is the point cloud data covered by the laser beam and not blocked by the obstacle, and the point cloud data in the invisible region is the point cloud data other than the point cloud data in the visible region; the point cloud data in the invisible region is filtered to obtain the first filtered point cloud data, which realizes the rough filtering of the point cloud data based on the physical rule, and can quickly filter out the obvious and typical noise in the point cloud data; then, the first filtered point cloud data is input into the bird's eye view perception model to obtain the first output result output by the bird's eye view perception model, wherein the first output result includes at least the first filtered point cloud data and the point cloud segmentation result corresponding to the first filtered point cloud data, the obstacle semantic category information and the obstacle model under different viewing angles, the point cloud segmentation result includes a first semantic category, the first semantic category is used to represent that the first filtered point cloud data is noise or an obstacle, and the obstacle semantic category information is used to represent that the point cloud data is an obstacle; the noise in the first filtered point cloud data is filtered according to the first output result to obtain second filtered point cloud data, the second filtered point cloud data is input into the point cloud denoising model, and the second output result output by the point cloud denoising model is obtained, the second output result includes the second filtered point cloud data segmented according to the set size and the second semantic category corresponding to the segmented second filtered point cloud data, and the second semantic category is used to represent that the second filtered point cloud data is noise or an obstacle; finally, the obstacle recognition result corresponding to the point cloud data can be determined according to the second filtered point cloud data, the obstacle model and the second output result, so that the obstacle around the vehicle can be determined according to the obstacle recognition result. The embodiment of the present application filters the point cloud data by multiple different filtering methods to filter different noise types in the point cloud data, so as to cope with complex mixed noise scenes, and then accurately perform intelligent driving based on the point cloud data, thereby ensuring the driving safety of the vehicle. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 A step flow chart of a method for identifying an obstacle provided in the embodiment of the present application;

[0027] Figure 2 A schematic diagram of the emission rule of the laser beam of a laser radar provided in the embodiment of the present application;

[0028] Figure 3 A schematic diagram of the visible region and the invisible region of point cloud data provided in the embodiment of the present application;

[0029] Figure 4 This is a schematic diagram of the structure of an obstacle recognition device provided in an embodiment of the present invention;

[0030] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0031] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0032] It should be noted that the embodiments of the present invention may involve the use of user data. In practical applications, user-specific personal data may be used in the scheme described herein within the scope permitted by applicable laws and regulations, provided that it complies with the applicable laws and regulations of the country (e.g., with the user's explicit consent, with the user being properly notified, etc.).

[0033] Reference Figure 1 The diagram illustrates a flowchart of an obstacle recognition method provided in an embodiment of the present invention. The method specifically includes the following steps:

[0034] Step 101: Obtain point cloud data collected by the vehicle's point cloud device.

[0035] In some embodiments, a point cloud device may refer to a LiDAR (Light Detection and Ranging) system installed on a vehicle. A LiDAR system generates point cloud data that reflects the environment surrounding the vehicle by emitting a laser beam and measuring the time it takes for the laser beam to return after hitting an object. Point cloud data consists of a large number of three-dimensional spatial points. Each point in the point cloud data contains multi-source feature information such as cloud space coordinates (X, Y, Z), echo intensity, object reflectivity, and a timestamp. Noise may be present in the point cloud data due to various reasons. Noise refers to points in the point cloud data that do not represent real physical obstacles. For example, noise may be spurious points generated by raindrops, dust, point cloud device errors, or specular reflection.

[0036] In some complex mixed noise scenes, the point cloud data usually has multiple different noise types, and exemplary noise types can include, but are not limited to: environmental noise: reflection points of raindrops / snowflakes in rainy / snowy weather, scattering points of suspended particles in foggy weather, forming diffuse distributed noise points; device noise: false point cloud generated by laser diffuse reflection or specular reflection on the vehicle itself (such as the roof); dynamic interference: transient noise points generated by moving objects such as flying birds and mosquitoes; signal noise: outlier noise points caused by sensor accuracy limitations and physical interference.

[0037] Step 102, determining the emission rule of the laser beam emitted by the point cloud device when collecting the point cloud data, and dividing the point cloud data into point cloud data of a visible region and point cloud data of an invisible region; the point cloud data of the visible region is point cloud data covered by the laser beam and not blocked by an obstacle, and the point cloud data of the invisible region is point cloud data other than the point cloud data of the visible region; filtering the point cloud data of the invisible region to obtain first filtered point cloud data.

[0038] In a specific implementation, the spatial distribution refers to the geometric relationship between points in the point cloud data, and the statistical relationship between the points and the global space of the environment in which the vehicle is located. According to the spatial distribution, the most obvious and typical noise in the point cloud data can be filtered, such as self-vehicle reflection noise, specular reflection (for example, after the laser beam emitted by the laser radar, a high-reflectivity sign on the path may reflect the laser beam to other objects to form a false image), and the like, thereby obtaining the first filtered point cloud data, which realizes rough filtering of the point cloud data. Not only can it reduce the data processing amount when generating the obstacle recognition result subsequently, improve the processing efficiency, but also can improve the accuracy of the generated obstacle recognition result.

[0039] Step 103, inputting the first filtered point cloud data into a bird's eye view perception model to obtain a first output result output by the bird's eye view perception model; the first output result includes at least point cloud segmentation results corresponding to the first filtered point cloud data under different viewing angles, obstacle semantic class information, and an obstacle model; the point cloud segmentation result includes a first semantic class, and the first semantic class is used to represent that the first filtered point cloud data is noise or an obstacle, and the obstacle semantic class information is used to represent that the first filtered point cloud data is an obstacle.

[0040] In a specific implementation, a bev (Bird's Eye View, bird's eye view / top view) perception model can be deployed at the vehicle or the cloud end in communication with the vehicle device. In the embodiment of the application, the first filtered point cloud data can be input into the bird's eye view perception model, so as to obtain a first output result output by the bird's eye view perception model.

[0041] Specifically, the first output result includes at least the point cloud segmentation result corresponding to the first filtered point cloud data under different perspectives, the obstacle semantic category information and the obstacle model. Wherein, the point cloud segmentation result corresponding to the first filtered point cloud data under different perspectives can include the point cloud segmentation result of the first filtered point cloud data under the bird's eye view (bev domain) and the point cloud segmentation result of the first filtered point cloud data under the distance perspective (range image), wherein the bird's eye view refers to the point cloud data formed by projecting the three-dimensional point cloud data onto a two-dimensional plane from top to bottom (usually the ground plane). The distance perspective refers to the point cloud data formed by projecting the three-dimensional point cloud data onto a two-dimensional cylindrical surface with the laser radar itself as the origin. The obstacle model refers to the bird's eye view perception model generated for the point cloud data of the obstacle, which has a certain contour information. Based on the model, the size of the obstacle can be determined.

[0042] The point cloud segmentation results corresponding to the bird's eye view and the distance perspective include the first semantic category, which is used to represent that the first filtered point cloud data is noise or an obstacle. For example, the bird's eye view perception model learns the characteristics of noise (such as raindrops, fog, dust, mirror reflection, etc.) on the Range Image, such as small shape, scattered distribution, abnormal reflection intensity, and discontinuity with the surrounding environment, etc. For point cloud data with these characteristics, the bird's eye view perception model will output the first semantic category of noise for the point cloud data. Of course, the bird's eye view perception model can also learn the contour features of obstacles (such as other vehicles, traffic signal poles or pedestrians, etc.), and for point cloud data with these contour features, the bird's eye view perception model will output the first semantic category of obstacle for the point cloud data.

[0043] The obstacle semantic category information is used to represent the first filtered point cloud data as an obstacle, and the obstacle semantic category information can include a whitelist obstacle bounding box and a general obstacle (general obstacle). The whitelist obstacle bounding box is used to represent the point cloud data of the first filtered point cloud data as an obstacle, and the obstacle has a corresponding obstacle semantic category. Specifically, the whitelist obstacle bounding box means that the aerial view perception model can perceive the point cloud data corresponding to the first filtered point cloud data as an obstacle, and further specifically identifies the obstacle semantic category corresponding to the first filtered point cloud data, for example, the obstacle semantic category can be other vehicles, traffic signal poles, pedestrians, or the like. The general obstacle is used to represent the point cloud data of the first filtered point cloud data as an obstacle, but the obstacle does not have a corresponding obstacle semantic category. Specifically, the general obstacle means that the aerial view perception model can perceive the point cloud data corresponding to the first filtered point cloud data as an obstacle, but cannot specifically identify the obstacle semantic category corresponding to the first filtered point cloud data. For example, the aerial view perception model can only perceive the point cloud data of the first filtered point cloud data as an obstacle, and the obstacle needs to be avoided for the safety of vehicle driving.

[0044] Step 104: filtering noise in the first filtered point cloud data according to the first output result to obtain second filtered point cloud data.

[0045] In the embodiment of the present application, after obtaining the first output result in the first filtered point cloud data, the point cloud data of the first filtered point cloud data with the first semantic category as noise can be filtered according to the first semantic category in the first output result, for example, rain, snow, fog, and dust particles produce large-area, low-density, and uniformly distributed noise points / noise, thereby obtaining second filtered point cloud data further removed from noise.

[0046] Step 105: inputting the second filtered point cloud data into a point cloud denoising model to obtain a second output result output by the point cloud denoising model; the second output result includes second filtered point cloud data segmented according to a set size and a second semantic category corresponding to the segmented second filtered point cloud data; and the second semantic category is used to represent the second filtered point cloud data as noise or an obstacle.

[0047] In a specific implementation, the point cloud denoising model can be deployed at a vehicle or a cloud end in communication with the vehicle device. In the embodiment of the present application, the second filtered point cloud data can be input into the point cloud denoising model, thereby obtaining the second output result output by the point cloud denoising model. The point cloud denoising model removes noise in the point cloud data, which can be difficult point cloud data between noise and a real object, for example, a fast flying bird or insect.

[0048] Specifically, the second output result can include second filtered point cloud data obtained by segmenting the second filtered point cloud data according to a set size in three dimensions, and the segmented second filtered point cloud data and a second semantic category corresponding to the segmented second filtered point cloud data, wherein the second semantic category is assigned to each point in the segmented second filtered point cloud data. In addition, in order to balance the accuracy and the power limit, the set size can be selected to be a suitable size according to actual needs. For example, in order to pursue accuracy, the set size can be set to a smaller size, or in order to avoid overload operation or too long operation time, the set size can be set to a larger size.

[0049] The second semantic category is used to represent that the second filtered point cloud data is noise or an obstacle. For example, the point cloud denoising model learns the features of noise or obstacles, and outputs the second semantic category of noise or obstacles for point cloud data that has these features.

[0050] In the embodiment of the present application, the point cloud denoising model can be a machine learning model, such as a PointNet structure model, which can identify fast flying birds or insects and other obstacles in point cloud data. Specifically, labeled sample point cloud data containing noise and obstacles, such as sample point cloud data including birds or insects and other obstacles and including noise, can be obtained, and then the sample point cloud data can be used to train the machine learning model to be trained, so that the machine learning model can learn the features of noise or obstacles. When the machine learning model reaches a preset convergence condition, such as when the loss value of the machine learning model reaches a preset loss value or the number of iterations reaches a preset number of iterations, the trained point cloud denoising model is obtained. The trained point cloud denoising model can determine the second semantic category corresponding to each point in the point cloud data or the segmented point cloud data (such as the segmented second filtered point cloud data), and the second semantic category can determine whether it is noise or an obstacle. It can be seen that the point cloud denoising model has a classification function (classification is obstacle or noise).

[0051] Step 106, determining an obstacle recognition result corresponding to the point cloud data according to the second filtered point cloud data, the obstacle model and the second output result.

[0052] In the embodiments of the present application, the obstacle recognition result corresponding to the point cloud data collected by the laser radar of the vehicle can be determined according to the second filtered point cloud data (point cloud data after several times of noise removal), the obstacle model output by the bird's eye view perception model, and the second output result output by the point cloud denoising model, wherein the obstacle recognition result can include a whitelist detection result and an obstacle result, the whitelist detection result is a result of explicitly identifying the specific obstacle around the vehicle, for example, it can be a pedestrian or other vehicles, etc., and the obstacle result is a result of not explicitly identifying the specific obstacle around the vehicle, that is, only knowing that the vehicle is an obstacle around, but it is not clear whether it is a pedestrian or other vehicles or other obstacles. In actual application, in order to ensure the safety of vehicle driving, the whitelist detection result and the obstacle result will be used as the basis for decision-making when the vehicle is intelligently driven during driving.

[0053] In the above obstacle identification method, first, the point cloud data collected by the point cloud device of the vehicle is obtained, the noise in the point cloud data can be filtered according to the spatial distribution of the point cloud data to obtain first filtered point cloud data, that is, the emission rule of the laser beam emitted by the point cloud device for collecting the point cloud data is determined, and the point cloud data is divided into point cloud data in the visible region and point cloud data in the invisible region; the point cloud data in the visible region is the point cloud data covered by the laser beam and not blocked by the obstacle, and the point cloud data in the invisible region is the point cloud data other than the point cloud data in the visible region; the point cloud data in the invisible region is filtered to obtain the first filtered point cloud data, which realizes rough filtering of the point cloud data based on physical rules, and can quickly filter out obvious and typical noise in the point cloud data; then, the first filtered point cloud data is input into the bird's eye view perception model to obtain a first output result output by the bird's eye view perception model, wherein the first output result includes at least the first filtered point cloud data and the point cloud segmentation result corresponding to the first filtered point cloud data under different viewing angles, obstacle semantic category information and obstacle model, the point cloud segmentation result includes a first semantic category, the first semantic category is used to represent that the first filtered point cloud data is noise or an obstacle, and the obstacle semantic category information is used to represent that the point cloud data is an obstacle; the noise in the first filtered point cloud data is filtered according to the first output result to obtain second filtered point cloud data; the second filtered point cloud data is input into a point cloud denoising model to obtain a second output result output by the point cloud denoising model, the second output result includes the second filtered point cloud data segmented according to a set size and a second semantic category corresponding to the segmented second filtered point cloud data, and the second semantic category is used to represent that the second filtered point cloud data is noise or an obstacle; finally, the obstacle identification result corresponding to the point cloud data can be determined according to the second filtered point cloud data, the obstacle model and the second output result, so that the obstacles around the vehicle can be determined according to the obstacle identification result. The embodiments of the present application filter the point cloud data through multiple different filtering methods to filter different noise types in the point cloud data, so as to cope with complex mixed noise scenes, and then accurately perform intelligent driving based on the point cloud data, thereby ensuring the driving safety of the vehicle.

[0054] In an embodiment of the present application, the step 106 of determining the obstacle identification result corresponding to the point cloud data according to the second filtered point cloud data, the obstacle model and the second output result can include:

[0055] inputting the second filtered point cloud data and the obstacle model into a binary classification model to obtain a classification result output by the binary classification model; the classification result includes a third semantic category corresponding to the second filtered point cloud data; and the third semantic category is used to represent that the point cloud data is noise or an obstacle.

[0056] According to the classification result, the historical frame model information, and the second output result, an obstacle recognition result corresponding to the point cloud data is determined; the historical frame model information is information that is continuous in time sequence in the process of recognizing the obstacle.

[0057] In a specific implementation, a binary classification model can be deployed at the vehicle or a cloud communicating with the vehicle device, where the binary classification model can be a machine learning model. In some embodiments, the binary classification model can be a model trained by xgbbost (eXtreme Gradient Boosting, optimized distributed gradient boosting library). Specifically, xgbbost is an ensemble learning algorithm based on decision trees, and is a high-efficiency algorithm in the field of machine learning. The binary classification model can output two results that the point cloud data is an obstacle or noise.

[0058] In an embodiment of the present application, the second filtered point cloud data and the obstacle model can be input into the binary classification model, so as to obtain a classification result output by the binary classification model, where the classification result can include the second filtered point cloud data and a third semantic category corresponding to the second filtered point cloud data, and the third semantic category is used to represent that the point cloud data is noise or an obstacle. For example, the binary classification model learns the features of noise or obstacles, and for the point cloud data with these features, the binary classification model outputs the third semantic category that the point cloud data is noise or an obstacle.

[0059] Before the second filtered point cloud data and the obstacle model are input into the binary classification model, the second filtered point cloud data can be subjected to connected component segmentation (clustering) and spatial density analysis, where the purpose of the connected component segmentation is to divide the point cloud data close to each other into different clusters (point cloud clusters), i.e., the point cloud data corresponding to the candidate obstacle, and then the spatial density analysis is performed on the point cloud clusters to determine the density of the points in the point cloud clusters. The point cloud clusters formed by rain and fog usually have very low density and loose and disordered point cloud distribution, based on which it can be determined whether the point cloud cluster is noise generated by rain and fog and thus filtered. In this way, the connected component segmentation and the spatial density analysis can effectively filter out the diffuse noise generated by rain and fog in the point cloud data.

[0060] Then, historical frame model information can be obtained, where the historical frame model information refers to information about the point cloud data, semantic category, track, etc. of the obstacle in the history or in time sequence in the tracking process of the obstacle in the process of identifying the obstacle recognition result of the point cloud data. Based on the historical frame model information, the real obstacle (such as a partially occluded vehicle) with blurred features and the transient noise (such as a floating plastic bag, a flying bird, and a flying insect) can be determined, which greatly improves the accuracy of the obstacle recognition result.

[0061] In some embodiments, before inputting the second filtered point cloud data and the obstacle model into the binary classification model, the point cloud data can be combined with time continuity analysis and same layer point cloud continuity verification to obtain historical frame model information, so that small obstacles such as floating plastic bags, flying birds, and flying insects can be avoided from being misjudged as noise according to the historical frame model information. The time continuity analysis is to assign a unique track id to the obstacle (target) through multi-target tracking technology, and continuously analyze its motion trajectory, speed change, and stability of classification result between different frames. In practice, the obstacle (such as a vehicle) usually has a smooth, reasonable motion rule and a stable type label, while the transient noise shows a chaotic trajectory, a very short existence time, or a dramatic type label fluctuation. By determining the context of the noise through time continuity analysis, the real target that appears temporarily but behaves reasonably can be effectively avoided from being misjudged as noise. The same layer point cloud continuity verification is to analyze whether the distribution of the point cloud data from the same laser beam corresponding to the scanning line (laser vector) is continuous before or after generating the point cloud cluster. In practice, an obstacle (such as a road pillar) will block the scanning line and produce a continuous and tangible point cloud sequence, while the points generated by the transient noise will appear independently and randomly, which will destroy the continuity of the scanning line. Based on the combination of time continuity analysis and same layer point cloud continuity verification, the historical frame model information and the spatial feature can be obtained respectively and input into the binary classification model.

[0062] Finally, the obstacle recognition result corresponding to the point cloud data can be determined according to the classification result of the binary classification model, the historical frame model information, and the second output result output by the point cloud denoising model.

[0063] In the embodiment of the present application, the classification result based on the output of the binary classification model, that is, whether it is an obstacle or noise, is the decision basis for the current point cloud data. Then, the classification result of the current point cloud data can be verified by the historical frame model information (that is, the past tracking trajectory and existence duration of the target (obstacle or noise), etc.) to assist in verifying whether the classification result of the current point cloud data is reasonable. For example, if the existence duration of the point cloud data exceeds the preset duration, it should not be determined as noise, or the tracking trajectory is greater than the preset trajectory length, it should not be determined as noise. Further, the second output result provided by the point cloud denoising model can also be used to assist in verifying whether the classification result of the current point cloud data is reasonable. For example, if the output of the point cloud denoising model for the point cloud data is an obstacle, but the binary classification model result is noise, the final obstacle recognition result can be obtained by comprehensive analysis. For example, the confidence level can be adjusted by comparing the classification result of the output of the binary classification model with the comparison result of the historical frame model information and the second output result, and finally the obstacle recognition result is output by comprehensively analyzing the confidence level. In addition, different weights are given to the historical frame model information and the second output result, and finally the obstacle recognition result is output by comprehensively analyzing the confidence level and the weight.

[0064] It should be noted that the reason why the embodiment of the present application uses different models (such as the point cloud denoising model and the binary classification model) and then analyzes the final obstacle recognition result by combining multiple data (such as the historical frame model information and the second output result) is to avoid the inaccuracy of a single data. Therefore, by combining multiple data for auxiliary analysis, the output obstacle recognition result can be more accurate, so as to better ensure the safety of the driver during intelligent driving.

[0065] In an embodiment of the present application, the step 102 of determining the emission rule of the laser beam emitted by the point cloud device for collecting the point cloud data divides the point cloud data into point cloud data of a visible region and point cloud data of an invisible region. The point cloud data of the visible region is the point cloud data covered by the laser beam and not blocked by the obstacle, and the point cloud data of the invisible region is the point cloud data other than the point cloud data of the visible region. The point cloud data of the invisible region in the point cloud data is filtered to obtain first filtered point cloud data, specifically:

[0066] In some cases, the front front vehicle (the front vehicle of the front vehicle) of the vehicle (the ego vehicle) is not supposed to be collected by the laser radar to generate point cloud data due to the occlusion of the front vehicle to the line of sight of the ego vehicle. In some cases, if there is a high-reflectivity sign (such as an advertising board) near the vehicle, a target may be formed under the high-reflectivity sign, which generally forms a false target of the front front vehicle of the ego vehicle. The front vehicle is a real vehicle, and the front front vehicle is an invisible area for the ego vehicle because there is a front vehicle in front of the ego vehicle. Therefore, the point cloud data corresponding to the target in the invisible area for the ego vehicle can be filtered out.

[0067] Exemplarily, with reference to Figure 2 A schematic diagram of the emission rule of the laser beam of the laser radar provided in an embodiment of the present application is shown. In a specific implementation, the laser radar (LIDAR) is usually installed on a roof platform with a certain height (H). The point cloud data is the laser beam emitted by the laser radar, which has a certain emission rule. For example, the laser radar emits a laser beam (the laser vector corresponding to the laser beam) and scans a certain field of view angle, which is generally 12 degrees or 360 degrees. The laser beams scanned have a certain longitudinal and lateral angle interval from the same emission point. As the distance increases, the real position difference in the interval area increases. The reachable area scanned by the laser beam that can be emitted in the point cloud data is the visible area, for example Figure 2 The rear area that cannot be scanned by the laser beam emitted in the point cloud data or is occluded by an object (for example Figure 2 The rear area that cannot be scanned by the laser beam emitted in the point cloud data or is occluded by an object (for example

[0068] With reference to Figure 3 A schematic diagram of the visible area and invisible area (blind area) of the point cloud data provided in an embodiment of the present application is shown. As can be seen from Figure 3 The laser beam emitted by the laser radar is blocked by an obstacle. The point cloud data corresponding to the area that cannot be reached by the laser beam is determined as the point cloud data belonging to the invisible area, which can be filtered out.

[0069] In the above embodiment, the emission rule of the laser beam is used to accurately identify and filter the mirror noise of the vehicle caused by the reflection of the vehicle body in the point cloud data, so that intelligent driving can be accurately performed based on the point cloud data filtered out of the noise, and the driving safety of the vehicle is ensured.

[0070] In an embodiment of the present application, the step 103 of inputting the first filtered point cloud data into the bird's eye view perception model to obtain the first output result output by the bird's eye view perception model includes:

[0071] acquire multi-view images collected by a camera device of the vehicle;

[0072] input the point cloud data and the multi-view images into a bird's eye view perception model to obtain a first output result output by the bird's eye view perception model.

[0073] In the embodiment of the present application, the point cloud data can be input into the bird's eye view perception model to obtain the first output result output by the bird's eye view perception model, and the multi-view images collected by the camera device of the vehicle, such as front view, rear view, left view, right view and the like, can be output into the bird's eye view perception model together to obtain the first output result output by the bird's eye view perception model, which can contain more information or be more accurate.

[0074] In the above embodiment, the point cloud data and the multi-view images can be input into the bird's eye view perception model together to obtain the first output result containing more information or being more accurate, so that the obstacle recognition result based on the first output result is more accurate.

[0075] In an embodiment of the present application, after the step 101 of acquiring the point cloud data collected by the point cloud device of the vehicle, the method can further include:

[0076] acquiring vehicle motion data collected by a sensor device of the vehicle; the sensor device at least includes an inertial measurement unit and a global positioning device;

[0077] performing motion distortion compensation on the point cloud data according to the vehicle motion data.

[0078] In a specific implementation, the motion distortion refers to the phenomenon that the point cloud data is distorted, stretched or compressed due to the fact that the ego vehicle is in a motion state (such as acceleration, deceleration or turning) during the process of collecting a frame of point cloud data by the laser radar.

[0079] In the embodiment of the present application, in order to compensate for the motion distortion of the ego vehicle, the embodiment of the present application can acquire the vehicle motion data collected by the sensor device of the vehicle, wherein the sensor device at least includes an inertial measurement unit (Inertial Measurement Unit, IMU) and a global positioning device (Global Positioning System, GPS), and then the point cloud data can be compensated for motion distortion through the vehicle motion data.

[0080] In the above embodiment, the point cloud data can be compensated for motion distortion according to the vehicle motion data collected by the sensor device of the vehicle, so that the point cloud data is accurate in geometry, and all subsequent denoising, obstacle result recognition and the like can be accurately and reliably implemented.

[0081] In conclusion, the embodiment of the present application proposes a multi-level, multi-source feature information fusion, data-driven and rule-driven collaborative noise filtering method for point cloud data, which mainly includes a three-level processing architecture: a hierarchical processing flow from coarse to fine (spatial distribution filtering → multi-source feature information fusion → data-driven optimization), which gradually filters out different types of noise; multi-source feature information fusion: simultaneously using point cloud spatial coordinates, echo intensity, object reflectivity, time continuity and scanning line distribution characteristics of point cloud data to accurately filter noise in point cloud data by integrating multi-source feature information; dual-driven collaborative mechanism: rule-driven (fast filtering based on physical constraints, i.e., filtering the most obvious and typical noise according to the emission law of the laser beam) and data-driven (adaptive parameter adjustment based on machine learning model), which significantly improves the accuracy of noise removal.

[0082] By applying the embodiment of the present application, the following problems can be solved: intensive noise removal in bad weather: diffuse noise points generated in rain and fog weather are effectively filtered out through connected domain segmentation and spatial density analysis; vehicle self-reflection noise identification: mirror noise points generated by vehicle body reflection are accurately identified and removed by using the emission law of the laser beam; dynamic small object misfiltering problem: combining time continuity analysis and same layer point cloud continuity verification, small obstacles such as birds are avoided to be misjudged as noise, so that complex mixed noise scenes can be dealt with, and intelligent driving based on point cloud data can be accurately performed, thereby ensuring the driving safety of the vehicle.

[0083] As shown in Figure 4 The present application discloses a structure diagram of an obstacle recognition device, which can include:

[0084] The point cloud data acquisition module 401 is configured to acquire point cloud data collected by a point cloud device of a vehicle.

[0085] The first point cloud data filtering module 402 is configured to determine an emission law of a laser beam emitted by the point cloud device when collecting the point cloud data, divide the point cloud data into point cloud data of a visible region and point cloud data of an invisible region, the point cloud data of the visible region being point cloud data covered by the laser beam and not blocked by an obstacle, and the point cloud data of the invisible region being point cloud data other than the point cloud data of the visible region; and filter the point cloud data of the invisible region in the point cloud data to obtain first filtered point cloud data.

[0086] The first output result output module 403 is configured to input the first filtered point cloud data into an aerial view perception model to obtain a first output result output by the aerial view perception model, wherein the first output result comprises at least a point cloud segmentation result corresponding to the first filtered point cloud data under different visual angles, obstacle semantic category information and an obstacle model; the point cloud segmentation result comprises a first semantic category, and the first semantic category is used to represent that the first filtered point cloud data is noise or an obstacle; and the obstacle semantic category information is used to represent that the first filtered point cloud data is an obstacle.

[0087] The second point cloud data filtering module 404 is configured to filter noise in the first filtered point cloud data according to the first output result to obtain second filtered point cloud data.

[0088] The second output result output module 405 is configured to input the second filtered point cloud data into a point cloud denoising model to obtain a second output result output by the point cloud denoising model, wherein the second output result comprises second filtered point cloud data obtained by segmenting the second filtered point cloud data according to a set size and a second semantic category corresponding to the segmented second filtered point cloud data; and the second semantic category is used to represent that the second filtered point cloud data is noise or an obstacle.

[0089] The obstacle recognition module is configured to determine an obstacle recognition result corresponding to the point cloud data according to the second filtered point cloud data, the obstacle model and the second output result.

[0090] In an embodiment of the present application, the second output result output module 405 is configured to:

[0091] input the second filtered point cloud data and the obstacle model into a binary classification model to obtain a classification result output by the binary classification model; the classification result comprises the second filtered point cloud data and a third semantic category corresponding to the second filtered point cloud data; and the third semantic category is used to represent that the point cloud data is noise or an obstacle.

[0092] determine the obstacle recognition result corresponding to the point cloud data according to the classification result, historical frame model information and the second output result; and the historical frame model information is information that is continuous in time sequence in the process of recognizing the obstacle.

[0093] In an embodiment of the present application, the second point cloud data filtering module 404 is configured to:

[0094] determine a transmission rule of a laser beam emitted by the point cloud device when collecting the point cloud data, divide the point cloud data into point cloud data of a visible region and point cloud data of an invisible region; the point cloud data of the visible region is point cloud data covered by the laser beam and not blocked by an obstacle, and the point cloud data of the invisible region is point cloud data other than the point cloud data of the visible region;

[0095] filter the point cloud data of the invisible region to obtain first filtered point cloud data.

[0096] In an embodiment of the present application, the point cloud data at least includes point cloud spatial coordinates, echo intensity, object reflectivity and time stamp.

[0097] In an embodiment of the present application, the point cloud segmentation result includes a point cloud segmentation result of the first filtered point cloud data in a bird's eye view and a point cloud segmentation result of the first filtered point cloud data in a distance view; the obstacle semantic category information includes a whitelist obstacle bounding box and a general obstacle, the whitelist obstacle bounding box is used to represent that the first filtered point cloud data is point cloud data of an obstacle and the obstacle has a corresponding obstacle semantic category, and the general obstacle is used to represent that the first filtered point cloud data is point cloud data of an obstacle but the obstacle does not have a corresponding obstacle semantic category.

[0098] In an embodiment of the present application, the first output result output module 403 is configured to:

[0099] acquire multi-view images collected by a camera device of the vehicle;

[0100] input the point cloud data and the multi-view images into a bird's eye view perception model to obtain a first output result output by the bird's eye view perception model.

[0101] In an embodiment of the present application, the device further includes a motion distortion compensation module configured to:

[0102] acquire vehicle motion data collected by a sensor device of the vehicle; the sensor device at least includes an inertial measurement unit and a global positioning device;

[0103] compensate for motion distortion of the point cloud data according to the vehicle motion data.

[0104] In an embodiment of the present application, the binary classification model is based on a machine learning model.

[0105] In this embodiment of the invention, firstly, point cloud data collected by a point cloud device of a vehicle is acquired. Noise in the point cloud data can be filtered according to its spatial distribution to obtain first filtered point cloud data. Specifically, the emission pattern of the laser beam emitted by the point cloud device for collecting the point cloud data is determined, and the point cloud data is divided into visible and invisible areas. Visible point cloud data refers to point cloud data covered by the laser beam and not obscured by obstacles, while invisible point cloud data refers to point cloud data excluding the visible areas. Filtering the invisible point cloud data yields the first filtered point cloud data, achieving coarse filtering of the point cloud data based on physical rules. This quickly removes obvious and typical noise from the point cloud data. Next, the first filtered point cloud data is input into a bird's-eye view perception model to obtain a first output result from the bird's-eye view perception model. The first output result includes at least the first output from different viewing angles. The system comprises a first filtered point cloud data and a corresponding point cloud segmentation result, obstacle semantic category information, and an obstacle model. The point cloud segmentation result includes a first semantic category, which is used to characterize the first filtered point cloud data as noise or an obstacle. The obstacle semantic category information is used to characterize the point cloud data as an obstacle. Based on the first output result, noise in the first filtered point cloud data is filtered to obtain second filtered point cloud data. The second filtered point cloud data is input into a point cloud denoising model to obtain a second output result. The second output result includes the second filtered point cloud data segmented according to a set size and the second semantic category corresponding to the segmented second filtered point cloud data. The second semantic category is used to characterize the second filtered point cloud data as noise or an obstacle. Finally, based on the second filtered point cloud data, the obstacle model, and the second output result, the obstacle recognition result corresponding to the point cloud data can be determined, thereby determining the obstacles around the vehicle based on the obstacle recognition result. The embodiments of the present invention filter point cloud data through various filtering methods to filter out different types of noise in the point cloud data, thereby dealing with complex mixed noise scenarios, and enabling accurate intelligent driving based on point cloud data, thus ensuring vehicle driving safety.

[0106] This invention also provides an electronic device, such as... Figure 5 As shown, it includes a processor 501, a device interface 502, and a memory.

[0107] 503 and bus 504;

[0108] Memory 503 is used to store computer programs;

[0109] The processor 501 performs the above steps when executing the program stored in the memory 503.

[0110] The bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0111] The memory can include a Random Access Memory (RAM) and can also include a non-volatile memory, such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0112] The processor mentioned above can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0113] The application further provides a storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the obstacle recognition method of the foregoing embodiment.

[0114] For the device embodiment, it is basically similar to the method embodiment, so the description is relatively simple, and the relevant part can be referred to the part of the method embodiment.

[0115] The algorithms and displays provided herein are not inherently related to any particular computer, virtual apparatus, or other device. Structures required to construct such devices as would be apparent to those of ordinary skill in the art based on the description given above. Moreover, the application is not specifically directed to any particular programming language. It will be appreciated that a variety of programming languages can be used to implement the present application described herein, and any references below to specific languages are provided for disclosure of the best mode of the application.

[0116] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application can be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

[0117] Similarly, it is to be understood that the embodiments of the present application can be readily combined with one another, and / or various aspects of the individual embodiments can be interchanged between the several embodiments. Similarly, it will be appreciated that, for the sake of conciseness, the disclosures of embodiments of the present application herein do not necessarily disclose all combinations of features described herein, or all combinations of features described in the art. It is therefore not intended that this application be limited in scope only to the dimensions specifically recited herein, but rather, the dimensions of this application are to be construed as broadly as the art will permit.

[0118] Those skilled in the art will appreciate that the modules in the apparatus of the embodiments can be adapted and placed in one or more apparatuses other than the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and further can be divided into more sub-modules or sub-units or sub-components. In addition to the fact that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstract and drawings), and all processes or units of any method or apparatus disclosed thus can be combined in any combination. Unless explicitly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract and drawings) can be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0119] Embodiments of the various components of the application can be implemented in hardware, or as software modules running in one or more processors, or combinations thereof. Those skilled in the art will appreciate that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functionality of some or all of the components in the sequencing apparatus according to the present application. The present application can also be implemented as a program for executing part or all of the methods described herein on a device or apparatus. Such a program can be stored on a computer readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier medium, or in any other form.

[0120] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that one skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a unit claim, several devices can be listed with a comma. The use of the term 'about' followed by a value and / or a term 'approximately' preceding a value means that the value can vary from the stated value by 10%. The use of any of the following terms in the claims is neither meant to limit the scope nor to introduce a non-combination limitation. The terms 'comprise', 'include', and 'contain' are not used in their exclusive sense. The use of the term 'first','second', and 'third' does not connote any order, quantity, creation or importance, but rather are used to denote one element from another. The use of these terms is interchangeable under appropriate circumstances.

[0121] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the devices, apparatuses and units described above can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0122] The above only represents the preferred embodiments of the present application, and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0123] The above only represents the preferred embodiments of the present application, and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0124] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that one skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a unit claim, several devices can be listed with a comma. The use of the term 'about' followed by a value and / or a term 'approximately' preceding a value means that the value can vary from the stated value by 10%. The use of any of the following terms in the claims is neither meant to limit the scope nor to introduce a non-combination limitation. The terms 'comprise', 'include', and 'contain' are not used in their exclusive sense. The use of the term 'first','second', and 'third' does not connote any order, quantity, creation or importance, but rather are used to denote one element from another. The use of these terms is interchangeable under appropriate circumstances.

Claims

1. An obstacle recognition method, characterized in that, The method includes: Acquire point cloud data collected by the vehicle's point cloud device; The emission pattern of the laser beam emitted by the point cloud device for collecting point cloud data is determined, and the point cloud data is divided into visible area point cloud data and invisible area point cloud data. The visible area point cloud data is the point cloud data covered by the laser beam and not obstructed by obstacles, and the invisible area point cloud data is the point cloud data excluding the visible area point cloud data. The invisible area point cloud data is filtered out to obtain the first filtered point cloud data. The first filtered point cloud data is input into the bird's-eye view perception model to obtain a first output result from the bird's-eye view perception model. The first output result includes at least point cloud segmentation results, obstacle semantic category information, and obstacle models corresponding to the first filtered point cloud data under different viewpoints. The point cloud segmentation result includes a first semantic category, which is used to characterize the first filtered point cloud data as noise or an obstacle. The obstacle semantic category information is used to characterize the first filtered point cloud data as an obstacle. Based on the first output result, the noise in the first filtered point cloud data is filtered to obtain the second filtered point cloud data; The second filtered point cloud data is input into the point cloud denoising model to obtain a second output result from the point cloud denoising model. The second output result includes the second filtered point cloud data after the second filtered point cloud data is divided according to a set size and the second semantic category corresponding to the divided second filtered point cloud data. The second semantic category is used to characterize the second filtered point cloud data as noise or obstacles. Based on the second filtered point cloud data, the obstacle model, and the second output result, the obstacle recognition result corresponding to the point cloud data is determined; The step of determining the obstacle recognition result corresponding to the point cloud data based on the second filtered point cloud data, the obstacle model, and the second output result includes: The second filtered point cloud data and the obstacle model are input into a binary classification model to obtain the classification result output by the binary classification model; the classification result includes the second filtered point cloud data and the third semantic category corresponding to the second filtered point cloud data; the third semantic category is used to characterize the point cloud data as noise or an obstacle; Based on the classification results, historical frame model information, and the second output results, the obstacle recognition result corresponding to the point cloud data is determined; the historical frame model information is the information that is continuous in time during the obstacle recognition process; the historical frame model information is obtained by combining the point cloud data with temporal continuity analysis and the continuity verification of the same layer point cloud.

2. The method according to claim 1, characterized in that, The point cloud data includes at least point cloud spatial coordinates, echo intensity, object reflectivity, and timestamp.

3. The method according to claim 1, characterized in that, The point cloud segmentation results include the point cloud segmentation results of the first filtered point cloud data under a bird's-eye view and the point cloud segmentation results of the first filtered point cloud data under a distance view; the obstacle semantic category information includes whitelisted obstacle detection boxes and general obstacles. The whitelisted obstacle detection boxes are used to characterize the first filtered point cloud data as point cloud data that is an obstacle and that the obstacle has a corresponding obstacle semantic category. The general obstacles are used to characterize the first filtered point cloud data as point cloud data that is an obstacle but that the obstacle does not have a corresponding obstacle semantic category.

4. The method according to claim 1, characterized in that, The step of inputting the first filtered point cloud data into the bird's-eye view perception model to obtain the first output result of the bird's-eye view perception model includes: Acquire multi-view images captured by the vehicle's camera equipment; The point cloud data and the multi-view image are input into the bird's-eye view perception model to obtain the first output result of the bird's-eye view perception model.

5. The method according to claim 1, characterized in that, After acquiring the point cloud data collected by the vehicle's point cloud device, the method further includes: The vehicle motion data is collected by the vehicle's sensor devices; the sensor devices include at least an inertial measurement unit and a global positioning system. Motion distortion compensation is performed on the point cloud data based on the vehicle motion data.

6. The method according to claim 1, characterized in that, The binary classification model is based on a machine learning model.

7. An obstacle recognition device, characterized in that, The device includes: The point cloud data acquisition module is used to acquire point cloud data collected by the vehicle's point cloud device. The first point cloud data filtering module is used to determine the emission pattern of the laser beam emitted by the point cloud device when collecting the point cloud data, and to divide the point cloud data into visible area point cloud data and invisible area point cloud data; the visible area point cloud data is the point cloud data covered by the laser beam and not obstructed by obstacles, and the invisible area point cloud data is the point cloud data excluding the visible area point cloud data; filtering the invisible area point cloud data in the point cloud data yields the first filtered point cloud data. The first output result module is used to input the first filtered point cloud data into the bird's-eye view perception model to obtain the first output result output by the bird's-eye view perception model; the first output result includes at least point cloud segmentation results, obstacle semantic category information and obstacle models corresponding to the first filtered point cloud data under different viewpoints; the point cloud segmentation result includes a first semantic category, the first semantic category is used to characterize the first filtered point cloud data as noise or obstacle, and the obstacle semantic category information is used to characterize the first filtered point cloud data as obstacle. The second point cloud data filtering module is used to filter noise in the first filtered point cloud data according to the first output result to obtain the second filtered point cloud data. The second output result module is used to input the second filtered point cloud data into a point cloud denoising model to obtain a second output result from the point cloud denoising model. The second output result includes second filtered point cloud data segmented according to a set size and a second semantic category corresponding to the segmented second filtered point cloud data. The second semantic category is used to characterize the second filtered point cloud data as noise or an obstacle. Determining the obstacle recognition result corresponding to the point cloud data based on the second filtered point cloud data, the obstacle model, and the second output result includes: inputting the second filtered point cloud data and the obstacle model into a binary classification model to obtain a classification result output by the binary classification model. The classification result includes the second filtered point cloud data and a third semantic category corresponding to the second filtered point cloud data. The third semantic category is used to characterize the point cloud data as noise or an obstacle. Determining the obstacle recognition result corresponding to the point cloud data based on the classification result, historical frame model information, and the second output result. The historical frame model information is information that is temporally continuous during the obstacle recognition process. The historical frame model information is obtained by combining temporal continuity analysis and same-layer point cloud continuity verification of the point cloud data. An obstacle recognition module is used to determine the obstacle recognition result corresponding to the point cloud data based on the second filtered point cloud data, the obstacle model, and the second output result.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the instructions to implement the obstacle recognition method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the mobile terminal, the mobile terminal is able to perform the obstacle recognition method as described in any one of claims 1 to 6.

10. A vehicle, characterized in that, The vehicle includes the electronic equipment as described in claim 8.

Images