Obstacle identification method and device and computer equipment

Abstract

The invention relates to an obstacle identification method and device and computer equipment. The method comprises the following steps: detecting point cloud data of a target scene based on a preset target detection integration model to obtain target sensing information of each obstacle; based on target sensing information, determining an entity uncertainty risk level of the corresponding obstacle, the target sensing information at least comprising first spatial position information of each obstacle; processing the point cloud data based on a preset rule algorithm to obtain a point cloud cluster of each obstacle; determining the obstacle type of the corresponding obstacle through the spatial geometric features of the point cloud cluster, and determining the second spatial position information of the corresponding obstacle based on the spatial position information of the point cloud cluster; and matching the first spatial position information with the second spatial position information to obtain a matching result, and determining an obstacle recognition result based on the matching result, the entity uncertainty risk level and the obstacle type. By adopting the method, the obstacle type identification precision can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an obstacle recognition method, device, and computer equipment. Background Art

[0002] The development of autonomous driving technology in unmanned mining areas can significantly improve transportation and loading efficiency, reduce commuting and labor costs, and achieve a high degree of automation in mine vehicle operations. Due to the harsh environment of open-pit mines and poor lighting conditions at night, lidar is more suitable for perception tasks than other sensors such as cameras. However, it still struggles to completely overcome the interference caused by severe dust.

[0003] In typical autonomous driving systems, the perception layer typically uses object detection technology to obtain semantic information about objects around the vehicle, while the decision-making layer relies entirely on this information. However, when the perception layer fails, this reliance can seriously impact system safety. To ensure efficient production, the system should avoid frequently misidentifying dust as physical objects, which could cause unmanned mining vehicles to brake unnecessarily. Furthermore, to ensure safety, the system should ensure that physical obstacles are not missed or misidentified as dust.

[0004] Related technologies use target detection technology to identify dust. However, in the complex and harsh working environment of mining areas, the frequent and diverse changes in dust form and density limit the stability of various target detection algorithms used in related technologies. As a result, the accuracy of obstacle recognition in related technologies is low. Summary of the Invention

[0005] Based on this, it is necessary to provide an obstacle identification method, device and computer equipment that can improve the identification accuracy of obstacle types in order to address the above technical problems.

[0006] In a first aspect, the present application provides a method for identifying an obstacle, comprising:

[0007] detecting point cloud data of a target scene based on a preset target detection integrated model to obtain target perception information of each obstacle in the target scene; and determining a physical uncertainty risk level of the corresponding obstacle based on the target perception information, wherein the target perception information includes at least first spatial position information of each obstacle;

[0008] Processing the point cloud data based on a preset rule algorithm to obtain a point cloud cluster for each obstacle; determining the obstacle type corresponding to the obstacle based on spatial geometric features of the point cloud cluster, and determining second spatial position information corresponding to the obstacle based on the spatial position information of the point cloud cluster;

[0009] Match the first spatial position information and the second spatial position information to obtain a matching result, and determine an obstacle recognition result based on the matching result, the entity uncertainty risk level, and the obstacle type.

[0010] In one embodiment, the determining the entity uncertainty risk level of the corresponding obstacle based on the target perception information includes:

[0011] Determine an entity uncertainty value based on a preset depth integration algorithm and the target perception information;

[0012] Compare the entity uncertainty value with a preset uncertainty boundary threshold to obtain a first comparison result, and determine the entity uncertainty risk level of the obstacle corresponding to the target perception information based on the first comparison result.

[0013] In one embodiment, the target perception information includes multiple initial perception information of the corresponding obstacle, and each initial perception information includes multiple attribute information of the obstacle. The attribute information at least includes the classification score of the obstacle. The determining the entity uncertainty value based on the preset depth integration algorithm and the target perception information includes:

[0014] Based on the number of initial perception information in the target perception information corresponding to the obstacle and each attribute information, determine the mean value of each attribute information; based on the multiple attribute information and the mean value corresponding to each attribute information, determine the variance value of the attribute information, and determine the variance value as the spatial information uncertainty of the obstacle;

[0015] Based on the mean value corresponding to the classification score, determine the Shannon entropy of the classification score, and correct the Shannon entropy based on the number of initial perception information in the target perception information corresponding to the obstacle and the number of preset target detection integration models to obtain the semantic uncertainty; determine the semantic uncertainty and the spatial information uncertainty as the entity uncertainty value of the obstacle.

[0016] In one embodiment, the determining the obstacle type of the corresponding obstacle through the spatial geometric features of the point cloud cluster includes:

[0017] Based on a preset entity degree algorithm, the spatial geometric features of the point cloud cluster, and multiple prior spatial geometric features in a preset obstacle feature library, determine the entity degree value of the obstacle corresponding to the point cloud cluster;

[0018] Compare the entity degree value with a preset entity degree boundary threshold to obtain a second comparison result, and determine the obstacle type of the obstacle corresponding to the point cloud cluster based on the second comparison result.

[0019] In one embodiment, determining the entity degree value of the obstacle corresponding to the point cloud cluster based on the preset entity degree algorithm, the spatial geometric features of the point cloud cluster, and multiple prior spatial geometric features in the preset obstacle feature library includes:

[0020] Calculating the covariance matrix of the spatial geometric features and multiple prior spatial geometric features in the preset obstacle feature library;

[0021] Based on the preset entity degree algorithm, processing the spatial geometric features, the multiple prior spatial geometric features, and the covariance matrix to obtain an initial similarity probability value;

[0022] Based on the preset normalization correction algorithm, correcting the initial similarity probability value to obtain the entity degree value of the obstacle, where the entity degree value represents the similarity probability between the point cloud cluster corresponding to the obstacle and the preset obstacle feature library.

[0023] In one embodiment, the method further includes:

[0024] Detecting the sample point cloud data set based on the preset target detection integration model to obtain sample perception information;

[0025] For each sample obstacle, detecting the sample perception information based on the preset perception conditions to obtain a correctness detection result; determining a sample entity uncertainty value based on the preset depth integration algorithm and the sample perception information; comparing the sample entity uncertainty with multiple sample uncertainty boundary thresholds respectively to determine multiple uncertainty detection results of the sample obstacle;

[0026] Based on the correctness detection result and multiple uncertainty detection results, respectively determining the evaluation scores corresponding to the respective sample uncertainty boundary thresholds;

[0027] Determining the sample uncertainty boundary threshold with the highest evaluation score as the preset uncertainty boundary threshold.

[0028] In one embodiment, the correctness detection result includes a correct detection result and an incorrect detection result, and the uncertainty detection result includes an uncertain detection result and a definite detection result. Determining the evaluation scores corresponding to the respective sample uncertainty boundary thresholds based on the correctness detection result and multiple uncertainty detection results includes:

[0029] For each of the uncertain detection results, determine the first quantity of the sample obstacles that meet the correct detection result and the sample obstacles corresponding to the determined detection result; determine the second quantity of the sample obstacles that meet the correct detection result and the sample obstacles corresponding to the uncertain detection result; determine the third quantity of the sample obstacles that meet the wrong detection result and the sample obstacles corresponding to the determined detection result; determine the fourth quantity of the sample obstacles that meet the wrong detection result and the sample obstacles corresponding to the uncertain detection result;

[0030] Determine the ratio of the fourth quantity to the sum value of the fourth quantity and the third quantity as the coverage rate; determine the ratio of the second quantity to the sum value of the fourth quantity and the fourth quantity as the false alarm rate; determine the first ratio of the second quantity to the sum value of the second quantity and the first quantity, and determine the ratio of the coverage rate to the first ratio as the quality score;

[0031] Based on the preset weight distribution coefficient, the coverage rate, the false alarm rate, and the quality score, determine the evaluation score corresponding to the sample uncertainty threshold.

[0032] In one embodiment, the method further includes:

[0033] Extract the sample space geometric features of the sample point cloud clusters corresponding to different sample obstacles in the preset obstacle feature library based on the preset entity degree algorithm; classify the sample point cloud clusters through the preset rule algorithm to obtain the bounding box information and sample entity degree values corresponding to each cluster of sample point clouds;

[0034] Compare the bounding box information with the preset bounding box information to obtain the spatial intersection over union ratio, and determine the success detection result based on the comparison result of the spatial intersection over union ratio and the preset spatial matching threshold; determine multiple entity degree detection results based on the comparison result of the sample entity degree value and multiple sample entity degree thresholds;

[0035] Based on the success detection result and multiple entity degree detection results, respectively determine the effective detection rates of each sample entity degree threshold;

[0036] Determine the sample entity degree threshold with the highest effective detection rate as the preset entity degree threshold.

[0037] In one embodiment, the success detection result includes a successful detection result and a wrong detection result, the entity degree detection result includes a high entity degree detection result and a low entity degree detection result, and the step of respectively determining the effective detection rates of each sample entity degree threshold based on the success detection result and multiple entity degree detection results includes:

[0038] For each entity degree detection result, determine the fifth quantity of the sample obstacles that satisfy the presence of the sample obstacles in the point cloud cluster and satisfy the successful detection result and the sample obstacles corresponding to the low entity degree detection result; determine the sixth quantity of the sample obstacles that satisfy the presence of the sample obstacles in the point cloud cluster and satisfy the successful detection result and the sample obstacles corresponding to the high entity degree detection result; determine the seventh quantity of the sample obstacles that satisfy the absence of the sample obstacles in the point cloud cluster and satisfy the error detection result and the low entity degree detection result; determine the eighth quantity of the sample obstacles that satisfy the absence of the sample obstacles in the point cloud cluster and satisfy the error detection result and the high entity degree detection result;

[0039] Determine the effective detection rate of the entity by taking the ratio of the sixth quantity to the sum of the fifth quantity and the sixth quantity, and determine the effective detection rate of the non-entity by taking the ratio of the seventh quantity to the sum of the seventh quantity and the eighth quantity;

[0040] Based on the sum of the effective detection rate of the entity and the effective detection rate of the non-entity, determine the effective detection rate of the sample entity degree demarcation threshold.

[0041] In one embodiment, the determining the obstacle recognition result based on the matching result, the entity uncertainty risk level, and the obstacle type includes:

[0042] If the matching result is that the first spatial position information and the second spatial position information exceed a preset spatial matching threshold, determine that the same type of obstacle is detected, and determine the initial recognition result of the obstacle based on the entity uncertainty risk level and the obstacle type;

[0043] If the matching result is that the first spatial position information and the second spatial position information are lower than the preset spatial matching threshold, determine that different types of obstacles are detected, then determine the initial recognition result of the obstacle corresponding to the first spatial position information based on the risk level, and determine the initial recognition result of the obstacle corresponding to the second spatial position information based on the obstacle type;

[0044] Determine the obstacle recognition result by taking the initial recognition result that satisfies the preset result screening condition, where the preset result screening condition is used to retain the initial recognition result that represents the entity obstacle or the initial recognition result of the obstacle corresponding to the first spatial position information.

[0045] In a second aspect, the present application also provides an obstacle recognition device, including:

[0046] A detection module, configured to detect the point cloud data of a target scene based on a preset target detection integrated model, so as to obtain the perception information of each obstacle in the target scene; and determine the entity uncertainty risk level of the corresponding obstacle based on the perception information, where the perception information at least includes the first spatial position information of each obstacle.

[0047] A determination module, configured to process the point cloud data based on a preset rule algorithm to obtain a point cloud cluster of each obstacle; determine the obstacle type of the corresponding obstacle through the spatial geometric features of the point cloud cluster, and determine the second spatial position information of the corresponding obstacle based on the spatial position information of the point cloud cluster.

[0048] An identification module, configured to match the first spatial position information and the second spatial position information to obtain a matching result, and determine an obstacle identification result based on the matching result, the entity uncertainty risk level, and the obstacle type.

[0049] In a third aspect, the present application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0050] Detect the point cloud data of a target scene based on a preset target detection integrated model, so as to obtain the target perception information of each obstacle in the target scene; and determine the entity uncertainty risk level of the corresponding obstacle based on the target perception information, where the target perception information at least includes the first spatial position information of each obstacle.

[0051] Process the point cloud data based on a preset rule algorithm to obtain a point cloud cluster of each obstacle; determine the obstacle type of the corresponding obstacle through the spatial geometric features of the point cloud cluster, and determine the second spatial position information of the corresponding obstacle based on the spatial position information of the point cloud cluster.

[0052] Match the first spatial position information and the second spatial position information to obtain a matching result, and determine an obstacle identification result based on the matching result, the entity uncertainty risk level, and the obstacle type.

[0053] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0054] Detect the point cloud data of the target scene based on a preset target detection integration model to obtain the target perception information of each obstacle in the target scene; and determine the entity uncertainty risk level of the corresponding obstacle based on the target perception information, where the target perception information at least includes the first spatial position information of each obstacle.

[0055] Process the point cloud data based on a preset rule algorithm to obtain the point cloud cluster of each obstacle; determine the obstacle type of the corresponding obstacle through the spatial geometric features of the point cloud cluster, and determine the second spatial position information of the corresponding obstacle based on the spatial position information of the point cloud cluster.

[0056] Match the first spatial position information and the second spatial position information to obtain a matching result, and determine the obstacle recognition result based on the matching result, the entity uncertainty risk level, and the obstacle type.

[0057] In a fifth aspect, the present application also provides a computer program product, including a computer program, which when executed by a processor implements the following steps:

[0058] Detect the point cloud data of the target scene based on a preset target detection integration model to obtain the target perception information of each obstacle in the target scene; and determine the entity uncertainty risk level of the corresponding obstacle based on the target perception information, where the target perception information at least includes the first spatial position information of each obstacle.

[0059] Process the point cloud data based on a preset rule algorithm to obtain the point cloud cluster of each obstacle; determine the obstacle type of the corresponding obstacle through the spatial geometric features of the point cloud cluster, and determine the second spatial position information of the corresponding obstacle based on the spatial position information of the point cloud cluster.

[0060] Match the first spatial position information and the second spatial position information to obtain a matching result, and determine the obstacle recognition result based on the matching result, the entity uncertainty risk level, and the obstacle type.

[0061] The above-mentioned obstacle recognition method, device and computer equipment detect point cloud data through a preset target detection integrated model to obtain target perception information, and determine the entity uncertainty risk level of the corresponding obstacle based on the target perception information; through a preset rule algorithm, obtain the point cloud cluster of the obstacle and determine the obstacle type of the obstacle; through the fusion method of the matching result, entity uncertainty risk level and obstacle type, obtain the final obstacle recognition result of the obstacle, which can reduce the interference of non-entity obstacles, more accurately identify entity obstacles and non-entity obstacles, ensure automated work in a dusty environment, and improve the transportation efficiency and safety of unmanned mining vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for the description of the embodiments of the present application or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0063] Figure 1 It is a schematic flowchart of an obstacle recognition method in an embodiment;

[0064] Figure 2 It is a process framework diagram of an obstacle recognition method in an embodiment;

[0065] Figure 3 It is a process framework diagram of an obstacle recognition method in an embodiment;

[0066] Figure 4 It is a process framework diagram of an obstacle recognition method in an embodiment;

[0067] Figure 5 It is a process framework diagram of an obstacle recognition method in an embodiment;

[0068] Figure 6 It is a process framework diagram of an obstacle recognition method in an embodiment;

[0069] Figure 7 It is a process framework diagram of an obstacle recognition method in an embodiment;

[0070] Figure 8 It is a structural block diagram of an obstacle recognition device in an embodiment;

[0071] Figure 9 It is an internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0072] To make the objectives, technical solutions, and advantages of this application more clear and understandable, the following further elaborates on this application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely used to explain this application and are not used to limit this application.

[0073] In an exemplary embodiment, as Figure 1 shown, a method for identifying obstacles is provided. In this embodiment, this method is exemplified by being applied to a terminal. It can be understood that this method can also be applied to a server and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0074] Step 101: Detect the point cloud data of the target scene based on a preset target detection integrated model to obtain the target perception information of each obstacle in the target scene; and based on the target perception information, determine the entity uncertainty risk level of the corresponding obstacle.

[0075] Among them, the preset target detection integrated model may include multiple preset target detection models, and the target scene may be an open-pit mining area, etc. The target perception information at least includes the first spatial position information, semantic category, and entity number information of each obstacle. The semantic category may be the category name of the obstacle. For example, mining truck 1, mining truck 2, car 1, and pedestrian 1, etc. The entity number information may reflect the number of entities of the detected obstacle. The entity uncertainty risk level may include high-uncertainty entities and low-uncertainty entities.

[0076] Specifically, the terminal can obtain the point cloud data of the target scene collected by the vehicle-mounted lidar sensor in real time or within a specified period. For each preset target detection model, the terminal can perform target detection on the same frame of point cloud data based on each preset target detection model to obtain the initial perception information of each obstacle. The terminal can integrate the initial perception information of the same obstacle to obtain multiple perception information of each obstacle in each preset target detection model, and determine the multiple perception information of each obstacle in each preset target detection model as the target perception information of each obstacle. The terminal can determine the entity uncertainty value of each obstacle based on the preset depth integration algorithm and the target perception information. The terminal can compare the entity uncertainty values of each obstacle with a preset uncertainty boundary threshold to obtain a comparison result, and determine the comparison result as the entity uncertainty risk level.

[0077] Step 102: Process the point cloud data based on a preset rule algorithm to obtain a point cloud cluster for each obstacle; determine the obstacle type of the corresponding obstacle through the spatial geometric features of the point cloud cluster, and determine the second spatial position information of the corresponding obstacle based on the spatial position information of the point cloud cluster.

[0078] Among them, the preset rule algorithm may include point cloud filtering, plane segmentation, point cloud clustering, and so on. The preset rule algorithm may be a rule clustering algorithm, and the spatial geometric features may include the shape features, size features, orientation features, and position features of the point cloud clusters, and so on. The obstacle types may include entities or non-entities. The spatial position information may be the coordinate position information of the point cloud clusters, and so on.

[0079] Specifically, the terminal may process the point cloud data of the same frame based on the preset rule algorithm to obtain the point cloud clusters of each obstacle. The terminal may extract the spatial geometric features of the point cloud clusters of each obstacle through an Ensemble of Shape Functions (ESF). The terminal may calculate the average reflection intensity of all points in each point cloud cluster based on the spatial geometric features of the point cloud cluster, and determine the entity degree value of the obstacle corresponding to the point cloud cluster based on the average reflection intensity and the preset prior spatial geometric features, and determine the obstacle type of the obstacle based on the entity degree value.

[0080] In one example, the specific steps for the terminal to obtain the point cloud clusters of each obstacle may include: The terminal may use a point cloud filtering algorithm to filter the initial point cloud data to obtain the filtered point cloud data. The terminal may remove the ground points in the filtered point cloud data through a plane segmentation algorithm to obtain the plane-processed point cloud data. The terminal may perform point cloud clustering on the plane-processed point cloud data based on a point cloud clustering algorithm to obtain the point cloud clusters of each obstacle. Exemplarily, the point cloud algorithm may be implemented through a voxel filter, the plane segmentation algorithm may be the Random Sample Consensus (RANSAC), and the point cloud clustering algorithm may be the Density-Based Spatial Clustering of Applications with Noise (DBSCAN).

[0081] The terminal may obtain the spatial position information of the point cloud cluster, project all the points in the point cloud cluster onto a two-dimensional plane to obtain two-dimensional position information, use the Graham algorithm to perform convex hull fitting on the two-dimensional position information corresponding to all the points to obtain the convex hull of the current point cloud cluster, determine the rectangle bounding box with the minimum area on the convex hull based on the rotating calipers algorithm, and determine the second spatial position information of the obstacle as the position information of the rectangle bounding box. The position information of the rectangle bounding box may include the position, size, and orientation of the rectangle bounding box, and so on.

[0082] Step 103: Match the first spatial position information and the second spatial position information to obtain a matching result, and determine the obstacle recognition result based on the matching result, the entity uncertainty risk level, and the obstacle type.

[0083] Among them, the obstacle recognition result is the entity type.

[0084] Specifically, the terminal can calculate the spatial intersection over union (IoU) of two obstacles based on the first spatial position information and the second spatial position information. If the spatial IoU exceeds the preset matching threshold, the matching result is that the two obstacles are matched, that is, the obstacle corresponding to the first spatial position information and the obstacle corresponding to the second spatial position information are the same obstacle. If the spatial IoU does not exceed the preset matching threshold, the matching result is an unmatched result, that is, the obstacle is detected only in one way. The terminal can obtain the initial recognition result of the obstacle based on the matching result, the entity uncertainty risk level, and the type of the obstacle, and only retain the results classified as entities and the low-uncertainty results detected only by step 101.

[0085] The above-mentioned obstacle recognition method detects the point cloud data through a preset target detection integration model to obtain target perception information, and determines the entity uncertainty risk level of the corresponding obstacle based on the target perception information; through a preset rule algorithm, obtains the point cloud cluster of the obstacle and determines the obstacle type of the obstacle; through the fusion method of the matching result, the entity uncertainty risk level, and the obstacle type, obtains the final obstacle recognition result of the obstacle, which can reduce the interference of non-entity obstacles, more accurately identify entity obstacles and non-entity obstacles, ensure the automated work in a multi-dust environment, and improve the transportation efficiency and safety of unmanned mining vehicles.

[0086] In an exemplary embodiment, the specific implementation process of step 101, "determine the entity uncertainty risk level of the corresponding obstacle based on the target perception information", may include:

[0087] Determine the entity uncertainty value based on a preset depth integration algorithm and the target perception information; compare the entity uncertainty value with a preset uncertainty boundary threshold to obtain a first comparison result, and based on the first comparison result, determine the entity uncertainty risk level of the obstacle corresponding to the target perception information.

[0088] Among them, the target perception information can reflect the spatial position and type of each obstacle. The target perception information may include the spatial position information and the classification score of the type corresponding to the obstacle. The obstacle is an entity obstacle.

[0089] Specifically, the terminal can calculate the mean of the target perception information based on the target perception information corresponding to each obstacle. The terminal can calculate the variance value of the target perception information based on the target perception information and the mean, and determine the variance value as the spatial information uncertainty. The terminal can calculate the mean of the classification scores of each type of obstacle and calculate the Shannon entropy based on the mean. The terminal determines the semantic uncertainty based on the Shannon entropy, and determines the entity uncertainty based on the spatial information uncertainty and the semantic uncertainty. The terminal can compare the entity uncertainty with a preset uncertainty boundary threshold to obtain a first comparison result. If the first comparison result is that the entity uncertainty is greater than the preset uncertainty boundary threshold, it is determined that the entity uncertainty risk level of the obstacle corresponding to the target perception information is a high-uncertainty entity. If the first comparison result is that the entity uncertainty is not greater than the preset uncertainty boundary threshold, it is determined that the entity uncertainty risk level of the obstacle corresponding to the target perception information is a low-uncertainty entity.

[0090] In this embodiment, the uncertainty value of the obstacle is calculated by a preset deep integration algorithm, the uncertainty corresponding to each obstacle is determined, and the risk level of the obstacle is divided based on a preset uncertainty boundary threshold, so as to determine that a high-uncertainty entity obstacle may actually be a non-entity obstacle such as dust, and the type of the obstacle can be determined more accurately.

[0091] In an exemplary embodiment, the specific implementation process of the step "determine the entity uncertainty value based on the preset deep integration algorithm and the target perception information" may include:

[0092] Based on the number of initial perception information in the target perception information corresponding to the obstacle and each attribute information, determine the mean of each attribute information; based on multiple attribute information and the mean corresponding to each attribute information, determine the variance value of the attribute information, and determine the variance value as the spatial information uncertainty of the obstacle; based on the mean corresponding to the classification score, determine the Shannon entropy of the classification score, and correct the Shannon entropy based on the number of initial perception information in the target perception information corresponding to the obstacle and the number of preset target detection integration models to obtain the semantic uncertainty; determine the semantic uncertainty and the spatial information uncertainty as the entity uncertainty value of the obstacle.

[0093] Among them, the target perception information includes multiple initial perception information corresponding to the obstacle, each initial perception information includes multiple attribute information of the obstacle, the attribute information includes the spatial position information of the corresponding obstacle and the classification score of the type, the spatial position information may include the center point coordinates of the obstacle, the size of the obstacle and the orientation information of the obstacle. The number of initial perception information in the target perception information may be the number of initial perception information of the target obstacle detected by multiple target detection models.

[0094] Specifically, the terminal can calculate the sum value of the attribute information of the target perception information of the obstacle, and determine the mean value of the attribute information corresponding to the obstacle based on the ratio of the sum value to the number of initial perception information in the target perception information corresponding to the obstacle. The specific formula can be:

[0095]

[0096] property∈{location,dimention,heading,p}

[0097] where, mean property can be the mean value corresponding to the attribute information, T is the number of initial perception information in the target perception information corresponding to the obstacle, satisfying T≤N, N is the number of preset target detection models, location can be the center point coordinates of the obstacle, dimention can be the size of the obstacle, heading can be the orientation information of the obstacle, and p can be the classification score of the type of the obstacle.

[0098] The terminal can calculate the variance value based on the attribute information and the corresponding mean value, and determine the variance value as the spatial information uncertainty of the obstacle. The specific calculation formula of the variance value is:

[0099]

[0100] The terminal can calculate the mean value of the classification scores corresponding to each type of obstacle. The terminal can determine the calculation method of the Shannon entropy of the obstacle based on whether the classification confidence of the mean value of the classification scores of each type of obstacle is independent or whether it has been normalized. Exemplarily, the types of obstacles include trucks, cars or people. The specific calculation formula for calculating the mean value of the classification scores corresponding to each type of obstacle is:

[0101] p c =mean p ,c∈{truck,car,person}

[0102] where, p c is the mean value of the classification scores corresponding to each type of obstacle, mean p is the mean value of the classification scores corresponding to each type of obstacle. c is the type of the obstacle.

[0103] As the semantic uncertainty, the Shannon entropy entropy can be specifically expressed by the formula:

[0104]

[0105] In addition, during the parallel execution of each preset target detection model, abnormal situations may occur. The abnormal situations may include missed detections or false detections. The terminal may determine the number of abnormal situations based on the number of initial perception information in the target perception information corresponding to the obstacle and the number of preset target detection models, correct the Shannon entropy based on the number of abnormal situations and a preset penalty factor to obtain the corrected Shannon entropy, and determine the corrected Shannon entropy as the semantic uncertainty.

[0106] The specific formula for the corrected Shannon entropy may be:

[0107] uncertainty=[1+f p (N-T)]entropy

[0108] where, uncertainty may be the semantic uncertainty, and f p is the penalty factor set for missed detections and false detection results in target detection.

[0109] It can be seen from this that when N = T, all preset target detection models detect the obstacle; when N > T, most target detection models detect the obstacle, and a small number of target detection models have missed detections; when N >> T, it means that most target detection models have missed detections, or only a small number of target detection models detect the entity, indicating that the entity may be a false detection.

[0110] The terminal may determine the semantic uncertainty and the spatial information uncertainty as the entity uncertainty value of the obstacle.

[0111] In this embodiment, by considering that missed detections or false detections may occur during the detection process of the preset target detection model, correcting the Shannon entropy corresponding to the obstacle, determining the semantic uncertainty, and comprehensively determining the entity uncertainty of the obstacle by combining the semantic uncertainty and the spatial uncertainty, the accuracy of detecting the entity uncertainty of the obstacle is improved. At the same time, multiple preset target detection models can run in parallel with multiple threads to meet the entity operation requirements while ensuring the recognition effect.

[0112] In an exemplary embodiment, the specific implementation process of step 102, "determine the obstacle type corresponding to the obstacle through the spatial geometric features of the point cloud cluster", may include:

[0113] Based on a preset entity degree algorithm, the spatial geometric features of the point cloud cluster, and multiple types of prior spatial geometric features in the preset obstacle feature library, determine the entity degree value of the obstacle corresponding to the point cloud cluster; compare the entity degree value with a preset entity degree threshold to obtain a second comparison result, and determine the obstacle type of the obstacle corresponding to the point cloud cluster based on the second comparison result.

[0114] Among them, the type of obstacle can be a physical obstacle or a non-physical obstacle. The preset obstacle feature library can include multiple known types of physical obstacles and the corresponding prior spatial geometric features of each type of obstacle. The obstacle types can include physical obstacles and non-physical obstacles. For example, physical obstacles can include trucks, cars, pedestrians, etc., and non-physical obstacles can be dust.

[0115] Specifically, the terminal can extract the spatial geometric features of the point cloud cluster of the obstacle based on the ESF feature descriptor. The terminal can calculate the covariance matrix of the spatial geometric features and multiple types of prior spatial geometric features in the preset obstacle feature library; based on the preset entity degree algorithm, process the spatial geometric features, multiple types of prior spatial geometric features, and the covariance matrix to obtain an initial similarity probability value; based on the preset normalization correction algorithm, correct the initial similarity probability value to obtain the entity degree value of the obstacle.

[0116] The terminal can compare the entity degree value with the preset entity degree threshold to obtain a second comparison result, and determine the obstacle type of the point cloud cluster corresponding to the obstacle based on the second comparison result. If the second comparison result is that the entity degree value is greater than the preset entity degree threshold, the terminal determines that the obstacle type of the point cloud cluster corresponding to the obstacle is physical; if the second comparison result is that the entity degree value is less than or equal to the preset entity degree threshold, the terminal determines that the obstacle type of the point cloud cluster corresponding to the obstacle is non-physical.

[0117] In this embodiment, by using the spatial feature descriptor and the point cloud reflection intensity feature to distinguish physical obstacles from dust, the feature descriptor can statistically analyze multiple overall and local spatial features of the point cloud cluster, and the reflection intensity can reflect the point cloud density feature, which can well adapt to dust with different distances, different shapes, and different densities, improving the robustness of the algorithm. Combining with the deep learning model can achieve more accurate identification.

[0118] In an exemplary embodiment, the specific implementation process of the step "determine the entity degree value of the obstacle corresponding to the point cloud cluster based on the preset entity degree algorithm, the spatial geometric features of the point cloud cluster, and multiple types of prior spatial geometric features in the preset obstacle feature library" can include:

[0119] Calculate the covariance matrix of the spatial geometric features and multiple types of prior spatial geometric features in the preset obstacle feature library; based on the preset entity degree algorithm, process the spatial geometric features, multiple types of prior spatial geometric features, and the covariance matrix to obtain an initial similarity probability value; based on the preset normalization correction algorithm, correct the initial similarity probability value to obtain the entity degree value of the obstacle.

[0120] Among them, the entity degree value can represent the similarity probability of the point cloud cluster corresponding to the obstacle and the preset obstacle feature library.

[0121] The terminal can calculate the spatial geometric features of the obstacle and the covariance matrix of each type of prior geometric feature. Taking the covariance matrix, the spatial geometric features of the obstacle, and the k-th type of prior spatial geometric features as inputs, it calculates the Mahalanobis distance between the spatial geometric features of the obstacle point cloud cluster and the four types of prior spatial geometric features through a preset entity degree algorithm, and converts the Mahalanobis distance into an initial similarity probability value P k . The specific calculation formula of the preset entity degree algorithm is as follows:

[0122]

[0123] where P k is the similarity probability value between the obstacle and the preset obstacle library, x is the spatial geometric feature of the current obstacle point cloud cluster, is the k-th type of prior spatial geometric feature, and S is the covariance matrix between x and .

[0124] The terminal can obtain the reflection intensity corresponding to all points in the point cloud cluster and calculate the average reflection intensity of all points in the point cloud cluster. The reflection intensity reflects the point cloud density feature. The specific formula for calculating the average reflection intensity is:

[0125]

[0126] where is the average reflection intensity, C is the number of points in the point cloud cluster, and I c is the reflection intensity corresponding to each point.

[0127] The terminal can correct the initial similarity probability value based on the average reflection intensity to obtain the similarity probability value. To simplify the parameters and achieve a better classification effect, only the similarity probability value of non-entity (dust) is corrected, and the similarity probability values of other categories remain unchanged, obtaining the similarity probability values of the current point cloud cluster with multiple categories in the preset obstacle feature library. The specific formula for correcting the initial similarity probability value is:

[0128]

[0129] where is the similarity probability value, I0 is the reflection intensity threshold compensation amount, I thre is the reflection intensity threshold obtained through spatial feature-intensity distribution analysis, is the average reflection intensity of the points in the current point cloud cluster, α and β are correction factors used to balance the magnitude gap between the reflection intensity and the similarity probability value, and K is the total number of categories, K = 4.

[0130] The terminal can determine the entity degree value of the obstacle corresponding to the point cloud cluster based on the similarity probability value between the current point cloud cluster and the k-th category and the similarity probability value between the current point cloud cluster and non-entity. The specific calculation formula for the entity degree value E can be:

[0131]

[0132] where K is the total number of categories. Since the categories include four types: autonomous mining trucks, cars, pedestrians, and dust, so K = 4, P k is the similarity probability value between the current point cloud cluster and the k-th category, and P4 is the similarity probability value between the current point cloud cluster and non-entity (dust).

[0133] In this embodiment, by using the spatial geometric features and the reflection intensity of the point cloud cluster to determine the entity degree value of the obstacle, it can well adapt to dust with different distances, different shapes, and different densities, improve the robustness of the algorithm, and can achieve more accurate identification in combination with the deep learning model.

[0134] In an exemplary embodiment, the obstacle recognition method further includes:

[0135] Detecting the sample point cloud data set based on a preset target detection integration model to obtain sample perception information; for each sample obstacle, detecting the sample perception information based on preset perception conditions to obtain a correctness detection result; determining a sample entity uncertainty value based on a preset depth integration algorithm and the sample perception information; comparing the sample entity uncertainty with multiple sample uncertainty threshold values respectively to determine multiple uncertainty detection results of the sample obstacle; determining the evaluation scores corresponding to the respective sample uncertainty threshold values based on the correctness detection result and the multiple uncertainty detection results; and determining the sample uncertainty threshold value with the highest evaluation score as the preset uncertainty threshold value.

[0136] Among them, the preset target detection integration model can include multiple preset target detection models. The sample point cloud data set can include the spatial position information and reflection intensity information, etc. of the obstacles in the target scene. The sample perception information can represent the spatial position information and semantic category of the sample point cloud data. The preset perception conditions can reflect the true spatial position information and true semantic category of the sample point cloud data.

[0137] Specifically, the terminal can use the sample data in the sample point cloud data set for testing and perform parallel operations on the perception information of each preset target detection integration model. Use the mean as the output of the integration, use the variance as the uncertainty of spatial information such as position, size, and direction, and use the Shannon entropy as the semantic uncertainty. The specific process of specifically determining the spatial information uncertainty and the semantic uncertainty is the same as the calculation process in the above embodiment and will not be elaborated here.

[0138] The terminal can obtain the preset perception conditions in the sample point cloud dataset. The terminal can determine the preset perception conditions through the validation set. The terminal can judge whether the sample perception information of the obstacle meets the preset perception conditions. The terminal can compare the spatial position information of the sample perception information with the spatial position information in the preset perception conditions to obtain the spatial intersection over union. If the semantic category classification is consistent when the sample perception information is compared with the preset perception conditions, and the spatial intersection over union exceeds the true value matching threshold, the correctness detection result is the correct detection result. If the semantic category classification is inconsistent when the sample perception information is compared with the preset perception conditions, and / or the spatial intersection over union does not exceed the true value matching threshold, the correctness detection result is the incorrect detection result.

[0139] The terminal can determine the sample entity uncertainty value based on the preset depth integration algorithm and the sample perception information. The specific steps for calculating the sample entity uncertainty value have been described in the foregoing embodiments and will not be elaborated here.

[0140] The terminal can set multiple uncertainty boundary thresholds, and can respectively compare them with the sample entity uncertainty value to obtain multiple uncertainty detection results. For each uncertainty boundary threshold, if the uncertainty detection result is that the sample entity uncertainty value exceeds the uncertainty boundary threshold, the uncertainty detection result is the uncertainty detection result. If the uncertainty detection result is that the sample entity uncertainty value does not exceed or is equal to the uncertainty boundary threshold, the uncertainty detection result is the certainty detection result. The terminal can determine the evaluation score based on the correctness detection result and the number of obstacles corresponding to the uncertainty detection result.

[0141] The terminal can determine the preset uncertainty boundary threshold by selecting the sample uncertainty boundary threshold with the highest evaluation score among the multiple evaluation scores.

[0142] In addition, the terminal can first set an uncertainty boundary threshold and perform a comparison of the uncertainty boundary threshold to obtain an uncertainty detection result. If the uncertainty detection result is that the sample entity uncertainty value exceeds the uncertainty boundary threshold, the uncertainty detection result is the uncertainty detection result. If the uncertainty detection result is that the sample entity uncertainty value does not exceed or is equal to the uncertainty boundary threshold, the uncertainty detection result is the certainty detection result. The terminal can determine the evaluation score based on the correctness detection result and the number of obstacles corresponding to the uncertainty detection result. The terminal can update the uncertainty boundary threshold and finally obtain the evaluation score. And so on, the terminal can obtain multiple evaluation scores.

[0143] In addition, the specific process of the terminal training the preset object detection integrated model may include the following steps: The terminal can use the training set data to train the first initial object detection model. In the initialization stage, the model parameters are randomly set. During the training process, before each round of traversing the training set, it is randomly shuffled until the loss function converges and reaches a certain accuracy to prove that the model is not underfitted, and at the same time, the accuracy performance of the validation set is similar to prove that the initial object detection model is not overfitted, so as to obtain the first preset object detection model. The terminal can record the number of training rounds at this time. Subsequently, repeat the above process to train multiple initial object detection models with the same network structure to obtain multiple preset object detection models, ensuring random initialization, random shuffling of the training set, and the same number of training rounds as the first model, as the basis for deep integration. The terminal can integrate multiple preset object detection models to obtain a preset object detection integrated model.

[0144] In this embodiment, based on the correctness detection results and multiple uncertainty detection results, the evaluation scores corresponding to the sample uncertainty boundary thresholds are respectively determined, and the sample uncertainty boundary threshold with the highest evaluation score is determined as the preset uncertainty boundary threshold, so as to obtain a reasonable preset uncertainty boundary threshold, which is convenient for more accurately determining the uncertain situation.

[0145] In an exemplary embodiment, the specific implementation process of the step "respectively determine the evaluation scores corresponding to the sample uncertainty boundary thresholds based on the correctness detection results and multiple uncertainty detection results" may include:

[0146] For each uncertainty detection result, determine the first quantity of the sample obstacles that meet the correct detection result and the uncertainty detection result; determine the second quantity of the sample obstacles that meet the correct detection result and the uncertainty detection result; determine the third quantity of the sample obstacles that meet the wrong detection result and the uncertainty detection result; determine the fourth quantity of the sample obstacles that meet the wrong detection result and the uncertainty detection result;

[0147] Determine the ratio of the fourth quantity to the sum of the fourth quantity and the third quantity as the coverage rate; determine the ratio of the second quantity to the sum of the fourth quantity and the fourth quantity as the false alarm rate; determine the first ratio of the second quantity to the sum of the second quantity and the first quantity, and determine the ratio of the coverage rate to the first ratio as the quality score;

[0148] Based on the preset weight distribution coefficient, coverage rate, false alarm rate and quality score, determine the evaluation score corresponding to the sample uncertainty boundary threshold.

[0149] Among them, the correctness detection results include correct detection results and wrong detection results, and the uncertainty detection results include uncertainty detection results and determination detection results.

[0150] Specifically, for each uncertain detection result, the terminal can classify the detection results into four categories based on two dimensions: whether it is a correct detection result and the uncertainty detection result. The first quantity n of the sample obstacles that satisfy the correct detection result and the corresponding certain detection result AC , the second quantity n of the sample obstacles that satisfy the correct detection result and the corresponding uncertain detection result AU , the third quantity n of the sample obstacles that satisfy the wrong detection result and the corresponding certain detection result IC , the fourth quantity n of the sample obstacles that satisfy the wrong detection result and the corresponding uncertain detection result IU .

[0151] The terminal can calculate the coverage rate CR, false alarm rate FAR, and quality score UQS based on the first quantity, second quantity, third quantity, and fourth quantity respectively. The specific formulas for the coverage rate, false alarm rate, and quality score can be:

[0152]

[0153] Based on the preset weight distribution coefficient, coverage rate, false alarm rate, and quality score, the calculation formula for determining the evaluation score score corresponding to the sample uncertainty threshold can be:

[0154] score = η CR CR - η FAR FAR + η UQS UQS

[0155] Among them, η CR is the weight coefficient of the coverage rate CR, η FAR is the weight coefficient of the false alarm rate FAR, η UQS is the weight coefficient of the quality score, which is used to adjust the weight ratio of the coverage rate, false alarm rate, and quality score in the comprehensive score. η CR The higher it is, the more it means that the proportion of wrong detection results with high uncertainty is expected to be more; η FAR , which means that the proportion of correct detection results with high uncertainty is expected to be less; η UQS The higher it is, the stronger the correlation between the expected high-uncertainty detection results and the wrong detection results is.

[0156] In this embodiment, by setting three indicators: the coverage rate, false alarm rate, and quality score, analyzing and evaluating on the sample data, and selecting a reasonable uncertainty threshold, the model is given the ability of online monitoring, requiring the coverage rate and quality score to be as high as possible, while the false alarm rate is as low as possible.

[0157] In an exemplary embodiment, the obstacle recognition method further includes:

[0158] Extract the sample space geometric features of the sample point cloud clusters corresponding to different sample obstacles in the preset obstacle feature library based on the preset entity degree algorithm; classify the sample point cloud clusters through the preset rule algorithm to obtain the bounding box information and sample entity degree value corresponding to each cluster of sample point clouds;

[0159] Compare the bounding box information with the preset bounding box information to obtain the spatial intersection over union (IoU), and determine the success detection result based on the comparison result between the spatial IoU and the preset spatial matching threshold; determine multiple entity degree detection results based on the comparison result between the sample entity degree value and multiple sample entity degree boundary thresholds;

[0160] Determine the effective detection rates of each sample entity degree boundary threshold based on the success detection result and multiple entity degree detection results respectively;

[0161] Determine the sample entity degree boundary threshold with the highest effective detection rate as the preset entity degree boundary threshold.

[0162] Among them, the preset entity degree algorithm can be the ESF feature descriptor, and the preset obstacle feature library can include obstacles existing in the target scene. For example, there are several obstacles such as mine cars, cars, pedestrians, and dust. The spatial geometric features can characterize the relationship between the shape functions (surface distance, angle, and area) between each point in the point cloud cluster and its neighboring points.

[0163] Specifically, the terminal can use the ESF feature descriptor to extract the spatial geometric features of the three-dimensional coordinates of the sample point cloud clusters of mine cars, cars, pedestrians, and dust. The terminal can perform statistical analysis on the sample space geometric features and sample reflection intensity features of all sample point cloud clusters. The terminal calculates the average value of the reflection intensities of all points in each cluster of sample point clouds as the sample reflection intensity feature of the sample point cloud cluster, and determines the reflection intensity threshold I according to the distribution of the reflection intensity features of the sample point cloud clusters thre 。

[0164] The terminal classifies the sample point cloud clusters through the preset rule algorithm to obtain the bounding box information and sample entity degree value of each cluster of sample point clouds. The terminal compares the bounding box information with the preset bounding box information to obtain the spatial IoU. The terminal can compare the spatial IoU with the preset spatial matching threshold θ truth to obtain the comparison result. If the comparison result is that the spatial IoU exceeds the preset spatial matching threshold θ truth , mark the success detection result as the success detection result, otherwise mark it as the error detection result.

[0165] The terminal can set multiple sample entity degree demarcation thresholds, and the terminal can determine multiple entity degree detection results based on the comparison results between the sample entity degree values and the multiple sample entity degree demarcation thresholds; for each sample entity degree demarcation threshold, if the entity degree detection result is that the sample entity degree value exceeds the sample entity degree demarcation threshold λ entity When it is, it is marked as an entity detection result, otherwise it is marked as a non-entity detection result. The terminal can determine multiple entity degree detection results based on the comparison results between the sample entity degree values and the multiple sample entity degree demarcation thresholds, and respectively determine the effective detection rates of each sample entity degree demarcation threshold based on the success detection results and the multiple entity degree detection results. The terminal can select the highest effective monitoring rate and determine the sample entity degree demarcation threshold with the highest effective detection rate as the preset entity degree demarcation threshold.

[0166] In addition, the terminal sets a sample entity degree demarcation threshold and calculates the effective detection rate corresponding to this sample entity degree demarcation threshold according to the above steps; the terminal can update this sample entity degree demarcation threshold and repeat the steps of the above embodiment to obtain the effective detection rate. The terminal can obtain multiple effective detection rates by updating the sample entity degree demarcation threshold.

[0167] In addition, the terminal can perform arithmetic averaging on the spatial geometric features of the same class of a total of M sample obstacles by dimension as the prior spatial geometric features of this class of obstacles The calculation formula of the prior spatial geometric spatial feature can be:

[0168]

[0169] Among them, is the prior spatial geometric feature of the kth class (since there are 4 types of prior features, k can only take values 1, 2, 3, and 4, representing trucks, cars, pedestrians, and dust respectively), M is the total number of samples of the mth class (Ω k ) of the offline prior feature vector, and k is the mth feature in the kth class of prior features.

[0170] In this embodiment, by aiming at the problem that the rule clustering model is prone to classification errors in a heavy dust environment, the preset entity degree demarcation threshold of the obstacle is determined, which can realize the subsequent differentiation of the entity degree of the obstacle, so as to well adapt to the dust with different distances, different shapes, and different densities, improve the robustness of the algorithm, and can achieve more accurate identification in combination with the deep learning model.

[0171] In an exemplary embodiment, respectively determining the effective detection rates of each sample entity degree demarcation threshold based on the success detection results and the multiple entity degree detection results includes:

[0172] For each entity degree detection result, determine the fifth quantity of the sample obstacles that satisfy the existence of the point cloud cluster of the sample obstacles and the sample obstacles corresponding to the successful detection result and the low entity degree detection result; determine the sixth quantity of the sample obstacles that satisfy the existence of the point cloud cluster of the sample obstacles and the sample obstacles corresponding to the successful detection result and the high entity degree detection result; determine the seventh quantity of the sample obstacles that satisfy the non-existence of the point cloud cluster of the sample obstacles and the sample obstacles corresponding to the error detection result and the low entity degree detection result; determine the eighth quantity of the sample obstacles that satisfy the non-existence of the point cloud cluster of the sample obstacles and the sample obstacles corresponding to the error detection result and the high entity degree detection result;

[0173] Determine the effective detection rate of the entity by taking the ratio of the sixth quantity to the sum of the fifth quantity and the sixth quantity, and determine the effective detection rate of the non-entity by taking the ratio of the seventh quantity to the sum of the seventh quantity and the eighth quantity;

[0174] Based on the sum of the effective detection rate of the entity and the effective detection rate of the non-entity, determine the effective detection rate of the sample entity degree demarcation threshold.

[0175] Among them, the success detection results include the successful detection result and the error detection result, and the entity degree detection results include the high entity degree detection result and the low entity degree detection result.

[0176] Specifically, for each entity degree detection result, the terminal can detect two dimensions of correct or incorrect and high or low entity degree, divide the detection results into five categories, and record the number of results C1 that exist in the dataset but are not detected, the fifth quantity C2 of the sample obstacles that satisfy the successful detection result and the low entity degree detection result, the sixth quantity C3 of the sample obstacles that satisfy the existence of the point cloud cluster of the sample obstacles and the successful detection result and the high entity degree detection result, the seventh quantity C4 of the sample obstacles that satisfy the non-existence of the point cloud cluster of the sample obstacles and the error detection result and the low entity degree detection result, and the eighth quantity C5 of the sample obstacles that satisfy the non-existence of the point cloud cluster of the sample obstacles and the error detection result and the high entity degree detection result.

[0177] The terminal can calculate the effective detection rate DR of the entity based on the fifth quantity, the sixth quantity, the seventh quantity, and the eighth quantity respectively entity , the effective detection rate DR of the non-entity (dust), dust , and the overall effective detection rate DR. The specific formulas for the effective detection rate of the entity, the effective detection rate of the non-entity (dust), and the overall effective detection rate can be:

[0178]

[0179] Among them, ω and are adjustable weight coefficients, and the setting of the weight coefficients can be adjusted according to the actual situation. For example, if more emphasis is placed on the safety of the result, ω can be increased; if more emphasis is placed on the transportation efficiency,

[0180] In one embodiment, based on the real point cloud dataset of the open-pit mining area collected on-site, it is divided into a training set, a validation set, and a test set. In each frame of data in the dataset, three-dimensional annotations are performed on entities such as mining trucks, cars, and pedestrians for the training of the deep learning model. During the annotation process, the pitch angle and roll angle are not considered, and information such as the position, size, orientation, and category of the cuboid bounding box of each entity is recorded and saved; in addition, in order to analyze the multivariate features of different obstacle point cloud clusters, a part of the data subset containing entities or non-entities is taken, and the three-dimensional coordinates and reflection intensity information of the mining truck, car, pedestrian, and dust point cloud clusters are recorded. The dust is regarded as a non-entity obstacle, and the rest of the categories are regarded as entity obstacles, forming a data pool for feature analysis.

[0181] In one embodiment, the process of preprocessing the real point cloud data may specifically include:

[0182] The terminal can process the real point cloud dataset of the open-pit mining area collected on-site according to a certain proportion, and extract and divide it into the training set, the validation set, and the test set from the continuous frames of multiple scenarios. The terminal can use a point cloud annotation tool (Point Cloud Annotation Tool, PCAT) etc. to visualize and annotate each frame of data in the dataset, distinguishing common entities such as mining trucks, cars, and pedestrians in the mining area;

[0183] The terminal can annotate and record information such as the position, size, orientation, and category of the cuboid bounding box of the entity point cloud of each category, and save the annotation results in files in formats such as.txt,.json,.pkl, etc., for easy calling by the algorithm model.

[0184] The terminal can process the real point cloud dataset of the open-pit mining area collected on-site according to a certain proportion, and extract and divide it into a feature extraction set from obstacles in different scenarios with various postures. Use the robot operating system visualization (ROS Visualization Tool, Rviz) tool to visualize each frame of data, distinguishing the entities (mining trucks, cars, pedestrians) and non-entities (dust) existing in the scenario.

[0185] The terminal can record the three-dimensional coordinates and reflection intensity information of the mining truck, car, pedestrian, and dust point cloud clusters respectively, and save the recording results in a.txt format file for point cloud multivariate feature analysis and testing.

[0186] In an exemplary embodiment, the specific implementation process of step 103 may include:

[0187] If the matching results of the first spatial position information and the second spatial position information exceed the preset spatial matching threshold, it is determined that obstacles of the same type are detected, and the initial recognition result of the obstacle is determined based on the entity uncertainty risk level and the obstacle type;

[0188] If the matching results of the first spatial position information and the second spatial position information are lower than the preset spatial matching threshold, it is determined that obstacles of different types are detected. Then, the initial recognition result of the obstacle corresponding to the first spatial position information is determined based on the risk level, and the initial recognition result of the obstacle corresponding to the second spatial position information is determined based on the obstacle type;

[0189] The initial recognition result that meets the preset result screening condition is determined as the obstacle recognition result.

[0190] Among them, the preset result screening condition is used to retain the initial recognition result representing the entity obstacle or the initial recognition result of the obstacle corresponding to the first spatial position information.

[0191] Specifically, if the matching results of the first spatial position information and the second spatial position information exceed the preset spatial matching threshold, the terminal determines that the same obstacle is detected by the two methods. The terminal can determine the initial recognition result of the obstacle based on the entity uncertainty risk level of the obstacle and the obstacle type. The initial recognition results can be divided into four categories: high-uncertainty entity results, high-uncertainty dust results, low-uncertainty entity results, and low-uncertainty dust results.

[0192] If the matching results of the first spatial position information and the second spatial position information are lower than the preset spatial matching threshold, the terminal determines that the obstacles detected by the two methods are not the same obstacle, that is, only one method detects the obstacle. If the obstacle is detected by the preset target detection integration model, the terminal can determine the initial recognition result of the obstacle based on the entity degree risk level; if the obstacle is detected by the preset rule algorithm, the terminal can determine the initial recognition result of the obstacle based on the type of the obstacle. The initial recognition results can be divided into four categories: high-uncertainty results detected only by the target detection model, low-uncertainty results detected only by the target detection model, dust results detected only by the preset rule algorithm, and entity results detected only by the preset rule algorithm.

[0193] The terminal can screen the initial recognition results based on the preset result screening condition and retain the results representing the entity of the obstacle in the initial recognition results. For example, retain the low-uncertainty entity results, retain the high-uncertainty entity results, the entity results detected only by the preset rule algorithm, and due to the difference in the detection performance of the target detection model and the preset rule algorithm in different situations, the results detected only by the target detection model also need to be retained.

[0194] Among them, the entity results with low uncertainty characterize positive samples that are highly certain to be entity obstacles; the entity results with high uncertainty characterize that although their uncertainty is high, since they are classified as entities, such results may be caused by unconventional situations outside the generalization ability of the deep learning model, such as abnormal postures of trucks, cars, and pedestrians, and there are relatively large risks and need to be retained; the entity results detected only by the preset rule algorithm characterize obstacle categories not in the training set, such as retaining walls on both sides of the road in the open-pit mining area and small mounds caused by slight collapses; the results detected only by the target detection model characterize that for obstacle targets that are far away and small in size, the clustering and matching processes of the preset rule algorithm are prone to being unable to effectively detect small targets, resulting in a high probability of missed detection. To avoid losing this key obstacle information.

[0195] In this embodiment, the entities in the preliminary detection scenario of the target detection model can be retained, the preset rule algorithm supplements the missed entities for it, and the dust misdetected as an entity by the target detection model is deleted, achieving a higher detection coverage, ensuring the safety of the autonomous driving mining truck transportation process, realizing the recognition of entity obstacles, and filtering out the dust misdetected as an entity, improving the safety and efficiency of mining area work.

[0196] In an exemplary embodiment, the specific implementation process of step 101, "detecting the point cloud data corresponding to the target scene based on the preset target detection integrated model to obtain the target perception information of each obstacle in the target scene", may include:

[0197] Detecting the point cloud data of the target scene through the multiple preset target detection models respectively to obtain multiple perception information sets, and the perception information set includes the initial perception information of multiple obstacles;

[0198] Integrating the initial perception information corresponding to each of the obstacles based on a preset information integration algorithm to obtain the target perception information of each of the obstacles.

[0199] Among them, the preset target detection integrated model includes multiple preset target detection models. The target perception information includes attribute information.

[0200] Specifically, the terminal can use the preset target detection integrated model E to perform target detection on the real-time sample point cloud data. Among them, each preset target detection model D in the preset target detection integrated model E n processes the same frame of data through a separate thread to obtain a total of N initial perception information P n , where N is the number of preset target detection models in the preset target detection integrated model, and n is the model number. The terminal can aggregate all the initial perception information together according to the model number to obtain P = {P1, P2, P3,..., PN}. The initial perception information P n is all the entity information detected by the corresponding model in the current frame. Except for the model number information, the spatial position and semantic category of each entity detection information are stored in B n,m , where m is the entity number. All the entity information B n contained in the initial perception information P n,m does not exceed M, and M is the number of entities detected by the model integration in the current real-time data frame.

[0201] The terminal can cluster the detection results of different models in the same frame through semantic category and spatial intersection-over-union based on the initial perception information P n corresponding to each model D n . If there are two detection results (attribute information) B i and B j in the initial perception information P i and P j of two different models D i,m and D j,m , and they satisfy the same category and the three-dimensional intersection-over-union exceeds the integrated matching threshold θ ensemble , it is regarded as a successful match, and these two detection results are clustered together. The terminal completes the clustering through the basic sequential algorithm considering sample exclusivity, reorganizes the initial perception information into a total of M target perception information, and obtains R = {R1, R2, R3, ……, R M}. The target perception information R m is the result of the corresponding entity detected by all models, mainly including the first spatial position information, semantic category, and entity number information of the entity. Use the basic sequential algorithm considering sample exclusivity to reorganize the initial perception information P n into the target perception information R m .

[0202] In this embodiment, the preset target detection integration model can run in parallel and multi-threaded simultaneously, and reorganize the operation results, which not only improves the efficiency of data analysis but also improves the accuracy of the obstacle detection results.

[0203] In an exemplary embodiment, the specific implementation steps of the step "integrate the initial perception information corresponding to each of the obstacles based on the preset information integration algorithm to obtain the target perception information of each of the obstacles" may include:

[0204] For the first initial perception information, establish corresponding M target perception information for each entity information in the first initial perception information; set an initial clustering identifier for the nth initial perception information, where 1 < n ≤ N and n is an integer; for the mth target perception information, determine the entity information in the nth initial perception information that meets the preset association conditions and place it in the mth initial perception information, and update the initial clustering identifier in the nth initial perception information to obtain the updated clustering identifier, where 1 < m ≤ M and m is an integer.

[0205] Among them, the initial perception information includes entity information corresponding to multiple obstacles, and the preset association conditions may include a category association condition, a spatial association condition, and an exclusivity association condition. For example, the category association condition may specifically be that the category identifier with the maximum confidence coefficient of the entity is the same as that of the target perception information, and the spatial association condition may specifically be that the maximum three-dimensional intersection over union ratio of the entity and the target perception information exceeds the integration matching threshold θ ensemble ; the exclusivity association condition may specifically be that there is no entity in the target perception information that is in the same initial perception information as the entity.

[0206] Specifically, the terminal can aggregate each initial perception information P = {P1, P2, P3, ……, P N} of the preset target detection integration model, read the first initial perception information P1, record the total number of entities detected by the first model as M, and establish target perception information R for each piece of attribute information B 1,m therein. m .

[0207] For the nth initial perception information P n , the terminal can initialize and set the clustering identifier flag n for P excl = 0 M . This clustering identifier is a vector of length M with a value of 0, indicating that different attribute information B n and B n,i in the initial perception information P n,j cannot be divided into the same target perception information R m .

[0208] For the mth target perception information R m , the terminal can judge each piece of attribute information B n in the current initial perception information P n,i and the three clustering association conditions of R m . If both the category association condition and the spatial association condition are satisfied, and the exclusivity association condition also passes the flag excl (m) = 0, then place B n,i into R mAnd update the clustering flag to flag excl (m)=1.

[0209] The terminal can repeat the above “for the mth target perception information R m " step until all current target perception information R is traversed m , if the initial perception information P n There are ΔM unassigned entity information B n,i , then establish target perception information for each of them and get R M+ΔM , and update the total amount of target perception information to M = M + ΔM.

[0210] The terminal can repeat "for the nth initial perception information P n "Until all initial perception information P is traversed n , complete the reorganization of perception information from model-based to entity-based, aggregate and output target perception information R = {R1, R2 R3, ..., R M}.

[0211] like Figure 2 As shown, according to the overall framework of the open-pit mine dust and entity recognition method based on the fusion of multivariate features of lidar point cloud and model uncertainty provided in this embodiment, the framework is divided into two parts: offline analysis and online recognition.

[0212] During the offline analysis process, a large amount of data is collected by the vehicle-mounted lidar sensor to obtain real data of the open-pit mine area in multiple scenarios. First, the offline collected data is divided into a training set, a validation set, and a test set according to a certain ratio for training the target detection model. At the same time, a certain number of point cloud cluster samples are extracted from them to form a feature extraction set. Next, multiple target detection models are trained and a target detection integration model is constructed. The model uncertainty is calculated based on the deep integration method, and then the preset uncertainty demarcation threshold is optimized according to the designed evaluation system. The specific details are as follows: Figure 3 As shown, Figure 3 This is a principle entity diagram for analyzing the threshold value of the degree of certainty of the target detection model. The spatial geometric features of the point cloud cluster are extracted using the ESF feature descriptor, and the degree of the point cloud cluster is calculated by combining the reflection intensity features. Then, the degree of certainty threshold is optimized according to the designed evaluation system. The specific details are as follows: Figure 4 As shown, Figure 3 It is a schematic diagram of the principle of performing a preset entity degree demarcation threshold analysis on a preset rule model.

[0213] During the online recognition process, the on-vehicle lidar sensor is used to obtain real-time scene data, which serves as the input data for the deep learning model (object detection model) and the rule clustering model (preset rule algorithm). Online recognition is performed based on the integrated model and prior features obtained from offline analysis. First, the real-time object detection results of the deep learning model in the current scene are reorganized to calculate the uncertainty of each detection result. All recognition results are classified by comparing the uncertainty boundary threshold. The specific details are as Figure 5 shown, Figure 5 which is a schematic diagram of the principle of real-time entity detection and online uncertainty estimation based on the preset object detection integrated model. At the same time, the similarity between the real-time point cloud clustering results of the rule clustering model in the current scene and the prior features is compared to calculate the entity degree of each clustering result. All recognition results are classified by comparing the entity degree threshold. The specific details are as Figure 6 shown, Figure 6 which is a schematic diagram of the principle of real-time entity detection and online entity degree estimation through the preset rule model. Subsequently, the detection results of the object detection model and the clustering results of the rule clustering model are spatially matched. Based on whether the matching is successful, all recognition results are fused and further classified in combination with three dimensions: the uncertainty level and the entity degree level. Finally, high-uncertainty entities, low-uncertainty entities, low-uncertainty results detected only by the object detection model, and entities detected only by the rule clustering model are retained as the final output to achieve the recognition of entity obstacles in the open-pit mining area and filter out the dust that is misdetected as an entity, improving the safety and efficiency of mining area work. The specific details are as Figure 7 shown, Figure 7 which is a schematic diagram of the principle of fusing the output results of the two models.

[0214] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily have to be executed at the same moment, but can be executed at different moments. The execution order of these steps or stages does not necessarily have to be sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0215] Based on the same inventive concept, an embodiment of the present application further provides an obstacle recognition device for implementing the above-mentioned obstacle recognition method. The solution provided by this device for solving problems is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the obstacle recognition device provided below can refer to the limitations on the obstacle recognition method in the foregoing, and will not be elaborated here.

[0216] In an exemplary embodiment, as Figure 8 shown, an obstacle recognition device 80 is provided, including: a detection module 81, a determination module 82, and an identification module 83, where:

[0217] The detection module 81 is configured to detect the point cloud data of the target scene based on a preset target detection integrated model to obtain the perception information of each obstacle in the target scene; and determine the entity uncertainty risk level of the corresponding obstacle based on the perception information, where the perception information at least includes the first spatial position information of each obstacle;

[0218] The determination module 82 is configured to process the point cloud data based on a preset rule algorithm to obtain the point cloud cluster of each obstacle; determine the obstacle type of the corresponding obstacle through the spatial geometric features of the point cloud cluster, and determine the second spatial position information of the corresponding obstacle based on the spatial position information of the point cloud cluster;

[0219] The identification module 83 is configured to match the first spatial position information and the second spatial position information to obtain a matching result, and determine the obstacle recognition result based on the matching result, the entity uncertainty risk level, and the obstacle type.

[0220] In one embodiment, the detection module 81 is specifically configured to determine the entity uncertainty value based on a preset depth integration algorithm and target perception information;

[0221] Compare the entity uncertainty value with a preset uncertainty boundary threshold to obtain a first comparison result, and determine the entity uncertainty risk level of the obstacle corresponding to the target perception information based on the first comparison result.

[0222] In one embodiment, the detection module 81 is specifically configured to determine the mean value of each attribute information based on the number of initial perception information in the target perception information corresponding to the obstacle and each attribute information; determine the variance value of the attribute information based on multiple attribute information and the mean value corresponding to each attribute information, and determine the variance value as the spatial information uncertainty of the obstacle;

[0223] Based on the mean value corresponding to the classification score, the Shannon entropy of the classification score is determined, and the Shannon entropy is corrected based on the number of initial perception information in the target perception information corresponding to the obstacle and the number of preset target detection integration models to obtain the semantic uncertainty; the semantic uncertainty and the spatial information uncertainty are determined as the entity uncertainty value of the obstacle.

[0224] In one embodiment, the determining module 82 is specifically configured to determine the entity degree value of the obstacle corresponding to the point cloud cluster based on a preset entity degree algorithm, the spatial geometric features of the point cloud cluster, and multiple types of prior spatial geometric features in a preset obstacle feature library;

[0225] The entity degree value is compared with a preset entity degree boundary threshold to obtain a second comparison result, and the obstacle type of the obstacle corresponding to the point cloud cluster is determined based on the second comparison result.

[0226] In one embodiment, the determining module 82 is specifically configured to calculate the covariance matrix of the spatial geometric features and multiple types of prior spatial geometric features in the preset obstacle feature library;

[0227] Based on a preset entity degree algorithm, the spatial geometric features, multiple types of prior spatial geometric features, and the covariance matrix are processed to obtain an initial similarity probability value;

[0228] Based on a preset normalization correction algorithm, the initial similarity probability value is corrected to obtain the entity degree value of the obstacle, and the entity degree value represents the similarity probability between the point cloud cluster corresponding to the obstacle and the preset obstacle feature library.

[0229] In one embodiment, the detection module 81 is further configured to detect the sample point cloud data set based on a preset target detection integration model to obtain sample perception information;

[0230] For each sample obstacle, the sample perception information is detected based on a preset perception condition to obtain a correctness detection result; the sample entity uncertainty value is determined based on a preset depth integration algorithm and the sample perception information; the sample entity uncertainty and multiple sample uncertainty boundary thresholds are respectively compared to determine multiple uncertainty detection results of the sample obstacle;

[0231] Based on the correctness detection result and multiple uncertainty detection results, the evaluation scores corresponding to the respective sample uncertainty boundary thresholds are determined;

[0232] The sample uncertainty boundary threshold with the highest evaluation score is determined as the preset uncertainty boundary threshold.

[0233] In one embodiment, the detection module 81 is further configured to, for each uncertain detection result, determine a first quantity of sample obstacles that meet the correct detection result and the sample obstacles corresponding to the determined detection result; determine a second quantity of sample obstacles that meet the correct detection result and the sample obstacles corresponding to the uncertain detection result; determine a third quantity of sample obstacles that meet the wrong detection result and the sample obstacles corresponding to the determined detection result; determine a fourth quantity of sample obstacles that meet the wrong detection result and the sample obstacles corresponding to the uncertain detection result;

[0234] Determine that the ratio of the fourth quantity to the sum value of the fourth quantity and the third quantity is the coverage rate; determine that the ratio of the second quantity to the sum value of the fourth quantity and the fourth quantity is the false alarm rate; determine a first ratio of the second quantity to the sum value of the second quantity and the first quantity, and determine the ratio of the coverage rate to the first ratio as the quality score;

[0235] Based on the preset weight distribution coefficient, coverage rate, false alarm rate, and quality score, determine the evaluation score corresponding to the sample uncertainty boundary threshold.

[0236] In one embodiment, the determination module 82 is further configured to extract the sample space geometric features of the sample point cloud clusters corresponding to different sample obstacles in the preset obstacle feature library based on the preset entity degree algorithm; classify the sample point cloud clusters through the preset rule algorithm to obtain the bounding box information and sample entity degree value corresponding to each cluster of sample point clouds;

[0237] Compare the bounding box information with the preset bounding box information to obtain the spatial intersection-over-union ratio, and determine the success detection result based on the comparison result of the spatial intersection-over-union ratio and the preset spatial matching threshold; determine multiple entity degree detection results based on the comparison result of the sample entity degree value and multiple sample entity degree boundary thresholds;

[0238] Based on the success detection result and multiple entity degree detection results, respectively determine the effective detection rates of each sample entity degree boundary threshold;

[0239] Determine the sample entity degree boundary threshold with the highest effective detection rate as the preset entity degree boundary threshold.

[0240] In one embodiment, the determination module 82 is further configured to, for each entity degree detection result, determine a fifth quantity of sample obstacles that satisfy the presence of the sample obstacle in the point cloud cluster and the sample obstacles corresponding to the success detection result and the low entity degree detection result; determine a sixth quantity of sample obstacles that satisfy the presence of the sample obstacle in the point cloud cluster and the sample obstacles corresponding to the success detection result and the high entity degree detection result; determine a seventh quantity of sample obstacles that satisfy the absence of the sample obstacle in the point cloud cluster and the sample obstacles corresponding to the wrong detection result and the low entity degree detection result; determine an eighth quantity of sample obstacles that satisfy the absence of the sample obstacle in the point cloud cluster and the sample obstacles corresponding to the wrong detection result and the high entity degree detection result;

[0241] Determine the ratio of the sixth quantity to the sum value of the fifth quantity and the sixth quantity as the effective detection rate of the entity, and determine the ratio of the seventh quantity to the sum value of the seventh quantity and the eighth quantity as the effective detection rate of the non-entity;

[0242] Based on the sum value of the effective detection rate of the entity and the effective detection rate of the non-entity, determine the effective detection rate of the sample entity degree boundary threshold.

[0243] In one embodiment, the recognition module 83 is specifically configured to, if the matching result is that the first spatial position information and the second spatial position information exceed the preset spatial matching threshold, determine that the same type of obstacle is detected, and determine the initial recognition result of the obstacle based on the entity uncertainty risk level and the obstacle type;

[0244] If the matching result is that the first spatial position information and the second spatial position information are lower than the preset spatial matching threshold, determine that different types of obstacles are detected, then determine the initial recognition result of the obstacle corresponding to the first spatial position information based on the risk level, and determine the initial recognition result of the obstacle corresponding to the second spatial position information based on the obstacle type;

[0245] Determine the initial recognition result that meets the preset result screening condition as the obstacle recognition result, and the preset result screening condition is used to retain the initial recognition result representing the entity obstacle or the initial recognition result of the obstacle corresponding to the first spatial position information.

[0246] Each module in the above obstacle recognition device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor in the computer device in the form of hardware or independent of it, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0247] In an exemplary embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 9As shown in the figure. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC), or other technologies. The computer program, when executed by the processor, implements a method for identifying obstacles. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, touchpad, or mouse, etc.

[0248] Those skilled in the art can understand that Figure 9 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0249] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0250] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0251] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0252] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0253] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include Read-Only Memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, Resistive Random Access Memory (ReRAM), Magnetoresistive Random Access Memory (MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (PCM), graphene memory, etc. Volatile memory can include Random Access Memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), etc. The database involved in the embodiments provided in this application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, Artificial Intelligence (AI) processors, etc., without limitation.

[0254] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this application.

[0255] The above-described embodiments merely represent several implementation manners of this application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of this application. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of this application shall be subject to the appended claims.

Claims

1. A method for identifying an obstacle, characterized in that The method includes: Detecting the point cloud data of the target scene based on a preset target detection integration model to obtain the target perception information of each obstacle in the target scene; and determining the entity uncertainty risk level of the corresponding obstacle based on the target perception information, where the target perception information at least includes the first spatial position information of each obstacle; Processing the point cloud data based on a preset rule algorithm to obtain a point cloud cluster for each obstacle; determining the obstacle type of the corresponding obstacle through the spatial geometric features of the point cloud cluster, and determining the second spatial position information of the corresponding obstacle based on the spatial position information of the point cloud cluster; Matching the first spatial position information and the second spatial position information to obtain a matching result, and determining the obstacle recognition result based on the matching result, the entity uncertainty risk level, and the obstacle type.

2. The method according to claim 1, characterized in that, The determining the entity uncertainty risk level of the corresponding obstacle based on the target perception information includes: Determining the entity uncertainty value based on a preset depth integration algorithm and the target perception information; Comparing the entity uncertainty value with a preset uncertainty boundary threshold to obtain a first comparison result, and determining the entity uncertainty risk level of the obstacle corresponding to the target perception information based on the first comparison result.

3. The method according to claim 2, wherein The target perception information includes multiple initial perception information of the corresponding obstacle, and each initial perception information includes multiple attribute information of the obstacle. The attribute information at least includes the classification score. The determining the entity uncertainty value based on a preset depth integration algorithm and the target perception information includes: Determining the mean value of each attribute information based on the number of initial perception information in the target perception information corresponding to the obstacle and each attribute information; determining the variance value of the attribute information based on the multiple attribute information and the mean value corresponding to each attribute information, and determining the variance value as the spatial information uncertainty of the obstacle; Determining the Shannon entropy of the classification score based on the mean value corresponding to the classification score, correcting the Shannon entropy based on the number of initial perception information in the target perception information corresponding to the obstacle and the number of preset target detection integration models to obtain the semantic uncertainty; determining the semantic uncertainty and the spatial information uncertainty as the entity uncertainty value of the obstacle.

4. The method according to claim 1, wherein The determining the obstacle type of the corresponding obstacle through the spatial geometric features of the point cloud cluster includes: Determining the entity degree value of the obstacle corresponding to the point cloud cluster based on a preset entity degree algorithm, the spatial geometric features of the point cloud cluster, and multiple prior spatial geometric features in a preset obstacle feature library; Comparing the entity degree value with a preset entity degree boundary threshold to obtain a second comparison result, and determining the obstacle type of the obstacle corresponding to the point cloud cluster based on the second comparison result.

5. The method according to claim 4, wherein The determining the entity degree value of the obstacle corresponding to the point cloud cluster based on a preset entity degree algorithm, the spatial geometric features of the point cloud cluster, and multiple prior spatial geometric features in a preset obstacle feature library includes: Calculate the covariance matrix of the spatial geometric features and multiple prior spatial geometric features in the preset obstacle feature library; Based on the preset entity degree algorithm, process the spatial geometric features, the multiple prior spatial geometric features, and the covariance matrix to obtain an initial similarity probability value; Based on the preset normalization correction algorithm, correct the initial similarity probability value to obtain the entity degree value of the obstacle, where the entity degree value represents the similarity probability between the point cloud cluster corresponding to the obstacle and the preset obstacle feature library.

6. The method according to claim 2, characterized in that The method further includes: Based on the preset target detection integration model, detect the sample point cloud data set to obtain sample perception information; For each sample obstacle, based on the preset perception conditions, detect the sample perception information to obtain a correctness detection result; based on the preset depth integration algorithm and the sample perception information, determine a sample entity uncertainty value; based on the sample entity uncertainty, compare it with multiple sample uncertainty boundary thresholds respectively to determine multiple uncertainty detection results of the sample obstacle; Based on the correctness detection result and multiple uncertainty detection results, determine the evaluation scores corresponding to the respective sample uncertainty boundary thresholds; Determine the sample uncertainty boundary threshold with the highest evaluation score as the preset uncertainty boundary threshold.

7. The method according to claim 6, characterized in that, The correctness detection result includes a correct detection result and an incorrect detection result, and the uncertainty detection result includes an uncertain detection result and a certain detection result. Determining the evaluation scores corresponding to the respective sample uncertainty boundary thresholds based on the correctness detection result and multiple uncertainty detection results includes: For each uncertain detection result, determine the first quantity of sample obstacles that satisfy the correct detection result and the certain detection result; determine the second quantity of sample obstacles that satisfy the correct detection result and the uncertain detection result; determine the third quantity of sample obstacles that satisfy the incorrect detection result and the certain detection result; determine the fourth quantity of sample obstacles that satisfy the incorrect detection result and the uncertain detection result; Determine the ratio of the fourth quantity to the sum of the fourth quantity and the third quantity as the coverage rate; determine the ratio of the second quantity to the sum of the fourth quantity and the fourth quantity as the false alarm rate; determine the first ratio of the second quantity to the sum of the second quantity and the first quantity, and determine the ratio of the coverage rate to the first ratio as the quality score; Based on the preset weight distribution coefficient, the coverage rate, the false alarm rate, and the quality score, determine the evaluation score corresponding to the sample uncertainty boundary threshold.

8. The method according to claim 4, wherein The method further includes: Extract the sample spatial geometric features of the sample point cloud clusters corresponding to different sample obstacles in the preset obstacle feature library based on the preset entity degree algorithm; classify the sample point cloud clusters through a preset rule algorithm to obtain the bounding box information and sample entity degree value corresponding to each cluster of sample point clouds; Compare the bounding box information with preset bounding box information to obtain the spatial intersection over union (IoU), and determine the success detection result based on the comparison result between the spatial IoU and the preset spatial matching threshold; determine multiple entity degree detection results based on the comparison result between the sample entity degree value and multiple sample entity degree boundary thresholds; Based on the success detection result and multiple entity degree detection results, determine the respective effective detection rates of each sample entity degree boundary threshold; Determine the sample entity degree boundary threshold with the highest effective detection rate as the preset entity degree boundary threshold.

9. The method according to claim 8, wherein The success detection result includes a successful detection result and an error detection result, and the entity degree detection result includes a high entity degree detection result and a low entity degree detection result. Determining the respective effective detection rates of each sample entity degree boundary threshold based on the success detection result and multiple entity degree detection results includes: For each entity degree detection result, determine the fifth quantity of sample obstacles that satisfy the presence of the sample obstacle in the point cloud cluster and the sample obstacles corresponding to the successful detection result and the low entity degree detection result; determine the sixth quantity of sample obstacles that satisfy the presence of the sample obstacle in the point cloud cluster and the sample obstacles corresponding to the successful detection result and the high entity degree detection result; determine the seventh quantity of sample obstacles that satisfy the absence of the sample obstacle in the point cloud cluster and the error detection result and the low entity degree detection result; determine the eighth quantity of sample obstacles that satisfy the absence of the sample obstacle in the point cloud cluster and the error detection result and the high entity degree detection result; Determine the effective detection rate of the entity by taking the ratio of the sixth quantity to the sum of the fifth quantity and the sixth quantity, and determine the effective detection rate of the non-entity by taking the ratio of the seventh quantity to the sum of the seventh quantity and the eighth quantity; Based on the sum of the effective detection rate of the entity and the effective detection rate of the non-entity, determine the effective detection rate of the sample entity degree boundary threshold.

10. The method according to claim 1, characterized in that The determining the obstacle recognition result based on the matching result, the entity uncertainty risk level, and the obstacle type includes: If the matching result is that the first spatial position information and the second spatial position information exceed the preset spatial matching threshold, determine that the same type of obstacle is detected, and determine the initial recognition result of the obstacle based on the entity uncertainty risk level and the obstacle type; If the matching result is that the first spatial position information and the second spatial position information are lower than the preset spatial matching threshold, determine that different types of obstacles are detected, then determine the initial recognition result of the obstacle corresponding to the first spatial position information based on the risk level, and determine the initial recognition result of the obstacle corresponding to the second spatial position information based on the obstacle type; Determine the initial recognition result that satisfies the preset result screening condition as the obstacle recognition result, and the preset result screening condition is used to retain the initial recognition result that represents the entity obstacle or the initial recognition result of the obstacle corresponding to the first spatial position information.

11. An obstacle recognition device, characterized in that, The device includes: A detection module, configured to detect point cloud data of a target scene based on a preset target detection integrated model, so as to obtain perception information of each obstacle in the target scene; and determine an entity uncertainty risk level of a corresponding obstacle based on the perception information, where the perception information at least includes first spatial position information of each of the obstacles; A determination module, configured to process the point cloud data based on a preset rule algorithm to obtain a point cloud cluster of each obstacle; determine an obstacle type of a corresponding obstacle through spatial geometric features of the point cloud cluster, and determine second spatial position information of the corresponding obstacle based on the spatial position information of the point cloud cluster; An identification module, configured to match the first spatial position information and the second spatial position information to obtain a matching result, and determine an obstacle identification result based on the matching result, the entity uncertainty risk level, and the obstacle type.

12. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 10.

Images