An out-of-distribution point cloud object detection method based on fuzzy membership supervision and evidence-based deep learning

Abstract

This application relates to the field of autonomous driving and proposes an out-of-distribution point cloud object detection method based on fuzzy membership supervision and evidence deep learning. The method includes the following steps: obtaining candidate foreground objects using a 3D object detection model; extracting foreground object point cloud features; calculating fuzzy membership based on fuzzy C-means clustering; training an evidence deep learning model using the fuzzy membership values ​​as labeling information; and using the trained evidence deep learning model to infer the uncertainty values ​​of the foreground objects and determine whether the objects belong to out-of-distribution samples. This application utilizes fuzzy membership obtained from fuzzy clustering as supervision information, trains evidence deep learning on a point cloud object detection dataset, and uses the uncertainty values ​​predicted by evidence deep learning for open-set point cloud object detection. This method is not only robust and highly interpretable but also ensures open-set detection performance in large-scale sparse point cloud scenarios.

Description

Technical Field

[0001] This application belongs to the field of autonomous driving, and in particular relates to a method for detecting out-of-distribution point cloud objects based on fuzzy membership supervision and evidence deep learning. Background Technology

[0002] Out-of-distribution object detection plays a crucial role in the safety and reliability of autonomous driving. Existing point cloud object detection techniques rely on learning from large amounts of data to obtain rich feature representations, thereby improving the accuracy and robustness of object detection to some extent. However, in open autonomous driving environments, these methods struggle to handle potentially unknown object categories.

[0003] Current out-of-distribution object detection methods typically use Dropout variational inference to estimate category uncertainty. This approach obtains an approximate estimate of the probability distribution through multiple forward propagations with Dropout during the inference phase, and uses the entropy of the estimated distribution to represent the uncertainty value, classifying samples with high uncertainty into unknown categories. However, this method requires multiple inferences, making it difficult to meet the real-time requirements of autonomous driving.

[0004] Another type of method estimates uncertainty through feature similarity. However, due to the significant variations in point cloud density in autonomous driving scenarios, the feature representations of similar objects may differ considerably in similarity, leading to weak feature consistency among objects of the same category. This density difference presents additional challenges to feature similarity estimation, further impacting the reliability and accuracy of uncertainty assessment. Therefore, achieving a balance between real-time performance and accuracy remains a critical issue that urgently needs to be addressed in out-of-distribution object detection. Summary of the Invention

[0005] The technical problem to be solved by this invention is to achieve efficient and accurate detection of distributed external objects in an autonomous driving environment, so as to improve the system's ability to identify unknown types of objects, thereby enhancing the safety and robustness of the autonomous driving system.

[0006] Therefore, this invention provides a method for detecting out-of-distribution point cloud objects based on fuzzy membership supervision and evidence deep learning.

[0007] The technical solution adopted by the present invention to solve its technical problem is:

[0008] An out-of-distribution point cloud object detection method based on fuzzy membership supervision and evidence deep learning includes the following steps:

[0009] Step 1: Point cloud data acquisition and preprocessing; acquire point cloud data from 3D sensors, and preprocess the point cloud, including noise reduction, filtering and downsampling, to improve the efficiency and accuracy of subsequent processing;

[0010] Step 2: Foreground object extraction; The preprocessed point cloud data is processed using the voxel-based 3D object detection model PV-RCNN to identify candidate foreground objects, i.e., regions that may contain objects of interest.

[0011] Step 3: Feature Extraction and Representation Learning: For the point cloud data of candidate foreground objects, PointNet is used to extract features to obtain d-dimensional feature vectors in order to capture the key information and features of the objects;

[0012] Step 4: Feature Memory Module: Since the number of foreground objects in each frame of point cloud is small and the number of samples is unevenly distributed, a feature memory module is used to maintain a set of feature vectors for each category. This memory module is updated by a sliding window to accumulate feature information of more samples.

[0013] The output of the feature memory module is provided to step five.

[0014] Step 5: Fuzzy membership degree calculation and cluster analysis: The extracted features are clustered using the fuzzy C-means clustering algorithm. The fuzzy membership degree of each point cloud data point is calculated to determine its degree of belonging to different object categories.

[0015] Step Six: Training and Optimization of Evidence Deep Learning Model: Using the calculated fuzzy membership values ​​as labeling information, and combining them with other attributes of the point cloud data, an evidence deep learning model is constructed and trained and optimized on the point cloud object detection dataset.

[0016] Step 7: Uncertainty value estimation and out-of-distribution detection: In the reasoning stage, the evidence deep learning model obtained in Step 6 is used to reason about the foreground object, predict its uncertainty value, and determine whether the object belongs to the out-of-distribution sample based on the uncertainty value.

[0017] By adopting the above technical solution, the fuzzy membership degree obtained by fuzzy clustering is used to alleviate the problem of lack of consideration for fuzzy concepts in manual annotation. This method is not only robust and highly interpretable, but also ensures the open set detection effect in large-scale sparse point cloud scenarios.

[0018] Furthermore, the optimization process for calculating the fuzzy membership degree of each foreground object belonging to different categories using the semi-supervised fuzzy C-means clustering algorithm is as follows: First, given the feature memory module... n samples, x i Let y represent the i-th sample. i Let c represent the label of the i-th sample, and randomly initialize c cluster centers using a matrix. This is represented, and then x is calculated for each sample. i to each cluster center v jThe distance is used to obtain the sample x. i Membership degree μ of different clusters ij The formula for calculating membership degree is:

[0019]

[0020] Where m is a hyperparameter for adjusting ambiguity, taking a value greater than 1. After calculating the membership degree of each sample, the cluster center matrix V is updated again, such that... Repeatedly update V and U until the cluster center V no longer changes significantly.

[0021] Furthermore, by optimizing the weighted loss function Where the loss function ψ represents the digamma function.

[0022] Furthermore, in the inference phase, the uncertainty value predicted by the evidence deep learning model is equal to... That is, the number of categories divided by the total evidence value. The more evidence value, the less uncertainty, indicating that the object is more likely to belong to a known category, and vice versa.

[0023] Beneficial effects

[0024] This invention enhances the estimation accuracy of evidence-based deep learning models for out-of-distribution point cloud objects, thereby improving the success rate of out-of-distribution object detection. This method addresses the neglect of out-of-distribution object detection in previous point cloud object detection methods and proposes a practical solution. This application fully utilizes the impact of the sparsity characteristics of point cloud modalities on uncertain estimation to achieve uncertain estimation and out-of-distribution detection of point cloud objects. This innovative method not only demonstrates superior detection accuracy but also possesses strong interpretability, contributing to enhanced safety in autonomous driving systems. Attached Figure Description

[0025] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0026] Figure 1 This is a flowchart of the method of the present invention;

[0027] Figure 2 This is a schematic diagram of the data processing of the present invention;

[0028] Figure 3 This invention addresses the annotation ambiguity problem in the autonomous driving point cloud scenario.

[0029] Figure 4 This is an example image of the out-of-distribution detection results of this invention on the Argoverse2 dataset. Detailed Implementation

[0030] The technical solutions provided in this application will be further described below with reference to specific embodiments and accompanying drawings. The advantages and features of this application will become clearer from the following description.

[0031] The prediction method proposed in this application uses computer technology and deep learning to detect objects outside the distribution of autonomous driving systems. This method integrates image processing, data processing, and neural network applications, and is expected to improve the reliability of autonomous driving in complex scenarios.

[0032] An out-of-distribution point cloud object detection method based on fuzzy membership supervision and evidence deep learning includes the following steps: (e.g.) Figure 1 , Figure 2 )

[0033] Step 1: Point cloud data acquisition and preprocessing. Point cloud data is acquired from the 3D sensor and preprocessed, including noise reduction, filtering, and downsampling, to improve the efficiency and accuracy of subsequent processing.

[0034] Step 2: Foreground Object Extraction. The preprocessed point cloud data is processed using the voxel-based 3D object detection model PV-RCNN to identify candidate foreground objects, i.e., regions that may contain objects of interest.

[0035] Foreground object extraction aims to predict the 3D bounding box B = {B1, ..., B2} of foreground objects using the acquired input point cloud information. N ,}, the i-th 3D bounding box B i The representation method is as follows: B i = [x, y, z, l, w, h, θ, c], where x, y, z are the 3D coordinates of the object, l, w, h are the length, width, and height of the object, θ is the azimuth or yaw angle of the object, and c is the object category label.

[0036] PV-RCNN (Point-Voxel Region-based Convolutional Neural Network) is an advanced model for 3D object detection, particularly suitable for processing LiDAR point cloud data. It combines voxel-based feature extraction with point cloud-based detail enhancement, improving the model's detection accuracy and efficiency in 3D scenes. This step aims to effectively identify potential target objects, providing accurate input for subsequent deep learning models.

[0037] Step 3: Feature Extraction and Representation Learning: For the point cloud data of candidate foreground objects, PointNet is used to extract features to obtain d-dimensional feature vectors in order to capture the key information and features of the objects.

[0038] PointNet is a deep learning model for processing irregular point cloud data. It transforms point clouds into high-dimensional feature vectors, which are then used as input to subsequent evidence-based deep learning models. This process aims to convert point cloud data into high-dimensional feature representations with semantic information, providing strong support for subsequent fuzzy membership calculations and the training of evidence-based deep learning models.

[0039] Step 4: Feature Memory Module: Since each frame of the point cloud contains a small number of foreground objects and the sample quantity is unevenly distributed, training using only the features from each frame's points may lead to biases in subsequent fuzzy membership calculations. Therefore, this invention proposes using a feature memory module to maintain a set of feature vectors for each category, and accumulating more feature information from samples by updating this memory module through a sliding window.

[0040] Specifically, for the k-th class of objects, the feature memory module stores n feature vectors for that class, each with dimension d. These feature vectors are updated using a sliding window approach; that is, when a new sample's feature vector is added to the set, the oldest feature vector is removed, thus maintaining the set's capacity. This design continuously accumulates and updates feature information for each class, helping to mitigate the impact of sample heterogeneity on fuzzy membership calculation and improving its accuracy.

[0041] The output of the feature memory module is provided to step five.

[0042] Step 5: Fuzzy membership degree calculation and cluster analysis.

[0043] The fuzzy C-means clustering algorithm is used to cluster the extracted features and calculate the fuzzy membership degree of each point cloud data point to determine its degree of belonging to different object categories.

[0044] Existing point cloud object detection methods use a uniform label to annotate the same type of objects with inconsistent point cloud density, but they do not consider the ambiguity caused by differences in point cloud density. Figure 3 As shown, for the two categories of pedestrians and cyclists, the ambiguity between these two categories gradually increases for objects with relatively sparse point cloud density. However, traditional dataset labeling methods assign these samples a single category label without considering the ambiguity caused by the sparse point cloud.

[0045] This invention introduces the concept of fuzzy membership, aiming to effectively describe the fuzzy boundaries between point cloud data points and different object categories through fuzzification, thereby providing accurate labeling information for the subsequent training of evidence-based deep learning models.

[0046] Specifically, a semi-supervised fuzzy C-means clustering algorithm is used to calculate the fuzzy membership degree of each foreground object to different categories. The process is as follows:

[0047] First, given the feature memory module n samples, x i Let y represent the i-th sample. i The label of the i-th sample is represented by the cluster center matrix, which is randomly initialized.

[0048] Then calculate x for each sample. i to each cluster center v j The distance is used to obtain the sample x. i Fuzzy membership degree μ belonging to different clusters ij The formula for calculating fuzzy membership degree is:

[0049]

[0050] Where m is a hyperparameter for adjusting ambiguity, taking a value greater than 1, i is the sample index, and j is the category index.

[0051] After calculating the membership degree of each sample, the cluster center matrix V is updated so that...

[0052] Repeatedly update V and U until the cluster center V no longer changes significantly.

[0053] Step Six: Training and Optimization of Evidence-Based Deep Learning Models.

[0054] The calculated fuzzy membership values ​​are used as labeling information, combined with other attributes of the point cloud data, to construct an evidence-based deep learning model, which is then trained and optimized on a point cloud object detection dataset. This step aims to utilize supervised learning methods to learn representations and classification patterns of different object categories from a large amount of point cloud data, thereby improving the model's generalization ability and detection accuracy.

[0055] The evidence deep learning model includes: a single-layer neural network, a non-negative activation function, and the calculation of Dirichlet distribution parameters.

[0056] Model input and structure:

[0057] The input to the evidence deep learning model is the d-dimensional feature vector h extracted in step three. The model's network structure is a single-layer neural network, and its weights are... Where K is the number of categories.

[0058] A single-layer neural network outputs a K-dimensional vector. Then, it is activated by the non-negative activation function ReLU to obtain a K-dimensional evidence value vector.

[0059] Evidence vector and Dirichlet distribution parameters:

[0060] The idea behind evidence-based deep learning is to add the prior distribution a to the evidence value vector e to obtain the parameters of the Dirichlet distribution. Prior distribution It is a k-dimensional vector of all 1s, representing the parameters of a Dirichlet distribution without prior information.

[0061] Loss function:

[0062] The loss function of traditional evidence-based deep learning: Evidence-based deep learning assumes that the predicted classification probability p is a single sample from the predicted Dirichlet distribution Dir(α). The loss function of evidence-based deep learning is to calculate the cross-entropy loss between the sampled p and the label y, and to calculate its expected value with respect to the Dirichlet distribution, as follows:

[0063]

[0064] in ψ represents the digamma function, and y is a label vector in K-dimensional one-hot code form. The model training for evidence-based deep learning is achieved by minimizing the above loss function.

[0065] Improved Loss Function: This invention comprehensively considers fuzzy membership degree and manual category labeling information, and proposes an optimized weighted loss function as follows:

[0066]

[0067] Where β is the weight, which is set to 1 in this embodiment; the loss function considers the fuzzy membership degree and the manually labeled information as follows:

[0068]

[0069]

[0070] ψ represents the digamma function. y is a label vector in K-dimensional one-hot code form; μ is the fuzzy membership degree, and j is the subscript of the category.

[0071] The above loss function is used to train the evidence deep learning model until the network reaches the expected stable state.

[0072] Step 7: Uncertainty estimation and out-of-distribution detection.

[0073] In the inference phase, the evidence-based deep learning model obtained in step six is ​​used to infer the foreground object, predict its uncertainty value, and determine whether the object belongs to an out-of-distribution sample, i.e., whether it is an unknown object or an anomalous object, based on the uncertainty value. This step aims to evaluate the model's confidence in the detected target through quantification, thereby improving the credibility and robustness of the detection results and ensuring the effectiveness of open set detection in large-scale sparse point cloud scenarios.

[0074] Furthermore, the uncertain value is equal to That is, the number of categories divided by the total evidence value. The more evidence value, the less uncertainty, indicating that the object is more likely to belong to a known category, and vice versa.

[0075] To determine whether a sample belongs to an unknown category, an uncertainty threshold τ is set. Samples with a predicted uncertainty value greater than τ are predicted to belong to the unknown category, while samples with an uncertainty value less than τ are predicted to belong to the known category.

[0076] The method for setting the uncertainty threshold τ is as follows: statistically analyze the distribution of uncertainty values ​​of all samples on the training set, and take the uncertainty value τ of more than 95% of the samples as the threshold.

[0077] To verify the effectiveness of the method of this invention, out-of-distribution point cloud target detection experiments were conducted on the open-source autonomous driving dataset Argovese2. The Argovese2 dataset has 28 labeled categories. To verify its ability to detect unknown categories, the experiment used 15 categories as known categories: "Ordinary Vehicles," "Pedestrians," "Boats," "Construction Cones," "Bicycles," "Large Vehicles," "Buses," "Box Trucks," "Trucks," "Motorcycles," "Bicycle Riders," "Motorcycle Riders," "Dogs," "School Buses," and "Articulated Buses." The remaining 13 categories—"Construction Buckets," "Stop Signs," "Wheeled Vehicles," "Signal Vehicles," "Truck Cabs," "Wheel Riders," "Baby Strollers," "Information Board Trailers," "Moving Pedestrian Signs," "Wheelchairs," "Tram Vehicles," and "Traffic Light Trailers"—were used as unknown categories. Example figures of out-of-distribution detection results are shown below. Figure 4 .

[0078] The performance comparison results of the proposed method and related methods on the Argovese2 dataset are shown in the table below. FRP@95, AUROC, and AURP are commonly used out-of-distribution detection metrics. A lower FRP@95 is better, while higher AUROC and AURP are better. It can be seen that the proposed method outperforms classic out-of-distribution detection methods on the point cloud out-of-distribution object detection task.

[0079] method FPR@95 AUROC AUPR Focal 70.34 62.16 10.99 MSP 100.00 52.44 9.98 MLS 100.00 53.53 10.32 OpemMax 100.00 54.84 10.74 Cosine Similarity 65.40 76.84 23.89 VOS 63.12 72.39 19.97 Method of the present invention 60.85 85.83 53.35

[0080] The above description is merely a description of preferred embodiments of this application and is not intended to limit the scope of this application in any way. Any changes or modifications made by those skilled in the art based on the above-disclosed technical content should be considered as equivalent and valid embodiments and fall within the scope of protection of the technical solution of this application.

Claims

1. A method for detecting out-of-distribution point cloud objects based on fuzzy membership supervision and evidence-based deep learning, characterized in that, Includes the following steps: Step 1: Point cloud data acquisition and preprocessing; acquire point cloud data from 3D sensors, and preprocess the point cloud, including noise reduction, filtering and downsampling operations, to improve the efficiency and accuracy of subsequent processing; Step 2: Foreground object extraction; The preprocessed point cloud data is processed using the voxel-based 3D object detection model PV-RCNN to identify candidate foreground objects, i.e., regions that may contain objects of interest. Step 3: Feature Extraction and Representation Learning: For the point cloud data of candidate foreground objects, PointNet is used for feature extraction. 3D feature vectors to capture key information and features of an object; Step 4: Feature Memory Module: Since the number of foreground objects in each frame of point cloud is small and the number of samples is unevenly distributed, a feature memory module is used to maintain a set of feature vectors for each category. This memory module is updated by a sliding window to accumulate feature information of more samples. The output of the feature memory module is provided to step five; Step 5: Fuzzy membership degree calculation and cluster analysis: The extracted features are clustered using the fuzzy C-means clustering algorithm. The fuzzy membership degree of each point cloud data point is calculated to determine its degree of belonging to different object categories. Step Six: Training and Optimization of Evidence Deep Learning Model: Using the calculated fuzzy membership values ​​as labeling information, combined with other attributes of the point cloud data, an evidence deep learning model is constructed and trained and optimized on the point cloud object detection dataset; Step 7: Uncertainty value estimation and out-of-distribution detection: In the reasoning stage, the evidence deep learning model obtained in Step 6 is used to reason about the foreground object, predict its uncertainty value, and determine whether the object belongs to the out-of-distribution sample based on the uncertainty value.

2. The out-of-distribution point cloud object detection method based on fuzzy membership supervision and evidence deep learning as described in claim 1, characterized in that, In step five, a semi-supervised fuzzy C-means clustering algorithm is used to calculate the fuzzy membership degree of each foreground object belonging to different categories. The process is as follows: Then calculate each sample To each cluster center The distance is used to obtain samples. Fuzzy membership degree of different clusters The formula for calculating fuzzy membership degree is: in This is a hyperparameter for adjusting fuzziness; it should be a value greater than 1. The subscript of the sample. Subscript for category; After calculating the membership degree of each sample, the cluster center matrix is ​​updated. , making Repeated iterative updates until the cluster center is satisfied. No further significant changes occurred.

3. The out-of-distribution point cloud object detection method based on fuzzy membership supervision and evidence deep learning as described in claim 1, characterized in that, In step six, the evidence deep learning model includes: a single-layer neural network, a non-negative activation function, and the calculation of Dirichlet distribution parameters; The input to the evidence deep learning model is the data extracted in step three. 3D feature vector The model's network structure is a single-layer neural network, and its weights are... ,in It is the number of categories; A single-layer neural network outputs one dimensional vector Then, it is activated by the non-negative activation function ReLU to obtain 3D evidence value vector ; Prior distribution It is A vector of dimension 1 represents the parameters of a Dirichlet distribution without prior information.

4. The out-of-distribution point cloud object detection method based on fuzzy membership supervision and evidence deep learning as described in claim 3, characterized in that, The loss function of the evidence deep learning model is as follows: Label vectors in one-hot encoding form; For fuzzy membership, The index is for the category.

5. The out-of-distribution point cloud object detection method based on fuzzy membership supervision and evidence deep learning as described in claim 1, characterized in that, In step seven, To determine whether a sample belongs to an unknown category, an uncertainty threshold is set. ; The uncertainty of the prediction is greater than The sample prediction decision is an unknown category, less than The sample is classified as a known category.

6. The out-of-distribution point cloud object detection method based on fuzzy membership supervision and evidence deep learning as described in claim 5, characterized in that, Uncertain threshold Setup method: Statistically analyze the distribution of uncertainty values ​​for all samples on the training set, and take the uncertainty value greater than 95% of the samples. As a threshold.

Images