Distributed exterior point cloud object detection method based on fuzzy membership degree supervision and evidence deep learning

Abstract

The invention relates to the field of automatic driving, and provides an out-of-distribution point cloud object detection method based on fuzzy membership supervision and evidence deep learning. Comprising the following steps: obtaining a candidate foreground object by using a 3D target detection model; foreground object point cloud feature extraction; calculating a fuzzy membership degree based on fuzzy C-mean clustering; the fuzzy membership degree value is used as mark information, and evidence deep learning model training is carried out; and inferring an uncertain value of a foreground object by using the trained evidence deep learning model, and judging whether the object belongs to an out-of-distribution sample or not. According to the method, the fuzzy membership degree obtained by fuzzy clustering is used as supervision information, evidence deep learning is used for training on the point cloud target detection data set, and the uncertain value predicted by the evidence deep learning is used for point cloud open set target detection, so that robustness and interpretability are high; and the open set detection effect can be ensured in a large-scale sparse point cloud scene.

Description

Technical Field

[0001] The present application belongs to the field of autonomous driving, and in particular, relates to an out-of-distribution point cloud object detection method based on fuzzy membership supervision and evidence deep learning. Background Art

[0002] Out-of-distribution object detection plays a vital role in the safety and reliability of autonomous driving. Existing point cloud object detection techniques rely on learning from a large amount of data to obtain rich feature representations, which improves the accuracy and robustness of object detection to a certain extent. However, in an open autonomous driving environment, these methods have difficulty dealing with objects of unknown categories that may appear.

[0003] Current out-of-distribution object detection methods usually use dropout variational inference to estimate category uncertainty. This method obtains an approximate estimate of the probability distribution through multiple forward propagations with dropout during the inference phase, and uses the entropy of the estimated distribution to represent the uncertainty value, classifying samples with higher uncertainty into unknown categories. However, this method requires multiple inferences and is difficult to meet the real-time requirements of autonomous driving.

[0004] Another type of method uses feature similarity to estimate uncertainty. However, due to the large variation in point cloud density in autonomous driving scenarios, the feature representations of objects of the same category may differ greatly in similarity, resulting in weak feature consistency for objects of the same category. This density difference poses additional challenges to feature similarity estimation, further affecting the reliability and accuracy of uncertainty assessment. Therefore, how to strike a balance between real-time and accuracy remains a key issue that needs to be addressed in out-of-distribution object detection. Summary of the invention

[0005] The technical problem to be solved by the present invention is to achieve efficient and accurate out-of-distribution object detection in an autonomous driving environment to improve the system's ability to recognize objects of unknown categories, thereby enhancing the safety and robustness of the autonomous driving system.

[0006] To this end, the present invention provides an out-of-distribution point cloud object detection method based on fuzzy membership supervision and evidence deep learning.

[0007] The technical solution adopted by the present invention to solve its technical problem is:

[0008] A method for out-of-distribution point cloud object detection based on fuzzy membership supervision and evidence deep learning, comprising the following steps:

[0009] Step 1: Point cloud data acquisition and preprocessing: Obtain point cloud data from the 3D sensor and preprocess the point cloud, including denoising, filtering, and downsampling, to improve the efficiency and accuracy of subsequent processing;

[0010] Step 2: Foreground object extraction: The preprocessed point cloud data is processed using the voxel-based 3D object detection model PV-RCNN to determine candidate foreground objects, that is, areas that may contain objects of interest;

[0011] Step 3: Feature extraction and representation learning: For the point cloud data of the candidate foreground object, use PointNet to extract features to obtain a d-dimensional feature vector to capture the key information and features of the object;

[0012] Step 4: Feature memory module: Since each frame of point cloud contains a small number of foreground objects and the number of samples is unevenly distributed, a feature memory module is used to maintain a feature vector set for each category. This memory module is updated through a sliding window to accumulate feature information of more samples.

[0013] The output of the feature memory module is provided to step five.

[0014] Step 5: Fuzzy membership calculation and cluster analysis: Use the fuzzy C-means clustering algorithm to cluster the extracted features and calculate the fuzzy membership of each point cloud data point to determine its degree of belonging to different object categories;

[0015] Step 6: Training and optimization of the evidence deep learning model: Use the calculated fuzzy membership value as labeling information, combine it with other attributes of the point cloud data, build an evidence deep learning model, and train and optimize it on the point cloud object detection dataset.

[0016] Step 7: Uncertainty value estimation and out-of-distribution detection: In the reasoning stage, the deep learning model with evidence obtained in step 6 is used to reason about the foreground object, predict its uncertainty value, and determine whether the object is an out-of-distribution sample based on the uncertainty value.

[0017] By adopting the above technical solution, the fuzzy membership obtained by fuzzy clustering is used to alleviate the problem of lack of consideration of fuzzy concepts in manual labeling. This method is not only robust and highly interpretable, but also can ensure the open set detection effect in large-scale sparse point cloud scenarios.

[0018] Furthermore, the optimization process of using the semi-supervised fuzzy C-means clustering algorithm to calculate the fuzzy membership of each foreground object belonging to different categories is as follows: first, given the feature memory module n samples in x i represents the i-th sample, y i Represents the label of the i-th sample, randomly initializes c cluster centers, and uses the matrix Representation, then calculate each sample x i To each cluster center v jThe distance is obtained according to the distance. i The membership degree μ of different clusters ij , where the calculation formula of membership degree is:

[0019]

[0020] Where m is a hyperparameter for adjusting the fuzziness and takes a value greater than 1. After calculating the membership of each sample, the cluster center matrix V is updated again so that Repeat the iterative update of V and U until the cluster center V no longer changes significantly.

[0021] Furthermore, by optimizing the weighted loss function The loss function ψ represents the digamma function,

[0022] Furthermore, in the inference stage, the uncertainty value predicted by the evidence deep learning model is equal to That is, the number of categories / total evidence value. The more evidence values ​​there are, the smaller the uncertainty is, which means that the object is more likely to be a known category, otherwise it is an unknown category.

[0023] Beneficial Effects

[0024] The present invention enhances the estimation accuracy of the evidence deep learning model on out-of-distribution point cloud objects and improves the success rate of out-of-distribution object detection. This method emphasizes that the detection of out-of-distribution objects has been neglected in previous point cloud target detection, and proposes a practical solution. This application makes full use of the impact of the sparse characteristics of the point cloud modality on uncertain estimation to achieve uncertain estimation and out-of-distribution detection of point cloud objects. This innovative method not only excels in detection accuracy, but also has strong interpretability, which helps to enhance the safety of autonomous driving systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0026] Figure 1 It is a processing flow chart of the method of the present invention;

[0027] Figure 2 It is a schematic diagram of data processing of the present invention;

[0028] Figure 3 This is the problem of annotation ambiguity in the autonomous driving point cloud scene proposed by the present invention;

[0029] Figure 4 This is an example diagram of the out-of-distribution detection results of the present invention on the Argoverse2 dataset. DETAILED DESCRIPTION

[0030] The technical solution provided by the present application will be further described below in conjunction with specific embodiments and accompanying drawings. The advantages and features of the present application will become more apparent with the following description.

[0031] The prediction method proposed in this application uses computer technology and deep learning to detect objects outside the autonomous driving distribution. This method combines image processing, data processing, and neural network applications, and is expected to improve the reliability of autonomous driving in complex scenarios.

[0032] A method for out-of-distribution point cloud object detection based on fuzzy membership supervision and evidence deep learning, comprising the following steps: (e.g. Figure 1 , Figure 2 )

[0033] Step 1: Point cloud data collection and preprocessing. Obtain point cloud data from the 3D sensor and preprocess the point cloud, including denoising, filtering, and downsampling, to improve the efficiency and accuracy of subsequent processing.

[0034] Step 2: Foreground object extraction: The preprocessed point cloud data is processed using the voxel-based 3D object detection model PV-RCNN to determine candidate foreground objects, that is, areas that may contain objects of interest.

[0035] The foreground object extraction aims to predict the 3D bounding box (Bounding Box) B = {B1, ..., B N ,}, the i-th 3D bounding box B i The expression method is as follows: B i =[x, y, z, l, w, h, θ, c], where x, y, z are the 3D coordinates of the object, l, w, h are the length, width and height of the object, θ is the azimuth or yaw angle of the object, and c is the object category label.

[0036] PV-RCNN (Point-Voxel Region-based Convolutional Neural Network) is an advanced model for 3D object detection, especially suitable for processing LiDAR point cloud data. It combines voxel-based feature extraction and point cloud-based detail enhancement to improve the detection accuracy and efficiency of the model in 3D scenes. This step aims to effectively identify potential target objects and provide accurate input for subsequent deep learning models.

[0037] Step 3: Feature extraction and representation learning: For the point cloud data of the candidate foreground object, use PointNet to extract features to obtain a d-dimensional feature vector to capture the key information and features of the object.

[0038] PointNet is a deep learning model for processing irregular point cloud data. It can convert point clouds into high-dimensional feature vectors and use them (i.e., feature vectors) as input for subsequent evidence deep learning models. This process aims to convert point cloud data into high-dimensional feature representations with semantic information, providing strong support for subsequent fuzzy membership calculations and training of evidence deep learning models.

[0039] Step 4: Feature memory module: Since the number of foreground objects contained in each frame of point cloud is small and the number of samples is unevenly distributed, if only the sample features in each frame of point cloud are used for training, it may cause deviations in the subsequent fuzzy membership calculation. Therefore, the present invention proposes the use of a feature memory module to maintain a feature vector set for each category, and to update this memory module through a sliding window to accumulate feature information of more samples.

[0040] Specifically, for the kth class of objects, the feature memory module stores n feature vectors of the class, each with a dimension of d. These feature vectors are updated using a sliding window, that is, when the feature vector of a new sample is added to the set, the earliest feature vector is removed, thereby keeping the capacity of the set unchanged. This design can continuously accumulate and update feature information for each class, which helps to alleviate the impact of sample inhomogeneity on fuzzy membership calculation and improve the accuracy of fuzzy membership calculation.

[0041] The output of the feature memory module is provided to step five.

[0042] Step 5: Fuzzy membership calculation and cluster analysis.

[0043] The fuzzy C-means clustering algorithm is used to cluster the extracted features and calculate the fuzzy membership of each point cloud data point to determine its degree of belonging to different object categories.

[0044] Existing point cloud object detection methods use uniform labels to mark the same type of objects with inconsistent point cloud density, but do not consider the ambiguity caused by the difference in point cloud density, such as Figure 3 As shown in the figure, for the two categories of pedestrians and cyclists, the ambiguity between the two categories gradually increases for objects with sparse point cloud density. However, the traditional dataset annotation method assigns these samples to a single category label without considering the ambiguity problem caused by the sparse point cloud.

[0045] The present invention introduces the concept of fuzzy membership, aiming to effectively describe the fuzzy boundaries between point cloud data points and different object categories through a fuzzification method, so as to provide accurate labeling information for the subsequent training of evidence deep learning models.

[0046] Specifically, the semi-supervised fuzzy C-means clustering algorithm is used to calculate the fuzzy membership of each foreground object belonging to different categories. The process is as follows:

[0047] First, given the feature memory module n samples in x i represents the i-th sample, y i Represents the label of the i-th sample and randomly initializes the cluster center matrix

[0048] Then calculate each sample x i To each cluster center v j The distance is obtained according to the distance. i Fuzzy membership μ of different clusters ij , where the calculation formula of fuzzy membership is:

[0049]

[0050] Where m is a hyperparameter for adjusting the fuzziness and takes a value greater than 1, i is the subscript of the sample, and j is the subscript of the category.

[0051] After calculating the membership of each sample, update the cluster center matrix V so that

[0052] Repeat the iterative update of V and U until the cluster center V no longer changes significantly.

[0053] Step 6: Evidence deep learning model training and optimization.

[0054] The calculated fuzzy membership value is used as label information, combined with other attributes of point cloud data, to build an evidence deep learning model, and trained and optimized on the point cloud object detection dataset. This step aims to use supervised learning methods to learn the representation and classification patterns of different object categories from a large amount of point cloud data, and improve the generalization ability and detection accuracy of the model.

[0055] The evidence deep learning model includes: a single-layer neural network, a non-negative activation function and Dirichlet distribution parameter calculation.

[0056] Model input and structure:

[0057] The input of the evidence deep learning model is the d-dimensional feature vector h extracted in step 3. The network structure of the model is a single-layer neural network, and its weight is Where K is the number of categories.

[0058] The output of a single-layer neural network is a K-dimensional vector Then it is activated by the non-negative activation function ReLU to obtain the K-dimensional evidence value vector

[0059] Evidence vector and Dirichlet distribution parameter:

[0060] The idea of ​​evidence deep learning is to add the evidence value vector e to the prior distribution a to obtain the parameters of the Dirichlet distribution Prior distribution is a k-dimensional vector of all 1s, representing the parameters of the Dirichlet distribution without prior information.

[0061] Loss function:

[0062] Loss function of traditional evidence deep learning: Evidence deep learning assumes that the predicted classification probability p is a result sampled from the predicted Dirichlet distribution Dir(α). The loss function of evidence deep learning is to calculate the cross entropy loss between the sampled p and the label y, and calculate its expected value with respect to the Dirichlet distribution. The specific form is as follows:

[0063]

[0064] in ψ represents the digamma function, and y is the label vector in the form of K-dimensional one-hot code. By minimizing the above loss function, the model training of evidence deep learning is realized.

[0065] Improved loss function: This paper comprehensively considers the fuzzy membership and manual category labeling information, and proposes an optimized weighted loss function as follows:

[0066]

[0067] Wherein, β is a weight, which is taken as 1 in this embodiment; the loss function considers the fuzzy membership and the manual category labeling information as follows:

[0068]

[0069]

[0070] ψ represents the digamma function, y is the label vector in K-dimensional one-hot encoding form; μ is the fuzzy membership, and j is the subscript of the category.

[0071] The evidential deep learning model is trained using the above loss function until the network reaches the desired stable state.

[0072] Step 7: Uncertain value estimation and out-of-distribution detection.

[0073] In the reasoning phase, the evidence deep learning model obtained in step 6 is used to reason about the foreground object, predict its uncertainty value, and determine whether the object belongs to an out-of-distribution sample based on the uncertainty value, that is, whether it is an unknown object or an abnormal object. This step aims to quantitatively evaluate the confidence of the model in the detection target, thereby improving the credibility and robustness of the detection results and ensuring the open set detection effect in large-scale sparse point cloud scenarios.

[0074] Furthermore, the uncertainty value is equal to That is, the number of categories / total evidence value. The more evidence values ​​there are, the smaller the uncertainty is, which means that the object is more likely to be a known category, otherwise it is an unknown category.

[0075] In order to determine whether a sample belongs to an unknown category, an uncertainty threshold τ is set. Samples with a predicted uncertainty value greater than τ are predicted to be of unknown category, and samples with a predicted uncertainty value less than τ are predicted to be of known category.

[0076] The method for setting the uncertainty threshold τ is as follows: the distribution of uncertainty values ​​of all samples in the training set is counted, and the uncertainty value τ greater than 95% of the samples is taken as the threshold.

[0077] In order to verify the effectiveness of the method of the present invention, an out-of-distribution point cloud target detection experiment was carried out on the open source autonomous driving dataset Argovese2. The Argovese2 dataset has a total of 28 annotated categories. In order to verify its ability to detect unknown categories, the experiment uses 15 categories such as "ordinary vehicles", "pedestrians", "bollards", "construction cones", "bicycles", "large vehicles", "buses", "box trucks", "trucks", "motorcycles", "cyclists", "motorcyclists", "dogs", "school buses" and "articulated buses" as known categories; and the remaining 13 categories such as "construction barrels", "stop signs", "wheeled devices", "signboards", "trailer vehicles", "truck cabs", "wheeled riders", "strollers", "information board trailers", "mobile pedestrian signs", "wheelchairs", "rail vehicles", and "traffic light trailers" are used as unknown categories. The example figure of the out-of-distribution detection result is as follows Figure 4 .

[0078] The performance comparison results of the proposed method and related methods on the Argovese2 dataset are shown in the following table, where FRP@95, AUROC and AURP are commonly used out-of-distribution detection indicators. The lower the FRP@95 indicator, the better, and the higher the AUROC and AUPR indicators, the better. It can be seen that the proposed method surpasses the classic out-of-distribution detection method in the point cloud out-of-distribution object detection task.

[0079] method FPR@95 AUROC AUPR Focal 70.34 62.16 10.99 MSP 100.00 52.44 9.98 MLS 100.00 53.53 10.32 OpemMax 100.00 54.84 10.74 Cosine Similarity 65.40 76.84 23.89 VOS 63.12 72.39 19.97 Method of the present invention 60.85 85.83 53.35

[0080] The above description is only a description of the preferred embodiments of the present application, and is not intended to limit the scope of the present application. Any changes or modifications made by any person skilled in the art based on the above disclosed technical contents shall be deemed as equivalent effective embodiments and shall fall within the scope of protection of the technical solution of the present application.

Claims

1. A method for out-of-distribution point cloud object detection based on fuzzy membership supervision and evidence deep learning, characterized in that: The following steps are involved: Step 1: Point cloud data acquisition and preprocessing: Obtain point cloud data from the 3D sensor and preprocess the point cloud, including denoising, filtering, and downsampling, to improve the efficiency and accuracy of subsequent processing; Step 2: Foreground object extraction: The preprocessed point cloud data is processed using the voxel-based 3D object detection model PV-RCNN to determine candidate foreground objects, that is, areas that may contain objects of interest; Step 3: Feature extraction and representation learning: For the point cloud data of the candidate foreground object, use PointNet to extract features to obtain a d-dimensional feature vector to capture the key information and features of the object; Step 4: Feature memory module: Since each frame of point cloud contains a small number of foreground objects and the number of samples is unevenly distributed, a feature memory module is used to maintain a feature vector set for each category. This memory module is updated through a sliding window to accumulate feature information of more samples. The output of the feature memory module is provided to step five; Step 5: Fuzzy membership calculation and cluster analysis: Use the fuzzy C-means clustering algorithm to cluster the extracted features and calculate the fuzzy membership of each point cloud data point to determine its degree of belonging to different object categories; Step 6: Training and optimization of the evidence deep learning model: The calculated fuzzy membership value is used as the label information, combined with other attributes of the point cloud data, to build the evidence deep learning model, and then trained and optimized on the point cloud object detection dataset; Step 7: Uncertainty value estimation and out-of-distribution detection: In the reasoning stage, the deep learning model with evidence obtained in step 6 is used to reason about the foreground object, predict its uncertainty value, and determine whether the object is an out-of-distribution sample based on the uncertainty value.

2. The out-of-distribution point cloud object detection method based on fuzzy membership supervision and evidence deep learning as claimed in claim 1, characterized in that: In step 5, the semi-supervised fuzzy C-means clustering algorithm is used to calculate the fuzzy membership of each foreground object to different categories. The process is as follows: First, given the feature memory module n samples in x i represents the i-th sample, y i Represents the label of the i-th sample, randomly initializes c cluster centers, and uses the matrix To express; Then calculate each sample x i To each cluster center v j The distance is obtained according to the distance. i Fuzzy membership μ of different clusters ij , where the calculation formula of fuzzy membership is: Where m is a hyperparameter for adjusting the fuzziness, which takes a value greater than 1, i is the subscript of the sample, and j is the subscript of the category; After calculating the membership of each sample, update the cluster center matrix V so that Repeat the iterative update of V and U until the cluster center V no longer changes significantly.

3. The out-of-distribution point cloud object detection method based on fuzzy membership supervision and evidence deep learning as claimed in claim 1, characterized in that: In step 6, the evidence deep learning model includes: a single-layer neural network, a non-negative activation function, and Dirichlet distribution parameter calculation; The input of the evidence deep learning model is the d-dimensional feature vector h extracted in step 3. The network structure of the model is a single-layer neural network, and its weight is Where K is the number of categories; The output of a single-layer neural network is a K-dimensional vector Then it is activated by the non-negative activation function ReLU to obtain the K-dimensional evidence value vector Add the evidence value vector e to the prior distribution a to get the parameters of the Dirichlet distribution Prior distribution is a k-dimensional vector of all 1s, representing the parameters of the Dirichlet distribution without prior information.

4. The out-of-distribution point cloud object detection method based on fuzzy membership supervision and evidence deep learning as claimed in claim 3, characterized in that: The loss function of the evidence deep learning model is as follows: in, Among them, β is the weight, ψ represents the digamma function, y is the label vector in K-dimensional one-hot encoding form; μ is the fuzzy membership, and j is the subscript of the category.

5. The out-of-distribution point cloud object detection method based on fuzzy membership supervision and evidence deep learning as claimed in claim 1, characterized in that: In step seven, The uncertainty value predicted by the evidence deep learning model is equal to That is, number of categories / total evidence value; In order to determine whether a sample belongs to an unknown category, an uncertainty threshold τ is set; samples with predicted uncertainty values ​​greater than τ are predicted to be of unknown category, and samples with predicted uncertainty values ​​less than τ are predicted to be of known category.

6. The out-of-distribution point cloud object detection method based on fuzzy membership supervision and evidence deep learning as claimed in claim 5, characterized in that: The method for setting the uncertainty threshold τ is as follows: the distribution of uncertainty values ​​of all samples in the training set is counted, and the uncertainty value τ greater than 95% of the samples is taken as the threshold.

Images