An unbiased pseudo-label learning method for semi-supervised three-dimensional target detection

Abstract

The application discloses a kind of semi-supervised three-dimensional target detection-oriented unbiased pseudo-label learning method, it is related to computer vision and three-dimensional perception technology, step is introduced confidence calibration, distribution uncertainty suppression and class re-weighting modulation to enhance tail class contribution;While combining dynamicIt solves the problem that the prior art under the condition of labeled data scarcity, due to weak small object perception ability and long-tail distribution of class, low pseudo-label quality, poor tail class recognition rate problem caused by.The application introduces local geometry branch in the full-supervised pre-training phase to enhance the small object local shape representation ability, and introduces class balance weight in semantic classification loss to alleviate the gradient flooding of long-tail class;In the semi-supervised training stage, a quality-aware pseudo-label generation and screening strategy is proposed, a quality score is constructed based on foreground confidence, class confidence and predicted intersection over union, and a screening mechanism of threshold constraint, hierarchical Top-K class quota and overlap suppression is further added to realize the unbiased selection of high-quality pseudo-labels.

Description

Technical Field

[0001] This invention relates to computer vision and 3D perception technology, specifically to an unbiased pseudo-label learning method for semi-supervised 3D object detection, which is applicable to practical application scenarios such as autonomous driving and intelligent robots where point cloud data is abundant but labeling costs are high. Background Technology

[0002] With the widespread adoption of LiDAR and depth sensors, 3D point clouds are increasingly used in autonomous driving, augmented reality, and intelligent robotics. However, accurately annotating 3D bounding boxes is not only time-consuming and labor-intensive but also requires a high level of expertise from the annotators, making it difficult to obtain large-scale labeled data. To alleviate the reliance on fully supervised training, semi-supervised 3D object detection methods have gained increasing attention in recent years. Among them, 3DIoUMatch is a representative work that generates pseudo-labels through a teacher-student framework and introduces the intersection-over-union ratio (3D IoU) of the predicted 3D bounding boxes as a location confidence metric to filter out low-quality predictions, significantly improving the performance of semi-supervised methods.

[0003] Despite this, existing methods still face significant challenges in unbiased learning under conditions of scarce labeled data: real-world 3D scene data generally exhibits a significant long-tail distribution—head categories such as "cars" and "pedestrians" constitute the vast majority of samples, while tail categories such as "traffic cones" and "trash cans" have extremely few samples. More importantly, tail categories contain a large number of small-scale objects with sparse point clouds and weak geometric structures. During feature abstraction processes such as voting aggregation, they are easily dominated by large objects, severely diluting local details and resulting in low localization accuracy and weak classification confidence. This makes it difficult for tail category samples to meet the confidence threshold during the pseudo-label generation stage, and even if they exist, they are often filtered out.

[0004] Furthermore, current pseudo-label selection mechanisms mostly use the product of foreground confidence and class confidence as the ranking criterion. This strategy is extremely harsh on tail classes: any low confidence level will cause the overall score to drop sharply, forming a significant bottleneck effect and limiting the diversity and coverage of pseudo-labels. Summary of the Invention

[0005] The technical problem to be solved by this invention is to propose an unbiased learning method for semi-supervised 3D object detection, which solves the problems of low pseudo-label quality and low tail class recognition rate caused by the weak perception ability of the model for small objects and the long tail distribution of the dataset under the condition of scarce labeled data. This improves the detection performance of the model in tail categories and the overall semi-supervised accuracy.

[0006] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0007] An unbiased pseudo-label learning method for semi-supervised 3D object detection includes the following steps:

[0008] Step 1: Acquire 3D point cloud scene data and perform preprocessing;

[0009] Step 2: Construct the detection backbone network and introduce a local geometry branch: The detection backbone network extracts features from the input point cloud to obtain seed point coordinates and seed point features; then, through the candidate box generation module, the seed points are voted to candidate centers using a voting mechanism, and then aggregated to generate candidate box center coordinates and candidate box features; the local geometry branch encodes the local neighborhood geometric information of each candidate box center in the original point cloud and generates local geometric features that match the dimensions of the candidate box features; the local geometric features and candidate box features are fused in a residual manner to obtain the enhanced features;

[0010] Step 3: Calculate the category balance weights: count the number of 3D bounding boxes for each semantic category, and calculate the category balance weights and their normalized forms;

[0011] Step 4: Teacher-student network initialization: Using gradient descent, the parameters of the detection backbone network, local geometric branches, and classification and regression heads are optimized under supervision using 3D bounding boxes and semantic category labels as supervision signals to obtain a pre-trained detection model;

[0012] The pre-trained detection model is replicated as a teacher network and a student network; the gradient of the teacher network is frozen during training, while the student network participates in gradient updates.

[0013] Step 5: Hybrid Batch Construction and Forward Inference: In each training batch, labeled and unlabeled data are sampled simultaneously; labeled data is used to supervise the student network; for unlabeled data, its strongly enhanced version is input into the student network to obtain the prediction result, and its weakly enhanced version is input into the teacher network to generate the prediction result required for the pseudo-label;

[0014] Step 6: Quality-Aware Pseudo-Label Generation and Screening: For each candidate bounding box of an unlabeled sample, calculate the basic linear quality score, interaction consistency term, and category distribution entropy penalty term based on the output of the teacher network, and combine them to obtain a comprehensive quality score; the comprehensive quality score is then augmented at the tail based on category balancing weights and smoothing modulation to obtain a weighted quality score; using the weighted quality score as the ranking criterion, and selecting Top-... under the premise of satisfying the basic confidence mask. Pseudo-labels; Align the filtered pseudo-labels with the student's online input in the same coordinate system;

[0015] Step 7: Pseudo-label supervision and parameter update: The filtered and aligned pseudo-labels are used to supervise the unlabeled predictions generated by the student network; the student network parameters are updated using gradient descent, and the teacher network parameters are updated using exponential moving average to complete iterative training.

[0016] Furthermore, step 2 includes the following steps:

[0017] Step 201: Construct a detection backbone network; the detection backbone network extracts features from the input point cloud to obtain seed point coordinates and seed point features; the candidate box generation module votes the seed points to the candidate centers through a voting mechanism, generating voting points and voting point features, and aggregating the voting points to obtain... Center of each candidate bounding box and its features ;

[0018] Step 202: Before the candidate box features output by the candidate box generation module enter the classification and regression head, a local geometry branch is introduced. The local geometry branch outputs local geometric features. With candidate box features The enhanced features are obtained by fusing using a residual method; the implementation process of the local geometric branch includes:

[0019] In the preprocessed point cloud, centered at the center of each candidate box With the center of the sphere as the radius, The ball query operator obtains the local neighborhood point set. ;

[0020] In the local neighborhood point set The process uses random sampling or nearest neighbor sampling to obtain at most 50 neighboring points, denoted as . ;when At that time, padding with zero vectors to 50 points is performed, and the effective point mask is recorded. The mask value corresponding to the real nearest point is 1, and the mask value corresponding to the padding point is 0.

[0021] Calculate the relative coordinates of neighboring points with respect to the center of the candidate box: ;

[0022] Each candidate box Corresponding neighborhood relative coordinates The input is a shared 3-layer multilayer perceptron, which performs point-by-point feature transformation to obtain local features. ;

[0023] The local geometric representation is obtained by performing masked max pooling aggregation on the local features of the 50 nearest neighbors. Subsequently, the local geometric representation will be... Aligned with candidate box features via linear mapping.

[0024] Furthermore, the category balance weights and their normalized forms are calculated as follows:

[0025] The number of 3D bounding boxes for each semantic category in the statistically labeled data is given by the following formula: The number of bounding boxes for the class is Set hyperparameters Calculate the category balance weights and its normalized form:

[0026]

[0027] in The total number of categories, For category Category balance weights, This refers to the normalized class balancing weights.

[0028] Furthermore, in step 4, supervised optimization is performed using supervised loss, which consists of foreground confidence loss, bounding box regression loss, and semantic classification loss. The semantic classification loss uses cross-entropy with class balancing weights, and the bounding box regression loss uses SmoothL1 loss, regressing the center residual, size residual, and angle residual respectively.

[0029] Furthermore, step 5 is detailed as follows:

[0030] Step 501: Simultaneously sample labeled and unlabeled data in each training batch; apply weak enhancement transformation and strong enhancement transformation to the same unlabeled point cloud to obtain weak enhancement version and strong enhancement version respectively;

[0031] Step 502: Labeled data is used to supervise the student network; for unlabeled data, its strongly enhanced version is input into the student network to obtain the prediction result, and its weakly enhanced version is input into the teacher network to generate the prediction result required for pseudo-labels. The output of the teacher network is used to generate pseudo-labels, and the output of the student network is used to receive pseudo-label supervision.

[0032] Furthermore, step 6 is detailed as follows:

[0033] Step 601: For each candidate bounding box of an unlabeled sample, obtain the foreground confidence score based on the teacher's network prediction results. Category confidence and predicted intersection-over-union ratio The category confidence score is obtained by using the maximum category probability from the semantic classification head output after temperature calibration. The corresponding temperature calibration category distribution is denoted as ;

[0034] Step 602: Calculate the overall quality score according to the quality score calculation formula:

[0035]

[0036] in, , , These are the linear weights corresponding to the prospect confidence, class confidence, and predicted intersection-over-union ratio, respectively. The weight coefficient for the interaction consistency term; The entropy penalty coefficient is defined as follows: The category distribution entropy and its normalized form are defined as:

[0037]

[0038] in, Category distribution after temperature calibration Information entropy Normalized information entropy;

[0039] Step 603: Let the predicted category of the candidate box be... Tail enhancement is achieved by introducing class balancing weights and smoothing modulation, and a weighted quality score is calculated:

[0040]

[0041] in, Normalized class balancing weights are used to predict the categories of candidate boxes. For the sigmoid function, For smoothing coefficients, As a regulating factor;

[0042] Step 604: Using weighted quality scores As the ranking criterion, Top-ranked candidates are selected under the premise of meeting basic confidence and dynamic threshold requirements. Pseudo-labels; the basic confidence level includes:

[0043]

[0044] in, , , Training steps Dynamic thresholds for foreground confidence, category confidence, and predicted intersection-over-union (IoU) are provided.

[0045] After satisfying the basic confidence level mask, a hierarchical Top- Mechanism: The global retention quantity is Each category must retain at least Use pseudo-tags to improve tail coverage:

[0046]

[0047] in, The quota sharpening factor is used; simultaneously, soft deduplication based on 3DIoU is performed on the candidate pseudo-label set before final retention:

[0048]

[0049] in, These are the candidate pseudo-labels currently pending processing. Candidate pseudo-labels The current weighted quality score, Among the currently selected pseudo-tags, The box with the largest overlap. Candidate pseudo-labels With selected pseudo-labels The three-dimensional intersection-union ratio, This is the soft deduplication attenuation parameter; then follow the updated... Sort by Top- And meet various quota constraints;

[0050] Step 605: Apply a geometric transformation to the filtered pseudo-labels that is consistent with the student network input, so that the pseudo-labels and the student network are aligned in the same coordinate system.

[0051] Furthermore, step 7 is detailed as follows:

[0052] Step 701: Use the filtered and geometrically transformed aligned pseudo-labels for the unlabeled predictions generated by the supervised student network; For the first... Calculate the center of each candidate bounding box. With the Alignment pseudo-label center European distance If satisfied Then the student candidate box Assign to the most recent pseudo-tag:

[0053]

[0054] in, This is the distance-gated threshold. Indicates the student candidate box The index of the nearest matching pseudo-label; obtaining the supervision set for distance gating:

[0055]

[0056] in, This is the set of student candidate box indices that satisfy distance constraints and participate in unlabeled consistency supervision.

[0057] Step 702: Calculate the total training loss of the student network:

[0058]

[0059] in, The total loss of students' online training For supervised loss based on labeled samples, For distance-gated supervised sets, the unlabeled consistency loss is... The unlabeled consistency loss weights are applied only to student candidate boxes in the distance-gated supervision set. The unlabeled consistency loss includes bounding box regression loss and semantic classification loss.

[0060] Step 703: Update the student network parameters using gradient descent and update the teacher network parameters using exponential moving average to complete iterative training.

[0061] Furthermore, the classification and regression head includes a foreground binary classification head, a semantic classification head, and an intersection-over-union (IoU) prediction head to obtain the foreground confidence score. Category distribution and maximum class probability Predicting the intersection and union ratio ;

[0062] The semantic classification head outputs the category distribution and the maximum category probability. Introducing temperature coefficient Calibrate the category distribution:

[0063]

[0064]

[0065] in, The logits of the semantic categories output by the semantic classification header.

[0066] Furthermore, the dynamic thresholds for foreground confidence, class confidence, and prediction intersection-over-union ratio are defined as follows:

[0067]

[0068]

[0069]

[0070] in, and These are the initial and upper limits of the foreground confidence threshold, respectively. and These are the initial and upper limits of the category confidence threshold, respectively. and These are the initial and upper limits for predicting the crossover ratio (CUP) threshold, respectively. is the threshold growth scale constant.

[0071] Furthermore, the unlabeled consistency loss Loss from bounding box regression With semantic classification loss composition:

[0072]

[0073] in, For semantic classification loss weights. For each Only assign pseudo-labels to them Bounding box parameter regression:

[0074]

[0075] in For the SmoothL1 loss, sum the center, size, and orientation components separately; To meet the distance gating condition for the number of student candidate frames, The first output for the student network Candidate box parameters, For the alignment pseudo-label box parameters that match it;

[0076] The semantic classification loss is the cross-entropy with class-balanced weights:

[0077]

[0078] in To be the set of student candidate boxes that satisfy the distance constraint, Match pseudo-label categories to it. For the normalized class balance weights of the corresponding categories, For student networks to select candidate boxes The probability of predicting the pseudo-label category.

[0079] The beneficial effects of this invention are:

[0080] (1) By introducing local geometric branches, the model’s ability to perceive the local shape of small objects is enhanced, and the accuracy of its bounding box localization and classification is improved, thus alleviating the problem of missed detection of small objects caused by sparse point clouds.

[0081] (2) A loss function based on class balance weights is adopted to suppress the phenomenon of head class dominance from the gradient level, so that tail classes can be fully learned in both labeled and unlabeled training, and achieve more balanced class coverage.

[0082] (3) A quality-aware pseudo-label scoring mechanism is proposed to avoid the "short-board effect" of traditional product sorting for small objects and tail classes. It integrates calibration, entropy penalty, dynamic threshold and hierarchical quota to achieve more inclusive and reliable pseudo-label selection, which greatly improves the stability and generalization performance of semi-supervised training.

[0083] (4) The present invention adopts a modular architecture design. The local geometric branch, category balance weighting mechanism and quality perception pseudo-label scoring module can be independently integrated into the existing three-dimensional target detection framework. It has good versatility and engineering practicality and is suitable for practical application scenarios with high annotation costs, such as autonomous driving and intelligent robots. Attached Figure Description

[0084] Figure 1 This is a flowchart of the model training process in an embodiment of the present invention;

[0085] Figure 2 This is a schematic diagram of the algorithm principle for model training in an embodiment of the present invention;

[0086] Figure 3 This is a flowchart of the model prediction process in an embodiment of the present invention. Detailed Implementation

[0087] This invention aims to propose an unbiased pseudo-label learning method for semi-supervised 3D object detection, addressing the problems of low pseudo-label quality and low tail category recognition rate in existing technologies under conditions of scarce labeled data, due to the model's weak perception of small objects and the existence of long-tailed distribution in the dataset. The method includes a training process consisting of a fully supervised pre-training stage and a semi-supervised pseudo-label generation and training stage. In the fully supervised pre-training stage, a small amount of labeled point cloud is used to supervise the training of the detection network. A local geometric branch is introduced after the candidate box generation module and before the classification and regression heads to enhance the geometric representation ability of small objects. Simultaneously, a class balancing weight is introduced into the semantic classification loss to alleviate the problem of long-tailed category gradient submersion. The resulting pre-trained model is used to initialize the teacher and student networks. In the semi-supervised pseudo-label generation and training stage, each training batch contains labeled and unlabeled data. The labeled data is used to supervise the student network, while the unlabeled data is used by the teacher network to generate pseudo-labels, which are then filtered based on a quality-aware scoring mechanism. The filtered pseudo-labels undergo a geometric transformation consistent with the student input and are used to supervise the student network update. The teacher network parameters are updated using an exponential moving average method.

[0088] Example:

[0089] The unbiased pseudo-label learning method for semi-supervised 3D object detection in this embodiment includes two parts: a fully supervised pre-training stage and a semi-supervised pseudo-label generation and training stage. These are described in detail below with reference to the accompanying drawings.

[0090] See Figure 1 The training phase includes:

[0091] Step 1: Acquire 3D point cloud scene data and perform preprocessing.

[0092] Step 101: Obtain a small amount of labeled 3D point cloud scene data. Each scene contains point cloud data and corresponding 3D bounding boxes and semantic category labels; where the point cloud can be represented as...

[0093]

[0094] Where P is the acquired point cloud set, For the point with index j, The number of sampling points. For point The three-dimensional coordinates Optional point features (including color, intensity, or height, etc.); the set of annotated 3D bounding boxes is represented as follows:

[0095]

[0096] Where B is the set of three-dimensional bounding boxes. Let t be the 3D bounding box of the t-th object. The number of objects in the scene. With the center coordinates, For size vectors, For the orientation angle, For semantic categories, This represents the total number of categories.

[0097] And through a random sampling function A fixed number of points are obtained by sampling the point cloud:

[0098]

[0099] in, For a set fixed number of points, This is the sampled point cloud set.

[0100] Step 102: Sample the point cloud data to obtain a fixed number of point cloud inputs, and select and normalize the point cloud attributes as needed. For example, sample 40,000 points on the ScanNet dataset, and retain the 3D coordinates and RGB color information; the normalization may include mean removal and scale normalization.

[0101] Step 103: Perform data augmentation on the sample data. Augmentation methods include rotation around the vertical axis, flipping along the coordinate axes, and scaling. Record the flip flag, rotation matrix, and scaling ratio to ensure consistency between the subsequent monitoring signal and the pseudo-label under different augmentation perspectives. Geometric augmentation can be uniformly represented as an affine transformation.

[0102]

[0103] Where p is the sampling point, To flip the matrix, Let be the rotation matrix about the vertical axis. This is the scale vector; and it records... Used for aligning the coordinates of subsequent tags and pseudo-tags.

[0104] Step 2: Construct the detection network and introduce local geometry branches:

[0105] Step 201: Construct a detection backbone network; the detection backbone network extracts features from the input point cloud to obtain the coordinates of seed points. Features of seed points The candidate box generation module uses a voting mechanism to vote seed points to candidate centers, generating voting points. and polling station characteristics ;in, Number of seed points For seed point feature dimensions, For the number of polling stations, For voting point feature dimensions.

[0106] Specifically, the candidate box generation module aggregates voting points to obtain... Center coordinates of candidate boxes and its candidate box features , The center of the candidate box can be obtained by averaging the coordinates of the voting points assigned to that candidate box.

[0107]

[0108] in, To be allocated to the A set of voting points in a central aggregation center The number of votes in this set. For the polling point index, For the first The three-dimensional coordinates of each voting point.

[0109] Step 202: Before the candidate box features output by the candidate box generation module enter the classification and regression head, a local geometry branch is introduced. This local geometry branch encodes the local neighborhood geometry information of each candidate box center in the original point cloud and generates local geometric features that match the candidate box feature dimensions. The local geometric features output by the local geometry branch... With candidate box features The enhanced features are obtained by fusing using a residual method:

[0110]

[0111] in, The enhanced candidate bounding box features are then fed into the detection network. After passing through the network's classification and regression heads, the foreground confidence scores are obtained. Category distribution and maximum class probability Predicting the intersection and union ratio The classification and regression heads include a foreground binary classification head, a semantic classification head, and an IoU (Intersection over Union) prediction head.

[0112] The foreground binary classification head outputs the logits of the foreground category and the background category, respectively denoted as... The foreground confidence is obtained by using a binary softmax (normalized exponential function) output. :

[0113]

[0114] in, The unnormalized predicted score for the foreground category. The unnormalized predicted score for the background category. It is an exponential function.

[0115] The semantic classification header outputs the logits of each semantic category, denoted as And the class probability distribution is obtained through multi-class softmax:

[0116]

[0117]

[0118] in, For semantic classification head pair Unnormalized predicted scores for the class For the first Class probability, The total number of categories, Index for summing categories, This represents the maximum class probability.

[0119] To reduce the overconfidence effect, a temperature coefficient can be introduced. Calibrate the category distribution:

[0120]

[0121]

[0122] The logit output of the IoU prediction header. Output after sigmoid Predict the crossover ratio score:

[0123]

[0124] in, The unnormalized prediction score output by the IoU prediction head. For the sigmoid function, The 3D intersection-union ratio quality score is the predicted value of the candidate bounding box.

[0125] As a further optimization, in step 202, the implementation process of the local geometric branch includes:

[0126] In the preprocessed point cloud, centered at the center of each candidate box With the center of the sphere as the radius, The ball query operator retrieves the local neighborhood point set:

[0127]

[0128] in, Centered on the candidate box Center and radius of the sphere The set of neighboring points within, For point clouds The Middle One point, Denotes the Euclidean norm. For the radius of the sphere, look in the set of neighboring points. The process uses random sampling or nearest neighbor sampling to obtain at most 50 neighboring points, denoted as . .when At that time, padding with zero vectors to 50 points is performed, and the effective point mask is recorded. The mask value corresponding to the real nearest point is 1, and the mask value corresponding to the padding point is 0.

[0129] Calculate the relative coordinates of neighboring points with respect to the center of the candidate box:

[0130]

[0131] Each candidate box Corresponding neighborhood relative coordinates The input is a shared 3-layer MLP (Multilayer Perceptron) for point-by-point feature transformation. The specific structure and computation process of the 3-layer MLP are as follows:

[0132] Neighborhood relative coordinates Pointwise, a shared MLP is obtained through the following three layers: The first layer uses a linear transformation to map 3D coordinates to 64D, followed by ReLU (rectified linear function); the second layer uses a linear transformation to map 64D to 128D, followed by ReLU; the third layer uses a linear transformation to preserve the 128D dimension, followed by ReLU, to obtain local features. ;

[0133] Masked max pooling is used to aggregate the local features of the 50 nearest neighbors to obtain the most discriminative local geometric representation. :

[0134]

[0135] in, For the first The first candidate box Valid point mask of neighboring points, This represents the local features obtained after sharing the MLP for the neighboring points. As a pre-defined large positive number, This is a penalty value for invalid fill points, used to ensure that fill points do not participate in the maximum value selection; the max pooling is a channel-by-channel maximum value operation. Then... Aligned to by linear mapping dimension.

[0136]

[0137] in , These are trainable parameters.

[0138] Step 3: Calculate the category balance weights.

[0139] The number of 3D bounding boxes for each semantic category in the statistically labeled data is given by the following formula: The number of bounding boxes for the class is Set hyperparameters Calculate the category balance weights and its normalized form:

[0140]

[0141] in The total number of categories, For category Category balance weights, These are the normalized class balancing weights, used for all subsequent weighted loss calculations.

[0142] Step 4: Teacher-Student Network Initialization:

[0143] Using gradient descent, and leveraging 3D bounding boxes and semantic category labels as supervision signals, supervised optimization is performed on the parameters of the detection backbone network, local geometric branches, and classification and regression heads to obtain a pre-trained detection model.

[0144]

[0145] in, The parameters of the detection model to be optimized are... For the parameters of the pre-trained detection model, For labeled datasets, To label point clouds and their 3D bounding boxes, For parameters The following is the supervised loss calculated from the labeled sample. (Supervised Loss) Loss of confidence due to prospect Bounding box regression loss and semantic classification loss composition:

[0146]

[0147] semantic classification loss Cross-entropy with class-balanced weights, bounding box regression loss SmoothL1 loss can be used to regress the central residual, dimensional residual, and angular residual separately.

[0148] The pre-trained detection model is replicated as a teacher network and a student network; the gradient of the teacher network is frozen during training, while the student network participates in gradient updates.

[0149] Step 5: Hybrid Batch Construction and Forward Inference:

[0150] Step 501: Simultaneously sample labeled and unlabeled data in each training batch; for the same unlabeled point cloud... Apply weak enhancement transformation respectively With strong enhancement transformation A weakly enhanced version was obtained. Enhanced version :

[0151]

[0152] Step 502: Labeled data is used to supervise the student network; for unlabeled data, its strongly augmented version is input into the student network to obtain prediction results, and its weakly augmented version is input into the teacher network to generate the prediction results needed for pseudo-labels, while retaining information such as the foreground confidence, class confidence, and prediction crossover ratio output by the teacher network; its teacher network... For the weak enhancement version Output prediction Student Network Enhanced version Output prediction The teacher's network output is used to generate pseudo-labels, and the student's network output is used for pseudo-label supervision. The teacher's output can be represented as:

[0153]

[0154] in, This is the set of candidate bounding box predictions from the teacher network on an unlabeled sample. The first output for teachers' network 3D bounding box parameters for each candidate bounding box , , The first The foreground confidence, class confidence, and predicted intersection-over-union ratio of each candidate box. For candidate box index, This represents the number of candidate boxes.

[0155] Step 6: Generation and screening of quality-aware pseudo-labels.

[0156] Step 601: For each candidate bounding box of an unlabeled sample, obtain the foreground confidence score based on the teacher's network output. Category confidence and predicted intersection-over-union ratio The category confidence score is the maximum category probability obtained from the semantic classification head output after temperature calibration in step 202. The corresponding temperature calibration category distribution is denoted as ;

[0157] Step 602: Calculate the overall quality score according to the quality score calculation formula. The overall quality score includes the basic linear quality score, the interaction consistency term, and the category distribution entropy penalty term.

[0158]

[0159] in, , , These are the linear weights corresponding to the foreground confidence, class confidence, and predicted intersection-union ratio, respectively, and satisfying the following conditions: , , , ; The interaction consistency term weight coefficient is used to emphasize the quality of candidate boxes when all three are high simultaneously. Let be the entropy penalty coefficient. The category distribution entropy and its normalized form are defined as:

[0160]

[0161] in, Category distribution after temperature calibration Information entropy Normalized information entropy is used to compress the scale of uncertainty to a comparable range; This is a penalty term for candidate boxes with high uncertainty.

[0162] Step 603: Let the predicted category of the candidate box be...

[0163]

[0164] Tail enhancement is performed by introducing class-balanced weights and smoothing modulation, and a weighted quality score is calculated:

[0165]

[0166] in, Normalized class balancing weights are used to predict the categories of candidate boxes. For smoothing coefficients, It is a regulating factor.

[0167] Step 604: Using weighted quality scores As the ranking criterion, Top-ranked candidates are selected under the premise of meeting basic confidence and dynamic threshold requirements. Pseudo-labels; basic confidence level includes

[0168]

[0169] in, , , Training steps The dynamic thresholds are: foreground confidence dynamic threshold, class confidence dynamic threshold, and prediction intersection-union dynamic threshold. Furthermore, to improve the stability of pseudo-label selection at different training stages, the three dynamic thresholds can be defined as follows:

[0170]

[0171]

[0172]

[0173] in, and These are the initial and upper limits of the foreground confidence threshold, respectively. and These are the initial and upper limits of the category confidence threshold, respectively. and These are the initial and upper limits for predicting the crossover ratio (CUP) threshold, respectively. is the threshold growth scale constant.

[0174] After satisfying the mask, a hierarchical Top-level approach using "global budget + intra-class quota" is adopted. Mechanism: The global retention quantity is Each category must retain at least Use pseudo-tags to improve tail coverage:

[0175]

[0176] in, This is the quota sharpening factor. Simultaneously, to reduce duplicate pseudo-labels at the same spatial location, soft deduplication based on 3D IoU is performed on the candidate pseudo-label set before final retention:

[0177]

[0178] in, These are the candidate pseudo-labels currently pending processing. Candidate pseudo-labels The current weighted quality score, Among the currently selected pseudo-tags, The box with the largest overlap. Candidate pseudo-labels With selected pseudo-labels The three-dimensional intersection-union ratio, These are the soft deduplication attenuation parameters. Then, follow the updated... Sort by Top- And meet various quota constraints.

[0179] Step 605: Apply a geometric transformation to the filtered pseudo-labels, consistent with the student network input, to align the pseudo-labels and the student network in the same coordinate system. Let the center, size, and orientation of the pseudo-label frame before the transformation be... , , After transformation, the center, size, and orientation of the aligned pseudo-label box are respectively... , , The corresponding transformation is:

[0180]

[0181] in, To create a flipped matrix that matches the student input, To provide a rotation matrix consistent with the student input, To enhance the scaling factor, This represents the change in orientation angle introduced by rotational enhancement. This represents the pseudo-label parameters before transformation, using... This represents the alignment pseudo-label parameter after transformation.

[0182] Step 7: Pseudo-label supervision and parameter update:

[0183] Step 701: Use the filtered and geometrically aligned pseudo-labels to supervise the unlabeled predictions generated by the student network. For the first... Calculate the center of each candidate bounding box. With the Alignment pseudo-label center Euclidean distance:

[0184]

[0185] in, Index the candidate boxes for student networks. For the filtered pseudo-label index, The first output for the student network Center of each candidate box, For the first geometric transformation after step 605 Align the pseudo-label center, The Euclidean distance between the two is... Let represent the Euclidean norm. If satisfies ... Then the student candidate box Assign to the most recent pseudo-tag:

[0186]

[0187] in, This is the distance-gated threshold. Indicates the student candidate box The nearest matching pseudo-label index. Bounding box parametric regression loss and semantic classification loss are applied only to student candidate boxes that satisfy the distance gating condition, and these participate in unlabeled consistency supervision. The supervision set for distance gating is represented as:

[0188]

[0189] in, A set of student candidate box indices that satisfy distance constraints and can participate in unlabeled consistency supervision.

[0190] Step 702: Calculate the total training loss of the student network:

[0191]

[0192] in, The total loss of students' online training For supervised loss on labeled samples, For consistency loss on unlabeled samples, This is used as the weight for the unlabeled consistency loss. Simultaneously, for unlabeled samples, neither the prospect confidence loss nor the voting loss is calculated or backpropagated to reduce the interference of pseudo-label noise on the optimization direction.

[0193] Unlabeled consistency loss Loss from bounding box regression With semantic classification loss composition:

[0194]

[0195] in, For semantic classification loss weights. For each Only assign pseudo-labels to them Bounding box parameter regression:

[0196]

[0197] in The SmoothL1 loss is summed over the center, size, and orientation components respectively. To meet the distance gating condition for the number of student candidate frames, The first output for the student network Candidate box parameters, For the alignment pseudo-label box parameters that match it

[0198] The semantic classification loss is the cross-entropy with class-balanced weights:

[0199]

[0200] in To be the set of student candidate boxes that satisfy the distance constraint, Match pseudo-label categories to it. To balance the weights for the corresponding normalized categories, For student networks to select candidate boxes The pseudo-label category prediction probability is obtained; and the voting loss and prospect confidence loss are not backpropagated for unlabeled samples, so as to reduce the interference of pseudo-label noise on the optimization direction.

[0201] Step 703: Update the student network parameters using gradient descent and the teacher network parameters using exponential moving average to complete one iteration of training. The formula for updating the teacher network parameters is:

[0202]

[0203] in, This represents the current number of training steps. For the first The updated teacher network parameters For the teacher network parameters of the previous training step, For the first The updated student network parameters are obtained by gradient descent. It is the momentum coefficient, and it can increase with the number of training steps. Perform adaptive behavior:

[0204]

[0205] For preset upper limit (e.g.) ).

[0206] The algorithm principle for model training in steps 1-7 above is as follows: Figure 2 As shown, the trained student network serves as a 3D object detection model for subsequent 3D object detection tasks.

[0207] The prediction process flow in this embodiment is shown below. Figure 3 It includes:

[0208] Step S1, Data Preprocessing:

[0209] In this step, the 3D point cloud scene data to be detected is preprocessed, including point cloud sampling to obtain a fixed number of input points, and point cloud attributes are normalized and coordinates are aligned as needed.

[0210] Step S2, Forward Inference and Candidate Box Generation:

[0211] In this step, the point cloud data preprocessed in step S1 is input into the trained 3D object detection model. The model generates candidate boxes through feature extraction and voting aggregation, and outputs prediction results such as the center position, size, orientation, semantic category confidence, and foreground confidence of each candidate box.

[0212] Step S3, Output of detection results:

[0213] In this step, the candidate bounding box prediction results output by the model are filtered to obtain the final 3D bounding box detection results and their corresponding semantic categories, which are used as the model prediction output.

[0214] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. An unbiased pseudo-label learning method for semi-supervised 3D object detection, characterized in that, Includes the following steps: Step 1: Acquire 3D point cloud scene data and perform preprocessing; Step 2: Construct the detection backbone network and introduce a local geometry branch: The detection backbone network extracts features from the input point cloud to obtain seed point coordinates and seed point features; then, through the candidate box generation module, the seed points are voted to candidate centers using a voting mechanism, and then aggregated to generate candidate box center coordinates and candidate box features; the local geometry branch encodes the local neighborhood geometric information of each candidate box center in the original point cloud and generates local geometric features that match the dimensions of the candidate box features; the local geometric features and candidate box features are fused in a residual manner to obtain the enhanced features; Step 3: Calculate the category balance weights: count the number of 3D bounding boxes for each semantic category, and calculate the category balance weights and their normalized forms; Step 4: Teacher-student network initialization: Using gradient descent, the parameters of the detection backbone network, local geometric branches, and classification and regression heads are optimized under supervision using 3D bounding boxes and semantic category labels as supervision signals to obtain a pre-trained detection model; The pre-trained detection model is replicated as a teacher network and a student network; the gradient of the teacher network is frozen during training, while the student network participates in gradient updates. Step 5: Hybrid Batch Construction and Forward Inference: In each training batch, labeled and unlabeled data are sampled simultaneously; labeled data is used to supervise the student network; for unlabeled data, its strongly enhanced version is input into the student network to obtain the prediction result, and its weakly enhanced version is input into the teacher network to generate the prediction result required for the pseudo-label; Step 6: Quality-Aware Pseudo-Label Generation and Screening: For each candidate bounding box of an unlabeled sample, calculate the basic linear quality score, interaction consistency term, and category distribution entropy penalty term based on the output of the teacher network, and combine them to obtain a comprehensive quality score; the comprehensive quality score is then augmented at the tail based on category balancing weights and smoothing modulation to obtain a weighted quality score; using the weighted quality score as the ranking criterion, and selecting Top-... under the premise of satisfying the basic confidence mask. Pseudo-labels; Align the filtered pseudo-labels with the student's online input in the same coordinate system; Step 7: Pseudo-label supervision and parameter update: Use the filtered and aligned pseudo-labels to supervise the unlabeled predictions generated by the student network; Gradient descent is used to update student network parameters, and exponential moving average is used to update teacher network parameters to complete iterative training.

2. The unbiased pseudo-label learning method for semi-supervised 3D target detection according to claim 1, characterized in that, Step 2 includes the following steps: Step 201: Construct a detection backbone network; the detection backbone network extracts features from the input point cloud to obtain seed point coordinates and seed point features; the candidate box generation module votes the seed points to the candidate centers through a voting mechanism, generating voting points and voting point features, and aggregating the voting points to obtain... Center of each candidate bounding box and its features ; Step 202: Before the candidate box features output by the candidate box generation module enter the classification and regression head, a local geometry branch is introduced. The local geometry branch outputs local geometric features. With candidate box features The enhanced features are obtained by fusing using a residual method; the implementation process of the local geometric branch includes: In the preprocessed point cloud, centered at the center of each candidate box With the center of the sphere as the radius, The ball query operator obtains the local neighborhood point set. ; In the local neighborhood point set The process uses random sampling or nearest neighbor sampling to obtain at most 50 neighboring points, denoted as . ;when At that time, padding with zero vectors to 50 points is performed, and the effective point mask is recorded. The mask value corresponding to the real nearest point is 1, and the mask value corresponding to the padding point is 0. Calculate the relative coordinates of neighboring points with respect to the center of the candidate box: ; Each candidate box Corresponding neighborhood relative coordinates The input is a shared 3-layer multilayer perceptron, which performs point-by-point feature transformation to obtain local features. ; The local geometric representation is obtained by performing masked max pooling aggregation on the local features of the 50 nearest neighbors. Subsequently, the local geometric representation will be... Aligned with candidate box features via linear mapping.

3. The unbiased pseudo-label learning method for semi-supervised 3D target detection according to claim 2, characterized in that, The category balance weights and their normalized forms are calculated as follows: The number of 3D bounding boxes for each semantic category in the statistically labeled data is given by the following formula: The number of bounding boxes for the class is Set hyperparameters Calculate the category balance weights and its normalized form: in The total number of categories, For category Category balance weights, This refers to the normalized class balancing weights.

4. The unbiased pseudo-label learning method for semi-supervised 3D object detection according to claim 3, characterized in that, In step 4, supervised optimization is performed using supervised loss, which consists of foreground confidence loss, bounding box regression loss, and semantic classification loss. The semantic classification loss uses cross-entropy with class balancing weights, and the bounding box regression loss uses SmoothL1 loss, regressing the center residual, size residual, and angle residual respectively.

5. The unbiased pseudo-label learning method for semi-supervised 3D target detection according to claim 4, characterized in that, Step 5 is described in detail below: Step 501: Simultaneously sample labeled and unlabeled data in each training batch; apply weak enhancement transformation and strong enhancement transformation to the same unlabeled point cloud to obtain weak enhancement version and strong enhancement version respectively; Step 502: Labeled data is used to monitor student networks; For unlabeled data, the strongly augmented version is input into the student network to obtain the prediction results, and the weakly augmented version is input into the teacher network to generate the prediction results required for pseudo-labels. The output of the teacher network is used to generate pseudo-labels, and the output of the student network is used to receive pseudo-label supervision.

6. The unbiased pseudo-label learning method for semi-supervised 3D target detection according to claim 5, characterized in that, Step 6 is as follows: Step 601: For each candidate bounding box of an unlabeled sample, obtain the foreground confidence score based on the teacher's network prediction results. Category confidence and predicted intersection-over-union ratio The category confidence score is obtained by using the maximum category probability from the semantic classification head output after temperature calibration. The corresponding temperature calibration category distribution is denoted as ; Step 602: Calculate the overall quality score according to the quality score calculation formula: in, , , These are the linear weights corresponding to the prospect confidence, class confidence, and predicted intersection-over-union ratio, respectively. The weight coefficient for the interaction consistency term; The entropy penalty coefficient is defined as follows: The category distribution entropy and its normalized form are defined as: in, Category distribution after temperature calibration Information entropy Normalized information entropy; Step 603: Let the predicted category of the candidate box be... Tail enhancement is achieved by introducing class balancing weights and smoothing modulation, and a weighted quality score is calculated: in, Normalized class balancing weights are used to predict the categories of candidate boxes. For the sigmoid function, For smoothing coefficients, As a regulating factor; Step 604: Using weighted quality scores As the ranking criterion, Top-ranked candidates are selected under the premise of meeting basic confidence and dynamic threshold requirements. Pseudo-labels; the basic confidence level includes: in, , , Training steps Dynamic thresholds for foreground confidence, category confidence, and predicted intersection-over-union (IoU) are provided. After satisfying the basic confidence level mask, a hierarchical Top- Mechanism: The global retention quantity is Each category must retain at least Use pseudo-tags to improve tail coverage: in, The quota sharpening factor is used; simultaneously, soft deduplication based on 3DIoU is performed on the candidate pseudo-label set before final retention: in, These are the candidate pseudo-labels currently pending processing. Candidate pseudo-labels The current weighted quality score, Among the currently selected pseudo-tags, The box with the largest overlap. Candidate pseudo-labels With selected pseudo-labels The three-dimensional intersection-union ratio, This is the soft deduplication attenuation parameter; then follow the updated... Sort by Top- And meet various quota constraints; Step 605: Apply a geometric transformation to the filtered pseudo-labels that is consistent with the student network input, so that the pseudo-labels and the student network are aligned in the same coordinate system.

7. The unbiased pseudo-label learning method for semi-supervised 3D object detection according to claim 6, characterized in that, Step 7 is described in detail below: Step 701: Use the filtered and geometrically transformed aligned pseudo-labels to supervise the unlabeled predictions generated by the student network; for the first... Calculate the center of each candidate bounding box. With the Alignment pseudo-label center European distance If satisfied Then the student candidate box Assign to the most recent pseudo-tag: in, This is the distance-gated threshold. Indicates the student candidate box The index of the nearest matching pseudo-label; obtaining the supervision set for distance gating: in, This is the set of student candidate box indices that satisfy distance constraints and participate in unlabeled consistency supervision. Step 702: Calculate the total training loss of the student network: in, The total loss of students' online training For supervised loss based on labeled samples, For distance-gated supervised sets, the unlabeled consistency loss is... The unlabeled consistency loss weights are applied only to student candidate boxes in the distance-gated supervision set. The unlabeled consistency loss includes bounding box regression loss and semantic classification loss. Step 703: Update the student network parameters using gradient descent and update the teacher network parameters using exponential moving average to complete iterative training.

8. The unbiased pseudo-label learning method for semi-supervised 3D target detection according to claim 7, characterized in that, The classification and regression heads include a foreground binary classification head, a semantic classification head, and an intersection-over-union (IoU) prediction head, which yields the foreground confidence score. Category distribution and maximum class probability Predicting the intersection and union ratio ; The semantic classification head outputs the category distribution and the maximum category probability. Introducing temperature coefficient Calibrate the category distribution: in, The logits of the semantic categories output by the semantic classification header.

9. The unbiased pseudo-label learning method for semi-supervised 3D target detection according to claim 8, characterized in that, The foreground confidence dynamic threshold, class confidence dynamic threshold, and prediction crossover ratio dynamic threshold are defined as follows: in, and These are the initial and upper limits of the foreground confidence threshold, respectively. and These are the initial and upper limits of the category confidence threshold, respectively. and These are the initial and upper limits for predicting the crossover ratio (CUP) threshold, respectively. is the threshold growth scale constant.

10. The unbiased pseudo-label learning method for semi-supervised 3D target detection according to claim 9, characterized in that, The unlabeled consistency loss Loss from bounding box regression With semantic classification loss composition: in, For each semantic classification loss weight; Only assign pseudo-labels to them Bounding box parameter regression: in For the SmoothL1 loss, sum the center, size, and orientation components separately; To meet the distance gating condition for the number of student candidate frames, The first output for the student network Candidate box parameters, For the alignment pseudo-label box parameters that match it; The semantic classification loss is the cross-entropy with class-balanced weights: in To be the set of student candidate boxes that satisfy the distance constraint, Match pseudo-label categories to it. For the normalized class balance weights of the corresponding categories, For student networks to select candidate boxes The probability of predicting the pseudo-label category.

Images