Point cloud detection method and system
By building a knowledge distillation network and introducing positioning guidance classification items, the problem of regression box scale uncertainty in laser point cloud detection is solved, the detection accuracy and accuracy are improved, and it is suitable for three-dimensional object detection tasks of laser point clouds.
Patent Information
- Application Number
- CN202310052085.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-02
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2043-02-02
AI Technical Summary
In the laser point cloud detection task, the occlusion of the target object leads to uncertainty in the regression frame scale. The existing knowledge distillation method is inefficient and is not suitable for three-dimensional data, making it difficult to improve detection accuracy.
A point cloud detection method was designed, and by constructing a knowledge distillation network, using the same point cloud encoding and feature extraction structures of the teacher detection network and the student detection network, distillation processing at the regression frame scale, and introducing positioning guidance classification terms to adjust the positive Sample-supervised label weights, optimized training sets to improve detection accuracy.
It effectively alleviates the uncertainty of object boundary caused by occlusion, improves the detection accuracy of obstructed objects, and reduces the confidence deviation between high-quality positive samples and low-quality positive samples, improving detection accuracy.
Smart Images

Figure CN116030023B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of laser point cloud, and specifically to a point cloud detection method and system. Background Art
[0002] Laser point cloud is an intuitive, flexible and storage-efficient 3D data representation method that has become indispensable in 3D vision. The emergence of large-scale LiDAR datasets and the great progress in end-to-end 3D representation learning have promoted the development of point cloud-based segmentation, generation and detection tasks. Therefore, learning and processing laser point cloud data is very important for promoting machines to understand the 3D world, which is conducive to promoting the development of many important real-world applications, such as autonomous driving technology.
[0003] Compared with traditional visual cameras, LiDAR can perceive more structural information and distance information, and is less affected by lighting. Therefore, LiDAR is rapidly popularized in driverless vehicles. Based on this trend, a large number of target detection algorithms based on laser point clouds have emerged. Generally, 3D detectors are divided into two categories: single-stage detection networks directly regress bounding boxes and confidences from features learned from the input; two-stage detection networks use region proposal alignment features in the second stage to optimize the first-stage predictions. Therefore, the two-stage detection network achieves higher accuracy by adding the second stage to make more precise corrections to the first-stage detection box, while the single-stage detection network usually runs faster due to its simpler network structure, and is more able to meet the real-time requirements in actual scenarios such as driverless driving.
[0004] The detection accuracy in target detection is greatly affected by the positioning accuracy or the quality of the regression box. In outdoor laser point cloud detection tasks, the target object may be occluded. According to the generation method of laser point cloud, this occlusion will make the point cloud representation of the object have no clear boundaries, making it difficult to determine the scale of the regression box during training. One way to alleviate this problem is knowledge distillation. Early knowledge distillation methods usually train the student network to imitate the classification probability distribution predicted by the teacher network. The knowledge distillation methods in the target detection network model are usually divided into three categories: Logit distillation method applied to classification supervision labels, but previous relevant scholars pointed out that the original logit distillation technology used for classification is inefficient and can only transfer semantic knowledge; applied to the feature layer to extract intermediate features based on different segmentation areas, adaptively align the feature maps of the teacher network and the student network, but the imbalance of a large number of background samples in the detection task makes the feature distillation defect obvious; "pseudo regression box" distillation method that uses the bounding box predicted by the teacher network model as an additional regression target. At present, the distillation methods applied to target detection are all designed based on two-dimensional images, which are not suitable for direct application to laser point cloud target detection. Summary of the invention
[0005] In order to meet the high precision and high efficiency requirements of tasks such as autonomous driving, this application designs a single-stage detection network to solve three-dimensional detection tasks based on laser point clouds.
[0006] In order to achieve the above object, the present application provides a point cloud detection method, the steps comprising:
[0007] Obtain laser point cloud data of the image to be detected in the target visual scene;
[0008] A training set is obtained by improving the scale accuracy of the regression frame of the laser point cloud data;
[0009] Optimizing the training set to obtain an optimized training set;
[0010] The optimized training set is used to construct a network model; and the network model is used to perform target detection on point cloud data.
[0011] Preferably, the method for obtaining the training set includes: constructing a knowledge distillation network; the knowledge distillation network includes: a teacher detection network and a student detection network; the teacher detection network and the student detection network have the same point cloud encoding and feature extraction structure.
[0012] Preferably, the method of obtaining the training set using the knowledge distillation network includes:
[0013] Discretizing the scale precision into continuous values and then performing probability fitting;
[0014] adding constraints to the student detection network;
[0015] The regression frame is distilled using the student detection network with added constraints to obtain the training set.
[0016] Preferably, the method for distilling the regression frame includes:
[0017] Process the scale of the positive sample regression box and record the data of each edge;
[0018] Setting the regression box size of the feature extraction and representing it by the teacher detection network and the student detection network respectively;
[0019] Converting the teacher detection network and the student detection network into probability representations;
[0020] By performing a distillation loss evaluation on the probability, an evaluation result is obtained;
[0021] The regression frame is distilled according to the data and the evaluation result.
[0022] Preferably, the method for obtaining the optimized training set includes: designing a positioning guided classification item to perform weight guidance on the positive sample supervision label to obtain the optimized training set.
[0023] Preferably, the method for performing the weight guidance includes:
[0024] The IOU value between the predicted bounding box corresponding to the positive sample and the true bounding box in the BEV space is used as the classification guidance weight to suppress the deviation between relatively high-quality positive samples and relatively low-quality positive samples and classification confidence, and the positioning guidance term g is introduced:
[0025]
[0026] Among them, the target supervision value is:
[0027]
[0028] In the formula, g represents the positioning guided classification item, i represents the IOU value of the predicted bounding box and the true bounding box, pos represents the positive sample, bev represents the dimensional representation of the bounding box in the BEV space, and reg pred Represents the predicted regression box deviation value, anchor represents the prior box, bbox perd represents the predicted regression box, represents the true label represented by the one-hot vector, and f represents the true label of the positive sample in the form of a softlabel representation that introduces a positioning guide.
[0029] The present application also provides a point cloud detection system, comprising: an acquisition module, a distillation module, an optimization module and a construction module;
[0030] The acquisition module is used to acquire laser point cloud data of the image to be detected in the target visual scene;
[0031] The distillation module is used to obtain a training set by improving the regression frame scale accuracy of the laser point cloud data;
[0032] The optimization module is used to optimize the training set to obtain an optimized training set;
[0033] The construction module uses the optimized training set to build a network model; and uses the network model to perform target detection on point cloud data.
[0034] Preferably, the distillation module comprises: a teacher unit and a student unit;
[0035] The teacher unit is used to provide network parameters;
[0036] The student unit is used to complete distillation based on the network parameters to obtain the training set.
[0037] Compared with the prior art, the beneficial effects of this application are as follows:
[0038] This application can effectively alleviate the uncertainty of object boundaries caused by occlusion and improve the detection accuracy of occluded objects; at the same time, this application also designs positioning-guided classification items to guide the supervision label weights of relatively high-quality positive samples and relatively low-quality positive samples, thereby alleviating the deviation between relatively high-quality positive samples or relatively low-quality positive samples and classification confidence, thereby improving detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. Embodiment 1 is an initial segmentation example, and Embodiment 2 is a region merging example. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0040] Figure 1 This is a schematic diagram of the method flow of Embodiment 1 of the present application;
[0041] Figure 2 Schematic diagram of the deviation between relatively high (low) quality positive samples and confidence levels in Example 1 of the present application
[0042] Figure 3 This is a schematic diagram of positioning guidance classification items of Example 1 of the present application;
[0043] Figure 4 This is a schematic diagram of the system structure of Example 2 of the present application. DETAILED DESCRIPTION
[0044] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0045] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0046] Embodiment 1
[0047] like Figure 1As shown, it is a schematic diagram of the method flow of an embodiment, and the steps include: obtaining laser point cloud data of the image to be detected in the target visual scene; obtaining a training set by improving the regression box scale accuracy of the laser point cloud data; optimizing the training set to obtain an optimized training set; using the optimized training set to build a network model; and using the network model to perform target detection on the point cloud data.
[0048] In this embodiment, the target visual scene is a scanning scene of the autonomous driving vehicle during driving.
[0049] First, the laser point cloud data of the image to be detected in the target visual scene is obtained. Then, the collected point cloud data is encoded and feature extracted. The point cloud is set as a cylinder on the xy plane, and each non-empty cylinder constitutes a group of sub-point clouds S x∈W,y∈H = {P i ,i=1,2,3,…,n x,y}, where each point P i Use a vector (x, y, z, r) to represent n x,y is the number of points in the corresponding set. (W,H) is the range of the number of points along the X and Y axes. x,y Point P in i , each point cloud represented by its vector (x, y, z) is encoded into a D = 9-dimensional vector (x, y, z, r, x c ,y c ,z c ,x p ,x p );(x c ,y c ,z c ) represents the distance from the current point to the arithmetic mean point of all points in the column; (x p ,y p ) represents the offset distance from the current lidar point to the x,y center of the cylinder. Set a maximum of P non-empty point columns. If the number of points in the column is greater than N, sample to N. If it is less than N, fill it with 0. Through the above method, a frame of point cloud is encoded into a dense tensor of dimension (D, P, N). Each point in the set is processed with a linear layer + BatchNorm + ReLU activation function to generate a tensor of dimension (C, P, N), where C is the feature channel; then the point column index value of each point is put back to the original corresponding point column x, y position to generate a pseudo image of dimension (C, H, W). The feature extraction network consists of a downsampling network and an upsampling network. The downsampling network block is represented as ConvBlock (C in ,C out ,S d ,N b ), where C is the number of feature channels, S dis the downsampling factor, Nb is the number of convolutional layers in each network block; the upsampling network block is represented as DeconvBlock (C in ,C out ,S u ), where S u is the upsampling factor for 2D deconvolution.
[0050] Afterwards, a knowledge distillation network is constructed to perform knowledge distillation on the processed point cloud to improve the accuracy of the regression box.
[0051] The knowledge distillation network includes a teacher detection network and a student detection network, wherein the feature extraction module and the prediction output module of the teacher network and the student network are the same. In this embodiment, the teacher network model is first trained, and then the teacher network parameters are frozen, and the teacher network model is preloaded when training the student network model. Unlike classification distillation that only transfers semantic knowledge, regression box distillation can transfer the position and scale information of the target object. The regression box scale from the teacher model is used as an additional regression target for the student model to help the student model converge to a better optimization point. This embodiment believes that this strategy can make the regression prediction of the student network model more robust and achieve better generalization ability, and can improve its detection effect when facing occluded targets or targets with uncertain boundaries.
[0052] In the 3D target detection of laser point cloud, the regression box prediction output of the network model is (x, y, z, l, w, h, θ), a total of 7 dimensions of data. In this embodiment, only the regression box scale (l, w, h) of the predicted output is distilled. In the target detection in the 2D image, the representation of its bounding box is usually (x, y, w, h) (center point coordinates, length and width), (x min ,y min ,x max ,y max )(the upper left corner and lower right corner of the regression box) and (t,b,l,r) (the distance from the sampling point to the top, bottom, left and right of the regression box), where (x,y,w,h) and (x min ,y min ,x max ,y max ) can be directly converted to each other. These two representation methods are further expressed by their sampling points (x s ,y s ) and the matching ground-truth box (x gt ,y gt ,w gt ,h gt) can calculate the distance from the sampling point to the top, bottom, left and right of the real frame, that is, (t, b, l, r). Regardless of whether it is an Anchor-Base type detection network or an Anchor-Free type detection network, the three representations of the above regression frame can be converted to each other according to the matching real frame, from discrete values to continuous values, and from continuous values to discrete values. However, in the three-dimensional target detection regression frame with a rotation angle, the center point, regression frame size and rotation angle are independent of each other. The idea of the regression frame distillation in this embodiment is to first discretize the variables regressed on the continuous domain, and finally perform probability fitting.
[0053] In this embodiment, the scale Dim = (l, w, h) of the positive sample regression box is selected for processing. The physical meaning of each variable of (l, w, h) is consistent. Each edge is recorded as e. Let m be the three regression box sizes predicted by the network, which are respectively determined by the m of the teacher network. T and student network S Indicates that the generalized SoftMax function S(·,τ)=SoftMax(·,τ) is used to convert m T and m S Convert to probability representation p T and p S When τ = 1, it is equivalent to the original SoftMax function; when τ > 1, the input parameters carry more information.
[0054] The distillation loss used to measure the probability similarity of two groups is implemented by the following formula:
[0055]
[0056] in Denoting KL-divergence loss, the distillation of three dimensions of regression box size S can be formulated as:
[0057]
[0058] Considering that in the prior art, the classification supervision vector of the detection network is generally assigned to the background class and the real target class, where the negative sample index is assigned to the background class and the positive sample index is assigned to the real target class, the number of negative samples is much larger than the number of positive samples; this will cause the problem of imbalance between positive and negative samples. In order to alleviate the deviation between relatively high-quality positive samples or relatively low-quality positive samples and classification confidence (such as Figure 2 ), without the need for additional network embedding, this embodiment designs a positioning guidance classification item to guide the weight of the positive sample supervision label. Figure 3As shown in the figure, in 3D object detection, the positioning points are usually defined by uniformly distributing bounding boxes of fixed size on the BEV feature map; then the network is trained by regressing the offset between the prior box and the ground truth. The network's positive sample regression prediction value is decoded with the matching prior box to obtain the predicted bounding box. A key advantage of objects in the BEV space of the point cloud is that their positions do not overlap. At the same time, the BEV space is also usually used for motion planning, so the positioning effect and positioning quality of real objects in the BEV space are excellent. The dimensions of the predicted bounding box and the true bounding box are expressed as bbox = (x, y, z, l, w, h, θ), and the vector (x, y, l, w, θ) of the positive sample set is cut out to calculate the IOU value in the BEV space. From the physical meaning of IOU, it can be seen that the higher the correlation between the two bounding boxes, the greater their IOU value. In the feature map, each sample point has a corresponding regression prediction, classification confidence, and true label; the higher the correlation between the regression prediction and the true bounding box, the higher the "quality" or "value" of the corresponding sample point, and the classification confidence corresponding to this sample point should be given more weight, and should not have the same weight as those relatively low-quality positive sample points. Based on this, this embodiment uses the IOU value of the predicted bounding box corresponding to the positive sample and the true bounding box in the BEV space as the classification guidance weight to suppress the deviation between relatively high-quality positive samples or relatively low-quality positive samples and classification confidence, and introduces the positioning guidance item g:
[0059]
[0060] Target supervision value:
[0061]
[0062] In the formula, g represents the positioning guided classification item, i represents the IOU value of the predicted bounding box and the true bounding box, pos represents the positive sample, bev represents the dimensional representation of the bounding box in the BEV space, and reg pred Represents the predicted regression box deviation value, anchor represents the prior box, bbox perd represents the predicted regression box, represents the true label represented by the one-hot vector, and f represents the true label of the positive sample in the form of a softlabel representation that introduces a positioning guide.
[0063] The current classification loss of the Pointpillars network is the Focal loss (FL) loss function. The original FL was used to solve the one-stage target detection scenario, where there is often an extreme imbalance between foreground and background classes during training. The general form of the focal loss is as follows:
[0064]
[0065] where \(y\in\{0,1\}\) represents the category of the true value, \(p\in[0,1]\) represents the category probability predicted by the model when the true label \(y = 1\), and \(\gamma\) is an adjustable focal parameter. The focal loss FL consists of the standard cross-entropy \(-\log(p t ) and a modulating factor \((1 - p t ) γ . To alleviate the bias between relatively high-quality positive samples or relatively low-quality positive samples and classification confidence, in this embodiment, a positioning guidance term \(g\) is introduced to the positive sample label. Thus, the true label of the positive sample changes from the original \(y = 0\) representing negative samples and \(y = 1\) representing positive samples to the soft label representation of \(f = 0\) representing negative samples and \(0\lt f\leq1\) representing positive samples. The original focal loss cannot meet the calculation requirements after introducing the positioning guidance term and needs to be modified. The focal loss of Pointpillars uses multi-binary classification of the sigmoid operator \(\alpha(\cdot)\) to achieve multi-classification, and the output of sigmoid is marked as \(\alpha\). The two parts of the focal loss Focal loss are extended. The cross-entropy part \(-\log(p t ) is extended to the complete representation form \(-((1 - y)\log(1 - \alpha)+y\log(\alpha))\). After substituting the positioning guidance term \(g\), the cross-entropy part becomes \(-((1 - f)\log(1 - \alpha)+f\log(\alpha))\); the scale factor part \((1 - p t ) γ is generalized and extended to the absolute value of the distance between the estimated \(\alpha\) and its continuous label, that is, expressed as \(|f - \alpha| β (\beta\geq0)\), where \(|\cdot|\) ensures non-negativity. Finally, the two extended parts are combined to form a complete classification loss:
[0066]
[0067] The network model is constructed through the above training set that has undergone knowledge distillation and added the positioning guidance classification term.
[0068] In this embodiment, the positioning regression loss selects the same regression loss as SECOND. The 3D representation of each true target or its prior box is represented by a 7-dimensional vector: \((x,y,z,l,w,h,\theta)\). Where \(x,y,z\) represent the center point coordinates of the 3D bounding box; \(l,w,h\) represent the length, width, and height of the 3D bounding box respectively; \(\theta\) represents the orientation angle of the 3D bounding box. The residual between the true bounding box and the prior box in the bounding box positioning regression task is defined as:
[0069]
[0070]
[0071] Δθ=sin(θ gt -θ a )
[0072] Among them, x gt and x a This represents the real bounding box and the prior box, The bounding box regression loss uses the Smooth L1 function:
[0073]
[0074] Softmax classification loss is used to learn the orientation of the target, and the orientation loss is recorded as
[0075] For classification loss, the modified focal loss function is used, namely:
[0076]
[0077] The regression distillation loss is:
[0078]
[0079] The total loss of the final network model can be expressed as:
[0080]
[0081] Where N pos is the number of positive samples, and the coefficients of each loss value are λ0=2.0,λ1=1.0,λ2=0.2,λ3=0.2.
[0082] Embodiment 2
[0083] like Figure 4 As shown, it is a schematic diagram of the system structure of this embodiment, including: an acquisition module, a distillation module, an optimization module and a construction module; wherein the acquisition module is used to acquire the laser point cloud data of the image to be detected in the target visual scene; the distillation module is used to obtain the training set by improving the scale accuracy of the regression box of the laser point cloud data; the optimization module is used to optimize the training set to obtain an optimized training set; the construction module uses the optimized training set to build a network model; and uses the network model to perform target detection on the point cloud data. The distillation module is used to construct a knowledge distillation network to improve the accuracy of the regression box. In addition, the distillation module includes: a teacher unit and a student unit; the teacher unit is used to provide network parameters; the student unit is used to complete distillation based on the network parameters to obtain a training set.
[0084] In this embodiment, the target visual scene is a scanning scene of the autonomous driving vehicle during driving.
[0085] First, the acquisition module is used to obtain the laser point cloud data of the image to be detected in the target visual scene. Then, the distillation module is used to perform point cloud encoding and feature extraction on the collected point cloud data.
[0086] The workflow of the distillation module includes: using teacher units to build a teacher detection network; using student units to build a student detection network; wherein the feature extraction module and prediction output module of the teacher network and the student network are the same. In this embodiment, the distillation module first trains the teacher network model, then freezes the teacher network parameters, and preloads the teacher network model when training the student network model. Unlike classification distillation that only transfers semantic knowledge, regression box distillation can transfer the position and scale information of the target object. The regression box scale from the teacher model is used as an additional regression target for the student model to help the student model converge to a better optimization point. This embodiment believes that this strategy can make the regression prediction of the student network model more robust and achieve better generalization ability, and can improve its detection effect when facing occluded targets or targets with uncertain boundaries.
[0087] In the 3D target detection of laser point cloud, the regression box prediction output of the network model is (x, y, z, l, w, h, θ), a total of 7 dimensions of data. In this embodiment, only the regression box scale (l, w, h) of the predicted output is distilled. In the target detection in the 2D image, the representation of its bounding box is usually (x, y, w, h) (center point coordinates, length and width), (x min ,y min ,x max ,y max )(the upper left corner and lower right corner of the regression box) and (t,b,l,r) (the distance from the sampling point to the top, bottom, left and right of the regression box), where (x,y,w,h) and (x min ,y min ,x max ,y max ) can be directly converted to each other. These two representation methods are further expressed by their sampling points (x s ,y s ) and the matching ground-truth box (x gt ,y gt ,w gt ,h gt) can calculate the distance from the sampling point to the top, bottom, left and right of the real frame, that is, (t, b, l, r). Regardless of whether it is an Anchor-Base type detection network or an Anchor-Free type detection network, the three representations of the above regression frame can be converted to each other according to the matching real frame, from discrete values to continuous values, and from continuous values to discrete values. However, in the three-dimensional target detection regression frame with a rotation angle, the center point, regression frame size and rotation angle are independent of each other. The idea of the regression frame distillation in this embodiment is to first discretize the variables regressed on the continuous domain, and finally perform probability fitting.
[0088] In this embodiment, the scale Dim = (l, w, h) of the positive sample regression box is selected for processing. The physical meaning of each variable of (l, w, h) is consistent. Each edge is recorded as e. Let m be the three regression box sizes predicted by the network, which are respectively determined by the m of the teacher network. T and student network S Indicates that the generalized SoftMax function S(·,τ)=SoftMax(·,τ) is used to convert m T and m S Convert to probability representation p T and p S When τ = 1, it is equivalent to the original SoftMax function; when τ > 1, the input parameters carry more information.
[0089] The distillation loss used to measure the probability similarity of two groups is implemented by the following formula:
[0090]
[0091] in Denoting KL-divergence loss, the distillation of three dimensions of regression box size S can be formulated as:
[0092]
[0093] Considering that in the prior art, the classification supervision vector of the detection network is generally assigned to the background class and the real target class, wherein the negative sample index is assigned to the background class, and the positive sample index is assigned to the real target class, and the number of negative samples is much larger than the number of positive samples; this will cause the problem of imbalance between positive and negative samples. In order to alleviate the deviation between relatively high-quality positive samples or relatively low-quality positive samples and classification confidence without the need for additional network embedding, this embodiment uses the optimization module to design a positioning guidance classification item for weight guidance of the positive sample supervision label.
[0094] In 3D object detection, the positioning points are usually defined by evenly distributing bounding boxes of fixed size on the BEV feature map; then the network is trained by regressing the offset between the prior box and the ground truth. The network's positive sample regression prediction value is decoded with the matching prior box to obtain the predicted bounding box. A key advantage of objects in the BEV space of the point cloud is that the positions do not overlap. At the same time, the BEV space is also usually used for motion planning, so the positioning effect and positioning quality of real objects in the BEV space are excellent. The dimensions of the predicted bounding box and the true bounding box are expressed as bbox = (x, y, z, l, w, h, θ), and the vector (x, y, l, w, θ) of the positive sample set is cut out to calculate the IOU value in the BEV space. From the physical meaning of IOU, it can be seen that the higher the correlation between the two bounding boxes, the greater their IOU value. In the feature map, each sample point has a corresponding regression prediction, classification confidence, and true label; the higher the correlation between the regression prediction and the true bounding box, the higher the "quality" or "value" of the corresponding sample point, and the classification confidence corresponding to this sample point should be given more weight, and should not have the same weight as those relatively low-quality positive sample points. Based on this, this embodiment uses the IOU value of the predicted bounding box corresponding to the positive sample and the true bounding box in the BEV space as the classification guidance weight to suppress the deviation between relatively high-quality positive samples or relatively low-quality positive samples and classification confidence, and introduces the positioning guidance item g:
[0095]
[0096]
[0097] Target supervision value:
[0098]
[0099] In the formula, g represents the positioning guided classification item, i represents the IOU value of the predicted bounding box and the true bounding box, pos represents the positive sample, bev represents the dimensional representation of the bounding box in the BEV space, and reg pred Represents the predicted regression box deviation value, anchor represents the prior box, bbox perd represents the predicted regression box, represents the true label represented by the one-hot vector, and f represents the true label of the positive sample in the form of a softlabel representation that introduces a positioning guide.
[0100] The current classification loss of the Pointpillars network is the Focal loss (FL) loss function. The original FL was used to solve the one-stage target detection scenario, where there is often an extreme imbalance between foreground and background classes during training. The general form of the focal loss is as follows:
[0101]
[0102] where y ∈ {0, 1} refers to the category of the true value, p ∈ [0, 1] refers to the category probability predicted by the model when the true label y = 1, and γ is the adjustable focal parameter. The focal loss FL consists of the standard cross-entropy -log(p t ) and a modulating factor (1 - p t ). γ The focal loss consists of two parts. To alleviate the bias between relatively high-quality positive samples or relatively low-quality positive samples and classification confidence, this embodiment introduces a positioning guidance term g to the positive sample label. Thus, the true label of the positive sample changes from the original y = 0 representing the negative sample and y = 1 representing the positive sample to the soft label representation of f = 0 representing the negative sample and 0 < f ≤ 1 representing the positive sample. The original focal loss cannot meet the calculation requirements after introducing the positioning guidance term and needs to be modified. The focal loss of Pointpillars uses multi-binary classification of the sigmoid operator α(·) to achieve multi-classification, and marks the output of sigmoid as α; the two parts of the focal loss Focal loss are extended. The cross-entropy part -log(p t ) is extended to the complete representation -((1 - y)log(1 - α) + ylog(α)). After substituting the positioning guidance term g, the cross-entropy part becomes -((1 - f)log(1 - α) + flog(α)); the scale factor part (1 - p t ) γ is generalized and extended to the absolute value of the distance between the estimated α and its continuous label, that is, expressed as |f - α| β (β ≥ 0), where |·| ensures non-negativity. Finally, the extended two parts are combined to form the complete classification loss:
[0103]
[0104] The network model is constructed through the above training set that has undergone knowledge distillation and added the positioning guidance classification term.
[0105] In this embodiment, the regression loss selected by the construction module is the same as that of SECOND. The 3D representation of each true target or its prior box is represented by a 7-dimensional vector: (x, y, z, l, w, h, θ). Where x, y, z represent the center point coordinates of the 3D bounding box; l, w, h represent the length, width, and height of the 3D bounding box respectively; θ represents the orientation angle of the 3D bounding box. The residual between the true bounding box and the prior box in the bounding box localization regression task is defined as:
[0106]
[0107]
[0108] Δθ=sin(θ gt -θ a )
[0109] Among them, x gt and x a This represents the real bounding box and the prior box, The bounding box regression loss uses the Smooth L1 function:
[0110]
[0111] Softmax classification loss is used to learn the orientation of the target, and the orientation loss is recorded as For classification loss, the modified focal loss function is used, namely:
[0112]
[0113] The regression distillation loss is:
[0114]
[0115] The total loss of the final network model can be expressed as:
[0116]
[0117] Where N pos is the number of positive samples, and the coefficients of each loss value are λ0=2.0,λ1=1.0,λ2=0.2,λ3=0.2.
[0118] The embodiments described above are only descriptions of the preferred methods of the present application, and are not intended to limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements made to the technical solutions of the present application by ordinary technicians in this field should fall within the scope of protection determined by the claims of the present application.
Claims
1. A point cloud detection method, characterized in that the steps include: Obtain laser point cloud data of the image to be detected in the target visual scene; A training set is obtained by improving the scale accuracy of the regression frame of the laser point cloud data; The method for obtaining the training set includes: constructing a knowledge distillation network; the knowledge distillation network includes: a teacher detection network and a student detection network; the teacher detection network and the student detection network have the same point cloud encoding and feature extraction structure; the method for obtaining the training set using the knowledge distillation network includes: discretizing the scale accuracy into continuous values and then performing probability fitting; adding constraints to the student detection network; and distilling the regression frame using the student detection network with added constraints to obtain the training set; The training set is optimized to obtain an optimized training set; the method for obtaining the optimized training set includes: designing a positioning guidance item to weight-guide the positive sample supervision label to obtain the optimized training set; the method for weight-guide includes: The IOU value between the predicted bounding box corresponding to the positive sample and the true bounding box in the BEV space is used as the classification guidance weight to suppress the deviation between relatively high-quality positive samples and relatively low-quality positive samples and classification confidence, and the positioning guidance term g is introduced: Among them, the target supervision value is: In the formula, g represents the positioning guide term, i represents the IOU value of the predicted bounding box and the true bounding box, pos represents the positive sample, bev represents the dimensional representation of the bounding box in the BEV space, and reg pred Represents the predicted regression box deviation value, anchor represents the prior box, bbox perd represents the predicted regression box, represents the true label represented by the one-hot vector, and f represents the true label of the positive sample in the form of a soft label representation with the introduction of the positioning guide item; A network model is constructed using the optimized training set; and target detection is performed on point cloud data using the network model.
2. The point cloud detection method according to claim 1, characterized in that: The method of distilling the regression frame includes: Process the scale of the positive sample regression box and record the data of each edge; Setting the regression box size of the feature extraction and representing it by the teacher detection network and the student detection network respectively; Converting the teacher detection network and the student detection network into probability representations; By performing a distillation loss evaluation on the probability, an evaluation result is obtained; The regression frame is distilled according to the data and the evaluation result.
3. A point cloud detection system, characterized in that: include: Acquisition module, distillation module, optimization module and construction module; The acquisition module is used to acquire laser point cloud data of the image to be detected in the target visual scene; The distillation module is used to obtain a training set by improving the scale accuracy of the regression frame of the laser point cloud data; the distillation module includes: a teacher unit and a student unit; the teacher unit is used to provide network parameters; the student unit is used to complete distillation based on the network parameters to obtain the training set; specifically, the teacher detection network and the student detection network have the same point cloud encoding and feature extraction structure; the method of using the knowledge distillation network to obtain the training set includes: discretizing the scale accuracy into continuous values and then performing probability fitting; adding constraints to the student detection network; using the student detection network with added constraints to distill the regression frame to obtain the training set; The optimization module is used to optimize the training set to obtain an optimized training set; the method for obtaining the optimized training set includes: designing a positioning guidance item to perform weight guidance on the positive sample supervision label to obtain the optimized training set; the method for performing the weight guidance includes: The IOU value between the predicted bounding box corresponding to the positive sample and the true bounding box in the BEV space is used as the classification guidance weight to suppress the deviation between relatively high-quality positive samples and relatively low-quality positive samples and classification confidence, and the positioning guidance term g is introduced: Among them, the target supervision value is: In the formula, g represents the positioning guide term, i represents the IOU value of the predicted bounding box and the true bounding box, pos represents the positive sample, bev represents the dimensional representation of the bounding box in the BEV space, and reg pred Represents the predicted regression box deviation value, anchor represents the prior box, bbox perd represents the predicted regression box, represents the true label represented by the one-hot vector, and f represents the true label of the positive sample in the form of a soft label representation with the introduction of the positioning guide item; The construction module uses the optimized training set to build a network model; and uses the network model to perform target detection on point cloud data.
