A class-guided roadside traffic target detection method

Abstract

The application discloses a kind of based on class guide roadside traffic target detection method, detection steps are as follows: S1., first, in feature extraction, considering the characteristics of 3D point cloud sparse, it is difficult to detect the target itself under the target feature of non-human design, to eliminate the dependence of 3D point cloud on artificial feature labeling, voxel feature extraction method is used;S2.after step S1 is completed, based on the 3D convolution network of voxel division can well preserve the point cloud information of laser radar, but the inference speed is slow and the target orientation estimation deviation is large, and the improved 3D sparse convolution network is used to improve the accuracy of roadside traffic target detection.The present application makes improvement on the existing method in feature extraction for road test traffic target detection to improve accuracy, and the present application also uses related data pre-training and parameter initialization in model training to ensure quality, and then trains on target class, so that the result can be accurately detected, and the accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of roadside traffic target detection, in particular to a roadside traffic target detection method based on category guidance. BACKGROUND

[0002] With the rapid development of automatic driving vehicles and the improvement of single vehicle intelligence specifications, the application range of roadside traffic target detection is expanding. In actual application, roadside traffic target detection provides single vehicles with super-perception information. Due to the single vehicle intelligence, it is still impossible to avoid the problems such as visual blind area, extreme weather and judgment delay. Therefore, it is crucial for the vehicle path planning to obtain effective information from the surrounding environment and assist the vehicle in making judgments. Therefore, the roadside traffic target detection can be used as an effective basis for the intelligent decision of the vehicle. Roadside traffic target detection can greatly compensate for the perception blind spot of single vehicle intelligence, and change the automatic driving from single combat to organized and efficient cooperation.

[0003] At present, in most applications, roadside device scanning is the main data source of roadside traffic target detection. When the roadside traffic target is detected, the model effect is required to be relatively high. When the distance of the target object from the scanning center is different, the scanning angle changes greatly, which will cause the phenomenon of uneven distribution of point cloud scanning density. The farther from the scanning center, the sparser the point cloud. In feature extraction, due to the sparse characteristics of 3D point cloud, it is difficult to detect the target itself under non-human designed target features. If the foreground points in the scene are much less than the background points, it will cause the problem of uneven categories. If a 3D detection network is used in model training, random initialization is difficult to train. If the same model is trained for 3 categories, the situation is different, the difficulty is great, and the effect is not good. Therefore, based on the above technical problems, the present application discloses a roadside traffic target detection method based on category guidance. SUMMARY

[0004] The purpose of the present application is to provide a roadside traffic target detection method based on category guidance. The present application uses improved spatial voxel feature extraction and neighborhood condensation feature extraction for feature extraction, and then uses related data for pre-training in model training, parameter initialization, and training on target categories. This can ensure accurate detection results under good quality and solve the problems raised in the background art.

[0005] To achieve the above purpose, the present application provides the following technical scheme: a roadside traffic target detection method based on category guidance, the detection steps are as follows:

[0006] S1. First, in feature extraction, considering the sparse characteristics of 3D point cloud, it is difficult to detect the target itself under non-human designed target features. In order to eliminate the dependence of 3D point cloud on artificial feature labeling, a voxel feature extraction method is used;

[0007] S2. After step S1 is completed, the 3D convolutional network based on voxel division can retain the point cloud information of the LiDAR well, but the inference speed is slow and the target orientation estimation deviation is large. The accuracy of roadside traffic target detection is improved by using an improved 3D sparse convolutional network.

[0008] S3. After step S2 is completed, in 3D sparse convolution, the 2D dense convolution algorithm is first used, F u,v,l,m G represents the filtered elements. u,v,l Let represent an image element, u and v be spatial position indices, representing the input l and output channels m, respectively. The function P(x,y) generates the input positions to be computed given the provided output positions. Therefore, the convolution output is Y. x,y,m ;

[0009] S4. After step S3 is completed, use a matrix multiplication-based algorithm to collect data and construct a matrix. Next, the conversion is performed using matrix multiplication;

[0010] S5. After step S4 is completed, then process the sparse data G' i,l and associated output Y' j,m Perform calculations;

[0011] S6. After step S5 is completed, as the number of network layers increases, many problems will arise, including gradient vanishing and gradient exploding. Deep learning updates parameters through chain-like backpropagation of errors. Once the derivative in the backpropagation of a certain layer is too small, the gradient will gradually vanish after multiple multiplications. The residual connection is equivalent to adding an identity number 1 to the derivative of each layer.

[0012] S7. After step S6 is completed, in the residual connection, this paper uses Gaussian error linear unit (GELU) instead of rectified linear unit (RELU) as the activation function of the network;

[0013] S8. After step S7 is completed, the RoI is obtained by generating corresponding features from the roadside 3D point cloud after feature extraction. Then, the Predicted Keypoint Weighting module is used to reduce the impact of non-foreground keypoint features on the refinement stage. The ground truth bounding boxes of 3D objects are used to provide object segmentation masks (separating objects from each other and objects from the background). Since there are far fewer foreground points than background points in the scene, a focus loss function is used to solve the class imbalance problem.

[0014] L focal (p t )=-α t (1-p t )γ log(p t )

[0015]

[0016] Where P t This represents the probability that a point cloud is either foreground or background. Once the features of the entire RoI are obtained, predictions can be made.

[0017] S9. After step S8 is completed, finally, in model training, use relevant data for pre-training and parameter initialization to ensure good quality, and then train on the target class to accurately detect the results and improve accuracy.

[0018] In a preferred embodiment of the present invention, the convolution output formula in step S3 is:

[0019]

[0020] Where u-u0 and v-v0 represent the kernel offsets u and v coordinates.

[0021] In a preferred embodiment of the present invention, the matrix multiplication formula executed in step S4 is as follows:

[0022]

[0023] Where F' u-u0,v-v0,l,m For F that has undergone matrix multiplication u-u0,v-v0,l,m .

[0024] In a preferred embodiment of the present invention, step S5 involves processing the sparse data G' i,l and associated output Y' j,m The calculation formula is:

[0025]

[0026] Where P'(j) is a function used to obtain the input index i and the filter offset (j);

[0027] The subscript k is the 1D convolution kernel offset of formula (1) with respect to u-u0 and v-v0.

[0028] In a preferred embodiment of the present invention, the residual in step S6 can be expressed as:

[0029]

[0030] Where f represents the convolution operation and x represents the input of the current layer.

[0031] In a preferred embodiment of the present invention, the activation function expression in step S7 is:

[0032] GELU(x)=xP(X≤x)=xΦ(x)

[0033] Since the above functions cannot be calculated directly, GELU can be represented by an approximation parameter 'tanh', and its expression is:

[0034] GELU(x)=0.5x(1+

[0035]

[0036] GELU can introduce the idea of ​​random regularization into activation, which is an effective description of the probability of neuron inputs, more intuitive and in line with natural cognition. At the same time, the non-linear expression of this function can make neural networks more deterministic in decision-making.

[0037] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0038] This invention introduces a category-guided method for roadside traffic target detection. This method does not rely on manual feature annotation, and it improves upon existing methods in feature extraction to enhance accuracy for roadside traffic target detection. Furthermore, this invention utilizes pre-training with relevant data and parameter initialization during model training to ensure quality, followed by training on the target classes. This approach enables accurate detection and further improves accuracy. Attached Figure Description

[0039] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0040] Fig. 1 This is a diagram of the spatial voxel feature extraction structure of the present invention;

[0041] Fig. 2 This is a structural diagram of the spatial voxel feature encoding layer of the present invention. Detailed Implementation

[0042] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0043] Please see Figs. 1-2 This invention provides a technical solution: a category-guided roadside traffic target detection method, the detection steps of which are as follows:

[0044] S1. First, in terms of feature extraction, considering the sparse nature of 3D point clouds, it is difficult to detect the target itself under non-human-designed target features. In order to eliminate the dependence of 3D point clouds on manual feature annotation, a voxel feature extraction method is used.

[0045] S2. After step S1 is completed, the 3D convolutional network based on voxel division can retain the point cloud information of the LiDAR well, but the inference speed is slow and the target orientation estimation deviation is large. The accuracy of roadside traffic target detection is improved by using an improved 3D sparse convolutional network.

[0046] S3. After step S2 is completed, in 3D sparse convolution, the 2D dense convolution algorithm is first used, F u,v,l,m G represents the filtered elements. u,v,l Let represent an image element, u and v be spatial position indices, representing the input l and output channels m, respectively. The function P(x,y) generates the input positions to be computed given the provided output positions. Therefore, the convolution output is Y. x,y,m ,

[0047]

[0048] Where u-u0 and v-v0 represent the kernel offsets u and v coordinates;

[0049] S4. After step S3 is completed, use a matrix multiplication-based algorithm to collect data and construct a matrix. Next, matrix multiplication is used for conversion, and the formula is as follows:

[0050]

[0051] Where F' u-u0,v-v0,l,m For F that has undergone matrix multiplication u-u0,v-v0,l,m ;

[0052] S5. After step S4 is completed, then process the sparse data G' i,l and associated output Y' j,m The calculation is performed using the following formula:

[0053]

[0054] Where P'(j) is a function used to obtain the input index i and the filter offset (j);

[0055] The subscript k is the 1D convolution kernel offset of formula (1) with respect to u-u0 and v-v0;

[0056] S6. After step S5 is completed, as the number of network layers increases, many problems arise, including vanishing gradients and exploding gradients. Deep learning updates parameters through chain-like backpropagation of errors. If the derivative in the backpropagation of a certain layer is too small, the gradient gradually vanishes after multiple multiplications. Residual connections are equivalent to adding an identity number 1 to the derivative of each layer's propagation.

[0057] The residual can be expressed as:

[0058]

[0059] Where f represents the convolution operation and x represents the input of the current layer;

[0060] S7. After step S6 is completed, in the residual connections, this paper uses the Gaussian Error Linear Unit (GELU) instead of the Rectified Linear Unit (RELU) as the activation function of the network, and its expression is:

[0061] GELU(x)=xP(X≤x)=xΦ(x)

[0062] Since the above functions cannot be calculated directly, GELU can be represented by an approximation parameter 'tanh', and its expression is:

[0063]

[0064] GELU can introduce the idea of ​​random regularization into activation, which is an effective description of the probability of neuron inputs, more intuitive and in line with natural cognition. At the same time, the non-linear expression of this function can make the neural network more deterministic in decision-making.

[0065] S8. After step S7 is completed, the RoI is obtained by generating corresponding features from the roadside 3D point cloud after feature extraction. Then, the Predicted Keypoint Weighting module is used to reduce the impact of non-foreground keypoint features on the refinement stage. The ground truth bounding boxes of 3D objects are used to provide object segmentation masks (separating objects from each other and objects from the background). Since there are far fewer foreground points than background points in the scene, a focus loss function is used to solve the class imbalance problem.

[0066] L focal (p t )=-α t (1-p t ) γ log(p t )

[0067]

[0068] Where P tThis represents the probability that a point cloud is either foreground or background. Once the features of the entire RoI are obtained, predictions can be made.

[0069] S9. After step S8 is completed, finally, in model training, use relevant data for pre-training and parameter initialization to ensure good quality, and then train on the target class to accurately detect the results and improve accuracy.

[0070] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the scope of the invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0071] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A category-guided roadside traffic target detection method, characterized in that: The testing steps are as follows: S1. First, in terms of feature extraction, considering the sparse nature of 3D point clouds, it is difficult to detect the target itself under non-human-designed target features. In order to eliminate the dependence of 3D point clouds on manual feature annotation, a voxel feature extraction method is used. S2. After step S1 is completed, the 3D convolutional network based on voxel division can retain the point cloud information of the LiDAR well, but the inference speed is slow and the target orientation estimation deviation is large. The accuracy of roadside traffic target detection is improved by using an improved 3D sparse convolutional network. S3. After step S2 is completed, in 3D sparse convolution, the 2D dense convolution algorithm is used first. Where u-u0 and v-v0 represent the kernel offsets u and v coordinates, F u,v,l,m G represents the filtered elements. u,v,l Representing image elements, u and v are spatial location indices, representing input l and output channels m, respectively. The function P(x,y) generates the input positions to be computed given the provided output positions. The convolution output is Y. x,y,m ; S4. After step S3 is completed, use a matrix multiplication-based algorithm to collect data and construct a matrix. Next, the conversion is performed using matrix multiplication; S5. After step S4 is completed, then process the sparse data G' i,l and associated output Y' j,m Perform calculations; S6. After step S5 is completed, as the number of network layers increases, many problems will arise, including: gradient vanishing and gradient exploding. Deep learning updates parameters through chain-like backpropagation of errors. Once the derivative in the backpropagation of a certain layer is too small, the gradient will gradually vanish after multiple multiplications. The residual connection is equivalent to adding an identity number 1 to the derivative of each layer. S7. After step S6 is completed, in the residual connection, the Gaussian error linear unit (GELU) is used to replace the rectified linear unit (RELU) as the activation function of the network; S8. After step S7 is completed, the RoI is obtained by generating corresponding features from the roadside 3D point cloud after feature extraction. Then, the Predicted Keypoint Weighting module is used to reduce the impact of non-foreground keypoint features on the refinement stage. The ground truth bounding boxes of 3D objects are used to provide object segmentation masks. These segmentation masks separate objects from each other and objects from the background. Since there are far fewer foreground points than background points in the scene, a focus loss function is used to solve the class imbalance problem. L focal (p t )=-a t (1-p t ) γ log(p t ) Where α is the class balance factor, adjusting the weight ratio of positive and negative samples; γ is the hard sample adjustment parameter, dynamically adjusting the loss weights to make the model pay more attention to samples that are difficult to classify; P t This represents the probability that a point cloud is either foreground or background. Once the features of the entire RoI are obtained, predictions can be made. S9. After step S8 is completed, finally, in model training, use relevant data for pre-training and parameter initialization to ensure good quality, and then train on the target class to accurately detect the results and improve accuracy.

2. The roadside traffic target detection method based on category guidance according to claim 1, characterized in that: The matrix multiplication formula executed in step S4 is as follows: Where F' u-u0,v-v0,l,m For F that has undergone matrix multiplication u-u0,v-v0,l,m .

3. The roadside traffic target detection method based on category guidance according to claim 2, characterized in that: In step S5, the sparse data G' i,l and associated output Y' j,m The calculation formula is: Where P'(j) is a function used to obtain the input index i and filter offset for sampling point (j); the subscript k is the 1D convolution kernel offset for u-u0 and v-v0.

4. The roadside traffic target detection method based on category guidance according to claim 3, characterized in that: The residual in step S6 can be expressed as: Where f represents the convolution operation, x represents the input of the current layer, and h is the convolution output.

5. The roadside traffic target detection method based on category guidance according to claim 4, characterized in that: The activation function expression in step S7 is: GELU(x)=xP(X≤x)=xΦ(x) P(X≤x) represents the probability that X is less than or equal to x in a random variable, and Φ(x) is the cumulative distribution function of the standard normal distribution. Since these functions cannot be directly calculated, GELU can be represented by the approximate parameter 'tanh', and its expression is: GELU can introduce the idea of ​​random regularization into activation, which is an effective description of the probability of neuron inputs, more intuitive and in line with natural cognition. At the same time, the non-linear expression of this function can make neural networks more deterministic in decision-making.

Images