A point cloud 3D target detection method, system, storage medium and device

Abstract

The present invention discloses a point cloud 3D target detection method, system, storage medium and device, the method comprising: obtaining image data of various road participants and different degrees of occlusion and truncation in different driving scenes, and performing preprocessing to obtain a target data set; constructing a point cloud 3D target detection model based on a B-spline activation function; importing a training set into the point cloud 3D target detection model for training, and using a test set to verify the point cloud 3D target detection model, and using a validation set to evaluate the point cloud 3D target detection model; deploying the point cloud 3D target detection model in a vehicle, and detecting scenes of occlusion and truncation of road participants in different driving scenes. The present invention is conducive to accurately judging road conditions, predicting the behavior of pedestrians and cyclists, improving driving efficiency and driving safety, enhancing system adaptability and robustness, and reducing the occurrence of traffic accidents by detecting different scenes in automatic driving.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a point cloud 3D target detection method, system, storage medium and device. Background Art

[0002] With the rapid development of autonomous driving technology, accurate perception of the surrounding environment has become one of the core requirements of autonomous driving systems. Traditional two-dimensional image processing technology has limitations in capturing three-dimensional information of objects. From the perspective of imaging principles: two-dimensional image processing technology mainly relies on two-dimensional images captured by image sensors, such as cameras. Although these images can show surface information such as the shape and color of objects, they cannot directly provide depth information of objects, that is, the position relationship of objects in three-dimensional space. Due to the lack of depth information in two-dimensional images, it is difficult for autonomous driving systems to accurately judge the actual distance between objects and vehicles, which is particularly critical in complex traffic environments. Especially at intersections or bends, if the distance to the vehicle or pedestrian in front cannot be accurately judged, serious traffic accidents may occur. From the perspective of environmental adaptability: two-dimensional image processing technology is very sensitive to changes in lighting conditions. Under harsh lighting conditions such as strong light, backlight or night, the image quality will be significantly reduced, thus affecting the accuracy and reliability of object recognition. Severe weather conditions such as rain, snow, fog and haze will also have a serious impact on two-dimensional image processing technology. These weather phenomena will cause problems such as image blur and reduced contrast, further increasing the difficulty of object recognition.

[0003] With the continuous development of deep learning, point cloud-based 3D object detection methods have become a hot research direction for autonomous driving monitoring. However, although the existing common object detection technologies have achieved good results in some scenarios, overall, there is still a lot of room for improvement in the performance of the algorithm, especially in terms of the real-time performance, reliability, accuracy and robustness of the algorithm.

[0004] Regarding the real-time and accuracy of target detection in the field of autonomous driving, there are currently the following challenges:

[0005] (1) Occlusion problem: In a complex traffic environment, vehicles, pedestrians, cyclists and other obstacles may block each other, causing the sensor to be unable to obtain complete target information. At the same time, the target object may also be blocked by trees, buildings or other non-moving objects, increasing the difficulty of detection.

[0006] (2) Small target detection: For small targets far away from the sensor, such as pedestrians or small obstacles in the distance, they occupy a small number of pixels or points in the point cloud data and are difficult to detect accurately.

[0007] (3) Real-time performance: Autonomous driving systems need to detect and respond to surrounding objects in real time to ensure driving safety. However, existing object detection methods still have a certain gap in real-time performance, especially when dealing with complex scenes and large amounts of data. Summary of the invention

[0008] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a point cloud 3D target detection method, system, storage medium and device. The present invention detects different scenarios in autonomous driving, which is conducive to accurately judging road conditions, predicting the behavior of pedestrians and cyclists, improving driving efficiency and driving safety, enhancing system adaptability and robustness, and reducing the occurrence of traffic accidents.

[0009] A first aspect of the present invention is to provide a point cloud 3D object detection method, the method comprising:

[0010] Obtain image data of various road participants and different degrees of occlusion and truncation in different driving scenarios, preprocess the image data to obtain the target data set, and divide the target data set into training set, test set and validation set according to the preset ratio;

[0011] Construct the model structure of the point cloud 3D object detection model based on B-spline activation function;

[0012] Importing the training set into the point cloud 3D object detection model for training, using the test set to verify the point cloud 3D object detection model, and using the validation set to evaluate the point cloud 3D object detection model;

[0013] The point cloud 3D target detection model is deployed in a vehicle, and the point cloud 3D target detection model is used to detect various road participants and scenes with different degrees of occlusion and truncation in different driving scenarios.

[0014] According to one aspect of the above technical solution, the steps of obtaining image data of various road participants and different degrees of occlusion and truncation in different driving scenes, preprocessing the image data to obtain a target data set, and dividing the target data set into a training set, a test set and a validation set according to a preset ratio include:

[0015] Obtain a target data set containing image data and lidar point cloud data collected in actual road scenes, or obtain image data of various road participants and different degrees of occlusion and truncation in different driving scenes to obtain a target data set;

[0016] Preprocessing the target data set and performing data annotation;

[0017] The target data set is divided into training set, test set and validation set in a ratio of 7:3:3.

[0018] According to one aspect of the above technical solution, in the step of importing the training set into the point cloud 3D object detection model for training, using the test set to verify the point cloud 3D object detection model, and using the verification set to evaluate the point cloud 3D object detection model, the step of importing the training set into the point cloud 3D object detection model for training includes:

[0019] Importing the training set into the point cloud 3D target detection model for training, and calculating the target detection loss according to a predetermined overall loss during the forward propagation calculation process;

[0020] Use the stochastic gradient descent algorithm for reverse derivation, and update the model parameters according to the gradient to complete the model training process of minimizing the loss function.

[0021] According to one aspect of the above technical solution, the overall loss includes total positioning loss, target classification loss and normalized classification loss, and the overall loss value of the overall loss is obtained by weighted average of the total positioning loss, the target classification loss and the normalized classification loss;

[0022] The total positioning loss includes center position loss, size loss, direction loss and rotation angle loss, and the target classification loss includes existence classification loss and category classification loss;

[0023] The calculation expression of the overall loss is:

[0024] ;

[0025] In the formula, Npos is the number of positive anchor points, is the total positioning loss, is the target classification loss, is the normalized classification loss, , , They are the weight coefficients of each loss, which are used to balance the importance of different loss items.

[0026] According to one aspect of the above technical solution, the calculation expression of the total positioning loss is:

[0027] ;

[0028] In the formula, L center is the center position loss, L dim For size loss, L ori For direction loss, Lrot is the rotation angle loss, λ center ,λ dim ,λ ori ,λ rot They are the weight coefficients of each loss, which are used to balance the importance of different loss items;

[0029] Among them, in the total positioning loss, the calculation expression of the center position loss is:

[0030] ;

[0031] Where b is the bounding box, is the target detection regression loss function, is the positioning regression residual between the real box and the anchor box, (x, y, z) is the position coordinate of the real box;

[0032] Among them, the positioning regression residual between the real box and the anchor box is defined as:

[0033] ;

[0034] In the formula, is the X-axis coordinate of the real frame, is the X-axis coordinate of the anchor box, is the Y-axis coordinate of the real frame, is the Y-axis coordinate of the anchor box, is the Z-axis coordinate of the real frame, is the Z-axis coordinate of the anchor box;

[0035] Among them, in the total positioning loss, the calculation expression of the size loss is:

[0036] ;

[0037] In the formula, is the width of the real frame, is the width of the anchor box, is the length of the real frame, is the length of the anchor box, is the height of the real frame, is the height of the anchor box;

[0038] Among them, in the total positioning loss, the calculation expression of the direction loss is:

[0039] ;

[0040] In the formula, is the rotation angle of the real frame, is the rotation angle of the anchor box;

[0041] Among them, in the total positioning loss, the calculation expression of the rotation angle loss is:

[0042] ;

[0043] In the formula, is the rotation angle of the real frame, The rotation angle of the anchor box.

[0044] According to one aspect of the above technical solution, the calculation expression of the target classification loss is:

[0045] ;

[0046] In the formula, L exist is the existence classification loss, L class is the category classification loss, and λ is a hyperparameter used to balance the importance of existence classification loss and category classification loss;

[0047] Among them, in the target classification loss, the calculation expression of the existence classification loss is:

[0048] ;

[0049] In the formula, N for Pillar total, p exist express Pillar The probability that a target exists in y exist is the true label, y exist The value is 0 or 1;

[0050] Among them, in the target classification loss, the calculation expression of the category classification loss is:

[0051] ;

[0052] In the formula, M For the purpose of existence Pillar quantity, C is the total number of categories, p class is the output probability of the model for each category, y class is the true label.

[0053] According to one aspect of the above technical solution, the calculation expression of the normalized classification loss is:

[0054] ;

[0055] In the formula, N for Pillar total, C is the total number of categories, y ic For sample i Belongs to category c The real label, p ic For sample i Belongs to category c The predicted probability of

[0056] in, , Zic For the model to sample i Belongs to category c The original output of Zij For sample i For Category j Output.

[0057] A second aspect of the present invention is to provide a point cloud 3D object detection system, which is applied to the method described in the above technical solution, and comprises:

[0058] A data acquisition module is used to acquire image data of various road participants and different degrees of occlusion and truncation in different driving scenarios, pre-process the image data to obtain a target data set, and divide the target data set into a training set, a test set, and a validation set according to a preset ratio;

[0059] A model building module is used to build the model structure of the point cloud 3D object detection model based on the B-spline activation function;

[0060] A model training module, used for importing the training set into the point cloud 3D object detection model for training, verifying the point cloud 3D object detection model using the test set, and evaluating the point cloud 3D object detection model using the validation set;

[0061] The target detection module is used to deploy the point cloud 3D target detection model in the vehicle, and use the point cloud 3D target detection model to detect various road participants and scenes with different degrees of occlusion and truncation in different driving scenarios.

[0062] A third aspect of the present invention is to provide a readable storage medium having a computer program stored thereon, which implements the method described in the above technical solution when executed by a processor.

[0063] The fourth aspect of the present invention is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the method described in the above technical solution when executing the computer program.

[0064] Compared with the prior art, the point cloud 3D target detection method, system, storage medium and device shown in the present invention have the following beneficial effects:

[0065] By adopting the point cloud 3D target detection technology based on B-spline activation function, different scenes in autonomous driving are detected, including vehicle scenes, pedestrian scenes, cyclist scenes and scenes with different degrees of occlusion and truncation. This is conducive to accurately judging road conditions and predicting the behavior of pedestrians and cyclists, thereby improving driving efficiency and driving safety, enhancing system adaptability and robustness, and reducing the occurrence of traffic accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0067] Figure 1 Schematic diagram of the process of a point cloud 3D object detection method in one embodiment of the present invention;

[0068] Figure 2 3D object detection system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0069] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings. Several embodiments of the present invention are shown in the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0070] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0071] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0072] Embodiment 1

[0073] See also Figure 1 The first embodiment of the present invention provides a point cloud 3D target detection method, the method comprising steps S10 to S40:

[0074] Step S10, obtaining image data of various road participants and different degrees of occlusion and truncation in different driving scenarios, preprocessing the image data to obtain a target data set, and dividing the target data set into a training set, a test set and a validation set according to a preset ratio.

[0075] In this embodiment, the steps of obtaining image data of various road participants and different degrees of occlusion and truncation in different driving scenes, preprocessing the image data to obtain a target data set, and dividing the target data set into a training set, a test set, and a validation set according to a preset ratio include:

[0076] Obtain a target data set containing image data and lidar point cloud data collected in actual road scenes, or obtain image data of various road participants and different degrees of occlusion and truncation in different driving scenes to obtain a target data set;

[0077] Preprocessing the target data set and performing data annotation;

[0078] The target data set is divided into training set, test set and validation set in a ratio of 7:3:3.

[0079] Specifically, we obtain image data of road participants such as vehicles, pedestrians, and cyclists in different driving scenarios, including urban areas, rural areas, and highways, with different degrees of occlusion and truncation, and pre-process the data. Then, we divide the overall data set into training set, test set, and validation set in a ratio of 7:3:3.

[0080] More specifically, the acquisition and processing of data sets include:

[0081] Dataset acquisition: You can obtain the Kitti dataset, which contains images and LiDAR point cloud data collected in actual road scenes, through the official website, or go to different scenes such as urban areas, rural areas, and highways to take pictures of vehicles, pedestrians, cyclists, and images with different degrees of occlusion and truncation.

[0082] Dataset processing: denoise and filter the data, use the ICP algorithm to spatially calibrate the point cloud data collected at different time points or different devices to ensure the consistency and integrity of the data. Then annotate the obtained data. In order to achieve a better data enhancement effect, randomly select some pictures for rotation and translation to simulate the observation results under different perspectives; select some pictures for scaling operations to simulate the size changes of target objects at different distances; and select some images for affine transformation to simulate more complex scene changes and improve the generalization ability of the model. Finally, the image data after data enhancement is annotated in the same way and added to the original data set as training data, thereby enhancing the adaptability and detection capabilities of the network model to learn different sizes and angles.

[0083] In this embodiment, for the annotation of data, the occluded vehicle shall be accurately annotated with the 3D frame of the vehicle based on the visible part of the point cloud data and the context information; if the occlusion is serious and cannot be accurately annotated, the occlusion situation shall be noted in the annotation information. Among them, the truncated vehicle shall reasonably infer and annotate the complete 3D frame of the vehicle based on the actual shape and size of the vehicle, and the truncation situation shall be noted in the annotation information. Among them, for the occluded pedestrians, their approximate outlines shall be annotated based on the visible part of the point cloud data and the context information. Among them, for the truncated pedestrians, the complete outline of the pedestrians shall be inferred and annotated according to the actual situation. Among them, for the occluded cyclists, their approximate outlines shall be annotated based on the visible part of the point cloud data and the context information. Among them, for the truncated cyclists, the complete outlines of the cyclists shall be inferred and annotated according to the actual situation.

[0084] Step S20, constructing a model structure of a point cloud 3D object detection model based on a B-spline activation function.

[0085] In this embodiment, a KanLinear layer module is designed and introduced in the key feature encoding KFE module of the network model, abandoning the traditional fixed linear weight parameters and using a highly parameterized univariate spline function to represent each weight. With its flexible weight representation, it can more accurately capture the subtle changes and inherent laws in the data, thereby improving the model's fitting accuracy to the data. This is especially important for the detection of occluders or small targets in autonomous driving.

[0086] Specifically, introduce the KanLinear layer module into the network model. Before using the KanLinear layer module, print the number of input channels (in_channels) and the number of output channels (out_channels), which helps to debug and understand the structure of the model. Then use a conditional statement to determine whether to add a batch normalization layer after the KANLinear layer. When the judgment is true, create a nn.BatchNorm1d instance to normalize the output of the KANLinear layer. It takes the number of output channels (out_channels) as a parameter, and eps and momentum are optional parameters, which are used for numerical stability and moving average calculations, respectively.

[0087] In the backbone module of the network model, the Kan convolution module is designed and introduced, which deeply integrates the attention mechanism, making the convolution process more intelligent and dynamic. Kan convolution can dynamically adjust these weights according to the characteristics of the input image, significantly improving the model's ability to capture important information of the image, making the feature extraction process more efficient and targeted.

[0088] Specifically, in the feature extraction block, a Kan convolution layer is constructed for each layer specified by the element in layer_nums, each convolution layer uses the same number of input and output channels (num_filters[idx]), the convolution kernel size is 3×3, the padding is 1, there is no bias term, and each convolution layer is followed by a batch normalization layer. In the upsampling process, the upsampling decoding block is constructed according to the stride, the transposed Kan convolution is used for upsampling, and a normalization layer is added after each upsampling / decoding block.

[0089] In the Detection Head module of the network model, three Kan convolution modules are designed and introduced to facilitate better category prediction, bounding box prediction and direction classification. It is similar to the ordinary convolution layer in structure, but a learnable nonlinear function is applied to each edge or each element, thereby improving the expressive power of the convolution layer.

[0090] Specifically, a category prediction convolution layer is defined using a Kan convolution to predict the bounding box parameters corresponding to each anchor point. The number of output channels is the total number of anchor points multiplied by the encoding size of the bounding box encoder (self.box_coder.code_size). A bounding box prediction convolution layer is defined using a Kan convolution to predict the bounding box parameters corresponding to each anchor point. The number of output channels is the total number of anchor points multiplied by the encoding size of the bounding box encoder (self.box_coder.code_size). A direction classification convolution layer is defined using a Kan convolution to predict the target direction corresponding to each anchor point. The number of output channels is the total number of anchor points multiplied by the number of categories for direction classification.

[0091] Step S30: importing the training set into the point cloud 3D object detection model for training, using the test set to verify the point cloud 3D object detection model, and using the validation set to evaluate the point cloud 3D object detection model.

[0092] In this embodiment, in the step of importing the training set into the point cloud 3D object detection model for training, using the test set to verify the point cloud 3D object detection model, and using the verification set to evaluate the point cloud 3D object detection model, the step of importing the training set into the point cloud 3D object detection model for training includes:

[0093] Importing the training set into the point cloud 3D target detection model for training, and calculating the target detection loss according to a predetermined overall loss during the forward propagation calculation process;

[0094] Use the stochastic gradient descent algorithm for reverse derivation, and update the model parameters according to the gradient to complete the model training process of minimizing the loss function.

[0095] Specifically, the overall loss includes the total positioning loss, the target classification loss and the normalized classification loss. The overall loss value of the overall loss is obtained by the weighted average of the total positioning loss, the target classification loss and the normalized classification loss;

[0096] Among them, the total positioning loss includes center position loss, size loss, direction loss and rotation angle loss, and the target classification loss includes existence classification loss and category classification loss;

[0097] The calculation expression of the overall loss is:

[0098] ;

[0099] In the formula, Npos is the number of positive anchor points, is the total positioning loss, is the target classification loss, is the normalized classification loss, , , They are the weight coefficients of each loss, which are used to balance the importance of different loss items.

[0100] In this embodiment, the calculation expression of the total positioning loss is:

[0101] ;

[0102] In the formula, L center is the center position loss, L dim For size loss, L ori For direction loss, L rot is the rotation angle loss, λ center , dim , ori , rot They are the weight coefficients of each loss, which are used to balance the importance of different loss items;

[0103] Among them, the calculation expression of the total positioning loss is:

[0104] ;

[0105] In the formula, L center is the center position loss, L dim For size loss, L ori For direction loss, L rot is the rotation angle loss, λ center ,λ dim ,λ ori ,λ rot They are the weight coefficients of each loss, which are used to balance the importance of different loss items;

[0106] Among them, the calculation expression of the center position loss in the total positioning loss is:

[0107] ;

[0108] Where b is the bounding box, is the target detection regression loss function, is the positioning regression residual between the real box and the anchor box, (x, y, z) is the position coordinate of the real box;

[0109] Among them, the positioning regression residual between the real box and the anchor box is defined as:

[0110] ;

[0111] In the formula, is the X-axis coordinate of the real frame, is the X-axis coordinate of the anchor box, is the Y-axis coordinate of the real frame, is the Y-axis coordinate of the anchor box, is the Z-axis coordinate of the real frame, is the Z-axis coordinate of the anchor box;

[0112] Among them, the calculation expression of size loss in the total positioning loss is:

[0113] ;

[0114] In the formula, is the width of the real frame, is the width of the anchor box, is the length of the real frame, is the length of the anchor box, is the height of the real frame, is the height of the anchor box;

[0115] Among them, the calculation expression of direction loss in the total positioning loss is:

[0116] ;

[0117] In the formula, is the rotation angle of the real frame, is the rotation angle of the anchor box;

[0118] Among them, the calculation expression of the rotation angle loss in the total positioning loss is:

[0119] ;

[0120] In the formula, is the rotation angle of the real frame, The rotation angle of the anchor box.

[0121] In this embodiment, the calculation expression of the target classification loss is:

[0122] ;

[0123] In the formula, L exist is the existence classification loss, L classis the category classification loss, and λ is a hyperparameter used to balance the importance of existence classification loss and category classification loss;

[0124] Among them, in the target classification loss, the calculation expression of the existence classification loss is:

[0125] ;

[0126] In the formula, N for Pillar total, p exist express Pillar The probability that a target exists in y exist is the true label, y exist The value is 0 or 1;

[0127] Among them, in the target classification loss, the calculation expression of the category classification loss is:

[0128] ;

[0129] In the formula, M For the purpose of existence Pillar quantity, C is the total number of categories, p class is the output probability of the model for each category, y class is the true label.

[0130] According to one aspect of the above technical solution, the calculation expression of the normalized classification loss is:

[0131] ;

[0132] In the formula, N for Pillar total, C is the total number of categories, y ic For sample i Belongs to category c The real label, p ic For sample i Belongs to category c The predicted probability of

[0133] in, , Zic For the model to sample i Belongs to category c The original output of Zij For sample i For Categoryj Output.

[0134] Step S40: deploy the point cloud 3D target detection model in the vehicle, and use the point cloud 3D target detection model to detect various road participants and scenes with different degrees of occlusion and truncation in different driving scenarios.

[0135] Specifically, after the point cloud 3D target detection model is constructed according to the above modules, and trained, verified and evaluated according to the above process, the final point cloud 3D target detection model will be obtained. At this time, it is only necessary to deploy the point cloud 3D target detection model in the vehicle, specifically in the control module of the vehicle used for intelligent driving decision-making. When the vehicle obtains the corresponding perception data through the perception device during driving, the real-time collected perception data is automatically imported into the point cloud 3D target detection model. The point cloud 3D target detection model is used to detect various road participants in different driving scenarios and scenes with different degrees of occlusion and truncation, thereby providing more accurate decision-making basis for the vehicle's intelligent driving.

[0136] In a more specific example, by obtaining image data of vehicles, pedestrians, cyclists and different degrees of occlusion and truncation in different scenes such as urban areas, rural areas, and highways, the data is preprocessed and annotated to obtain a data set that can be used for network model training, and the structure of the 3D point cloud target detection model based on the B-spline activation function is constructed, including designing and introducing the KanLinear layer module in the key feature encoding KFE module of the network model, designing and introducing the Kan convolution module in the backbone module of the network model, and designing and introducing 3 Kan convolution modules in the Detection Head module of the network model, so as to better perform category prediction, bounding box prediction and direction classification. By training the training set into the detection model, the model is verified using the test set and the model performance is evaluated using the validation set, and finally the detection target in autonomous driving is accurately detected.

[0137] In summary, this embodiment uses point cloud 3D target detection technology based on B-spline activation function to detect different scenes in autonomous driving, including vehicle scenes, pedestrian scenes, cyclist scenes, and scenes with different degrees of occlusion and truncation. This is conducive to accurately judging road conditions and predicting the behavior of pedestrians and cyclists, thereby improving driving efficiency and driving safety, enhancing system adaptability and robustness, and reducing the occurrence of traffic accidents.

[0138] Embodiment 2

[0139] See also Figure 2The second embodiment of the present invention provides a point cloud 3D target detection system, which is applied to the method described in the above embodiment. The system includes: a data acquisition module 10, a model construction module 20, a model training module 30 and a target detection module 40.

[0140] The data acquisition module 10 is used to acquire image data of various road participants and different degrees of occlusion and truncation in different driving scenes, pre-process the image data to obtain a target data set, and divide the target data set into a training set, a test set and a validation set according to a preset ratio;

[0141] A model building module 20 is used to build a model structure of a point cloud 3D object detection model based on a B-spline activation function;

[0142] A model training module 30 is used to import the training set into the point cloud 3D object detection model for training, verify the point cloud 3D object detection model using the test set, and evaluate the point cloud 3D object detection model using the validation set;

[0143] The target detection module 40 is used to deploy the point cloud 3D target detection model in the vehicle, and use the point cloud 3D target detection model to detect various road participants and scenes with different degrees of occlusion and truncation in different driving scenarios.

[0144] This embodiment uses point cloud 3D target detection technology based on B-spline activation function to detect different scenes in autonomous driving, including vehicle scenes, pedestrian scenes, cyclist scenes and scenes with different degrees of occlusion and truncation. This is conducive to accurately judging road conditions and predicting the behavior of pedestrians and cyclists, thereby improving driving efficiency and driving safety, enhancing system adaptability and robustness, and reducing the occurrence of traffic accidents.

[0145] Embodiment 3

[0146] A third embodiment of the present invention provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method described in the above embodiment is implemented.

[0147] Embodiment 4

[0148] A fourth embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the method described in the above embodiments when executing the computer program.

[0149] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0150] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A point cloud 3D object detection method, characterized in that: The method comprises: Obtain image data of various road participants and different degrees of occlusion and truncation in different driving scenarios, preprocess the image data to obtain the target data set, and divide the target data set into training set, test set and validation set according to the preset ratio; Construct the model structure of the point cloud 3D object detection model based on B-spline activation function; Importing the training set into the point cloud 3D object detection model for training, using the test set to verify the point cloud 3D object detection model, and using the validation set to evaluate the point cloud 3D object detection model; The point cloud 3D object detection model is deployed in a vehicle, and the point cloud 3D object detection model is used to detect various road participants and scenes with different degrees of occlusion and truncation in different driving scenarios; Among them, in the steps of importing the training set into the point cloud 3D object detection model for training, using the test set to verify the point cloud 3D object detection model, and using the verification set to evaluate the point cloud 3D object detection model, the step of importing the training set into the point cloud 3D object detection model for training includes: Importing the training set into the point cloud 3D target detection model for training, and calculating the target detection loss according to a predetermined overall loss during the forward propagation calculation process; Use the stochastic gradient descent algorithm for reverse derivation and update the model parameters according to the gradient to complete the model training process of minimizing the loss function; The overall loss includes the total positioning loss, the target classification loss and the normalized classification loss, and the overall loss value of the overall loss is obtained by weighted average of the total positioning loss, the target classification loss and the normalized classification loss; The total positioning loss includes center position loss, size loss, direction loss and rotation angle loss, and the target classification loss includes existence classification loss and category classification loss; The calculation expression of the overall loss is: ； In the formula, Npos is the number of positive anchor points, is the total positioning loss, is the target classification loss, is the normalized classification loss, , , They are the weight coefficients of each loss, which are used to balance the importance of different loss items; Among them, the calculation expression of the total positioning loss is: ； In the formula, L center is the center position loss, L dim For size loss, L ori For direction loss, L rot is the rotation angle loss, λ center , dim , ori , rot They are the weight coefficients of each loss, which are used to balance the importance of different loss items; Among them, in the total positioning loss, the calculation expression of the center position loss is: ； Where b is the bounding box, is the target detection regression loss function, is the positioning regression residual between the real box and the anchor box, (x, y, z) is the position coordinate of the real box; Among them, the positioning regression residual between the real box and the anchor box is defined as: ； In the formula, is the X-axis coordinate of the real frame, is the X-axis coordinate of the anchor box, is the Y-axis coordinate of the real frame, is the Y-axis coordinate of the anchor box, is the Z-axis coordinate of the real frame, is the Z-axis coordinate of the anchor box; Among them, in the total positioning loss, the calculation expression of the size loss is: ； In the formula, is the width of the real frame, is the width of the anchor box, is the length of the real frame, is the length of the anchor box, is the height of the real frame, is the height of the anchor box; Among them, in the total positioning loss, the calculation expression of the direction loss is: ； In the formula, is the rotation angle of the real frame, is the rotation angle of the anchor box; Among them, in the total positioning loss, the calculation expression of the rotation angle loss is: ； In the formula, is the rotation angle of the real frame, is the rotation angle of the anchor box; Among them, the calculation expression of the target classification loss is: ； In the formula, L exist is the existence classification loss, L class is the category classification loss, and λ is a hyperparameter used to balance the importance of existence classification loss and category classification loss; Among them, in the target classification loss, the calculation expression of the existence classification loss is: ； In the formula, N for Pillar total, p exist express Pillar The probability that a target exists in y exist is the true label, y exist The value is 0 or 1; Among them, in the target classification loss, the calculation expression of the category classification loss is: ； In the formula, M For the purpose of existence Pillar quantity, C is the total number of categories, p class is the output probability of the model for each category, y class is the true label; Among them, the calculation expression of the normalized classification loss is: ； In the formula, N for Pillar total, C is the total number of categories, y ic For sample i Belongs to category c The real label, p ic For sample i Belongs to category c The predicted probability of in, , Zic For the model to sample i Belongs to category c The original output of Zij For sample i For Category j Output.

2. The point cloud 3D target detection method according to claim 1, characterized in that: The steps of obtaining image data of various road participants and different degrees of occlusion and truncation in different driving scenarios, preprocessing the image data to obtain a target data set, and dividing the target data set into a training set, a test set, and a validation set according to a preset ratio include: Obtain a target data set containing image data and lidar point cloud data collected in actual road scenes, or obtain image data of various road participants and different degrees of occlusion and truncation in different driving scenes to obtain a target data set; Preprocessing the target data set and performing data annotation; The target data set is divided into training set, test set and validation set in a ratio of 7:3:

3.

3. A point cloud 3D object detection system, characterized in that: The system is applied to the method described in any one of claims 1 to 2, and the system comprises: A data acquisition module is used to acquire image data of various road participants and different degrees of occlusion and truncation in different driving scenarios, pre-process the image data to obtain a target data set, and divide the target data set into a training set, a test set, and a validation set according to a preset ratio; A model building module is used to build the model structure of the point cloud 3D object detection model based on the B-spline activation function; A model training module, used to import the training set into the point cloud 3D object detection model for training, verify the point cloud 3D object detection model using the test set, and evaluate the point cloud 3D object detection model using the validation set; The target detection module is used to deploy the point cloud 3D target detection model in the vehicle, and use the point cloud 3D target detection model to detect various road participants and scenes with different degrees of occlusion and truncation in different driving scenarios.

4. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 2 is implemented.

5. A computer device comprising a memory, a processor, and a computer program stored in the memory and capable of being executed on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 2 is implemented.

Images