Method for improving training effect of pointpillars model

Abstract

The invention is suitable for the technical field of model training of deep learning, and provides a method for improving the training effect of a pointpillars model, and the method comprises the steps: S1, carrying out the voxelization of input point cloud data to generate corresponding tensor data D * P * N, D being the dimension of an augmented vector of each point cloud, P being the number of point cloud columns pillars mapped by an original point cloud, and N being the number of point clouds in each pillar; according to the method, image feature information lost along with deepening of the number of network layers is reserved, more return gradient information is obtained in the back propagation process, the quality of model training can be effectively improved, auxiliary branches are abandoned during reasoning, and the target detection effect of the pointpillars model is effectively improved while the reasoning time is kept unchanged.

Description

Technical Field

[0001] This invention belongs to the field of deep learning model training technology, and in particular relates to a method for improving the training effect of pointpillars models. Background Technology

[0002] In the development of autonomous driving technology, 3D object detection has always been a crucial field. Autonomous vehicles need to accurately detect and identify various objects on the road, such as vehicles, pedestrians, and traffic signs, to ensure safe driving and effective traffic planning. In recent years, the rapid development of deep learning technology has brought new opportunities for autonomous driving object detection. By training on point cloud datasets, deep learning models can automatically learn and extract features from point clouds, thereby achieving accurate object detection and recognition, representing their position and pose with 3D bounding boxes. This is of great significance for environmental perception and decision-making in autonomous driving.

[0003] However, despite the significant progress made by deep learning models in object detection, several challenges remain in autonomous driving applications. One major challenge is balancing inference speed and accuracy. Traditionally, improving model accuracy often requires increasing model complexity and the number of parameters, which can lead to increased inference time.

[0004] In autonomous driving scenarios, delays in inference time can prevent vehicles from reacting in a timely manner, thus affecting driving safety. Increasing model complexity to improve inference and detection performance is not suitable for autonomous driving. Pointpillars models, as classic point cloud detection models, are widely used in autonomous driving due to their low computation time. However, this low computation time and low model complexity correspond to poor target detection performance. In contrast, this invention provides a method to improve the training performance of pointpillars models to address the aforementioned technical problems. Summary of the Invention

[0005] The purpose of this invention is to provide a method for improving the training effect of pointpillars models, aiming to solve the problems mentioned in the background art.

[0006] The present invention is implemented as follows: a method to improve the training effect of the PointPillars model is to connect a new auxiliary invertible branch after the PointPillars model generates pseudo-images, add three new detection heads, and obtain more return gradient information. The method includes the following steps: S1: For the input point cloud data, voxelize it to generate corresponding tensor data D*P*N, where D is the dimension of the augmented vector of each point cloud, P is the number of point cloud pillars mapped from the original point cloud, and N is the number of point clouds in each pillar. Then, according to the Scatter operator, the read Pillar features are transformed into pseudo-feature maps H*W*C, where H is the length of the pseudo-image, W is the width of the pseudo-image, and C is the number of features in each cell. S2: Compared with the original Pointpillars structure, a new reversible auxiliary branch is introduced. The feature map of H*W*C is propagated forward through this branch and the features extracted from the original branch are fused to obtain the feature map of the new branch. S3: Process the feature maps extracted from the original Pointpillars main branch and the feature maps extracted from the new branch separately. The corresponding detection head is convolved to output the corresponding category, orientation, and size, and the loss is calculated by comparing it with the true value; S4: Update the parameters of the main branch and auxiliary branches by backpropagation based on the loss, and iterate through training to obtain... The trained model; S5: Use the trained model for inference, discarding auxiliary branches, and only process the point cloud through the main branch. Voxelization and feature extraction are performed to obtain the final 3D target detection results.

[0007] As a further limitation of the technical solution of the embodiments of the present invention,

[0008] As a further limitation of the technical solution of this embodiment of the invention, in step S2, the reversible auxiliary branch refers to obtaining a feature map by passing the generated pseudo image through 4 convolution kernels, obtaining three feature maps by passing the three downsampled feature maps of the main branch through the CBLinear layer to unify the dimensions, and finally stitching these 4 feature maps together and inputting them into the SSD detection head to obtain three outputs.

[0009] As a further limitation of the technical solution of this embodiment of the invention, in step S3, the Loss includes three types: First, the SmoothL1 loss function is used to regress the seven variables: the center point x, y, z, length, width, height l, h, w, and orientation angle yaw. Secondly, the FocalLoss loss function is used to calculate the classification loss; Thirdly, the SoftMax loss function is used to learn the orientation of objects; The final total loss is the weighted sum of the three losses mentioned above.

[0010] Compared with the prior art, the beneficial effects of the present invention are as follows: It retains the image feature information lost as the network depth increases, obtains more return gradient information during backpropagation, which can effectively improve the quality of model training. Furthermore, during inference, it discards auxiliary branches, effectively improving the target detection performance of the PointPillars model while keeping the inference time constant. Attached Figure Description

[0011] Figure 1 A flowchart of the method provided in the embodiments of the present invention; Figure 2 The method provided in this embodiment of the invention generates pseudo-images by voxelizing point clouds; Figure 3 The model structure of the method provided in the embodiments of the present invention. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0013] Figure 1 A flowchart of the method provided by an embodiment of the present invention is shown.

[0014] Specifically, this invention provides a method to improve the training performance of PointPillars models. More specifically, it is a method to improve the inference performance of models by increasing the training cost while keeping the inference time constant. This invention can be widely applied in various fields such as autonomous driving and industrial automation. The method specifically includes the following steps: S1: Point cloud preprocessing voxelization to generate pseudo-images; For the input point cloud data, it is voxelized to generate corresponding tensor data D*P*N, where D is the dimension of the augmented vector of each point cloud, P is the number of point cloud pillars mapped from the original point cloud, and N is the number of point clouds in each pillar. Then, according to the Scatter operator, the read Pillar features are transformed into pseudo-feature maps H*W*C, where H is the length of the pseudo-image, W is the width of the pseudo-image, and C is the number of features in each cell. S2: Features are extracted through the main branch and auxiliary branches; Compared to the original Pointpillars structure, a new reversible auxiliary branch is introduced. The feature map of H*W*C is propagated forward through this branch, and the features extracted from the original branch are fused to obtain the feature map of the new branch. S3: Input the feature maps extracted from the two branches into the detection head respectively; The feature maps extracted from the original Pointpillars main branch and the feature maps extracted from the new branch are convolved through the corresponding detection heads to output the corresponding category, orientation and size, and the loss is calculated with the true value. S4: Training the model; Backpropagation is performed based on the loss to update the parameters of the main branch and auxiliary branches, and iterative training is performed to obtain the trained model. S5: Discard the auxiliary branch reasoning model; The trained model is used for inference. Auxiliary branches are discarded, and the point cloud is only processed through the main branch for voxelization and feature extraction to obtain the final 3D object detection result.

[0015] In this embodiment of the invention, by increasing the training cost, the object detection performance of the PointPillars model is improved while keeping the inference cost constant. The model structure mentioned in this invention will be described in detail below with reference to the accompanying drawings; In step S1, the original point cloud is voxelized according to the preprocessing of the PointPillars model and mapped onto the pseudo-image, as shown in the attached figure. Figure 2 As shown; In step S2, the auxiliary reversible branch is as follows: Figure 3 As shown, the pseudo-image passes through the branch indicated by the dashed line, merges with the feature map extracted from the main branch, outputs the final result to the detection head, and calculates the loss of this branch for backpropagation.

[0016] Furthermore, in the embodiments of the present invention: In step S2, the reversible auxiliary branch refers to passing the generated pseudo image through four convolutional kernels to obtain a feature map, passing the three downsampled feature maps from the main branch through a CBLinear layer to unify the dimensions to obtain three feature maps, and finally concatenating these four feature maps together and inputting them into the SSD detection head to obtain three outputs.

[0017] Furthermore, in the embodiments of the present invention: In step S3, the Loss includes three types: First, the SmoothL1 loss function is used to regress the seven variables: the center point x, y, z, length, width, height l, h, w, and orientation angle yaw. Secondly, the FocalLoss loss function is used to calculate the classification loss; Thirdly, the SoftMax loss function is used to learn the orientation of objects; The final total loss is the weighted sum of the three losses mentioned above.

[0018] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0019] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0020] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0021] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

[0022] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for improving the training effect of a pointpillars model, characterized in that: After the pointpillars model generates a pseudo image, a new auxiliary reversible branch is connected and three new detection heads are added to obtain more return gradient information; The method comprises the following steps: S1: For the input point cloud data, voxelize it to generate the corresponding tensor data D*P*N, where D is the dimension of the augmented vector of each point cloud, P is the number of point cloud pillars mapped by the original point cloud, and N is the number of point clouds in each pillar. Then, according to the Scatter operator, the read Pillar features are converted to pseudo feature maps H*W*C, where H is the length of the pseudo image, W is the width of the pseudo image, and C is the number of features of each grid; S2: Compared with the original Pointpillars structure, a new reversible auxiliary branch is introduced, the feature map of H*W*C is forward propagated through this branch, and the features extracted from the original branch are fused to obtain the feature map of the new branch; S3: The feature map extracted by the original Pointpillars main branch and the feature map extracted by the new branch are respectively The corresponding detection head convolution outputs the corresponding category, direction and size, and calculates the Loss loss with the true value; S4: Update the parameters of the main branch and auxiliary branch by back propagation according to the Loss, and iterate the training to obtain The trained model; S5: Use the trained model for reasoning, discard the auxiliary branch, and the point cloud only passes through the main branch Voxelization and feature extraction are performed to obtain the final 3D object detection results.

2. The method for improving the training effect of the pointpillars model according to claim 1, characterized in that: In step S2, the reversible auxiliary branch refers to obtaining a feature map by passing the generated pseudo image through 4 convolution kernels, unifying the dimensions of the three downsampled feature maps of the main branch through the CBLinear layer to obtain three feature maps, and finally splicing these four feature maps together, inputting them into the SSD detection head, and obtaining three outputs.

3. The method for improving the training effect of the pointpillars model according to claim 1, characterized in that: In step S3, the Loss includes three types: First, the SmoothL1 loss function is used to regress the seven variables of the detection box center point x, y, z, length, width, height l, h, w and direction angle yaw; Second, the FocalLoss loss function is used to calculate the classification loss; Third, it is the SoftMax loss function used to learn the object direction; The final total Loss is the weighted sum of the above three Losses.

Images