A laser radar environment perception method based on contrast learning

Abstract

The application discloses a kind of laser radar environment perception methods based on contrast learning, comprising the following steps: S1.utilizes laser radar scanning, collects the laser point cloud data of target environment, and pre-processes laser point cloud data, obtains point cloud feature data;S2.building MLP dimension reduction model, the feature of each point in point cloud feature data is reduced, filtered and learned after update, and mature MLP dimension reduction model is obtained;S3.building MLP classification model, and with the original classification branch and rebalancing branch are combined, gradient is calculated and weight is updated reversely, and mature MLP classification model is obtained;S4.environment perception is carried out to the point cloud data newly collected in target environment.The application effectively realizes the balance of training process sample, effectively guarantees the classification precision of model obtained by training.

Description

Technical Field

[0001] This invention relates to environmental perception, and in particular to a lidar environmental perception method based on contrastive learning. Background Technology

[0002] LiDAR is a commonly used sensor, and detecting point cloud information acquired by LiDAR is a key issue in autonomous driving. While 2D image detection technology has matured, understanding and analyzing 3D targets remains a significant challenge. Currently, most LiDAR-based point cloud detection algorithms are trained for classification based on the original point cloud data before detection and classification.

[0003] Point cloud perception methods are mostly applied in outdoor scenarios, such as road scenes. Directly using raw points for classification learning in outdoor scenes may lead to imbalanced training of the classification model; existing perception methods do not pay much attention to this problem in their design. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a lidar environment perception method based on contrastive learning, which effectively achieves the balance of samples in the training process and effectively ensures the classification accuracy of the trained model.

[0005] The objective of this invention is achieved through the following technical solution: a lidar environment perception method based on contrastive learning, comprising the following steps:

[0006] S1. Use lidar to scan and collect lidar point cloud data of the target environment, and preprocess the lidar point cloud data to obtain point cloud feature data;

[0007] S2. Construct an MLP dimensionality reduction model. After performing dimensionality reduction, filtering, and learning updates on the features of each point in the point cloud feature data, a mature MLP dimensionality reduction model is obtained.

[0008] S3. Construct an MLP classification model, and combine the original classification branch and the rebalancing branch to calculate the gradient and update the weights in reverse, thus obtaining a mature MLP classification model.

[0009] S4. After preprocessing the newly acquired point cloud data in the target environment according to step S1, new point cloud feature data is obtained. The features of each point in the new point cloud feature data are sent to a mature MLP dimensionality reduction model for dimensionality reduction, and then sent to a mature MLP classification model. The MLP classification model outputs the classification result of each point, which is used as the environmental perception result of the lidar.

[0010] The beneficial effects of this invention are as follows: First, this invention constructs an MLP dimensionality reduction model for training in order to obtain the comparative features after dimensionality reduction. Then, based on the combination of the original classification branch and the rebalancing branch, the training of the MLP classification model is completed. In the early stage of training, the focus is on learning the rebalancing branch. As training progresses, the focus gradually shifts to the original classification branch, which effectively achieves the balance of samples in the training process and effectively ensures the classification accuracy of the trained model. Attached Figure Description

[0011] Figure 1 This is a flowchart of the method of the present invention;

[0012] Figure 2 This is a schematic diagram illustrating the specific principles of this application. Detailed Implementation

[0013] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the following description.

[0014] like Figures 1-2 As shown, a lidar environment perception method based on contrastive learning includes the following steps:

[0015] S1. Use lidar to scan and collect lidar point cloud data of the target environment, and preprocess the lidar point cloud data to obtain point cloud feature data;

[0016] The preprocessing process includes:

[0017] S101. The collected laser point cloud data is used as the input point cloud. The input point cloud contains multiple points, each of which has a truth value. The truth value category and truth box of each point are marked.

[0018] S102. Use a deep model to extract features from the input point cloud to obtain point cloud feature data; the point cloud feature data contains multiple points and the features of each point;

[0019] For the deep learning model, PointNet++ can be used. This model mainly performs downsampling on the input point cloud, as well as KNN and feature aggregation operations to extract multiple points, each with a corresponding feature. Feature aggregation is implemented using max pooling.

[0020] S2. Construct an MLP dimensionality reduction model. After performing dimensionality reduction, filtering, and learning updates on the features of each point in the point cloud feature data, a mature MLP dimensionality reduction model is obtained:

[0021] An MLP dimensionality reduction model is constructed using the MLP algorithm. After dimensionality reduction of the features of each point, the resulting points are filtered, the comparison point loss is calculated, and the dimensionality reduction model is updated to complete the learning of the dimensionality reduction model.

[0022] S201. Construct an MLP dimensionality reduction model based on the MLP algorithm. This dimensionality reduction model can be viewed as a comparison point encoder:

[0023] The MLP dimensionality reduction model has more nodes in the input layer than in the output layer, and the number of nodes in the hidden layer is the same as the number of nodes in the input layer. The activation function is ReLU. The number of nodes in the output layer corresponds to the dimension before feature dimensionality reduction, and the number of nodes in the output layer corresponds to the dimension after feature dimensionality reduction.

[0024] The features of each point obtained in step S1 are fed into the MLP model for dimensionality reduction to obtain comparative features;

[0025] Contrastive learning, as a common pre-training paradigm, has garnered significant attention in the image processing domain. Unlike commonly used supervised learning, contrastive learning does not rely on large amounts of labeled data. Its core principle is to learn the similarity between positive sample pairs and to extrapolate the distance between negative sample pairs in the feature space. In this way, the model is guided to implicitly classify data during training, providing better initialization parameters for subsequent downstream tasks.

[0026] S202. Use a point filter to filter each point after dimensionality reduction:

[0027] During filtering, the score for each point is first calculated, and then points with scores below the set score threshold are filtered out.

[0028] The point filter calculates the score for each point as follows:

[0029]

[0030] Where f, b, l, r, u, d represent the distances of the point from the front, back, left, right, top, and bottom surfaces of the truth box, respectively.

[0031] S203. For the filtered points, optimize the model based on the comparison point loss:

[0032] The optimization process involves calculating a loss function. As the loss function decreases during the learning process, the weights are updated to achieve the desired effect. The loss function aims to bring the feature distance between positive samples closer and widen the feature distance between negative samples.

[0033] Specifically, in the point contrast loss method, let the point cloud formed by the filtered points be... The contrast representation of the i-th point, y i s represents the truth value of the i-th point. i represents the score of the i-th point, and N represents the number of filtered points;

[0034] The comparison point loss is defined as follows:

[0035]

[0036]

[0037] Where Q(i) is the set of points excluding itself, and P(i) is the set of points whose truth class is the same as that of the i-th point.

[0038] Based on the comparison point loss, the MLP dimensionality reduction model is updated using gradient backpropagation, and the above update is repeated at each point to obtain a mature MLP dimensionality reduction model.

[0039] S3. Construct an MLP classification model, and combine the original classification branch and the rebalancing branch to calculate the gradient and update the weights in reverse, thus obtaining a mature MLP classification model.

[0040] S301. Construct an MLP classification model based on the MLP neural network algorithm; in this classification model, the number of hidden layer nodes is equal to the number of input layer nodes, and the activation function is ReLU;

[0041] The classification categories are predefined; the input layer nodes are used to input the features of the points to be classified, and each node in the output layer is assigned a classification category; the output of each node in the output layer is normalized using softmax to obtain the probability of belonging to each category;

[0042] S302. For each point obtained in step S1, calculate the score as follows:

[0043]

[0044] Where f, b, l, r, u, d represent the distances of the point from the front, back, left, right, top, and bottom surfaces of the truth box, respectively.

[0045] Let the score at point i be denoted as i = 1, 2, ..., N, where N represents the number of points obtained in step S1;

[0046] S303. Prediction is performed under the original classification branch:

[0047] For each point obtained in step S1, its features are directly input into the MLP classification model to obtain the classification result, and the probability of the point belonging to each category is obtained by normalization through softmax.

[0048] Let p be the vector of probabilities of the i-th point in each category. i , i = 1, 2, ..., N;

[0049] S304. Prediction under the rebalanced classification branch:

[0050] For each point obtained in step S1, it is fed into a mature MLP dimensionality reduction model for dimensionality reduction, then input into an MLP classification model to obtain the classification result, and normalized by softmax to obtain the probability that the point belongs to each category.

[0051] Let the vector of probabilities of the i-th point in each category be denoted as . i = 1, 2, ..., N;

[0052] S305. Calculate the original classification branch loss:

[0053]

[0054] Calculate the classification loss of the rebalanced branch:

[0055]

[0056] S306. Let the total number of training epochs be cur_epoch and the total number of training epochs be total_epoch. Then the parameter α introduced in the current epoch is:

[0057]

[0058] S307. Calculate the overall loss for the current round using parameter α:

[0059] L cls =α·L Bcls +(1-α)·L Vcls

[0060] In the early stages of training, the focus is on learning the rebalanced branch, and as training progresses, the focus gradually shifts to the original classification branch.

[0061] After obtaining the overall classification loss, gradient backpropagation is used to update the MLP classification model based on the overall classification loss to complete the training of the current round. Then, steps S303 to S307 are repeated until the total number of training rounds reaches total_epoch. At this point, a mature MLP classification model is obtained.

[0062] S4. After preprocessing the newly acquired point cloud data according to step S1, point cloud feature data is obtained. The features of each point in the point cloud feature data are sent to a mature MLP dimensionality reduction model for dimensionality reduction, and then sent to a mature MLP classification model. The MLP classification model outputs the classification result of each point, which is used as the environmental perception result of the lidar.

[0063] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A lidar environment perception method based on contrastive learning, characterized in that: Includes the following steps: S1. Use lidar to scan and collect lidar point cloud data of the target environment, and preprocess the lidar point cloud data to obtain point cloud feature data; S2. Construct an MLP dimensionality reduction model. After performing dimensionality reduction, filtering, and learning updates on the features of each point in the point cloud feature data, a mature MLP dimensionality reduction model is obtained. Step S2 includes the following sub-steps: S201. Constructing an MLP dimensionality reduction model based on the MLP algorithm: The MLP dimensionality reduction model has more nodes in the input layer than in the output layer, and the number of nodes in the hidden layer is the same as the number of nodes in the input layer. The activation function is ReLU. The number of nodes in the output layer corresponds to the dimension before feature dimensionality reduction, and the number of nodes in the output layer corresponds to the dimension after feature dimensionality reduction. The features of each point obtained in step S1 are fed into the MLP model for dimensionality reduction to obtain comparative features; S202. Use a point filter to filter each point after dimensionality reduction: During filtering, the score for each point is first calculated, and then points with scores below the set score threshold are filtered out. The point filter calculates the score for each point as follows: Where f, b, l, r, u, d represent the distances of the point from the front, back, left, right, top, and bottom surfaces of the truth box, respectively. S203. For the filtered points, optimize the model based on the comparison point loss: The optimization process involves calculating a loss function; as the loss function decreases during the learning process, the weights are updated. Let the point cloud formed by the points obtained after filtering be... The contrast representation of the i-th point, y i s represents the truth value of the i-th point. i represents the score of the i-th point, and N represents the number of filtered points; The comparison point loss is defined as follows: Where Q(i) is the set of points excluding itself, and P(i) is the set of points whose truth class is the same as that of the i-th point; Based on the comparison point loss, the MLP dimensionality reduction model is updated using gradient backpropagation, and the above update is repeated at each point to obtain a mature MLP dimensionality reduction model. S3. Construct an MLP classification model, and combine the original classification branch and the rebalancing branch to calculate the gradient and update the weights in reverse, thus obtaining a mature MLP classification model. S4. After preprocessing the newly acquired point cloud data in the target environment according to step S1, new point cloud feature data is obtained. The features of each point in the new point cloud feature data are sent to a mature MLP dimensionality reduction model for dimensionality reduction, and then sent to a mature MLP classification model. The MLP classification model outputs the classification result of each point, which is used as the environmental perception result of the lidar.

2. The lidar environment perception method based on contrastive learning according to claim 1, characterized in that: The preprocessing of laser point cloud data in step S1 includes: S101. The collected laser point cloud data is used as the input point cloud. The input point cloud contains multiple points, each of which has a truth value. The truth value category and truth box of each point are marked. S102. Use a deep model to extract features from the input point cloud to obtain point cloud feature data; the point cloud feature data contains multiple points and the features of each point.

3. The lidar environment perception method based on contrastive learning according to claim 1, characterized in that: Step S3 includes: S301. Construct an MLP classification model based on the MLP neural network algorithm; in this classification model, the number of hidden layer nodes is equal to the number of input layer nodes, and the activation function is ReLU; The classification categories are predefined; the input layer nodes are used to input the features of the points to be classified, and each node in the output layer is assigned a classification category; the output of each node in the output layer is normalized using softmax to obtain the probability of belonging to each category; S302. For each point obtained in step S1, calculate the score as follows: Where f, b, l, r, u, d represent the distances of the point from the front, back, left, right, top, and bottom surfaces of the truth box, respectively. Let the score at point i be denoted as i = 1, 2, ..., N, where N represents the number of points obtained in step S1; S303. Predict under the original classification branch: For each point obtained in step S1, its features are directly input into the MLP classification model to obtain the classification result, and the probability of the point belonging to each category is obtained by normalization through softmax. Let p be the vector of probabilities of the i-th point in each category. i , i = 1, 2, ..., N; S304. Prediction under the rebalanced classification branch: For each point obtained in step S1, it is fed into a mature MLP dimensionality reduction model for dimensionality reduction, then input into an MLP classification model to obtain the classification result, and normalized by softmax to obtain the probability that the point belongs to each category. Let the vector of probabilities of the i-th point in each category be denoted as . i = 1, 2, ..., N; S305. Calculate the original classification branch loss: Calculate the classification loss of the rebalanced branch: S306. Let the total number of training epochs be cur_epoch and the total number of training epochs be total_epoch. Then the parameter α introduced in the current epoch is: S307. Calculate the overall loss for the current round using parameter α: L cls =α·L Bcls +(1-a)·L Vcls In the early stages of training, the focus is on learning the rebalanced branch, and as training progresses, the focus gradually shifts to the original classification branch. After obtaining the overall classification loss, gradient backpropagation is used to update the MLP classification model based on the overall classification loss to complete the training of the current round. Then, steps S303 to S307 are repeated until the total number of training rounds reaches total_epoch. At this point, a mature MLP classification model is obtained.

Images