A point cloud intensity prediction method and system fusing physical modeling

Abstract

This invention discloses a point cloud intensity prediction method and system that integrates physical modeling. The method includes: acquiring point cloud data and its corresponding geometric features and semantic information through sensors; converting the point cloud data into a two-dimensional multi-channel feature map to construct a dataset, and dividing the dataset into a training set, a validation set, and a test set; establishing a neural network model that integrates physical modeling; inputting the training set, validation set, and test set into the neural network model to complete model training and testing; during model training, calculating the loss based on the predicted intensity value output by the data-driven branch and the physical predicted intensity value output by the fusion correction module, and optimizing the neural network model based on the loss; inputting the point cloud data to be predicted into the trained neural network model to obtain the final predicted point cloud intensity. This invention improves data utilization and system robustness, and has significant application value in fields such as simulation data generation and target detection.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a point cloud intensity prediction method and system that integrates physical modeling. Background Technology

[0002] LiDAR (Light Detection and Ranging) simultaneously outputs three-dimensional coordinates and reflection intensity. Intensity information is crucial for material identification, SLAM (Simultaneous Localization and Mapping) loop closure, and target detection. For example, in target detection, when multiple targets are closely packed together (such as in a parking lot scenario), a pure geometric point cloud often only provides a uniform mass of points. However, with intensity information, it's possible to clearly identify whether a point is a car body or concrete. Therefore, obtaining more accurate and realistic intensity information is very helpful for LiDAR-related work.

[0003] Existing methods generally rely solely on pixel-level or point-level numerical error losses (such as L1 or L2 losses) during training, lacking explicit modeling and constraints on the physical laws of optical reflection. This leads to physical inconsistencies in the point cloud intensity predicted by the model, for example: The intensity of the same material at different incident angles does not follow a cosine decay law: For a uniform material surface, the predicted intensity fails to decay reasonably with increasing laser incident angle. The predicted reflectance of the same material fluctuates greatly at different distances: Due to the lack of explicit modeling of distance decay effects, the model's estimates of reflectance for the same material at near, medium, and far distances may differ significantly, severely violating optical reflectance consistency. There is a lack of constraints on the consistency of material reflectance: During training, the model does not enforce that regions with the same semantic labels or color features have similar reflectances, leading to physically unreasonable prediction results and affecting the reliability of downstream tasks such as material recognition and object classification.

[0004] Most existing methods learn reflectance as a latent variable within the network, failing to establish an explicit and stable mapping relationship between reflectance and observable and interpretable external information (such as RGB color and semantic category). This results in poor interpretability of prediction results, making it difficult for users and downstream systems to understand why the model outputs a certain reflectance value, and also making it impossible to manually verify or intervene in reflectance based on color or semantic information. Summary of the Invention

[0005] The purpose of this invention is to provide a point cloud intensity prediction method and system that integrates physical modeling, thereby improving data utilization and system robustness, and having significant application value in fields such as simulation data generation, target detection, and sensor calibration.

[0006] To achieve the above objectives, the present invention provides the following solution: A point cloud intensity prediction method integrating physical modeling includes the following steps: S1. Obtain point cloud data and its corresponding geometric features and semantic information through sensors. The geometric features include at least the three-dimensional coordinates and normal vector of each point, and the semantic information is the semantic label of each point. S2. Convert the point cloud data into a two-dimensional multi-channel feature map, construct a dataset, and divide the dataset into a training set, a validation set, and a test set; S3. Establish a neural network model that integrates physical modeling, including a data-driven branch, a physical modeling branch, a physical calculation layer, and a fusion correction module; S4. Input the training set, validation set, and test set into the neural network model to complete the training and testing of the model; during the model training process, construct a joint loss function, calculate the loss based on the predicted intensity value output by the data-driven branch and the physical predicted intensity value output by the fusion correction module, and optimize the neural network model based on the loss; S5. Input the point cloud data to be predicted into the trained neural network model to obtain the final predicted point cloud intensity.

[0007] Preferably, in S1, the point cloud data is converted into a two-dimensional multi-channel feature map, specifically including: Perform a spherical coordinate transformation: (x,y,z) is transformed into (r,θ,Φ); where r = x 2 +y 2 +z 2 θ = arctan(y / x), Φ = arcsin(z / r), where r is the distance, θ is the azimuth angle, and Φ is the pitch angle; then, the image coordinates (r, θ, Φ) are generated and converted to (u, v); where u = 0.5 × (θ / π + 1) × W, u is the horizontal pixel coordinate, corresponding to the sensor's horizontal rotation angle, and W is the width; v = (Φ - Φ min ) / (Φ max -Φ min )×H, where v is the vertical pixel coordinate corresponding to the sensor processing channel, H is the height, and Φ is the depth. min For the minimum pitch angle, Φ max This is the maximum pitch angle.

[0008] Preferably, in S3, the neural network model is specifically a U2P-Net model, including: The data-driven branch is used to directly regress and predict intensity values ​​based on input features; The physical modeling branch is used to explicitly predict reflectance values ​​that are independent of observation conditions; The physical computation layer receives the reflectivity values ​​predicted by the physical modeling branch, as well as the incident angle and distance information from the input features, and calculates the physically derived intensity values ​​based on the radar equations. The fusion correction module is used to receive the physical derivation intensity value, correct the physical terms, and output the physical correction intensity value.

[0009] Preferably, in the physical calculation layer, the physically derived intensity value is calculated based on the radar equations, as shown in the following formula:

[0010] in, For physical derivation of strength, For reflectivity, For reflected power, For the receiving aperture area, Angle of incidence The distance is specified; the fusion correction module is specifically an MLP, with a three-layer fully connected structure.

[0011] Preferably, in S3, the training process of the neural network model specifically includes: The system receives the input distance r, reflection intensity I, semantic label, and the cosine value of the incident angle. It first predicts the intensity I using the training output of the first half of the UNET network. pre and predicted reflectivity ρ pre Then ρ pre The physical strength I is obtained through calculation using physical formulas. phys ; will I phys The input is then processed by another part of the model, the MLP, to correct the physics term, resulting in the predicted physical intensity I. physpre Finally, based on I pre and I physpre The loss is calculated using the formula for the joint loss function, the gradient is backpropagated, and the parameters of the data-driven branch, the physical modeling branch, and the fusion correction module are optimized.

[0012] Preferably, the formula for the joint loss function is as follows:

[0013] in, For data-driven loss, For physical correction loss, This is a hyperparameter.

[0014] This invention also provides a point cloud intensity prediction system based on physical modeling, for performing any of the above-mentioned point cloud intensity prediction methods based on physical modeling, comprising: The data acquisition module is used to acquire point cloud data and its corresponding geometric features and semantic information through sensors; The data conversion module is used to convert point cloud data into two-dimensional multi-channel feature maps, construct a dataset, and divide the dataset into training set, validation set, and test set. The model building module is used to build a neural network model that integrates physical modeling, including a data-driven branch, a physical modeling branch, a physical calculation layer, and a fusion correction module; The model training module is used to input the training set, validation set, and test set into the neural network model to complete the training and testing of the model. During the model training process, a joint loss function is constructed, and the loss is calculated based on the predicted intensity value output by the data-driven branch and the physical predicted intensity value output by the fusion correction module. The neural network model is then optimized based on the loss. The point cloud intensity prediction module is used to input the point cloud data to be predicted into a trained neural network model to obtain the final predicted point cloud intensity.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements a point cloud intensity prediction method that integrates physical modeling as described above.

[0016] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: This invention effectively solves the core problems of poor physical consistency and lack of interpretability in traditional point cloud intensity prediction methods by embedding a differentiable physical model into a neural network architecture and employing a dual-branch collaborative training mechanism. First, it significantly improves prediction accuracy and generalization ability, ensuring that intensity values ​​strictly follow optical reflection laws at different distances and angles, thus significantly enhancing reliability in complex scenarios such as autonomous driving. Second, by explicitly outputting physically meaningful reflectivity parameters and establishing an interpretable mapping with semantic information, it greatly improves the model's transparency, providing directly usable physical parameters for downstream tasks such as material identification and quality inspection. Furthermore, this method can rationally complete missing intensity data according to physical laws based on geometric and semantic information, improving data utilization and system robustness, and has significant application value in fields such as simulation data generation and sensor calibration. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the neural network model that integrates physical modeling in this invention; Figure 2 This is a schematic diagram of the training process of the neural network model of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] This invention provides a point cloud intensity prediction method that integrates physical modeling, comprising the following steps: S1. Obtain point cloud data and its corresponding geometric features and semantic information through sensors. The geometric features include at least the three-dimensional coordinates and normal vector of each point, and the semantic information is the semantic label of each point. S2. Convert the point cloud data into a two-dimensional multi-channel feature map, construct a dataset, and divide the dataset into a training set, a validation set, and a test set; S3. Establish a neural network model that integrates physical modeling, including a data-driven branch, a physical modeling branch, a physical calculation layer, and a fusion correction module; S4. Input the training set, validation set, and test set into the neural network model to complete the training and testing of the model; during the model training process, construct a joint loss function, calculate the loss based on the predicted intensity value output by the data-driven branch and the physical predicted intensity value output by the fusion correction module, and optimize the neural network model based on the loss; S5. Input the point cloud data to be predicted into the trained neural network model to obtain the final predicted point cloud intensity.

[0022] Furthermore, in S3, the neural network model is specifically the U2P-Net model, which includes: The data-driven branch is used to directly regress and predict intensity values ​​based on input features; The physical modeling branch is used to explicitly predict reflectance values ​​that are independent of observation conditions; The physical computation layer receives the reflectivity values ​​predicted by the physical modeling branch, as well as the incident angle and distance information from the input features, and calculates the physically derived intensity values ​​based on the radar equations. The fusion correction module, specifically an MLP, is designed as a three-layer fully connected structure. It is used to receive the physical derivation strength value, perform physical term correction, and output the physical correction strength value.

[0023] The method of this invention specifically includes: 1. Data processing stage: The goal of data processing is to transform existing datasets into multi-channel feature maps that can be used as input to the model. Here, we use the SemanticKITTI and VoxelScape datasets, with the latter having a structure largely consistent with the former. Subsequent explanations will only use the SemanticKITTI dataset as an example.

[0024] The SemanticKITTI dataset, created by the Computer Vision Group at the University of Trier, Germany, is an extension and improvement upon the KITTI dataset. It focuses on providing dense semantic annotations for LiDAR point clouds, aiming to advance research in LiDAR-based semantic segmentation and object detection. With rich semantic annotations and containing point cloud data from over 43,000 scans, it is an important resource for the field of autonomous driving.

[0025] According to the official documentation, the steps to convert the SemanticKITTI dataset into a multi-channel feature map are as follows: Loading point clouds and labels: The SemanticKITTI dataset typically includes .bin and .label files. First, a frame of point cloud is loaded from the .bin file, which is an N×4 array, where N is the number of points and the four channels represent the 3D coordinates (x, y, z) and laser reflection intensity of the corresponding point; at the same time, the original 32-bit label array is loaded from the corresponding .label file.

[0026] Extracting semantic labels: According to the format definition of the SemanticKITTI dataset, the lower 16 bits of the original 32-bit label store the semantic label ID. The corresponding 16-bit ID is extracted by bitwise AND operation.

[0027] Spherical Projection: Since SemanticKITTI uses a Velodyne HDL-64E sensor, the projected 2D image size should be 64×1024 or 64×2048. Considering model training efficiency, a resolution of 64×1024 (height H=64, width W=1024) is used. The specific projection process for each point is as follows: (1) Perform spherical coordinate transformation: (x,y,z) is transformed into (r,θ,Φ); where r = x 2 +y 2 +z 2 θ = arctan(y / x), Φ = arcsin(z / r), where r is the distance, θ is the azimuth angle, and Φ is the pitch angle; then, the image coordinates (r, θ, Φ) are generated and converted to (u, v); where u = 0.5 × (θ / π + 1) × W, u is the horizontal pixel coordinate, corresponding to the sensor's horizontal rotation angle, and W is the width; v = (Φ - Φ min ) / (Φ max -Φ min)×H, where v is the vertical pixel coordinate corresponding to the sensor processing channel, H is the height, and Φ is the depth. min For the minimum pitch angle, Φ max This is the maximum pitch angle.

[0028] This yields the coordinates of the points in the dataset within the 2D image.

[0029] (2) Calculation of incident angle: According to the formula cosα= We obtain cosα. Since we will use the value of cosα later, we will directly save the value of cosα here (which also saves us the normalization step).

[0030] (3) Numerical normalization: The distance feature is normalized using Min-Max, while data beyond 80m is truncated (this part of the data is very small and the accuracy is significantly reduced); the intensity feature is linearly scaled; the cos value of the incident angle has been limited to 0-1; the label feature retains the original SemanticLabel.

[0031] (4) Data set partitioning: 70% of the dataset is used as the training set, 15% as the validation set, and 15% as the test set. The partitioning is done by random sampling.

[0032] After these steps are completed, the original point cloud data will be transformed into a two-dimensional tensor with fixed shape, fixed channels, and aligned labels, which can be directly applied to network training.

[0033] 2. Model Design: Based on the model's structural characteristics, it is named "U2P-Net" (UNET-Physics-Net). The model's training process (e.g.) Figure 2 (As shown): The system receives input data such as distance r, reflection intensity I, semantic label, and the cosine value of the incident angle. It first uses the training output of the first half of the UNET network to predict the intensity I. pre and predicted reflectivity ρ pre Then ρ pre The physical strength I is obtained through calculation using physical formulas. phys ; will I phys The input is then processed by another part of the model, the MLP, to correct the physics term, resulting in the predicted physical intensity I. physpre Finally, use I pre and I physpre Calculate the loss and backpropagate the gradient.

[0034] Below is a detailed description of the model (e.g.) Figure 1 (as shown) (1) UNET input channel: The input channel includes distance R, incident angle θ and reflection intensity I, etc., with a size of 64×1024. These data have been normalized and can be directly sent to the first layer of the network.

[0035] (2) UNET encoding path: four levels of downsampling, with two 3×3 convolutions (BN+ReLU) followed by 2×2 max pooling at each level. The channel progression is 64→128→256→512, and the spatial resolution is halved at each level to extract multi-scale context.

[0036] (3) UNET decoding path: Four-level upsampling, with each level using a 2×2 transposed convolution to restore the spatial size. The encoded features at the same level are concatenated via skip connections and then fused through two 3×3 convolutions (BN+ReLU). The number of channels decreases from 512 to 256 to 128 to 64, maintaining symmetry.

[0037] (4) UNET output header: A 1×1 convolution projects the 64-channel features into 2 channels, with dimensions of 64×1024×2. Channel 0 is the predicted intensity I. pre Channel 1 is the predicted reflectance ρ pre The two channels have independent parameters and only share front-end features.

[0038] (5) Physical calculation: In the forward propagation of UNET, obtain the incident angle cosα value from the input, and use formula I phys =ρ pre × cosα×R -2 Calculate I phys The dimensions here remain 64×1024×1.

[0039] (6) MLP Input: Input I phys The image is unfolded pixel by pixel into a one-dimensional scalar, forming the only input channel of the MLP, with dimensions of 64×1024×1 → N×1.

[0040] (7) MLP structure design: The structure design of MLP is a three-layer fully connected (1→32, 32→16, 16→1). Three layers are sufficient to describe the possible parameters not indicated in the formula (such as transmitter and receiver aperture, atmospheric attenuation coefficient, etc., and the three-layer fully connected can describe about 600 parameters). At the same time, it also reduces the risk of overfitting caused by multiple layers.

[0041] (8) MLP output: Change the output size from N×1 back to 64×1024×1.

[0042] Loss constraint design: Loss constraints are used to uniformly guide the collaborative optimization of the two-branch network. The model contains two parallel strength output branches: the data-driven branch outputting I from UNET direct regression. preAnd the physical-data fusion branch output I calculated by physical formulas and corrected by MLP physpre Both must be aligned with the actual sensor intensity reading I. A weighted double-loss mechanism is employed: total loss... L total Data-driven loss L data and physical correction loss L phys The weighted summation constitutes, i.e. .

[0043] Hyperparameters The range [0,1] is used to dynamically balance the contributions of the two branches in the early, middle, and late stages of training, and can be adjusted according to the training effect. The loss of each branch is calculated using masked L2 loss (mask mean square error) to ignore invalid or unlabeled pixels, ensuring that the gradient comes only from valid training samples. During backpropagation, L data The gradient is only updated in UNET with respect to the predicted intensity I. pre Directly related parameters; and L phys The gradient will simultaneously update the reflectivity prediction branch of UNET (affecting ρ). pre This, along with the subsequent MLP correction network, enables end-to-end propagation of physical knowledge from formulas to data.

[0044] This invention also provides a point cloud intensity prediction system based on physical modeling, for performing any of the above-mentioned point cloud intensity prediction methods based on physical modeling, comprising: The data acquisition module is used to acquire point cloud data and its corresponding geometric features and semantic information through sensors; The data conversion module is used to convert point cloud data into two-dimensional multi-channel feature maps, construct a dataset, and divide the dataset into training set, validation set, and test set. The model building module is used to build a neural network model that integrates physical modeling, including a data-driven branch, a physical modeling branch, a physical calculation layer, and a fusion correction module; The model training module is used to input the training set, validation set, and test set into the neural network model to complete the training and testing of the model. During the model training process, a joint loss function is constructed, and the loss is calculated based on the predicted intensity value output by the data-driven branch and the physical predicted intensity value output by the fusion correction module. The neural network model is then optimized based on the loss. The point cloud intensity prediction module is used to input the point cloud data to be predicted into a trained neural network model to obtain the final predicted point cloud intensity.

[0045] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements a point cloud intensity prediction method that integrates physical modeling as described above.

[0046] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0047] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A point cloud intensity prediction method integrating physical modeling, characterized in that, Includes the following steps: S1. Acquire point cloud data and its corresponding geometric features and semantic information through sensors. The geometric features include at least the three-dimensional coordinates and normal vector of each point, and the semantic information is the semantic label of each point. S2. Convert the point cloud data into a two-dimensional multi-channel feature map, construct a dataset, and divide the dataset into a training set, a validation set, and a test set; S3. Establish a neural network model that integrates physical modeling, including a data-driven branch, a physical modeling branch, a physical calculation layer, and a fusion correction module; S4. Input the training set, validation set, and test set into the neural network model to complete the training and testing of the model; during the model training process, construct a joint loss function, calculate the loss based on the predicted intensity value output by the data-driven branch and the physical predicted intensity value output by the fusion correction module, and optimize the neural network model based on the loss; S5. Input the point cloud data to be predicted into the trained neural network model to obtain the final predicted point cloud intensity.

2. The point cloud intensity prediction method based on physical modeling according to claim 1, characterized in that, In step S1, converting the point cloud data into a two-dimensional multi-channel feature map specifically includes: Perform a spherical coordinate transformation: (x,y,z) is transformed into (r,θ,Φ); where r = x 2 +y 2 +z 2 θ = arctan(y / x), Φ = arcsin(z / r), where r is the distance, θ is the azimuth angle, and Φ is the pitch angle; then, the image coordinates (r, θ, Φ) are generated and converted to (u, v); where u = 0.5 × (θ / π + 1) × W, u is the horizontal pixel coordinate, corresponding to the sensor's horizontal rotation angle, and W is the width; v = (Φ - Φ min ) / (Φ max -Φ min )×H, where v is the vertical pixel coordinate corresponding to the sensor processing channel, H is the height, and Φ is the depth. min For the minimum pitch angle, Φ max This is the maximum pitch angle.

3. The point cloud intensity prediction method based on physical modeling according to claim 1, characterized in that, In S3, the neural network model is specifically a U2P-Net model, including: The data-driven branch is used to directly regress and predict intensity values ​​based on input features; The physical modeling branch is used to explicitly predict reflectance values ​​that are independent of observation conditions; The physical computation layer receives the reflectivity values ​​predicted by the physical modeling branch, as well as the incident angle and distance information from the input features, and calculates the physically derived intensity values ​​based on the radar equations. The fusion correction module is used to receive the physical derivation intensity value, correct the physical terms, and output the physical correction intensity value.

4. The point cloud intensity prediction method based on physical modeling according to claim 3, characterized in that, In the physical calculation layer, the physically derived intensity value is calculated based on the radar equations, as shown in the following formula: in, For physical derivation of strength, For reflectivity, For reflected power, For the receiving aperture area, Angle of incidence The distance is specified; the fusion correction module is specifically an MLP, with a three-layer fully connected structure.

5. The point cloud intensity prediction method based on physical modeling according to claim 1, characterized in that, In S3, the training process of the neural network model specifically includes: The system receives the input distance r, reflection intensity I, semantic label, and the cosine value of the incident angle. It first predicts the intensity I using the training output of the first half of the UNET network. pre and predicted reflectivity ρ pre Then ρ pre The physical strength I is obtained through calculation using physical formulas. phys ; will I phys The input is then processed by another part of the model, the MLP, to correct the physics term, resulting in the predicted physical intensity I. physpre Finally, based on I pre and I physpre The loss is calculated using the formula for the joint loss function, the gradient is backpropagated, and the parameters of the data-driven branch, the physical modeling branch, and the fusion correction module are optimized.

6. The point cloud intensity prediction method based on physical modeling according to claim 5, characterized in that, The formula for the joint loss function is as follows: in, For data-driven loss, For physical correction loss, This is a hyperparameter.

7. A point cloud intensity prediction system integrating physical modeling, used to execute the point cloud intensity prediction method integrating physical modeling as described in any one of claims 1-6, characterized in that, include: The data acquisition module is used to acquire point cloud data and its corresponding geometric features and semantic information through sensors; The data conversion module is used to convert the point cloud data into a two-dimensional multi-channel feature map, construct a dataset, and divide the dataset into a training set, a validation set, and a test set. The model building module is used to build a neural network model that integrates physical modeling, including a data-driven branch, a physical modeling branch, a physical calculation layer, and a fusion correction module; The model training module is used to input the training set, validation set, and test set into the neural network model to complete the training and testing of the model. During the model training process, a joint loss function is constructed, and the loss is calculated based on the predicted intensity value output by the data-driven branch and the physical predicted intensity value output by the fusion correction module. The neural network model is then optimized based on the loss. The point cloud intensity prediction module is used to input the point cloud data to be predicted into a trained neural network model to obtain the final predicted point cloud intensity.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a point cloud intensity prediction method that integrates physical modeling as described in claims 1-6.

Images