Point cloud generation method, electronic equipment and computer readable storage medium

Abstract

The invention discloses a point cloud generation method, electronic equipment and a computer readable storage medium. The point cloud generation method comprises the following steps: performing Gaussian initialization on an initial point cloud to obtain first point cloud distribution; performing parameter modeling on the first point cloud distribution to at least obtain an intensity parameter and a light loss probability; performing Gaussian flattening processing on the first point cloud distribution to obtain a Gaussian plane; calculating the distance from the radar to the Gaussian plane; inputting the intensity parameter, the light loss probability and the distance into a range graph to obtain range graph representation; the range graph representation is converted into a target point cloud, and the target point cloud and the initial point cloud are point clouds of different view angles. Through the method, the point cloud rendering effect and precision can be improved, and the multi-view point cloud with higher geometric details can be generated.

Description

Technical Field

[0001] The present application relates to the field of point cloud processing technology, and in particular to a point cloud generation method, an electronic device, and a computer-readable storage medium. Background Art

[0002] A new perspective synthesis method for radar point clouds is a highly flexible simulation technology that can dynamically adjust radar sensor parameters, such as resolution, frequency, and field of view, in a virtual environment to accurately simulate the characteristics of different sensor types. Furthermore, this method supports modifying the sensor's position and rotation angle, enabling the generation of radar data from multiple perspectives and locations. This technology not only helps evaluate sensor performance in different configurations but also generates diverse training data for autonomous driving algorithms in complex scenarios, thereby enhancing their robustness and adaptability in real-world environments.

[0003] However, there are many defects in related technologies, such as high labeling cost, slow rendering speed of neural radiation field method and limited applicability. Summary of the Invention

[0004] The point cloud generation method, electronic device, and computer-readable storage medium provided in this application can improve the point cloud rendering effect and accuracy, and generate multi-view point clouds with higher geometric details.

[0005] To solve the above technical problems, a technical solution adopted in the present application is: to provide a point cloud generation method, which includes: performing Gaussian initialization on the initial point cloud to obtain a first point cloud distribution; performing parameter modeling on the first point cloud distribution to obtain at least an intensity parameter and a light loss probability; and performing Gaussian flattening on the first point cloud distribution to obtain a Gaussian plane; calculating the distance from the radar to the Gaussian plane; inputting the intensity parameter, the light loss probability, and the distance into a range graph to obtain a range graph representation; and converting the range graph representation into a target point cloud, wherein the target point cloud and the initial point cloud are point clouds from different perspectives.

[0006] The initial point cloud is Gaussian initialized to obtain a first point cloud distribution, including: performing Gaussian initialization on the covariance matrix of the Gaussian distribution and the spatial coordinates of each point in the initial point cloud to obtain the first point cloud distribution.

[0007] The step of performing Gaussian flattening on the first point cloud distribution to obtain a Gaussian plane includes: performing Gaussian flattening on the first point cloud distribution using a minimum scale factor to obtain a Gaussian plane.

[0008] Particularly, parameter modeling is performed on the first point cloud distribution to obtain at least an intensity parameter and a light loss probability, including: obtaining a first direction vector, a first distance, and a feature representation; wherein the first direction vector represents a direction vector of light from the radar sensor position to the Gaussian basis element, and the first distance represents a distance from the radar sensor position to the Gaussian basis element; and the first direction vector, the first distance, and the feature representation are input into a multilayer perceptron to obtain the intensity parameter and the light loss probability.

[0009] Calculating the distance from the radar to the Gaussian plane includes: obtaining the radar's ray equation and the plane equation corresponding to the Gaussian plane; solving the ray equation and the plane equation to obtain the value of the distance parameter in the ray equation; determining the intersection position between the ray and the Gaussian plane based on the value; and calculating the distance from the radar to the Gaussian plane based on the intersection position and the center position of the radar.

[0010] The range graph is trained in the following manner: the training point cloud is input into the range graph to obtain a training range graph representation; the training point cloud is annotated with true intensity values, true depth values, true sparse mask values, and rendered normal vector values, and the information represented by the training range graph includes predicted intensity values, predicted depth values, and predicted sparse mask values; an intensity loss value is calculated based on the true intensity value and the predicted intensity value; a first depth loss value and a second depth loss value are calculated based on the true depth value and the predicted depth value; a sparse mask loss value is calculated based on the true sparse mask value and the predicted sparse mask value; a normal-depth loss value is calculated based on the rendered normal vector value and the predicted normal vector value; the predicted normal vector value is converted from the predicted depth value; and the model parameters of the range graph are adjusted based on the intensity loss value, the first depth loss value, the second depth loss value, the sparse mask loss value, and the normal-depth loss value.

[0011] The first depth loss value and the second depth loss value are calculated according to the actual depth value and the predicted depth value, including: calculating the first depth loss value using the following formula: Where N represents the number of depth values, d i represents the true depth value of the i-th pixel, represents the predicted depth value of the i-th pixel; the second depth loss value is calculated using the following formula:

[0012] The normal-depth loss value is calculated according to the rendered normal vector value and the predicted normal vector value, including: the normal-depth loss value is calculated using the following formula: Among them, N represents the number of normal vector values, n i Represents the rendering normal vector value of the i-th pixel, Represents the predicted normal vector value of the i-th pixel, w iRepresents the weight that is dynamically adjusted based on deep reliability.

[0013] To solve the above technical problems, another technical solution adopted in this application is: to provide an electronic device, which includes a processor and a memory coupled to the processor; wherein the memory is used to store computer programs, and the processor is used to execute the computer programs to implement the method provided by any of the above technical solutions.

[0014] To solve the above technical problems, another technical solution adopted in this application is: providing a computer-readable storage medium, which is used to store a computer program. When the computer program is executed by a processor, it is used to implement the method provided by any of the above technical solutions.

[0015] The beneficial effects of the embodiments of the present application are as follows: Different from the prior art, the point cloud generation method, electronic device, and computer-readable storage medium provided by the present application first perform Gaussian initialization on the initial point cloud to provide a basis for subsequent modeling. On this basis, parameter modeling is performed on the first point cloud distribution to obtain at least intensity parameters and light loss probability to accurately simulate the response characteristics of a real radar. In order to enhance the structuring effect, the first point cloud distribution is Gaussian flattened to obtain a Gaussian plane. The Gaussian distribution is flattened, so that the subsequently generated target point cloud has higher geometric accuracy and realism, which can improve the overall point cloud rendering effect and accuracy, and generate a multi-view point cloud with higher geometric details. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without inventive efforts. Among them:

[0017] Figure 1 This is a flow chart of an embodiment of a point cloud generation method provided by this application;

[0018] Figure 2 yes Figure 1 A flow chart of an embodiment of step 12;

[0019] Figure 3 yes Figure 1 A flow chart of an embodiment of step 14;

[0020] Figure 4 This is a flow chart of an embodiment of a range graph training method provided by the present application;

[0021] Figure 5This is a structural diagram of an embodiment of an electronic device provided by the present application;

[0022] Figure 6 It is a structural diagram of an embodiment of a computer-readable storage medium provided by this application. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It will be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some, rather than all, structures related to the present application are shown in the drawings. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0024] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0025] A new perspective synthesis method for radar point clouds is a highly flexible simulation technology that can dynamically adjust radar sensor parameters, such as resolution, frequency, and field of view, in a virtual environment to accurately simulate the characteristics of different sensor types. Furthermore, this method supports modifying the sensor's position and rotation angle, enabling the generation of radar data from multiple perspectives and locations. This technology not only helps evaluate sensor performance in different configurations but also generates diverse training data for autonomous driving algorithms in complex scenarios, thereby enhancing their robustness and adaptability in real-world environments.

[0026] Deep learning generation method: A large amount of real radar scan data is collected and annotated, and radar point cloud features are learned through a deep learning network to synthesize radar data from new perspectives. When the generalization performance is high enough, the network can recognize a variety of scenes and realize radar simulation from different perspectives.

[0027] Physics-based simulation: Traditional methods use physics engines and hand-crafted 3D scenes to simulate radar signals through ray casting, generating radar point cloud data in a virtual environment. However, this method typically relies on a large number of 3D model resources, and the generated point cloud data is significantly different from the real data.

[0028] Multi-sensor fusion simulation: Combining data from sensors such as lidar, cameras, and ultrasonic sensors, multi-sensor fusion simulation is implemented in a virtual environment to improve the accuracy of synthesizing new perspectives. This approach can effectively compensate for the shortcomings of a single sensor and make the generated radar data more realistic.

[0029] Synthesis methods based on neural radiance fields: In recent years, techniques such as neural radiance fields (NeRF) have been used to generate radar data from new perspectives. These methods learn implicit scene representations to generate radar point clouds from diverse perspectives. These methods perform well in small-scale scenes but have limitations in large-scale, complex scenes.

[0030] However, deep learning methods suffer from high annotation costs: Deep learning-based synthesis methods require large amounts of labeled radar data, which is costly and complex, making them difficult to apply in a wide range of scenarios. Furthermore, the required labeled data often cannot be directly reused from existing data in other driver assistance systems, increasing data acquisition costs.

[0031] Physics engine simulation relies on 3D resources and suffers from significant realism gaps: Physics engine-based simulation methods rely on high-quality 3D scene resources, but these resources are expensive and difficult to build. The generated data differs significantly from real data, affecting the realism and applicability of new perspective synthesis.

[0032] The technical cost and complexity of multi-sensor fusion are high: Multi-sensor fusion simulation generates high accuracy at different perspectives, but requires high hardware resources. At the same time, it also requires high accuracy in sensor calibration and synchronization, which increases the complexity of the system.

[0033] The neural radiation field method has slow rendering speed and limited applicability: Although the synthesis method based on the neural radiation field performs well in small-scale static scenes, it has a slow rendering speed in large-scale complex scenes and dynamic environments and cannot meet real-time requirements.

[0034] Based on this, the present application performs Gaussian initialization through the initial point cloud to provide a basis for subsequent modeling. On this basis, the first point cloud distribution is parameter modeled to obtain at least the intensity parameter and the probability of light loss to accurately simulate the response characteristics of the real radar. In order to enhance the structured effect, the first point cloud distribution is Gaussian flattened to obtain a Gaussian plane. The Gaussian distribution is flattened so that the subsequently generated target point cloud has higher geometric accuracy and realism, which can improve the point cloud rendering effect and accuracy as a whole and generate a multi-view point cloud with higher geometric details. Please refer to any of the following embodiments for details.

[0035] See Figure 1 , Figure 1 : This is a flow chart of an embodiment of a point cloud generation method provided by this application. The method includes:

[0036] Step 11: Perform Gaussian initialization on the initial point cloud to obtain the first point cloud distribution.

[0037] In some embodiments, the initial point cloud may be collected by a point cloud collection device such as a radar, wherein the initial point cloud may be collected in real time or in advance.

[0038] In some embodiments, Gaussian initialization may be performed in combination with the covariance matrix of the Gaussian distribution and the spatial coordinates of each point in the initial point cloud to obtain a first point cloud distribution.

[0039] In some embodiments, Gaussian initialization can be performed in the following manner:

[0040] Initialize the input radar point cloud with a Gaussian distribution, treat each point in the point cloud as a Gaussian distribution, and preliminarily construct the Gaussian structure of the 3D point cloud, laying the foundation for subsequent feature modeling and adjustment. Specifically, based on the input 3D point cloud data, the spatial coordinate μ of each point is regarded as the mean of the 3D Gaussian distribution, and the formula is as follows:

[0041]

[0042] Where x is an arbitrary position in three-dimensional space, and ∑ is the covariance matrix of the Gaussian distribution, which is used to describe the directionality and shape of the distribution. To ensure the positive definiteness of the covariance matrix, its structure is defined as follows:

[0043] ∑=RSS T R T .

[0044] Where S is the scaling matrix that controls the scale of the Gaussian distribution, and R is the rotation matrix that adjusts the directionality of the distribution. This 3D Gaussian structure inherits the differential properties of voxel representation while supporting efficient tile-based rasterization.

[0045] Step 12: Perform parameter modeling on the first point cloud distribution to obtain at least intensity parameters and light loss probability.

[0046] In some embodiments, the intensity (reflectance value) and sparse distribution of each point in the point cloud are modeled based on the radar's reflection characteristics. This modeling can more accurately reproduce the radar's sparse distribution characteristics and intensity variations, laying the foundation for generating simulation data that conforms to actual physical properties.

[0047] The specific process is as follows. First, according to the Gaussian initialization described above, the scene is represented as a set of Gaussian primitives, where each Gaussian primitive is modeled by the following parameters:

[0048] Position parameter: μ∈R 3, represents the three-dimensional space position of the Gaussian basis element.

[0049] Rotation matrix: R∈R (3×3) , which is used to describe the orientation of the Gaussian basis element.

[0050] Scaling parameter: S∈R 3 , corresponding to the scale component of the Gaussian covariance matrix.

[0051] Opacity: α∈R, which indicates the degree to which light penetrates the Gaussian primitive.

[0052] Intensity parameter: λ∈R, simulating the LiDAR reflection intensity.

[0053] Light loss probability (radar sparsity characteristic): ρ∈R, which simulates the probability of light attenuation due to material, angle or distance, that is, modeling the sparse characteristics of radar.

[0054] In some embodiments, see Figure 2 , step 12 can be the following process:

[0055] Step 121: Obtain a first direction vector, a first distance, and a feature representation; wherein the first direction vector represents the direction vector of the light from the radar sensor position to the Gaussian basis element, and the first distance represents the distance from the radar sensor position to the Gaussian basis element.

[0056] Step 122: Input the first direction vector, the first distance, and the feature representation into a multilayer perceptron to obtain an intensity parameter and a light loss probability.

[0057] In one application scenario, the intensity parameter λ and the light loss probability ρ are generated by a neural network model. By combining multiple factors such as material, angle, and distance, the radar response characteristics in real physical scenes are accurately modeled:

[0058] λ k =F λ (f k ,r k ,d k ).

[0059] ρ k =F ρ (f k ,r k ,d k ).

[0060] Among them, f k is a learnable feature representation that describes the material properties of the object, r k represents the direction vector (first direction vector) of the light from the radar sensor position to the Gaussian basis element, d k Indicates the distance from the radar sensor position to the Gaussian primitive (first distance), Fλ and F ρ A learnable multi-layer perceptron (MLP) maps light direction, distance, and feature embeddings into intensity and light loss probability, respectively. The intensity parameter ranges from [0, 255], and the probability of light loss is [0, 1], representing a range from no light loss to complete light loss.

[0061] Step 13: Perform Gaussian flattening on the first point cloud distribution to obtain a Gaussian plane.

[0062] In some embodiments, Gaussian flattening is performed on the first point cloud distribution using a minimum scale factor to obtain a Gaussian plane.

[0063] In some embodiments, to improve the structural accuracy of the generated point cloud, the initialized Gaussian distribution (first point cloud distribution) is flattened. Specifically, based on the covariance matrix of the Gaussian distribution, it is compressed using a minimum scale factor, causing the three-dimensional Gaussian ellipsoid to degenerate into a plane perpendicular to the direction of the minimum scale factor. The equation of this plane can be accurately calculated using the direction matrix and scale factor matrix of the Gaussian ellipsoid. Through this processing method, the surface structure of the point cloud is more consistent, effectively reducing the deviation caused by changes in the three-dimensional structure, and ensuring that the generated point cloud has richer geometric details. After the flattening process, these planes derived from the degeneration of the Gaussian sphere lay the foundation for subsequent refined ray distance rendering optimization. Specifically, the plane equation and the line equation of the radar ray can accurately calculate the intersection position and distance information of the ray and the plane. This method not only improves the accuracy of the point cloud surface structure, but also significantly improves the depth resolution of the data under the new radar perspective when simulating the distance distribution of radar detected objects. In practical applications, this refined ray distance calculation method can more realistically reflect the geometric morphology of objects detected by the radar, thereby optimizing the overall quality of point cloud generation.

[0064] Step 14: Calculate the distance from the radar to the Gaussian plane.

[0065] In some embodiments, see Figure 3 , step 14 can be the following process:

[0066] Step 141: Obtain the radar ray equation and the plane equation corresponding to the Gaussian plane.

[0067] Specifically, the radar ray equation is expressed as:

[0068] r(t)=T c +t·(K -1 p).

[0069] Among them, T cis the center position of the radar, K is the internal parameter matrix of the radar, p is the pixel coordinate in the direction of the ray, t is the distance parameter along the direction of the ray,

[0070] At the same time, the plane equation generated by Gaussian flattening can be expressed as:

[0071] n k ·(x-μ)=0.

[0072] Among them, μ is the Gaussian center point, n k is the normal vector to the plane, defined as the direction of the shortest axis of the Gaussian distribution.

[0073] Step 142: Solve the ray equation and the plane equation to obtain the value of the distance parameter in the ray equation.

[0074] The ray equation r(t)=Tc+t·(K -1 p) and the plane equation n k (x-μ)=0, we can solve for the ray distance parameter t:

[0075]

[0076] Step 143: Determine the intersection point between the outgoing ray and the Gaussian plane according to the numerical value.

[0077] In some embodiments, after t is calculated, it is substituted into the ray equation to obtain the end point of the ray, that is, the actual intersection position T of the real radar ray and the Gaussian basis plane. p .

[0078] Step 144: Calculate the distance from the radar to the Gaussian plane based on the intersection point position and the center position of the radar.

[0079] In some embodiments, the distance between the intersection point and the center position of the radar is calculated to obtain the refined distance from the radar emission ray to the Gaussian basis element.

[0080] L refine =∥T p -T c ∥.

[0081] Compared to traditional methods that only consider the distance from the Gaussian center point to the radar position, this method combines the Gaussian flattened plane equation with the ray equation, introduces the geometric properties of the Gaussian flattened plane and its normal vector, and accurately calculates the intersection and distance between the ray and the plane, achieving high-precision calculation of the true distance between the ray and the object surface, significantly improving the calculation accuracy of distance information. It can not only truly reflect the actual distance between the radar ray and the object surface, but also significantly enhance the depth resolution in multi-view point cloud rendering. In particular, in the field of high-precision radar imaging and three-dimensional point cloud generation in complex scenes, this method has demonstrated excellent applicability and performance improvement. Furthermore, it can more accurately simulate the distance distribution of objects detected by radar and significantly improve the depth accuracy of multi-view data.

[0082] Step 15: Input the intensity parameter, light loss probability, and distance into the range graph to obtain the range graph representation.

[0083] The range graph can be trained in advance using the corresponding training samples.

[0084] In some embodiments, the view angle corresponding to the target point cloud may also be input into the range graph, and the range graph may then generate a range graph representation of the corresponding view angle.

[0085] Step 16: Convert the range graph representation into a target point cloud, where the target point cloud and the initial point cloud are point clouds from different perspectives.

[0086] The generated range map representation is converted into a radar point cloud representation (target point cloud), resulting in the final new-perspective radar data. This process generates radar data with higher geometric accuracy and realism, enabling testing and verification of downstream autonomous driving modules.

[0087] In this embodiment, the initial point cloud is first Gaussian initialized to provide a foundation for subsequent modeling. Based on this, parametric modeling is performed on the first point cloud distribution, obtaining at least intensity parameters and light loss probability to accurately simulate the response characteristics of a real radar. To enhance the structural effect, the first point cloud distribution is Gaussian flattened to obtain a Gaussian plane. This flattening process ensures that the subsequently generated target point cloud has higher geometric accuracy and realism, improving the overall rendering quality and accuracy of the point cloud and generating a multi-view point cloud with higher geometric detail.

[0088] See Figure 4 , the range graph is trained in the following way:

[0089] Step 41: Input the training point cloud into the range graph to obtain the predicted range graph representation.

[0090] In some embodiments, the training point cloud is annotated with true intensity values, true depth values, true sparse mask values, and rendered normal vector values, and the information represented by the predicted range graph includes predicted intensity values, predicted depth values, and predicted sparse mask values.

[0091] In some embodiments, to convert 3D radar data into a range graph representation, given 3D coordinates (x, y, z), the azimuth angle β and elevation angle α of each point are first calculated:

[0092] β=π-arctan2(y,x);

[0093]

[0094] Among them, fov_down represents the downward viewing angle of the radar, Represents the angle between the point (x, y, z) and the horizontal plane.

[0095] Then β and α are mapped to the coordinate system of the range image,

[0096]

[0097] Where c is the horizontal pixel coordinate of the range image, r is the vertical pixel coordinate of the range image, H and W are the height and width of the range image respectively, and fov is the overall emission angle of the radar.

[0098] Therefore, the true intensity value, true depth value, true sparse mask value and rendering normal vector value can be obtained from the mapped range graph representation.

[0099] At the same time, the training point cloud is input into the range graph to obtain the predicted range graph representation of the range graph output. The predicted range graph representation information includes predicted intensity value, predicted depth value and predicted sparse mask value.

[0100] Step 42: Calculate the strength loss value based on the actual strength value and the predicted strength value.

[0101] In some embodiments, the strength loss value is calculated using the following formula:

[0102] Among them, t i is the true intensity value of the i-th pixel, represents the predicted intensity value of the i-th pixel.

[0103] Step 43: Calculate a first depth loss value and a second depth loss value according to the true depth value and the predicted depth value.

[0104] In some embodiments, the first depth loss value is calculated using the following formula:

[0105] Where N represents the number of depth values, d i represents the true depth value of the i-th pixel, Represents the predicted depth value of the i-th pixel.

[0106] In some embodiments, the second depth loss value is calculated using the following formula:

[0107]

[0108] To reduce the errors caused by the sparsity and noise of lidar data, the calculation process of the second depth loss value incorporates spatial structure priors, imposes constraints on the depth prediction results, and significantly improves the consistency of depth representation in key areas.

[0109] Step 44: Calculate the sparse mask loss value based on the true sparse mask value and the predicted sparse mask value.

[0110] In some embodiments, the sparse mask loss value is calculated using the following formula:

[0111]

[0112] Among them, r i represents the true sparse mask value of the i-th pixel, Represents the predicted sparse mask value.

[0113] Step 45: Calculate a normal-depth loss value based on the rendered normal vector value and the predicted normal vector value; wherein the predicted normal vector value is obtained by converting the predicted depth value.

[0114] In some embodiments, the normal-depth loss value is calculated using the following formula: Among them, N represents the number of normal vector values, n i Represents the rendering normal vector value of the i-th pixel, Represents the predicted normal vector value of the i-th pixel, w i Represents the weight that is dynamically adjusted based on deep reliability.

[0115] To ensure the consistency between the predicted normal vector and the depth value, the calculation of the normal-depth loss value can dynamically adjust the weight according to the reliability of the depth value to improve the prediction accuracy of the high confidence area.

[0116] Step 46: Adjust the model parameters of the range map according to the intensity loss value, the first depth loss value, the second depth loss value, the sparse mask loss value, and the normal-depth loss value.

[0117] In some embodiments, corresponding weights can be set for the intensity loss value, the first depth loss value, the second depth loss value, the sparse mask loss value, and the normal-depth loss value. These values are then weighted and summed to obtain a final loss value. The model parameters of the range graph are then adjusted based on the final loss value. Training ends when the range graph meets the requirements.

[0118] The joint optimization of these loss values not only significantly improves the accuracy of rendered depth and intensity maps, but also better preserves the sparse characteristics of lidar data and enhances the geometric fidelity and sparsity characteristics of synthesized new perspective data.

[0119] That is, in the training process of the range graph, in order to achieve higher quality range graph representation, a multi-head loss function supervision mechanism is introduced to constrain the range graph output from multiple dimensions to make it more consistent with the distribution characteristics of actual radar sensors, thereby significantly improving the quality and accuracy of synthetic data. Specifically, a variety of loss functions are designed, including random depth structure loss, adaptive normal-depth consistency loss, and multi-view loss. Figure 1 These loss functions work together during the training process to effectively enhance the geometric consistency and robustness of the model.

[0120] See Figure 5 , Figure 5 1 is a schematic diagram of the structure of an embodiment of an electronic device provided in this application. The electronic device 50 includes a processor 51 and a memory 52 coupled to the processor 51; wherein the memory 52 is used to store a computer program, and the processor 51 is used to execute the computer program to implement the following method:

[0121] The initial point cloud is Gaussian initialized to obtain a first point cloud distribution; the first point cloud distribution is parameterized to obtain at least an intensity parameter and a light loss probability; and the first point cloud distribution is Gaussian flattened to obtain a Gaussian plane; the distance from the radar to the Gaussian plane is calculated; the intensity parameter, the light loss probability, and the distance are input into a range graph to obtain a range graph representation; and the range graph representation is converted into a target point cloud, wherein the target point cloud and the initial point cloud are point clouds from different perspectives.

[0122] It can be understood that the processor 51 is also used to execute computer programs to implement the method of any of the above embodiments, which will not be described in detail here.

[0123] In some embodiments, the electronic device 50 may be a device with computing capabilities, such as a mobile terminal, a personal computer, a vehicle-mounted terminal, or a vehicle.

[0124] See Figure 6 , Figure 66 is a schematic diagram of the structure of an embodiment of a computer-readable storage medium provided by the present application. The computer-readable storage medium 60 is used to store a computer program 61. When the computer program 61 is executed by a processor, it is used to implement the following method:

[0125] The initial point cloud is Gaussian initialized to obtain a first point cloud distribution; the first point cloud distribution is parameterized to obtain at least an intensity parameter and a light loss probability; and the first point cloud distribution is Gaussian flattened to obtain a Gaussian plane; the distance from the radar to the Gaussian plane is calculated; the intensity parameter, the light loss probability, and the distance are input into a range graph to obtain a range graph representation; and the range graph representation is converted into a target point cloud, wherein the target point cloud and the initial point cloud are point clouds from different perspectives.

[0126] It can be understood that when the computer program 61 is executed by the processor, it is also used to implement the method of any of the above embodiments, which will not be described in detail here.

[0127] In summary, the point cloud generation method, electronic device, and computer-readable storage medium provided by this application first perform Gaussian initialization on the initial point cloud to provide a basis for subsequent modeling. On this basis, parameter modeling is performed on the first point cloud distribution to obtain at least intensity parameters and light loss probability to accurately simulate the response characteristics of a real radar. In order to enhance the structured effect, the first point cloud distribution is Gaussian flattened to obtain a Gaussian plane. The Gaussian distribution is flattened, so that the subsequently generated target point cloud has higher geometric accuracy and realism, which can improve the overall point cloud rendering effect and accuracy, and generate a multi-view point cloud with higher geometric details.

[0128] Furthermore, this application does not require complex 3D resources and data annotation: through three-dimensional Gaussian splattering technology, high-quality radar synthetic data can be generated directly using existing point cloud data, avoiding the tedious annotation and 3D scene construction process.

[0129] Furthermore, the real-time performance and flexibility of this application are improved: three-dimensional Gaussian splashing achieves fast rendering, can meet real-time requirements, and supports flexible adjustment of radar sensor parameters and positions in a virtual environment to simulate diverse scenarios.

[0130] Furthermore, compared with traditional methods, the radar data (point cloud) generated by the present application has higher geometric accuracy and richer details, which enhances the realism of the simulation effect.

[0131] Furthermore, the present application better simulates the sparse characteristics of radar: the present application can more accurately reproduce the sparse distribution characteristics of radar data, providing more realistic data support for autonomous driving algorithms.

[0132] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the circuits or units is merely a logical function division. In actual implementation, other division methods may be used, such as combining or integrating multiple units or components into another system, or ignoring or not implementing certain features.

[0133] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of this embodiment.

[0134] In addition, each functional unit in each embodiment of the present application may be integrated into a first processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The above-mentioned integrated units may be implemented in the form of hardware or software functional units.

[0135] The above description is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made according to the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A point cloud generation method, characterized in that: The point cloud generation method comprises: Perform Gaussian initialization on the initial point cloud to obtain the first point cloud distribution; Performing parameter modeling on the first point cloud distribution to obtain at least an intensity parameter and a light loss probability; and performing Gaussian flattening on the first point cloud distribution to obtain a Gaussian plane; Calculating the distance from the radar to the Gaussian plane; Inputting the intensity parameter, the light loss probability, and the distance into a range graph to obtain a range graph representation; The range graph representation is converted into a target point cloud, wherein the target point cloud and the initial point cloud are point clouds from different perspectives.

2. The point cloud generation method according to claim 1, characterized in that: The Gaussian initialization of the initial point cloud to obtain a first point cloud distribution includes: Gaussian initialization is performed based on the covariance matrix of the Gaussian distribution and the spatial coordinates of each point in the initial point cloud to obtain a first point cloud distribution.

3. The point cloud generation method according to claim 2, characterized in that: The performing Gaussian flattening on the first point cloud distribution to obtain a Gaussian plane includes: A minimum scale factor is used to perform Gaussian flattening processing on the first point cloud distribution to obtain a Gaussian plane.

4. The point cloud generation method according to claim 1, characterized in that: The performing parameter modeling on the first point cloud distribution to obtain at least an intensity parameter and a light loss probability includes: Obtaining a first direction vector, a first distance, and a feature representation; wherein the first direction vector represents a direction vector of light from the radar sensor position to the Gaussian basis element, and the first distance represents a distance from the radar sensor position to the Gaussian basis element; The first direction vector, the first distance, and the feature representation are input into a multilayer perceptron to obtain the intensity parameter and the light loss probability.

5. The point cloud generation method according to claim 1, characterized in that: The calculating the distance from the radar to the Gaussian plane includes: Obtaining a ray equation of the radar and a plane equation corresponding to the Gaussian plane; Solving the ray equation and the plane equation to obtain a value of a distance parameter in the ray equation; Determine the intersection position between the outgoing ray and the Gaussian plane according to the numerical value; The distance from the radar to the Gaussian plane is calculated according to the intersection point position and the center position of the radar.

6. The point cloud generation method according to claim 1, characterized in that: The range graph is trained in the following way: Inputting the training point cloud into a range graph to obtain a training range graph representation; wherein the training point cloud is annotated with true intensity values, true depth values, true sparse mask values, and rendered normal vector values, and the information represented by the training range graph includes predicted intensity values, predicted depth values, and predicted sparse mask values; Calculating a strength loss value according to the true strength value and the predicted strength value; Calculating a first depth loss value and a second depth loss value according to the true depth value and the predicted depth value; Calculating a sparse mask loss value according to the true sparse mask value and the predicted sparse mask value; Calculating a normal-depth loss value based on the rendered normal vector value and the predicted normal vector value; wherein the predicted normal vector value is obtained by converting the predicted depth value; Adjusting model parameters of the range graph according to the intensity loss value, the first depth loss value, the second depth loss value, the sparse mask loss value, and the normal-depth loss value.

7. The point cloud generation method according to claim 6, characterized in that: The calculating a first depth loss value and a second depth loss value according to the true depth value and the predicted depth value includes: The first depth loss value is calculated using the following formula: Where N represents the number of depth values, d i represents the true depth value of the i-th pixel, represents the predicted depth value of the i-th pixel; The second depth loss value is calculated using the following formula:

8. The point cloud generation method according to claim 6, characterized in that: The calculating the normal-depth loss value according to the rendered normal vector value and the predicted normal vector value includes: The normal-depth loss value is calculated using the following formula: Among them, N represents the number of normal vector values, n i Represents the rendering normal vector value of the i-th pixel, Represents the predicted normal vector value of the i-th pixel, w i Represents the weight that is dynamically adjusted based on deep reliability.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory coupled to the processor; wherein the memory is used to store a computer program, and the processor is used to execute the computer program to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the computer program is used to implement the method according to any one of claims 1 to 8.

Images