Three-dimensional simulation point cloud generation method, system and device and storage medium

Abstract

The invention provides a three-dimensional simulation point cloud generation method, system and device and a storage medium, and relates to the technical field of three-dimensional perception detection, and the method comprises the steps: building a three-dimensional scene model of a target object, and loading three-dimensional vertex data; acquiring laser radar simulation parameters to generate an angle sequence emitted by each laser beam, and rendering each angle direction of the angle sequence into coordinates of pixel points of the three-dimensional scene model according to a preset pixel and angle mapping rule; calculating the three-dimensional scene model according to the angle sequence and the three-dimensional vertex data to obtain a pixel point depth value; performing projection matrix inverse operation on the depth value to obtain an actual distance from the starting point of each laser beam to the surface of the three-dimensional scene model; calculating a point cloud three-dimensional coordinate according to the actual distance and each angle direction; and generating point cloud data. In this way, laser radar simulation and graphic rendering are fused, laser radar scanning is converted into pixel-level calculation, point cloud tracking rays one by one is replaced, the simulation calculation order of magnitude is improved, and consumed time is shortened.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional perception detection, in particular to a three-dimensional simulation point cloud generation method, system, device and storage medium. BACKGROUND

[0002] Point cloud generation technology refers to a method of obtaining, reconstructing or synthesizing three-dimensional point cloud data from a real world or virtual environment through various sensors, algorithms and computing models. Point cloud is composed of a large number of discrete three-dimensional coordinate points (X, Y, Z) and additional attributes (such as color, reflectivity, etc.), and is widely used in autonomous driving, three-dimensional modeling, industrial detection, tunnel engineering construction and other fields.

[0003] The existing research method divides the point cloud generation technology into two technical methods of physical sensor modeling and geometric model generation. The physical sensor modeling method realizes point cloud reconstruction through the interaction coordinate calculation of rays and scene geometry, traverses all the collisions of rays and scenes, and generates million-level point clouds. The geometric model generation method generates point clouds by randomly sampling on the surface of geometric primitives (such as spheres, cubes), applies geometric transformations (such as translation, rotation, noise injection) to the generated point cloud data, learns the point cloud distribution characteristics through a neural network, and generates new point cloud data.

[0004] However, when calculating the ray projection of a large-scale scene, the traditional physical sensor modeling method takes several hours to generate million-level point cloud coordinates, resulting in a serious time consumption; the geometric model generation method requires large-scale labeled data training for hundreds of GPU hours, and the inference delay is high and time-consuming, so the traditional CPU point cloud coordinate simulation is time-consuming and low in efficiency, which needs to be improved. SUMMARY

[0005] Therefore, the present application aims to provide a three-dimensional simulation point cloud generation method, system, device and storage medium, which simulates the point cloud data scanned by laser radar through a graphics rendering method, so that the pixels and point clouds are one-to-one corresponding, and the calculation speed is much faster than CPU simulation, thereby solving the problem of serious time consumption in ray projection simulation calculation of large-scale scenes.

[0006] To achieve the above object, in a first aspect, the application provides a three-dimensional simulation point cloud generation method, comprising: obtaining configured laser radar simulation parameters, establishing a three-dimensional scene model of a target object, and loading three-dimensional vertex data of the target object; generating an angle sequence of each laser beam emission in simulation based on the laser radar simulation parameters, and rendering each angle direction in the angle sequence as a coordinate of a pixel point of the three-dimensional scene model in an image space according to a preset pixel-angle mapping rule; performing depth calculation on the three-dimensional scene model according to the angle sequence and the three-dimensional vertex data to obtain a depth value of the pixel point; performing inverse operation on the depth value by a projection matrix to obtain an actual distance from a starting point of each laser beam to a surface of the three-dimensional scene model; calculating a point cloud three-dimensional coordinate according to the actual distance from the starting point of each laser beam to the surface of the three-dimensional scene model and an angle direction corresponding to the coordinate of each pixel point; and generating point cloud data according to the point cloud three-dimensional coordinate.

[0007] In the embodiment, the three-dimensional scene model of the target object is established, the three-dimensional vertex data of the target object in the environment to be simulated is obtained, and the configured laser radar simulation parameters are obtained at the same time, so that a plurality of laser beams are generated in simulation based on the laser radar simulation parameters, each laser beam has a respective angle direction and forms an angle sequence, each angle direction of the laser beam emission is rendered as a coordinate of a pixel point of the three-dimensional scene model in a graphics space according to a preset pixel-angle mapping rule, the depth value of the pixel point is obtained by performing depth calculation on the coordinate of the pixel point according to the angle sequence and the three-dimensional vertex data, the actual distance from the starting point of each laser beam to the surface of the three-dimensional scene model is obtained by performing inverse operation on the depth value, and the point cloud three-dimensional coordinate is calculated according to the actual distance from the starting point of each laser beam to the surface of the three-dimensional scene model and the angle direction corresponding to the coordinate of each pixel point, so as to obtain the three-dimensional point cloud data. In this way, the laser radar simulation and the graphics rendering depth are fused, the scanning of the laser radar is converted into a pixel-level calculation task, the simulation mode of the traditional CPU point cloud ray-by-ray tracking is replaced, the order of magnitude of the simulation calculation is improved, the point cloud simulation time consumption is shortened, and the ray projection calculation efficiency for a large-scale scene is improved.

[0008] In some embodiments, the laser radar simulation parameters include a scanning field of view angle and an angular resolution, the angular direction includes an azimuth angle and an elevation angle, the generating an angle sequence of each laser beam emission in simulation based on the laser radar simulation parameters, and rendering each angular direction in the angle sequence to a coordinate of a pixel point of the three-dimensional scene model in an image space according to a preset pixel-angle mapping rule includes: generating a corresponding azimuth angle and elevation angle of each laser beam emission in simulation based on the scanning field of view angle and the angular resolution of the laser radar; and rendering the corresponding azimuth angle and elevation angle of each laser beam emission to the coordinate of the pixel point of the three-dimensional scene model in the image space according to the preset pixel-angle mapping rule.

[0009] In this way, the azimuth angle and the elevation angle of the laser beam emission are generated based on the scanning field of view angle and the angular resolution of the laser radar, so that the azimuth angle and the elevation angle of the laser beam emission are rendered to the coordinate of the pixel point according to the preset mapping rule, so that each pixel point represents a laser emission direction in the image space, thereby providing an accurate spatial positioning basis for subsequent ray tracing.

[0010] In some embodiments, the coordinate of the pixel point includes a horizontal pixel index and a vertical pixel index, the azimuth angle in the angular direction corresponding to the coordinate of each pixel point is proportional to the horizontal pixel index, and the elevation angle in the angular direction corresponding to the coordinate of each pixel point is proportional to the vertical pixel index.

[0011] In this way, it is ensured that each pixel point in the image space uniquely corresponds to a laser beam emission direction, and is uniformly distributed and has no omissions, which helps to distinguish and define each pixel point as a two-dimensional position identifier of the three-dimensional scene model in the image space.

[0012] In some embodiments, the depth calculation of the three-dimensional scene model according to the angle sequence and the three-dimensional vertex data to obtain the depth value of the pixel point includes: performing ray tracing on the three-dimensional scene model according to the laser beams of multiple angular directions to construct a ray emitted by the laser radar towards the three-dimensional scene model; and calculating the depth value of the pixel point based on the intersection of the ray and the three-dimensional vertex data.

[0013] In this way, the depth value of the pixel point is obtained by ray tracing and intersection calculation, which helps to simultaneously process intersection operations of millions of rays, thereby significantly improving the calculation efficiency of the depth value.

[0014] In some embodiments, the laser radar simulation parameters include a return mode and a diffusion characteristic of a laser, and the calculation of the depth value of the pixel point based on the intersection of the ray and the three-dimensional vertex data includes: obtaining an intersection point of the ray and the three-dimensional scene model based on the ray and the three-dimensional vertex data, calculating an intersection distance of the laser radar to the intersection point to obtain a first depth value; performing multi-level buffer processing on the first depth value to obtain a second depth value; adding depth noise to the second depth value to obtain a third depth value; and filtering and fusing the third depth value according to the return mode and the diffusion characteristic of the laser to obtain the depth value of the pixel point.

[0015] In this way, the depth value is closer to the actual laser radar collected data after multi-level buffer, noise addition and filtering and fusion, and the reliability and authenticity of the point cloud data are improved.

[0016] In some embodiments, the inverse projection matrix operation on the depth value to obtain the actual distance from the laser beam starting point of each pixel point to the surface of the three-dimensional scene model includes: performing inverse projection matrix operation on the depth value to obtain an intermediate depth value; and combining the intermediate depth value with a nonlinear transformation formula to restore the actual distance from the laser beam starting point of each pixel point to the surface of the three-dimensional scene model.

[0017] In this way, the compressed depth value is converted into an intermediate depth value in a linear space after inverse projection matrix operation, and the actual distance from the laser beam corresponding to the pixel point to the surface of the three-dimensional scene model is inversely calculated through a nonlinear transformation formula, thereby helping to improve the accuracy of spatial positioning of the pixel point.

[0018] In some embodiments, the generation of the point cloud data according to the point cloud three-dimensional coordinates includes: loading a material map of the target object with reflectivity information; calculating a surface reflectivity of the material map corresponding to each laser beam emission direction based on the three-dimensional scene model and the material map to obtain a reflectivity value; and generating the point cloud data according to the point cloud three-dimensional coordinates and the reflectivity value.

[0019] In this way, the interaction of the laser and the surface of the object is simulated through the material map to obtain the reflectivity value corresponding to each pixel point, so that the reflectivity value can be combined with the point cloud three-dimensional coordinates to generate the point cloud data, which helps to enrich the information dimension of the point cloud data and is used for identifying material types, monitoring obstacle properties and other perception tasks.

[0020] In a second aspect, an embodiment of the present application provides a three-dimensional simulation point cloud generation system, the system comprising: an acquisition module configured to acquire configured laser radar simulation parameters, and to establish a three-dimensional scene model of a target object and load three-dimensional vertex data of the target object; a rendering module configured to generate an angle sequence of each laser beam emission in simulation based on the laser radar simulation parameters, and to render each angle direction in the angle sequence as a coordinate of a pixel point of the three-dimensional scene model in an image space according to a preset pixel-angle mapping rule; a processing module configured to perform depth calculation on the three-dimensional scene model according to the angle sequence and the three-dimensional vertex data, to obtain a depth value of the pixel point; to perform inverse projection matrix operation on the depth value to obtain an actual distance from a starting point of each laser beam to a surface of the three-dimensional scene model; to calculate a point cloud three-dimensional coordinate according to the actual distance from the starting point of each laser beam to the surface of the three-dimensional scene model and an angle direction corresponding to the coordinate of each pixel point; and to generate point cloud data according to the point cloud three-dimensional coordinate.

[0021] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, the memory storing a computer program capable of being executed by the processor, and the processor being capable of executing the computer program to implement the three-dimensional simulation point cloud generation method of the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the three-dimensional simulation point cloud generation method of the first aspect.

[0023] In order to make the above objectives, characteristics and advantages of the present application more apparent, clear and easy to understand, the following will describe a preferred embodiment in detail, and the accompanying drawings will be referred to, and the detailed description will be as follows. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to make the above objectives, characteristics and advantages of the present application more apparent, clear and easy to understand, the following will describe a preferred embodiment in detail, and the accompanying drawings will be referred to, and the detailed description will be as follows.

[0025] Figure 1 A three-dimensional simulation point cloud generation method flowchart provided by an embodiment of the present application; Figure 2 A laser radar scanning range schematic diagram provided by an embodiment of the present application; Figure 3 A three-dimensional simulation point cloud generation method flowchart provided by an embodiment of the present application; Figure 2 A three-dimensional simulation point cloud generation method flowchart provided by an embodiment of the present application; Figure 4 A schematic diagram of the functional modules of the three-dimensional simulation point cloud generation system provided in the embodiments of the present invention; Figure 5 A block diagram of an electronic device provided in an embodiment of the present invention.

[0026] Icons: 1000 - 3D simulation point cloud generation system; 1100 - Acquisition module; 1200 - Rendering module; 1300 - Processing module; 2000 - Electronic device; 2100 - Processor; 2200 - Memory; 2300 - Bus; 2400 - Communication interface. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0028] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0029] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0030] The following is a brief introduction to some concepts that may be involved in the embodiments of the present invention.

[0031] As mentioned in the background, the main method for modeling based on physical sensors is ray casting: it simulates the emission, reflection, and reception process of LiDAR light, calculates the collision points between each ray and the scene to generate a point cloud, requires traversing all collisions between rays and the scene, and generating millions of point clouds takes several hours. Methods based on geometric models mainly include parametric random generation, point cloud deformation enhancement, and deep learning generation. These methods require hundreds of GPU hours of training on large-scale labeled data, and inference latency is also significant. Therefore, traditional CPU point cloud simulation is time-consuming and inefficient, and needs improvement.

[0032] To address the aforementioned issues, this invention provides a method, system, device, and storage medium for generating three-dimensional simulation point clouds. By transforming the traditional LiDAR point cloud simulation process into efficient parallel processing based on the graphics rendering pipeline, a one-to-one correspondence is established between pixels in graphics rendering and point clouds. The generation latency for millions of point clouds is less than 10ms, and the calculation speed is much faster than CPU simulation, thus solving the problem of time-consuming ray projection simulation calculations for large-scale scenes.

[0033] See Figure 1 , Figure 1 The present invention provides a flowchart of a three-dimensional simulation point cloud generation method, which includes steps S100 to S600.

[0034] S100: Obtain the configured lidar simulation parameters, establish a 3D scene model of the target object, and load the 3D vertex data of the target object.

[0035] In this embodiment, when establishing a 3D scene model of the target object, the 3D vertex data of the target object is loaded based on the input 3D scene model data of the target object. The 3D vertex data includes data on the shape and surface properties of the target object. Simultaneously, the simulation parameters of the lidar are configured. These parameters include the scanning method (e.g., mechanical or solid-state), scanning range (horizontal and vertical angle range), scanning resolution (sampling density per angle), echo mode (whether multi-echo detection is supported), and physical characteristics such as laser diffusion and intensity.

[0036] For example, the scanning method indicates the scanning mechanism used by the lidar, such as mechanical rotation, MEMS micromirror control, or OPA optical phased array. The scanning range defines the maximum horizontal and vertical angular range that the laser beam can cover, such as a 360° horizontal field of view and ±25° vertical field of view. The scanning resolution represents the minimum step angle of the laser beam in the horizontal and vertical directions, determining the spatial density of the point cloud. The echo mode defines whether it supports the reception of multiple reflected signals, such as single-echo, dual-echo, or multi-echo modes. Laser diffusion characterizes the degree of divergence during laser beam propagation, affecting detection accuracy. The laser intensity sets the emitted energy value of the laser pulse, used for subsequent reflection intensity calculations.

[0037] S200: Generate the angle sequence of each laser beam emitted in the simulation based on the simulation parameters of the lidar, and render each angle direction in the angle sequence as the coordinates of the pixel point of the three-dimensional scene model in the image space according to the preset pixel-angle mapping rules.

[0038] In this embodiment, a virtual laser emission path is constructed in the graphics rendering pipeline based on the LiDAR scanning parameters, mainly involving the configuration of the vertex shader. The vertex shader maps each scanning direction to a pixel in the image space according to the LiDAR scanning method. For example, a horizontal 360-degree scan will have 3600 pixels, with one pixel every 0.1 degrees. If there is a 40-degree range in the vertical direction, there will be 400 pixels, with one pixel every 0.1 degrees.

[0039] Specifically, the vertex shader is responsible for rendering the emission angle direction (composed of azimuth angle φ and pitch angle θ) of each laser beam in the angle sequence as pixel coordinates (u, v) in image space. In other words, each pixel uniquely corresponds to the emission direction of a laser beam. The preset pixel-to-angle mapping rule is implemented through a projection matrix in a normalized device coordinate system (-1 to 1), ensuring that each laser beam is accurately projected onto its corresponding pixel position.

[0040] For example, the lidar simulation parameters also include the scanning field of view and angular resolution. The angular directions include azimuth and elevation angles. Based on the lidar's scanning field of view and angular resolution, the azimuth and elevation angles corresponding to the emission of each laser beam in the simulation are generated.

[0041] Specifically, angular resolution includes horizontal angular resolution and vertical angular resolution. Horizontal angular resolution is the minimum interval (e.g., 0.1°, 0.2°) in azimuth between two adjacent scan lines (or two adjacent emissions on the same line), determining the density of the point cloud in the horizontal direction. Vertical angular resolution is the minimum interval (e.g., 2°, 0.4°) in elevation between two adjacent laser lines (or vertical scan steps), determining the density of the point cloud in the vertical direction. When simulating and generating the scan angle sequence (φ and θ), steps or sampling are performed according to the designed resolution. For example, a 360° horizontal scan with a resolution of 0.2° requires 360 / 0.2 = 1800 azimuth steps. Field of view includes horizontal and vertical field of view. The horizontal field of view is the maximum horizontal angle range that the radar can scan (e.g., 360°, 120°). The vertical field of view is the maximum vertical angle range that the radar can scan (e.g., 40°, 30°). The φ and θ sequences generated in the simulation must be within the defined FOV range; points outside the FOV will not be simulated.

[0042] Furthermore, based on the scanning field of view and angular resolution, a series of laser emission directions are generated sequentially in the virtual environment at set step intervals. Each direction is characterized by both the azimuth angle φ and the elevation angle θ. For example, if the horizontal scanning field of view is 360° and the angular resolution is 0.1°, a total of 3600 azimuth directions are generated; if the vertical field of view is 40° and the angular resolution is 0.1°, a total of 400 elevation directions are generated. This constitutes a complete angular sequence containing 3600 × 400 = 1,440,000 angular directions.

[0043] Further, see Figure 2 , Figure 2 This is a schematic diagram of the laser radar scanning range provided in an embodiment of the present invention. The azimuth angle is the angle between the projection of the laser beam onto a horizontal plane (usually a plane parallel to the radar mounting plane) and a certain reference direction (usually directly in front of the radar). Figure 2 The 120° is represented by the symbol φ (phi); its range is [0°, 360°) or [-180°, 180°]. For example, in a mechanically rotating radar, φ is determined by the real-time rotation angle of the internal rotating motor (such as a turntable or rotating mirror). Therefore, for uniform rotation (mechanical), φ = (current_step / total_steps_per_rotation) * 360°. The elevation angle is the angle between the laser beam and the horizontal plane. Up is positive, down is negative. The symbol is θ (theta); its range depends on the radar design (e.g., -25° to +25°). Figure 2In the case of a multi-line mechanical rotating radar, θ is determined by vertically arranged fixed lidars. Each lidar has a fixed elevation angle offset θ_line. Therefore, when the lidar rotates, each line scans on its own vertical plane. Another example is a single-line rotating radar, which is typically fixed at 0° (horizontal scan) or a fixed small angle.

[0044] For example, the azimuth and elevation angles corresponding to the emission of each laser beam are rendered as the coordinates of the pixels of the three-dimensional scene model in the image space according to the preset pixel-angle mapping rules.

[0045] In this embodiment, the preset pixel-angle mapping rule converts the two-dimensional Cartesian coordinate system in the image space into an angle sequence centered on the lidar, and combines it with the distance information from the pixel to the lidar to construct a complete mapping rule. For example, pixel coordinates (u,v) - corresponding laser beam angle direction (φ, θ): (0,0) - (φ=-180°, θ=+25°), (1,0) - (φ=-179.9°, θ=+25°), (...-...), (3599,351) - (φ=+180°, θ=-25°). Thus, the direction of each laser beam is encoded as the coordinates of a pixel on the image.

[0046] In some embodiments, the coordinates of a pixel include a horizontal pixel index and a vertical pixel index. The azimuth angle in the angular direction corresponding to the coordinates of each pixel is proportional to the horizontal pixel index, and the pitch angle in the angular direction corresponding to the coordinates of each pixel is proportional to the vertical pixel index.

[0047] In this embodiment, the azimuth angle φ is proportional to the horizontal pixel index, and the elevation angle θ is proportional to the vertical pixel index, achieving a one-to-one mapping relationship. The horizontal scanning range (i.e., the scanning field of view) of the LiDAR is divided into several equally spaced step angles, with each step corresponding to a horizontal pixel index. Therefore, there is a linear relationship between the azimuth angle φ and the horizontal pixel index, for example, φ = (horizontal pixel index / image width) × horizontal field of view. Similarly, there is also a linear relationship between the elevation angle θ and the vertical pixel index, for example, θ = θ_min + (vertical pixel index / image height) × vertical field of view span. This linear proportional relationship between the pixel index and the scanning angle of the LiDAR ensures that each pixel represents a laser emission direction in the image space, providing a precise spatial positioning basis for subsequent ray tracing.

[0048] S300: Perform depth calculation on the 3D scene model based on the angle sequence and 3D vertex data to obtain the depth value of each pixel.

[0049] In this embodiment, ray tracing is performed on the 3D scene model based on laser beams from multiple angles to construct rays emitted by the LiDAR towards the 3D scene model. The depth value of a pixel is calculated based on the intersection of the ray and the 3D vertex data. It can be understood that by using a ray tracing algorithm in the graphics rendering pipeline, the intersection of each ray and the 3D vertex data is calculated to obtain the nearest intersection point between the ray and the surface of the 3D model. The distance between this intersection point and the starting point of the ray is the depth value corresponding to that pixel.

[0050] For example, constructing a virtual laser emission path in the graphics rendering pipeline based on LiDAR scanning parameters also involves configuring the fragment shader. The fragment shader is used to simulate the interaction between light and the scene, calculating the depth of the laser beam path corresponding to each pixel based on the angle sequence and 3D vertex data, and outputting a depth map. The depth map reflects the distance of the laser to the object in each direction, which is the depth value of the pixel.

[0051] S400: Perform inverse projection matrix operation on the depth value to obtain the actual distance from the laser beam origin of each pixel to the surface of the 3D scene model.

[0052] In this embodiment, the depth values ​​are inversely projected using a projection matrix to obtain intermediate depth values. These intermediate depth values ​​are then combined with a nonlinear transformation formula to restore the actual distance from the laser beam origin of each pixel to the surface of the 3D scene model. The reason for performing the inverse operation to obtain intermediate depth values, and then transforming them into actual distances, is that the pixel depth values ​​stored in the depth map do not directly represent physical distances, but are normalized depth values ​​compressed to the range [0,1] after perspective projection transformation. This is because the GPU's depth buffer has limited precision (typically 24-bit or 32-bit floating-point numbers), and to optimize graphics operations such as Z-buffer sorting, the original distance values ​​must be nonlinearly compressed. To restore the true physical distance, these compressed values ​​must undergo an inverse projection transformation, for example, Z... =2Z depth -1, Z depth Z represents the depth value in the depth map. This is the intermediate depth value, often referred to as the "linear depth" or "view space depth." This value is then used to inversely calculate the final distance value using the original projection function. For example, the final distance value... .

[0053] For example, if the normalized depth value of a pixel is 0.5, and the near clipping plane distance of the LiDAR is 0.1 meters and the far clipping plane distance is 100 meters, then the true distance between the laser beam corresponding to the pixel and the obstacle can be calculated through inverse transformation to be approximately 9.1 meters, so as to ensure the quality of point cloud data.

[0054] S500: Calculate the three-dimensional coordinates of the point cloud based on the actual distance from the laser beam origin of each pixel to the surface of the three-dimensional scene model and the angle direction corresponding to the coordinates of each pixel.

[0055] In this embodiment, based on the actual distance and the corresponding angular direction (azimuth φ and pitch θ), the 3D coordinates of each point are calculated using the conversion formulas from spherical coordinates to Cartesian coordinates: x = r·cosθ·cosφ; y = r·cosθ·sinφ; z = r·sinθ. Wherein, cos(θ): projects the hypotenuse r onto the horizontal plane XY, obtaining the horizontal projection length r_h = r*cos(θ). cos(φ) and sin(φ): decompose the horizontal projection length r_h into X-axis and Y-axis components. sin(θ): directly obtains the Z-axis component (height).

[0056] S600: Generate point cloud data based on the three-dimensional coordinates of the point cloud.

[0057] In this embodiment, a material texture map containing reflectivity information is loaded onto the target object. Based on the 3D scene model and the material texture map, the surface reflectivity of the material texture map corresponding to each laser beam emission direction is calculated to obtain the reflectivity value. Point cloud data is generated based on the point cloud 3D coordinates and reflectivity values.

[0058] For example, point cloud data is a data format that expresses the three-dimensional coordinates of points and attributes such as reflectivity and color. It typically consists of a set of m x n floating-point numbers, where m represents the attribute of each point, such as x, y, and z coordinates, and for reflectivity, m = 4; n is the total number of points. It can be understood that point cloud data not only contains three-dimensional spatial location information but also additional attributes reflecting the surface characteristics of an object. Therefore, after establishing a three-dimensional scene model of the target object and loading its three-dimensional vertex data, a material map containing reflectivity information is further loaded. Based on the geometry and material distribution of the three-dimensional scene model, the surface reflection intensity in each laser beam irradiation direction is calculated. Specifically, in the fragment shader, the interaction between the laser beam and the object surface is simulated by combining the incident angle and material map information, thereby obtaining the reflectivity value corresponding to each pixel. Finally, this reflectivity value is combined with the existing three-dimensional coordinates of the point cloud to generate complete point cloud data, where each point contains at least (x, y, z) coordinates and reflectivity information. By introducing the reflective properties of materials, the information dimensions of point cloud data are enriched, making it not only usable for spatial modeling, but also for higher-order perception tasks such as identifying material types and detecting obstacle properties.

[0059] Specifically, reflectivity depends on surface material properties and lighting geometry. Different materials (metals, plastics, etc.) have different light-reflecting capabilities. The angle of incidence is the angle between the light ray and the surface normal (following the Fresnel effect). Surface roughness is the scattering effect of surface microstructure on reflection. Calculating the reflectivity of point cloud points in the fragment shader essentially involves simulating the interaction between light and the surface using a physically based lighting model and leveraging GPU resources to accelerate the computation of the entire scene. For example, the calculation process may include: Prepare the basic vectors: obtain the normal vector N of the current point (perpendicular to the surface); calculate the ray direction L (unit vector from the point to the light source); calculate the line-of-sight direction V (unit vector from the point to the camera); calculate the half-angle vector H (the intermediate vector between L and V: H = normalize(L + V)).

[0060] The diffuse reflection component is calculated using the Lambert model: reflectivity_diff = max(0, dot(N, L)) *baseReflectivity; dot(N, L) calculates the cosine of the angle between the normal and the ray (range 0~1); baseReflectivity is the material's base reflectivity (0~1).

[0061] Specular reflection component calculation, normal distribution function (NDF): The normal distribution of the micro-surface is calculated using the GGX model; formula: NDF=α² / [π*(cos²θ*(α²-1)+1)²]; where α is the surface roughness, and θ is the angle between the normal and the half-angle vector.

[0062] Fresnel equation (F): using the Schlick approximation formula; formula: F = F0 + (1 - F0) * (1 - cos 5 θ); F0 is the reflectivity at perpendicular incidence, and cosθ is the angle between the line of sight and the half-angle vector. Composite specular reflection: Specular reflection = (NDF*F) / (4*max(dot(N,V),0.001)*max(dot(N,L),0.001)).

[0063] The final reflectance is calculated as follows: Total reflectance = diffuse component + specular component; formula: reflectance = reflectance_diff + reflectance_spec. Energy conservation constraints are required to ensure the total reflectance does not exceed 1.0; specular reflection is diffused on rough surfaces and concentrated on smooth surfaces. Output processing involves mapping reflectance values ​​in the [0,1] range to grayscale or pseudo-color, outputting fragment color values.

[0064] It's worth noting that because the entire point cloud generation process is implemented entirely based on the graphics rendering pipeline and developed using the internationally standard OpenGL interface, it can be directly embedded into mainstream game engines such as Unity and Unreal Engine. There's no need to create a new 3D scene model specifically for point cloud simulation; it can be used on any display device without code modifications, significantly improving compatibility. Furthermore, leveraging mature graphics rendering technologies, such as ray tracing, for simulating LiDAR ranging echoes greatly enhances the realism of the simulation. All assets used can be directly applied to graphics rendering and share asset resources in video memory, improving data transfer performance.

[0065] In some embodiments, to obtain more realistic and accurate depth values, further processing of the obtained depth values ​​is required. This is done after configuring the LiDAR's echo mode and laser diffusion characteristic parameters, such as... Figure 3 As shown, Figure 3 for Figure 2 The flowchart of the sub-steps in step S300 includes S301 to S304: S301. Based on the ray and 3D vertex data, obtain the intersection point of the ray and the 3D scene model, calculate the intersection point distance of the laser radar to the intersection point, and obtain the first depth value.

[0066] In this embodiment, the distance between the laser radar and the nearest intersection point between the ray and the surface of the three-dimensional model can be calculated using a ray tracing algorithm to obtain the first depth value.

[0067] S302. Perform multi-level buffering on the first depth value to obtain the second depth value.

[0068] In this embodiment, since the depth buffer precision of the GPU is limited, typically 32-bit floating-point, it is necessary to perform multi-level buffering to improve the depth resolution and obtain a more accurate second depth value.

[0069] S303. Add depth noise to the second depth value to obtain the third depth value.

[0070] In this embodiment, in order to simulate the scanning behavior of a real lidar, depth noise, such as Gaussian noise, is added to the second depth value to reproduce the measurement error of the lidar in actual operation, thus obtaining the third depth value.

[0071] S304. The third depth value is filtered and fused according to the echo pattern and the diffusion characteristics of the laser to obtain the depth value of the pixel.

[0072] In this embodiment, based on the echo mode (such as single echo, double echo, or multiple echo) and the laser diffusion characteristics (such as beam divergence) in the lidar simulation parameters, the processed third depth value is filtered and fused to obtain the final pixel depth value, so as to eliminate invalid or interfering echoes, weighted fuse multiple valid echoes, and improve the depth accuracy of each pixel in the final image.

[0073] Based on the above method, embodiments of the present invention also provide a system corresponding to the above method, such as... Figure 4 As shown, Figure 4 This is a functional module diagram of the three-dimensional simulation point cloud generation system 1000 provided in this embodiment of the invention. It should be noted that the basic principle and technical effects of the three-dimensional simulation point cloud generation system 1000 provided in this embodiment are the same as those in the above method embodiments. For the sake of brevity, parts not mentioned in this embodiment can be referred to the corresponding content in the method embodiments.

[0074] In this embodiment, the 3D simulation point cloud generation system 1000 includes an acquisition module 1100, a rendering module 1200, and a processing module 1300. The acquisition module 1100 is used to acquire configured LiDAR simulation parameters, establish a 3D scene model of the target object, and load the 3D vertex data of the target object. It can be understood that the acquisition module 1100 is used to perform the above-described step S100.

[0075] The rendering module 1200 is used to generate an angle sequence of each laser beam emitted in the simulation based on the LiDAR simulation parameters, and to render each angle direction in the angle sequence as the coordinates of a pixel point in the image space of the 3D scene model according to a preset pixel-to-angle mapping rule. It can be understood that the rendering module 1200 is used to perform the above step S200.

[0076] Processing module 1300 is used to perform depth calculations on the 3D scene model based on angle sequences and 3D vertex data to obtain the depth values ​​of pixels; to perform inverse projection matrix operations on the depth values ​​to obtain the actual distance from the laser beam origin of each pixel to the surface of the 3D scene model; to calculate the 3D coordinates of the point cloud based on the actual distance from the laser beam origin of each pixel to the surface of the 3D scene model and the angle direction corresponding to the coordinates of each pixel; and to generate point cloud data based on the 3D coordinates of the point cloud. It can be understood that processing module 1300 is used to execute the above steps S300~S600.

[0077] In some embodiments, the lidar simulation parameters include the scanning field of view and angular resolution, and the angular directions include azimuth and elevation angles. Pixel coordinates include horizontal and vertical pixel indices. The azimuth angle corresponding to each pixel's coordinates is proportional to the horizontal pixel index, and the elevation angle corresponding to each pixel's coordinates is proportional to the vertical pixel index. The rendering module 1200 is also used to generate the azimuth and elevation angles corresponding to the emission of each laser beam in the simulation based on the lidar's scanning field of view and angular resolution. The azimuth and elevation angles corresponding to the emission of each laser beam are then rendered as pixel coordinates of the 3D scene model in image space according to a preset pixel-to-angle mapping rule.

[0078] In some embodiments, the processing module 1300 is further configured to perform ray tracing on the three-dimensional scene model based on laser beams from multiple angles and directions to construct rays emitted by the lidar toward the three-dimensional scene model; and calculate the depth value of a pixel based on the intersection of the rays and the three-dimensional vertex data.

[0079] In some embodiments, the lidar simulation parameters include the echo pattern and the laser diffusion characteristics. The processing module 1300 is further configured to obtain the intersection point of the ray and the three-dimensional scene model based on the ray and three-dimensional vertex data, calculate the intersection point distance from the lidar to the intersection point, and obtain a first depth value; perform multi-level buffering processing on the first depth value to obtain a second depth value; add depth noise to the second depth value to obtain a third depth value; and filter and fuse the third depth value according to the echo pattern and the laser diffusion characteristics to obtain the depth value of the pixel. It can be understood that the processing module 1300 is also configured to perform the above steps S301~S304.

[0080] In some embodiments, the processing module 1300 is further configured to perform an inverse projection matrix operation on the depth value to obtain an intermediate depth value; and to combine the intermediate depth value with a nonlinear transformation formula to restore the actual distance from the laser beam origin of each pixel to the surface of the three-dimensional scene model.

[0081] In some embodiments, the acquisition module 1100 is further configured to load a material map of the target object containing reflectivity information.

[0082] In some embodiments, the processing module 1300 is further configured to calculate the reflectivity of the material map surface corresponding to each laser beam emission direction based on the three-dimensional scene model and material map, and obtain the reflectivity value; and generate point cloud data based on the point cloud three-dimensional coordinates and reflectivity value.

[0083] Based on the same inventive concept disclosed above, the present invention also provides a block diagram of an electronic device 2000 performing the above method. Please refer to... Figure 5 , Figure 5This is a block diagram of an electronic device 2000 provided in an embodiment of the present invention. The electronic device 2000 includes a processor 2100, a memory 2200, a bus 2300, and a communication interface 2400. The processor 2100 and the memory 2200 are connected via the bus 2300, and the processor 2100 communicates with external devices via the communication interface 2400.

[0084] Processor 2100 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed through integrated logic circuits in the hardware of processor 2100 or through software instructions. The processor 2100 may be a general-purpose processor 2100, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0085] The memory 2200 is used to store computer programs. For example, the three-dimensional simulation point cloud generation system 1000 in this embodiment of the invention includes at least one software function module that can be stored in the memory 2200 in the form of software or firmware. After receiving the execution instruction, the processor 2100 executes the program to implement the three-dimensional simulation point cloud generation method in this embodiment of the invention.

[0086] The memory 2200 may include high-speed random access memory (RAM) or non-volatile memory. Optionally, the memory 2200 may be a storage device built into the processor 2100 or a storage device independent of the processor 2100.

[0087] Bus 2300 can be ISA bus 2300, PCI bus 2300 or EISA bus 2300, etc. Figure 5 It is indicated by only one double-headed arrow, but does not mean that there is only one bus 2300 or one type of bus 2300.

[0088] Electronic devices 2000 can be mobile phones, tablets, laptops, desktop computers, and other computer devices.

[0089] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor 2100, implements the three-dimensional simulation point cloud generation method described above. This computer-readable storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0090] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating three-dimensional simulation point clouds, characterized in that, The method includes: Obtain the configured lidar simulation parameters, establish a three-dimensional scene model of the target object, and load the three-dimensional vertex data of the target object; Based on the LiDAR simulation parameters, an angle sequence of each laser beam emitted in the simulation is generated, and according to the preset pixel-to-angle mapping rules, each angle direction in the angle sequence is rendered as the coordinates of the pixel point of the three-dimensional scene model in the image space. Based on the angle sequence and the three-dimensional vertex data, the depth of the three-dimensional scene model is calculated to obtain the depth value of the pixel. The depth value is inversely projected to obtain the actual distance from the laser beam origin of each pixel to the surface of the three-dimensional scene model. The three-dimensional coordinates of the point cloud are calculated based on the actual distance from the laser beam origin of each pixel to the surface of the three-dimensional scene model and the angle direction corresponding to the coordinates of each pixel. Point cloud data is generated based on the three-dimensional coordinates of the point cloud.

2. The method according to claim 1, characterized in that, The lidar simulation parameters include scanning field of view and angular resolution, and the angular directions include azimuth and elevation angles. The process of generating an angle sequence for each laser beam emitted in the simulation based on these lidar simulation parameters, and rendering each angular direction in the angle sequence as the coordinates of a pixel in the image space of the 3D scene model according to a preset pixel-to-angle mapping rule, includes: Based on the scanning field of view and angular resolution of the lidar, the azimuth and elevation angles corresponding to the emission of each laser beam in the simulation are generated; The azimuth and elevation angles corresponding to the emission of each laser beam are rendered as the coordinates of the pixel points of the three-dimensional scene model in the image space according to the preset pixel-angle mapping rules.

3. The method according to claim 2, characterized in that, The coordinates of the pixel include a horizontal pixel index and a vertical pixel index. The azimuth angle in the angular direction corresponding to the coordinates of each pixel is proportional to the horizontal pixel index, and the pitch angle in the angular direction corresponding to the coordinates of each pixel is proportional to the vertical pixel index.

4. The method according to claim 1, characterized in that, The step of calculating the depth of the three-dimensional scene model based on the angle sequence and the three-dimensional vertex data to obtain the depth value of the pixel includes: Ray tracing is performed on the 3D scene model based on laser beams from multiple angles and directions to construct the rays emitted by the lidar toward the 3D scene model; The depth value of the pixel is calculated by intersecting the ray and the three-dimensional vertex data.

5. The method according to claim 4, characterized in that, The lidar simulation parameters include the echo mode and the laser diffusion characteristics. The depth value of the pixel obtained by calculating the intersection of the ray and the three-dimensional vertex data includes: Based on the ray and the three-dimensional vertex data, the intersection point of the ray and the three-dimensional scene model is obtained, and the intersection distance of the laser radar to the intersection point is calculated to obtain the first depth value; The first depth value is buffered at multiple levels to obtain the second depth value; Add depth noise to the second depth value to obtain a third depth value; The third depth value is filtered and fused based on the echo pattern and the diffusion characteristics of the laser to obtain the depth value of the pixel.

6. The method according to claim 1, characterized in that, The step of performing an inverse projection matrix operation on the depth value to obtain the actual distance from the laser beam origin of each pixel to the surface of the 3D scene model includes: The intermediate depth value is obtained by performing an inverse projection matrix operation on the depth value. The intermediate depth value is then combined with a nonlinear transformation formula to restore the actual distance from the laser beam origin of each pixel to the surface of the 3D scene model.

7. The method according to claim 1, characterized in that, The step of generating point cloud data based on the three-dimensional coordinates of the point cloud includes: Load the material texture map containing reflectivity information of the target object; Based on the three-dimensional scene model and the material map, the surface reflectivity of the material map corresponding to the emission direction of each laser beam is calculated to obtain the reflectivity value; Point cloud data is generated based on the three-dimensional coordinates of the point cloud and the reflectivity value.

8. A three-dimensional simulation point cloud generation system, characterized in that, The system includes: The acquisition module is used to acquire the configured lidar simulation parameters, establish a three-dimensional scene model of the target object, and load the three-dimensional vertex data of the target object. The rendering module is used to generate an angle sequence of each laser beam emitted in the simulation based on the LiDAR simulation parameters, and to render each angle direction in the angle sequence as the coordinates of a pixel point of the three-dimensional scene model in the image space according to a preset pixel-angle mapping rule. The processing module is configured to perform depth calculation on the 3D scene model based on the angle sequence and the 3D vertex data to obtain the depth value of each pixel; to perform inverse projection matrix operation on the depth value to obtain the actual distance from the laser beam origin of each pixel to the surface of the 3D scene model; to calculate the 3D coordinates of the point cloud based on the actual distance from the laser beam origin of each pixel to the surface of the 3D scene model and the angle direction corresponding to the coordinates of each pixel; and to generate point cloud data based on the 3D coordinates of the point cloud.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program that can be executed by the processor to implement the three-dimensional simulation point cloud generation method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the three-dimensional simulation point cloud generation method as described in any one of claims 1-7.

Images