High-performance point cloud distance sampling simulation method
By introducing linear depth processing shaders into the game engine and optimizing calculations using the depth test process, the performance bottleneck of traditional lidar simulation methods is solved, efficient point cloud distance sampling is achieved, and real-time simulation needs of autonomous driving systems are met.
Patent Information
- Application Number
- CN202510461547.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-14
- Publication Date
- 2025-07-22
AI Technical Summary
Traditional lidar simulation methods have high computing overhead in large-scale scenarios, which is difficult to meet real-time requirements, and the simulation efficiency is inefficient.
By introducing a linear depth processing shader into the game engine, using the depth test process, obtaining and calculating the distance of objects within the camera's visual range, reducing ray collision detection, using depth map sampling and buffer parameters for distance conversion, generating efficient point cloud data.
It significantly reduces computing overhead, improves simulation efficiency, realizes high-performance and real-time point cloud distance sampling, and provides reliable simulation data support for autonomous driving systems.
Smart Images
Figure CN120355871A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of point cloud simulation, and particularly relates to a high-performance point cloud distance sampling simulation method. Background Art
[0002] With the rapid development of automotive intelligence and autonomous driving technologies, lidar systems have become one of the important sensors for vehicle environmental perception. Lidar can provide high-resolution point cloud data for detecting obstacles, pedestrians, vehicles, etc. in the surrounding environment, providing key information for the decision-making and control of autonomous driving systems.
[0003] In the research and development of autonomous driving systems, in order to train and test algorithms, researchers usually need a large amount of driving data in real scenarios. However, collecting real driving data faces high costs, time consumption, and safety risks. To solve this problem, simulation technologies have emerged, allowing developers to use virtual environments to generate driving data for algorithm training and testing.
[0004] Current mainstream simulation methods use game engines to build virtual scenes and simulate real-world objects through cameras and sensors. However, in terms of lidar distance sampling simulation, traditional methods have performance bottlenecks and are difficult to meet the real-time requirements. Specifically, using a camera to emit virtual rays towards surrounding objects, obtaining the collision points between the rays and surrounding objects through ray detection, and then calculating the distance based on the collision points and camera coordinates consumes a large amount of performance and is difficult to achieve efficient distance sampling in complex scenarios.
[0005] Traditional lidar simulation methods use camera-based ray detection. A common implementation of this method is to use a game engine to build a virtual scene. Taking the Unity game engine as an example, in a scene containing a large number of faces, for a distance detection with a resolution of 1920*1080, it requires a running time of hundreds of milliseconds. Traditional methods using camera-based ray detection need to perform detailed collision detection and complex distance calculations for each ray, which results in significant computational overhead in large-scale scenes, is difficult to meet the real-time requirements, has low simulation efficiency, and cannot meet the refresh requirements above 60 Hz. This result shows that traditional simulation methods have obvious performance bottlenecks. Summary of the Invention
[0006] To solve the problems raised in the above background art, the present invention provides a high-performance point cloud distance sampling simulation method to solve the problems of obvious performance bottlenecks and low simulation efficiency of traditional simulation methods.
[0007] To achieve the above object, the present invention provides the following technical solutions: A high-performance point cloud distance sampling simulation method, comprising the following steps: S1: Obtain depth sampling data from the camera rendering process of the game engine. The depth sampling data represents the distance from the camera in the screen to each pixel point in the rendered scene. The depth sampling data consists of multiple four-dimensional floating-point arrays; S2: Introduce a shader for linear depth processing in the game engine, declare the depth map sampling data of the shader, and assign the depth sampling data to the depth map sampling data; S3: In the fragment shader function of the shader, sample the depth map sampling data to obtain the depth data of each point on the screen. The depth data is a component of the four-dimensional floating-point array; S4: Define the depth buffer parameter bufferParams based on the near plane and far plane of the camera; S5: In the fragment shader function, calculate the linear depth data through bufferParams, and obtain an image of the linear depth data through shader rendering. Its resolution is the same as the screen resolution. Any point on the screen can obtain the corresponding object distance data through the depth data; S6: Take values point by point from the obtained linear depth data, perform distance conversion, and map the linear depth data to the actual distance by setting the camera far plane distance to generate point cloud data.
[0008] Preferably, the game engine uses the Unity engine.
[0009] Preferably, the floating-point value range of the depth data in the four-dimensional floating-point array is between 0 and 1.
[0010] Preferably, in S2, depth testing is declared to be always enabled in the shader to ensure that the depth information is always valid.
[0011] Preferably, in S4, the depth buffer parameter bufferParams is expressed as: bufferParams = float4((1.0 - far / near), (far / near), (x / far), (y / far)); Where far is the camera far plane, near is the camera near plane, float4 is a four-dimensional floating-point array, and the four components in the four-dimensional floating-point array are represented by x, y, z, and w respectively. Among them, x is (1.0 - far / near), y is (far / near), z is (x / far), and w is (y / far).
[0012] Preferably, the specific calculation method of the linear depth data depth in S5 is as follows: depth = 1 / (bufferParams.x * z + bufferParams.y).
[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: By introducing the method of linear depth testing, this application modifies the graphics processing flow of the game engine to achieve high-performance detection of the distance of objects within the visible range of the camera. Through linear depth testing, this application makes full use of the depth testing process of the GPU during rendering, reducing the need for detailed collision detection for each ray. This application samples the depth map in the fragment shader function to achieve efficient calculation of the depth of the object surface. Compared with the traditional method, this application not only significantly reduces the computational overhead, but also shows higher performance in large-scale scenes, greatly improving the simulation efficiency. Therefore, this application can achieve the purpose of high-performance and real-time point cloud distance sampling by optimizing the lidar simulation method, providing more reliable and efficient data support for the simulation of autonomous driving systems. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 is a schematic flow chart of this application; Figure 2 is a schematic diagram of this scene construction; Figure 3 Schematic diagram of the scene photographed by the camera; Figure 4 is a schematic diagram of the original depth; Figure 5 is a schematic diagram of the processed depth. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0015] To facilitate the understanding of the technical content of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and specific examples. It should be understood that the specific examples described herein are only used to explain the present invention and are not used to limit the present invention.
[0016] Embodiment 1 As Figure 1 shown, a high-performance point cloud distance sampling simulation method includes: First, obtain depth sampling data from the camera rendering process of the game engine. Taking a resolution of 1920 * 1080 as an example, the depth sampling data consists of 2,073,600 four-dimensional floating-point arrays; When the depth texture of the camera is sampled in the shader, it returns a float4 (four-dimensional floating-point vector), where the last component usually corresponds to the depth value, that is, the depth floating-point number, and the remaining components may be used for alignment format or left as null values. A four-dimensional floating-point array refers to the data structure when the depth of each pixel is carried in the RGBA format, and those floating-point values between 0 and 1 represent the pixel depth in the depth texture.
[0017] Introduce a shader for linear depth processing. The purpose of this shader is to perform linear depth processing on the depth sampling data of the camera to improve the accuracy of depth information. In the linear depth processing shader, declare the depth map sampling data and assign the depth sampling data of the camera to this depth map sampling data. In the shader, it is declared that the depth test (ZTestAlways) is always enabled to ensure that the depth information is always valid; In the fragment shader function (frag function) of the shader, sample the depth map sampling data to obtain the depth data z of each point on the screen; Define the depth buffer parameters (bufferParams) based on the near plane and far plane of the camera for subsequent linear depth calculation. Specifically: bufferParams = float4((1.0 - far / near), (far / near), (x / far), (y / far)); Where far is the far plane of the camera and near is the near plane of the camera. In the graphics engine, the four components of the float4 four-dimensional array are generally represented by x, y, z, and w respectively, that is, x is (1.0 - far / near), y is (far / near), z is (x / far), and w is (y / far).
[0018] In the fragment shader function, calculate the linear depth data depth through bufferParams. The calculation method of the linear depth data depth is as follows: depth = 1 / (bufferParams.x * z + bufferParams.y); Through shader rendering, obtain an image of the linear depth data, whose resolution is the same as the screen resolution. Any point on the screen can obtain the corresponding object distance data by taking its linear depth data depth; Take the values of the obtained linear depth data one by one and perform distance conversion to generate point cloud data. This step can map the depth data to the actual distance by setting the distance (L) of the camera far plane.
[0019] Now take the Unity engine as an example for illustration: Such as Figure 2 AndFigure 3 As shown, add a drone in the three-dimensional terrain, mount a camera on the drone, and set the depth rendering material. The position of the camera is (-77, 3, 276). Place a cube on the ground. For the convenience of calculation, place it at the center of the camera's view. Its center position is (-192, -21, 299). The camera uses a distance simulation camera for sampling; The straight-line distance L from the center of the camera to the center of the cube is: ; L ≈ 119.7 meters; Set the near plane distance near of the camera to 1 meter and the far plane distance far to 300 meters. Then: bufferParams = float4(-299, 300, -0.9967, 1); At this time, the depth data read from the engine at the position of the cube is: z = 0.995; The converted linear depth data is: Z_linear = 1 / (bufferParams.x * z + bufferParams.y); Furthermore: Z_linear = 1 / ((-299) * z + 300) = 0.399; Sampling distance = Z_linear * far plane distance = 0.399 * 300 = 119.7 meters. The final calculation result meets the expectation.
[0020] The above steps constitute the main process of this application. By optimizing the graphics processing process of the game engine and making full use of the depth test process of the GPU, high-performance detection of the distance of objects within the visible range of the camera is achieved.
[0021] In the rendering process of this application, depth data is obtained through the camera, and a shader for linear depth processing is introduced to perform linear depth processing on the camera depth data to improve the accuracy of depth information. In the linear depth processing shader, sampling of the depth map is performed. By introducing predefined depth buffer parameters, the sampled values of the depth map are converted, as Figure 4 shown, to obtain the floating-point raw depth data. Using the depth buffer parameters, linear depth calculation is performed in the fragment shader function. The purpose of this calculation is to map the depth data to the actual linear depth. Through the rendering process, linear depth data is obtained, as Figure 5As shown, its resolution is consistent with the screen resolution. Any point on the screen corresponds to a linear depth data, which represents the linear relative distance between the object surface and the camera. The linear depth data obtained is sampled point by point, and the distance conversion is performed. This step maps the linear depth data into the actual distance through the setting of the camera far plane distance, and finally generates the point cloud data.
[0022] Compared with the traditional technology, in this application, by introducing a linear depth processing shader in the game engine and using the depth buffer parameter, the high-performance detection of the object distance within the visible range of the camera is realized. Compared with the traditional ray collision detection method, this optimization greatly reduces the calculation overhead and improves the simulation efficiency. After sampling the depth map in the shader, the depth buffer parameter is introduced to realize the linear relationship processing of the depth data, which increases the accuracy of the depth information. By performing distance conversion on the obtained relative linear depth data, the relative depth is mapped into the actual distance, and finally the point cloud data is generated. This step is the key to realizing the actual distance data and provides high-quality simulation data for the autonomous driving system.
Claims
1. A high-performance point cloud distance sampling simulation method, characterized in that, It includes the following steps: S1: Obtain depth sampling data from the camera rendering process of the game engine. The depth sampling data represents the distance from the camera to each pixel point in the rendered scene on the screen, and the depth sampling data consists of multiple four-dimensional floating-point arrays; S2: Introduce a shader for linear depth processing in the game engine, declare the depth map sampling data of the shader, and assign the depth sampling data to the depth map sampling data; S3: In the fragment shader function of the shader, sample the depth map sampling data to obtain the depth data of each point on the screen. The depth data is a component of a four-dimensional floating-point array; S4: Define the depth buffer parameter bufferParams based on the near plane and far plane of the camera; S5: In the fragment shader function, calculate the linear depth data through bufferParams, and obtain an image of the linear depth data through shader rendering. Its resolution is the same as the screen resolution, and the distance data of the corresponding object can be obtained for any point on the screen through the depth data; S6: Take values point by point for the obtained linear depth data, perform distance conversion, and map the linear depth data to the actual distance by setting the distance of the camera far plane to generate point cloud data.
2. A high-performance point cloud distance sampling simulation method according to claim 1, characterized in that, The game engine uses the Unity engine.
3. A high-performance point cloud distance sampling simulation method according to claim 1, characterized in that, The floating-point value range of the depth data in the four-dimensional floating-point array is between 0 and 1.
4. A high-performance point cloud distance sampling simulation method according to claim 1, characterized in that In S2, depth testing is declared to be always enabled in the shader to ensure that the depth information is always valid.
5. A high-performance point cloud distance sampling simulation method according to claim 3, characterized in that, In S4, the depth buffer parameter bufferParams is expressed as: bufferParams = float4((1.0 - far / near), (far / near), (x / far), (y / far)); where far is the camera far plane, near is the camera near plane, float4 is a four-dimensional floating-point array, and the four components in the four-dimensional floating-point array are represented by x, y, z, and w respectively. Among them, x is (1.0 - far / near), y is (far / near), z is (x / far), and w is (y / far).
6. A high-performance point cloud distance sampling simulation method according to claim 5, characterized in that, The specific calculation method of the linear depth data depth in S5 is as follows: depth = 1 / (bufferParams.x * z + bufferParams.y).