Method and system for accelerating acquisition of point cloud data in simulation scene

Abstract

The invention relates to the technical field of point cloud data acquisition, and provides a method for accelerating acquisition of point cloud data in a simulation scene, which comprises the following steps: S1, setting main acquisition equipment for acquiring point cloud data of three-dimensional space information in the simulation scene, according to the simulation parameter information of the main acquisition device, setting an auxiliary acquisition device having the same visual field range as the main acquisition device; s2, in the simulation process of the simulation scene, according to the sampling parameter information of the main acquisition equipment, acquiring point cloud data with the same sampling interval as the main acquisition equipment from the sampling data of the auxiliary acquisition equipment, and mapping the point cloud data acquired by the auxiliary acquisition equipment into the point cloud data of the main acquisition equipment; and S3, calculating three-dimensional coordinates of the point cloud data in the world space coordinates, and calculating other information of the point cloud data. According to the technical scheme, the acquisition process of the point cloud data in the simulation scene is accelerated in a mode of combining two kinds of point cloud acquisition equipment, and the acquisition efficiency of the point cloud data in the simulation scene is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud data acquisition, and in particular to a method and system for accelerating the acquisition of point cloud data in a simulation scenario. Background Art

[0002] In the existing methods of acquiring point cloud data in simulation scenarios, point cloud data is generally acquired by only one single method, such as acquiring point cloud data by using a single laser radar method.

[0003] LiDAR is widely used in intelligent body perception modules, especially in the field of autonomous driving. It acquires three-dimensional spatial information of the environment by emitting laser pulses and receiving echoes. In simulation testing, in order to simulate the actual working process of LiDAR, it is usually necessary to simulate LiDAR.

[0004] Most existing LiDAR simulation methods are executed on the CPU. The specific process is to emit a large number of rays within a certain range of space according to the position, attitude, number of lines, detection distance, horizontal field of view angle, vertical field of view angle, rotation speed, point frequency and other parameters of the LiDAR. These rays perform ray detection with the surrounding environment, record the collision position with the object, and then generate LiDAR data. The point cloud data output by the LiDAR includes the position of the collision point and the echo intensity, where the echo intensity is attenuated according to the distance from the collision point to the LiDAR.

[0005] Since LiDAR can emit tens of thousands to millions of pulses per second, traditional CPU calculation methods need to perform the same number of large-scale ray detection. Ray detection requires complex geometric intersection tests in three-dimensional space, which requires huge calculations and results in low operating efficiency. At the same time, when calculating the echo intensity, the existing methods only consider distance attenuation, and do not consider factors such as the reflectivity of the object surface and the angle of incidence, resulting in inaccurate intensity information and unable to truly reflect the impact of the object material on the laser. Summary of the invention

[0006] In view of the above problems, the purpose of the present invention is to provide a method and system for accelerating the acquisition of point cloud data in simulation scenarios, which accelerates the acquisition process of point cloud data in simulation scenarios by combining two point cloud acquisition devices, and at the same time utilizes the parallel computing capability of GPU to significantly improve the simulation efficiency and increase the accuracy of echo intensity calculation.

[0007] The above-mentioned object of the present invention is achieved through the following technical solutions:

[0008] A method for accelerating acquisition of point cloud data in a simulation scenario comprises the following steps:

[0009] S1: Setting a main acquisition device for acquiring point cloud data of three-dimensional spatial information in a simulation scene, and setting an auxiliary acquisition device having the same field of view as the main acquisition device according to simulation parameter information of the main acquisition device;

[0010] S2: During the simulation process of the simulation scene, the point cloud data with the same sampling interval as that of the main acquisition device is acquired from the sampling data of the auxiliary acquisition device according to the sampling parameter information of the main acquisition device, and the point cloud data acquired by the auxiliary acquisition device is mapped to the point cloud data of the main acquisition device;

[0011] S3: Calculate the three-dimensional coordinates of the point cloud data in the world space coordinates, and calculate other information of the point cloud data.

[0012] Further, in step S1, a main acquisition device for acquiring point cloud data of three-dimensional space information in a simulation scene is set, and an auxiliary acquisition device having the same field of view as the main acquisition device is set according to simulation parameter information of the main acquisition device, specifically:

[0013] Selecting a laser radar as the main acquisition device and selecting an auxiliary camera as the auxiliary acquisition device;

[0014] The auxiliary camera having the same field of view as the laser radar is set according to the simulation parameter information of the laser radar, and the auxiliary camera is initialized according to the simulation parameter information, specifically:

[0015] According to the installation position and installation angle of the laser radar, create the corresponding auxiliary camera;

[0016] The vertical field of view angle of the auxiliary camera is set according to the vertical field of view angle of the laser radar, and the far clipping plane of the auxiliary camera is set according to the detection distance of the laser radar;

[0017] According to the rotation speed of the laser radar and the current simulation frame rate of the simulation scene, the horizontal field of view angle of each frame of the auxiliary camera is updated separately so that the auxiliary camera simulates the rotation scanning of the laser radar.

[0018] Furthermore, in step S1, it also includes:

[0019] The rendering pipeline of the auxiliary camera only runs to the depth preprocessing stage in each frame to process the depth information of objects in the simulation scene and generate the visibility buffer of the auxiliary camera;

[0020] The visibility buffer stores visibility information of an object corresponding to each pixel in the simulation scene.

[0021] Further, in step S2, during the simulation process of the simulation scene, the point cloud data with the same sampling interval as that of the main acquisition device is obtained from the sampling data of the auxiliary acquisition device according to the sampling parameter information of the main acquisition device, and the point cloud data obtained by the auxiliary acquisition device is mapped to the point cloud data of the main acquisition device, specifically:

[0022] During the simulation of each frame of the simulation scene, obtaining the visibility buffer of the auxiliary camera;

[0023] Calculating the sampling interval of the visibility buffer in the horizontal direction and the vertical direction according to the number of lines and the point frequency of the laser radar;

[0024] According to the calculated sampling interval, corresponding data points are sampled from the visibility buffer, and the relevant information of the data points including the object depth information and the visibility information are integrated and mapped to the format and requirements of the point cloud data of the lidar.

[0025] Further, according to the calculated sampling interval, corresponding data points are sampled from the visibility buffer, and relevant information of the data points including the object depth information and the visibility information are integrated and mapped to the format and requirements of the point cloud data of the laser radar, further comprising:

[0026] Parallel accelerated sampling using compute shaders running on the graphics processing unit (GPU);

[0027] At the same time, when the visibility buffer in the rendering channel is bound to the compute shader, a memory barrier is established for coordinating access to the memory occupied by the visibility buffer by different processing units to prevent data read and write conflicts;

[0028] The visibility buffer is bound to a resource slot of the compute shader to achieve correct access to the visibility buffer.

[0029] Furthermore, in step S3, the three-dimensional coordinates of the point cloud data in the world space coordinates are calculated, specifically:

[0030] Get the screen space two-dimensional coordinates of the point cloud data in the screen space, denoted as (x s ,y s ), and it is known that the width of the screen for simulating the simulation scene is W, and the height is H;

[0031] The coordinates of the point cloud data are converted from the two-dimensional screen space coordinates to the two-dimensional NDC space coordinates (x ndc,y ndc ), the formula is:

[0032]

[0033] A fixed Z value is selected, usually the near plane position of the auxiliary camera, to expand the two-dimensional NDC space coordinates to the NDC space secondary coordinates, as follows:

[0034]

[0035] Get the view matrix M v and the projection matrix M p , the view matrix M v and the projection matrix M p The inverse matrix of the product is applied to the NDC space second coordinate to obtain the world space second coordinate, the formula is:

[0036]

[0037] The world space coordinates are normalized to obtain the three-dimensional coordinates of the point cloud data in the world space coordinates. The formula is:

[0038]

[0039] Furthermore, in step S3, other information of the point cloud data is calculated, specifically:

[0040] The other information includes color information of the point cloud data, and echo intensity of the point cloud data calculated by taking into account both the reflectivity and the incident angle of the object;

[0041] The UV coordinates of the corresponding primitive are calculated by using the instance ID, primitive ID and centroid coordinates of the object stored in the visibility cache, the corresponding material information is obtained from the material lookup table using the material ID, and the material is sampled by the UV coordinates to obtain PBR parameters including reflectivity and refractive index, and finally the light source attributes are obtained and combined with the PBR parameters to perform illumination calculation to obtain the color information of the point cloud data;

[0042] The PBR illumination model is set based on the PBR parameters, the light source is set to the pulse signal of the laser radar, the self-luminescence and transparency factors of the object are ignored, the reflectivity and incident angle of the object are considered, and the echo intensity of the point cloud data is calculated.

[0043] A system for accelerating the acquisition of point cloud data in a simulation scene for executing the method for accelerating the acquisition of point cloud data in a simulation scene as described above, comprising:

[0044] An auxiliary device initialization module is used to set a main acquisition device for acquiring point cloud data of three-dimensional space information in a simulation scene, and to set an auxiliary acquisition device with the same field of view as the main acquisition device according to simulation parameter information of the main acquisition device;

[0045] A point cloud data sampling module, used for acquiring the point cloud data with the same sampling interval as that of the main acquisition device from the sampling data of the auxiliary acquisition device according to the sampling parameter information of the main acquisition device during the simulation process of the simulation scene, and mapping the point cloud data acquired by the auxiliary acquisition device to the point cloud data of the main acquisition device;

[0046] The point cloud data calculation module is used to calculate the three-dimensional coordinates of the point cloud data in the world space coordinates and calculate other information of the point cloud data.

[0047] A computer device comprises a memory and one or more processors, wherein the memory stores computer codes, and when the computer codes are executed by the one or more processors, the one or more processors execute the above method.

[0048] A computer-readable storage medium stores computer codes. When the computer codes are executed, the above method is executed.

[0049] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0050] (1) A method for accelerating the acquisition of point cloud data in a simulation scene is provided, characterized in that it includes: S1: setting a main acquisition device for acquiring point cloud data of three-dimensional spatial information in a simulation scene, and setting an auxiliary acquisition device with the same field of view as the main acquisition device according to the simulation parameter information of the main acquisition device; S2: in the simulation process of the simulation scene, acquiring the point cloud data with the same sampling interval as the main acquisition device in the sampling data of the auxiliary acquisition device according to the sampling parameter information of the main acquisition device, and mapping the point cloud data acquired by the auxiliary acquisition device to the point cloud data of the main acquisition device; S3: calculating the three-dimensional coordinates of the point cloud data in the world space coordinates, and calculating other information of the point cloud data. The above technical solution accelerates the acquisition process of point cloud data in a simulation scene by combining two point cloud acquisition devices, thereby improving the efficiency of acquiring point cloud data in a simulation scene.

[0051] (2) The visibility buffer is introduced to replace the traditional ray detection method, which avoids a large number of geometric intersection calculations and improves computational efficiency.

[0052] (3) Compute shaders are used to perform parallel sampling and calculations on the visibility buffer, giving full play to the parallel processing capabilities of the GPU and accelerating the generation of lidar point cloud data. A large number of computing tasks are transferred from the CPU to the GPU, and the parallel computing capabilities of the GPU are used to significantly reduce the simulation time and achieve real-time or near real-time lidar simulation. The CPU's computing pressure is reduced, making system resource allocation more reasonable and improving the overall system operation efficiency.

[0053] (4) In the calculation of echo intensity, the reflectivity of the object surface and the angle of incidence are taken into account, which improves the accuracy of the echo intensity information. By accurately calculating the echo intensity, more realistic and accurate point cloud data is provided, which helps to improve the performance and reliability of subsequent perception algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is an overall flow chart of the method for accelerating acquisition of point cloud data in a simulation scenario according to the present invention;

[0055] Figure 2 A detailed flow chart of the method for accelerating acquisition of point cloud data in a simulation scenario according to the present invention;

[0056] Figure 3 This is an overall structural diagram of a system for accelerating the acquisition of point cloud data in a simulation scenario according to the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0058] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0059] The technical abbreviations involved in the present invention include:

[0060] (1) LIDAR (Light Detection and Ranging)

[0061] Definition: LIDAR is a remote sensing technology that determines the distance of a target object by emitting a laser beam and measuring the time it takes for the reflected light to return. It can quickly acquire a large number of three-dimensional spatial data points, which constitute the so-called "point cloud".

[0062] Working principle: The laser radar device emits a laser pulse, which is reflected back when it encounters the surface of an object. By accurately measuring the time interval between the emission and return of the laser pulse, combined with the speed of light, the distance between the object and the laser radar can be calculated. For example, in an autonomous vehicle, the laser radar is installed on the top of the vehicle and continuously emits lasers to the surrounding environment, thereby building a three-dimensional map around the vehicle to help the vehicle perceive surrounding obstacles, other vehicles, pedestrians, etc.

[0063] (2) FOV (Field of View)

[0064] Definition: Field of view refers to the range of angles that an optical instrument (such as a camera, telescope, human eye, etc.) can observe. It describes the range of lateral and longitudinal angles that the instrument can see at a given distance.

[0065] Calculation method: It is usually divided into horizontal field of view angle and vertical field of view angle. For simple optical systems, the field of view angle can be calculated by the focal length of the lens and the sensor size. For example, for a camera lens, the horizontal field of view angle (in radians) can be approximately calculated using the formula, where is the horizontal width of the sensor and is the focal length of the lens.

[0066] (3) PBR (Physically Based Rendering)

[0067] Definition: Physically based rendering is a rendering technique that simulates the physical behavior of light in the real world, such as reflection, refraction, scattering, etc., to generate more realistic images.

[0068] Principle and process: PBR takes into account the material properties of objects (such as roughness, metalness, refractive index, etc.) and the lighting environment. During the rendering process, light starts from the light source and interacts with the surface of the object. For example, when light shines on a rough non-metallic surface, diffuse reflection occurs, and the light is reflected evenly in all directions; while for a smooth metal surface, specular reflection occurs, and the light is reflected at a certain angle. By accurately calculating the interaction of these lights and combining them with the physical properties of the material, high-quality, realistic images can be generated.

[0069] (4) GPU (Graphics Processing Unit)

[0070] Definition: A graphics processing unit is a processor that is specialized for handling graphics and image-related tasks. It has a highly parallel computing architecture that can quickly process large amounts of graphics data.

[0071] How it works: The GPU contains many small processing cores that can handle multiple pixel or vertex calculation tasks at the same time. For example, when rendering a 3D scene, the GPU can simultaneously perform operations such as coordinate transformation and lighting calculation on the vertices of thousands of triangles in the scene, and perform color filling and texture mapping on the pixels inside each triangle. Its parallel computing capabilities are much more efficient than traditional CPUs in graphics processing tasks.

[0072] (5)CPU (Central Processing Unit)

[0073] Definition: The central processing unit is the core component of a computer. It is responsible for executing the instructions of the computer program, performing data processing, logical operations and controlling the various parts of the computer.

[0074] How it works: The CPU reads instructions from memory, then decodes and executes them. It consists of a control unit, an arithmetic logic unit (ALU), and a cache. The control unit is responsible for directing the operation flow of the entire computer system, the ALU performs arithmetic and logical operations, and the cache is used to store frequently accessed data and instructions to improve the CPU's operating efficiency. For example, when running a word processing software, the CPU is responsible for processing user input (such as text entered on the keyboard), typesetting the text, spell checking, and other operations.

[0075] (6) Compute Shader

[0076] Definition: A compute shader is a program that runs on a GPU and is used to perform general computing tasks, not just traditional graphics rendering tasks. It can use the parallel computing capabilities of the GPU to accelerate various data processing operations.

[0077] Working mechanism: Compute shaders implement parallel computing by executing the same computational code simultaneously on multiple processing units of the GPU. Developers can organize the data to be processed into buffers (such as texture buffers or structured buffers), and then pass these buffers as input to the compute shader. The code in the compute shader can perform various operations on this data, such as mathematical calculations, data sorting, physical simulation, etc. For example, in a fluid simulation, the compute shader can calculate the velocity and position changes of each particle in the fluid according to the laws of physics.

[0078] (7)Visibility Buffer

[0079] Definition: The visibility buffer is a data structure used for graphics rendering, which is mainly used to store the visibility information of objects corresponding to each pixel in the scene.

[0080] How it works: During the rendering process, the graphics system needs to determine which objects are visible and which are occluded at a given viewing angle. The visibility buffer records this information. For example, it can store the object ID corresponding to each pixel, or a Boolean value indicating whether the pixel is occluded by an object. In this way, in subsequent rendering steps, such as lighting calculations or post-processing, operations can be performed only on visible objects based on the information in the visibility buffer, thereby improving rendering efficiency.

[0081] (8)Depth Buffer

[0082] Definition: The depth buffer, also known as the Z-buffer, is a buffer used for graphics rendering that stores the depth information corresponding to each pixel, that is, the distance from the camera's perspective to the surface of the object in the scene.

[0083] How it works: During the rendering process, when a new pixel is drawn, the depth value of the object corresponding to the pixel is compared with the depth value already in the depth buffer. If the depth value of the new pixel is less than (that is, closer to the camera) the value in the depth buffer, then the new pixel will be drawn and the value in the depth buffer will be updated; otherwise, the new pixel will be discarded. This ensures that in the final rendering result, only the pixels of the object in the front (closest to the camera) are displayed, thereby correctly handling the occlusion relationship between objects.

[0084] (9) CUDA (Compute Unified Device Architecture, parallel computing platform)

[0085] Definition: CUDA is a parallel computing platform and programming model launched by NVIDIA, which allows developers to use the powerful parallel computing capabilities of NVIDIA GPUs to accelerate various computing tasks.

[0086] How it works: The CUDA programming model uses a C / C++-like programming language, where developers can write code called "kernels" that run on the GPU. These kernels can be executed simultaneously on multiple processing units of the GPU, enabling highly parallel computing. For example, in deep learning training, CUDA can accelerate the forward propagation and backpropagation calculations of neural networks. The data is divided into multiple small blocks, each of which is calculated on a processing unit of the GPU, and then the results are aggregated, greatly increasing the computing speed.

[0087] (10) UV (texture coordinates)

[0088] Definition: UV is a coordinate system used to map a two-dimensional texture to a three-dimensional surface. U and V represent the horizontal and vertical coordinate axes in the texture space, respectively, and the value range is usually between 0 and 1.

[0089] How it works: When applying a texture to a 3D surface, UV coordinates are needed to determine where each pixel on the texture should be mapped to the surface of the object. For example, for a simple cube model, by specifying UV coordinates for each face, an image texture (such as a wood texture) can be accurately mapped to the six faces of the cube, making the cube look like it is made of wood.

[0090] (11) Screen space coordinates

[0091] Two-dimensional coordinates in pixels, usually with (0,0) at the upper left corner and increasing to the right and downward.

[0092] (12) World space coordinates

[0093] Real-world coordinates in three-dimensional space, representing the location of an object in the world.

[0094] (13) NDC space coordinates (normalized device coordinates)

[0095] Represents screen coordinates in the normalized range [-1,1].

[0096] The invention of the present invention is described below by means of specific embodiments:

[0097] First embodiment

[0098] like Figure 1 and Figure 2 As shown, this embodiment provides a method for accelerating the acquisition of point cloud data in a simulation scenario, comprising the following steps:

[0099] S1: Setting a main acquisition device for acquiring point cloud data of three-dimensional space information in a simulation scene, and setting an auxiliary acquisition device with the same field of view as the main acquisition device according to simulation parameter information of the main acquisition device.

[0100] In this implementation, step S1 is specifically as follows:

[0101] Selecting a laser radar as the main acquisition device and selecting an auxiliary camera as the auxiliary acquisition device;

[0102] The auxiliary camera having the same field of view as the laser radar is set according to the simulation parameter information of the laser radar, and the auxiliary camera is initialized according to the simulation parameter information, specifically:

[0103] (1) Create the corresponding auxiliary camera according to the installation position and installation angle of the laser radar. In this embodiment, the following scheme can be specifically adopted:

[0104] explain:

[0105] In systems involving LiDAR and other vision-related processing (such as simulation scenarios), the role of the auxiliary camera is to assist in obtaining data information related to LiDAR perception. LiDAR has specific installation positions and angles on vehicles, robots and other equipment, such as being installed in the center of the top of the vehicle and placed horizontally, or being installed in front of the robot's head and tilted at a certain angle. Creating a corresponding auxiliary camera means accurately setting the initial position and orientation of the auxiliary camera based on the actual three-dimensional spatial position of the LiDAR and its pointing angle (angle information of the three axes). The purpose of this is to ensure that the field of view "seen" by the auxiliary camera can match the area covered by the LiDAR as much as possible in space, laying the foundation for subsequent data collaborative processing based on the two. For example, if the LiDAR is mainly used to detect environmental information within a certain range in front of and on both sides of the vehicle, the auxiliary camera should also be set in a position that covers a similar area, and its viewing direction should be roughly consistent with the detection direction of the LiDAR.

[0106] Technical implementation:

[0107] Coordinate system alignment:

[0108] First, it is necessary to determine the three-dimensional coordinate system used by the entire system. Generally, the center of the vehicle or robot is used as the origin to establish a coordinate system that conforms to the right-hand rule (for example, the x-axis is forward, the y-axis is left, and the z-axis is upward). The installation position of the lidar has a clear coordinate value in this coordinate system, such as (x_lidar, y_lidar, z_lidar), and its angle information can be expressed by Euler angles (pitch angle, yaw angle, roll angle) or rotation matrix.

[0109] When creating the auxiliary camera, set its initial position to the same coordinate value as the lidar installation position to ensure that their starting positions in space coincide.

[0110] Direction vector calculation:

[0111] Calculate the direction vector of the laser radar in three-dimensional space based on its angle information. For example, for a laser radar with yaw, pitch, and roll angles (yaw, pitch, roll), the direction vector components in the x, y, and z directions can be obtained through trigonometric functions and rotation matrix operations.

[0112] Set the auxiliary camera to the same direction vector as the LiDAR so that the two have the same observation direction. This can be achieved by setting the camera's view matrix, which determines the camera's observation direction and position in space. In graphics programming, the view matrix is ​​usually set using API functions such as OpenGL or DirectX, with its parameters set to the direction vector and position coordinates calculated based on the LiDAR angle.

[0113] (1) The vertical field of view of the auxiliary camera is set according to the vertical field of view of the laser radar, and the far clipping plane of the auxiliary camera is set according to the detection distance of the laser radar. In this embodiment, the following schemes can be specifically adopted:

[0114] explain:

[0115] Vertical field of view setting:

[0116] The laser radar itself has an angle range that can be detected in the vertical direction, that is, the vertical field of view. The auxiliary camera is set with a corresponding vertical field of view in order to match the field of view of the two in the vertical dimension. For example, if the vertical field of view of the laser radar is from 30 degrees below the horizontal to 15 degrees above the horizontal, then the auxiliary camera should also be set to the same vertical field of view range, so that when the two work at the same time, the scene information obtained in the vertical direction is consistent in angle coverage. This helps to ensure the accuracy of the correspondence between the data in the vertical direction when converting the data obtained by the auxiliary camera (such as visibility buffer data) into a data form similar to the laser radar perception (such as point cloud data), so as to facilitate accurate environmental perception and analysis.

[0117] Far clipping plane settings:

[0118] The LiDAR has a maximum distance that it can effectively detect, which is the detection distance. The far clipping plane of the auxiliary camera is set to the detection distance of the corresponding LiDAR, which means that in the imaging model of the camera, objects beyond this distance are considered to be out of its effective observation range (similar to the fact that objects outside this distance will not have corresponding effective information in the image or data generated by the camera). This is because we hope that the data obtained by the auxiliary camera matches the detection range of the LiDAR in terms of spatial depth. For example, if the LiDAR can detect objects within 200 meters in front, the far clipping plane of the auxiliary camera is set to 200 meters, so that when processing the data of both, the analysis can be based on the same spatial depth range, such as determining whether the object is in the effective detection area of ​​common concern of the LiDAR and the auxiliary camera, and subsequent calculations based on depth information (such as converting screen space coordinates to world space coordinates, etc.), the data is matched in the depth dimension.

[0119] Technical Implementation

[0120] Vertical field of view setting:

[0121] In graphics programming, the vertical field of view angle parameter of the auxiliary camera can usually be specified directly in the camera's property settings. This parameter is generally expressed as an angle value. For example, in OpenGL, the vertical field of view angle can be specified when setting the perspective projection matrix through a function call.

[0122] Get the vertical field of view value from the lidar (assuming it is fov_vertical_lidar), and then set the auxiliary camera's vertical field of view parameter to the same value. This ensures that in the vertical direction, the auxiliary camera's field of view is consistent with the lidar.

[0123] Far clipping plane settings:

[0124] When setting the perspective projection matrix, the far clipping plane is one of the important parameters. In programming, the value of the far clipping plane can be specified through related function calls.

[0125] Get the detection range of the LiDAR (assuming it is range_lidar) and set the far clipping plane parameter of the auxiliary camera to this value. In this way, in the graphics rendering pipeline, objects that are farther away from the auxiliary camera than the LiDAR detection range will not be processed, ensuring the consistency of the spatial depth detection range between the auxiliary camera and the LiDAR.

[0126] (3) According to the rotation speed of the laser radar and the current simulation frame rate of the simulation scene, the horizontal field of view angle of each frame of the auxiliary camera is updated respectively, so that the auxiliary camera simulates the rotation scanning of the laser radar. In this embodiment, the following scheme can be specifically adopted:

[0127] explain:

[0128] LiDAR rotation scanning principle:

[0129] Some laser radars achieve all-round environmental scanning by continuously rotating. For example, common mechanical laser radars rotate around a vertical axis, continuously emitting laser pulses and receiving reflected signals during the rotation process to obtain point cloud data of the surrounding environment. The rotation speed determines the angle range it can scan per unit time.

[0130] Horizontal field of view update basis:

[0131] In the simulation environment, in order to accurately simulate the effect of the rotational scanning of the laser radar, the horizontal field of view of the auxiliary camera needs to be updated every frame. Specifically, the horizontal field of view value that should be set for each frame of the auxiliary camera is calculated based on the actual rotation speed of the laser radar (usually measured in revolutions per minute or revolutions per second) and the frame rate of the current simulation system (i.e., the number of frames generated per second). For example, if the laser radar rotates one circle per second (360 degrees), and the current simulation frame rate is 30 frames per second, then the average rotation angle of the laser radar per frame is 360 degrees / 30 = 12 degrees. At this time, the horizontal field of view of the auxiliary camera should be set to update 12 degrees per frame, so that the horizontal angle range covered by the rotational scanning of the laser radar in the frame time period can be simulated in each frame, so that the data obtained by the auxiliary camera can also correspond to the scanning effect of the laser radar in the horizontal direction, and provide suitable basic data for subsequent accurate generation of point cloud data similar to the laser radar.

[0132] Technical implementation:

[0133] Parameter acquisition and calculation:

[0134] In the system, obtain the rotation speed parameter of the lidar (assuming rotation_speed_lidar, in degrees / second) and the current simulation frame rate (assuming frame_rate, in frames / second).

[0135] The calculation formula for the horizontal field of view angle updated per frame (fov_horizontal_per_frame) is: fov_horizontal_per_frame = rotation_speed_lidar / frame_rate. For example, if the lidar rotation speed is 360 degrees / second and the simulation frame rate is 60 frames / second, the horizontal field of view angle update value per frame is 6 degrees.

[0136] Update implementation:

[0137] Before rendering each frame, the auxiliary camera's horizontal field of view angle parameters are modified in graphics programming to achieve the update. The specific implementation method depends on the graphics API used. For example, in some game engines or graphics libraries, the update operation can be completed by accessing the properties of the camera object and assigning the newly calculated horizontal field of view angle value. In this way, in each frame, the auxiliary camera's horizontal field of view angle can accurately simulate the rotation scanning angle range of the lidar during that time period.

[0138] In step S1, it also includes: the rendering pipeline of the auxiliary camera only runs to the depth preprocessing stage per frame to process the depth information of the objects in the simulation scene, and generates a visibility buffer of the auxiliary camera; wherein the visibility buffer stores the visibility information of the objects corresponding to each pixel in the simulation scene. In this embodiment, the following scheme can be specifically adopted:

[0139] Purpose of deep preprocessing stage:

[0140] In this solution, the rendering pipeline of the auxiliary camera only runs to the depth preprocessing stage per frame. This stage mainly performs preliminary processing on the depth information of objects in the scene, such as calculating the distance from the object to the camera, and generates a visibility buffer. The visibility buffer stores the object visibility information corresponding to each pixel in the scene, such as whether a pixel is blocked by an object. Obtaining this visibility buffer is the key to this solution, because it is used to perform subsequent conversions and processing related to the lidar point cloud data.

[0141] Reasons for not performing follow-up processing:

[0142] LiDAR obtains environmental information by emitting laser pulses and receiving reflected pulses. The number of its pulses is relatively limited, unlike ordinary cameras that can obtain very rich full-color image data at the pixel level. Therefore, for the visibility buffer data obtained from the auxiliary camera, there is no need to perform rasterization (converting geometry into pixels) and shading (assigning attributes such as color to pixels) like conventional rendering, because these processes are mainly aimed at generating high-quality color images. This solution focuses on using the data in the visibility buffer to simulate the perception effect of LiDAR and generate point cloud data, etc. It does not require detailed sampling and processing of attributes such as color for each pixel. It only needs to use basic information such as object visibility and depth in the visibility buffer. This can save computing resources and focus on key steps related to LiDAR data processing.

[0143] Technical implementation:

[0144] Rendering pipeline customization:

[0145] In the implementation of the graphics rendering pipeline, there are usually a series of programmable stages and fixed-function stages. Through the graphics API (such as OpenGL's Shader programming or DirectX's related programming model), the execution flow of the rendering pipeline can be customized.

[0146] Write a special rendering pipeline program so that only the depth preprocessing stage is executed when rendering each frame. This stage usually involves calculating the depth values ​​of objects in the scene and storing them in the visibility buffer. You can use the depth buffer related technology and programming interface to achieve this function. For example, in OpenGL, you can enable depth testing (glEnable(GL_DEPTH_TEST)) and set the appropriate code in the fragment shader to store the depth information in a custom visibility buffer.

[0147] Skip rasterization and shading:

[0148] In the rendering pipeline program, the rasterization and shading stages are skipped through conditional judgment or specific programming logic. This can be achieved by not calling the relevant rasterization and shading functions in the rendering pipeline process or skipping the code execution corresponding to these stages. For example, in some graphics programming models, instructions to not execute the rasterization stage can be set in the pipeline configuration, or code blocks related to shading processing can be skipped in the code through return statements, etc.

[0149] Data utilization considerations:

[0150] Since the working principle of LiDAR is based on limited pulses to obtain environmental information, it does not need to process a large amount of pixel-level color information like traditional cameras. Therefore, the data obtained from the visibility buffer is mainly used to simulate the point cloud data generation of LiDAR. In subsequent processing, the depth and object visibility information in the visibility buffer is converted into point cloud data, without the need for pixel-by-pixel sampling and processing like processing color images. This targeted processing method can reduce unnecessary calculations and improve system efficiency.

[0151] S2: During the simulation process of the simulation scene, the point cloud data with the same sampling interval as that of the main acquisition device is acquired from the sampling data of the auxiliary acquisition device according to the sampling parameter information of the main acquisition device, and the point cloud data acquired by the auxiliary acquisition device is mapped to the point cloud data of the main acquisition device.

[0152] In this embodiment, step S2 is specifically as follows:

[0153] (1) During the simulation process of each frame of the simulation scene, the visibility buffer of the auxiliary camera is obtained.

[0154] Simulation process overview: In simulation systems involving LiDAR simulation or related scene reconstruction, each frame represents the system state at a specific moment. The entire simulation process is continuous, simulating the dynamic changes of the real scene over time by advancing frame by frame.

[0155] Auxiliary camera role review: The auxiliary camera has been initialized and set up before (such as according to the installation position, angle, field of view, etc. of the lidar), and its main purpose is to assist in obtaining data information related to lidar perception. During the operation of the rendering pipeline, it will generate some intermediate data, including the visibility buffer.

[0156] The significance of obtaining the visibility buffer: The visibility buffer stores the object visibility information corresponding to each pixel in the scene, such as whether a pixel is blocked by an object. It is obtained in the simulation process of each frame because it is necessary to further process and generate point cloud data similar to that obtained by the lidar based on this visibility information and other related data in the buffer (such as depth information, etc.). It is the key intermediate data source in the entire process of converting from auxiliary camera data to simulated lidar point cloud data, and provides basic data support for subsequent sampling, mapping and other operations.

[0157] (2) Calculate the sampling interval of the visibility buffer in the horizontal and vertical directions according to the number of lines and point frequency of the laser radar.

[0158] The meaning of laser radar line number and point frequency:

[0159] Line count: The line count of a LiDAR usually refers to the number of laser beams in the vertical direction. For example, a LiDAR with 32 lines means that it will simultaneously emit and receive 32 laser beams at different angles in the vertical direction to detect environmental information. LiDARs with different line counts have different vertical resolutions. The more lines, the stronger the ability to perceive details of objects in the vertical direction.

[0160] Point frequency: Point frequency refers to the frequency at which the LiDAR emits laser pulses and receives reflected pulses per second to generate valid data points. For example, a point frequency of 100kHz (kilohertz) means that 100,000 valid data points can be generated per second. It reflects the speed and density of LiDAR data acquisition.

[0161] The purpose of calculating the sampling interval: In order to accurately map the data in the visibility buffer to the point cloud data of the LiDAR, it is necessary to determine the sampling interval in the horizontal and vertical directions of the visibility buffer according to the characteristics of the LiDAR itself (number of lines and point frequency). This is because the way and density of LiDAR data collection are different from the data distribution of the visibility buffer. By reasonably calculating the sampling interval, the data sampled from the visibility buffer can be as close as possible to the data actually collected by the LiDAR in terms of spatial distribution, thereby ensuring that the subsequently generated point cloud data can accurately simulate the perception effect of the LiDAR.

[0162] Specific method for calculating sampling interval:

[0163] In the vertical direction:

[0164] Assuming that the total number of pixel rows in the visibility buffer in the vertical direction is V_total and the number of laser radar lines is L, the calculation formula for the sampling interval I_vertical in the vertical direction is:

[0165] I_vertical=V_total / L

[0166] In the horizontal direction:

[0167] Assuming that the total number of pixel columns in the visibility buffer in the horizontal direction is H_total, the point frequency of the lidar is F, and the current simulation frame rate is R (frames / second), the calculation formula for the sampling interval I_horizontal in the horizontal direction is:

[0168] I_horizontal=H_total / (F / R)

[0169] (3) According to the calculated sampling interval, corresponding data points are sampled from the visibility buffer, and relevant information of the data points including the object depth information and the visibility information are integrated and mapped into the format and requirements of the point cloud data of the lidar.

[0170] The above technical solution also includes:

[0171] (31) Parallel accelerated sampling is performed using compute shaders running on a graphics processing unit (GPU).

[0172] Compute shader features: A compute shader is a program that runs on the GPU. It has powerful parallel computing capabilities and can process multiple data elements at the same time. Unlike traditional graphics rendering shaders (such as vertex shaders, fragment shaders, etc.), compute shaders are mainly used to perform general computing tasks, not just limited to graphics rendering related operations.

[0173] Advantages of parallel accelerated sampling: When processing the data in the visibility buffer to generate lidar point cloud data, since the visibility buffer usually contains a large amount of pixel-level data (corresponding to various objects and areas in the scene), if the traditional sequential processing method is used, it will take a lot of time and computing resources. By using the parallel computing power of the compute shader for parallel accelerated sampling, multiple data points can be sampled at the same time, greatly improving the speed of data processing. For example, on multiple processing units of the GPU, the compute shader can sample pixel data at different locations in the visibility buffer at the same time, just like multiple workers performing collection work in different places at the same time, which is extremely efficient.

[0174] Data mapping process: Mapping the data in the visibility buffer to the point cloud data of the LiDAR, specifically involves selecting the corresponding data points from the visibility buffer according to certain rules based on the previously calculated sampling interval, and integrating and converting the relevant information of these data points (such as depth information, object visibility information, etc.) to make them conform to the format and requirements of the LiDAR point cloud data. For example, LiDAR point cloud data usually represents each data point with three-dimensional coordinates (x, y, z) and possible reflection intensity information. In the mapping process, it is necessary to use the depth information in the visibility buffer to convert the pixel coordinates into three-dimensional coordinates through inverse projection and other technologies, and combine the object visibility information to determine the validity and other related attributes of each data point, thereby completing the mapping process from the visibility buffer data to the LiDAR point cloud data.

[0175] (32) When the visibility buffer in the rendering channel is bound to the compute shader, a memory barrier is established to coordinate access to the memory occupied by the visibility buffer by different processing units to prevent data read and write conflicts.

[0176] Relationship between rendering pass and visibility buffer: In the whole process of graphics rendering, rendering pass is a collection of processing steps used to gradually convert 3D scene data into the final 2D image or related data output. As an important data structure generated during the rendering process, the visibility buffer may be used in different rendering stages and processing links. In this scenario, it needs to be extracted from the rendering pass and bound to the compute shader so that it can be processed using the functions of the compute shader.

[0177] The importance of ensuring data consistency and access coordination: When multiple processing units (such as different processing cores in a GPU) access (read or write) the visibility buffer at the same time, data inconsistency can easily occur without effective coordination and management. For example, one processing unit may be reading a data point in the visibility buffer while another processing unit is writing to the data point at the same time, which will cause the read data to be inaccurate or tampered with, thus affecting subsequent calculations and data processing results.

[0178] The role of establishing a memory barrier: A memory barrier is a mechanism used to coordinate access to shared memory (here, the memory occupied by the visibility buffer) by different processing units. By establishing a memory barrier, it can be stipulated that before a certain processing unit completes a specific operation (such as a write operation) on the visibility buffer, other processing units cannot perform corresponding operations (such as a read operation) on the buffer, and vice versa. This can effectively prevent data read and write conflicts, ensure the consistency and accuracy of data when accessed by different processing units, and provide a guarantee for the subsequent accurate processing of the visibility buffer using compute shaders.

[0179] (33) Bind the visibility buffer to the resource slot of the compute shader to achieve correct access to the visibility buffer.

[0180] The concept of compute shader resource slots: compute shaders need to access various data resources during operation. In order to facilitate the management and use of these resources, the GPU provides a series of resource slots for compute shaders. These resource slots are like specific "slots". Different types of resources (such as textures, buffers, etc.) can be inserted into the corresponding resource slots so that the compute shader can accurately identify and access these resources.

[0181] The significance of binding the visibility buffer: Binding the visibility buffer to the resource slot of the compute shader is to "insert" the visibility buffer as a data resource into a specific location that the compute shader can access. The purpose of this is to allow the compute shader to correctly identify and access the data in the visibility buffer during operation, so as to process it according to the predetermined program (such as parallel accelerated sampling, data mapping, etc.). Only through correct binding can the compute shader obtain the required visibility buffer data, thereby achieving effective use of the buffer and smooth completion of subsequent data processing tasks.

[0182] S3: Calculate the three-dimensional coordinates of the point cloud data in the world space coordinates, and calculate other information of the point cloud data.

[0183] In step S3, the three-dimensional coordinates of the point cloud data in the world space coordinates need to be calculated first, specifically:

[0184] Get the screen space two-dimensional coordinates of the point cloud data in the screen space, denoted as (x s ,y s ), and it is known that the width of the screen for simulating the simulation scene is W, and the height is H;

[0185] The coordinates of the point cloud data are converted from the two-dimensional screen space coordinates to the two-dimensional NDC space coordinates (x ndc ,y ndc ), the formula is:

[0186]

[0187]

[0188] A fixed Z value is selected (for example, set to -1), usually the near plane position of the auxiliary camera, to expand the two-dimensional NDC space coordinates to NDC space secondary coordinates, as follows:

[0189]

[0190] Get the view matrix M v and the projection matrix M p , the view matrix M v and the projection matrix M p The inverse matrix of the product is applied to the NDC space second coordinate to obtain the world space second coordinate, the formula is:

[0191]

[0192] The world space coordinates are normalized to obtain the three-dimensional coordinates of the point cloud data in the world space coordinates. The formula is:

[0193]

[0194] In step S3, other information of the point cloud data is also calculated, and the other information includes color information of the point cloud data and echo intensity of the point cloud data calculated by considering the reflectivity and incident angle of the object, specifically:

[0195] (1) The UV coordinates of the corresponding primitives are calculated using the instance ID, primitive ID, and centroid coordinates of the objects stored in the visibility cache. The corresponding material information is obtained from the material lookup table using the material ID. The material is sampled using the UV coordinates to obtain PBR parameters including reflectivity and refractive index. Finally, the light source attributes are obtained and combined with the PBR parameters to perform illumination calculations to obtain the color information of the point cloud data.

[0196] Examples of specific technical solutions are as follows:

[0197] UV coordinate calculation:

[0198] Read instance ID, primitive ID, and center of gravity coordinates from the visibility buffer. This also requires data extraction through a suitable programming interface based on the storage format of the visibility buffer. For example, in some graphics engines, there may be dedicated functions for obtaining specific types of data from the buffer.

[0199] The UV coordinates of the corresponding primitives are calculated based on the information obtained. The specific implementation method depends on the geometric shape of the object and the mapping algorithm used. For simple geometric shapes (such as a cube), the primitive ID can be used to determine which face the primitive belongs to, and then the UV coordinates can be calculated through a simple linear mapping combined with the barycentric coordinates. For complex geometric shapes, more complex algorithms may be required, such as mapping algorithms based on parametric surfaces. Generally speaking, different object instances are distinguished based on the instance ID. For the primitives in each object instance, a series of geometric transformations and calculations are performed based on the primitive ID and barycentric coordinates to map their positions in three-dimensional space to the UV coordinate positions in the texture space. This may involve mathematical operations such as matrix multiplication and vector operations, and different calculation code segments may need to be written according to the specific geometric shapes.

[0200] Material information acquisition and processing:

[0201] Using the obtained material ID, the corresponding material information is obtained from the material lookup table. The material lookup table can be a predefined data structure, such as an array or a hash table, which stores the attribute data of various materials with the material ID as the index. In the programming implementation, through a simple index access operation, the corresponding material information can be obtained according to the material ID, including roughness, metalness, color and other attributes.

[0202] The acquired material is sampled by UV coordinates to obtain PBR related parameters. This may involve using texture sampling functions in graphics programming to read corresponding data points from the material texture according to the UV coordinates, and then obtain PBR parameters such as reflectivity and refractive index. For example, in OpenGL, the texture2D function (for two-dimensional textures) can be used to sample the material texture according to the UV coordinates to obtain the required parameter values.

[0203] Lighting calculation:

[0204] Get the properties of the light source, including the position, intensity, color, and other information of the light source. This may be done by defining a light source object in the scene and accessing its properties to obtain this information. For example, in a game development scenario, there may be a dedicated light source class that creates and initializes a light source object, and then accesses its properties to obtain various parameters of the light source.

[0205] Combining the acquired PBR related parameters and light source properties, the lighting calculation model is used to perform lighting calculation. Common lighting calculation models include physics-based lighting models (such as the PBR lighting model itself). In programming implementation, the corresponding code segment will be written according to the specific lighting calculation model, and the PBR parameters and light source properties will be used as input. Through a series of mathematical operations (such as simulation calculations of physical processes such as light propagation, reflection, and scattering), the color that each data point should present under the current lighting conditions is calculated, thereby obtaining the color information of the point cloud.

[0206] (2) A PBR illumination model is set based on the PBR parameters, the light source is set to the pulse signal of the laser radar, self-luminescence and transparency factors of the object are ignored, the reflectivity and incident angle of the object are considered, and the echo intensity of the point cloud data is calculated.

[0207] Examples of specific technical solutions are as follows:

[0208] Set the light source based on the PBR lighting model:

[0209] In programming implementation, the first thing to do is to explicitly set the light source as the laser radar pulse signal in the code. This may involve defining a special light source type or modifying the properties of an existing light source object to make it conform to the characteristics of the laser radar pulse signal. For example, in a program that simulates the laser radar detection environment, you can create a light source class that specifically represents the laser radar pulse signal and set its related properties (such as the emission direction, the law of intensity change with distance, etc.) to simulate the working principle of the laser radar.

[0210] Consider the key factors and calculate:

[0211] When calculating the echo intensity, the self-illumination and transparency factors of the object are ignored according to the settings. In the code implementation, this means that the self-illumination and transparency properties of the object are not included in the calculation equation when performing relevant calculations. For example, in functions or code segments related to lighting calculations, only parameters related to reflectivity and incident angle are considered, and variables related to self-illumination and transparency are not processed.

[0212] Consider the reflectivity and angle of incidence of the object for accurate calculation. The reflectivity property of the object can be obtained through the material properties (such as the reflectivity parameters of the material) obtained in the previous material information acquisition step. The angle of incidence needs to be calculated based on the emission direction of the laser radar pulse signal and the position of the object surface in the world space. In the programming implementation, it may be necessary to determine the angle of incidence through vector operations. For example, the angle of incidence is determined by calculating the angle between the emission direction vector of the laser radar pulse signal and the normal vector of the object surface. Then, according to the principles of physical optics (such as the law of reflection, etc.), combined with the reflectivity and the angle of incidence, the echo intensity is accurately calculated by writing the corresponding mathematical calculation code segment. For example, according to the law of reflection, the reflectivity and the angle of incidence are substituted into the formula using the trigonometric function relationship to calculate the received echo intensity value.

[0213] Summary: In this embodiment, the present invention is a laser radar simulation acceleration method based on visibility buffer and compute shader, which makes full use of the efficient parallel computing capability of GPU and solves the problem of large computational complexity and low efficiency of traditional CPU methods. At the same time, by accurately calculating the echo intensity, the accuracy and realism of point cloud data are improved. This method has broad application prospects and good scalability, and provides an efficient and accurate solution for the field of laser radar simulation.

[0214] The application scenarios of the present invention include:

[0215] Autonomous driving simulation test: This method can be used for simulation testing of autonomous driving systems, providing high-precision lidar data and supporting testing and verification of modules such as environmental perception, path planning, and decision control.

[0216] Robot navigation and perception: Applicable to the navigation and perception of simulated robots in complex environments, providing accurate environmental information and supporting the development and testing of robot algorithms.

[0217] Virtual Reality and Game Development: In virtual reality or game scenarios that need to simulate real environment perception, this method can be used to generate high-quality point cloud data and enhance user experience.

[0218] In addition, the present invention can also use the following alternatives:

[0219] Solution 1: Use depth buffer:

[0220] You can use the depth buffer instead of the visibility buffer to obtain the location information of the point cloud. The disadvantage is that the depth buffer can only provide depth information, but cannot obtain color and reflection intensity, which limits the richness and accuracy of the point cloud data.

[0221] Solution 2: Sampling using CUDA:

[0222] Using CUDA to sample and calculate the visibility buffer can give full play to the computing power of NVIDIA graphics cards. The disadvantage is that CUDA can only run on NVIDIA graphics cards, which limits the scope of hardware application and is not conducive to promotion and application on different devices and platforms.

[0223] Second embodiment

[0224] like Figure 3 As shown, this embodiment provides a system for accelerating the acquisition of point cloud data in a simulation scene for executing the method for accelerating the acquisition of point cloud data in a simulation scene in the first embodiment, including:

[0225] Auxiliary device initialization module 1, used to set a main acquisition device for acquiring point cloud data of three-dimensional space information in a simulation scene, and to set an auxiliary acquisition device with the same field of view as the main acquisition device according to simulation parameter information of the main acquisition device;

[0226] A point cloud data sampling module 2 is used to obtain the point cloud data with the same sampling interval as that of the main acquisition device from the sampling data of the auxiliary acquisition device according to the sampling parameter information of the main acquisition device during the simulation process of the simulation scene, and map the point cloud data obtained by the auxiliary acquisition device to the point cloud data of the main acquisition device;

[0227] The point cloud data calculation module 3 is used to calculate the three-dimensional coordinates of the point cloud data in the world space coordinates and calculate other information of the point cloud data.

[0228] A computer-readable storage medium stores a computer code. When the computer code is executed, the above method is executed. A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium. The storage medium can include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0229] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

[0230] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0231] It should be noted that the above embodiments can be freely combined as needed. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention.

Claims

1. A method for accelerating the acquisition of point cloud data in a simulation scenario, characterized in that: The following steps are involved: S1: Setting a main acquisition device for acquiring point cloud data of three-dimensional spatial information in a simulation scene, and setting an auxiliary acquisition device having the same field of view as the main acquisition device according to simulation parameter information of the main acquisition device; S2: During the simulation process of the simulation scene, the point cloud data with the same sampling interval as that of the main acquisition device is acquired from the sampling data of the auxiliary acquisition device according to the sampling parameter information of the main acquisition device, and the point cloud data acquired by the auxiliary acquisition device is mapped to the point cloud data of the main acquisition device; S3: Calculate the three-dimensional coordinates of the point cloud data in the world space coordinates, and calculate other information of the point cloud data.

2. The method for accelerating acquisition of point cloud data in a simulation scenario according to claim 1, characterized in that: In step S1, a main acquisition device for acquiring point cloud data of three-dimensional space information in a simulation scene is set, and an auxiliary acquisition device having the same field of view as the main acquisition device is set according to simulation parameter information of the main acquisition device, specifically: Selecting a laser radar as the main acquisition device and selecting an auxiliary camera as the auxiliary acquisition device; The auxiliary camera having the same field of view as the laser radar is set according to the simulation parameter information of the laser radar, and the auxiliary camera is initialized according to the simulation parameter information, specifically: According to the installation position and installation angle of the laser radar, create the corresponding auxiliary camera; The vertical field of view angle of the auxiliary camera is set according to the vertical field of view angle of the laser radar, and the far clipping plane of the auxiliary camera is set according to the detection distance of the laser radar; According to the rotation speed of the laser radar and the current simulation frame rate of the simulation scene, the horizontal field of view angle of each frame of the auxiliary camera is updated separately so that the auxiliary camera simulates the rotation scanning of the laser radar.

3. The method for accelerating acquisition of point cloud data in a simulation scenario according to claim 2, characterized in that: In step S1, it also includes: The rendering pipeline of the auxiliary camera only runs to the depth preprocessing stage in each frame to process the depth information of objects in the simulation scene and generate the visibility buffer of the auxiliary camera; The visibility buffer stores visibility information of an object corresponding to each pixel in the simulation scene.

4. The method for accelerating acquisition of point cloud data in a simulation scenario according to claim 3, characterized in that: In step S2, during the simulation process of the simulation scene, the point cloud data with the same sampling interval as that of the main acquisition device is obtained from the sampling data of the auxiliary acquisition device according to the sampling parameter information of the main acquisition device, and the point cloud data obtained by the auxiliary acquisition device is mapped to the point cloud data of the main acquisition device, specifically: During the simulation of each frame of the simulation scene, obtaining the visibility buffer of the auxiliary camera; Calculating the sampling interval of the visibility buffer in the horizontal direction and the vertical direction according to the number of lines and the point frequency of the laser radar; According to the calculated sampling interval, corresponding data points are sampled from the visibility buffer, and the relevant information of the data points including the object depth information and the visibility information are integrated and mapped to the format and requirements of the point cloud data of the lidar.

5. The method for accelerating acquisition of point cloud data in a simulation scenario according to claim 4, characterized in that: According to the calculated sampling interval, corresponding data points are sampled from the visibility buffer, and relevant information of the data points including the object depth information and the visibility information are integrated and mapped to the format and requirements of the point cloud data of the laser radar, which also includes: Parallel accelerated sampling using compute shaders running on the graphics processing unit (GPU); At the same time, when the visibility buffer in the rendering channel is bound to the compute shader, a memory barrier is established for coordinating access to the memory occupied by the visibility buffer by different processing units to prevent data read and write conflicts; The visibility buffer is bound to a resource slot of the compute shader to achieve correct access to the visibility buffer.

6. The method for accelerating acquisition of point cloud data in a simulation scenario according to claim 2, characterized in that: In step S3, the three-dimensional coordinates of the point cloud data in the world space coordinates are calculated, specifically: Get the screen space two-dimensional coordinates of the point cloud data in the screen space, denoted as (x s ,y s ), and it is known that the width of the screen for simulating the simulation scene is W, and the height is H; The coordinates of the point cloud data are converted from the two-dimensional screen space coordinates to the two-dimensional NDC space coordinates (x ndc ,y ndc ), the formula is: A fixed Z value is selected, usually the near plane position of the auxiliary camera, to expand the two-dimensional NDC space coordinates to the NDC space secondary coordinates, as follows: Get the view matrix M v and the projection matrix M p , the view matrix M v and the projection matrix M p The inverse matrix of the product is applied to the NDC space second coordinate to obtain the world space second coordinate, the formula is: The world space coordinates are normalized to obtain the three-dimensional coordinates of the point cloud data in the world space coordinates. The formula is:

7. The method for accelerating acquisition of point cloud data in a simulation scenario according to claim 3, characterized in that: In step S3, other information of the point cloud data is calculated, specifically: The other information includes color information of the point cloud data, and echo intensity of the point cloud data calculated by taking into account both the reflectivity and the incident angle of the object; The UV coordinates of the corresponding primitive are calculated by using the instance ID, primitive ID and centroid coordinates of the object stored in the visibility cache, the corresponding material information is obtained from the material lookup table using the material ID, and the material is sampled by the UV coordinates to obtain PBR parameters including reflectivity and refractive index, and finally the light source attributes are obtained and combined with the PBR parameters to perform illumination calculation to obtain the color information of the point cloud data; The PBR illumination model is set based on the PBR parameters, the light source is set to the pulse signal of the laser radar, the self-luminescence and transparency factors of the object are ignored, the reflectivity and incident angle of the object are considered, and the echo intensity of the point cloud data is calculated.

8. A system for accelerating the acquisition of point cloud data in a simulation scene for executing the method for accelerating the acquisition of point cloud data in a simulation scene according to any one of claims 1 to 7, characterized in that: include: An auxiliary device initialization module is used to set a main acquisition device for acquiring point cloud data of three-dimensional space information in a simulation scene, and to set an auxiliary acquisition device with the same field of view as the main acquisition device according to simulation parameter information of the main acquisition device; A point cloud data sampling module, used for acquiring the point cloud data with the same sampling interval as that of the main acquisition device from the sampling data of the auxiliary acquisition device according to the sampling parameter information of the main acquisition device during the simulation process of the simulation scene, and mapping the point cloud data acquired by the auxiliary acquisition device to the point cloud data of the main acquisition device; The point cloud data calculation module is used to calculate the three-dimensional coordinates of the point cloud data in the world space coordinates and calculate other information of the point cloud data.

9. A computer device comprising a memory and one or more processors, wherein the memory stores computer codes, and when the computer codes are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 7. 10 . A computer-readable storage medium storing a computer code. When the computer code is executed, the method according to claim 1 is executed.

Images