A laser radar simulation method, device, electronic device and storage medium

Abstract

The present application provides a method, apparatus, electronic device, and storage medium for laser radar simulation. The method includes: constructing a three-dimensional simulation scene, and reading parameters for laser radar simulation. The parameters for laser radar simulation include: the incident angle of each laser radar ray, then performing motion distortion processing on the incident angle of each laser radar ray to obtain the incident angle after distortion processing, and then performing laser radar simulation based on the three-dimensional simulation scene, the incident angle after distortion processing, and other parameters except the incident angle of the laser radar ray. The present application provides a method, apparatus, electronic device, and storage medium for laser radar simulation that can make laser radar simulation closer to the real situation, thereby achieving more accurate simulation of the laser radar on the simulation platform.

Description

Technical Field

[0001] The present application relates to the field of laser radar simulation technology, and in particular to a laser radar simulation method, device, electronic device and storage medium. Background Art

[0002] LiDAR simulation technology is widely used in autonomous driving, drones, and robotics. On a simulation platform, LiDAR simulation technology can provide simulated data from LiDAR sensors, which serves as input for mobile robot perception, path planning, control, and positioning and navigation.

[0003] In real scenarios, the working performance of lidar is easily affected by various factors. Therefore, how to simulate lidar more accurately on a simulation platform becomes a key issue. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, electronic device and storage medium for laser radar simulation to solve at least one of the above technical problems.

[0005] The above-mentioned invention objectives of this application are achieved through the following technical solutions:

[0006] In a first aspect, a laser radar simulation method is provided, comprising:

[0007] Construct 3D simulation scenes;

[0008] Reading parameters for laser radar simulation, wherein the parameters for laser radar simulation include: an incident angle of each laser radar ray;

[0009] Performing motion distortion processing on the incident angle of each laser radar ray to obtain a distorted incident angle;

[0010] The laser radar simulation is performed based on the three-dimensional simulation scene, the incident angle after the distortion processing, and other parameters except the incident angle of the laser radar ray.

[0011] In a possible implementation, constructing a three-dimensional simulation scene includes any of the following:

[0012] Acquiring sensor data, and constructing the three-dimensional simulation scene based on the sensor data;

[0013] Constructing the three-dimensional simulation scene through a physical engine;

[0014] The three-dimensional simulation scene includes: a three-dimensional simulation scene described by a triangular grid or a three-dimensional simulation scene described by a square grid.

[0015] In another possible implementation, the parameters used for the laser radar simulation include: field of view angle and angular resolution;

[0016] The step of reading parameters for laser radar simulation further includes:

[0017] A three-dimensional incident angle table is established based on the field of view angle and the angular resolution.

[0018] In another possible implementation, the method of reading the incident angle of the laser radar ray further includes:

[0019] Get multiple frames of measurement results;

[0020] removing noise points from the multi-frame measurement results;

[0021] Calculate the incident angle for each effective point corresponding to each ray in each frame;

[0022] Based on the incident angle corresponding to the i-th ray in each frame, the incident angle corresponding to the i-th ray is determined, where i∈[1,n], and n is the number of rays contained in each frame.

[0023] In another possible implementation, performing motion distortion processing on the incident angle of each laser radar ray includes:

[0024] Get the current pose information;

[0025] Determine a relative transformation matrix generated by motion distortion based on the current posture information;

[0026] The incident angle of each laser radar ray is subjected to motion distortion processing based on the relative transformation matrix.

[0027] In another possible implementation, performing motion distortion processing on the incident angle of each laser radar ray based on the relative transformation matrix to obtain the distorted incident angle includes:

[0028] Determine the unit direction vector of the incident angle of each laser radar ray in a specific coordinate system to obtain the unit direction vector corresponding to each ray;

[0029] Determining the unit direction vector of each ray after motion distortion based on the unit direction vector corresponding to each ray and the relative transformation matrix;

[0030] The incident angle of each ray after the distortion is determined based on the unit direction vector of each ray after the motion distortion.

[0031] In another possible implementation, performing lidar simulation based on the three-dimensional simulation scene, any distorted incident angle, and other parameters except the incident angle of the lidar ray includes:

[0032] Constructing a spherical three-dimensional simulation sub-scene based on the three-dimensional simulation scene;

[0033] performing collision detection based on the spherical three-dimensional simulation sub-scene, any one of the distorted incident angles, and other parameters except the incident angle of the laser radar ray;

[0034] Based on the collision detection results and the beam model, a lidar simulation is performed.

[0035] In another possible implementation, the other parameters besides the incident angle of the laser radar ray include: an effective detection range of the laser radar, the vertical angular resolution, the horizontal angular resolution, and the diameter resolution;

[0036] The collision detection based on the spherical three-dimensional simulation sub-scene, any incident angle after distortion processing, and other parameters except the incident angle of the laser radar ray includes:

[0037] Performing spherical element rasterization on the spherical sub-scene according to the vertical angular resolution, the horizontal angular resolution, and the diameter resolution to obtain a spherical element rasterized three-dimensional simulation sub-scene;

[0038] Collision detection is performed based on a three-dimensional simulation sub-scene rasterized with spherical elements and any incident angle after the distortion processing.

[0039] In another possible implementation, the spherical element rasterized three-dimensional simulation sub-scene includes a plurality of spherical element grid voxels corresponding to each ray; and the method further includes:

[0040] Determining index values ​​corresponding to a plurality of meta-grid voxels corresponding to each ray based on the incident angle of each ray after distortion processing, the propagation distance of each ray, the effective detection range of the laser radar, the vertical angular resolution, and the horizontal angular resolution;

[0041] A three-dimensional data group is created based on the index values ​​corresponding to the plurality of meta-grid voxels corresponding to each ray.

[0042] In another possible implementation, the spherical element rasterized three-dimensional simulation sub-scene includes a plurality of spherical element raster voxels;

[0043] The collision detection based on the three-dimensional simulation sub-scene rasterized by spherical elements and any incident angle after the distortion processing includes:

[0044] Acquiring semantic information in the 3D simulation sub-scene rasterized by the spherical element;

[0045] Performing bounding box division on obstacles in each spherical element grid voxel based on semantic information in the spherical element rasterized three-dimensional simulation sub-scene;

[0046] Determine the coordinates of any ray in the laser radar spherical coordinate system based on any distorted incident angle, the position of the laser radar in the simulation scene, and the spherical three-dimensional simulation sub-scene;

[0047] Determine a distance from any ray to the obstacle surface based on the coordinates of any ray in the laser radar spherical coordinate system and the plurality of spherical grid voxels;

[0048] Determining the size of a light spot generated by any ray to the obstacle surface based on the distance from any ray to the obstacle surface and the beam model;

[0049] Determining an incident angle of a ray after distortion processing that satisfies a first preset condition, where the incident angle of the ray after distortion processing that satisfies the first preset condition is the incident angle of the ray after distortion processing corresponding to a ray centered on the direction vector of any ray and having the spot size as a radius;

[0050] Determine the coordinates of each ray that meets the first preset condition in the laser radar spherical coordinate system based on each incident angle after distortion processing that meets the first preset condition, the position of the laser radar in the simulation scene, and the three-dimensional simulation sub-scene of the sphere;

[0051] Based on the coordinates of each ray that meets the first preset condition in the laser radar spherical coordinate system and the multiple spherical grid voxels, the distance between each ray that meets the first preset condition and the obstacle surface is determined.

[0052] In another possible implementation, the determining of the incident angle after the ray distortion processing that satisfies the first preset condition further includes:

[0053] Determine the number of sampling points;

[0054] Based on the number of sampling points, a sampling range with the direction vector of any ray as the center and the light spot radius as the radius is divided;

[0055] Sampling the laser radar ray set based on each divided range to obtain a sampled laser radar ray set;

[0056] The step of determining the coordinates of each ray meeting the first preset condition in the laser radar spherical coordinate system based on each incident angle after distortion processing that meets the first preset condition, the position of the laser radar in the simulation scene, and the spherical three-dimensional simulation sub-scene includes:

[0057] Based on the incident angle of each ray after distortion processing of the sampled laser radar ray set, the position of the laser radar in the simulation scene and the three-dimensional simulation sub-scene of the sphere, the coordinates of each ray that meets the first preset condition in the laser radar spherical coordinate system are determined.

[0058] In another possible implementation, the coordinates of any one of the rays in the laser radar spherical coordinate system are (m, n);

[0059] The determining, based on the coordinates of the any ray in the laser radar spherical coordinate system and the plurality of spherical grid voxels, a distance from the any ray to the obstacle surface includes:

[0060] From k=k min to k=k max Traversing the spherical element grid voxels with index (m, n, k) in the three-dimensional data group, and performing bounding box detection on the traversed spherical element grid voxels;

[0061] If an intersection exists, the traversal is stopped, and the distance from any ray to the obstacle surface is calculated based on ray detection.

[0062] In another possible implementation, the collision detection result of any ray includes: the distance from any ray to the obstacle surface; the beam model includes the beam waist value and the wavelength of the beam;

[0063] The performing of laser radar simulation based on the collision detection result and the beam model includes:

[0064] Determining a spot radius based on the distance from any one of the rays to the obstacle surface, the beam waist value of the light beam, and the wavelength of the light beam;

[0065] Determine a laser radar ray set centered on any one of the ray direction vectors and having the light spot radius as a radius;

[0066] The divergence phenomenon of rays during propagation is simulated based on the laser radar ray set.

[0067] In another possible implementation, the laser radar simulation result includes: a distance value from each ray to an obstacle; and the method further includes:

[0068] In combination with the noise model, the distance value from each ray to the obstacle is processed to obtain the distance value corresponding to each ray after adding noise;

[0069] Combined with the weather model, the distance value corresponding to each ray after adding noise is processed to obtain the laser intensity corresponding to each ray under different weather types;

[0070] Based on the laser intensity corresponding to each ray under the target weather type, the echo pattern and the dropoff mechanism are simulated;

[0071] Based on the incident angle after distortion processing of each laser radar ray and the simulation results, the three-dimensional point cloud data of each laser radar ray and the normalized intensity value are determined through three-dimensional point solution processing and normalization processing.

[0072] In another possible implementation, the laser radar simulation result includes: a distance value from each ray to an obstacle;

[0073] Among them, the laser radar simulation results are processed in combination with the noise model, which also includes:

[0074] Obtaining a variance of the measured distance, and determining an expectation and a variance of a noise model based on the variance of the measured distance;

[0075] Among them, the laser radar simulation results are processed in combination with the noise model, including:

[0076] Noise is added to the distance value from each ray to the obstacle based on the expectation and variance of the noise model to obtain a distance value after the noise is added.

[0077] In another possible implementation, the weather model includes a correspondence between weather type, measurement range, and attenuation;

[0078] Combined with the weather model, the distance value corresponding to each ray after adding noise is processed to obtain the laser intensity corresponding to each ray under the target weather type, including:

[0079] Determining whether the distance value corresponding to each ray after adding the noise falls within the measurement range corresponding to the target weather type;

[0080] If there is a ray belonging to the measurement range corresponding to the target weather type, then obtaining the attenuation rate corresponding to the target weather type;

[0081] Based on the distance value corresponding to the ray meeting the measurement range and the attenuation rate corresponding to the target weather type, the laser intensity corresponding to the ray meeting the measurement range under the target weather type is determined.

[0082] In another possible implementation, based on the laser intensity corresponding to any ray under the target weather type, the echo pattern is simulated, including:

[0083] determining an echo pattern of any of the rays;

[0084] If the echo mode of any of the rays is a single echo mode, the first laser intensity and the corresponding first distance value are returned. The first laser intensity is the laser intensity corresponding to any ray under the target weather type, and the first distance value is the distance from the ray corresponding to the first laser intensity to the obstacle.

[0085] If the echo mode of any of the rays is a multi-echo mode, the first laser intensity, the first distance value, the laser intensity that meets the second preset condition, and the distance value corresponding to the laser intensity that meets the second preset condition are returned, and the laser intensity that meets the second preset condition is the first N laser intensities selected from large to small among the laser intensities corresponding to the rays that meet the first preset condition; the distance value corresponding to the laser intensity that meets the second preset condition is the first N distance values ​​selected from large to small among the distance values ​​corresponding to the rays that meet the first preset condition.

[0086] In another possible implementation, based on the laser intensity corresponding to any ray under the target weather type, the simulation is performed through the Dropoff mechanism, including:

[0087] Determining whether a second laser intensity is greater than a first intensity threshold, where the second laser intensity is the laser intensity corresponding to any ray under the target weather type, and the first intensity threshold is the laser intensity threshold for generating a dropoff phenomenon;

[0088] If it is greater than the first intensity threshold, outputting the second laser intensity and the corresponding second distance value;

[0089] If it is not greater than the first intensity threshold, calculating the attenuation value based on the second laser intensity;

[0090] If the attenuation value is greater than a second intensity threshold, the second laser intensity and the first distance value are output, and the second intensity threshold is a random value.

[0091] In another possible implementation, the simulation result includes: a third laser intensity and a third distance value, where the third laser intensity is the laser intensity corresponding to any laser radar ray, and the third distance value is the distance value of the ray corresponding to the third laser intensity;

[0092] Based on the incident angle after distortion processing of any laser radar ray, the second laser intensity, and the second distance value, three-dimensional point cloud data of any laser radar ray and the normalized intensity value are determined through three-dimensional point solution processing and normalization processing, including:

[0093] Calculate the three-dimensional point coordinates measured by any laser radar ray in the laser radar Cartesian coordinate system based on the incident angle of any laser radar ray after distortion processing, the third laser intensity, and the third distance value;

[0094] A normalized intensity value is determined based on the third laser intensity, the maximum laser intensity, and the minimum laser intensity, where the maximum laser intensity and the minimum laser intensity are the maximum and minimum laser intensity values ​​corresponding to the respective rays.

[0095] In a second aspect, a laser radar simulation device is provided, comprising:

[0096] Construction module, used to construct three-dimensional simulation scenes;

[0097] A reading module, configured to read parameters for laser radar simulation, wherein the parameters for laser radar simulation include: an incident angle of each laser radar ray;

[0098] a motion distortion processing module, configured to perform motion distortion processing on the incident angle of each laser radar ray to obtain a distorted incident angle;

[0099] The laser radar simulation module is used to perform laser radar simulation based on the three-dimensional simulation scene, the incident angle after the distortion processing, and other parameters except the incident angle of the laser radar ray.

[0100] In a possible implementation, the construction module is specifically used for any of the following when constructing a three-dimensional simulation scene:

[0101] Acquiring sensor data, and constructing the three-dimensional simulation scene based on the sensor data;

[0102] Constructing the three-dimensional simulation scene through a physical engine;

[0103] The three-dimensional simulation scene includes: a three-dimensional simulation scene described by a triangular grid or a three-dimensional simulation scene described by a square grid.

[0104] In another possible implementation, the parameters used for the laser radar simulation include: field of view angle and angular resolution;

[0105] The device further comprises: an establishment module, wherein:

[0106] The establishing module is used to establish a three-dimensional incident angle table based on the field of view angle and the angular resolution.

[0107] In another possible implementation, the device further includes: a first acquisition module, a noise point removal module, a calculation module, and a first determination module, wherein:

[0108] The first acquisition module is used to obtain multi-frame measurement results;

[0109] The noise point removal module is used to remove noise points from the multi-frame measurement results;

[0110] The calculation module is used to calculate the incident angle of the effective point corresponding to each ray in each frame;

[0111] The first determining module is configured to determine the incident angle corresponding to the i-th ray based on the incident angle corresponding to the i-th ray in each frame, where i∈[1,n], and n is the number of rays contained in each frame.

[0112] In another possible implementation, when performing motion distortion processing on the incident angle of each laser radar ray, the motion distortion processing module is specifically configured to:

[0113] Get the current pose information;

[0114] Determine a relative transformation matrix generated by motion distortion based on the current posture information;

[0115] The incident angle of each laser radar ray is subjected to motion distortion processing based on the relative transformation matrix.

[0116] In another possible implementation, when the motion distortion processing module performs motion distortion processing on the incident angle of each laser radar ray based on the relative transformation matrix to obtain the incident angle after distortion processing, it is specifically configured to:

[0117] Determine the unit direction vector of the incident angle of each laser radar ray in a specific coordinate system to obtain the unit direction vector corresponding to each ray;

[0118] Determining the unit direction vector of each ray after motion distortion based on the unit direction vector corresponding to each ray and the relative transformation matrix;

[0119] The incident angle of each ray after the distortion is determined based on the unit direction vector of each ray after the motion distortion.

[0120] In another possible implementation, when performing lidar simulation based on the three-dimensional simulation scene, any distorted incident angle, and other parameters except the incident angle of the lidar ray, the lidar simulation module is specifically configured to:

[0121] Constructing a spherical three-dimensional simulation sub-scene based on the three-dimensional simulation scene;

[0122] performing collision detection based on the spherical three-dimensional simulation sub-scene, any one of the distorted incident angles, and other parameters except the incident angle of the laser radar ray;

[0123] Based on the collision detection results and the beam model, a lidar simulation is performed.

[0124] In another possible implementation, the other parameters besides the incident angle of the laser radar ray include: an effective detection range of the laser radar, the vertical angular resolution, the horizontal angular resolution, and the diameter resolution;

[0125] The laser radar simulation module, when performing collision detection based on the spherical three-dimensional simulation sub-scene, any incident angle after distortion processing, and other parameters except the incident angle of the laser radar ray, is specifically used to:

[0126] Performing spherical element rasterization on the spherical sub-scene according to the vertical angular resolution, the horizontal angular resolution, and the diameter resolution to obtain a spherical element rasterized three-dimensional simulation sub-scene;

[0127] Collision detection is performed based on a three-dimensional simulation sub-scene rasterized with spherical elements and any incident angle after the distortion processing.

[0128] In another possible implementation, the spherical element-rasterized three-dimensional simulation sub-scene includes a plurality of element-raster voxels corresponding to each ray; the device further includes: a second determination module and a creation module, wherein,

[0129] The second determining module is configured to determine an index value corresponding to each of the plurality of meta-grid voxels corresponding to each ray based on the incident angle of each ray after distortion processing, the propagation distance of each ray, the effective detection range of the laser radar, the vertical angular resolution, and the horizontal angular resolution;

[0130] The creation module is used to create a three-dimensional data group based on the index values ​​corresponding to the multiple meta-grid voxels corresponding to each ray.

[0131] In another possible implementation, the spherical element rasterized three-dimensional simulation sub-scene includes a plurality of spherical element raster voxels;

[0132] When performing collision detection based on the three-dimensional simulation sub-scene rasterized by the spherical element and any incident angle after the distortion processing, the laser radar simulation module is specifically used to:

[0133] Acquiring semantic information in the 3D simulation sub-scene rasterized by the spherical element;

[0134] Performing bounding box division on obstacles in each spherical element grid voxel based on semantic information in the spherical element rasterized three-dimensional simulation sub-scene;

[0135] Determine the coordinates of any ray in the laser radar spherical coordinate system based on any distorted incident angle, the position of the laser radar in the simulation scene, and the spherical three-dimensional simulation sub-scene;

[0136] Determine a distance from any ray to the obstacle surface based on the coordinates of any ray in the laser radar spherical coordinate system and the plurality of spherical grid voxels;

[0137] Determining the size of a light spot generated by any ray to the obstacle surface based on the distance from any ray to the obstacle surface and the beam model;

[0138] Determining an incident angle of a ray after distortion processing that satisfies a first preset condition, where the incident angle of the ray after distortion processing that satisfies the first preset condition is the incident angle of the ray after distortion processing corresponding to a ray centered on the direction vector of any ray and having the spot size as a radius;

[0139] Determine the coordinates of each ray that meets the first preset condition in the laser radar spherical coordinate system based on each incident angle after distortion processing that meets the first preset condition, the position of the laser radar in the simulation scene, and the three-dimensional simulation sub-scene of the sphere;

[0140] Based on the coordinates of each ray that meets the first preset condition in the laser radar spherical coordinate system and the multiple spherical grid voxels, the distance between each ray that meets the first preset condition and the obstacle surface is determined.

[0141] In another possible implementation, the apparatus further includes: a third determining module, a dividing module, and a sampling module, wherein:

[0142] The third determining module is used to determine the number of sampling points;

[0143] The division module is configured to divide the sampling range centered on any ray direction vector and having the light spot radius as a radius based on the number of sampling points;

[0144] The sampling module is used to sample the laser radar ray set based on each divided range to obtain a sampled laser radar ray set;

[0145] The laser radar simulation module is specifically used to determine the coordinates of each ray that meets the first preset condition in the laser radar spherical coordinate system based on each incident angle after distortion processing that meets the first preset condition, the position of the laser radar in the simulation scene, and the spherical three-dimensional simulation sub-scene:

[0146] Based on the incident angle of each ray after distortion processing of the sampled laser radar ray set, the position of the laser radar in the simulation scene and the three-dimensional simulation sub-scene of the sphere, the coordinates of each ray that meets the first preset condition in the laser radar spherical coordinate system are determined.

[0147] In another possible implementation, the coordinates of any one of the rays in the laser radar spherical coordinate system are (m, n);

[0148] The laser radar simulation module is specifically configured to, when determining the distance from any ray to the obstacle surface based on the coordinates of any ray in the laser radar spherical coordinate system and the plurality of spherical grid voxels:

[0149] From k=k min to k=k max Traversing the spherical element grid voxels with index (m, n, k) in the three-dimensional data group, and performing bounding box detection on the traversed spherical element grid voxels;

[0150] If an intersection exists, the traversal is stopped, and the distance from any ray to the obstacle surface is calculated based on ray detection.

[0151] In another possible implementation, the collision detection result of any ray includes: the distance from any ray to the obstacle surface; the beam model includes the beam waist value and the wavelength of the beam;

[0152] When performing the laser radar simulation based on the collision detection result and the beam model, the laser radar simulation module is specifically used to:

[0153] Determining a spot radius based on the distance from any one of the rays to the obstacle surface, the beam waist value of the light beam, and the wavelength of the light beam;

[0154] Determine a laser radar ray set centered on any one of the ray direction vectors and having the light spot radius as a radius;

[0155] The divergence phenomenon of rays during propagation is simulated based on the laser radar ray set.

[0156] In another possible implementation, the laser radar simulation result includes: the distance value from each ray to the obstacle; the device further includes: a first processing module, a second processing module, a simulation module and a fourth determination module, wherein,

[0157] The first processing module is configured to process the distance value from each ray to the obstacle in combination with the noise model to obtain the distance value corresponding to each ray after adding noise;

[0158] The second processing module is used to process the distance value corresponding to each ray after adding noise in combination with the weather model to obtain the laser intensity corresponding to each ray under different weather types;

[0159] The simulation module is used to simulate the laser intensity corresponding to each ray under the target weather type through the echo mode and the dropoff mechanism;

[0160] The fourth determination module is used to determine the three-dimensional point cloud data and the normalized intensity value of each laser radar ray based on the incident angle after distortion processing and the simulation results of each laser radar ray through three-dimensional point solution processing and normalization processing.

[0161] In another possible implementation, the laser radar simulation result includes: a distance value from each ray to an obstacle;

[0162] The device further includes: a second acquisition module and a fifth determination module, wherein:

[0163] The second acquisition module is used to obtain the variance of the measured distance;

[0164] The fifth determining module is configured to determine an expectation and a variance of a noise model based on the variance of the measured distance;

[0165] The first processing module, when processing the laser radar simulation results in combination with the noise model, is specifically used to:

[0166] Noise is added to the distance value from each ray to the obstacle based on the expectation and variance of the noise model to obtain a distance value after the noise is added.

[0167] In another possible implementation, the weather model includes a correspondence between weather type, measurement range, and attenuation;

[0168] The second processing module processes the distance value corresponding to each ray after adding noise in combination with the weather model to obtain the laser intensity corresponding to each ray under the target weather type, specifically for:

[0169] Determining whether the distance value corresponding to each ray after adding the noise falls within the measurement range corresponding to the target weather type;

[0170] If there is a ray belonging to the measurement range corresponding to the target weather type, then obtaining the attenuation rate corresponding to the target weather type;

[0171] Based on the distance value corresponding to the ray meeting the measurement range and the attenuation rate corresponding to the target weather type, the laser intensity corresponding to the ray meeting the measurement range under the target weather type is determined.

[0172] In another possible implementation, the simulation module is specifically configured to:

[0173] determining an echo pattern of any of the rays;

[0174] If the echo mode of any of the rays is a single echo mode, the first laser intensity and the corresponding first distance value are returned. The first laser intensity is the laser intensity corresponding to any ray under the target weather type, and the first distance value is the distance from the ray corresponding to the first laser intensity to the obstacle.

[0175] If the echo mode of any of the rays is a multi-echo mode, the first laser intensity, the first distance value, the laser intensity that meets the second preset condition, and the distance value corresponding to the laser intensity that meets the second preset condition are returned, and the laser intensity that meets the second preset condition is the first N laser intensities selected from large to small among the laser intensities corresponding to the rays that meet the first preset condition; the distance value corresponding to the laser intensity that meets the second preset condition is the first N distance values ​​selected from large to small among the distance values ​​corresponding to the rays that meet the first preset condition.

[0176] In another possible implementation, the simulation module, when simulating based on the laser intensity corresponding to any ray under the target weather type and through the Dropoff mechanism, is specifically configured to:

[0177] Determining whether a second laser intensity is greater than a first intensity threshold, where the second laser intensity is the laser intensity corresponding to any ray under the target weather type, and the first intensity threshold is the laser intensity threshold for generating a dropoff phenomenon;

[0178] If it is greater than the first intensity threshold, outputting the second laser intensity and the corresponding second distance value;

[0179] If it is not greater than the first intensity threshold, calculating the attenuation value based on the second laser intensity;

[0180] If the attenuation value is greater than a second intensity threshold, the second laser intensity and the first distance value are output, and the second intensity threshold is a random value.

[0181] In another possible implementation, the simulation result includes: a third laser intensity and a third distance value, where the third laser intensity is the laser intensity corresponding to any laser radar ray, and the third distance value is the distance value of the ray corresponding to the third laser intensity;

[0182] The fourth determination module is specifically configured to determine the three-dimensional point cloud data of any laser radar ray and the normalized intensity value thereof through three-dimensional point solution processing and normalization processing based on the incident angle after the distortion processing of any laser radar ray, the second laser intensity, and the second distance value:

[0183] Calculate the three-dimensional point coordinates measured by any laser radar ray in the laser radar Cartesian coordinate system based on the incident angle of any laser radar ray after distortion processing, the third laser intensity, and the third distance value;

[0184] A normalized intensity value is determined based on the third laser intensity, the maximum laser intensity, and the minimum laser intensity, where the maximum laser intensity and the minimum laser intensity are the maximum and minimum laser intensity values ​​corresponding to the respective rays.

[0185] According to a third aspect, an electronic device is provided, comprising:

[0186] one or more processors;

[0187] Memory;

[0188] One or more applications, wherein one or more applications are stored in a memory and configured to be executed by one or more processors, and the one or more programs are configured to: perform operations corresponding to the method of lidar simulation shown in any possible implementation of the first aspect.

[0189] In a fourth aspect, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement a method for laser radar simulation as shown in any possible implementation method of the first aspect.

[0190] In a fifth aspect, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the lidar simulation method provided in the aforementioned optional implementation.

[0191] In summary, this application includes at least one of the following beneficial technical effects:

[0192] The present application provides a method, device, electronic device and storage medium for laser radar simulation. In the present application, a three-dimensional simulation scene is constructed and laser radar simulation parameters including the incident angle of each laser radar ray are read, and the incident angle of each laser radar ray is subjected to motion distortion processing to obtain the distorted incident angle. The laser radar simulation is performed based on the three-dimensional simulation scene, the distorted incident angle and other parameters required for the laser radar simulation. That is, when performing the laser radar simulation, the incident angle of each laser radar ray is subjected to motion distortion processing to simulate the influence of motion distortion on the laser radar ray in the real scene, so that the laser radar simulation can be made closer to the real situation, and thus a more accurate simulation of the laser radar can be achieved on the simulation platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0193] Figure 1 This is a schematic diagram of a method flow chart for laser radar simulation provided by an embodiment of the present application;

[0194] Figure 2 is a schematic diagram of the incident angle of the laser radar ray provided in an embodiment of the present application;

[0195] Figure 3 Schematic diagram of a 3D spherical simulation sub-scene provided in an embodiment of the present application;

[0196] Figure 4 This is a schematic diagram of spherical element rasterization provided by an embodiment of the present application;

[0197] Figure 5 This is a schematic diagram of adding a beam model for simulation provided in an embodiment of the present application;

[0198] Figure 6 is a schematic diagram of sampling a set of incident angles within a light spot range provided by an embodiment of the present application;

[0199] Figure 7 This is a schematic diagram of the structure of a laser radar simulation device provided in an embodiment of the present application;

[0200] Figure 8This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0201] The present application is further described in detail below with reference to the accompanying drawings.

[0202] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed, but as long as they are within the scope of the claims of the present application, they are protected by the patent law.

[0203] In real-world scenarios, lidar performance (e.g., distance measurement and intensity) is susceptible to complex test conditions (e.g., highly reflective objects, nearby obstacles, sunlight, and multiple radar beams). Therefore, replicating lidar performance in real-world scenarios on simulation platforms often presents significant challenges. Furthermore, the realism and efficiency of lidar simulation significantly impact the simulation testing and evaluation of high-standard mobile robots. For example, lidar simulation data fails to account for motion distortion, lidar parameters are susceptible to environmental factors (e.g., weather and smog), and lidar echo intensity is too low, potentially causing it to be misclassified as noise.

[0204] To address the above technical issues, the present invention provides a high-fidelity LiDAR simulation that fully considers the environmental factors in the simulation scene and the functional performance of various LiDARs in the current industry, and can provide high-fidelity LiDAR simulation data. For example, the present invention adds a motion distortion model to the LiDAR simulation, which can generate LiDAR simulation data caused by motion distortion, making the simulation data closer to the LiDAR data measured in the real scene; the present invention fully considers the environmental factors of the simulation scene, introduces a weather model and a noise model, and achieves more realistic LiDAR simulation data (measured distance, intensity); the present invention provides a simple and effective echo model that can achieve multi-echo or single-echo LiDAR simulation; the present invention provides a dropoff mechanism that can simulate the phenomenon of random noise misjudgment when the laser intensity is too low; the present invention uses real LiDAR measurement data to calibrate the parameters required for the simulation model, making the simulation results closer to the real measurement; the present invention constructs a spherical three-dimensional simulation sub-scene and stores the sub-scene in a spherical element grid to accelerate the retrieval speed of collision detection, which is faster than the retrieval speed of a general grid.

[0205] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0206] In this document, the term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document, unless otherwise specified, generally indicates an "or" relationship between the related objects.

[0207] The embodiments of the present application are described in further detail below with reference to the accompanying drawings.

[0208] The embodiment of the present application provides a method for simulating a laser radar. The method for simulating a laser radar can be performed by an electronic device, which can be a server or a terminal device. The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. The terminal device can be a smartphone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto. The terminal device and the server can be directly or indirectly connected via wired or wireless communication. The embodiment of the present application does not limit this. The electronic device in the embodiment of the present application can be a vehicle with an autonomous driving function, a mobile robot, a drone, etc.

[0209] like Figure 1 As shown, the method may include:

[0210] Step S101: construct a three-dimensional simulation scene.

[0211] A three-dimensional simulation scene refers to a realistic virtual environment generated by computer technology that has multiple senses such as sight, hearing, touch, and taste. Users can use various sensory devices through their natural skills to interact with entities in the virtual environment.

[0212] Specifically, constructing a 3D simulation scene includes: acquiring sensor data and constructing the 3D simulation scene based on the sensor data; or constructing the 3D simulation scene using a physics engine. In the embodiment of the present application, the sensor data may include: point cloud data from an inertial measurement unit, a lidar, or image information obtained by a camera.

[0213] The three-dimensional simulation scene includes: a three-dimensional simulation scene described by a triangular grid or a three-dimensional simulation scene described by a square grid.

[0214] Step S102: Read parameters for laser radar simulation.

[0215] For the embodiment of the present application, the parameters for the laser radar simulation are read from the outside; wherein, the parameters for the laser radar simulation include: the incident angle of each laser radar ray. In addition, the parameters for the laser radar simulation may also include: measurement range, echo parameters and operating frequency, etc.

[0216] It is worth noting that step S101 can be performed before step S102, or after step S102, or simultaneously with step S102, which is not limited in the present embodiment. Figure 1 This is only one possible implementation method and is not intended to limit the embodiments of the present application.

[0217] Step S103: Perform motion distortion processing on the incident angle of each laser radar ray to obtain the incident angle after distortion processing.

[0218] Since the laser radar accompanies the carrier during movement, each laser point is generated at a different reference posture, so the point cloud data obtained are not all point cloud data at the same reference posture. Especially when the laser radar scanning frequency is relatively low, the motion error of the laser frame caused by the movement of the laser radar carrier cannot be ignored.

[0219] In order to simulate the laser radar more realistically, a motion distortion model is added to the laser radar simulation in the embodiment of the present application. That is, the incident angle of each laser radar ray is processed by the motion distortion model to obtain the incident angle after distortion processing.

[0220] It is worth noting that step S101 can be performed before step S103, or after step S103, or simultaneously with step S103, which is not limited in the embodiment of the present application. Figure 1 This is only one possible implementation method and is not intended to limit the embodiments of the present application.

[0221] Step S104: perform lidar simulation based on the three-dimensional simulation scene, the incident angle after distortion processing, and other parameters except the incident angle of the lidar ray.

[0222] Furthermore, a lidar simulation is performed based on the constructed three-dimensional simulation scene, the incident angle after distortion processing, the measurement range, the echo parameters, the operating frequency, and other parameters used for lidar simulation. Specifically, the lidar simulation can be performed on a corresponding simulation platform based on the three-dimensional simulation scene, the incident angle after distortion processing, and other parameters except the incident angle of the lidar ray.

[0223] An embodiment of the present application provides a method for simulating a laser radar. In the embodiment of the present application, a three-dimensional simulation scene is constructed and laser radar simulation parameters including the incident angle of each laser radar ray are read. The incident angle of each laser radar ray is subjected to motion distortion processing to obtain the incident angle after distortion processing. The laser radar simulation is performed based on the three-dimensional simulation scene, the incident angle after distortion processing, and other parameters required for the laser radar simulation. That is, when performing the laser radar simulation, the incident angle of each laser radar ray is subjected to motion distortion processing to simulate the influence of motion distortion on the laser radar ray in the real scene, so that the laser radar simulation can be made closer to the real situation, and thus a more accurate simulation of the laser radar can be achieved on the simulation platform.

[0224] Furthermore, common techniques for constructing a 3D scene based on vision in step S101 include structure from motion (SFM) and shape from template (SFT). Common techniques for constructing a 3D scene based on multi-sensor fusion include vision fusion with lidar and an inertial measurement unit (R3live), lidar-enhanced SFM, and vision fusion with an inertial measurement unit and a global positioning system (GPS) using an extended Kalman filter (ECF). Furthermore, 3D simulation scenes can also be manually designed using physics engines, such as Blender and Unreal Engine. Furthermore, 3D simulation scenes can be constructed by adding semantic information to obstacles in the scene through data annotation or semantic segmentation, outputting a 3D simulation scene with semantic information.

[0225] Next, read the parameters used for LiDAR simulation and configure the simulated LiDAR model. First, read the LiDAR operating parameters (measurement range, echo parameters, field of view, angular resolution, number of output points, operating frequency, laser wavelength, and measurement noise) from the LiDAR manual. Also required is the LiDAR's position in the 3D simulation scene and the LiDAR ray's angle of incidence (θ, φ). This angle of incidence (θ, φ) can be calculated by combining the field of view, angular resolution, and number of output points, or measured.

[0226] It should be noted that in a real environment, the laser radar performs multi-frame measurements in a stationary state and ensures the validity of each measurement point. The incident angle of the laser radar ray read in step S102 can be the incident angle after denoising. That is, before reading the incident angle of the laser radar ray in step S102, the following steps may be performed: obtaining multi-frame measurement results; removing noise points from the multi-frame measurement results; calculating the incident angle for the valid points corresponding to each ray in each frame; and determining the incident angle corresponding to the i-th ray based on the incident angle corresponding to the i-th ray in each frame. Wherein, i∈[1,n], n is the number of rays contained in each frame.

[0227] The RANSAC (Random Sampling Consensus) algorithm is used on the multi-frame measurement results to remove noise points (outliers). The incident angle is then calculated for the valid points corresponding to each ray in each frame, and the average of the incident angles calculated over multiple frames is taken as the incident angle of the current lidar.

[0228] Specifically, the incident angle of the i-th ray is determined by the following formula (1), where:

[0229] as well as Formula (1);

[0230] Among them, (θ i ,φ i ) is the incident angle of the i-th ray obtained last, , θ i,m is the vertical incident angle of the i-th ray in the m-th frame measurement result, that is, the angle between the lidar ray and the positive z-axis, φ i,m is the horizontal incident angle of the i-th ray in the m-th frame measurement result, (x i,m， y i,m ， z i,m ) is the measurement result of the i-th ray in the m-th frame. k is the number of valid points after filtering out noise or invalid points using the RANSAC algorithm.

[0231] In addition, the measurement noise can also be calibrated using the same method as above. x , σ y , σ z is the measurement noise variance, σ d is the variance of the measurement distance noise and can be used to model the noise in the simulation model. , , It is the mean of the valid points after filtering out the noise points using the RANSAC algorithm.

[0232] in, ; ; ; in, .

[0233] Furthermore, the parameters used for the laser radar simulation include: field of view angle and angular resolution; the parameters used for the laser radar simulation are read in step S102, and then it can also include: establishing a three-dimensional incident angle table based on the field of view angle and angular resolution.

[0234] Specifically, a 3D incident angle table (m, n, k) is created based on the field of view and angular resolution. θ is the vertical incident angle, φ is the horizontal incident angle, and k is the number of output points. This table is used to store the 3D incident angles to be simulated. The simulation output can be obtained by traversing this array in a certain order, thereby generating ordered point cloud data.

[0235] Specifically, the calculation method of m and n in the dimension (m, n, k) is shown in formula (2), where,

[0236] ; Formula (2)

[0237] Among them, the vertical field of view angle range is: (θ min ,θ max ), horizontal field of view angle range: (φ min ,φ max ), where each vertical field of view angle and each horizontal field of view angle are as follows Figure 2 As shown, each adjacent vertical field of view angle differs by a horizontal angular resolution, and each adjacent vertical field of view angle differs by a vertical angular resolution. Further, Figure 2 In the figure, the vertical axis is the φ axis, and along the axis, φ is from φ min to φ max Change (that is, the value range of m), the horizontal axis to the right is the θ axis, along the axis to the right, θ changes from θ min to θ max Change (that is, the range of n). Because it is a three-dimensional data structure, that is, Figure 2 In addition to the φ axis and the θ axis, it also includes the k axis. Figure 2 There are multiple tables from near to far, that is, representing different k values, and from near to far, k is also from k min to k max For example, from near to far, the k value corresponding to the first table is k min , the k value corresponding to the farthest table is k max , Furthermore, each table represents a two-dimensional data structure, based on k from k min to k maxGet multiple tables, that is, get a three-dimensional data structure, that is, the three-dimensional data angle table mentioned above, that is, the three-dimensional data angle table is obtained by Figure 2 Store as shown.

[0238] Wherein, vertical angular resolution: σ θ， Horizontal angular resolution: σ φ ; LiDAR ray (θ i ,φ j )Index: (m j ,n i ),in, , .

[0239] Furthermore, after reading the incident angle of each laser radar ray through the above embodiment, in order to simulate the laser radar more realistically, the incident angle of each laser radar ray read is passed through a motion distortion model to obtain the incident angle after distortion processing.

[0240] Specifically, in step S103, the incident angle of each laser radar ray is subjected to motion distortion processing, which may include: obtaining current posture information; determining the relative transformation matrix generated by the motion distortion based on the current posture information; and performing motion distortion processing on the incident angle of each laser radar ray based on the relative transformation matrix. In an embodiment of the present application, other sensors, such as IMU and fisheye camera, can be used to provide positioning posture. After obtaining the positioning posture, the relative transformation matrix T can be obtained by interpolation. i We can also assume that the laser radar moves at a constant speed during the period of motion distortion, that is, multiply the current posture by an appropriately small random number to obtain a small posture offset as T i , so that according to T i The incident angle of each laser radar ray is subjected to motion distortion processing to obtain the distorted incident angle.

[0241] Specifically, the incident angle of each laser radar ray is subjected to motion distortion processing based on the relative transformation matrix to obtain the incident angle after distortion processing, which may specifically include: determining the unit direction vector of the incident angle of each laser radar ray in a specific coordinate system to obtain the unit direction vector corresponding to each ray; determining the unit direction vector of each ray after motion distortion based on the unit direction vector corresponding to each ray and the relative transformation matrix; and determining the incident angle of each ray after distortion processing based on the unit direction vector of each ray after motion distortion.

[0242] Through the above embodiment, the incident angle of the ith laser radar ray after calibration is known, which is expressed as (θ i ,φ i ), in the LiDAR Cartesian coordinate system, θ iis the angle between the lidar ray and the positive z-axis, φ i is the azimuth angle between the projection of the laser radar ray on the xy plane and the positive x-axis. The starting point of the laser radar ray is the origin of the laser radar coordinate system. The unit direction vector of the laser radar ray in the laser radar Cartesian coordinate system is (x i , y i , z i ). Angle of incidence (θ i ,φ i ) and the unit direction vector (x i , y i , z i ) is shown in formula (3). After the motion distortion is obtained, the unit direction vector of the laser radar ray is ( , , ) and the relative transformation matrix T generated by the motion distortion obtained by the above embodiment i The relationship between the two is shown in formula (4), that is, the unit direction vector of the laser radar ray after motion distortion is obtained by formula (4), and the incident angle after motion distortion processing is obtained based on formula (5).

[0243] in, , , formula (3);

[0244] in, Formula (4);

[0245] in, , , formula (5).

[0246] After obtaining the incident angle after distortion processing through the above embodiment, laser radar simulation is performed based on the three-dimensional simulation scene, the incident angle after distortion processing, and other parameters except the incident angle of the laser radar ray.

[0247] Specifically, the laser radar simulation is performed based on the three-dimensional simulation scene, any incident angle after distortion processing, and other parameters except the incident angle of the laser radar ray, which may include: step Sa (not shown in the figure), step Sb (not shown in the figure), and step Sc (not shown in the figure), wherein:

[0248] Step Sa: construct a sphere three-dimensional simulation sub-scene based on the three-dimensional simulation scene.

[0249] In order to improve the simulation speed, based on the three-dimensional simulation scene constructed in step S101, sub-scenes are constructed to reduce redundant scenes and accelerate the retrieval speed of collision detection. Since the incident angle of the laser radar ray is in the laser radar spherical coordinate system, it is more convenient to use the voxel grid to retrieve the sub-scene. Specifically, based on the three-dimensional simulation scene, with the origin of the laser radar current coordinate system {Lidar} as the origin, the laser radar effective detection range is the radius [r min , r max ], construct a spherical three-dimensional simulation sub-scene, such as Figure 3 shown.

[0250] Step Sb: performing collision detection based on the spherical three-dimensional simulation sub-scene, any incident angle after distortion processing, and other parameters except the incident angle of the laser radar ray.

[0251] Specifically, in addition to the incident angle of the laser radar ray, other parameters may also include: the effective detection range of the laser radar, the vertical angular resolution, the horizontal angular resolution, and the diameter resolution; in step Sb, collision detection is performed based on the spherical three-dimensional simulation sub-scene, any incident angle after distortion processing, and other parameters except the incident angle of the laser radar ray, which may specifically include: step Sb1 (not shown in the figure) and step Sb2 (not shown in the figure), wherein,

[0252] Step Sb1: spherical element rasterization is performed on the spherical sub-scene according to the vertical angular resolution, horizontal angular resolution and diameter resolution to obtain a 3D simulation sub-scene of spherical element rasterization. θ , horizontal angular resolution σ φ And the diameter resolution r d Rasterize the sphere sub-scene into sphere elements, such as Figure 4 As shown in the figure, since the above parameters are fixed, a three-dimensional array can be created before the simulation starts to store the sub-scenes.

[0253] Among them, the diameter resolution r d It is not a working parameter of the laser radar, but a parameter required for laser radar simulation. Furthermore, the diameter resolution r d It can be set according to the computational efficiency or computing power in the simulation. The higher the resolution, the more sub-scenes are divided, and the higher the memory usage.

[0254] Furthermore, the spherical meta-rasterized three-dimensional simulation sub-scene contains multiple meta-grid voxels corresponding to each ray; in order to quickly find the corresponding spherical meta-grid voxels later, a three-dimensional data group can be created, which is used to store each meta-grid voxel and determine the index value corresponding to each, so that the corresponding meta-grid voxel can be quickly found later based on the index value. The creation of the three-dimensional data group can specifically include: determining the index value corresponding to each of the multiple meta-grid voxels corresponding to each ray based on the incident angle of each ray after distortion processing, the propagation distance of each ray, the effective detection range of the laser radar, the vertical angular resolution, and the horizontal angular resolution; and creating the three-dimensional data group based on the index value corresponding to each of the multiple meta-grid voxels corresponding to each ray.

[0255] For example, the i-th lidar ray (θ i ,φ i ) is determined by the following formula:

[0256] ; ; ;

[0257] Where (m, n, k) is the index of the spherical grid voxel that the i-th lidar ray passes through, r is the propagation distance of the lidar ray, m, n, k are all integers, σ θ is the vertical angular resolution, σ φ is the horizontal angular resolution and r d is the diameter resolution.

[0258] In the LiDAR spherical coordinate system, the point (θ, φ, r) in the spherical grid voxel with index (m, n, k) must meet the following conditions:

[0259] 1. The range of θ is [θ i – σ θ / 2 , θ i + σ θ / 2 )

[0260] 2. The range of φ is [φ i – σ φ / 2 , φ i + σ φ / 2 )

[0261] 3. r range is [(k-1) • r d +r min , k • r d +r min )

[0262] Furthermore, after obtaining a 3D simulated scene with rasterized spherical elements through the above embodiment (step Sb1), collision detection simulation is performed. Collision detection generally has two meanings: one is physical collision detection, and the other is mathematical collision detection. In the present embodiment, collision detection refers to the collision detection between the LiDAR ray and obstacles in the simulated scene. This is a purely mathematical collision detection method, namely, determining whether objects intersect (or contain or overlap) and calculating the intersection point. In general 3D physics engines, blocks, spheres, rays / line segments are often used to replace intersection detection for complex shapes. If further accuracy is required, triangles or meshes are used for collision detection, that is, simplified bounding volumes (3D) are used instead of the original body for collision detection. The two most commonly used shapes are bounding boxes and bounding spheres, which provide the fastest collision detection speed. The bounding box detection algorithm can quickly identify obstacles that collide with LiDAR rays in the 3D simulated scene. In the embodiments of this application, a simulation scene created through 3D reconstruction or a physics engine can add semantic information to each obstacle in the scene through data annotation or semantic segmentation. This semantic information can be used to quickly establish a bounding volume for the obstacle. Based on the obstacles retrieved by the bounding box detection algorithm, ray casting is used to accurately calculate the distance from the lidar ray to the obstacle surface. Specific collision methods are detailed in the following embodiments.

[0263] Step Sb2: performing collision detection based on the three-dimensional simulation sub-scene rasterized with spherical elements and any incident angle after distortion processing.

[0264] The spherical element rasterized three-dimensional simulation sub-scene includes a plurality of spherical element raster voxels;

[0265] Specifically, collision detection is performed based on the 3D simulation sub-scene rasterized by spherical elements and any incident angle after distortion processing, which may include: step Sb21 (not shown in the figure), step Sb22 (not shown in the figure), step Sb23 (not shown in the figure), step Sb24 (not shown in the figure), step Sb25 (not shown in the figure), step Sb26 (not shown in the figure), step Sb27 (not shown in the figure) and step Sb28 (not shown in the figure), wherein,

[0266] Step Sb21: Acquire semantic information in the rasterized 3D simulation sub-scene of the spherical element.

[0267] In the embodiments of the present application, it is noted in the above embodiments that after constructing a 3D simulation sub-scene, semantic information can be added to obstacles in the simulation scene based on data annotation or semantic segmentation to obtain a 3D simulation scene with semantic information. Based on this, after obtaining a 3D simulation sub-scene rasterized with spherical elements, semantic information within the 3D simulation sub-scene rasterized with spherical elements can also be obtained.

[0268] Step Sb22 : performing bounding box division on the obstacles in each spherical element grid voxel based on the semantic information in the spherical element rasterized three-dimensional simulation sub-scene.

[0269] Bounding box is an algorithm for finding the optimal bounding space of a discrete set of points. The basic idea is to use a slightly larger geometric body with simpler characteristics (called a bounding box) to approximate complex geometric objects. Common bounding box algorithms include axis-aligned bounding box (AABB), bounding sphere, oriented bounding box (OBB), and fixed directions hull (FDH).

[0270] Step Sb23: Determine the coordinates of any ray in the laser radar spherical coordinate system based on any incident angle after distortion processing, the position of the laser radar in the simulation scene, and the spherical three-dimensional simulation sub-scene.

[0271] For the embodiment of the present application, the incident angle of any motion-distorted laser radar ray ( , ), the laser radar's position in the simulation scene and the spherical three-dimensional simulation sub-scene obtained in step Sa, determine the coordinates (m, n) of the ray in the laser radar spherical coordinate system, the index m, n can be obtained by the laser radar ray incident angle ( , )Sure.

[0272] Step Sb24: Determine the distance from any ray to the obstacle surface based on the coordinates of any ray in the laser radar spherical coordinate system and multiple spherical grid voxels.

[0273] For the embodiment of the present application, the coordinates of any ray in the laser radar spherical coordinate system are (m, n); based on the coordinates of any ray in the laser radar spherical coordinate system and a plurality of spherical grid voxels, determining the distance from any ray to the obstacle surface may specifically include: starting from k=k min to k=k maxIn the three-dimensional data set, the spherical element grid voxels with indexes (m, n, k) are traversed, and bounding box detection is performed on the traversed spherical element grid voxels; if there is an intersection, the traversal is stopped, and the distance from any ray to the obstacle surface is calculated based on ray detection. That is, in the embodiment of the present application, the indexes of the multiple spherical element grid voxels are (m, n, k), starting from k min to k=k max Traverse the spherical grid voxels with index (m, n, k) in the subscene. For example, if the lidar measurement range is 200 meters and the resolution is 1 meter, the lidar measurement range is divided into 200 segments, that is, k is [0,199], k min =0, k max =199; perform bounding box detection on the traversed spherical element grid voxels to determine whether there is an intersection. If not, skip it. If there is an intersection, stop traversal and further use ray casting to accurately calculate the distance from the lidar ray to the obstacle surface.

[0274] Furthermore, in an embodiment of the present application, the distance from any ray to the obstacle surface is determined based on the coordinates of the ray in the laser radar spherical coordinate system. If there are multiple rays, the determination method corresponding to the distance from each ray to the obstacle surface can be based on the above embodiment, or it can be traversed from small to large based on θ, and then traversed from small to large based on φ. The specific traversal method will not be repeated in the embodiment of the present application.

[0275] Step Sb25: Determine the size of the light spot generated by any ray reaching the obstacle surface based on the distance from any ray to the obstacle surface and the beam model.

[0276] Specifically, the beam model in the embodiment of the present application is shown in formula (6), that is, the spot size generated by any ray to the obstacle surface is determined based on the distance from any ray to the obstacle surface determined in step Sb24 and formula (6).

[0277] in, , formula (6); where the distance from any laser radar ray to the obstacle is the beam propagation distance m, the beam waist r0 of the Gaussian beam, and the wavelength λ, where the beam waist r0 and the wavelength λ are laser radar parameters.

[0278] Step Sb26: Determine the incident angle after the ray distortion processing that meets the first preset condition.

[0279] The incident angle after ray distortion processing that meets the first preset condition is the incident angle after distortion processing corresponding to a ray with the direction vector of any ray as the center and the spot size as the radius.

[0280] For example, the incident angle after the ray distortion processing that meets the first preset condition can be a laser radar ray set with the laser radar ray direction vector as the center and the spot radius as the radius { , In the embodiment of the present application, the laser radar ray set { , } can be stored in the three-dimensional incident angle table involved in the above embodiment, and an index corresponding to each incident angle is generated.

[0281] Step Sb27, based on each incident angle after distortion processing that meets the first preset condition, the position of the laser radar in the simulation scene and the spherical three-dimensional simulation sub-scene, determine the coordinates of each ray that meets the first preset condition in the laser radar spherical coordinate system.

[0282] In the embodiment of the present application, based on the laser radar ray set { , }, the position of the laser radar in the simulation scene, and the spherical simulation sub-scene, determine the coordinates of each incident angle in the laser radar spherical coordinate system. For details, please refer to the implementation method corresponding to step Sb3, which will not be repeated here.

[0283] Step Sb28: Based on the coordinates of each ray that meets the first preset condition in the laser radar spherical coordinate system and multiple spherical grid voxels, determine the distance between each ray that meets the first preset condition and the obstacle surface.

[0284] Furthermore, each ray satisfying the first preset condition involved in the embodiment of the present application refers to the laser radar set { , In the embodiment of the present application, the distance between each ray satisfying the first preset condition and the obstacle surface is determined in detail in the implementation method corresponding to step Sb4, which will not be repeated here.

[0285] Step Sc: perform lidar simulation based on the collision detection results and the beam model.

[0286] For the embodiment of the present application, the collision detection result of any ray includes: the distance from any ray to the surface of the obstacle; the beam model includes the beam waist value and the wavelength of the beam. In the embodiment of the present application, in order to realistically simulate the propagation process of the laser radar ray in the medium, a beam model is added to the laser radar simulation in the embodiment of the present application to simulate the spot effect generated by the divergence of the laser beam during the propagation process. The generated spot will be used for the echo mode. In order not to affect the real-time performance of the simulation, a Gaussian beam model can be used in the embodiment of the present application. In optics, a Gaussian beam is an electromagnetic wave beam whose transverse electric field and irradiance distribution approximately satisfy the Gaussian function.

[0287] Specifically, in step Sc, based on the collision detection result and the beam model, laser radar simulation is performed, which may include: step Sc1 (not shown in the figure), step Sc2 (not shown in the figure), and step Sc3 (not shown in the figure), wherein:

[0288] Step Sc1: Determine the spot radius based on the distance from any ray to the obstacle surface, the beam waist value, and the wavelength of the beam. In the embodiment of the present application, the spot radius is determined by formula (7), where:

[0289] Formula (7);

[0290] in, , m represents the propagation distance of the light beam, r represents the spot radius, r0 represents the waist of the Gaussian beam, and λ represents the wavelength.

[0291] Step Sc2: determine a set of laser radar rays centered on any ray direction vector and having a spot radius as a radius.

[0292] For the embodiment of the present application, the light spot is a light spot with a radius r and a direction vector of the laser radar ray as the center. The distance from the laser radar ray to the obstacle is the beam propagation distance m. Specifically, Figure 5 shown.

[0293] According to the size of the spot radius, we can get the laser radar ray set with the laser radar ray direction vector as the center and the spot radius as the radius .

[0294] Step Sc3: simulate the divergence of rays during propagation based on the laser radar ray set.

[0295] Specifically, the laser radar ray set obtained by the above embodiment is As a simulation of the divergence of laser radar rays during propagation. In addition, the laser radar ray set Can be used in echo mode to return multi-echo measurements. It can be used in the echo mode. Please refer to the following embodiments for details, which will not be described again here.

[0296] Furthermore, in order to improve the simulation speed and reduce the calculation cost, the incident angle set in the spot range is Sampling is performed. In the embodiment of the present application, the sampling range refers to the light spot. In order to simulate the echo mode, a laser ray may generate multiple sampling values. Therefore, the simulated ray will generate a light spot when hitting an obstacle. A range of incident angles will be generated within the light spot range (that is, the incident angles after the ray distortion processing that meets the first preset condition). In order to obtain multiple sampling values, sampling is performed within the light spot range, and the incident angles of several rays are selected. Then, the intersection of these rays with the obstacle is simulated, so that multiple values ​​can be obtained. Specifically, in step Sb26, the incident angles after the ray distortion processing that meets the first preset condition are determined, and then the following steps may be included: determining the number of sampling points; dividing the sampling range formed by taking any ray direction vector as the center and the light spot radius as the radius based on the number of sampling points; sampling the laser radar ray set based on each divided range to obtain the sampled laser radar ray set. That is, in the embodiment of the present application, the number of sampling points n determined can be set to twice the maximum number of echoes in the laser radar parameters. Secondly, in the spot, the sampling range is divided into equal intervals according to the spot center and the spot radius. The number of divided ranges is a multiple of the number of sampling points to ensure that the same number of points can be collected in each range, such as Figure 6 As shown, finally, random sampling is performed in each divided range, and the set of incident angles of all sampling points is output. Among them, the incident angle set of all sampling points is The number of laser radar rays is not greater than the set obtained in the above embodiment .

[0297] In step Sb27, based on the incident angle after each distortion processing that satisfies the first preset condition, the position of the lidar in the simulation scene, and the 3D simulation sub-scene of the sphere, the coordinates of each ray that satisfies the first preset condition in the lidar spherical coordinate system are determined. Specifically, this may include: based on the incident angle after distortion processing of each ray in the sampled lidar ray set, the position of the lidar in the simulation scene, and the 3D simulation sub-scene of the sphere, the coordinates of each ray that satisfies the first preset condition in the lidar spherical coordinate system are determined. In the embodiment of the present application, the method for determining the coordinates of each ray that satisfies the first preset condition in the lidar spherical coordinate system is detailed in the above embodiment and is not repeated here.

[0298] In another possible implementation of the embodiment of the present application, the laser radar simulation result includes: the distance value from each ray to the obstacle; the method further includes: step S1 (not shown in the figure), step Sm (not shown in the figure), step Sn (not shown in the figure), and step So (not shown in the figure), wherein steps S1 to step So can be performed after obtaining the distance value from each ray to the obstacle, for example, steps S1 to step So can be performed after step Sb, wherein,

[0299] Step S1: Combined with the noise model, the distance value from each ray to the obstacle is processed to obtain the distance value corresponding to each ray after adding noise.

[0300] To simulate the measurement noise of the LiDAR, we add noise to the distance values ​​from each ray to the obstacle obtained in the above example. Typical noise types include Gaussian noise and shot noise. It's worth noting that since the distance values ​​are used in the laser intensity calculation, there's no need to add an additional noise model to the intensity values.

[0301] Processing the LiDAR simulation results in conjunction with the noise model may also include obtaining the variance of the measured distance and determining the expected and variance of the noise model based on the variance of the measured distance. In the embodiment of the present application, the variance of the measured distance is a parameter read for LiDAR simulation in the above embodiment, and the expected μ and variance σ of the noise model are calculated using this parameter.

[0302] After obtaining the expected μ and variance σ of the noise model, the laser radar simulation results are processed in combination with the noise model. Specifically, this may include: adding noise to the distance value from each ray to the obstacle based on the expected and variance of the noise model to obtain the distance value after adding noise. In the embodiment of the present application, the distance value after adding noise is determined by the following formula (8), where:

[0303] Formula (8).

[0304] in, It is used to represent the distance value after adding noise, z is used to represent the distance value from each ray to the obstacle, and n is used to represent the noise.

[0305] Furthermore, LiDAR performance is also dependent on weather. For example, in fog, snow, rain, and other weather conditions, the LiDAR's measurement range will be affected. Weather also affects the attenuation rate of light in air, significantly impacting laser intensity calculations. Therefore, a weather model is used to simulate the impact of different weather types on LiDAR performance. See the following embodiment for details. Step Sm: Based on the weather model, the distance values ​​corresponding to each ray after adding noise are processed to obtain the laser intensity corresponding to each ray under the target weather type.

[0306] Specifically, the weather model stores the correspondence between weather types, measurement ranges, and attenuation degrees. In the embodiment of the present application, the measurement ranges and attenuation degrees corresponding to the four weather types of rain, snow, fog, and clear are given, as shown in Table 1.

[0307] Table 1

[0308] weather Measuring range (m) Attenuation rate (1 / m) rain 110 0.032 Snow 80 0.0412 fog 50m 0.005 clear 120m 0.0001

[0309] Specifically, after obtaining the measurement range and attenuation corresponding to each weather type, the distance value corresponding to each ray after adding noise is processed in combination with the weather model to obtain the laser intensity corresponding to each ray under the target weather type. Specifically, it may include: determining whether the distance value corresponding to each ray after adding noise belongs to the measurement range corresponding to the target weather type; if there is a ray belonging to the measurement range corresponding to the target weather type, obtaining the attenuation rate corresponding to the target weather type; based on the distance value corresponding to the ray that meets the measurement range and the attenuation rate corresponding to the target weather type, determining the laser intensity corresponding to the ray that meets the measurement range under the target weather type.

[0310] Specifically, based on the distance value corresponding to the ray that meets the measurement range and the attenuation corresponding to the target weather type, the laser intensity corresponding to the ray that meets the measurement range under the target weather type is determined by the following formula (9).

[0311] Formula (9)

[0312] Among them, E0 is the pulse energy emitted by the laser, is the attenuation rate of the laser during propagation, is the distance value after adding noise. The pulse energy E0 can be obtained from the lidar operating parameters read above, or can be roughly calculated using E = hc / λ, where h is Planck's constant, c is the speed of light in a vacuum, and λ is the laser wavelength (usually obtained from the lidar data sheet).

[0313] In an embodiment of the present application, the target weather type can be set by the user or randomly selected by the simulation platform. For example, the target weather type can be rain, which is not limited in the embodiment of the present application.

[0314] For example, when the target weather type is rain, the measurement range and attenuation are 110m and 0.032 1 / m respectively. For another example, =100m, E0=1, that is, not greater than 110m, then the attenuation of the weather type is determined to be rain, which is 0.032, and then based on the distance value after adding noise And the attenuation is 0.032, and the laser intensity corresponding to the ray in the rainy weather environment is determined to be: Intensity=e -3.2 .

[0315] Furthermore, if the distance value after adding noise obtained in the above embodiment is larger than the measurement range corresponding to the target type, for example, larger than the measurement range corresponding to the weather type of rain, it is ignored.

[0316] It is worth noting that each ray can be simulated in the above manner when combining the weather type to determine the corresponding laser intensity under the target weather type, which is not limited in the embodiments of the present application.

[0317] Furthermore, after simulating the laser intensity for each ray under the target weather type, an echo pattern and a dropoff mechanism can be simulated based on the laser intensity corresponding to each ray under the target weather type. For details, see the following embodiment. Step Sn: Based on the laser intensity corresponding to each ray under the target weather type, simulation is performed using an echo pattern and a dropoff mechanism.

[0318] Echo modes include Strongest Return, Last Return, Single Return, and Dual Return. When set to Dual Return mode, target detail is enhanced, with twice the data volume compared to a single return. Dual returns are only generated when the distance between two objects is greater than 1 meter. Due to beam divergence, any single laser emission can potentially generate multiple laser returns. As the laser pulse is emitted, the spot size gradually increases. Assuming a large enough spot size, it can strike multiple targets, generating multiple reflections. Generally, the farther away the target is, the weaker its energy is at the receiver. Bright or reflective surfaces may experience the opposite effect, so simulation of the echo mode is necessary.

[0319] Furthermore, the Dropoff mechanism simulates the possibility that when laser intensity is too low, the return value may be mistakenly interpreted as measurement noise and filtered out. When the laser intensity falls below a certain threshold, it may be too low, which may be due to measurement noise or too low intensity due to a long propagation distance. Therefore, there is a possibility of being filtered out or returning a normal value. Therefore, the Dropoff mechanism is used to implement this random possibility to reduce the phenomenon of false positives caused by lidar noise.

[0320] Based on the effects of the above-mentioned echo mode and Dropoff mechanism, the following embodiment introduces a specific implementation method of laser radar simulation through the echo mode and Dropoff mechanism.

[0321] Specifically, based on the laser intensity corresponding to any ray under the target weather type and simulation through the echo mode, it can specifically include: determining the echo mode of any ray; if the echo mode of any ray is a single echo mode, returning the first laser intensity and the corresponding first distance value; if the echo mode of any ray is a multi-echo mode, returning the first laser intensity, the first distance value, the laser intensity that meets the second preset condition, and the distance value corresponding to the laser intensity that meets the second preset condition.

[0322] Specifically, the parameters read for lidar simulation in the above embodiment may include the echo pattern corresponding to each laser ray. The first laser intensity is the laser intensity corresponding to any ray under the target weather type, and the first distance value is the distance value from the ray corresponding to the first laser intensity to the obstacle. The laser intensity that meets the second preset condition is the first N laser intensities selected from the laser intensities corresponding to the rays that meet the first preset condition, in descending order; the distance value corresponding to the laser intensity that meets the second preset condition is the first N distance values ​​selected from the distance values ​​corresponding to the rays that meet the first preset condition, in descending order.

[0323] The general echo mode is selected according to the intensity value. If it is a single echo mode, the ray is directly returned ( , ) measurement value, if it is multi-echo mode, in addition to returning the measurement value of the ray, according to the laser radar ray set { , } Select the corresponding number of measurement values ​​with larger intensity values ​​as other echoes. For example, in dual-echo mode, a ray needs to return two measurement values. In addition to the measurement result of the ray, another measurement result with the largest intensity needs to be returned. According to the light beam model, a ray hits an obstacle to produce a certain light spot, and then a ray set can be obtained with this light spot as the radius. Through sampling, a smaller number of ray sets can be further obtained. Then, all rays in the ray set (that is, the incident angle set, the incident angle is the direction vector of the ray, representing a ray, and the rays will not intersect) are subjected to collision detection and intensity value calculation, and the measurement result with the largest intensity value in this ray set is selected as the second measurement value. Three echoes are returned. Generally, lidar only has single echo and dual echo modes.

[0324] It is worth noting that if echo pattern simulation is performed on multiple laser radar rays, the echo pattern simulation can be performed on any laser radar ray in the above manner to obtain the echo pattern simulation results corresponding to each ray.

[0325] Furthermore, based on the laser intensity corresponding to any ray under the target weather type, and through the Dropoff mechanism, simulation can specifically include: determining whether the second laser intensity is greater than the first intensity threshold; if it is greater than the first intensity threshold, outputting the second laser intensity and the corresponding second distance value; if it is not greater than the first intensity threshold, calculating the attenuation value based on the second laser intensity; if the attenuation value is greater than the second intensity threshold, outputting the second laser intensity and the first distance value.

[0326] Wherein, the second laser intensity is the laser intensity corresponding to any ray under the target weather type, the first intensity threshold is the laser intensity threshold that produces the Dropoff phenomenon; and the second intensity threshold is a random value. In the embodiment of the present application, the first intensity threshold can be represented by D1. That is, after obtaining the laser intensity of a ray under the target weather type through the above embodiment, it is determined whether the laser intensity is greater than D1. If it is greater than D1, the ray intensity value and distance value are considered not to be noise and cannot be ignored, and the intensity value is output. If the ray intensity value is not greater than D1, the Dropoff is calculated based on the laser intensity value of the ray under the target weather type and is calculated using formula (10). If the Dropoff value is greater than the Random value (that is, the second intensity threshold mentioned above), the ray intensity value and distance value are considered not to be noise and cannot be ignored, and the intensity value and distance value are output. Otherwise, the measurement result of the ray is ignored.

[0327] Formula (10);

[0328] Among them, Do is the probability of getting the return value when Intensity is 0. When the intensity value is 0, for example, D o is 70%, meaning there's a 70% probability of obtaining a distance measurement with an intensity value of 0. This value is generally empirically determined. If it's less than 50%, typical lidars have internal filtering that may filter out measurements with an intensity value of 0. Dl is the intensity threshold for dropoff, where intensity is the intensity of the laser beam in the target's weather conditions. For example, D1 can be determined empirically by the user. Alternatively, D1 can be the intensity resolution. For example, if the intensity resolution is 0.1, then D1 can be 0.1.

[0329] In order to convert the distance measured by the lidar into the coordinates of a point in the lidar coordinate system, and because the measurement ranges of different lidar models are different, the ranges of intensity values ​​in the simulation model may be different. In order to convert the distance into coordinates and unify the measurement of intensity values, three-dimensional point solution processing and normalization processing are performed. For details, see step So and the corresponding implementation method.

[0330] Step So, based on the incident angle after distortion processing of each laser radar ray and the simulation results, three-dimensional point solution processing and normalization processing are performed to determine the three-dimensional point cloud data of each laser radar ray and the intensity value after normalization processing.

[0331] Specifically, the simulation results include: a third laser intensity and a third distance value, where the third laser intensity is the laser intensity corresponding to any laser radar ray, and the third distance value is the distance value of the ray corresponding to the third laser intensity.

[0332] Specifically, based on the incident angle after distortion processing, the second laser intensity, and the second distance value of any laser radar ray, three-dimensional point cloud data and the normalized intensity value of any laser radar ray are determined through three-dimensional point solution processing and normalization processing. Specifically, this may include: calculating the three-dimensional point coordinates measured by any laser radar ray in the laser radar Cartesian coordinate system based on the incident angle after distortion processing, the third laser intensity, and the third distance value of any laser radar ray; and determining the normalized intensity value based on the third laser intensity, the maximum laser intensity, and the minimum laser intensity. In this embodiment of the present application, the maximum laser intensity and the minimum laser intensity are the maximum and minimum values ​​of the laser intensity values ​​corresponding to each ray, respectively.

[0333] It is worth noting that the step of calculating the three-dimensional point coordinates measured by any laser radar ray in the laser radar Cartesian coordinate system based on the incident angle, the third laser intensity and the third distance value after the distortion processing of any laser radar ray can be performed before the step of determining the normalized intensity value based on the third laser intensity, the maximum laser intensity and the minimum laser intensity, or after the step of determining the normalized intensity value based on the third laser intensity, the maximum laser intensity and the minimum laser intensity, or can be performed simultaneously with the step of determining the normalized intensity value based on the third laser intensity, the maximum laser intensity and the minimum laser intensity, which is not limited in the embodiments of the present application.

[0334] Specifically, based on the incident angle, the third laser intensity, and the third distance value after the distortion processing of any laser radar ray, the coordinates of the three-dimensional point measured by any laser radar ray in the laser radar Cartesian coordinate system are calculated. Specifically, the coordinates may include: according to the incident angle and the measured distance value corresponding to the laser radar ray, the three-dimensional point (x l,i， y l,i， z l,i ).

[0335] ; ; ;Formula (11)

[0336] Among them, (x l,i， y l,i， z l,i ) is the three-dimensional point measured by the laser radar ray in the laser radar Cartesian coordinate system, ( , ) is the incident angle of the motion-distorted lidar ray (i.e., the incident angle of any lidar ray after distortion processing), k iis the distance measurement value of the laser radar ray obtained in the above embodiment.

[0337] Specifically, based on the third laser intensity, the maximum laser intensity, and the minimum laser intensity, the normalized intensity value is determined, which may include: calculating the intensity values ​​of all rays, and normalizing the intensity value of each ray based on formula (12), generally in the range of [0, 255].

[0338] in,

[0339] Formula (12);

[0340] Among them, I max is the maximum value among all intensities, I min is the minimum value among all intensities, I g is the normalized intensity value, and I is the Intensity value of the above embodiment.

[0341] Furthermore, through the above embodiment, the three-dimensional point cloud corresponding to the ray and the corresponding intensity value can be obtained.

[0342] Furthermore, for each incident angle in the three-dimensional incident angle group established in the above embodiment, the ray corresponding to each incident angle can be calculated based on the incident angle after distortion processing of any laser radar ray, the third laser intensity and the third distance value, and the three-dimensional point coordinates measured by any laser radar ray in the laser radar Cartesian coordinate system can be calculated; based on the third laser intensity, the maximum laser intensity and the minimum laser intensity, the normalized intensity value can be determined, and the corresponding three-dimensional point cloud and its corresponding intensity value {x l,i， y l,i， z l,i， I i}. Wherein, i is used to represent the ray corresponding to the incident angle in the three-dimensional incident angle group.

[0343] The above embodiment introduces a method for simulating a laser radar from the perspective of a method flow, and the following embodiment introduces a device for simulating a laser radar from the perspective of a module. For details, please refer to the following embodiment.

[0344] The embodiment of the present application provides a laser radar simulation device, such as Figure 7 As shown, the laser radar simulation device 70 may specifically include: a construction module 71, a reading module 72, a motion distortion processing module 73 and a laser radar simulation module 74, wherein:

[0345] A construction module 71 is used to construct a three-dimensional simulation scene;

[0346] A reading module 72 is used to read parameters for laser radar simulation, where the parameters for laser radar simulation include: an incident angle of each laser radar ray;

[0347] A motion distortion processing module 73 is used to perform motion distortion processing on the incident angle of each laser radar ray to obtain the incident angle after distortion processing;

[0348] The laser radar simulation module 74 is used to perform laser radar simulation based on the three-dimensional simulation scene, the incident angle after distortion processing, and other parameters except the incident angle of the laser radar ray.

[0349] In one possible implementation of the embodiment of the present application, the construction module 71 is specifically used for any of the following when constructing a three-dimensional simulation scene:

[0350] Acquire sensor data and build a three-dimensional simulation scene based on the sensor data;

[0351] Build a three-dimensional simulation scene through the physical engine;

[0352] The three-dimensional simulation scene includes: a three-dimensional simulation scene described by a triangular grid or a three-dimensional simulation scene described by a square grid.

[0353] In another possible implementation of the embodiment of the present application, the parameters used for laser radar simulation include: field of view angle and angular resolution; wherein the device 70 further includes: an establishment module, wherein,

[0354] Create a module for creating a three-dimensional incidence angle table based on field of view angle and angular resolution.

[0355] In another possible implementation of the embodiment of the present application, the apparatus 70 further includes: a first acquisition module, a noise point removal module, a calculation module, and a first determination module, wherein:

[0356] A first acquisition module is used to obtain multi-frame measurement results;

[0357] Noise point removal module, used to remove noise points from multi-frame measurement results;

[0358] A calculation module, used to calculate the incident angle of each effective point corresponding to each ray in each frame;

[0359] The first determining module is configured to determine the incident angle corresponding to the i-th ray based on the incident angle corresponding to the i-th ray in each frame, where i∈[1,n], and n is the number of rays contained in each frame.

[0360] In another possible implementation of the embodiment of the present application, the motion distortion processing module 73 is specifically configured to:

[0361] Get the current pose information;

[0362] Determine the relative transformation matrix generated by motion distortion based on the current pose information;

[0363] The incident angle of each lidar ray is motion-distorted based on the relative transformation matrix.

[0364] In another possible implementation of the embodiment of the present application, the motion distortion processing module 73 performs motion distortion processing on the incident angle of each laser radar ray based on the relative transformation matrix to obtain the incident angle after the distortion processing, specifically for:

[0365] Determine the unit direction vector of the incident angle of each laser radar ray in a specific coordinate system, and obtain the unit direction vector corresponding to each ray;

[0366] Based on the unit direction vector corresponding to each ray and the relative transformation matrix, determine the unit direction vector of each ray after motion distortion;

[0367] The incident angle of each ray after the distortion is determined based on the unit direction vector of each ray after the motion distortion.

[0368] In another possible implementation of the embodiment of the present application, the laser radar simulation module 74 is specifically configured to:

[0369] Construct a spherical three-dimensional simulation sub-scene based on the three-dimensional simulation scene;

[0370] Collision detection is performed based on the 3D simulation sub-scene of the sphere, any incident angle after distortion processing, and other parameters except the incident angle of the lidar ray;

[0371] Perform lidar simulation based on collision detection results and beam model.

[0372] In another possible implementation of the embodiment of the present application, other parameters besides the incident angle of the laser radar ray include: the effective detection range of the laser radar, the vertical angular resolution, the horizontal angular resolution, and the diameter resolution;

[0373] The laser radar simulation module 74 is specifically used to perform collision detection based on the spherical three-dimensional simulation sub-scene, any incident angle after distortion processing, and other parameters other than the incident angle of the laser radar ray:

[0374] The spherical sub-scene is rasterized into spherical elements according to the vertical angular resolution, the horizontal angular resolution and the diameter resolution to obtain a 3D simulation sub-scene of spherical element rasterization;

[0375] Collision detection is performed based on a 3D simulation sub-scene rasterized by spherical elements and any incident angle after distortion processing.

[0376] In another possible implementation of the embodiment of the present application, the spherical element-rasterized three-dimensional simulation sub-scene includes multiple element-raster voxels corresponding to each ray; the device 70 further includes: a second determination module and a creation module, wherein:

[0377] A second determination module is configured to determine the index values ​​corresponding to the plurality of meta-grid voxels corresponding to each ray based on the incident angle of each ray after distortion processing, the propagation distance of each ray, the effective detection range of the laser radar, the vertical angular resolution, and the horizontal angular resolution;

[0378] The creation module is used to create a three-dimensional data group based on the index values ​​corresponding to the multiple spherical grid voxels corresponding to each ray.

[0379] In another possible implementation of the embodiment of the present application, the spherical element rasterized three-dimensional simulation sub-scene includes a plurality of spherical element raster voxels;

[0380] When performing collision detection based on the three-dimensional simulation sub-scene rasterized by the sphere element and any incident angle after distortion processing, the laser radar simulation module 74 is specifically used to:

[0381] Obtaining semantic information in the 3D simulation sub-scene of spherical element rasterization;

[0382] Based on the semantic information in the 3D simulation sub-scene of spherical element rasterization, the obstacles in each spherical element grid voxel are divided into bounding boxes;

[0383] Determine the coordinates of any ray in the LiDAR spherical coordinate system based on any distorted incident angle, the LiDAR's position in the simulation scene, and the spherical 3D simulation subscene.

[0384] Determine the distance from any ray to the obstacle surface based on the coordinates of any ray in the lidar spherical coordinate system and a plurality of spherical grid voxels;

[0385] Based on the distance from any ray to the obstacle surface and the beam model, determine the spot size generated by any ray to the obstacle surface;

[0386] Determining an incident angle after distorted ray processing that satisfies a first preset condition, where the incident angle after distorted ray processing that satisfies the first preset condition is an incident angle after distorted processing corresponding to a ray centered on a direction vector of any ray and having a spot size as a radius;

[0387] Determine the coordinates of each ray that meets the first preset condition in the laser radar spherical coordinate system based on each incident angle after distortion processing that meets the first preset condition, the position of the laser radar in the simulation scene, and the spherical three-dimensional simulation subscene;

[0388] Based on the coordinates of each ray that meets the first preset condition in the laser radar spherical coordinate system and a plurality of spherical grid voxels, the distance between each ray that meets the first preset condition and the obstacle surface is determined.

[0389] In another possible implementation of the embodiment of the present application, the apparatus 70 further includes: a third determination module, a division module, and a sampling module, wherein:

[0390] A third determining module is used to determine the number of sampling points;

[0391] A division module is used to divide the sampling range centered on any ray direction vector and composed of the spot radius as the radius based on the number of sampling points;

[0392] A sampling module is used to sample the lidar ray set based on each divided range to obtain a sampled lidar ray set;

[0393] The laser radar simulation module 74 is specifically used to determine the coordinates of each ray that meets the first preset condition in the laser radar spherical coordinate system based on each incident angle after distortion processing that meets the first preset condition, the position of the laser radar in the simulation scene, and the spherical three-dimensional simulation sub-scene:

[0394] Based on the incident angle of each ray after distortion processing of the sampled lidar ray set, the position of the lidar in the simulation scene and the three-dimensional simulation sub-scene of the sphere, the coordinates of each ray that meets the first preset condition in the lidar spherical coordinate system are determined.

[0395] In another possible implementation of the embodiment of the present application, the coordinates of any ray in the laser radar spherical coordinate system are (m, n); the laser radar simulation module 74 is specifically configured to determine the distance from any ray to the obstacle surface based on the coordinates of any ray in the laser radar spherical coordinate system and a plurality of spherical grid voxels:

[0396] From k=k min to k=k max Traversing the spherical element grid voxels with index (m, n, k) in the three-dimensional data group, and performing bounding box detection on the traversed spherical element grid voxels;

[0397] If there is an intersection, the traversal stops and the distance from any ray to the obstacle surface is calculated based on ray detection.

[0398] In another possible implementation of the embodiment of the present application, the collision detection result of any ray includes: the distance from any ray to the obstacle surface; the beam model includes the beam waist value and the beam wavelength; the laser radar simulation module 74, when performing laser radar simulation based on the collision detection result and the beam model, is specifically used to:

[0399] Determine the spot radius based on the distance from any ray to the obstacle surface, the beam waist value, and the wavelength of the beam;

[0400] Determine the set of lidar rays centered on any ray direction vector and with the spot radius as the radius;

[0401] The divergence of rays during propagation is simulated based on the lidar ray set.

[0402] In another possible implementation of the embodiment of the present application, the laser radar simulation result includes: the distance value from each ray to the obstacle; the device 70 further includes: a first processing module, a second processing module, a simulation module and a fourth determination module, wherein,

[0403] The first processing module is used to process the distance value from each ray to the obstacle in combination with the noise model to obtain the distance value corresponding to each ray after adding noise;

[0404] The second processing module is used to process the distance value corresponding to each ray after adding noise in combination with the weather model to obtain the laser intensity corresponding to each ray under different weather types;

[0405] The simulation module is used to simulate the laser intensity corresponding to each ray under the target weather type through the echo pattern and dropoff mechanism;

[0406] The fourth determination module is used to determine the three-dimensional point cloud data and the normalized intensity value of each laser radar ray based on the incident angle after distortion processing and the simulation results of each laser radar ray through three-dimensional point solution processing and normalization processing.

[0407] In another possible implementation of the embodiment of the present application, the laser radar simulation result includes: the distance value from each ray to the obstacle; wherein the device 70 further includes: a second acquisition module and a fifth determination module, wherein,

[0408] The second acquisition module is used to obtain the variance of the measured distance;

[0409] a fifth determination module, configured to determine an expectation and a variance of a noise model based on the variance of the measured distance;

[0410] The first processing module, when processing the laser radar simulation results in combination with the noise model, is specifically used to:

[0411] Noise is added to the distance value from each ray to the obstacle based on the expectation and variance of the noise model to obtain the distance value after adding noise.

[0412] In another possible implementation of the embodiment of the present application, the weather model includes a correspondence between weather type, measurement range, and attenuation; the second processing module processes the distance value corresponding to each ray after adding noise in combination with the weather model to obtain the laser intensity corresponding to each ray under the target weather type, specifically for:

[0413] Determine whether the distance value corresponding to each ray after adding noise falls within the measurement range corresponding to the target weather type;

[0414] If there is a ray belonging to the measurement range corresponding to the target weather type, then the attenuation rate corresponding to the target weather type is obtained;

[0415] Based on the distance value corresponding to the ray meeting the measurement range and the attenuation rate corresponding to the target weather type, the laser intensity corresponding to the ray meeting the measurement range under the target weather type is determined.

[0416] In another possible implementation of the embodiment of the present application, the simulation module is specifically configured to:

[0417] Determine the echo pattern for any ray;

[0418] If the echo mode of any ray is single echo mode, the first laser intensity and the corresponding first distance value are returned. The first laser intensity is the laser intensity corresponding to any ray under the target weather type, and the first distance value is the distance from the ray corresponding to the first laser intensity to the obstacle.

[0419] If the echo mode of any ray is a multi-echo mode, the first laser intensity, the first distance value, the laser intensity that meets the second preset condition, and the distance value corresponding to the laser intensity that meets the second preset condition are returned. The laser intensity that meets the second preset condition is the first N laser intensities selected from large to small among the laser intensities corresponding to the ray that meets the first preset condition; the distance value corresponding to the laser intensity that meets the second preset condition is the first N distance values ​​selected from large to small among the distance values ​​corresponding to the ray that meets the first preset condition.

[0420] In another possible implementation of the embodiment of the present application, the simulation module is specifically configured to:

[0421] Determine whether the second laser intensity is greater than a first intensity threshold, where the second laser intensity is the laser intensity corresponding to any ray under the target weather type, and the first intensity threshold is the laser intensity threshold for generating a dropoff phenomenon;

[0422] If it is greater than the first intensity threshold, a second laser intensity and a corresponding second distance value are output;

[0423] If it is not greater than the first intensity threshold, calculating the attenuation value based on the second laser intensity;

[0424] If the attenuation value is greater than a second intensity threshold, the second laser intensity and the first distance value are output, and the second intensity threshold is a random value.

[0425] In another possible implementation of the embodiment of the present application, the simulation result includes: a third laser intensity and a third distance value, where the third laser intensity is the laser intensity corresponding to any laser radar ray, and the third distance value is the distance value of the ray corresponding to the third laser intensity;

[0426] The fourth determination module is specifically used to determine the three-dimensional point cloud data of any laser radar ray and the normalized intensity value based on the incident angle, the second laser intensity and the second distance value after the distortion processing of any laser radar ray through three-dimensional point solution processing and normalization processing:

[0427] Calculate the three-dimensional point coordinates measured by any laser radar ray in the laser radar Cartesian coordinate system based on the incident angle after distortion processing of any laser radar ray, the third laser intensity, and the third distance value;

[0428] Based on the third laser intensity, the maximum laser intensity and the minimum laser intensity, a normalized intensity value is determined, where the maximum laser intensity and the minimum laser intensity are the maximum value and the minimum value of the laser intensity values ​​corresponding to each ray.

[0429] An embodiment of the present application provides a device for laser radar simulation. In the embodiment of the present application, a three-dimensional simulation scene is constructed and laser radar simulation parameters including the incident angle of each laser radar ray are read, and the incident angle of each laser radar ray is subjected to motion distortion processing to obtain the incident angle after distortion processing, so as to perform laser radar simulation based on the three-dimensional simulation scene, the incident angle after distortion processing and other parameters required for laser radar simulation. That is, when performing laser radar simulation, the incident angle of each laser radar ray is subjected to motion distortion processing to simulate the influence of motion distortion on the laser radar ray in a real scene, so that the laser radar simulation can be made closer to the real situation, and thus a more accurate simulation of the laser radar can be achieved on the simulation platform.

[0430] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the laser radar simulation device described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0431] An electronic device is provided in an embodiment of the present application, such as Figure 8 As shown, Figure 8 The electronic device 800 shown includes a processor 801 and a memory 803. The processor 801 and the memory 803 are connected, for example, via a bus 802. Optionally, the electronic device 800 may further include a transceiver 804. It should be noted that in actual applications, the number of transceivers 804 is not limited to one, and the structure of the electronic device 800 does not constitute a limitation on the embodiments of the present application.

[0432] Processor 801 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 801 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0433] Bus 802 may include a path for transmitting information between the aforementioned components. Bus 802 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. Bus 802 may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0434] The memory 803 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0435] The memory 803 is used to store application code for executing the solution of the present application, and the execution is controlled by the processor 801. The processor 801 is used to execute the application code stored in the memory 803 to implement the content shown in the above method embodiment.

[0436] The electronic devices include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. They may also include servers, and in the embodiments of the present application, the server may be a cloud server. Figure 8 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0437] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed on a computer, enables the computer to execute the corresponding contents of the aforementioned method embodiment. In the embodiment of the present application, a three-dimensional simulation scene is constructed and laser radar simulation parameters including the incident angle of each laser radar ray are read, and the incident angle of each laser radar ray is subjected to motion distortion processing to obtain the incident angle after the distortion processing, so as to perform laser radar simulation according to the three-dimensional simulation scene, the incident angle after the distortion processing, and other parameters required for laser radar simulation. That is, when performing laser radar simulation, the incident angle of each laser radar ray is subjected to motion distortion processing to simulate the influence of motion distortion on the laser radar ray in a real scene, so that the laser radar simulation can be made closer to the real situation, and thus a more accurate simulation of the laser radar can be achieved on the simulation platform.

[0438] Those skilled in the art will clearly understand that for the sake of convenience and brevity, the division of the above-mentioned functional modules is only used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working processes of the above-mentioned systems, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0439] In the embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0440] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0441] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0442] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0443] The above embodiments are merely a detailed introduction to the technical solutions of the present application. However, the description of the above embodiments is only intended to help understand the method and core concept of the present application and should not be construed as limiting the present application. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application.

Claims

1. A laser radar simulation method, characterized in that: include: Construct 3D simulation scenes; Reading parameters for laser radar simulation, wherein the parameters for laser radar simulation include: an incident angle of each laser radar ray; Performing motion distortion processing on the incident angle of each laser radar ray to obtain a distorted incident angle; The laser radar simulation is performed based on the three-dimensional simulation scene, the incident angle after the distortion processing, and other parameters except the incident angle of the laser radar ray.

2. The method according to claim 1, characterized in that The constructing of a three-dimensional simulation scene includes any of the following: Acquiring sensor data, and constructing the three-dimensional simulation scene based on the sensor data; Constructing the three-dimensional simulation scene through a physical engine; The three-dimensional simulation scene includes: a three-dimensional simulation scene described by a triangular grid or a three-dimensional simulation scene described by a square grid.

3. The method according to claim 1, characterized in that The parameters used for laser radar simulation include: field of view angle and angular resolution; The step of reading parameters for laser radar simulation further includes: A three-dimensional incident angle table is established based on the field of view angle and the angular resolution.

4. The method according to claim 1, wherein Reading the incident angle of the laser radar ray, which also includes: Get multiple frames of measurement results; removing noise points from the multi-frame measurement results; Calculate the incident angle for each effective point corresponding to each ray in each frame; Based on the incident angle corresponding to the i-th ray in each frame, the incident angle corresponding to the i-th ray is determined, where i∈[1,n], and n is the number of rays contained in each frame.

5. The method according to claim 1, wherein The performing motion distortion processing on the incident angle of each laser radar ray includes: Get the current pose information; Determine a relative transformation matrix generated by motion distortion based on the current posture information; The incident angle of each laser radar ray is subjected to motion distortion processing based on the relative transformation matrix.

6. The method according to claim 5, characterized in that The performing motion distortion processing on the incident angle of each laser radar ray based on the relative transformation matrix to obtain the incident angle after the distortion processing includes: Determine the unit direction vector of the incident angle of each laser radar ray in a specific coordinate system to obtain the unit direction vector corresponding to each ray; Determining the unit direction vector of each ray after motion distortion based on the unit direction vector corresponding to each ray and the relative transformation matrix; The incident angle of each ray after the distortion is determined based on the unit direction vector of each ray after the motion distortion.

7. The method according to claim 5 or 6, characterized in that Performing a laser radar simulation based on the three-dimensional simulation scene, any incident angle after distortion processing, and other parameters except the incident angle of the laser radar ray, including: Constructing a spherical three-dimensional simulation sub-scene based on the three-dimensional simulation scene; performing collision detection based on the spherical three-dimensional simulation sub-scene, any one of the distorted incident angles, and other parameters except the incident angle of the laser radar ray; Based on the collision detection results and the beam model, a lidar simulation is performed.

8. The method according to claim 7, characterized in that The other parameters besides the incident angle of the laser radar ray include: the effective detection range, vertical angular resolution, horizontal angular resolution and diameter resolution of the laser radar; The collision detection based on the spherical three-dimensional simulation sub-scene, any incident angle after distortion processing, and other parameters except the incident angle of the laser radar ray includes: Performing spherical element rasterization on the spherical sub-scene according to the vertical angular resolution, the horizontal angular resolution, and the diameter resolution to obtain a spherical element rasterized three-dimensional simulation sub-scene; Collision detection is performed based on a three-dimensional simulation sub-scene rasterized with spherical elements and any incident angle after the distortion processing.

9. The method according to claim 8, characterized in that The spherical element-rasterized three-dimensional simulation sub-scene includes a plurality of element-raster voxels corresponding to each ray; the method further includes: Determining index values ​​corresponding to the plurality of spherical grid voxels corresponding to each ray based on the incident angle of each ray after distortion processing, the propagation distance of each ray, the effective detection range of the laser radar, the vertical angular resolution, and the horizontal angular resolution; A three-dimensional data group is created based on the index values ​​corresponding to the plurality of meta-grid voxels corresponding to each ray.

10. The method according to claim 8 or 9, characterized in that The spherical element rasterized three-dimensional simulation sub-scene includes a plurality of spherical element grid voxels; The collision detection based on the three-dimensional simulation sub-scene rasterized by spherical elements and any incident angle after the distortion processing includes: Acquiring semantic information in the 3D simulation sub-scene rasterized by the spherical element; Performing bounding box division on obstacles in each spherical element grid voxel based on semantic information in the spherical element rasterized three-dimensional simulation sub-scene; Determine the coordinates of any ray in the laser radar spherical coordinate system based on any distorted incident angle, the position of the laser radar in the simulation scene, and the spherical three-dimensional simulation sub-scene; Determine a distance from any ray to the obstacle surface based on the coordinates of any ray in the laser radar spherical coordinate system and the plurality of spherical grid voxels; Determining the size of a light spot generated by any ray to the obstacle surface based on the distance from any ray to the obstacle surface and the beam model; Determining an incident angle of a ray after distortion processing that satisfies a first preset condition, where the incident angle of the ray after distortion processing that satisfies the first preset condition is the incident angle of the ray after distortion processing corresponding to a ray centered on the direction vector of any ray and having the spot size as a radius; Determine the coordinates of each ray that meets the first preset condition in the laser radar spherical coordinate system based on each incident angle after distortion processing that meets the first preset condition, the position of the laser radar in the simulation scene, and the three-dimensional simulation sub-scene of the sphere; Based on the coordinates of each ray that meets the first preset condition in the laser radar spherical coordinate system and the multiple spherical grid voxels, the distance between each ray that meets the first preset condition and the obstacle surface is determined.

11. The method according to claim 10, characterized in that The step of determining the incident angle after the ray distortion processing that satisfies the first preset condition further includes: Determine the number of sampling points; Based on the number of sampling points, a sampling range with the direction vector of any ray as the center and the spot radius as the radius is divided; Sampling the laser radar ray set based on each divided range to obtain a sampled laser radar ray set; The step of determining the coordinates of each ray meeting the first preset condition in the laser radar spherical coordinate system based on each incident angle after distortion processing that meets the first preset condition, the position of the laser radar in the simulation scene, and the spherical three-dimensional simulation sub-scene includes: Based on the incident angle of each ray after distortion processing of the sampled laser radar ray set, the position of the laser radar in the simulation scene and the three-dimensional simulation sub-scene of the sphere, the coordinates of each ray that meets the first preset condition in the laser radar spherical coordinate system are determined.

12. The method according to claim 11, characterized in that The coordinates of any ray in the laser radar spherical coordinate system are (m, n); The determining, based on the coordinates of the any ray in the laser radar spherical coordinate system and the plurality of spherical grid voxels, a distance from the any ray to the obstacle surface includes: From k=k min to k=k max Traversing the spherical element grid voxels with index (m, n, k) in the three-dimensional data group, and performing bounding box detection on the traversed spherical element grid voxels; If an intersection exists, the traversal is stopped, and the distance from any ray to the obstacle surface is calculated based on ray detection.

13. The method according to claim 7, characterized in that The collision detection result of any ray includes: the distance from any ray to the obstacle surface; the beam waist value and the wavelength of the beam included in the beam model; The performing of laser radar simulation based on the collision detection result and the beam model includes: Determining a spot radius based on the distance from any one of the rays to the obstacle surface, the beam waist value of the light beam, and the wavelength of the light beam; Determine a laser radar ray set centered on any one of the ray direction vectors and having the light spot radius as a radius; The divergence phenomenon of rays during propagation is simulated based on the laser radar ray set.

14. The method according to claim 1, wherein The laser radar simulation result includes: the distance value from each ray to the obstacle; the method further includes: In combination with the noise model, the distance value from each ray to the obstacle is processed to obtain the distance value corresponding to each ray after adding noise; Combined with the weather model, the distance value corresponding to each ray after adding noise is processed to obtain the laser intensity corresponding to each ray under the target weather type; Based on the laser intensity corresponding to each ray under the target weather type, the echo pattern and the dropoff mechanism are simulated; Based on the incident angle after distortion processing of each laser radar ray and the simulation results, the three-dimensional point cloud data of each laser radar ray and the normalized intensity value are determined through three-dimensional point solution processing and normalization processing.

15. The method according to claim 14, characterized in that The laser radar simulation results include: the distance value from each ray to the obstacle; Among them, the laser radar simulation results are processed in combination with the noise model, which also includes: Obtaining a variance of the measured distance, and determining an expectation and a variance of a noise model based on the variance of the measured distance; Among them, the laser radar simulation results are processed in combination with the noise model, including: Noise is added to the distance value from each ray to the obstacle based on the expectation and variance of the noise model to obtain a distance value after the noise is added.

16. The method according to claim 15, characterized in that The weather model includes a correspondence between weather type, measurement range, and attenuation; Combined with the weather model, the distance value corresponding to each ray after adding noise is processed to obtain the laser intensity corresponding to each ray under the target weather type, including: Determining whether the distance value corresponding to each ray after adding the noise falls within the measurement range corresponding to the target weather type; If there is a ray belonging to the measurement range corresponding to the target weather type, then obtaining the attenuation rate corresponding to the target weather type; Based on the distance value corresponding to the ray meeting the measurement range and the attenuation rate corresponding to the target weather type, the laser intensity corresponding to the ray meeting the measurement range under the target weather type is determined.

17. The method according to claim 16, characterized in that Based on the laser intensity corresponding to any ray under the target weather type, it is simulated through the echo pattern, including: determining an echo pattern of any of the rays; If the echo mode of any of the rays is a single echo mode, the first laser intensity and the corresponding first distance value are returned. The first laser intensity is the laser intensity corresponding to any ray under the target weather type, and the first distance value is the distance from the ray corresponding to the first laser intensity to the obstacle. If the echo mode of any of the rays is a multi-echo mode, the first laser intensity, the first distance value, the laser intensity that meets the second preset condition, and the distance value corresponding to the laser intensity that meets the second preset condition are returned, and the laser intensity that meets the second preset condition is the first N laser intensities selected from large to small among the laser intensities corresponding to the rays that meet the first preset condition; the distance value corresponding to the laser intensity that meets the second preset condition is the first N distance values ​​selected from large to small among the distance values ​​corresponding to the rays that meet the first preset condition.

18. The method according to claim 16 or 17, characterized in that Based on the laser intensity corresponding to any ray under the target weather type, it is simulated through the Dropoff mechanism, including: Determining whether a second laser intensity is greater than a first intensity threshold, where the second laser intensity is the laser intensity corresponding to any ray under the target weather type, and the first intensity threshold is the laser intensity threshold for generating a dropoff phenomenon; If it is greater than the first intensity threshold, outputting the second laser intensity and the corresponding second distance value; If it is not greater than the first intensity threshold, calculating the attenuation value based on the second laser intensity; If the attenuation value is greater than a second intensity threshold, the second laser intensity and the first distance value are output, and the second intensity threshold is a random value.

19. The method according to claim 14, wherein The simulation result includes: a third laser intensity and a third distance value, wherein the third laser intensity is the laser intensity corresponding to any laser radar ray, and the third distance value is the distance value of the ray corresponding to the third laser intensity; Based on the incident angle, the second laser intensity, and the second distance value after the distortion processing of any laser radar ray, the three-dimensional point cloud data and the normalized intensity value of any laser radar ray are determined through three-dimensional point solution processing and normalization processing, including: Calculate the three-dimensional point coordinates measured by any laser radar ray in the laser radar Cartesian coordinate system based on the incident angle of any laser radar ray after distortion processing, the third laser intensity, and the third distance value; A normalized intensity value is determined based on the third laser intensity, the maximum laser intensity, and the minimum laser intensity, where the maximum laser intensity and the minimum laser intensity are the maximum and minimum laser intensity values ​​corresponding to the respective rays.

20. A laser radar simulation device, characterized in that: include: Construction module, used to construct three-dimensional simulation scenes; A reading module, configured to read parameters for laser radar simulation, wherein the parameters for laser radar simulation include: an incident angle of each laser radar ray; a motion distortion processing module, configured to perform motion distortion processing on the incident angle of each laser radar ray to obtain a distorted incident angle; The laser radar simulation module is used to perform laser radar simulation based on the three-dimensional simulation scene, the incident angle after the distortion processing, and other parameters except the incident angle of the laser radar ray.

21. An electronic device, characterized in that: It includes: One or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to: execute a lidar simulation method according to any one of claims 1 to 19.

22. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a laser radar simulation method as described in any one of claims 1 to 19 is implemented.

Images