Multi-source heterogeneous laser radar simulation method, device, equipment and medium

Abstract

The invention discloses a multi-source heterogeneous laser radar simulation method, device, equipment and medium, and relates to the technical field of laser radar simulation, and the method comprises the steps: obtaining the angle parameters of a plurality of laser radars, and constructing the total field angle of the plurality of laser radars according to the angle parameters; an angle distribution table and an angle correspondence table are established based on the total field angle, the angle distribution table comprises a set used for simulating all light beam directions needed by the multiple laser radars, and the angle correspondence table comprises the corresponding relation between all the light beam directions and all the laser radars; creating a simulation environment, taking the angle distribution table as input, calculating intersection point information of each light beam direction and an object in the simulation environment by adopting a single ray tracing algorithm, and outputting a frame of mixed point cloud data; and analyzing the mixed point cloud data according to the angle correspondence table, and outputting a plurality of point cloud data subsets corresponding to the laser radar. According to the invention, only one virtual sensor module is configured, so that the problem of simulation function limitation caused by the limitation of the number of software sensor modules can be solved.

Description

Technical Field

[0001] This application relates to the field of lidar simulation technology, specifically to a multi-source heterogeneous lidar simulation method, device, equipment, and medium. Background Technology

[0002] Using simulation testing to verify autonomous driving systems has become a common and necessary method in the industry. When testing systems integrated with LiDAR, the primary approach is LiDAR point cloud injection. The main idea is to build a model simulating a real-world scene in simulation software, then simulate the physical characteristics of the LiDAR, and finally simulate ray detection to obtain distance information in the current simulation environment to generate a LiDAR point cloud.

[0003] Utilizing Nvidia's ray tracing technology to compute point cloud data in simulation scenarios is a common industry solution, and it's also the most widely used simulation software in the autonomous driving simulation field, VTD. Currently, high-level autonomous driving systems typically employ multiple LiDAR sensors. When simulating multiple LiDARs, each LiDAR sensor is configured with a sensor module. For each cycle of point cloud data, this sensor module runs once, using the corresponding ray tracing algorithm to obtain the LiDAR's point cloud data. Multiple LiDARs require multiple sensor modules. Since ray tracing algorithms are resource-intensive (GPU), simultaneous computation by multiple sensor modules leads to time lags. In real vehicles, multiple LiDARs operate simultaneously, and these time lags affect simulation accuracy. Furthermore, the computation by multiple sensor modules can directly cause simulation software to malfunction, even impacting the simulation's functionality. Each simulation software has a limited number of configurable sensor modules, some only allowing a maximum of four, which is insufficient for multi-LiDAR simulation. Summary of the Invention

[0004] This application provides a multi-source heterogeneous lidar simulation method, apparatus, equipment, and medium, which can solve the technical problems existing in the prior art where there are time differences between multiple radar data and the simulation software is limited by the number of sensor modules.

[0005] In a first aspect, embodiments of this application provide a simulation method for multi-source heterogeneous lidar, the simulation method comprising: Obtain the angle parameters of multiple lidars, and construct the total field of view of the multiple lidars based on the multiple angle parameters; An angle distribution table and an angle correspondence table are established based on the total field of view. The angle distribution table includes a set of all beam directions required to simulate multiple lidars, and the angle correspondence table includes the correspondence between each beam direction and each lidar. A simulation environment is created, and the angle distribution table is used as input. A single ray tracing algorithm is used to calculate the intersection information of each beam direction with the object in the simulation environment, so as to output a frame of mixed point cloud data. The mixed point cloud data is parsed according to the angle correspondence table to output multiple subsets of point cloud data corresponding to the lidar.

[0006] In conjunction with the first aspect, in one implementation, acquiring the angle parameters of multiple lidars and constructing the total field of view of the multiple lidars based on the multiple angle parameters includes: Obtain the installation orientation angle, horizontal field of view, vertical field of view, horizontal angular resolution, and vertical angular resolution for each lidar. Based on the installation orientation angle, horizontal field of view angle, and vertical field of view angle of each lidar, the spatial distribution of the field of view of each lidar in the vehicle coordinate system is calculated through coordinate transformation. Based on the spatial distribution, the minimum outer envelope field of view angle covering all lidar fields of view is determined as the total field of view angle.

[0007] In conjunction with the first aspect, in one embodiment, establishing the angle distribution table and the angle correspondence table based on the total field of view includes: The horizontal and vertical ranges of the total field of view are used as boundaries; Generate an azimuth-elevation grid within the boundary according to a preset precision; Store the azimuth-elevation pairs of each point in the grid in order as an angle distribution table; Confirm the correspondence between each azimuth-elevation pair in the angle distribution table and the actual field of view of each lidar; Establish a mapping relationship between the azimuth-elevation angle pairs and their corresponding lidar identifiers; Store all mapping relationships as the angle correspondence table.

[0008] In conjunction with the first aspect, in one implementation, the preset accuracy is determined by finding the greatest common divisor of the horizontal and vertical angular resolutions of the multiple lidars.

[0009] In conjunction with the first aspect, in one implementation, the creation of the simulation environment, using the angle distribution table as input, employs a single-ray ray tracing algorithm to calculate the intersection information of all beam directions with objects in the simulation environment, to output a frame of mixed point cloud data, including: Read each beam direction in the angle distribution table sequentially; Rays are emitted from each lidar installation location, based on each beam direction; Mark the nearest intersection point between each ray and an object in the simulation environment; Record the point cloud data points at each intersection point to output a frame of mixed point cloud data; The hybrid point cloud data includes multiple point cloud data points, and each point cloud data point includes the three-dimensional coordinates, distance information, and reflection intensity information of each intersection point.

[0010] In conjunction with the first aspect, in one implementation, the step of parsing the mixed point cloud data according to the angle correspondence table to output multiple subsets of point cloud data corresponding to each LiDAR includes: Read multiple point cloud data points included in the mixed point cloud data; Based on the index position of each point cloud data point in the angle distribution table, find the corresponding LiDAR identifier in the angle correspondence table; Each point cloud data point is added to the point cloud data subset of its corresponding LiDAR.

[0011] In conjunction with the first aspect, in one implementation, after parsing the mixed point cloud data according to the angle correspondence table to output multiple subsets of point cloud data corresponding to the lidar, the method includes: Each subset of point cloud data is encapsulated according to the original data format of its corresponding LiDAR. Add device identifiers and data frame headers for multiple LiDARs; The information is transmitted via the LiDAR communication protocol to a designated port of the autonomous vehicle's perception system.

[0012] Secondly, embodiments of this application provide a multi-source heterogeneous lidar simulation device, the multi-source heterogeneous lidar simulation device comprising: The acquisition module is used to acquire the angle parameters of multiple lidars and construct the total field of view of the multiple lidars based on the multiple angle parameters. A module is established to create an angle distribution table and an angle correspondence table based on the total field of view. The angle distribution table includes a set of all beam directions required to simulate multiple lidars, and the angle correspondence table includes the correspondence between each angle parameter and each lidar. The calculation module is used to create a simulation environment. Taking the angle distribution table as input, it uses a single ray tracing algorithm to calculate the intersection information of all beam directions with objects in the simulation environment, and outputs a frame of mixed point cloud data. The parsing module is used to parse the mixed point cloud data according to the angle correspondence table to output multiple subsets of point cloud data corresponding to each lidar.

[0013] Thirdly, this application provides a multi-source heterogeneous lidar simulation device, which includes a processor, a memory, and a multi-source heterogeneous lidar simulation program stored in the memory and executable by the processor. When the multi-source heterogeneous lidar simulation program is executed by the processor, it implements the steps of the multi-source heterogeneous lidar simulation method as described in any of the above embodiments.

[0014] Fourthly, embodiments of this application provide a computer-readable storage medium storing a multi-source heterogeneous lidar simulation program, wherein when the multi-source heterogeneous lidar simulation program is executed by a processor, it implements the steps of the multi-source heterogeneous lidar simulation method described in any of the above embodiments.

[0015] The beneficial effects of the technical solutions provided in this application include: This application embodiment simulates multiple LiDARs using a single ray tracing calculation, significantly improving simulation real-time performance and reducing computational resource consumption. By unifying all beam directions through a pre-calculated angle distribution table, only one ray tracing operation is needed to generate hybrid point cloud data covering the entire radar field of view, greatly reducing GPU load and avoiding simulation stuttering or interruptions caused by GPU resource exhaustion. Furthermore, this application embodiment requires only one virtual sensor module; by changing the contents of the angle distribution table and angle correspondence table, any number of LiDARs can be flexibly simulated, thus solving the simulation function limitations caused by the limitation on the number of software sensor modules. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating an embodiment of the multi-source heterogeneous lidar simulation method of this application; Figure 2 This is a schematic diagram of the angle distribution table for the multi-source heterogeneous lidar simulation method of this application; Figure 3 This is a functional module diagram of an embodiment of the multi-source heterogeneous lidar simulation device of this application; Figure 4 This is a schematic diagram of the hardware structure of the multi-source heterogeneous lidar simulation device involved in the embodiments of this application. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0019] This application provides a multi-source heterogeneous lidar simulation method, apparatus, equipment, and medium, which can solve the technical problems existing in the prior art where there are time differences between multiple radar data and the simulation software is limited by the number of sensor modules.

[0020] In a first aspect, embodiments of this application provide a simulation method for multi-source heterogeneous lidar.

[0021] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the multi-source heterogeneous lidar simulation method of this application. Figure 1 As shown, the simulation method for multi-source heterogeneous lidar includes the following steps: Step S1: Obtain the angle parameters of multiple lidars, and construct the total field of view of multiple lidars based on the multiple angle parameters.

[0022] Specifically, step S1 includes the following steps: Step S11: Obtain the installation orientation angle, horizontal field of view, vertical field of view, horizontal angular resolution, and vertical angular resolution of each lidar.

[0023] Specifically, the angle parameters for each LiDAR are extracted from the installation configuration. The installation orientation angle represents the LiDAR's orientation relative to the vehicle coordinate system; the horizontal field of view angle represents the maximum angular range the LiDAR can cover in the horizontal direction; the vertical field of view angle represents the maximum angular range the LiDAR can cover in the vertical direction; the horizontal angular resolution represents the smallest distinguishable angular unit in the horizontal direction; and the vertical angular resolution represents the smallest distinguishable angular unit in the vertical direction. These angle parameters provide the necessary parameters for constructing the subsequent total field of view, ensuring that the field of view characteristics of each LiDAR are accurately reflected in the total field of view.

[0024] Step S12: Based on the installation orientation angle, horizontal field of view angle, and vertical field of view angle of each lidar, calculate the spatial distribution of the field of view of each lidar in the vehicle coordinate system through coordinate transformation.

[0025] Specifically, a local coordinate system is established with the installation location of each lidar as the origin, and the direction vector of the boundary ray is determined based on its horizontal and vertical field of view angles. A rotation matrix is ​​constructed based on the installation orientation angle and installation location of each lidar, transforming the boundary vectors in the local coordinate system to the vehicle coordinate system, thus obtaining the spatial distribution of each lidar's field of view in the vehicle coordinate system. This embodiment of the application, by confirming the position and shape of each lidar's field of view in the vehicle coordinate system, provides an accurate spatial reference for determining the total field of view angle, ensuring that the total field of view angle can reasonably cover the fields of view of all lidars.

[0026] Step S13: Based on the spatial distribution, determine the minimum outer envelope field of view angle that covers all lidar fields of view as the total field of view angle.

[0027] Specifically, in the horizontal direction, the minimum coverage angle range of all lidar fields of view projected onto the horizontal plane is determined; in the vertical direction, the minimum coverage angle range of all fields of view projected onto the vertical plane is determined. These two angle ranges together constitute the total field of view angle covering all lidar fields of view.

[0028] Step S2: Establish an angle distribution table and an angle correspondence table based on the total field of view. The angle distribution table includes a set of all beam directions required to simulate multiple lidars, and the angle correspondence table includes the correspondence between each beam direction and each lidar.

[0029] Specifically, step S2, which involves establishing the angle distribution table and the angle correspondence table based on the total field of view, includes the following steps: Step S21: Use the horizontal and vertical ranges of the total field of view as boundaries.

[0030] Specifically, based on the total field of view obtained in step S1, its horizontal and vertical ranges are extracted, and these two ranges are used as the boundary conditions for simulation. In some other embodiments of this application, boundary alignment techniques can be used to discretize the continuous field of view range into regular sampling intervals to ensure the accurate inclusion of boundary values.

[0031] Step S22: Generate an azimuth-elevation grid within the boundary according to the preset accuracy.

[0032] The preset accuracy is determined by finding the greatest common divisor (GCD) of the horizontal and vertical angular resolutions of multiple lidars. Specifically, the GCD of the horizontal and vertical angular resolution sets of each lidar is calculated and used as the basic sampling interval to ensure that the grid is compatible with the resolution of each lidar. In the horizontal direction, starting from the left boundary of the total field of view, a series of azimuth angles are generated with the preset accuracy in the horizontal direction as the step size. In the vertical direction, starting from the lower boundary of the total field of view, a series of elevation angles are generated with the preset accuracy in the vertical direction as the step size. This forms a set of grid points covering the entire range of the total field of view. A reasonable selection of the preset accuracy ensures no resolution conflicts during subsequent data splitting and avoids interpolation errors.

[0033] Step S23: Store the azimuth-elevation pairs of each point in the grid in order as an angle distribution table.

[0034] Specifically, the set of grid points formed in step S22 is stored in a table structure according to a certain order (e.g., lexicographical order of horizontal first, then vertical) to form an angle distribution table. Each entry in the angle distribution table contains specific values ​​for azimuth and elevation angles, representing a beam direction. The setting of the angle distribution table provides a clear set of beam directions for the ray tracing algorithm, while also facilitating subsequent fast indexing and traversal, thus improving simulation efficiency.

[0035] Step S24: Confirm the correspondence between each azimuth-elevation pair in the angle distribution table and the actual field of view of each lidar.

[0036] Specifically, the field-of-view boundary of each lidar is extracted, and the grid points are verified to ensure that they meet the horizontal and vertical field-of-view constraints of the lidar. The association between the grid point and the lidar is recorded. This ensures the accuracy of beam assignment.

[0037] Step S25: Establish a mapping relationship between the azimuth-elevation angle pairs and their corresponding lidar identifiers; Specifically, if a grid point belongs to the field of view of a LiDAR, then the index of the grid point is bound to the identifier of its corresponding LiDAR. The mapping relationship can also be implemented using a dictionary or an array.

[0038] Step S26: Store all mapping relationships as an angle correspondence table.

[0039] Specifically, all the mapping relationships established in step S25 are summarized and stored as an angle correspondence table, thereby improving the automation of data processing and reducing manual intervention.

[0040] Figure 2 This is a schematic diagram of the angle distribution table for the multi-source heterogeneous lidar simulation method of this application. See also... Figure 2In a specific embodiment, the implementation process of constructing a multi-source heterogeneous simulation system based on lidar A and lidar B is described. Lidar A has a lateral field of view of 25° (resolution 5°) and a longitudinal field of view of 12° (resolution 3°), while lidar B has a lateral field of view of 28° (resolution 7°) and a longitudinal field of view of 9° (resolution 0.9°). The two lidars are installed at angles 90° apart and at the same horizontal height. A parameter system for the total field of view is generated through spatial geometric calculations, where the X-direction represents the lateral total field of view and the Y-direction represents the longitudinal total field of view. The lateral total field of view is 116.5°, and the resolution is taken as the greatest common divisor (GCD) of three values ​​(5, 7, 116.5), which is 0.5°. The longitudinal total field of view is 12°, and the resolution is taken as the GCD of three values ​​(3, 0.9, 12), which is 0.3°. The angle distribution table established accordingly is stored in a matrix structure, containing 234 columns and 41 rows. Each cell stores a specific azimuth-elevation angle pair, and adjacent cells increase at intervals of 0.5° × 0.3°.

[0041] The synchronously constructed angle correspondence table uses the same dimension (234×41), and the beam affiliation is identified by a flag bit. Specifically, the area corresponding to LiDAR A is the grid cell with row index 1-41 (step 10) and column index 1-51 (step 10) in the matrix, and the area corresponding to LiDAR B is the grid cell with row index 6-36 (step 3) and column index 178-234 (step 14). The affiliation area is marked as 1, and the other areas are marked as 0.

[0042] The embodiments of this application optimize sampling accuracy through the greatest common divisor algorithm, which not only ensures complete coverage of the beam direction, but also avoids complex calculations of data, significantly improving the accuracy and real-time performance of multi-radar simulation.

[0043] Step S3: Create a simulation environment. Using the angle distribution table as input, use a single ray tracing algorithm to calculate the intersection information of each beam direction with the object in the simulation environment, and output a frame of mixed point cloud data.

[0044] Specifically, step S3 includes the following steps: Step S31: Read each beam direction in the angle distribution table in sequence.

[0045] Specifically, the azimuth and elevation pairs corresponding to each grid point in the angle distribution table are read sequentially. These angle pairs represent the emission direction of the ray in the simulation environment.

[0046] Step S32: Based on each beam direction, emit rays from each lidar mounting location.

[0047] Specifically, based on the angle distribution table, the starting point and direction vector of the ray are determined, and the ray is emitted from the installation position of the lidar along the direction vector of the ray.

[0048] Step S33: Mark the nearest intersection point between each ray and an object in the simulation environment.

[0049] Specifically, ray tracing algorithms are used to detect the intersection of each ray with all objects in the simulation environment. The location of the nearest intersection point between each ray and an object is recorded, and the 3D coordinates, distance information, and reflection intensity information of the intersection point are marked.

[0050] Step S34: Record the point cloud data points of each intersection point to output a frame of mixed point cloud data.

[0051] The hybrid point cloud data includes multiple point cloud data points, each containing the 3D coordinates, distance information, and reflection intensity information of each intersection point. The 3D coordinates represent the position of the intersection point in the vehicle coordinate system, the distance information represents the distance from the ray origin to the intersection point, and the reflection intensity information represents the intensity value based on the object's material and laser reflection characteristics.

[0052] Therefore, the embodiments of this application can realize the simulation of multiple lidars through one sensor module, breaking through the limitation of the number of sensor modules in the simulation software and solving the problem that the simulation scene cannot be fully reproduced due to insufficient number of modules.

[0053] Step S4: Parse the mixed point cloud data according to the angle correspondence table to output multiple subsets of point cloud data corresponding to the LiDAR.

[0054] Specifically, step S4 includes the following steps: Step S41: Read multiple point cloud data points included in the mixed point cloud data.

[0055] Step S42: Based on the index position of each point cloud data point in the angle distribution table, find the corresponding LiDAR identifier in the angle correspondence table.

[0056] Specifically, since the angle correspondence table stores the mapping relationship between each index position and the LiDAR identifier, the LiDAR to which point cloud data point belongs can be determined by looking up the table.

[0057] Step S43: Add each point cloud data point to the point cloud data subset of its corresponding LiDAR.

[0058] Specifically, a subset of point cloud data from each LiDAR is stored separately for subsequent processing and analysis.

[0059] In this embodiment of the application, after step S4, the following is included: Step S51: Encapsulate each subset of point cloud data according to the original data format of its corresponding LiDAR; Step S52: Add device identifiers and data frame headers for multiple LiDARs; Step S53: Send the signal to the designated port of the autonomous vehicle's perception system via the LiDAR communication protocol.

[0060] By sending multiple subsets of point cloud data to the perception system of autonomous vehicles according to the communication protocol of LiDAR, the strict requirements of autonomous driving systems for data timeliness and accuracy are met, thereby effectively improving the perception capabilities and decision-making efficiency of autonomous driving systems.

[0061] This application embodiment simulates multiple LiDARs using a single ray tracing calculation, significantly improving simulation real-time performance and reducing computational resource consumption. By unifying all beam directions through a pre-calculated angle distribution table, only one ray tracing operation is needed to generate hybrid point cloud data covering the entire radar field of view, greatly reducing GPU load and avoiding simulation stuttering or interruptions caused by GPU resource exhaustion. Furthermore, this application embodiment requires only one virtual sensor module; by changing the contents of the angle distribution table and angle correspondence table, any number of LiDARs can be flexibly simulated, thus solving the simulation function limitations caused by the limitation on the number of software sensor modules.

[0062] Secondly, embodiments of this application also provide a multi-source heterogeneous lidar simulation device.

[0063] In one embodiment, reference is made to Figure 3 , Figure 3 This is a functional module diagram of an embodiment of the multi-source heterogeneous lidar simulation device of this application. Figure 3 As shown, the multi-source heterogeneous lidar simulation device includes: The acquisition module is used to acquire the angle parameters of multiple lidars and construct the total field of view of the multiple lidars based on the multiple angle parameters. The module is used to establish an angle distribution table and an angle correspondence table based on the total field of view. The angle distribution table includes a set of all beam directions required to simulate multiple lidars, and the angle correspondence table includes the correspondence between each angle parameter and each lidar. The calculation module is used to create a simulation environment. Taking the angle distribution table as input, it uses a single ray tracing algorithm to calculate the intersection information of all beam directions with objects in the simulation environment, and outputs a frame of mixed point cloud data. The parsing module is used to parse the mixed point cloud data according to the angle correspondence table to output multiple subsets of point cloud data corresponding to each LiDAR.

[0064] The functions of each module in the above-mentioned multi-source heterogeneous lidar simulation device correspond to the steps in the above-mentioned multi-source heterogeneous lidar simulation method embodiment, and their functions and implementation processes will not be described in detail here.

[0065] Thirdly, embodiments of this application provide a multi-source heterogeneous lidar simulation device. The multi-source heterogeneous lidar simulation device includes a processor, a memory, and a multi-source heterogeneous lidar simulation program stored in the memory and executable by the processor. When the multi-source heterogeneous lidar simulation program is executed by the processor, it implements the steps of the multi-source heterogeneous lidar simulation method as described in any of the above embodiments.

[0066] Figure 4 This is a schematic diagram of the hardware structure of the multi-source heterogeneous lidar simulation device involved in the embodiments of this application. The multi-source heterogeneous lidar simulation device can be a personal computer (PC), laptop computer, server, or other device with data processing capabilities. In the embodiments of this application, the multi-source heterogeneous lidar simulation device may include a processor, memory, communication interface, and communication bus.

[0067] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0068] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting devices within the multi-source heterogeneous lidar simulation equipment, as well as interfaces used for interconnecting the multi-source heterogeneous lidar simulation equipment with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0069] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0070] The processor can be a general-purpose processor, which can call the multi-source heterogeneous lidar simulation program stored in the memory and execute the multi-source heterogeneous lidar simulation method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the multi-source heterogeneous lidar simulation program is called can be referred to the various embodiments of the multi-source heterogeneous lidar simulation method of this application, and will not be repeated here.

[0071] Those skilled in the art will understand that Figure 4 The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0072] Fourthly, embodiments of this application provide a computer-readable storage medium storing a multi-source heterogeneous lidar simulation program, wherein when the multi-source heterogeneous lidar simulation program is executed by a processor, it implements the steps of the multi-source heterogeneous lidar simulation method of any of the above embodiments.

[0073] This application stores a multi-source heterogeneous lidar simulation program on a computer-readable storage medium, wherein when the multi-source heterogeneous lidar simulation program is executed by a processor, it implements the steps of the multi-source heterogeneous lidar simulation method described above.

[0074] The method implemented when the multi-source heterogeneous lidar simulation program is executed can be referred to in the various embodiments of the multi-source heterogeneous lidar simulation method of this application, and will not be repeated here.

[0075] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0076] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0077] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0078] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0079] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0080] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0081] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A simulation method for multi-source heterogeneous lidar, characterized in that, The simulation method for multi-source heterogeneous lidar includes: Obtain the angle parameters of multiple lidars, and construct the total field of view of the multiple lidars based on the multiple angle parameters; An angle distribution table and an angle correspondence table are established based on the total field of view. The angle distribution table includes a set of all beam directions required to simulate multiple lidars, and the angle correspondence table includes the correspondence between each beam direction and each lidar. A simulation environment is created, and the angle distribution table is used as input. A single ray tracing algorithm is used to calculate the intersection information of each beam direction with the object in the simulation environment, so as to output a frame of mixed point cloud data. The mixed point cloud data is parsed according to the angle correspondence table to output multiple subsets of point cloud data corresponding to the lidar.

2. The simulation method for multi-source heterogeneous lidar according to claim 1, characterized in that, The step of acquiring the angle parameters of multiple lidars and constructing the total field of view of the multiple lidars based on the multiple angle parameters includes: Obtain the installation orientation angle, horizontal field of view, vertical field of view, horizontal angular resolution, and vertical angular resolution for each lidar. Based on the installation orientation angle, horizontal field of view angle, and vertical field of view angle of each lidar, the spatial distribution of the field of view of each lidar in the vehicle coordinate system is calculated through coordinate transformation. Based on the spatial distribution, the minimum outer envelope field of view angle covering all lidar fields of view is determined as the total field of view angle.

3. The simulation method for multi-source heterogeneous lidar according to claim 2, characterized in that, The establishment of the angle distribution table and angle correspondence table based on the total field of view includes: The horizontal and vertical ranges of the total field of view are used as boundaries; Generate an azimuth-elevation grid within the boundary according to a preset precision; Store the azimuth-elevation pairs of each point in the grid in order as an angle distribution table; Confirm the correspondence between each azimuth-elevation pair in the angle distribution table and the actual field of view of each lidar; Establish a mapping relationship between the azimuth-elevation angle pairs and their corresponding lidar identifiers; Store all mapping relationships as the angle correspondence table.

4. The simulation method for multi-source heterogeneous lidar according to claim 3, characterized in that, The preset accuracy is determined by finding the greatest common divisor of the horizontal and vertical angular resolutions of multiple lidars.

5. The simulation method for multi-source heterogeneous lidar according to claim 4, characterized in that, The simulation environment is created by using the angle distribution table as input and employing a single-ray ray tracing algorithm to calculate the intersection information of all beam directions with objects in the simulation environment, outputting a frame of mixed point cloud data, including: Read each beam direction in the angle distribution table sequentially; Rays are emitted from each lidar installation location, based on each beam direction; Mark the nearest intersection point between each ray and an object in the simulation environment; Record the point cloud data points at each intersection point to output a frame of mixed point cloud data; The hybrid point cloud data includes multiple point cloud data points, and each point cloud data point includes the three-dimensional coordinates, distance information, and reflection intensity information of each intersection point.

6. The simulation method for multi-source heterogeneous lidar according to claim 5, characterized in that, The step of parsing the mixed point cloud data according to the angle correspondence table to output multiple subsets of point cloud data corresponding to each LiDAR includes: Read multiple point cloud data points included in the mixed point cloud data; Based on the index position of each point cloud data point in the angle distribution table, find the corresponding LiDAR identifier in the angle correspondence table; Each point cloud data point is added to the point cloud data subset of its corresponding LiDAR.

7. The simulation method for multi-source heterogeneous lidar according to claim 1, characterized in that, After parsing the mixed point cloud data according to the angle correspondence table to output multiple subsets of point cloud data corresponding to the lidar, the process includes: Each subset of point cloud data is encapsulated according to the original data format of its corresponding LiDAR. Add device identifiers and data frame headers for multiple LiDARs; The information is transmitted via the LiDAR communication protocol to a designated port of the autonomous vehicle's perception system.

8. A multi-source heterogeneous lidar simulation device, characterized in that, The multi-source heterogeneous lidar simulation device includes: The acquisition module is used to acquire the angle parameters of multiple lidars and construct the total field of view of the multiple lidars based on the multiple angle parameters. A module is established to create an angle distribution table and an angle correspondence table based on the total field of view. The angle distribution table includes a set of all beam directions required to simulate multiple lidars, and the angle correspondence table includes the correspondence between each angle parameter and each lidar. The calculation module is used to create a simulation environment. Taking the angle distribution table as input, it uses a single ray tracing algorithm to calculate the intersection information of all beam directions with objects in the simulation environment, and outputs a frame of mixed point cloud data. The parsing module is used to parse the mixed point cloud data according to the angle correspondence table to output multiple subsets of point cloud data corresponding to each lidar.

9. A multi-source heterogeneous lidar simulation device, characterized in that, The multi-source heterogeneous lidar simulation device includes a processor, a memory, and a multi-source heterogeneous lidar simulation program stored in the memory and executable by the processor, wherein when the multi-source heterogeneous lidar simulation program is executed by the processor, it implements the steps of the multi-source heterogeneous lidar simulation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a multi-source heterogeneous lidar simulation program, wherein when the multi-source heterogeneous lidar simulation program is executed by a processor, it implements the steps of the multi-source heterogeneous lidar simulation method as described in any one of claims 1 to 7.

Images