Method for simulating operation process of lidar, and computing device

Abstract

The present disclosure provides a method for simulating the operation process of a lidar, and a computing device. The method for simulating the operation process of a lidar may comprise the following steps: setting parameters of a lidar; configuring a simulation scenario associated with the lidar; simulating, in the simulation scenario, energy changes of a laser light emitted by the lidar; and at least on the basis of the simulated energy changes, determining an echo signal of the emitted laser light received by the lidar.

Description

Method and computing device for simulating the working process of laser radar

[0001] This application claims priority to the Chinese patent application entitled “Method and computing device for simulating the working process of a lidar” and application number 202311205247.2 filed on September 18, 2023. The contents of the priority application are incorporated herein by reference in their entirety. Technical Field

[0002] The present disclosure relates to the field of laser radar, and in particular to simulation testing of laser radar detection. Background Art

[0003] With the widespread adoption and application of LiDAR in areas such as assisted driving and autonomous driving, the exploration of its performance has become crucial for the standardization and maturity of LiDAR products. During the design and development process, LiDAR must be tested to verify its performance and reliability during operation.

[0004] Traditional testing methods involve field testing, which requires setting up a field test setup for each test, resulting in high testing costs and low efficiency.

[0005] To quickly and efficiently verify the effectiveness of the LiDAR solution, a virtual simulation test of the LiDAR has been proposed. This simulation simulates the LiDAR's operating scenarios (for example, different lighting conditions, weather conditions, and the reflection mechanism of the laser on objects) and sets relevant operating parameters. This covers a wide range of test scenarios, thereby reducing testing costs and improving testing efficiency.

[0006] Chinese patent application CN 104268323 A discloses a ray-tracing-based LiDAR scene simulation solution. This solution uses ray tracing technology to calculate the reflection between laser light and the environment, simulating and emulating LiDAR detection at the laser light level. This ray-tracing-based simulation solution primarily focuses on the reflection of laser light (both direct and indirect), enabling simulation testing at the laser light level.

[0007] PCT patent application WO 2023 / 010540 A1 discloses a method, apparatus, device, and storage medium for verifying lidar scanning results. This method outputs corresponding simulated point cloud information by specifying a lidar model and simulation scenario. This solution focuses on generating simulated point clouds and performing simulation tests by invoking the specified model and scenario.

[0008] Therefore, there is a need in the art to efficiently and accurately provide simulation scenarios to achieve complete and accurate signal-level simulation of lidar detection.

[0009] Summary of the Invention

[0010] The present disclosure is provided in order to achieve a complete and accurate signal-level simulation solution for lidar detection.

[0011] According to one aspect of the present disclosure, a method is provided for simulating the working process of a laser radar, the method comprising: setting parameters of the laser radar; setting a simulation scene associated with the laser radar; simulating energy changes of laser emitted by the laser radar in the simulation scene; and determining an echo signal of the emitted laser received by the laser radar based at least on the simulated energy changes.

[0012] In the method as described above, setting the parameters of the laser radar includes: setting the energy distribution of the laser emitted by the laser radar.

[0013] A method as described in any of the above items, wherein the laser radar includes one or more channels, and setting the energy distribution of the laser emitted by the laser radar includes: setting the laser emitted by the channel to a number of rays with preset emission angles and energy ratios, and the energy ratio represents the ratio of the energy of each ray to the energy of the emitted laser.

[0014] A method as described in any of the above items, wherein determining the echo signal of the emitted laser received by the lidar includes: determining the echo signal of the light in the emitted laser based at least on the simulated energy change; and superimposing the determined echo signal of the light based at least on the energy ratio of the light in the laser emitted by the channel to generate a spot echo superposition signal.

[0015] A method as described in any of the above items, wherein setting a simulation scene associated with the laser radar includes: calling a scene and / or a material library generated by three-dimensional rendering as an element of the simulation scene.

[0016] A method as described in any of the above items, wherein, in the case of calling a material library, setting a simulation scene associated with the laser radar further comprises: randomly setting at least part of the parameters of at least one simulation scene based on the material library.

[0017] A method as described in any of the above items, wherein simulating the energy change of the laser emitted by the laser radar in the simulation scene includes: determining the energy change based at least on simulated weather factors.

[0018] The method as described in any one of the above items, wherein determining the energy change based at least on simulated weather factors includes: determining the energy change based at least on visibility changes under the simulated weather factors.

[0019] The method as described in any one of the above items, wherein the simulated weather factors include sunny days or foggy days.

[0020] A method as described in any of the above items, wherein determining the energy change based at least on simulated weather factors includes: simulating the distribution of particulate matter in the simulation scene under the simulated weather factors; and determining the energy change based at least on the energy loss when the emitted laser hits the particulate matter.

[0021] A method as described in any of the above items, wherein the simulated weather factors include rainy days, the particulate matter includes raindrops, and determining the energy change based at least on the energy loss when the emitted laser hits the particulate matter includes: simulating each light ray in the emitted laser to lose a specific proportion of energy when hitting the raindrops; and simulating the light ray that has suffered energy loss to continue to propagate after being randomly deviated by a certain angle.

[0022] The method as described in any one of the above items, wherein the simulated weather factors include snowy days or hazy days,

[0023] The particulate matter includes snow or haze particles, and determining the energy change based at least on energy loss when the emitted laser light hits the particulate matter includes simulating each ray of the emitted laser light as having at least partial absorption of energy when hitting the snow or haze particles.

[0024] A method as described in any of the above items, wherein setting a simulation scenario associated with the operation of the laser radar includes: setting the simulated weather factors.

[0025] A method as described in any of the above items, wherein simulating the energy change of the laser emitted by the laser radar in the simulation scene includes: determining the energy change based at least on the material of the object irradiated by the emitted laser.

[0026] A method as described in any of the above items, wherein setting a simulation scene associated with the operation of the laser radar includes: defining a model of a surface in the simulation scene.

[0027] The method as described in any one of the above items, wherein the surface model in the simulation scene includes a first type of surface, and the power of the emitted laser light reflected from the first type of surface is determined based on a bidirectional reflectance distribution function.

[0028] A method as described in any of the above items, wherein setting a simulation scene associated with the operation of the lidar includes: calling a scene generated by three-dimensional rendering, and defining a model of a surface in the simulation scene also includes: correcting the model parameters of the surface in the scene generated by the called three-dimensional rendering based on the determined power of the reflected light.

[0029] A method as described in any of the above items, wherein the surface model in the simulation scene includes at least one of a second type of surface, a third type of surface, or a fourth type of surface. The second type of surface includes a specular reflective surface. The third type of surface includes a transparent or translucent surface. The fourth type of surface includes a micro-prismatic structure or a micro-bead structure.

[0030] A method as described in any of the above items, wherein simulating the energy change of the laser emitted by the lidar in the simulation scene includes: determining the energy change based at least on the distance traveled by the emitted laser to hit the object and / or the angle of incidence of the object on which the emitted laser is incident.

[0031] The method as described in any one of the above items further comprises: performing an ambient light noise simulation.

[0032] A method as described in any of the above items, wherein setting a simulation scene associated with the operation of the laser radar includes: setting parameters of an ambient light source.

[0033] According to another aspect of the present disclosure, a device is provided, comprising: a device configured to set parameters of a laser radar; a device configured to set a simulation scene associated with the laser radar; a device configured to simulate energy changes of laser light emitted by the laser radar in the simulation scene; and a device configured to determine an echo signal of the emitted laser light received by the laser radar based at least on the simulated energy changes.

[0034] According to yet another aspect of the present disclosure, a computing device is provided, comprising: a processor; and a memory coupled to the processor, the memory comprising instructions, which, when executed, enable the processor to perform any one of the above methods.

[0035] According to yet another aspect of the present disclosure, a computer-readable storage medium is provided, which stores instructions, which, when executed, cause a computing device to perform any one of the above methods.

[0036] The present disclosure obtains the laser echo signal based on the energy change of the laser in the simulation scene, which can realize the signal-level simulation of the lidar detection, and has high simulation value for virtual road testing, perception algorithm training, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Various embodiments of the present disclosure are described with reference to the accompanying drawings, in which:

[0038] FIG1 illustrates a flowchart of an exemplary method for simulating the operation process of a lidar, consistent with some embodiments of the present disclosure.

[0039] FIG2 shows a diagram of an exemplary process for setting parameters of a lidar, consistent with some embodiments of the present disclosure.

[0040] FIG3 shows an exemplary graphical representation of energy distribution of laser light emitted by a lidar according to a channel setting consistent with some embodiments of the present disclosure.

[0041] FIG4 illustrates a flow chart of an exemplary process for determining an echo signal of emitted laser light received by a laser radar, consistent with some embodiments of the present disclosure.

[0042] FIG. 5 illustrates a diagram of a point cloud obtained by simulation according to methods consistent with some embodiments of the present disclosure involving an exemplary corner case.

[0043] FIG6 illustrates a diagram of a point cloud obtained by simulation according to methods consistent with some embodiments of the present disclosure involving another exemplary corner case.

[0044] 7 shows an illustration of a process for setting up an exemplary simulation scenario associated with the operation of a lidar, consistent with some embodiments of the present disclosure.

[0045] FIG8 shows a diagram of an exemplary process for customizing a simulation scenario consistent with some embodiments of the present disclosure.

[0046] FIG9 illustrates a flowchart of an exemplary process for randomly setting simulation scenarios consistent with some embodiments of the present disclosure.

[0047] FIG10 illustrates a diagram of an exemplary process of energy variation of laser light emitted by a simulated lidar in a simulated scene, consistent with some embodiments of the present disclosure.

[0048] 11 shows a diagram of an exemplary process for determining energy changes based on simulated weather factors, consistent with some embodiments of the present disclosure.

[0049] 12 shows a diagram of an exemplary process for determining energy changes based at least on energy loss of emitted laser light when it strikes particulate matter, consistent with some embodiments of the present disclosure.

[0050] 13 shows a diagram of an exemplary process for defining a model of a surface in a simulation scene, consistent with some embodiments of the present disclosure.

[0051] FIG. 14 illustrates a schematic structural diagram of an exemplary microprism simulating a high-reflective object consistent with some embodiments of the present disclosure.

[0052] FIG15 illustrates a block diagram of an exemplary computing device for simulating the operation of a lidar, consistent with some embodiments of the present disclosure.

[0053] FIG16 illustrates a block diagram of an exemplary non-volatile computer-readable storage medium for simulating the operation of a lidar, consistent with some embodiments of the present disclosure. DETAILED DESCRIPTION

[0054] While various embodiments are described herein in conjunction with the motion of an automobile, the scope of the present disclosure is not limited thereto. The embodiments of the present disclosure are applicable to simulating detection by a laser radar mounted on any other vehicle (e.g., an aircraft, other land vehicles, ships, etc.) and any other moving or non-moving object.

[0055] In this disclosure, the term "high retroreflectivity" refers to a condition in which a high proportion of the energy returned by light is reflected at the angle of incident light. For example, high retroreflectivity can refer to 50% or higher, 60% or higher, 70% or higher, 80% or higher, 90% or higher, 95% or higher, 99% or higher, or 100%. As an example, objects with high retroreflectivity may include traffic signs, warning strips, etc.

[0056] In the present disclosure, the expression "energy is at least partially absorbed" means 50% or more of the energy is absorbed, 60% or more of the energy is absorbed, 70% or more of the energy is absorbed, 80% or more of the energy is absorbed, 90% or more of the energy is absorbed, 95% or more of the energy is absorbed, 99% or more of the energy is absorbed, or 100% of the energy is absorbed.

[0057] In the present disclosure, the expression “substantially reflected as is” means that 90% or more of the incident light energy is reflected, for example, 95% or more is reflected, 99% or more is reflected, for example, completely reflected.

[0058] In the present disclosure, the terms "or" and "and / or" describe the association relationship between related objects and represent a non-exclusive inclusion. For example, "A and / or B" and "A or B" may include: only "A" exists, only "B" exists, and "A" and "B" exist at the same time, where "A" and "B" can be singular or plural. For another example, "A, B and / or C" and "A, B or C" may include: only "A" exists, only "B" exists, only "C" exists, "A" and "B" exist at the same time, "A" and "C" exist at the same time, "B" and "C" exist at the same time, and "A", "B" and "C" exist at the same time, where "A", "B" and "C" can be singular or plural. In addition, the symbol " / " in the present disclosure indicates that there is an "or" relationship between the related objects before and after the symbol. In the present disclosure, the term "at least one A or B" has the same meaning as the above-mentioned "A or B". The term "at least one A, B or C" has the same meaning as the above-mentioned "A, B or C".

[0059] According to one aspect of the present disclosure, a method for simulating a working process of a laser radar is provided.

[0060] FIG1 illustrates a flow chart of an exemplary method 100 for simulating the operation of a lidar, consistent with some embodiments of the present disclosure.

[0061] Method 100 may include step 110: setting parameters for a laser radar. Method 100 may include step 130: setting a simulation scene associated with the laser radar. Method 100 may include step 150: simulating energy variations of laser light emitted by the laser radar in the simulation scene. For example, the energy variations may be energy loss caused by interaction between the laser light emitted by the laser radar and the simulation scene. Method 100 may include step 170: determining an echo signal of the laser light emitted and received by the laser radar based at least on the simulated energy variations.

[0062] In some embodiments, after method 100 executes step 170, one or more of steps 110, 130, 150, and 170 may be repeatedly executed to update radar parameters and / or simulation scenes, thereby obtaining echo signals based on the updated radar parameters and / or simulation scene frames to obtain multiple simulation results of lidar detection.

[0063] By considering the energy changes during the interaction between the laser emitted by the lidar and the simulation scene in the simulation of the lidar detection process, the echo signal received by the lidar detector in the simulation environment can be simulated, thereby achieving signal-level simulation.

[0064] In some embodiments, the echo signal can be further processed to obtain simulated point cloud information, which can be used for virtual drive testing, perception algorithm training, etc.

[0065] For virtual drive testing applications, method 100 can partially or completely replace actual LiDAR road testing, identifying and resolving issues in advance. Compared to conventional automotive road testing, LiDAR's operating environment is affected not only by specific road surfaces and traffic conditions, but also by numerous other factors. Therefore, the challenge of road testing is that even with extensive testing, it's impossible to exhaust the combined impact of these numerous factors. Compared to conducting actual roadside tests in the field, applying method 100 to virtual drive testing applications offers greater efficiency and lower costs, providing high simulation value.

[0066] For perception algorithm training applications, method 100 can quickly and massively acquire high-precision simulated point cloud information, thereby increasing the diversity of training data and making the perception model more extensible and easier to identify uncommon situations (e.g., corner cases).

[0067] Furthermore, because method 100 can perform signal-level simulation of lidar echo signals, simulated point cloud information can be further derived from this simulated echo signal in a variety of ways. For example, the obtained echo light pulse and channel ambient light intensity can be used as inputs into a simulation model of a photoelectric sensor to generate a digital model. Simulated point cloud information can then be generated using existing lidar data processing algorithms / hardware. Similarly, method 100 can also be used to implement hardware-in-the-loop simulation of processing hardware used to process lidar data.

[0068] FIG2 shows a diagram of an exemplary process 200 for setting parameters of a lidar consistent with some embodiments of the present disclosure. The process 200 may include processes similar to those involved in step 110 of the method 100 shown in FIG1 .

[0069] In some embodiments, process 200 includes step 210: selecting / defining a lidar model. In some embodiments, step 210 may include selecting a lidar model, for example, selecting a brand, model, etc. of the lidar. Alternatively, process 200 may include customizing a lidar model. As an example, the parameters of the definable lidar model may include: the number of detection channels, the angular information of one or more detection channels, the position of one or more detection channels at which light is emitted in the lidar coordinate system, the order in which one or more detection channels emit light and the corresponding time intervals, the detection range / capability of one or more detection channels at a specific reflectivity, the pulse waveform of the emitted laser, etc.

[0070] In some embodiments, process 200 includes step 230: setting the installation parameters of the LiDAR. For example, for an application scenario where the LiDAR is mounted on a vehicle, step 230 may include determining the installation position of the LiDAR relative to the vehicle, the angle of the installation plane, etc.

[0071] In some embodiments, process 200 includes step 250: setting the energy distribution of the laser emitted by the laser radar. In some embodiments, the laser radar includes one or more channels, and step 250 includes step 255: setting the laser emitted by a channel to a plurality of rays with preset angles and energy ratios, wherein the energy ratio represents the ratio of the energy of each ray to the energy of the laser emitted by the laser radar. As an example, the laser of each channel can be described as a series of rays forming a specified width / cross-sectional area, and the ratio of the energy carried by each ray can be stored accordingly (for example, stored in a matrix). The implementation of step 255 will be further described below in conjunction with Figure 3.

[0072] Through step 255, light rays in the same channel with associated preset angles and energy ratios are stored in a correlated manner. This facilitates subsequent correlated processing of the echo signals generated by these light rays, making the simulation results closer to the actual laser spot, and thus helping to discover corner cases that are difficult to find during actual lidar testing. This will be further described below in conjunction with Figures 4-6.

[0073] In an embodiment, process 200 may include one or more of steps 210 , 230 , and 250 .

[0074] FIG3 shows an exemplary graphical representation 300 of energy distribution of laser light emitted by a lidar according to a channel configuration, consistent with some embodiments of the present disclosure.

[0075] Graphical representation 300 shows that the laser light emitted by each channel of the lidar can be associated with several regions, which are represented by different legends in Figure 3. As an example, these regions may include strip-shaped regions 310, 320, and 330 from the outside to the inside, and a central region 340. The central region 340 may further include rectangular regions 341, 342, 343, 344, 345, and 346 from the center to the sides. The above-mentioned regional divisions in Figure 3 are merely exemplary.

[0076] The laser beams emitted by each channel can be associated with specific regions based on their emission angles, and the energy ratio of the beams within each region can be set to be the same or different. For example, the energy ratio of the beams corresponding to the region near the center is higher, while the energy ratio decreases towards the sides or periphery. This allows the beams within each region to be stored in an associated manner, and these associated beams can be further superimposed. When irradiated onto a surface in the simulated scene, a light spot with a specified width / cross-sectional area is generated. Subsequent processing steps can essentially be based on the echo signals of the light spots, which is closer to the actual detection scenario of the lidar.

[0077] As an example, the energy ratio of light at the corresponding position may be set based on the position defined by the horizontal coordinate and the vertical coordinate in FIG. 3 , thereby storing the energy ratio of the laser light emitted by each channel as an energy distribution matrix.

[0078] FIG4 illustrates a flow chart of an exemplary process for determining an echo signal of emitted laser light received by a laser radar, consistent with some embodiments of the present disclosure.

[0079] Process 400 may be similar to the process involved in step 170 of method 100 shown in FIG. 1 , and may be directed to a process including step 255 of process 200 in FIG. 2 .

[0080] Process 400 includes step 410 of determining an echo signal of light in the emitted laser light based at least on the simulated energy variation.

[0081] The process 400 includes step 430 : superimposing the determined echo signals of the light based on at least the energy ratio of the light in the laser light emitted by the channel to generate a spot echo superposition signal.

[0082] By superimposing the echo signals of the associated laser light, such as those set in step 255 described above, to generate a spot echo superposition signal, compared with the light-based simulation scheme for lidar detection, it can effectively reflect and identify corner cases, thereby enhancing the value of virtual testing.

[0083] As an example, the pulse function of a single light ray can be expressed by the following equation (1): i (t) = m i *F(t) (1)

[0084] Among them, m i It is the ratio of the energy of the light to the energy of the laser beam emitted in the channel where the light is located, which can be the energy ratio of the light set in step 255 described above in conjunction with Figure 2; F(t) is the complete light pulse waveform function of the laser beam emitted by the corresponding channel.

[0085] Assume that the peak power of the laser beam is For a linear system, the peak power of the echo is Among them, K i is the attenuation coefficient of the return power of the light. Accordingly, the echo pulse corresponding to the light can be derived by the following formula (2): i (t) = m i K i *F(t-2L i / C) (2)

[0086] Among them, t is the time for the light to travel, L i is the distance traveled by the light ray after it hits the object, and c is the speed of light in a vacuum. The total echo pulse corresponding to each light ray in the channel can be obtained by calculating the echo pulse corresponding to each light ray in the channel according to formula (2) and superimposing the calculated results.

[0087] As an example, a simplified algorithm can first obtain the unit peak input pulse Its peak power is P t =1, and then calculate the probability distribution function of the peak power of all light with respect to the distance L, and Performing a convolution operation with this function yields the returned optical waveform signal. This completes the signal-level simulation of the echo signal that the lidar receiver module can receive.

[0088] By considering the impact of the laser spot echo signal on the simulation results, we can effectively reflect some of the corner cases that are of interest in LiDAR simulation testing. These corner cases may include high reflection broadening, drag points, and missing sidewalks.

[0089] The effect of the light spot on the characteristics of the point cloud is affected by many factors. For example, the effect of the light spot on the characteristics of the point cloud is affected by the detection distance. For example, in the case of close-range detection, the light spot is small and the effect is small; while in the case of long-range detection, the light spot is large and the effect is large. Therefore, different results can be seen from the point cloud for close-range detection and long-range detection. For example, the effect of the light spot on the characteristics of the point cloud is affected by the reflectivity of the object. For example, some features related to high back reflectivity will also produce different point cloud results. For example, the size of the incident angle has an impact on the point cloud results. For example, for ground lines with oblique incidence of laser light, the bending and disappearance of zebra crossings are all related to the size of the incident angle.

[0090] For point cloud training, without considering the spot size, the point cloud results obtained by incident on objects with different reflectivity at different distances and angles may be similar. In contrast, when considering the spot size, the point cloud results obtained will be different, depending on the definition of the spot (for example, the setting of the associated light), and this case is closer to the real lidar detection situation, because the real lidar emits a spot with a specified width / cross-section.

[0091] FIG. 5 shows an illustration 500 of a point cloud simulated according to methods consistent with some embodiments of the present disclosure involving an exemplary corner case.

[0092] Diagram 500 shows corner case portions 510 and 530 associated with high anti-broadening. Corner case portion 510 shows a point cloud 512 corresponding to an obstacle and point clouds 514 and 516 at its edges. Corner case portion 530 shows a point cloud 531 corresponding to a street sign and point cloud 534 at its edges.

[0093] For a laser spot of a certain size (e.g., width / cross-sectional area) emitted by a lidar, although the center of the channel may not cover an object with a high back reflectivity, its edge may cover the object with a high back reflectivity. Although the energy of the light at the edge of the spot is weaker than that of the light at the center, given that objects with a high back reflectivity have a higher back reflectivity, the light at the edge of the spot can still reflect back some of the energy so that it can be received by the lidar detector. In this case, the center of the channel generates the correct point cloud result corresponding to the channel, while the light at the edge of the channel generates an erroneous point cloud result (i.e., noise in the point cloud result), resulting in point cloud expansion for objects with a high back reflectivity, which is also called high back reflectivity broadening. Compared to the simulation process that does not consider the spot size, by considering the influence of the echo signal of the laser in the form of a spot on the simulation results, the high back reflectivity broadening phenomenon of concern in the lidar simulation test can be effectively reflected, thereby improving the accuracy of the simulation of lidar detection.

[0094] FIG. 6 shows an illustration 600 of a point cloud obtained by simulation according to methods consistent with some embodiments of the present disclosure, relating to another exemplary corner case.

[0095] Diagram 600 illustrates an example of corners associated with zebra crossing jitter. For zebra crossings on the road that are farther from the lidar, the incident angle of the laser emitted by the lidar increases, elongating the light spot and reducing measurement accuracy. For zebra crossings on the road, if the laser spot emitted by the lidar covers both the zebra crossing and the adjacent ground, the difference in reflectivity between the zebra crossing and the adjacent ground causes the majority of the light spot's echo energy to be concentrated at the zebra crossing, resulting in zebra crossing jitter.

[0096] 7 shows a diagram of a process 700 for setting up an exemplary simulation scenario associated with the operation of a lidar, consistent with some embodiments of the present disclosure. The process 700 can be similar to the process involved in step 130 of the method 100 shown in FIG1 .

[0097] In some embodiments, process 700 includes step 710: customizing a simulation scenario. The following will describe the steps involved in customizing a simulation scenario in conjunction with FIG8 .

[0098] In some embodiments, process 700 includes step 730: randomly setting simulation scenarios. The steps involved in randomly generating simulation scenarios are described below in conjunction with FIG9 . By simulating lidar detection in a variety of randomly set simulation scenarios, it is possible to simulate a variety of possible application environments, thereby facilitating the discovery of edge cases that are difficult to detect during actual lidar detection testing.

[0099] In some embodiments, process 700 may include customizing a portion of parameters of the simulation scenario and randomly setting a portion of parameters of the simulation scenario.

[0100] FIG8 illustrates a diagram of a process 800 for customizing an exemplary simulation scenario consistent with some embodiments of the present disclosure. Process 800 may be similar to the process involved in step 710 of process 700 shown in FIG7 .

[0101] In some embodiments, process 800 includes step 810: selecting a scene. As an example, a scene map can be selected from a map library. Alternatively, a map can be modified first and then the modified map can be selected.

[0102] In some embodiments, process 800 includes step 820: calling the scene and / or material library generated by three-dimensional rendering. By calling the scene generated by three-dimensional rendering and only performing simple parameter settings on a few scenes, the model in the simulation scene is made applicable to the road scene resources of the current autonomous driving platform, thereby saving the work of constructing a three-dimensional simulation work scene by oneself and reducing the difficulty of obtaining virtual scenes for virtual road test applications. In addition, since the scene generated by three-dimensional rendering can directly generate photos from the camera's perspective, calling the existing scene generated by three-dimensional rendering and simulating lidar detection based on this can generate simulated point cloud information and simulated visual information based on the same virtual scene for the autonomous driving test level, which is conducive to multi-sensor fusion simulation testing.

[0103] In some embodiments, process 800 includes step 830: setting a motion trajectory and motion state of the laser radar in the scene. In some embodiments, setting the motion trajectory may include setting automatic tracking driving. As examples, the configurable motion states may include turning, climbing, accelerating, decelerating, etc.

[0104] In some embodiments, process 800 includes step 840: setting motion parameters of a dynamic object (e.g., a person, a car, etc.) in the scene. As an example, the trajectory of the moving object may be set. Alternatively, the motion parameters for autonomous driving may be set.

[0105] In some embodiments, process 800 may include step 850: setting simulated weather factors, such as sunny, foggy, rainy, snowy, or hazy weather.

[0106] On clear days, it can be defined that laser energy propagation through the air is not affected by weather factors. On foggy days, it can be defined that laser energy propagation through the air is affected by visibility. On rainy days, raindrops can be simulated uniformly or randomly in three-dimensional space, and it can be defined that the laser light will lose a specific proportion of energy when it encounters raindrops during propagation, and then continue to propagate after being randomly deflected by a certain angle. On snowy, hazy, or other weather conditions containing opaque particles, snow, haze particles, or other opaque particles can be simulated in three-dimensional space, and it can be defined that the laser light's energy is at least partially absorbed when it encounters them.

[0107] In some embodiments, process 800 includes step 860: defining a model of a surface in the simulation scene. The definition of various surface models will be further described below in conjunction with FIG.

[0108] In some embodiments, process 800 may include step 870: setting parameters of an ambient light source. By setting the parameters of the ambient light source, a noise simulation of the ambient light may be performed.

[0109] In reality, the illuminance value measured against the direction of the sun is often used to describe the overall level of ambient light in the current scene. However, due to the effects of the atmosphere's non-uniform reflection and refraction of sunlight, multiple reflections after sunlight hits objects, shadows, and other effects, in actual scenes, the amount of ambient light received by each channel and angle of the lidar is different. In the scene of the rendering engine, the illuminance value of sunlight can be set as the overall input, and the reflection and refraction parameters of the sky light can be set so that the final scene rendering color is obtained after the rendering engine is run. Accordingly, a mapping relationship can be constructed between the final rendering color and the base color of the object, the illuminance on the object, and the camera exposure. The current illuminance value of the object is calculated by obtaining the rendering color, the base color, and the current exposure. There is a specific relationship (for example, a linear relationship) between the calculated current channel illuminance value and the effective value of the ambient light noise received by the channel. Based on this relationship, the echo signal determined in step 170 of method 100 in Figure 1 can be adjusted to further determine a more realistic simulation result.

[0110] In an embodiment, process 800 may include one or more of steps 810 to 870. In some embodiments, by fully considering and traversing weather factors and / or surface and / or ambient light noise that may be encountered in real testing in the simulation of lidar detection, more accurate and realistic simulation results can be obtained, thereby effectively improving the value of virtual testing.

[0111] FIG9 is a flowchart illustrating a process 900 of randomly setting a simulation scenario according to an embodiment of the present disclosure. The process 900 may be the process involved in step 730 of the process 700 shown in FIG7 .

[0112] In some embodiments, process 900 includes step 910: calling a material library. By directly calling the material library, the setting of the simulation scene can be simplified, thereby improving the efficiency of simulating the laser radar detection.

[0113] In some embodiments, process 900 includes step 930: randomly generating at least some parameters of at least one simulation scenario based on the material library. Randomly generating at least some parameters of the simulation scenario facilitates discovering edge cases that are difficult to detect during actual lidar detection testing, thereby enhancing the value of virtual testing.

[0114] FIG10 illustrates a process 1000 for simulating energy variations of laser light emitted by a laser radar in a simulated scene, consistent with some embodiments of the present disclosure. The process 1000 may be similar to the process involved in step 150 of the method 100 shown in FIG1 .

[0115] In some embodiments, process 1000 includes step 1010 of determining energy changes based on simulated weather factors. Step 1010 may be performed in conjunction with step 850 in FIG. 8 above and will be further described below in conjunction with FIG. 11-FIG .

[0116] In some embodiments, process 1000 includes step 1030: determining energy variation based on the material of the object irradiated by the emitted laser light. Step 1010 can be performed in conjunction with step 860 in FIG. 8 above and will be further described below in conjunction with FIG. 13 .

[0117] In some embodiments, process 1000 includes step 1050 of determining an energy change based on the distance traveled by the emitted laser light before striking an object. For example, the energy change can be determined based on a general attenuation pattern of laser light energy in a typical environment. For example, the power of laser light may decay inversely or inversely square with propagation distance.

[0118] In some embodiments, process 1000 includes step 1070 of determining an energy change based on an angle of incidence of the emitted laser light on the object.

[0119] In an embodiment, process 1000 may include one or more of steps 1010, 1030, 1050, and 1070. As an example, the emitted laser power P t and the received laser power P r The following relationship (3) can be satisfied: r =KP t (3)

[0120] Where K is the attenuation coefficient of laser power.

[0121] As an example, K may be a function of one or more of the factors involved in the above steps 1010, 1030, 1050, and 1070, as shown in the following formula (4): K = f(k 天气 , k 材质 , k 距离 , k 入射角 ) (4)

[0122] Among them, k 天气 is the attenuation coefficient of laser power caused by weather factors, k 材质 is the attenuation coefficient of the laser power caused by the material of the object irradiated by the laser, k 距离 is the attenuation coefficient of the laser power due to the distance the laser travels to hit the object, k 入射角 It is the attenuation coefficient of the laser power caused by the incident angle of the laser into the object.

[0123] As an example, K may be the product of the above-mentioned attenuation coefficients, as shown in the following equation (5), but the scope of the present disclosure is not limited thereto: K=k 天气 k 材质 k 距离 k 入射角 (5)

[0124] 11 shows a diagram of an exemplary process 1100 for determining energy changes based on simulated weather factors, consistent with some embodiments of the present disclosure. Process 1100 can be similar to the process involved in step 1010 of process 1000 shown in FIG.

[0125] In some embodiments, process 1100 includes step 1110 of determining an energy change based at least on a visibility change under simulated weather factors.

[0126] In some embodiments, the simulated weather factors may include a sunny day. On a sunny day, it can be considered that the power of the laser only decays with the propagation distance without further decay caused by weather factors.

[0127] In some embodiments, the simulated weather factors may include foggy weather. For foggy weather, the attenuation of the simulated laser pulse signal can be considered as the attenuation acting uniformly on each channel. As an example, the power of the laser after propagating a certain distance can be calculated according to the following formula (6):

[0128] Where P0 is the laser emission power, d0 is the visibility on an ideal sunny day, d is the visibility in the current weather (foggy day), D is the distance the laser propagates (here, D is the one-way distance of the laser propagation, and the actual round-trip propagation distance of the laser is 2D), and k is a constant related to the wavelength of the selected laser.

[0129] The term in formula (6) It can reflect the attenuation of laser power caused by weather factors. It can be combined with the term k in formula (5) 天气 For example, for an ideal sunny day, d = d0, that is, k 天气 =1, that is, there is no further attenuation caused by weather factors. The term in formula (6) It can reflect the attenuation of laser power with propagation distance. As an example, It can be combined with the term k in formula (5) 距离 correspond.

[0130] In some embodiments, process 1100 includes step 1130 related to particulate matter under simulated weather conditions.

[0131] Step 1130 may include step 1132: simulating the distribution of particulate matter in the simulated scene under simulated weather conditions. For example, the particulate matter may be raindrops, snow, haze particles, or other opaque particulate matter. For example, the particulate matter may be simulated to appear randomly in three-dimensional space and occupy a specific space.

[0132] For example, for a raindrop, the expected probability E of its occurrence at a specific location in three-dimensional space can be proportional to the set unit precipitation amount and the simulation time step. Alternatively, raindrops can be made to appear according to a set distribution, with the number of raindrops appearing per unit space set to be proportional to the unit precipitation amount and the simulation time step.

[0133] As an example, for snow, haze particles or other opaque particles, the size of the particles can be set and generated in three-dimensional space in a similar manner to the generation of raindrops described above.

[0134] Step 1130 may include step 1134: determining an energy change based at least on energy loss when the emitted laser light strikes the particles. Step 1134 will be further described below in conjunction with FIG.

[0135] FIG12 shows an example process 1200 for determining energy change based at least on energy loss when emitted laser light strikes particles, consistent with some embodiments of the present disclosure. The process 1200 shown in FIG12 can be similar to the process involved in step 1134 of the process 1100 shown in FIG11.

[0136] In some embodiments, the simulated weather may include rain, and the associated particulate matter may be raindrops. Accordingly, process 1200 may include step 1210: simulating each day's emitted laser light to lose a specific percentage of its energy upon striking a raindrop. Process 1200 may also include step 1230: simulating the light that has experienced energy loss to continue propagating after being randomly deflected by a certain angle. Thus, steps 1210 and 1230 may be repeated each time the light encounters a raindrop until the light strikes a surface.

[0137] In some embodiments, the simulated weather may include snow or haze, and the associated particulate matter may be snow or haze particles. Accordingly, process 1200 may include step 1250: simulating each ray of the emitted laser as having its energy at least partially absorbed when hitting the snow or haze particles. For this simulated weather, the energy of the laser light that is not absorbed when hitting the particulate matter accounts for a low proportion, and its continued propagation may be chosen to be considered or not in the actual simulation of the lidar detection. In some cases, although the calculation becomes complicated when considering the continued propagation of the laser light that is not absorbed by the particulate matter in this simulated weather, the corner cases (e.g., heavy snow) can be reflected through simulation.

[0138] FIG13 shows an illustration of an exemplary process 1300 for defining a model of a surface in a simulation scene, consistent with some embodiments of the present disclosure. The process 1300 shown in FIG13 may be similar to the process involved in step 860 of the process 800 shown in FIG8.

[0139] In some embodiments, the model of the surface in the simulation scene includes a first type of surface. In some embodiments, the first type of surface includes a rough, opaque surface. Process 1300 includes step 1310 of determining the power of light reflected from the first type of surface by the emitted laser light using a bidirectional reflectance distribution function (BRDF).

[0140] The BRDF model is a basic model used in graphics algorithms to describe light reflection phenomena. It is applicable to general rough opaque surfaces. For such a surface, to calculate the echo energy / power of the laser reflected from the surface, the power irradiated on the surface is P i The power P of the light received by the laser radar can be obtained by derivation. r It can be determined by the following formula (7): r =Ωf BRDF (θ i )cosθ i P i (7)

[0141] Where Ω is the solid angle, f BRDF The term can be defined as the following formula (8):

[0142] Where L is the luminance of the emitted light, and its unit is lm / (sr·m 2 )(lumens / (steradian·m 2 )), E is the incident light illuminance, its unit is lm / m 2 ,θ i is the angle of incidence of the light ray on the surface. BRDF The unit of the term is sr -1 , that is, the ratio of reflected power per unit solid angle. BRDF The result of multiplying by the solid angle Ω represents the reflection ratio of the incident light power on the surface within a specific solid angle direction. r The solid angle of the laser radar / lidar receiver channel corresponding to the reflecting surface at distance L can be determined according to the following equation (9):

[0143] The power P of the laser light finally received by the laser radar r It can be determined according to the following formula (10):

[0144] From the above formula (9), it can be seen that in addition to the power attenuation caused by the laser propagating in the air, the power of the reflected laser light received by the laser radar is also related to the distance between the object and the laser radar, the angle at which the laser light is incident on the surface of the object, and f BRDF Here, f BRDF Items are items associated with the material properties of an object.

[0145] In some embodiments, step 1310 may include step 1315: modifying model parameters of surfaces in the scene generated by the invoked 3D rendering based on the determined power of the reflected light. Step 1315 may be performed in conjunction with step 820 in process 800 described above in conjunction with FIG.

[0146] In the simulation scene generated by 3D rendering, the model called by the mainstream rendering engine usually contains the material information of each surface, which can be obtained through f BRDF The item is associated with the material information used in the rendering operation, and then the power of the light finally received by the laser radar corresponding to the specific surface can be calculated through the above formula (10). Therefore, the scene and / or material library generated by the 3D rendering can be directly called as an element of the simulation scene for the simulation process of the laser radar detection. Since the 3D rendering process usually does not consider the physical conservation of energy, in some embodiments, when using the BRDF model, the relevant parameters can be corrected to ensure that the energy of the reflected light in the model is conserved with the energy of the incident light and the energy of the absorbed light, so as to accurately reflect the interaction process between the laser and the surface.

[0147] By utilizing the BRDF model in combination with some customized models, the scenes generated by existing three-dimensional rendering can be used for virtual road testing after simple processing, reducing the difficulty of obtaining virtual scenes.

[0148] In addition, using a BRDF-based model for simulation can adapt to the materials of current mainstream rendering engines (for example, Unity, Unreal Engine, etc.), which can save the work of building a three-dimensional simulation scene by yourself.

[0149] In addition to simulating general rough, opaque objects using the above-mentioned BRDF model, in some embodiments, process 1300 may also include defining other special surfaces.

[0150] In some embodiments, the special surface includes a second type of surface, which includes a specular reflective surface. Process 1300 may include step 1330: simulating the second type of surface as a specular reflective surface that reflects light substantially at its original energy when it strikes it. For an opaque mirror, when laser light strikes the surface, it is reflected at a specular angle substantially at its original energy. The laser light changes its propagation direction at the point of incidence and continues to propagate, while other aspects, such as the laser energy, remain substantially unchanged.

[0151] In some embodiments, the special surface includes a third type of surface, and the third type of surface includes a transparent or translucent surface. Process 1300 may include step 1350: simulating the third type of surface as a surface with transmission and specular reflection when light is irradiated. For a transparent or translucent surface, when the laser is irradiated on the surface during simulation, a portion of the light continues to pass through (for example, for window glass). The energy transmitted can be determined by dividing the energy of the incident light by the refractive index of the object material (or in the rendering material, transparency can also be used). As an example, if the surface of such an object is smooth (for example, a water surface, a puddle, etc.), the remaining energy is reflected out as specular reflection.

[0152] In some embodiments, the special surface includes a fourth type of surface, and the fourth type of surface includes a micro-prismatic structure or a micro-bead structure. The process 1300 includes step 1370: simulating the surface of the object with high retro-reflectivity as a micro-prismatic structure or a micro-bead structure.

[0153] High-reflectivity objects have an extremely high reflectivity, which manifests as extremely high energy returned at the angle of incident light (as distinct from specular reflection). This characteristic can affect LiDAR detection results. Examples of high-reflectivity objects include traffic signs and warning signs. These high-reflectivity objects cannot be considered individual surfaces; they are typically micro-prismatic or micro-bead structures.

[0154] The microprism structure may include many mutually perpendicular mirrors, an exemplary structure of which is shown in FIG14 . If the object itself has color, it is also necessary to take the average value of its color channel or the value B of the red channel (in some embodiments, the value can be normalized to the interval [0, 1]) and determine the power P of the reflected laser light according to the following formula (11): out : P out =cos 2 θ i BP in (11)

[0155] Among them, θ i is the angle of incidence of the light incident on the surface of the object, P in is the power of the incident light.

[0156] For light incident in a certain direction, the microprism structure can make most of the energy return along the original path with the reflected light. This is because the light passes through a layer of glass or transparent plastic (used to converge the light) before entering the microprism structure, and the energy of this part of the reflection increases with the increase of the incident angle, which is in line with the Fresnel principle, as shown in the following equations (12)-(15):

[0157] Taking the Schlick approximation, we have: R(θ)=R0+(1-R0)(1-cosθ) 5 (16)

[0158] In the above formulas (12) to (17), η i is the refractive index of the incident medium (for air, it is assumed to be 1), η t is the refractive index of glass or transparent plastic (greater than 1), then equation (17) becomes the following equation (18):

[0159] When cosθ=1 (ie, normal incidence), the following equation (19) is obtained: out =BP in =R(θ=0)BP in (19)

[0160] For other incident angles, the following equation (20) is obtained: out (θ)=R(θ)BP in (20)

[0161] FIG. 14 illustrates a schematic structural diagram of an exemplary microprism simulating a high-reflective object consistent with some embodiments of the present disclosure.

[0162] The upper diagram in Figure 14 shows the situation where light is incident on the surface of the micro-prism structure. It can be seen that the light is reflected in a direction perpendicular to the incident surface substantially according to the original energy.

[0163] The lower figure in Figure 14 shows the situation where light is incident obliquely on the surface of the micro-prism structure. It can be seen that the light is reflected in a direction parallel to the incident direction but opposite to the incident direction, basically according to the original energy.

[0164] According to another aspect of the present disclosure, a computing device is provided, which may include devices each configured to execute each step of the method described above in conjunction with FIG. 1 to FIG. 14 .

[0165] According to yet another aspect of the present disclosure, a computing device is provided.

[0166] FIG15 illustrates a block diagram of an exemplary computing device 1500 for simulating the operation of a lidar, consistent with some embodiments of the present disclosure.

[0167] The computing device 1500 may include a processor 1510 and a memory 1530. The memory 1530 is coupled to the processor 1510. The memory 1530 may include instructions 1550 that, when executed, cause the processor 1510 to perform the methods described above in conjunction with FIG. 1-FIG . 14.

[0168] According to yet another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided.

[0169] FIG16 illustrates a block diagram of an exemplary non-volatile computer-readable storage medium 1600 for simulating the operation of a lidar, consistent with some embodiments of the present disclosure.

[0170] The computer-readable storage medium 1600 may include instructions 1650. When executed, the instructions 1650 may cause a computing device to perform the method described above in conjunction with Figures 1-14.

[0171] Non-limiting embodiments of the present disclosure have been described with reference to the accompanying drawings, with the embodiments being presented by way of illustration and not limitation.

Claims

1. A method for simulating the working process of a laser radar, comprising: Set the parameters of the laser radar; Setting a simulation scene associated with the laser radar; Simulating energy changes of lasers emitted by the laser radar in the simulation scene; as well as Based at least on the simulated energy variation, an echo signal of the emitted laser light received by the laser radar is determined.

2. The method according to claim 1, characterized in that The parameters of the laser radar are set as follows: Set the energy distribution of the laser emitted by the laser radar.

3. The method according to claim 2, characterized in that The laser radar includes one or more channels; Setting the energy distribution of the laser emitted by the laser radar includes: The laser emitted by the channel is set to be a plurality of light rays with a preset emission angle and energy ratio, wherein the energy ratio represents the ratio of the energy of each light ray to the energy of the emitted laser.

4. The method according to claim 3, characterized in that Determining the echo signal of the emitted laser received by the laser radar comprises: determining an echo signal of light in the emitted laser light based at least on the simulated energy variation; and At least based on the energy ratio of the light in the laser emitted by the channel, the determined echo signals of the light are superimposed to generate a spot echo superposition signal.

5. The method according to claim 1, characterized in that Setting up a simulation scenario associated with the laser radar includes: The scene and / or material library generated by three-dimensional rendering is called as an element of the simulation scene.

6. The method according to claim 5, characterized in that In the case of calling the material library, setting the simulation scene associated with the laser radar further includes: Based on the material library, at least part of the parameters of at least one simulation scene are randomly set.

7. The method according to claim 1, characterized in that Simulating the energy change of the laser emitted by the laser radar in the simulation scene includes: The energy change is determined based at least on simulated weather factors.

8. The method according to claim 7, characterized in that Determining the energy change based at least on simulated weather factors includes: The energy change is determined based at least on a visibility change under the simulated weather factor.

9. The method according to claim 8, characterized in that The simulated weather factors include sunny days or foggy days.

10. The method according to claim 7, characterized in that Determining the energy change based at least on simulated weather factors includes: Simulating the distribution of particulate matter in the simulation scene under the simulated weather factors; The energy change is determined based at least on energy loss of the emitted laser light when it strikes the particle.

11. The method according to claim 10, characterized in that The simulated weather factors include rainy days, The particulate matter includes raindrops, and The energy is determined based at least on the energy loss of the emitted laser when it hits the particle. Changes include: Simulating each ray of the emitted laser light to lose a specific fraction of its energy when striking the raindrop; and The light that has suffered energy loss is simulated as continuing to propagate after being randomly deflected by a certain angle.

12. The method according to claim 10, characterized in that The simulated weather factors include snowy days or hazy days. The particulate matter includes snow or haze particles, and Determining the energy change based at least on the energy loss of the emitted laser light when it hits the particle comprises: Each ray of the emitted laser light is simulated to have its energy at least partially absorbed when it strikes the snow or haze particles.

13. The method according to claim 8, characterized in that Setting up the simulation scenario associated with the operation of the LiDAR includes: Sets the weather factors for the simulation.

14. The method according to claim 1, wherein: Simulating the energy change of the laser emitted by the laser radar in the simulation scene includes: The energy change is determined based at least on the material of an object irradiated by the emitted laser light.

15. The method according to claim 14, characterized in that Setting up the simulation scenario associated with the operation of the LiDAR includes: A model is defined for a surface in the simulation scene.

16. The method according to claim 15, characterized in that The model of the surface in the simulation scene includes a first type of surface; The power of the emitted laser light reflected from the first type of surface is determined based on the bidirectional reflectance distribution function.

17. The method according to claim 16, characterized in that Setting up the simulation scenario associated with the operation of the LiDAR includes: Calling the scene generated by 3D rendering, and The model defining the surface in the simulation scene also includes: Model parameters of surfaces in a scene generated by the invoked three-dimensional rendering are modified based on the determined power of the reflected light.

18. The method according to claim 15, characterized in that The model of the surface in the simulation scene includes at least one of a second type of surface, a third type of surface, or a fourth type of surface; The second type of surface comprises a specular reflective surface; The third type of surface includes a transparent or translucent surface; The fourth type of surface includes a micro-prismatic structure or a micro-bead structure.

19. The method according to claim 1, wherein: Simulating the energy change of the laser emitted by the laser radar in the simulation scene includes: The energy change is determined based at least on a distance traveled by the emitted laser light to strike an object and / or an angle of incidence of the emitted laser light to the object.

20. The method of claim 1, wherein: Also includes: Performs an ambient light noise simulation.

21. The method of claim 20, wherein: Setting up the simulation scenario associated with the operation of the LiDAR includes: Sets the parameters of the ambient light source.

22. A device comprising: A device configured to set parameters of a laser radar; A device configured to set a simulation scene associated with the laser radar; A device configured to simulate energy changes of laser light emitted by the laser radar in the simulation scene; as well as A device configured to determine an echo signal of the emitted laser light received by the laser radar based at least on the simulated energy change.

23. A device comprising: processor; as well as A memory is coupled to the processor, the memory comprising instructions, which when executed cause the processor to perform the method as claimed in any one of claims 1 to 21.

24. A non-transitory computer-readable storage medium storing instructions which, when executed, cause a computing device to perform the method of any one of claims 1-21.

Images