Method and computing device for simulating a working process of lidar

Abstract

The invention discloses a method for simulating a working process of a laser radar and computing equipment. The method for simulating the working process of the laser radar comprises the following steps: setting parameters of the laser radar; setting a simulation scene associated with the work of the laser radar; simulating the energy change of the laser emitted by the laser radar in the simulation scene; and determining an echo signal of the emitted laser received by the laser radar at least based on the simulated energy change.

Description

Technical Field

[0001] The present invention generally relates to the field of laser radar, and in particular to simulation testing of laser radar detection. Background Art

[0002] With the popularization and application of LiDAR in the fields of assisted driving and autonomous driving, the exploration of its performance has become the key to the standardization and maturity of LiDAR products. During the design and development process of LiDAR, it is necessary to conduct sufficient testing on LiDAR to verify its performance and reliability during operation.

[0003] Traditional testing methods involve field testing. For field testing, each test requires setting up a field test exchange unit, which is time-consuming, resulting in high testing costs and low efficiency.

[0004] In order to quickly and efficiently verify the effectiveness of the LiDAR solution, a virtual simulation test of the LiDAR is also proposed. The simulation test simulates the working scenes of the LiDAR (for example, different lighting, weather, the reflection mechanism of the laser on the object, etc.) and sets the relevant working parameters. It can cover different test scenes, thereby reducing the test cost and improving the test efficiency.

[0005] Chinese patent application publication CN 104268323 A discloses a laser radar scene simulation solution based on ray tracing, which uses ray tracing technology to calculate the reflection effect between the laser and the environment, and realizes the simulation and simulation of laser radar detection from the level of laser light. The simulation solution based on ray tracing mainly focuses on the reflection effect of laser light (including direct reflection and indirect reflection) to realize simulation testing at the laser light level.

[0006] PCT patent application publication WO 2023 / 010540 A1 discloses a method, device, equipment and storage medium for verifying the scanning results of a laser radar, which outputs corresponding simulated point cloud information by specifying a laser radar model and a simulation scene. The solution focuses on the generation of simulated point clouds and performs simulation tests by calling specified models and scenes.

[0007] Therefore, there is a need in the art to efficiently and accurately provide simulation scenarios to achieve complete and accurate signal-level simulation of lidar detection. Summary of the invention

[0008] The present invention is provided in order to achieve a complete and accurate signal level simulation solution for laser radar detection.

[0009] According to one aspect of the present invention, there is provided a method for simulating the working process of a laser radar, the method comprising: setting parameters of the laser radar; setting a simulation scene associated with the working of the laser radar; simulating the energy change of the laser emitted by the laser radar in the simulation scene; and determining the echo signal of the emitted laser received by the laser radar at least based on the simulated energy change.

[0010] In the method as described above, setting the parameters of the laser radar includes: setting the energy distribution of lasers emitted by each channel of the laser radar.

[0011] A method as described in any of the above items, wherein setting the energy distribution of the laser emitted by each channel of the laser radar includes: setting the laser emitted by each channel to a number of rays with preset emission angles and energy ratios, the energy ratio representing the ratio of the energy of each ray to the energy of the emitted laser.

[0012] A method as described in any of the above items, wherein determining the echo signal of the emitted laser received by the laser radar includes: calculating the echo signal of the light in the emitted laser based at least on the simulated energy change; and superimposing the calculated echo signal of the light based at least on the energy ratio of the light in the laser emitted by each channel to generate a spot echo superposition signal.

[0013] A method as described in any of the above items, wherein setting a simulation scene associated with the operation of the laser radar includes: calling a scene and / or a material library generated by an existing three-dimensional rendering as an element of the simulation scene.

[0014] A method as described in any of the above items, wherein, in the case of calling an existing material library, setting a simulation scene associated with the operation of the laser radar further comprises: randomly setting at least part of the parameters of at least one simulation scene based on the existing material library.

[0015] A method as described in any of the above items, wherein simulating the energy change of the laser emitted by the laser radar in the simulation scene includes: determining the energy change based at least on simulated weather factors.

[0016] A method as described in any of the above items, wherein determining the energy change based at least on simulated weather factors comprises: determining the energy change based at least on visibility changes under the simulated weather factors.

[0017] A method as described in any one of the above items, wherein the simulated weather factors include sunny days or foggy days.

[0018] A method as described in any of the above items, wherein determining the energy change based at least on simulated weather factors comprises: simulating the distribution of particulate matter in the simulation scene under the simulated weather factors; and determining the energy change based at least on the energy loss when the emitted laser hits the particulate matter.

[0019] A method as described in any of the above items, wherein the simulated weather factors include rainy days, the particulate matter includes raindrops, and determining the energy change based at least on the energy loss when the emitted laser hits the particulate matter includes: simulating each light ray in the emitted laser to lose a specific proportion of energy when hitting the raindrops; and simulating the light ray that has suffered energy loss to continue to propagate after being randomly deviated by a certain angle.

[0020] A method as described in any one of the above, wherein the simulated weather factors include snowy days or hazy days,

[0021] The particulate matter includes snow or haze particles, and determining the energy change based at least on energy loss when the emitted laser hits the particulate matter includes simulating each ray of the emitted laser as having energy at least partially absorbed when hitting the snow or haze particles.

[0022] A method as described in any of the above items, wherein setting a simulation scene associated with the operation of the laser radar includes: setting the simulated weather factors.

[0023] A method as described in any of the above items, wherein simulating the energy change of the laser emitted by the laser radar in the simulation scene includes: determining the energy change based at least on the material of the object irradiated by the emitted laser.

[0024] A method as described in any of the above items, wherein setting a simulation scene associated with the operation of the laser radar includes: defining a model of a surface in the simulation scene.

[0025] The method as described in any one of the above items, wherein defining the model of the surface in the simulation scene comprises: using a bidirectional reflectance distribution function to determine the power of the light reflected by the emitted laser light from the rough opaque surface.

[0026] A method as described in any of the above items, wherein setting a simulation scene associated with the operation of the lidar includes: calling an existing scene generated by three-dimensional rendering, and defining a model of a surface in the simulation scene also includes: correcting model parameters of the surface in the scene generated by the called three-dimensional rendering based on the determined power of the reflected light.

[0027] A method as described in any of the above items, wherein the model defining the surface in the simulation scene includes at least one of the following: simulating an opaque mirror surface as a mirror reflection surface that is reflected substantially at the original energy when light is irradiated; simulating the surface of a transparent or translucent object as a surface with transmission and mirror reflection when light is irradiated; simulating the surface of an object with high reflectivity as a micro-prism structure or a micro-bead structure.

[0028] A method as described in any of the above items, wherein simulating the energy change of the laser emitted by the lidar in the simulation scene includes: determining the energy change based at least on the distance traveled by the emitted laser to hit the object and / or the incident angle of the emitted laser to the object.

[0029] The method as described in any one of the above items further comprises: performing an ambient light noise simulation on the echo signal of the emitted laser light.

[0030] A method as described in any of the above items, wherein setting a simulation scene associated with the operation of the laser radar includes: setting parameters of an ambient light source.

[0031] According to another aspect of the present invention, a computing device is provided, which includes: a device configured to set parameters of a laser radar; a device configured to set a simulation scene associated with the operation of the laser radar; a device configured to simulate energy changes of laser emitted by the laser radar in the simulation scene; and a device configured to determine an echo signal of the emitted laser received by the laser radar based at least on the simulated energy changes.

[0032] According to another aspect of the present invention, a computing device is provided, comprising: a processor; and a memory coupled to the processor, wherein the memory comprises instructions, and when the instructions are executed, the processor is caused to perform any one of the above methods.

[0033] According to yet another aspect of the present invention, a computer-readable storage medium is provided, which stores instructions, and when the instructions are executed, a computing device is caused to perform any of the above methods.

[0034] The present invention obtains the laser echo signal based on the energy change of the laser in the simulation scene, which can realize accurate signal level simulation of lidar detection and has high simulation value for virtual road testing, perception algorithm training, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Embodiments of the present invention are described with reference to the accompanying drawings, in which:

[0036] Figure 1 is a flow chart showing a method for simulating a working process of a laser radar according to an embodiment of the present invention.

[0037] Figure 2 is a diagram showing a process of setting parameters of a laser radar according to an embodiment of the present invention.

[0038] Figure 3 is a graphical representation showing energy distribution of laser light emitted by a laser radar according to channel settings according to an embodiment of the present invention.

[0039] Figure 4 is a flowchart illustrating a process of determining an echo signal of an emitted laser received by a laser radar according to an embodiment of the present invention.

[0040] Figure 5 is a diagram showing a point cloud involving a corner case obtained by simulation according to a method of an embodiment of the present invention.

[0041] Figure 6 is a diagram showing a point cloud involving another corner case obtained by simulation according to the method of an embodiment of the present invention.

[0042] Figure 7 is a diagram showing a process of setting up a simulation scene associated with the operation of a lidar according to an embodiment of the present invention.

[0043] Figure 8 is a diagram illustrating a process of customizing a simulation scenario according to an embodiment of the present invention.

[0044] Fig. 9 is a flowchart illustrating a process of randomly setting simulation scenarios according to an embodiment of the present invention.

[0045] Fig.10 It is a diagram showing the process of energy change of laser emitted by a simulated laser radar in a simulation scene according to an embodiment of the present invention.

[0046] Fig.11 is a diagram illustrating a process of determining energy changes based on simulated weather factors according to an embodiment of the present invention.

[0047] Fig.12 is a diagram illustrating a process of determining energy change based at least on energy loss when emitted laser light hits particles according to an embodiment of the present invention.

[0048] Fig.13 is a diagram illustrating a process of defining a model of a surface in a simulation scene according to an embodiment of the present invention.

[0049] Fig.14 It is a schematic structural diagram showing a micro-prism that simulates a high-reflection object according to an embodiment of the present invention.

[0050] Fig.15 is a block diagram showing a computing device for simulating a working process of a laser radar according to an embodiment of the present invention.

[0051] Fig.16 It is a block diagram showing a computer-readable storage medium for simulating the working process of a laser radar according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In the present application, various embodiments are described mainly in conjunction with the movement of a car, but the scope of the present application is not limited thereto. The embodiments of the present application are applicable to simulating the detection of a laser radar mounted on any other vehicle (e.g., an aircraft, other land vehicles, ships, etc.) and any other moving / non-moving object.

[0053] In this application, the term "high reflectivity" refers to a situation where the energy returned by light at the angle of incident light accounts for a high proportion. For example, high reflectivity can refer to 50% or more, 60% or more, 70% or more, 80% or more, 90% or more, 95% or more, 99% or more, or 100%. As an example, high reflectivity objects may include traffic signs, warning strips, etc.

[0054] In the present application, the expression "energy is at least partially absorbed" means 50% or more of the energy is absorbed, 60% or more of the energy is absorbed, 70% or more of the energy is absorbed, 80% or more of the energy is absorbed, 90% or more of the energy is absorbed, 95% or more of the energy is absorbed, 99% or more of the energy is absorbed, or 100% of the energy is absorbed.

[0055] In the present application, the expression “substantially reflected as is” means that 90% or more of the incident light energy is reflected, for example, 95% or more is reflected, 99% or more is reflected, for example, completely reflected.

[0056] According to one aspect of the present invention, a method for simulating a working process of a laser radar is provided.

[0057] Figure 1 is a flow chart showing a method 100 for simulating a working process of a laser radar according to an embodiment of the present invention.

[0058] The method 100 may include step 110: setting parameters of the laser radar. The method 100 may include step 130: setting a simulation scene associated with the operation of the laser radar. The method 100 may include step 150: simulating energy changes of the laser emitted by the laser radar in the simulation scene. As an example, the energy change may be energy loss caused by the interaction of the laser emitted by the laser radar with the simulation scene. The method 100 may include step 170: determining an echo signal of the emitted laser received by the laser radar based at least on the simulated energy change.

[0059] In practice, after method 100 executes step 170, one or more of steps 110, 130, 150, and 170 may be repeatedly executed to update radar parameters and / or simulation scenes, thereby obtaining echo signals based on the updated radar parameters and / or simulation scene frames to obtain multiple simulation results of lidar detection.

[0060] By considering the energy changes in the interaction between the laser emitted by the lidar and the simulation scene in the simulation of the lidar detection process, the echo signal received by the lidar detector in the simulation environment can be accurately simulated, thereby achieving accurate signal level simulation.

[0061] In some embodiments, the echo signal may be further processed to obtain simulated point cloud information, which may be widely used in virtual road testing, perception algorithm training, and the like.

[0062] For virtual road test applications, method 100 can partially or completely replace the actual road test of the lidar to discover and solve problems in advance. Compared with ordinary automobile road tests, the working environment of the lidar is not only affected by the specific road surface and road traffic conditions, but also by many other factors. Therefore, the challenge of road testing is that even a large number of road tests cannot exhaust the combination of the combined influence of many factors. Therefore, compared with performing real road tests in the field, applying method 100 to virtual road test applications is more efficient and less costly, and has a high simulation value.

[0063] For perception algorithm training applications, method 100 can quickly and massively acquire high-precision simulated point cloud information, ensuring the diversity of training data, thereby enabling the perception model to have more scalability and more easily identify uncommon situations (e.g., corner cases).

[0064] In addition, since the method 100 can complete the signal-level simulation of the laser radar echo signal, the simulated point cloud information can be further obtained based on the simulated echo signal in a variety of ways. For example, the obtained echo light pulse and channel ambient light illumination can be used as inputs to enter the simulation model of the photoelectric sensor to generate a digital model, and then the existing laser radar data processing algorithm / hardware is used to generate simulated point cloud information. Similarly, the method 100 can also be used to implement hardware-in-the-loop simulation of the processing hardware used to process laser radar data.

[0065] Figure 2 2 is a diagram illustrating a process 200 for setting parameters of a laser radar according to an embodiment of the present invention. The process 200 may be Figure 1 The process involved in step 110 of method 100 shown in FIG.

[0066] In some embodiments, process 200 may include step 210: selecting / defining a laser radar model. In some embodiments, step 210 may include selecting an existing laser radar model, for example, a brand, model, etc. of the laser radar may be selected. Alternatively, process 200 may include a custom laser radar model. As an example, the parameters of the definable laser radar model may include: the number of detection channels, the angle information of each channel, the position where each channel emits light in the laser radar coordinate system, the order of light emission of each channel and the corresponding time interval, the detection range / capability of each transmitting and receiving channel at a specific reflectivity, the pulse waveform of the emitted laser, etc.

[0067] In some embodiments, process 200 may include step 230: setting installation parameters of the laser radar. As an example, for an application scenario where the laser radar is mounted on a car, step 230 may include: determining the installation position of the laser radar relative to the entire vehicle, the angle of the installation plane, etc.

[0068] In some embodiments, process 200 may include step 250: setting the energy distribution of the laser emitted by each channel of the laser radar. In some embodiments, step 250 may include step 255: setting the laser emitted by each channel to a plurality of rays with preset angles and energy ratios, wherein the energy ratio represents the ratio of the energy of each ray to the energy of the laser emitted by the laser radar. As an example, the laser of each channel may be described as a series of rays forming a specified width / cross-sectional area, and the ratio of the energy carried by each ray may be stored accordingly (for example, stored in a matrix). The following will be combined with Figure 3 The implementation of step 255 is further described.

[0069] Through step 255, the light rays with associated preset angles and energy ratios in the same channel are stored in association, which helps to further associate and process the echo signals generated by these light rays, so that the simulation results are closer to the real laser spot, and further helps to find corner cases that are difficult to find in the actual laser radar test process. This will be further combined with Figure 4-Figure 6 to describe.

[0070] In an embodiment, process 200 may include one or more of steps 210 , 230 , 250 .

[0071] Figure 3 is a graphical representation 300 showing energy distribution of laser light emitted by a lidar according to channel settings according to an embodiment of the present invention.

[0072] Graphical representation 300 shows that the laser light emitted by each channel of the lidar can be associated with a number of regions. Figure 3 As an example, these areas may include band-shaped areas 310, 320, 330 from outside to inside and a central area 340, wherein the central area 340 may further include rectangular areas 341, 342, 343, 344, 345, 346 from the middle to both sides. Figure 3 The above-mentioned area divisions are only exemplary.

[0073] The associated light rays can be associated with the specified area according to the emission angle of the laser light emitted by each channel, and the energy ratio of the light rays in each area can be set to be the same. As an example, the energy ratio of the light rays corresponding to the area close to the middle is higher, while the energy ratio of the light rays decreases as it extends to the sides or the periphery. Thus, the light rays in each area can be stored in association, and the associated stored light rays can be further superimposed, and a light spot with a specified width / cross-sectional area can be generated when irradiated onto the surface in the simulation scene, so that in the subsequent processing steps, processing can be performed based on the echo signal of the light spot, which is closer to the actual detection scenario of the laser radar.

[0074] As an example, based on Figure 3 The energy ratio of the light at the corresponding position is set according to the position defined by the horizontal coordinate and the vertical coordinate, so that the energy ratio of the light of the laser emitted by each channel can be stored as an energy distribution matrix.

[0075] Figure 4 4 is a flow chart illustrating a process 400 of determining an echo signal of an emitted laser received by a laser radar according to an embodiment of the present invention.

[0076] Process 400 may be Figure 1The process involved in step 170 of the method 100 shown in FIG. 1 and may be directed to include Figure 2 The process of step 255 in process 200.

[0077] Process 400 may include step 410 of calculating a return signal of light in the emitted laser light based at least on the simulated energy variation.

[0078] The process 400 may include step 430: superimposing the calculated echo signals of the light based on at least the energy ratio of the light in the laser emitted by each channel to generate a spot echo superposition signal.

[0079] By superimposing the echo signals of the associated laser light rays set in step 255 described above, for example, to generate a spot echo superposition signal, compared with the light-based simulation scheme for lidar detection in the prior art, it can effectively reflect and identify corner cases, thereby greatly improving the value of virtual testing.

[0080] As an example, the pulse function of a single light ray can be expressed by the following equation (1):

[0081] F i (t) = m i *F(t) (1)

[0082] Among them, m i is the ratio of the energy of the light to the energy of the laser beam emitted in the channel where the light is located, which can be the above combined with Figure 2 The energy ratio of the light set in step 255 is described; F(t) is a complete light pulse waveform function of the laser beam emitted by the corresponding channel.

[0083] Assume that the peak power of the laser beam is For a linear system, the peak power of the echo is Among them, K i is the attenuation coefficient of the return power of the light. Accordingly, the echo pulse corresponding to the light can be derived by the following formula (2):

[0084] F i (t) = m i K i *F(t-2L i / c) (2)

[0085] Among them, t is the propagation time of the light, L i is the distance traveled by the light after it hits the object, and c is the speed of light in a vacuum. The total echo pulse corresponding to each light ray in the channel is obtained by calculating the echo pulse corresponding to each light ray in the channel according to formula (2) and superimposing the calculation results.

[0086] As an example, a simplified algorithm can first find the unit peak input pulse Its peak power is P t =1, and then count the probability distribution function of the peak power of all light rays with respect to the distance L, and The returned light waveform signal can be obtained by convolution operation with this function. Thus, the signal level simulation of the echo signal that can be received by the laser radar receiver module is completed.

[0087] By considering the impact of the laser echo signal in the form of a spot on the simulation results, a large part of the corner cases that are of concern in the LiDAR simulation test can be effectively reflected. These corner cases may include high reflection broadening, drag points, and disappearance of sidewalks.

[0088] The influence of the light spot on the characteristics of the point cloud varies with many factors. For example, in the case of close-range detection, the light spot is small and has little influence; while in the case of long-range detection, the light spot is large and has a great influence. Therefore, different results can be seen from the point cloud for close-range detection and long-range detection. For example, some features related to high reflectivity will also produce different point cloud results. For example, the size of the incident angle has a great influence on the point cloud results. For example, for ground lines with oblique incident lasers, the bending and disappearance of zebra crossings are related to this.

[0089] For point cloud training, without considering the light spot, the point cloud results obtained by incident on objects with different reflectivity at different distances and angles may be similar. In contrast, when considering the light spot, the point cloud results obtained will be different, depending on the definition of the light spot (i.e., the setting of the associated light rays), and this case is closer to the actual lidar detection situation, because the real lidar emits a light spot with a specified width / cross-section.

[0090] Figure 5 FIG. 5 is a diagram showing a point cloud involving a corner case obtained by simulation according to a method of an embodiment of the present invention.

[0091] Diagram 500 shows corner case portions 510, 530 associated with high anti-broadening. A point cloud 512 corresponding to an obstacle and point clouds 514, 516 of its edges are shown in corner case portion 510. A point cloud 531 corresponding to a road sign and point cloud 534 of its edges are shown in corner case portion 530.

[0092] For a laser spot of a certain size (e.g., width / cross-sectional area) emitted by a laser radar, although the center of the channel may not cover an object with a high back reflectivity, its edge may cover the object with a high back reflectivity. Although the energy of the light at the edge of the spot is weaker than that of the light at the center, given that the object with a high back reflectivity has a higher back reflectivity, the light at the edge of the spot can still reflect back part of the energy so that it can be received by the laser radar detector. In this case, the center of the channel generates the correct point cloud result corresponding to the channel, while the light at the edge of the channel generates an erroneous point cloud result (i.e., noise in the point cloud result), resulting in point cloud expansion for the object with a high back reflectivity, which is also called high back broadening. Compared with the simulation process that does not consider the spot size, by considering the influence of the echo signal of the laser in the form of a spot on the simulation results, the high back broadening phenomenon of concern in the laser radar simulation test can be effectively reflected, thereby improving the accuracy of the simulation of the laser radar detection.

[0093] Figure 6 FIG. 600 is a diagram showing a point cloud involving another corner case obtained by simulation according to a method of an embodiment of the present invention.

[0094] Diagram 600 shows a corner case associated with zebra crossing jitter. For a zebra crossing on the road that is far away from the laser radar, the incident angle of the laser emitted by the laser radar is large, and the light spot is elongated, resulting in a significant decrease in measurement accuracy. For a zebra crossing on the road, if the light spot of the laser emitted by the laser radar covers the zebra crossing and its adjacent ground, due to the difference in the return reflectivity between the zebra crossing and the adjacent ground, most of the echo energy of the light spot will be concentrated at the zebra crossing, resulting in the zebra crossing jitter phenomenon.

[0095] Figure 7 7 is a diagram illustrating a process 700 for setting up a simulation scene associated with the operation of a laser radar according to an embodiment of the present invention. The process 700 may be Figure 1 The process involved in step 130 of method 100 shown in FIG.

[0096] In some embodiments, process 700 may include step 710: customizing the simulation scenario. Figure 8 Describes the steps involved in customizing simulation scenarios.

[0097] In some embodiments, process 700 may include step 730: randomly setting a simulation scene. Fig. 9 Describe the steps involved in randomly generating simulation scenarios. By simulating LiDAR detection in a variety of randomly set simulation scenarios, it is possible to effectively simulate a variety of possible application environments, thereby helping to discover corner cases that are difficult to find in the actual LiDAR detection test process.

[0098] In some embodiments, process 700 may include customizing a portion of parameters of the simulation scenario and randomly setting a portion of parameters of the simulation scenario.

[0099] Figure 8 8 is a diagram illustrating a process 800 for customizing a simulation scenario according to an embodiment of the present invention. The process 800 may be Figure 7 The process involved in step 710 in process 700 shown in FIG.

[0100] In some embodiments, process 800 may include step 810: selecting a scene. As an example, a scene map may be selected from an existing map library. Alternatively, an existing map may be modified first, and then the modified map may be selected.

[0101] In some embodiments, process 800 may include step 820: calling existing scenes and / or material libraries generated by three-dimensional rendering. By calling existing scenes generated by three-dimensional rendering and only performing simple parameter settings on a few scenes, the model in the simulation scene is adapted to the road scene resources of the current autonomous driving platform, thereby greatly saving the work of building a three-dimensional simulation work scene by oneself and reducing the difficulty of obtaining virtual scenes for virtual road test applications. In addition, since the scene generated by three-dimensional rendering can directly generate photos from the camera's perspective, calling the scene generated by existing three-dimensional rendering and simulating lidar detection based on it can generate simulated point cloud information and simulated visual information based on the same virtual scene for the autonomous driving test level, which is conducive to multi-sensor fusion simulation testing.

[0102] In some embodiments, process 800 may include step 830: setting a motion trajectory and motion state of the laser radar in the scene. In some embodiments, setting the motion trajectory may include setting automatic tracking driving. As an example, the motion state that can be set may include turning, going uphill, accelerating, decelerating, etc.

[0103] In some embodiments, process 800 may include step 840: motion parameters of dynamic objects (eg, people, cars, etc.) in the scene. As an example, the trajectory of the moving object may be set. Alternatively, autonomous driving may be set.

[0104] In some embodiments, process 800 may include step 850: setting simulated weather factors. As an example, an ideal sunny day, foggy day, rainy day, snowy day, haze day, etc. may be set.

[0105] For an ideal sunny day, it can be defined that the laser propagates in the air without losing energy due to weather factors. For foggy days, it can be defined that the laser propagates in the air and is affected by visibility. For rainy days, raindrops can be simulated uniformly or randomly in three-dimensional space, and it is defined that the laser light will lose a certain proportion of energy when it encounters raindrops during propagation, and continue to propagate after randomly deviating at a certain angle. For snowy days, haze days, or weather containing other opaque particles, snow, haze particles, or other opaque particles can be simulated in three-dimensional space, and it is defined that the energy of the laser light is at least partially absorbed when it encounters them.

[0106] In some embodiments, process 800 may include step 860: defining a model of a surface in a simulation scene. Fig.13 The definitions of various surface models are further described.

[0107] In some embodiments, process 800 may include step 870: setting parameters of an ambient light source. By setting the parameters of the ambient light source, noise simulation of ambient light may be performed.

[0108] In reality, the illuminance value measured against the direction of the sun is often used to describe the overall level of ambient light in the current scene. However, due to the non-uniform reflection and refraction of sunlight by the atmosphere, multiple reflections after sunlight hits objects, shadows and other effects, in actual scenes, the amount of ambient light received by each channel of the lidar at each angle is different. In the scene of the rendering engine, the illuminance value of sunlight can be set as the overall input, and the reflection and refraction parameters of the sky light can be set so that the final scene rendering color can be obtained after the rendering engine runs. Accordingly, a mapping relationship can be constructed between the final rendering color and the base color of the object, the illuminance on the object, and the camera exposure. The current illuminance value of the object is inferred by obtaining the rendering color, the base color and the current exposure. There is a specific relationship (for example, a linear relationship) between the inferred current channel illuminance value and the effective value of the ambient light noise received by the channel. Based on this relationship, the illumination value of the current channel can be adjusted. Figure 1 The echo signal determined in step 170 of method 100 is used to further determine a more realistic simulation result.

[0109] In an embodiment, process 800 may include one or more of steps 810 to 870. In some embodiments, by fully considering and traversing weather factors and / or surface and / or ambient light noise that may be encountered in real tests in the simulation of lidar detection, more accurate and realistic simulation results may be obtained, thereby effectively improving the value of virtual testing.

[0110] Fig. 9 is a flow chart showing a process 900 of randomly setting simulation scenarios according to an embodiment of the present invention. The process 900 may be Figure 7The process involved in step 730 in process 700 shown in FIG.

[0111] The process 900 may include step 910: calling an existing material library. By directly calling an existing material library, the setting of the simulation scene can be simplified, thereby improving the efficiency of simulating laser radar detection.

[0112] The process 900 may include step 930: randomly generating at least some parameters of at least one simulation scene based on the existing material library. By randomly generating at least some parameters of the simulation scene, it is helpful to find corner cases that are difficult to find in the actual laser radar detection test process, thereby improving the value of virtual testing.

[0113] Fig.10 1 is a diagram showing a process 1000 of simulating energy changes of lasers emitted by a laser radar in a simulation scene according to an embodiment of the present invention. The process 1000 may be Figure 1 The process involved in step 150 of method 100 shown in FIG.

[0114] In some embodiments, process 1000 may include step 1010: determining energy changes based on simulated weather factors. Step 1010 may be combined with the above Figure 8 Step 850 is performed, and will be combined with Figure 11-Figure 12 Further description.

[0115] In some embodiments, process 1000 may include step 1030: determining energy changes based on the material of the object irradiated by the emitted laser. Figure 8 Step 860 is performed, and will be combined with Fig.13 Further description.

[0116] In some embodiments, process 1000 may include step 1050: determining an energy change based on the distance traveled by the emitted laser to hit the object. As an example, the energy change may be determined based on a general attenuation law of the energy of the laser light in a conventional environment. For example, the power of the laser light decays inversely or inversely square with the propagation distance.

[0117] In some embodiments, process 1000 may include step 1070 of determining an energy change based on an incident angle of the emitted laser light incident on the object.

[0118] In an embodiment, process 1000 may include one or more of steps 1010, 1030, 1050, 1070. As an example, the emitted laser power P t and the received laser power P r The following relationship (3) can be satisfied:

[0119] P r =KP t (3)

[0120] Where K is the attenuation coefficient of laser power.

[0121] As an example, K may be a function of one or more of the factors involved in the above steps 1010, 1030, 1050, and 1070, as shown in the following formula (4):

[0122] K=f(k 天气 , k 材质 , k 距离 , k 入射角 ) (4)

[0123] Among them, k 天气 is the attenuation coefficient of laser power caused by weather factors, k 材质 is the attenuation coefficient of the laser power caused by the material of the object irradiated by the laser, k 距离 is the attenuation coefficient of the laser power due to the distance traveled by the laser to hit the object, and k 入射角 It is the attenuation coefficient of the laser power caused by the incident angle of the laser into the object.

[0124] As an example, K may be the product of the above-mentioned attenuation coefficients, as shown in the following equation (5), but the scope of the present invention is not limited thereto:

[0125] K=k 天气 k 材质 k 距离 k 入射角 (5)

[0126] Fig.11 1 is a diagram illustrating a process 1100 for determining energy changes based on simulated weather factors according to an embodiment of the present invention. The process 1100 may be Fig.10 The process involved in step 1010 in process 1000 shown in FIG.

[0127] In some embodiments, process 1100 may include step 1110 of determining an energy change based at least on a visibility change under simulated weather factors.

[0128] In some embodiments, the simulated weather factors may include a sunny day. On an ideal sunny day, it can be considered that the power of the laser only decays with the propagation distance without further attenuation caused by weather factors.

[0129] In some embodiments, the simulated weather factors may include foggy weather. For foggy weather, the attenuation of the simulated laser pulse signal may be regarded as the attenuation acting uniformly on each channel. As an example, the power of the laser after propagating a certain distance may be calculated according to the following formula (6):

[0130]

[0131] Among them, P0 is the emission power of the laser, d0 is the visibility on an ideal sunny day, d is the visibility in the current weather (foggy day), D is the distance of laser propagation (here, D is the one-way distance of laser propagation, and the actual round-trip propagation distance of the laser is 2D), and k is a constant related to the wavelength of the selected laser.

[0132] The term in formula (6) It can reflect the attenuation of laser power caused by weather factors. As an example, It can correspond to the term kweather in equation (5). For example, for an ideal sunny day, d = d0, that is, kweather = 1, that is, there is no further attenuation caused by weather factors. The term in equation (6) It can reflect the attenuation of laser power with propagation distance. As an example, It can correspond to the term k distance in equation (5).

[0133] In some embodiments, process 1100 may include step 1130 related to particulate matter under simulated weather factors.

[0134] Step 1130 may include step 1132: simulating the distribution of particles in the simulation scene under simulated weather factors. As an example, the particles may be raindrops, snow, haze particles, other opaque particles, etc. As an example, the particles may be simulated to appear randomly in the three-dimensional space and occupy a specific space.

[0135] As an example, for raindrops, the expected probability E of their occurrence at a specific position in the three-dimensional space may be proportional to the set unit precipitation and the simulation time step. Alternatively, the raindrops may be made to appear according to a set distribution, and the number of raindrops appearing in the set unit space is proportional to the unit precipitation and the simulation time step.

[0136] As an example, for snow, haze particles or other opaque particles, the size of the particles can be set and generated in three-dimensional space in a similar manner to the generation of raindrops described above.

[0137] Step 1130 may include step 1134: determining the energy change based at least on the energy loss of the emitted laser when it hits the particle. Step 1134 will be described below in conjunction with Fig.12 Further description.

[0138] Fig.12 is a diagram illustrating a process 1200 of determining energy change based at least on energy loss when emitted laser light strikes a particle in accordance with an embodiment of the present invention. Fig.12 The process 1200 shown in FIG. 1 may be Fig.11 The process involved in step 1134 in process 1100 shown in FIG.

[0139] In some embodiments, the simulated weather may include rainy days, and the related particulate matter may be raindrops. Accordingly, process 1200 may include step 1210: simulating the daily light in the emitted laser to lose a certain proportion of energy when hitting raindrops. Process 1200 may also include step 1230: simulating the light that has suffered energy loss to continue to propagate after randomly deviating at a certain angle. Thus, the above steps 1210 and 1230 may be repeated each time the light encounters raindrops until the light encounters the surface of the object.

[0140] In some embodiments, the simulated weather may include snow or haze, and the associated particulate matter may be snow or haze particles. Accordingly, process 1200 may include step 1250: simulating each ray of the emitted laser as having its energy at least partially absorbed when hitting snow or haze particles. For this simulated weather, the energy of the laser light that is not absorbed when hitting the particulate matter accounts for a low proportion, and its continued propagation is generally not considered in actual simulations of lidar detection. In some cases, although the calculation becomes complicated when considering the continued propagation of the laser light that is not absorbed by the particulate matter in this simulated weather, this is also feasible when corner cases (e.g., heavy snow) can be reflected through simulation.

[0141] Fig.13 is a diagram illustrating a process 1300 of defining a model of a surface in a simulation scene according to an embodiment of the present invention. Fig.13 The process 1300 shown in FIG. 1 may be Figure 8 The process involved in step 860 in process 800 shown in FIG.

[0142] In some embodiments, process 1300 may include step 1310 of determining the power of light reflected from a rough opaque surface by an emitted laser using a bidirectional reflectance distribution function (BRDF).

[0143] The BRDF model is a basic model used in graphics algorithms to describe light reflection phenomena. It is particularly suitable for general rough opaque surfaces. For such a surface, to calculate the echo energy / power of the laser reflected from the surface, the power irradiated on the surface is P iThe power P of the light received by the laser radar can be obtained by derivation. r It can be determined by the following formula (7):

[0144] P r =Ωf BRDF (θ i )cosθ i P i (7)

[0145] Where Ω is the solid angle, f BRDF The term can be defined as the following formula (8):

[0146]

[0147] Where L is the luminance of the emitted light, and its unit is lm / (sr·m 2 )(lumens / (steradian meter 2 )), E is the incident light illuminance, its unit is lm / m 2 ,θ i is the angle of incidence of the light ray on the surface. BRDF The unit of the term is sr -1 , which is the reflected power ratio per unit solid angle. BRDF The result of multiplying by the solid angle Ω represents the reflection ratio of the incident light power on the surface within a specific solid angle direction. r The solid angle of the laser radar / lidar receiver channel corresponding to the reflecting surface at distance L can be determined according to the following formula (9):

[0148]

[0149] Correspondingly, the power P of the laser light finally received by the laser radar is r It can be determined according to the following formula (10):

[0150]

[0151] From the above formula 9, it can be seen that in addition to the power attenuation caused by the laser propagating in the air, the power of the reflected laser light received by the laser radar is related to the distance between the object and the laser radar, the angle at which the laser light is incident on the surface of the object, and f BRDF Here, f BRDF The terms are those associated with the material properties of the object and have little to do with the parameters of the lidar.

[0152] In some embodiments, step 1310 may include step 1315: modifying the model parameters of the surface in the scene generated by the invoked three-dimensional rendering based on the determined power of the reflected light. Figure 8 The process 800 is described as step 820 being performed.

[0153] In the simulation scene generated by 3D rendering, the model called by the mainstream rendering engine usually contains the material information of each surface, so it can be BRDF The item is associated and mapped with the material information used in the rendering operation, and then the power of the light finally received by the laser radar corresponding to the specific surface can be calculated through the above formula (10). Therefore, the scene and / or material library generated by the existing 3D rendering can be directly called as an element of the simulation scene for the simulation process of laser radar detection. Since the 3D rendering process usually does not consider the physical conservation of energy, in some embodiments, when using the BRDF model, the relevant parameters can be corrected to ensure that the energy of the reflected light in the model is conserved with the energy of the incident light and the energy of the absorbed light, so as to accurately reflect the interaction process between the laser and the surface.

[0154] By utilizing the BRDF model in combination with some custom models, the scenes generated by existing three-dimensional rendering can be used for virtual road testing after simple processing, which greatly reduces the difficulty of obtaining virtual scenes.

[0155] In addition, the use of BRDF-based models for simulation can adapt to the materials of current mainstream rendering engines (for example, Unity, Unreal Engine, etc.), which can save a lot of work in building three-dimensional simulation scenes by yourself.

[0156] In addition to simulating general rough, opaque objects using the above-mentioned BRDF model, in some embodiments, process 1300 may also include defining some other special surfaces.

[0157] In some embodiments, process 1300 may include step 1330: simulating an opaque mirror surface as a mirror-reflecting surface that is reflected substantially at the original energy when the light is irradiated. For an opaque mirror surface, when the laser light irradiates the surface, it is reflected at the mirror angle substantially at the original energy. In other words, it is equivalent to the laser changing the propagation direction at the incident point and continuing to propagate, while other aspects such as the laser energy remain substantially unchanged.

[0158] In some embodiments, process 1300 may include step 1350: simulating the surface of a transparent or translucent object as a surface with projection and specular reflection when light is irradiated. For a transparent or translucent object, when the laser irradiates the surface during simulation, a portion of the light is allowed to continue to pass through (for example, for window glass). The energy that is transmitted can be determined by dividing the energy of the incident light by the refractive index of the object material (or in the rendering material, transparency can also be used). As an example, if the surface of such an object is smooth (for example, a level surface, a puddle, etc.), the remaining energy is reflected out as specular reflection.

[0159] In some embodiments, process 1300 may include step 1370 : simulating the surface of the high reflectivity object as a micro-prism structure or a micro-bead structure.

[0160] The point cloud generated by high-reflectivity objects has a very high reflectivity, which is manifested as extremely high energy returned at the angle of the incident light (different from specular reflection), which is one of the characteristics that affects the detection results of LiDAR. As an example, high-reflectivity objects may include traffic signs, warning strips, etc. Such high-reflectivity objects cannot be regarded as separate surfaces, and their actual objects are usually micro-prismatic structures or micro-bead structures.

[0161] The micro-prism structure may include a number of mutually perpendicular mirrors. Fig.14 If the object itself has color, it is also necessary to take the average value of its color channel or the value B of the red channel (normalized to the interval [0, 1]) and determine the power P of the reflected laser light according to the following formula (11): out :

[0162] P out =cos 2 θ i BP in (11)

[0163] Among them, θ i is the incident angle of the light hitting the surface of the object, P in is the power of the incident light.

[0164] The characteristic of the micro-prism structure is that, for light incident in a certain direction, most of the energy can be returned along the original path with the reflected light. This is because the light passes through a layer of glass or transparent plastic (for focusing light) before entering the micro-prism structure, and the energy of this part of the reflection increases with the increase of the incident angle, which is in line with the Fresnel principle, as shown in the following equations (12)-(15):

[0165]

[0166]

[0167]

[0168]

[0169] Taking the Schlick approximation, we have:

[0170] R(θ)=R0+(1-R0)(1-cosθ) 5 (16)

[0171]

[0172] In the above formulas (12) to (17), η i is the refractive index of the incident medium (for air, it is assumed to be 1), η t is the refractive index of glass or transparent plastic (greater than 1), then equation (17) becomes the following equation (18):

[0173]

[0174] When cosθ=1 (i.e., normal incidence), the following equation (19) is obtained:

[0175] P out =BP in =R(θ=0)BP in (19)

[0176] For general incident angles, the following equation (20) is obtained:

[0177] P out (θ)=R(θ)BP in (20)

[0178] Fig.14 It is a schematic structural diagram showing a micro-prism that simulates a high-reflection object according to an embodiment of the present invention.

[0179] exist Fig.14 The upper figure in the figure shows the situation where the light is incident on the surface of the micro-prismatic structure. It can be seen that the light is reflected in the direction perpendicular to the incident surface basically according to the original energy.

[0180] exist Fig.14 The figure below shows the situation where light is incident obliquely on the surface of the micro-prism structure. It can be seen that the light is reflected in a direction parallel to the incident direction but opposite to the incident direction according to the original energy.

[0181] According to another aspect of the present invention, a computing device is provided. The computing device may include each of which is configured to perform the above combination Figure 1-Figure 14 The apparatus for each step of the method is described.

[0182] According to yet another aspect of the present invention, a computing device is provided.

[0183] Fig.15 is a block diagram showing a computing device 1500 for simulating a working process of a laser radar according to an embodiment of the present invention.

[0184] The computing device 1500 may include a processor 1510 and a memory 1530. The memory 1530 is coupled to the processor 1510. The memory 1530 may include instructions 1550, which when executed, cause the processor 1510 to perform the above-mentioned Figure 1-Figure 14 Describe the method.

[0185] According to yet another aspect of the present invention, a computer-readable storage medium is provided.

[0186] Fig.16 is a block diagram showing a computer-readable storage medium 1600 for simulating the working process of a laser radar according to an embodiment of the present invention.

[0187] The computer readable storage medium 1600 may include instructions 1650. The instructions 1650, when executed, may cause the computing device to perform the above combined Figure 1-Figure 14 Describe the method.

[0188] Non-limiting embodiments of the present invention have been described with reference to the accompanying drawings, the embodiments being intended to be illustrative rather than limiting.

Claims

1. A method for simulating a working process of a laser radar, comprising: Set the parameters of the lidar; Setting a simulation scenario associated with the operation of the laser radar; Simulating energy changes of lasers emitted by the laser radar in the simulation scene; as well as Based at least on the simulated energy variation, an echo signal of the emitted laser light received by the laser radar is determined.

2. The method according to claim 1, characterized in that The parameters of the laser radar are set as follows: The energy distribution of the laser emitted by each channel of the laser radar is set.

3. The method according to claim 2, characterized in that Setting the energy distribution of the laser emitted by each channel of the laser radar includes: The laser emitted by each channel is set to be a plurality of light rays with a preset emission angle and energy ratio, wherein the energy ratio represents the ratio of the energy of each light ray to the energy of the emitted laser.

4. The method according to claim 3, characterized in that Determining the echo signal of the emitted laser received by the laser radar comprises: calculating an echo signal of light in the emitted laser light based at least on the simulated energy variation; and At least based on the energy ratio of the light in the laser emitted by each channel, the calculated echo signals of the light are superimposed to generate a spot echo superposition signal.

5. The method according to claim 1, characterized in that Setting up the simulation scenario associated with the operation of the LiDAR includes: The existing scene and / or material library generated by three-dimensional rendering is called as an element of the simulation scene.

6. The method according to claim 5, characterized in that In the case of calling the existing material library, setting the simulation scene associated with the operation of the laser radar further includes: Based on the existing material library, at least part of the parameters of at least one simulation scene are randomly set.

7. The method according to claim 1, characterized in that Simulating the energy change of the laser emitted by the laser radar in the simulation scene includes: The energy change is determined based at least on simulated weather factors.

8. The method according to claim 7, characterized in that Determining the energy change based at least on simulated weather factors includes: The energy change is determined based at least on a visibility change under the simulated weather factor.

9. The method according to claim 8, characterized in that The simulated weather factors include sunny days or foggy days.

10. The method according to claim 7, characterized in that Determining the energy change based at least on simulated weather factors includes: Simulating the distribution of particulate matter in the simulation scene under the simulated weather factors; The energy change is determined based at least on energy loss of the emitted laser light when it strikes the particle.

11. The method according to claim 10, characterized in that The simulated weather factors include rainy days, The particulate matter includes raindrops, and Determining the energy change based at least on the energy loss of the emitted laser light when it hits the particle comprises: Simulating each ray of the emitted laser light to lose a specific fraction of its energy when striking the raindrop; and The light that has suffered energy loss is simulated as continuing to propagate after being randomly deflected by a certain angle.

12. The method according to claim 10, characterized in that The simulated weather factors include snowy days or hazy days. The particulate matter includes snow or haze particles, and Determining the energy change based at least on the energy loss of the emitted laser light when it hits the particle comprises: Each ray of the emitted laser light is simulated to have its energy at least partially absorbed when it strikes the snow or haze particles.

13. The method according to claim 8, characterized in that Setting up the simulation scenario associated with the operation of the LiDAR includes: Sets the weather factors for the simulation.

14. The method according to claim 1, wherein: Simulating the energy change of the laser emitted by the laser radar in the simulation scene includes: The energy change is determined based at least on the material of an object irradiated by the emitted laser light.

15. The method according to claim 14, characterized in that Setting up the simulation scenario associated with the operation of the LiDAR includes: A model is defined for a surface in the simulation scene.

16. The method according to claim 15, characterized in that The model defining the surface in the simulation scene includes: The power of the emitted laser light reflected from the rough opaque surface is determined using a bidirectional reflectance distribution function.

17. The method according to claim 16, characterized in that Setting up the simulation scenario associated with the operation of the LiDAR includes: Calling the scene generated by the existing 3D rendering, and The model defining the surface in the simulation scene also includes: Model parameters of surfaces in a scene generated by the invoked three-dimensional rendering are modified based on the determined power of the reflected light.

18. The method according to claim 15, characterized in that The model defining the surface in the simulation scene includes at least one of the following: The opaque mirror surface is simulated as a specular reflective surface that is reflected at substantially the original energy when light strikes it; Simulate the surface of transparent or translucent objects as a surface with transmission and specular reflection when light hits it; The surface of a high reflectivity object is simulated as a micro-prism structure or a micro-bead structure.

19. The method of claim 1, wherein: Simulating the energy change of the laser emitted by the laser radar in the simulation scene includes: The energy change is determined based at least on a distance traveled by the emitted laser light to strike an object and / or an angle of incidence of the emitted laser light to the object.

20. The method of claim 1, wherein: Also includes: An ambient light noise simulation is performed on the echo signal of the emitted laser light.

21. The method of claim 20, wherein: Setting up the simulation scenario associated with the operation of the LiDAR includes: Sets the parameters of the ambient light source.

22. A computing device comprising: A device configured to set parameters of a laser radar; means configured to set a simulation scenario associated with operation of the laser radar; A device configured to simulate energy changes of laser light emitted by the laser radar in the simulation scene; as well as A device configured to determine an echo signal of the emitted laser light received by the laser radar based at least on the simulated energy change.

23. A computing device comprising: processor; as well as A memory is coupled to the processor, the memory comprising instructions, which when executed cause the processor to perform the method as claimed in any one of claims 1 to 21.

24. A computer-readable storage medium storing instructions which, when executed, cause a computing device to perform the method of any one of claims 1-21.

Images