Point cloud distortion effect restoration method and apparatus, electronic device and storage medium

Abstract

The present disclosure relates to the technical field of autonomous driving and provides a point cloud distortion effect restoration method, a point cloud distortion effect restoration apparatus, an electronic device and a storage medium. The method comprises: if an autonomous driving simulation environment is a low-frame-rate simulation environment, controlling a LiDAR to simultaneously emit multiple lasers at different angles on the basis of a horizontal angular resolution, and determining multiple lasers captured in each simulation frame; in each simulation frame, using a ray tracing algorithm to separately determine collision point coordinates corresponding to each laser beam in the simulation frame and, according to the current vehicle speed vector of a host vehicle corresponding to each laser beam and a laser emission time difference, correcting the collision point coordinates corresponding to each laser beam, so as to determine corrected collision coordinates; and, according to the corrected collision point coordinates corresponding to each column of lasers, obtaining a point cloud having an added distortion effect. The present disclosure adds the distortion effect to the point cloud by means of the motion characteristics of the host vehicle, thus rendering the autonomous driving environment more real, and improving the precision of subsequent operations.

Description

Point cloud distortion effect restoration method, device, electronic device and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure claims priority to Chinese patent application number 202310913118.2, filed with the State Intellectual Property Office of China on July 25, 2023, entitled “Point cloud distortion effect restoration method, device, electronic device and storage medium,” the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0003] The present disclosure relates to the field of autonomous driving technology, and in particular to a method, device, electronic device, and storage medium for restoring point cloud distortion effects. Background Art

[0004] In autonomous driving technology, lidar point clouds play a very important role in perception. Due to the density of the point clouds generated by lidar, lidar has become one of the most important perception tools in the field of autonomous driving perception. In the development of autonomous driving simulation software, it is necessary to simulate the lidar point cloud as real perception data to verify the control algorithm and perception and fusion algorithm.

[0005] Ray tracing technology is currently an important technology for outputting LiDAR point clouds and has been widely used in LiDAR simulations. However, a shortcoming of point clouds currently created using ray tracing technology is that they cannot reflect distortion. This is because the ray tracing algorithm needs to perform calculations within a fixed frame, while the real scene is dynamic and the laser pulse emission of the scanning LiDAR is also dynamic. This causes the point cloud generated by the ray tracing algorithm to be too perfect and unable to reflect the real simulation environment.

[0006] Summary of the Invention

[0007] In view of this, the present disclosure at least provides a point cloud distortion effect restoration method, a point cloud distortion effect restoration device, an electronic device and a storage medium. By adding distortion effects to the point cloud according to the motion characteristics of the main vehicle in high and low frequency simulations, the autonomous driving environment is made more realistic and the subsequent calculation accuracy is improved.

[0008] The present disclosure mainly includes at least the following aspects:

[0009] Some embodiments of the present disclosure provide a method for restoring point cloud distortion effects, which may include:

[0010] Obtain a simulation frame rate corresponding to the autonomous driving simulation environment and determine whether the autonomous driving simulation environment is an ultra-high frame rate simulation environment based on the simulation frame rate. The autonomous driving simulation environment includes a main vehicle equipped with a laser radar, which is in a moving state. If the autonomous driving simulation environment is an ultra-high frame rate simulation environment, the laser radar is controlled to rotate and sequentially emit multiple lasers to obtain a point cloud with an added distortion effect. If the autonomous driving simulation environment is a low frame rate simulation environment, the laser radar is controlled to simultaneously emit multiple groups of lasers along different angles according to the horizontal angular resolution, and the multiple groups of lasers captured in each simulation frame are determined. Within each simulation frame, the ray tracing algorithm is used to determine the collision point coordinates corresponding to each laser beam within the simulation frame, and the collision point coordinates corresponding to each laser beam are corrected based on the current vehicle speed vector corresponding to each laser beam and the laser emission time difference to determine the corrected collision point coordinates. The point cloud with the added distortion effect is obtained from the corrected collision point coordinates corresponding to each laser beam.

[0011] In a preferred embodiment, whether the autonomous driving simulation environment is an ultra-high frame rate simulation environment can be determined in the following manner: obtain the rotation frequency and horizontal angular resolution corresponding to the laser radar; determine the laser emission time difference corresponding to the laser radar based on the rotation frequency and horizontal angular resolution; calculate the simulation time difference based on the simulation frequency; if the laser emission time difference is greater than or equal to the simulation time difference, determine that the autonomous driving simulation environment is an ultra-high frame rate simulation environment; if the laser emission time difference is less than the simulation time difference, determine that the autonomous driving simulation environment is a low frame rate simulation environment.

[0012] In a preferred embodiment, the step of determining the laser emission time difference corresponding to the laser radar based on the rotation frequency and horizontal angular resolution may include: determining the total rotation time taken by the laser radar to rotate one circle based on the rotation frequency corresponding to the laser radar; determining the total number of lasers emitted when the laser radar rotates one circle based on the horizontal angular resolution; and determining the ratio between the total rotation time and the total number of lasers emitted as the laser emission time difference.

[0013] In a preferred embodiment, if the autonomous driving simulation environment is an ultra-high frame rate simulation environment, a point cloud with added distortion effect can be obtained in the following manner: lasers are emitted sequentially according to the time difference of laser emission; for each emitted laser, the following processing is performed: the surrounding environment information and the main vehicle position recorded in the simulation frame corresponding to the emitted laser are extracted; the ray tracing data generated by the emitted laser in its corresponding simulation frame is calculated using the ray tracing algorithm, the surrounding environment information of the main vehicle and the main vehicle position, and the ray tracing data includes the distance information and emission angle information between the main vehicle and the collision point of its surrounding environment; when the laser radar rotates one circle, polar coordinate conversion is performed according to the distance information and emission angle information corresponding to each emitted laser to obtain a point cloud with added distortion effect.

[0014] In a preferred embodiment, the corrected collision point coordinates corresponding to each laser beam can be determined in the following manner: the product of the current vehicle speed vector corresponding to the laser beam and the laser emission time difference is determined as the displacement vector of the vehicle corresponding to the laser beam; the difference between the collision point coordinates and the displacement vector of the vehicle is determined as the corrected collision point coordinates corresponding to the laser beam.

[0015] In a preferred embodiment, the step of obtaining a point cloud with a distortion effect added from the corrected collision point coordinates corresponding to each laser beam may include: for each laser beam, performing polar coordinate conversion on the corrected collision point coordinates of the laser beam to obtain the distance information and emission angle information corresponding to the laser beam; and using the distance information and emission angle information corresponding to each laser beam to output a point cloud with a distortion effect added.

[0016] Some other embodiments of the present disclosure further provide a point cloud distortion effect restoration device, which may include: a frame rate simulation environment judgment module, the frame rate simulation environment judgment module being configured to: obtain a simulation frame rate corresponding to an autonomous driving simulation environment and judge whether the autonomous driving simulation environment is an ultra-high frame rate simulation environment according to the simulation frame rate, wherein the autonomous driving simulation environment includes a main vehicle equipped with a laser radar, and the main vehicle is in a moving state; a high frame rate distortion restoration module, the high frame rate distortion restoration module being configured to: if the autonomous driving simulation environment is an ultra-high frame rate simulation environment, control the laser radar to rotate and sequentially emit multiple lasers to obtain a point cloud with a distortion effect added; a first low frame rate distortion restoration module, the first low frame rate distortion restoration module The restoration module is configured as follows: if the autonomous driving simulation environment is a low frame rate simulation environment, the laser radar is controlled to simultaneously emit multiple groups of lasers along different angles according to the horizontal angular resolution, and the multiple groups of lasers captured in each simulation frame are determined; the correction module is configured as follows: in each simulation frame, the ray tracing algorithm is used to respectively determine the collision point coordinates corresponding to each laser beam in the simulation frame, and the collision point coordinates corresponding to each laser beam are corrected according to the current vehicle speed vector of the main vehicle corresponding to each laser beam and the laser emission time difference, and the corrected collision point coordinates are determined; a second low frame rate distortion restoration module, the second low frame rate distortion restoration module is configured as follows: the corrected collision point coordinates corresponding to each laser beam are used to obtain a point cloud with an added distortion effect.

[0017] In a preferred embodiment, the frame rate simulation environment judgment module can also be configured to: determine whether the autonomous driving simulation environment is an ultra-high frame rate simulation environment in the following manner: obtain the rotation frequency and horizontal angular resolution corresponding to the laser radar; determine the laser emission time difference corresponding to the laser radar based on the rotation frequency and horizontal angular resolution; calculate the simulation time difference based on the simulation frequency; if the laser emission time difference is greater than or equal to the simulation time difference, determine that the autonomous driving simulation environment is an ultra-high frame rate simulation environment; if the laser emission time difference is less than the simulation time difference, determine that the autonomous driving simulation environment is a low frame rate simulation environment.

[0018] In a preferred embodiment, the frame rate simulation environment judgment module can also be configured to: determine the laser emission time difference corresponding to the laser radar based on the rotation frequency and horizontal angular resolution, which may include: determining the total rotation time taken for the laser radar to rotate one circle based on the rotation frequency corresponding to the laser radar; determining the total number of lasers emitted when the laser radar rotates one circle based on the horizontal angular resolution; and determining the ratio between the total rotation time and the total number of lasers emitted as the laser emission time difference.

[0019] In a preferred embodiment, the high frame rate distortion restoration module can also be configured as follows: if the autonomous driving simulation environment is an ultra-high frame rate simulation environment, a point cloud with added distortion effect can be obtained in the following manner: lasers are emitted sequentially according to the time difference of laser emission; for each emitted laser, the following processing is performed: the surrounding environment information and the main vehicle position recorded in the simulation frame corresponding to the emitted laser are extracted; the ray tracing data generated by the emitted laser in its corresponding simulation frame is calculated using the ray tracing algorithm, the surrounding environment information of the main vehicle and the main vehicle position, and the ray tracing data includes the distance information and emission angle information between the main vehicle and the collision point of its surrounding environment; when the laser radar rotates one circle, polar coordinate conversion is performed according to the distance information and emission angle information corresponding to each emitted laser to obtain a point cloud with added distortion effect.

[0020] In a preferred embodiment, the correction module can also be configured to: determine the corrected collision point coordinates corresponding to each laser beam in the following manner: determine the product of the current vehicle speed vector of the main vehicle corresponding to the laser beam and the laser emission time difference as the main vehicle displacement vector corresponding to the laser beam, and determine the difference between the collision point coordinates and the main vehicle displacement vector as the corrected collision point coordinates corresponding to the laser beam.

[0021] In a preferred embodiment, the second low frame rate distortion restoration module can also be configured as follows: the step of obtaining a point cloud with an added distortion effect from the corrected collision point coordinates corresponding to each laser beam can include: for each laser beam, performing polar coordinate conversion on the corrected collision point coordinates of the laser beam to obtain the distance information and emission angle information corresponding to the laser beam; using the distance information and emission angle information corresponding to each laser beam, outputting a point cloud with an added distortion effect.

[0022] Some further embodiments of the present disclosure further provide an electronic device, which may include: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the point cloud distortion effect restoration method in some of the above embodiments or any possible implementation methods of some of the above embodiments.

[0023] Still other embodiments of the present disclosure provide a computer-readable storage medium, which may store a computer program. When the computer program is executed by a processor, the steps of restoring the point cloud distortion effect in some of the above embodiments or any possible implementation of the above embodiments are executed.

[0024] The embodiments of the present disclosure provide a method for restoring point cloud distortion effects, a device for restoring point cloud distortion effects, an electronic device, and a storage medium. If the autonomous driving simulation environment is a low frame rate simulation environment, the laser radar is controlled to simultaneously emit multiple groups of lasers along different angles according to the horizontal angular resolution, and the multiple groups of lasers captured in each simulation frame are determined; within each simulation frame, the ray tracing algorithm is used to respectively determine the collision point coordinates corresponding to each laser beam in the simulation frame, and the collision point coordinates corresponding to each laser beam are corrected according to the current vehicle speed vector of the main vehicle corresponding to each laser beam and the laser emission time difference, and the corrected collision coordinates are determined; the corrected collision point coordinates corresponding to each column of lasers are used to obtain a point cloud with added distortion effects. The present disclosure adds distortion effects to the point cloud based on the motion characteristics of the main vehicle in high and low frequency simulations, thereby making the autonomous driving environment more realistic and improving the accuracy of subsequent calculations.

[0025] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0027] FIG1 shows a flow chart of a method for restoring point cloud distortion effects provided by an embodiment of the present disclosure;

[0028] FIG2 shows a schematic diagram of the laser radar operation in the ultra-high frame rate simulation environment of the present disclosure;

[0029] FIG3 shows a schematic structural diagram of a point cloud distortion effect restoration device provided by an embodiment of the present disclosure;

[0030] FIG4 shows a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. It should be understood that the drawings in the present disclosure are only for the purpose of illustration and description and are not used to limit the scope of protection of the present disclosure. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in the present disclosure illustrate operations implemented according to some embodiments of the present disclosure. It should be understood that the operations of the flowchart can be implemented out of sequence, and steps that do not have a logical context relationship can be reversed in order or implemented simultaneously. In addition, those skilled in the art, guided by the contents of the present disclosure, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0032] In addition, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings herein can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the disclosure claimed for protection, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present disclosure.

[0033] As an important sensor output parameter, point cloud output has been calculated by a large number of algorithms. The most famous of these is the use of ray tracing technology, which detects the collision point in the direction of each laser pulse and outputs the coordinates of the collision point to generate a point cloud. Ray tracing technology is currently an important technology for lidar point cloud output and has been widely used in lidar simulation.

[0034] Ray tracing technology is calculated based on the rendering environment and is based on computer graphics. One shortcoming of the point cloud currently created using ray tracing technology is that it cannot reflect distortion phenomena. This is because the ray tracing algorithm needs to perform calculations within a fixed frame, while the real scene is dynamic and the laser pulse emission of the scanning lidar is also dynamic. This causes the point cloud generated by the ray tracing algorithm to be too perfect, which is not conducive to our verification of the real perception algorithm.

[0035] Based on this, the disclosed embodiments provide a method for restoring point cloud distortion effects. This method adds distortion effects to point clouds by leveraging the motion characteristics of the vehicle, making the autonomous driving environment more realistic and improving the accuracy of subsequent calculations. The details are as follows:

[0036] Please refer to Figure 1, which shows a flow chart of a method for restoring point cloud distortion effects provided by an embodiment of the present disclosure. As shown in Figure 1, the method for restoring point cloud distortion effects provided by an embodiment of the present disclosure may include the following steps:

[0037] S100: Obtain a simulation frame rate corresponding to an autonomous driving simulation environment.

[0038] S200: Determine whether the autonomous driving simulation environment is an ultra-high frame rate simulation environment based on the simulation frame rate.

[0039] The autonomous driving simulation environment includes a main vehicle equipped with a lidar, which is in a moving state.

[0040] S300: If the autonomous driving simulation environment is an ultra-high frame rate simulation environment, control the laser radar to rotate and emit multiple lasers in sequence to obtain a point cloud with a distortion effect added.

[0041] S400: If the autonomous driving simulation environment is a low frame rate simulation environment, control the laser radar to simultaneously emit multiple groups of lasers along different angles according to the horizontal angular resolution, and determine the multiple groups of lasers captured in each simulation frame.

[0042] S500 , in each simulation frame, using a ray tracing algorithm to respectively determine the coordinates of the collision point corresponding to each laser beam in the simulation frame.

[0043] S600 , in each simulation frame, correct the collision point coordinates corresponding to each laser beam according to the current vehicle speed vector of the host vehicle and the laser emission time difference corresponding to each laser beam, and determine the corrected collision point coordinates.

[0044] S700 , obtaining a point cloud with added distortion effect based on the corrected collision point coordinates corresponding to each column of lasers.

[0045] In step S200, whether the autonomous driving simulation environment is an ultra-high frame rate simulation environment may be determined by:

[0046] Obtain the rotation frequency and horizontal angular resolution corresponding to the lidar, determine the laser emission time difference corresponding to the lidar based on the rotation frequency and horizontal angular resolution, calculate the simulation time difference based on the simulation frequency, and if the laser emission time difference is greater than or equal to the simulation time difference, determine that the autonomous driving simulation environment is an ultra-high frame rate simulation environment; if the laser emission time difference is less than the simulation time difference, determine that the autonomous driving simulation environment is a low frame rate simulation environment.

[0047] Specifically, the rotation frequency of the laser radar refers to the number of rotations of the laser radar per second, and the horizontal angular resolution refers to the angular interval between two adjacent laser beams of the laser radar.

[0048] In another preferred embodiment, the step of determining the laser emission time difference corresponding to the laser radar according to the rotation frequency and the horizontal angular resolution may include:

[0049] According to the corresponding rotation frequency of the laser radar, the total rotation time taken by the laser radar to rotate one circle is determined. According to the horizontal angular resolution, the total number of lasers emitted when the laser radar rotates one circle is determined. The ratio between the total rotation time and the total number of lasers emitted is determined as the laser emission time difference.

[0050] The laser emission time difference can be calculated by the following formula:

[0051] In this formula, dt represents the time difference of laser emission, n represents the total number of lasers emitted during one rotation of the laser radar, and f represents the total number of lasers emitted during one rotation of the laser radar. Lidar Indicates the rotation frequency corresponding to the laser radar.

[0052] The total number of lasers emitted during one rotation of the laser radar, n, can be determined by the following formula:

[0053] In this formula, θ represents the horizontal angular resolution corresponding to the lidar.

[0054] Among them, the simulation time difference is the inverse of the simulation frequency. Specifically, Among them, f sim Indicates the simulation frequency.

[0055] In this disclosure, for different simulation frame rates, the corresponding algorithm can be selected to reconstruct the point cloud distortion corresponding to the laser radar. The judgment condition is when:

[0056] At this time, the autonomous driving simulation environment is an ultra-high frame rate simulation environment, that is, the simulation frequency is high enough to meet the computing requirements of each laser beam in different frames.

[0057] when:

[0058] At this time, the autonomous driving simulation environment is a low frame rate simulation environment, that is, the simulation frequency cannot meet the requirements of the laser radar frame number, that is, there are lasers that cannot be captured by the simulation frame.

[0059] Step S300 may include:

[0060] Lasers are emitted sequentially according to the time difference of laser emission. For each emitted laser, the following processing is performed: the surrounding environment information and the main vehicle position recorded in the simulation frame corresponding to the emitted laser are extracted, and the ray tracing data generated by the emitted laser in its corresponding simulation frame are calculated using the ray tracing algorithm, the surrounding environment information and the main vehicle position. The ray tracing data includes the distance information and emission angle information between the main vehicle and the collision point with its surrounding environment. After the laser radar rotates one circle, polar coordinate conversion is performed according to the distance information and emission angle information corresponding to each emitted laser to obtain a point cloud with added distortion effect.

[0061] The surrounding environment information of the host vehicle may include other vehicles or obstacles located around the host vehicle, and is not specifically limited here.

[0062] Specifically, the ray tracing algorithm in the related art is that all laser rays are emitted synchronously and calculated at the same time, so there is no distortion effect. In the present disclosure, the laser radar is emitted in sequence according to its own corresponding horizontal angular resolution. Please refer to Figure 2, which shows a schematic diagram of the laser radar action in the ultra-high frame rate simulation environment in the present disclosure. As shown in Figure 2, the horizontal angular resolution of the laser radar is 45°, and it rotates one circle to obtain 1 to 8 laser beams as needed. Taking the first laser beam and the second laser beam as an example, after the first laser beam is emitted, the second laser beam is emitted after the laser emission time difference, and so on, until the laser radar rotates one circle. In this way, for the two adjacent laser beams, due to the existence of the laser emission time difference, the main vehicle carrying the laser radar will be displaced within the laser emission time difference, which can ensure that the main vehicle and its surrounding environment are in dynamic change, so that the point cloud output by the laser radar can be more realistic.

[0063] The distance information and emission angle information corresponding to each emitted laser beam are saved together. When the laser radar rotates one circle, the distance information and emission angle information of all emitted lasers will be output as a point cloud (that is, the conversion from polar coordinates to rectangular coordinates).

[0064] In the present disclosure, considering that the laser radar laser emission time difference may be between 2 frames, when the simulation frame rate is greater than or equal to 1 times the total number of laser emissions, for each laser beam, a random approximate frame is sufficient.

[0065] In step S400, the simulation frame rate is insufficient, resulting in the inability to emit the laser emitted by the laser radar in a separate frame. The above-mentioned algorithm for the ultra-high frame rate simulation environment is not applicable. In order to deal with this situation, it is necessary to control the laser radar to simultaneously emit multiple groups of lasers along different angles according to the horizontal angular resolution, and then calculate the allocation index. The allocation index represents the number of simulation frames generated in the autonomous driving simulation environment during the time when the laser radar rotates one circle. The allocation indication is the ratio between the simulation frame rate and the rotation frequency. After obtaining the allocation index, all lasers emitted within one circle of the laser radar can be evenly distributed to different simulation frames to determine the multiple groups of lasers captured in each simulation frame.

[0066] The allocation index is determined by the following formula:

[0067] In this formula, N represents the allocation index.

[0068] At this time, what is calculated in each simulation frame is no longer one laser beam but multiple laser beams. If only the ray tracing algorithm is used to calculate the point cloud, the calculation result will not have a distortion effect. In the present disclosure, the calculation of each laser beam is corrected through step S500 to perform distortion correction for the final point cloud.

[0069] In a preferred embodiment, in step S600, the corrected collision point coordinates corresponding to each laser beam can be determined by:

[0070] The product of the current vehicle speed vector of the host vehicle corresponding to the laser beam and the laser emission time difference is determined as the host vehicle displacement vector corresponding to the laser beam, and the difference between the collision point coordinates and the host vehicle displacement vector is determined as the corrected collision point coordinates corresponding to the laser beam.

[0071] Specifically, the corrected collision point coordinates of each laser beam can be determined by the following formula: ji =P ji0 -v j dt

[0072] Among them, P ji represents the corrected collision point coordinates corresponding to the i-th laser beam in the j-th simulation frame, P ji0 represents the coordinates of the collision point obtained after the i-th laser beam collides in the j-th simulation frame, v j Represents the current speed vector of the main vehicle extracted in the j-th simulation frame.

[0073] In this disclosure, compared to directly using ray tracing to calculate the collision point coordinates in each simulation frame, it takes into account the main vehicle speed vector and the time difference of laser emission, and makes a correction to the final output collision point coordinates to add a distortion effect to the subsequent point cloud output.

[0074] In a preferred embodiment, step S700 may include:

[0075] For each laser beam, the coordinates of the collision point after correction of the laser beam are transformed into polar coordinates to obtain the corresponding distance information and emission angle information of the laser beam. The distance information and emission angle information corresponding to each laser beam are used to output a point cloud with added distortion effect.

[0076] The processing of distortion effects by autonomous driving perception algorithms is still an important part of algorithm verification. If the point cloud generated by the lidar does not have a distortion effect, it is impossible to verify the algorithm's processing of distortion. Based on this, in this disclosure, starting from the principle of distortion, based on the classic approach of constructing point clouds using the ray tracing algorithm, different strategies are used to generate point clouds with distortion effects for different simulation environments corresponding to the simulation frame rate, so as to facilitate the verification of the autonomous driving perception algorithm's processing of distortion.

[0077] Based on the same application concept, the embodiment of the present disclosure also provides a point cloud distortion effect restoration device corresponding to the point cloud distortion effect restoration method provided in the above embodiment. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the point cloud distortion effect restoration method in the above embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0078] Please refer to Figure 3, which shows a schematic diagram of the structure of a point cloud distortion effect restoration device provided by an embodiment of the present disclosure. As shown in Figure 3, the point cloud distortion effect restoration device may include:

[0079] a frame rate simulation environment determination module 800 configured to obtain a simulation frame rate corresponding to an autonomous driving simulation environment and determine, based on the simulation frame rate, whether the autonomous driving simulation environment is an ultra-high frame rate simulation environment, wherein the autonomous driving simulation environment includes a host vehicle equipped with a laser radar, and the host vehicle is in a moving state;

[0080] A high frame rate distortion restoration module 810 is configured to: if the autonomous driving simulation environment is an ultra-high frame rate simulation environment, control the laser radar to rotate and sequentially emit multiple lasers to obtain a point cloud with an added distortion effect;

[0081] a first low-frame-rate distortion restoration module 820 configured to: if the autonomous driving simulation environment is a low-frame-rate simulation environment, control the laser radar to simultaneously emit multiple groups of lasers at different angles according to a horizontal angular resolution, and determine the multiple groups of lasers captured in each simulation frame;

[0082] Correction module 830, configured to: determine the collision point coordinates corresponding to each laser beam within each simulation frame using a ray tracing algorithm, and correct the collision point coordinates corresponding to each laser beam based on the current vehicle speed vector corresponding to each laser beam and the laser emission time difference, thereby determining the corrected collision point coordinates;

[0083] The second low frame rate distortion restoration module 840 is configured to obtain a point cloud with added distortion effect based on the corrected collision point coordinates corresponding to each laser beam.

[0084] Preferably, the frame rate simulation environment judgment module 800 is also configured to: obtain the rotation frequency and horizontal angular resolution corresponding to the laser radar; determine the laser emission time difference corresponding to the laser radar based on the rotation frequency and horizontal angular resolution; calculate the simulation time difference based on the simulation frequency; if the laser emission time difference is greater than or equal to the simulation time difference, determine that the autonomous driving simulation environment is an ultra-high frame rate simulation environment; if the laser emission time difference is less than the simulation time difference, determine that the autonomous driving simulation environment is a low frame rate simulation environment.

[0085] Preferably, the frame rate simulation environment judgment module 800 is also configured to: determine the total rotation time taken by the laser radar to rotate one circle according to the rotation frequency corresponding to the laser radar; determine the total number of lasers emitted when the laser radar rotates one circle according to the horizontal angular resolution; and determine the ratio between the total rotation time and the total number of lasers emitted as the laser emission time difference.

[0086] Preferably, the high frame rate distortion restoration module 810 is also configured to: perform the following processing for each emitted laser: extract the surrounding environment information and the main vehicle position recorded in the simulation frame corresponding to the emitted laser; use the ray tracing algorithm, the surrounding environment information of the main vehicle and the main vehicle position to respectively calculate the ray tracing data generated by the emitted laser in its corresponding simulation frame, the ray tracing data including the distance information and emission angle information between the main vehicle and its surrounding collision point; when the laser radar rotates one circle, polar coordinate conversion is performed according to the distance information and emission angle information corresponding to each emitted laser to obtain a point cloud with added distortion effect.

[0087] Preferably, the correction module 830 is further configured to: determine the product of the current vehicle speed vector of the main vehicle corresponding to the laser beam and the laser emission time difference as the main vehicle displacement vector corresponding to the laser beam, and determine the difference between the collision point coordinates and the main vehicle displacement vector as the corrected collision point coordinates corresponding to the laser beam.

[0088] Preferably, the second low frame rate distortion restoration module 840 is further configured to: for each laser beam, perform polar coordinate conversion on the coordinates of the collision point after correction of the laser beam to obtain the distance information and emission angle information corresponding to the laser beam; and use the distance information and emission angle information corresponding to each laser beam to output a point cloud with added distortion effect.

[0089] Based on the same application concept, please refer to Figure 4, which shows a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. Electronic device 900 may include: a processor 910, a memory 920, and a bus 930. The memory 920 stores machine-readable instructions executable by the processor 910. When the electronic device 900 is running, the processor 910 and the memory 920 communicate via the bus 930. The machine-readable instructions are executed by the processor 910 to execute the steps of the point cloud distortion effect restoration method provided in any of the above embodiments.

[0090] Based on the same application concept, an embodiment of the present disclosure further provides a computer-readable storage medium, which may store a computer program. When the computer program is executed by a processor, the steps of the point cloud distortion effect restoration method provided in the above embodiment are executed.

[0091] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0092] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0093] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0094] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0095] The above are only specific embodiments of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims. Industrial Applicability

[0096] The present disclosure provides a method for restoring point cloud distortion effects, a device for restoring point cloud distortion effects, an electronic device, and a storage medium, and relates to the field of autonomous driving technology, including: if the autonomous driving simulation environment is a low frame rate simulation environment, controlling a laser radar to simultaneously emit multiple groups of lasers along different angles according to the horizontal angular resolution, and determining the multiple groups of lasers captured in each simulation frame; within each simulation frame, using a ray tracing algorithm to respectively determine the collision point coordinates corresponding to each laser beam in the simulation frame, and correcting the collision point coordinates corresponding to each laser beam based on the current vehicle speed vector of the host vehicle corresponding to each laser beam and the laser emission time difference, to determine the corrected collision coordinates; and obtaining a point cloud with added distortion effects from the corrected collision point coordinates corresponding to each column of lasers. The present disclosure adds distortion effects to the point cloud through the motion characteristics of the host vehicle, making the autonomous driving environment more realistic and improving the accuracy of subsequent calculations.

[0097] Furthermore, it is understood that the disclosed point cloud distortion effect restoration device is reproducible and can be used in a variety of industrial applications. For example, the disclosed point cloud distortion effect restoration device can be used in any application that requires a point cloud distortion effect restoration device to make the autonomous driving environment more realistic.

Claims

1. A method for restoring point cloud distortion effects, wherein: The point cloud distortion effect restoration method comprises: Obtaining a simulation frame rate corresponding to an autonomous driving simulation environment, and determining whether the autonomous driving simulation environment is an ultra-high frame rate simulation environment according to the simulation frame rate, wherein the autonomous driving simulation environment includes a main vehicle equipped with a laser radar, and the main vehicle is in a moving state; If the autonomous driving simulation environment is an ultra-high frame rate simulation environment, controlling the laser radar to rotate and sequentially emit multiple lasers to obtain a point cloud with a distortion effect added; If the autonomous driving simulation environment is a low frame rate simulation environment, controlling the laser radar to simultaneously emit multiple groups of lasers along different angles according to the horizontal angular resolution, and determining the multiple groups of lasers captured in each simulation frame; In each simulation frame, the ray tracing algorithm is used to determine the coordinates of the collision point corresponding to each laser beam in the simulation frame, and the coordinates of the collision point corresponding to each laser beam are corrected according to the current vehicle speed vector of the host vehicle corresponding to each laser beam and the laser emission time difference, and the corrected collision point coordinates are determined; The point cloud with added distortion effect is obtained from the corrected collision point coordinates corresponding to each laser beam.

2. The method for restoring point cloud distortion effect according to claim 1, wherein: Whether the autonomous driving simulation environment is an ultra-high frame rate simulation environment is determined by the following method: Obtaining the rotation frequency and horizontal angular resolution corresponding to the laser radar; Determining a laser emission time difference corresponding to the laser radar according to the rotation frequency and the horizontal angle resolution; Calculating a simulation time difference according to the simulation frequency; If the laser emission time difference is greater than or equal to the simulation time difference, determining that the autonomous driving simulation environment is an ultra-high frame rate simulation environment; If the laser emission time difference is less than the simulation time difference, it is determined that the autonomous driving simulation environment is a low frame rate simulation environment.

3. The method for restoring point cloud distortion effect according to claim 2, wherein: The step of determining the laser emission time difference corresponding to the laser radar according to the rotation frequency and the horizontal angle resolution comprises: Determine the total rotation time taken by the laser radar to rotate one circle according to the rotation frequency corresponding to the laser radar; Determine the total number of lasers emitted by the laser radar during one rotation according to the horizontal angular resolution; The ratio between the total rotation time and the total number of laser shots is determined as the laser shot time difference.

4. The method for restoring point cloud distortion effect according to any one of claims 1 to 3, wherein: If the autonomous driving simulation environment is an ultra-high frame rate simulation environment, the point cloud with added distortion effect is obtained by the following method: The lasers are emitted sequentially according to the laser emission time difference; For each laser shot, the following processing is performed: Extracting the surrounding environment information and the position of the main vehicle recorded in the simulation frame corresponding to the emitted laser; Using the ray tracing algorithm, the surrounding environment information of the main vehicle and the position of the main vehicle, the ray tracing data generated by the emitted laser in its corresponding simulation frame are calculated respectively, and the ray tracing data includes the distance information between the main vehicle and the collision point of its surrounding environment and the emission angle information; After the laser radar rotates one circle, polar coordinate conversion is performed based on the distance information and emission angle information corresponding to each emitted laser to obtain a point cloud with added distortion effect.

5. The method for restoring point cloud distortion effect according to any one of claims 1 to 4, wherein: The corrected collision point coordinates for each laser beam are determined by: The product of the current vehicle speed vector of the host vehicle corresponding to the laser beam and the laser emission time difference is determined as the displacement vector of the host vehicle corresponding to the laser beam; The difference between the collision point coordinates and the host vehicle displacement vector is determined as the corrected collision point coordinates corresponding to the laser beam.

6. The method for restoring point cloud distortion effect according to any one of claims 1 to 5, wherein: The steps of obtaining a point cloud with a distortion effect added from the corrected collision point coordinates corresponding to each laser beam include: For each laser beam, the coordinates of the collision point after correction of the laser beam are transformed into polar coordinates to obtain the distance information and emission angle information corresponding to the laser beam; Using the distance information and emission angle information corresponding to each laser beam, a point cloud with distortion effect added is output.

7. A point cloud distortion effect restoration device, wherein: The point cloud distortion effect restoration device comprises: a frame rate simulation environment judgment module, wherein the frame rate simulation environment judgment module is configured to: obtain a simulation frame rate corresponding to an autonomous driving simulation environment and judge whether the autonomous driving simulation environment is an ultra-high frame rate simulation environment according to the simulation frame rate, wherein the autonomous driving simulation environment includes a main vehicle equipped with a laser radar, and the main vehicle is in a moving state; A high frame rate distortion restoration module, wherein the high frame rate distortion restoration module is configured to: if the autonomous driving simulation environment is an ultra-high frame rate simulation environment, control the laser radar to rotate and sequentially emit multiple lasers to obtain a point cloud with a distortion effect added; a first low frame rate distortion restoration module, wherein the first low frame rate distortion restoration module is configured to: if the autonomous driving simulation environment is a low frame rate simulation environment, control the laser radar to simultaneously emit multiple groups of lasers along different angles according to a horizontal angular resolution, and determine the multiple groups of lasers captured by each simulation frame; A correction module, wherein the correction module is configured to: determine the collision point coordinates corresponding to each laser beam in each simulation frame by using a ray tracing algorithm, and correct the collision point coordinates corresponding to each laser beam according to the current vehicle speed vector of the host vehicle corresponding to each laser beam and the laser emission time difference, and determine the corrected collision point coordinates; The second low frame rate distortion restoration module is configured to obtain a point cloud with added distortion effect from the corrected collision point coordinates corresponding to each laser beam.

8. The point cloud distortion effect restoration device according to claim 7, wherein: The frame rate simulation environment judgment module is further configured to: judge whether the autonomous driving simulation environment is an ultra-high frame rate simulation environment by: Obtaining the rotation frequency and horizontal angular resolution corresponding to the laser radar; Determining a laser emission time difference corresponding to the laser radar according to the rotation frequency and the horizontal angle resolution; Calculating a simulation time difference according to the simulation frequency; If the laser emission time difference is greater than or equal to the simulation time difference, determining that the autonomous driving simulation environment is an ultra-high frame rate simulation environment; If the laser emission time difference is less than the simulation time difference, it is determined that the autonomous driving simulation environment is a low frame rate simulation environment.

9. The point cloud distortion effect restoration device according to claim 8, wherein: The frame rate simulation environment judgment module is further configured to: determine the laser emission time difference corresponding to the laser radar according to the rotation frequency and the horizontal angle resolution, comprising: Determine the total rotation time taken by the laser radar to rotate one circle according to the rotation frequency corresponding to the laser radar; Determine the total number of lasers emitted by the laser radar during one rotation according to the horizontal angular resolution; The ratio between the total rotation time and the total number of laser shots is determined as the laser shot time difference.

10. The point cloud distortion effect restoration device according to any one of claims 7 to 9, wherein: The high frame rate distortion restoration module is further configured to: if the autonomous driving simulation environment is an ultra-high frame rate simulation environment, obtain a point cloud with added distortion effect by: The lasers are emitted sequentially according to the laser emission time difference; For each laser shot, the following processing is performed: Extracting the surrounding environment information and the position of the main vehicle recorded in the simulation frame corresponding to the emitted laser; Using the ray tracing algorithm, the surrounding environment information of the main vehicle and the position of the main vehicle, the ray tracing data generated by the emitted laser in its corresponding simulation frame are calculated respectively, and the ray tracing data includes the distance information between the main vehicle and the collision point of its surrounding environment and the emission angle information; After the laser radar rotates one circle, polar coordinate conversion is performed based on the distance information and emission angle information corresponding to each emitted laser to obtain a point cloud with added distortion effect.

11. The point cloud distortion effect restoration device according to any one of claims 7 to 10, wherein: The correction module is further configured to determine the corrected collision point coordinates corresponding to each laser beam by: The product of the current vehicle speed vector of the host vehicle corresponding to the laser beam and the laser emission time difference is determined as the displacement vector of the host vehicle corresponding to the laser beam; The difference between the collision point coordinates and the host vehicle displacement vector is determined as the corrected collision point coordinates corresponding to the laser beam.

12. The point cloud distortion effect restoration device according to any one of claims 7 to 11, wherein: The second low frame rate distortion restoration module is further configured to: obtain a point cloud with added distortion effect from the corrected collision point coordinates corresponding to each laser beam, comprising: For each laser beam, the coordinates of the collision point after correction of the laser beam are transformed into polar coordinates to obtain the distance information and emission angle information corresponding to the laser beam; Using the distance information and emission angle information corresponding to each laser beam, a point cloud with distortion effect added is output.

13. An electronic device, wherein: The electronic device includes: a processor, a memory and a bus, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the point cloud distortion effect restoration method as described in any one of claims 1 to 6.

14. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the point cloud distortion effect restoration method according to any one of claims 1 to 6 are executed.

Images