Simulation point cloud data generation method and device, electronic equipment and storage medium

Abstract

The invention provides a simulation point cloud data generation method and device, electronic equipment and a storage medium, and the method comprises the steps: collecting a multi-view image for a target object model background, carrying out the reconstruction of a three-dimensional geometric structure according to the multi-view image, and obtaining a three-dimensional point cloud model of the target object model background; based on a scanning mechanism of a laser radar, simulating a reflection process of laser beams with different directions and scanning line numbers on the surface of the three-dimensional point cloud model of the target object model background, and obtaining simulation point cloud data of the target object model background; and according to topographic factors in the simulation point cloud data of the target object model background, fitting a pre-acquired target object model point cloud to the simulation point cloud data of the target object model background in an adaptive manner to obtain simulation point cloud data including the target object model and the target object model background. Therefore, a large amount of diversified target object point cloud data can be generated without depending on real laser radar acquisition, so that the cost is saved, and the training data set is remarkably expanded.

Description

Technical Field

[0001] One or more embodiments of the present disclosure relate to the technical field of point cloud generation, and in particular, to a method, apparatus, electronic device, and storage medium for generating simulated point cloud data. Background Art

[0002] With the rapid development of mine autonomous driving technology, the demand for high-quality vehicle perception data for training deep learning models has increased significantly. However, in a complex, variable, and unique environment such as a mine, traditional perception data acquisition methods face many challenges in obtaining high-quality lidar point cloud data of specific target objects (such as vehicles, mechanical equipment, etc.):

[0003] First, the difficulty of point cloud data acquisition is high. The mine terrain has large undulations and a complex operating environment. Operating lidar equipment on rough ground and ensuring data quality is a highly challenging task, making the acquisition of real and effective point cloud data not only costly but also require a large amount of human input.

[0004] Second, the amount of point cloud data is insufficient. Due to the significant regional and scenario dependence of mine scenes, the specific conditions (such as geological structure, equipment configuration, etc.) of each mine vary greatly, and general perception data sets are difficult to comprehensively cover the typical working conditions unique to mines, resulting in a severe shortage of available high-quality training data. This limitation restricts the performance improvement and wide application of deep learning models in mine autonomous driving.

[0005] Therefore, there is an urgent need for a method that can efficiently generate simulated point cloud data conforming to the characteristics of the mine environment to fill this gap. Summary of the Invention

[0006] To solve the problem that traditional perception data acquisition methods are not convenient for obtaining high-quality lidar point cloud data of specific target objects in complex environments, the present disclosure provides a method for generating simulated point cloud data, the method comprising:

[0007] Collect multi-view images of the background of the target object model, and perform three-dimensional geometric structure reconstruction based on the multi-view images to obtain a three-dimensional point cloud model of the background of the target object model;

[0008] Based on the scanning mechanism of the lidar, simulate the reflection process of laser beams in different directions and with different numbers of scan lines on the surface of the three-dimensional point cloud model of the background of the target object model, so as to obtain the simulated point cloud data of the background of the target object model;

[0009] According to the terrain factors in the simulated point cloud data of the target object model background, adapt and fit the pre-acquired target object model point cloud to the simulated point cloud data of the target object model background, so as to obtain the simulated point cloud data including the target object model and the target object model background.

[0010] Optionally, the scanning mechanism based on lidar simulates the reflection process of laser beams with different directions and numbers of scan lines on the surface of the three-dimensional point cloud model of the target object model background, so as to obtain the simulated point cloud data of the target object model background, including:

[0011] Based on the scanning mechanism of the lidar, simulate and generate laser beams with different directions and numbers of scan lines;

[0012] Use the ray-triangle intersection algorithm to calculate the intersection points of each generated laser beam with the surface of the three-dimensional point cloud model of the target object model background;

[0013] Calculate the reflection direction of each generated laser beam according to the law of reflection to simulate the lidar receiving the reflected signal, and use the received signal intensity and the intersection point of each laser beam as the simulated point cloud data of the target object model background.

[0014] Optionally, the simulating and generating laser beams with different directions and numbers of scan lines based on the scanning mechanism of the lidar includes:

[0015] Based on the scanning mechanism of the lidar, simulate and generate laser beams with different directions and numbers of scan lines, and add Gaussian noise to the measurement distance of the laser beams;

[0016] Simulate the multipath reflection of the laser beams on the surface of the three-dimensional point cloud model of the target object model background to obtain additional noise points.

[0017] Optionally, after obtaining the simulated point cloud data of the target object model background, the method further includes:

[0018] Extract the point cloud data of the target object in the simulated point cloud data of the target object model background, and calculate the main direction of the target object according to the normal vector distribution of the point cloud data of the target object; and calculate the three-dimensional coordinates of the target object according to the geometric center or centroid of the point cloud data of the target object.

[0019] Perform three-dimensional annotation on the target object in the simulated point cloud data of the target object model background according to the three-dimensional coordinates and main direction of the target object, as well as the pre-defined target object size and category information.

[0020] Optionally, the method further includes: adapting and fitting the pre-acquired target object model point cloud to the simulated point cloud data of the target object model background according to the terrain factors in the simulated point cloud data of the target object model background, including:

[0021] Determine the terrain factors in the simulated point cloud data of the target object model background according to the three-dimensionally labeled target object;

[0022] Fit the target object model point cloud to the simulated point cloud data of the target object model background through a collision detection algorithm and the terrain factors in the simulated point cloud data of the target object model background;

[0023] Determine the partial point cloud of the fitted target object model point cloud occluded by the existing obstacles in the simulated point cloud data of the target object model background, and delete the partial point cloud.

[0024] Optionally, determining the terrain factors in the simulated point cloud data of the target object model background according to the three-dimensionally labeled target object includes:

[0025] Project the three-dimensionally labeled target object onto a two-dimensional plane to obtain a two-dimensionally labeled target object;

[0026] Count the number of original point clouds in the area where the two-dimensionally labeled target object is located to determine the terrain factors in the simulated point cloud data of the target object model background and the position where the target object model point cloud is to be fitted.

[0027] Optionally, fitting the target object model point cloud to the simulated point cloud data of the target object model background through a collision detection algorithm and the terrain factors in the simulated point cloud data of the target object model background includes:

[0028] If the two-dimensionally labeled target object includes a road surface, move the lowest point of the target object model point cloud to the road surface;

[0029] Perform a collision detection on the position where the target object model point cloud is to be fitted and the labeled target object;

[0030] If there is no collision between the position where the target object model point cloud is to be fitted and the labeled target object, fit the target object model point cloud to the simulated point cloud data of the target object model background according to the position to be fitted.

[0031] The present disclosure also provides a simulated point cloud data generation device, the device includes:

[0032] A reconstruction unit, configured to collect multi-view images of the background of a target object model, and perform three-dimensional geometric structure reconstruction based on the multi-view images to obtain a three-dimensional point cloud model of the background of the target object model;

[0033] A simulation unit, configured to simulate the reflection process of laser beams emitted in different directions and densities on the surface of the three-dimensional point cloud model of the background of the target object model based on the scanning mechanism of a lidar, so as to obtain simulated point cloud data of the background of the target object model;

[0034] A fitting unit, configured to adapt and fit a pre-acquired point cloud of a target object model to the simulated point cloud data of the background of the target object model according to terrain factors in the simulated point cloud data of the background of the target object model, so as to obtain simulated point cloud data including the target object model and the background of the target object model.

[0035] The present disclosure further provides an electronic device, including a communication interface, a processor, a memory, and a bus, where the communication interface, the processor, and the memory are interconnected through the bus;

[0036] Machine-readable instructions are stored in the memory, and the processor executes the above method by calling the machine-readable instructions.

[0037] The present disclosure further provides a machine-readable storage medium, where machine-readable instructions are stored in the machine-readable storage medium, and when the machine-readable instructions are called and executed by a processor, the above method is implemented.

[0038] Through the embodiments of the present disclosure, first, multi-view images of the background of a target object model are collected, and three-dimensional geometric structure reconstruction is performed using these images to generate a three-dimensional point cloud model of the background, so as to accurately capture the spatial information of a complex environment; further, based on the scanning mechanism of a lidar, the reflection process of laser beams in different directions and with different numbers of scan lines on the surface of the three-dimensional point cloud model of the background of the target object model is simulated to obtain simulated point cloud data of the background of the target object model, so as to truly reproduce the process of obtaining point cloud data by the lidar in an actual environment; finally, according to terrain factors (such as ground height, obstacle distribution, etc.) in the generated simulated point cloud data, the pre-acquired point cloud of the target object model is adaptively fitted into the simulated point cloud data of the background of the target object model.

[0039] In the above manner, the technical solution of the present disclosure realizes seamless fusion of the target object model and the background environment by comprehensively applying multi-view image reconstruction, lidar scanning mechanism simulation, and intelligent adaptation technology, and then generates high-quality and high-fidelity simulated point cloud data. Accordingly, the present disclosure effectively solves the problem of difficult large-scale acquisition of high-quality perception data in a complex environment, and provides rich and reliable training data support for various application scenarios. Brief Description of the Drawings

[0040] To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments recorded in the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0041] Figure 1 is a flowchart of a method for aligning point cloud and image pixels shown in an exemplary embodiment;

[0042] Figure 2 is a schematic diagram of another method for generating simulated point cloud data shown in an exemplary embodiment;

[0043] Figure 3 is a schematic diagram of simulated point cloud data including a target object model and the background of the target object model after fitting shown in an exemplary embodiment;

[0044] Figure 4 is a schematic diagram of yet another method for generating simulated point cloud data shown in an exemplary embodiment;

[0045] Figure 5 is a hardware structure diagram of an electronic device shown in an exemplary embodiment;

[0046] Figure 6 is a block diagram of a device for generating simulated point cloud data shown in an exemplary embodiment. Detailed Description of the Embodiments

[0047] In order to enable those skilled in the art to better understand the technical solutions in the present disclosure, the following will clearly and completely describe the technical solutions in the embodiments of the present disclosure in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present disclosure.

[0048] It should be noted that: in other embodiments, the steps of the corresponding methods are not necessarily executed in the order shown and described in the present disclosure. In some other embodiments, the steps included in the method may be more or less than those described in the present disclosure. In addition, a single step described in the present disclosure may be decomposed into multiple steps for description in other embodiments; and multiple steps described in the present disclosure may also be combined into a single step for description in other embodiments.

[0049] With the rapid development of autonomous driving technology in mines, the demand for high-quality vehicle perception data for training deep learning models has increased significantly. However, in the complex, ever-changing and unique environment of mines, traditional perception data collection methods face many challenges in obtaining high-quality lidar point cloud data of specific target objects (such as vehicles, mechanical equipment, etc.):

[0050] First of all, the difficulty of point cloud data collection is high. The terrain in mines has large undulations and complex operating environments. Operating lidar equipment on rough ground and ensuring data quality is an extremely challenging task, making the collection of real and effective point cloud data not only require high costs but also a large amount of human input.

[0051] Secondly, the amount of point cloud data is insufficient. Due to the significant regional and scenario dependencies of mine scenes, the specific conditions (such as geological structures, equipment configurations, etc.) of each mine vary greatly, and general perception datasets are difficult to comprehensively cover the typical working conditions unique to mines, resulting in a serious shortage of available high-quality training data. This limitation restricts the performance improvement and wide application of deep learning models in mine autonomous driving.

[0052] Therefore, there is an urgent need for a method that can efficiently generate simulation point cloud data conforming to the characteristics of the mine environment to make up for this gap.

[0053] In view of this, the present disclosure aims to propose a technical solution for generating high-quality simulation point cloud data in different terrain environments.

[0054] This technical solution first collects multi-view images of the background of the target object model, uses these images for three-dimensional geometric structure reconstruction to generate a three-dimensional point cloud model of the background, so as to accurately capture the spatial information of the complex environment; further, based on the scanning mechanism of lidar, simulate the reflection process of laser beams with different directions and numbers of scan lines on the surface of the three-dimensional point cloud model of the background of the target object model to obtain the simulation point cloud data of the background of the target object model, so as to truly reproduce the process of lidar obtaining point cloud data in the actual environment; finally, according to the terrain factors (such as ground height, obstacle distribution, etc.) in the generated simulation point cloud data, fit the pre-obtained point cloud of the target object model to the simulation point cloud data of the background of the target object model.

[0055] For example, the simulation point cloud data of an autonomous vehicle (as the target object) is generated in a mine environment. First, multiple high-resolution photos are taken at different positions in the mine using drones or ground mobile devices. Generally, it is necessary to ensure that there is sufficient overlapping area between the photos to support subsequent 3D reconstruction. The processor uses the collected multi-view photos and performs 3D geometric structure reconstruction using stereo vision algorithms to generate a 3D point cloud model, which accurately captures the spatial information of the complex mine environment, such as terrain undulations, pit contours, and the positions of various obstacles. Then, based on the scanning mechanism of the lidar (such as scanning period, emission angle, number of scan lines, etc.), the processor simulates the reflection process of laser beams with different directions and densities on the surface of the above-generated 3D point cloud model to obtain the simulation point cloud data of the mine background, truly reproducing the process of the lidar obtaining point cloud data in the actual environment. Finally, according to the terrain factors (such as ground height, obstacle distribution, etc.) in the generated simulation point cloud data of the mine background, the pre-acquired vehicle point cloud is adapted and fitted into the simulation point cloud data of the mine background. This step can include terrain analysis, collision detection, and adaptation and fitting to ensure that the vehicle point cloud naturally integrates into the background point cloud.

[0056] It should be noted that this method is not only applicable to mine environments but also can be applied to many other environments, such as urban streets, forests, or airports. At the same time, the target object is not limited to vehicles and can be any object that needs to be modeled, such as pedestrians, animals, or luggage handling robots.

[0057] It can be seen that the technical solution of the present disclosure, by comprehensively using multi-view image reconstruction, lidar scanning mechanism simulation, and intelligent adaptation technology, fits the target object model into the background point cloud, realizes the seamless integration of the target object and the background environment, and thus generates high-quality and high-fidelity simulation point cloud data. Accordingly, the present disclosure effectively solves the problem of difficult large-scale acquisition of high-quality perception data in complex environments and provides rich and reliable training data support for various application scenarios.

[0058] Next, the present disclosure will be described through specific embodiments in combination with specific application scenarios.

[0059] Please refer to Figure 1 , Figure 1 which is a flowchart of a method for aligning point cloud and image pixels shown in an exemplary embodiment. The method can perform the following steps:

[0060] Step 102: Collect multi-view images of the background of the target object model to perform 3D geometric structure reconstruction based on the multi-view images to obtain a 3D point cloud model of the background of the target object model.

[0061] For example, in a mine environment, simulated point cloud data of an autonomous vehicle (as the target object) is generated. Multiple high-resolution photos at different angles need to be taken using drones or ground mobile devices at different positions in the mine, and usually, sufficient overlapping areas between the photos need to be ensured to support subsequent 3D reconstruction. Through these multi-view photos collected, the processor uses stereo vision algorithms to reconstruct the 3D geometric structure and generate a 3D point cloud model, which accurately captures the spatial information of the complex mine environment, such as terrain undulations, pit contours, and the positions of various obstacles.

[0062] Among them, in 3D reconstruction technology, it is usually divided into two stages: sparse reconstruction and dense reconstruction. Sparse reconstruction is mainly to restore the basic geometric structure of the scene and generate a sparse point cloud of the scene. This process usually includes steps such as feature point detection, matching, epipolar geometric constraint, triangulation, and bundle adjustment. The result of sparse reconstruction is usually a set of relatively few but sufficient point cloud data to describe the general shape of the scene. These points are often located at the edges of objects or where there are obvious texture changes. Dense reconstruction is based on the result of sparse reconstruction, calculates the depth information pixel by pixel, and generates a dense 3D point cloud model to further restore more surface details representing the scene. This step involves techniques such as depth map generation and optimization, such as depth estimation algorithms based on epipolar line search. After generating the dense 3D point cloud model, post-processing is usually required to improve the visual effect and practicality of the model. Common post-processing methods include point cloud filtering, meshing, and texture mapping.

[0063] Step 104: Based on the scanning mechanism of the lidar, simulate the reflection process of laser beams with different directions and numbers of scan lines on the surface of the 3D point cloud model of the background of the target object model, so as to obtain the simulated point cloud data of the background of the target object model.

[0064] For example, based on the scanning mechanism of the lidar (such as scanning period, emission angle, number of scan lines, etc.), the processor simulates the reflection process of laser beams with different directions and densities on the surface of the above-generated 3D point cloud model, and obtains the simulated point cloud data of the mine background, truly reproducing the process of the lidar obtaining point cloud data in the actual environment.

[0065] Among them, the scanning period refers to the time required for the lidar to complete one full scan. By adjusting the scanning period, the speed and accuracy of the lidar collecting data can be controlled. The emission angle is the angle of the laser beam emitted by the lidar relative to the horizontal plane, which determines the coverage range and density of the laser beam. Different emission angles can simulate scanning effects at different heights and directions. The number of scan lines of the lidar determines the density of the point cloud. The more the number of lines, the higher the point cloud density.

[0066] Step 106: According to the terrain factors in the simulated point cloud data of the target object model background, adapt and fit the pre-acquired target object model point cloud to the simulated point cloud data of the target object model background, so as to obtain the simulated point cloud data including the target object model and the target object model background.

[0067] For example, according to the terrain factors (such as ground height, obstacle distribution, etc.) in the generated simulated point cloud data of the mine background, adapt and fit the pre-acquired vehicle point cloud to the simulated point cloud data of the mine background. This step may include terrain analysis, collision detection, and adaptation and fitting to ensure that the vehicle point cloud is naturally integrated into the background point cloud.

[0068] Among them, the three-dimensionally labeled target object can be projected onto a two-dimensional plane from the top view angle, and the number of original point clouds in this area can be counted to evaluate the terrain features (such as ground height, obstacle distribution, etc.). The collision detection algorithm is used to confirm whether there is a conflict between the placement position of the target object and other objects. The distance and relative position between the target object model point cloud and the simulated point cloud data of the target object model background can be calculated to avoid unnecessary overlap or interference. During adaptation and fitting, if an obstacle (such as a pile of gravel) is detected, it will not be fitted. If a road surface point is detected, the lowest point position of the target object will be adjusted according to the road surface height so that it just touches the ground. Through the above method, it can be ensured that the target object naturally integrates into the background environment and avoids suspension or sinking into the ground.

[0069] In an illustrated embodiment, the lidar-based scanning mechanism simulates the reflection process of laser beams with different directions and numbers of scan lines on the surface of the three-dimensional point cloud model of the target object model background, so as to obtain the simulated point cloud data of the target object model background, including: based on the lidar-based scanning mechanism, simulating and generating laser beams with different directions and numbers of scan lines; using the ray-triangle intersection algorithm to calculate the intersection points of each generated laser beam with the surface of the three-dimensional point cloud model of the target object model background; calculating the reflection direction of each generated laser beam according to the law of reflection to simulate the lidar receiving the reflected signal, and taking the received signal intensity and the intersection point of each laser beam as the simulated point cloud data of the target object model background.

[0070] For example, the scanning cycle of the laser radar is 10 times per second, the emission angle ranges from -30 degrees to +30 degrees, and the number of scanning lines is 64. The processor generates laser beams of different directions and densities based on these parameters to simulate the scanning mechanism of the laser radar. Then, the intersection of each laser beam and the surface of the three-dimensional point cloud model is calculated using the ray and triangle patch intersection algorithm. For example, a laser beam intersects with a triangle patch at the edge of the mine at a certain angle, and the processor records the position coordinates (x, y, z) of the intersection. Another laser beam may intersect with the top of the vehicle, and the processor also records the position of its intersection. The processing method for the remaining laser beams is similar. Finally, according to the law of reflection, the reflection direction of each laser beam is determined, and the process of the laser radar receiving the reflected signal is simulated. For example, the incident angle of a laser beam is 30 degrees. According to the law of reflection, the processor calculates that the reflection angle is also 30 degrees, and records the intensity of the reflected signal. The intersection of each laser beam and the received signal intensity are used as the simulation point cloud data of the target object model background.

[0071] Among them, the ray and triangle intersection algorithm is a method for determining whether a ray intersects with a triangle and the location of the intersection point. In the process of generating 3D point cloud data, this method is often used to simulate the working principle of LiDAR and calculate the intersection of each laser beam (ray) with the surface of an object in the environment (represented by a triangular mesh). The signal strength reflects the strength of the laser beam reflected back by the object. The calculation of signal strength can be based on factors such as the reflectivity of the material and the distance. For example, the reflectivity of the metal surface of a vehicle is higher, so the signal strength is higher; while the reflectivity of the soil at the bottom of the mine pit is lower, so the signal strength is relatively low.

[0072] In one embodiment shown, the scanning mechanism based on the laser radar simulates the generation of laser beams in different directions and with different numbers of scanning lines, including: based on the scanning mechanism of the laser radar, simulates the generation of laser beams in different directions and with different numbers of scanning lines, and adds Gaussian noise to the measurement distance of the laser beam; simulates the multi-path reflection of the laser beam on the surface of the three-dimensional point cloud model of the background of the target object model to obtain additional noise points.

[0073] For example, the scanning mechanism of a 16-line lidar includes a horizontal angle range from 0 degrees to 360 degrees, generating a laser beam every 0.2 degrees, a vertical angle range from -15 degrees to +15 degrees, generating a laser beam every 2 degrees (a total of 16 scanning lines), a scanning period of 0.1 seconds, and generating a complete scan every 0.1 seconds. Simulate the scanning mechanism of the 16-line lidar, simulate the generation of laser beams in different directions and with different numbers of scanning lines, and for the measured distance of each laser beam, add a Gaussian noise with a mean of 0 and a standard deviation of 0.02 meters. For example, if the measured distance of a laser beam is 10 meters, it may become 9.98 meters or 10.02 meters after adding the noise. Also, simulate the situation of multiple reflections of laser beams in a complex environment. For example, when a laser beam hits the metal surface of a vehicle, two reflections may occur: the first reflection occurs on the top of the vehicle, and the second reflection occurs at the bottom of the mine pit. The processor records the signal intensity and intersection position after each reflection and generates additional noise points.

[0074] Among them, Gaussian noise is a kind of random noise that conforms to the Gaussian distribution (normal distribution), with a mean of 0, and the standard deviation determines the intensity of the noise. Adding Gaussian noise in the simulation can simulate the measurement error of the lidar in the actual environment and improve the authenticity of the simulated point cloud data. Multipath reflection refers to the phenomenon that laser beams are reflected multiple times on complex surfaces. Simulating multipath reflection in the simulation can generate additional noise points and further improve the authenticity of the simulated point cloud data.

[0075] In an illustrated embodiment, after obtaining the simulated point cloud data of the background of the target object model, the method further includes: extracting the point cloud data of the target object from the simulated point cloud data of the background of the target object model, and calculating the main direction of the target object according to the normal vector distribution of the point cloud data of the target object; and calculating the three-dimensional coordinates of the target object according to the geometric center or centroid of the point cloud data of the target object; performing three-dimensional annotation on the target object in the simulated point cloud data of the background of the target object model according to the three-dimensional coordinates and the main direction of the target object, as well as the pre-defined size and category information of the target object.

[0076] For example, the background of the target object model is a mine. The simulated point cloud data of the mine usually contains the point cloud data of objects such as the ground and gravel piles, which can reflect the geographical factors of the mine. Therefore, it is necessary to separate the point cloud data of the ground and gravel piles from the simulated point cloud data of the mine, and this process can be achieved through a semantic segmentation algorithm. Then, calculate the normal vectors of the separated ground point cloud data and the separated gravel pile point cloud data, so as to obtain the main directions of the ground and gravel piles. For example, the ground is roughly parallel to the Z-axis, and the gravel pile is roughly parallel to the X-axis. Next, take the average values of the coordinates of the ground point cloud data and the gravel pile point cloud data respectively, so as to obtain the centroid positions of the ground point cloud data and the gravel pile point cloud data. For example, the three-dimensional coordinates of the centroid position of the ground point cloud data are (50, 20, 0), and the three-dimensional coordinates of the centroid position of the gravel pile point cloud data are (60, 25, 5), which represent the specific positions of the ground and gravel piles in the mine background. Finally, according to the three-dimensional coordinates and main directions of the target object, as well as the pre-defined target object size and category information, perform three-dimensional annotation on the target object in the simulated point cloud data of the target object model background. For example: The size of the ground is 100 meters long, 50 meters wide, and 0 meters high (assuming the ground is flat). The size of the gravel pile is 30 meters long, 0.5 meters wide, and 10 meters high. The category information is "ground" and "gravel pile" respectively. Then, based on the three-dimensional coordinates and directions obtained previously, mark the positions and directions of the ground and gravel piles in the simulated point cloud data of the mine, and form three-dimensional bounding boxes with semantic labels of corresponding sizes for subsequent analysis or training of machine learning models.

[0077] Among them, semantic segmentation is to use a deep learning model to classify the point cloud to identify different categories of objects (such as the ground, gravel piles, etc.). For each point cloud, the principal component analysis method is usually used to calculate its local normal vector, calculate the covariance matrix within the neighborhood of this point, and then solve the eigenvalues and eigenvectors. The eigenvector corresponding to the largest eigenvalue is the main direction. The three-dimensional bounding box is a common method for marking the position and direction of the target object in the point cloud data. The bounding box is usually a minimum bounding box that contains all the points of the target object and is aligned with the main direction of the target object. The target object size and category information are the pre-defined size (such as length, width, height) and category (such as ground, gravel pile, etc.) of the target object, which are used to accurately mark and identify the target object.

[0078] In this embodiment, through the automatic annotation method, the need for manual annotation can be greatly reduced, and the efficiency of obtaining simulated point cloud data is improved.

[0079] In an illustrated embodiment, adapting and fitting the pre-acquired target object model point cloud to the simulated point cloud data of the target object model background according to the terrain factors in the simulated point cloud data of the target object model background includes: determining the terrain factors in the simulated point cloud data of the target object model background according to the three-dimensionally labeled target object; fitting the target object model point cloud to the simulated point cloud data of the target object model background through a collision detection algorithm and the terrain factors in the simulated point cloud data of the target object model background; determining the partial point cloud of the fitted target object model point cloud that is blocked by the existing obstacles in the simulated point cloud data of the target object model background, and deleting the partial point cloud.

[0080] For example, the target object model background is a mine, and the simulated point cloud data of the mine contains the point cloud data of target objects such as the ground and the gravel pile. The target object model is a vehicle, and the processor needs to adapt and fit the pre-acquired vehicle point cloud data to the simulated point cloud data of the mine background. According to the three-dimensionally labeled target objects (i.e., the ground and the gravel pile), analyze the terrain factors in the simulated point cloud data of the mine background, such as the ground height, the position and height of the gravel pile, etc. Assume that the ground height is 0 meters, the gravel pile is located at the coordinates (60, 25), the height is 10 meters, the width is 0.5 meters, and the length is 30 meters. The processor uses a collision detection algorithm to ensure that the placement position of the vehicle does not conflict with other objects. For example, the size of the vehicle is 5 meters in length, 2 meters in width, and 3 meters in height. Assume that the initial position of the vehicle is (50, 20), and calculate whether this position overlaps or interferes with the ground or the gravel pile through the collision detection algorithm. If there is no conflict, confirm this position; if there is a conflict, adjust the position of the vehicle. After confirming the placement position of the vehicle, if a part of the vehicle is blocked by the gravel pile, delete these blocked partial point clouds to ensure that the final simulated point cloud data looks natural and has no overlap.

[0081] Among them, collision detection is a technology used to confirm whether the placement position of the target object model conflicts with other objects. Commonly used collision detection methods include geometric shape-based detection and distance field-based detection. Adapting and fitting refers to the process of naturally integrating the target object model point cloud into the background point cloud. The blocked partial point clouds can be deleted using ray tracing technology. This technology emits rays from each point of the target object and determines whether these rays are blocked by the objects included in the background point cloud. If the ray is blocked, it is considered that the point cloud is blocked, and then this point cloud needs to be deleted.

[0082] In an illustrated embodiment, determining the terrain factors in the simulated point cloud data of the background of the target object model according to the three-dimensional labeled target object includes: projecting the three-dimensional labeled target object onto a two-dimensional plane to obtain the two-dimensional labeled target object; counting the number of original point clouds in the area where the two-dimensional labeled target object is located to determine the terrain factors in the simulated point cloud data of the background of the target object model and the position where the point cloud of the target object model is to be fitted.

[0083] For example, the background of the target object model is a mine, and the simulated point cloud data of the mine contains the point cloud data of the ground and gravel piles, which are the target objects. The target object model is a vehicle with dimensions of 5 meters in length, 2 meters in width, and 3 meters in height. The processor needs to adapt and fit the pre-acquired vehicle point cloud data to the simulated point cloud data of the mine background. The processor can project the three-dimensional labeled target objects (i.e., the ground and gravel piles) onto the two-dimensional plane of the top view. This can simplify the calculation and make it easier to count the number of point clouds in this two-dimensional area. For example, the number of ground point clouds is 100, and the number of point clouds in the gravel pile part is 20. After obtaining the specific number of point clouds, it can be evaluated together with the predefined dimensions of the target object (the predefined dimensions of the target object may not be completely accurate), and then the terrain features, such as the ground height and obstacle distribution, and the position where the vehicle is to be fitted, such as several positions where the vehicle does not collide with the gravel pile and has reasonable contact with the ground, can be determined more accurately.

[0084] Among them, in the process of mapping the target object in three-dimensional space onto a two-dimensional plane, the top view (XY plane) is usually selected for projection to simplify the calculation. The top view projection projects the top view of the target object onto the XY plane by ignoring the Z-axis coordinate. For example, the three-dimensional coordinates (X, Y, Z) of the target object become (X, Y) after projection. The number of point clouds in the area reflects the terrain complexity of this area. For example, in a flat area such as the ground, the reflection signals of the laser beams are more concentrated, and the point cloud distribution is uniform and the density is high. While the terrain of the gravel pile area is complex, which may cause the reflection signals of the laser beams to be scattered, manifested as a lower or uneven point cloud density. However, a lower point cloud density does not necessarily directly indicate the existence of obstacles. It may also be due to the laser beam being blocked or the reflection signal being weak, and further analysis needs to be combined with the distribution pattern of the point clouds (such as whether an obvious obstacle contour is formed). The position to be fitted is to determine the appropriate placement position of the target object model according to the terrain factors and the collision detection results, which can ensure the seamless fusion of the target object model with the background environment and avoid overlap or interference. For example, if it is determined that the simulated point cloud data of the mine contains a retaining wall on a steep slope, the vehicle physically cannot enter this environment, so it is not suitable to adapt and fit the pre-acquired vehicle point cloud data to the simulated point cloud data of the mine background, and the position to be fitted is not selected in this frame of the simulated point cloud data of the mine background.

[0085] In one of the illustrated embodiments, fitting the point cloud of the target object model to the simulated point cloud data of the background of the target object model by means of the collision detection algorithm and the terrain factors in the simulated point cloud data of the background of the target object model includes: if the target object with two-dimensional annotation includes a road surface, moving the lowest point of the point cloud of the target object model to the road surface; performing collision detection on the position where the point cloud of the target object model is to be fitted and the annotated target object; if there is no collision between the position where the point cloud of the target object model is to be fitted and the annotated target object, fitting the point cloud of the target object model to the simulated point cloud data of the background of the target object model according to the to-be-fitted position.

[0086] For example, the background of the target object model is a mine, and the simulated point cloud data of the mine contains the point cloud data of target objects such as the ground and the gravel pile. The target object model is a vehicle with dimensions of 5 meters in length, 2 meters in width, and 3 meters in height, and the initial position is set to (50, 20, Z), where Z represents the height coordinate. The processor needs to adapt and fit the pre-acquired vehicle point cloud data to the simulated point cloud data of the mine background. Assuming that the height of the ground is 0 meters, it is necessary to adjust the lowest point of the vehicle point cloud to the position where Z = 0 meters so that it just touches the ground. Then, the processor uses the collision detection algorithm to check whether the vehicle collides with the gravel pile or other obstacles at the new position (50, 20, 0). If at the new position (50, 20, 0), the vehicle is closest to the obstacle gravel pile but does not collide with it, then the point cloud data of the vehicle can be safely placed at this position and can be fitted to the simulated point cloud data of the mine background.

[0087] Among them, in order to make the simulation environment as close to the actual situation as possible, any object placed on the ground should be accurately located on the ground. For example, the vehicle should not float in the air or be partially embedded in the ground. Correct height adjustment helps to more accurately simulate the physical behavior in reality, thereby providing reliable data support for subsequent analysis and applications.

[0088] It should be noted that after obtaining the simulated point cloud data including the target object model and the background of the target object model by fitting, data augmentation operations such as random scaling, random rotation, and adding noise can be performed on it to further expand the data set.

[0089] To help those skilled in the art better understand the above embodiments, the following is combined with Figures 2 to 4 to illustrate the above embodiments.

[0090] Please refer to Figure 2 , Figure 2 which is a schematic diagram of another method for generating simulated point cloud data shown in an exemplary embodiment. AsFigure 2 As shown in the figure, first, multiple images (Image 1, Image 2, …, Image n) of the mine scene are taken from different angles. Then, based on these multi-view images, 3D geometric structure reconstruction is performed to obtain a 3D point cloud model of the background of the target object model. Through the optical flow tracking method, according to the scanning mechanism of the lidar, the reflection process of laser beams emitted by the lidar in different directions and with different numbers of scan lines on the surface of the 3D point cloud model of the background of the target object model is simulated, so as to obtain the simulated point cloud data of the background of the target object model. To increase authenticity, Gaussian noise may be added to the measured distance of the laser beam in this process, and the multi-path reflection of the laser beam on the surface of the 3D point cloud model of the background of the target object model is simulated, so as to obtain additional noise points and enrich the data. Next, the point cloud data of the target object in the simulated point cloud data of the background of the target object model is extracted, so as to calculate the main direction of the target object according to the normal vector distribution of the point cloud data of the target object, and calculate the 3D coordinates of the target object according to the geometric center or centroid of the point cloud data of the target object; based on the 3D coordinates and main direction of the target object, as well as the predefined size and category information of the target object, 3D annotation is performed on the target object in the simulated point cloud data of the background of the target object model to obtain the simulated point cloud data of the background of the target object model after annotation. Finally, through collision detection, the pre-acquired point cloud of the target object model is adapted and fitted to the simulated point cloud data of the background of the target object model, so as to obtain the simulated point cloud data of the fitted target object model and the background of the target object model.

[0091] Please refer to Figure 3 , Figure 3 Figure is a schematic diagram of the simulated point cloud data of the fitted target object model and the background of the target object model shown in an exemplary embodiment. As Figure 3 shown, the vehicle is located at an appropriate position in the mine environment, the vehicle is close to the ground and will not collide with surrounding obstacles.

[0092] Next, in combination with Figure 4 , taking the collection of the point cloud containing the vehicle in the mine environment as an example, the generation process of the simulated point cloud data in the technical solution of the present disclosure is introduced.

[0093] Please refer to Figure 4 , Figure 4 Figure is a schematic diagram of another method for generating simulated point cloud data shown in an exemplary embodiment. As Figure 4As shown in the figure, the process first obtains the labeled mine point cloud and the vehicle model point cloud sample. Among them, the vehicle model point cloud sample can be obtained in advance, and the labeled mine point cloud is obtained in the following way: After performing three-dimensional geometric structure reconstruction on multi-view images of the mine to generate a three-dimensional point cloud model, simulate the reflection process of laser beams with different directions and densities emitted by the lidar on the surface of the above three-dimensional point cloud model, so as to obtain the simulated point cloud data of the background of the target object model. Then, based on the three-dimensional coordinates, main direction, and the predefined size and category information of the target object in the mine point cloud, the target object is three-dimensionally labeled to obtain the labeled mine point cloud. Then, convert the three-dimensional detection box of the labeled mine point cloud into a two-dimensional detection box to facilitate counting the number of point clouds in the detection box and judging the situation of the target object in the detection box. If there are earth wall points in the target object in the detection box, it means that the environment is steep and the vehicle cannot physically enter this environment, so it is not suitable to fit the pre-obtained vehicle model point cloud sample to the labeled mine point cloud data and no fitting is performed. If the target object in the detection box collides with the pre-obtained vehicle model point cloud sample, no fitting is performed either. Only when there is no collision can fitting be carried out. If there are ground points in the target object point cloud in the detection box, fit the lowest point of the vehicle model point cloud to the road surface to make it conform to the physical reality. If there are no ground points in the target object point cloud in the detection box, the position of the vehicle model point cloud remains unchanged. Finally, obtain the vehicle model point cloud sample that can be fitted and its corresponding three-dimensional detection box, fit the vehicle model point cloud sample to the labeled mine point cloud, and then determine whether some point clouds of the fitted vehicle model point cloud sample are blocked by existing obstacles in the background. If there is occlusion, delete these occluded partial point clouds to ensure that the final simulated point cloud data looks natural and has no overlap. If there is no occlusion, retain all the point cloud data after fitting.

[0094] Corresponding to the embodiment of the above simulation point cloud data generation method, the present disclosure also provides an embodiment of a simulation point cloud data generation device.

[0095] Please refer to Figure 5 , Figure 5 is a hardware structure diagram of an electronic device shown in an exemplary embodiment. At the hardware level, the device includes a processor 502, an internal bus 504, a network interface 506, a memory 508, and a non-volatile memory 510. Of course, other required hardware may also be included. One or more embodiments of the present disclosure can be implemented in a software manner. For example, the processor 502 reads the corresponding computer program from the non-volatile memory 510 into the memory 508 and then runs it. Of course, in addition to the software implementation manner, one or more embodiments of the present disclosure do not exclude other implementation manners, such as logical devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logical unit, and can also be hardware or logical devices.

[0096] Please refer to Figure 6 , Figure 6 which is a block diagram of an exemplary simulation point cloud data generation device shown in an exemplary embodiment. The simulation point cloud data generation device 600 can be applied to an electronic device as shown in Figure 5 to implement the technical solution of the present disclosure.

[0097] The device includes:

[0098] A reconstruction unit 602, configured to collect multi-view images of the background of a target object model, and perform three-dimensional geometric structure reconstruction based on the multi-view images to obtain a three-dimensional point cloud model of the background of the target object model;

[0099] A simulation unit 604, configured to simulate the reflection process of laser beams emitted in different directions and densities on the surface of the three-dimensional point cloud model of the background of the target object model based on the scanning mechanism of a lidar, so as to obtain simulation point cloud data of the background of the target object model;

[0100] A fitting unit 606, configured to adapt and fit a pre-acquired point cloud of a target object model to the simulation point cloud data of the background of the target object model according to the terrain factors in the simulation point cloud data of the background of the target object model, so as to obtain simulation point cloud data including the target object model and the background of the target object model.

[0101] In some embodiments, the simulation unit includes:

[0102] A first simulation unit, configured to simulate and generate laser beams in different directions and with different numbers of scan lines based on the scanning mechanism of the lidar;

[0103] A calculation unit, configured to calculate the intersection points of each generated laser beam with the surface of the three-dimensional point cloud model of the background of the target object model by using a ray-triangle intersection algorithm;

[0104] A second simulation unit, configured to calculate the reflection direction of each generated laser beam according to the law of reflection, so as to simulate the lidar receiving the reflected signal, and use the received signal intensity and the intersection point of each laser beam as the simulation point cloud data of the background of the target object model.

[0105] In some embodiments, the first simulation unit includes:

[0106] A first simulation subunit, configured to simulate and generate laser beams in different directions and with different numbers of scan lines based on the scanning mechanism of the lidar, and add Gaussian noise to the measurement distances of the laser beams;

[0107] A second simulation subunit, configured to simulate multi-path reflections of the laser beam on the surface of the three-dimensional point cloud model of the background of the target object model, so as to obtain additional noise points.

[0108] In some embodiments, the apparatus further includes:

[0109] An extraction unit, configured to extract the point cloud data of the target object from the simulated point cloud data of the background of the target object model, so as to calculate the main direction of the target object according to the normal vector distribution of the point cloud data of the target object; and calculate the three-dimensional coordinates of the target object according to the geometric center or centroid of the point cloud data of the target object.

[0110] A labeling unit, configured to perform three-dimensional labeling on the target object in the simulated point cloud data of the background of the target object model according to the three-dimensional coordinates and main direction of the target object, as well as the predefined size and category information of the target object.

[0111] In some embodiments, the fitting unit includes:

[0112] A determination unit, configured to determine the terrain factors in the simulated point cloud data of the background of the target object model according to the three-dimensionally labeled target object.

[0113] A first fitting unit, configured to fit the point cloud of the target object model to the simulated point cloud data of the background of the target object model through a collision detection algorithm and the terrain factors in the simulated point cloud data of the background of the target object model.

[0114] A deletion unit, configured to determine the partial point cloud of the point cloud of the fitted target object model that is blocked by the existing obstacles in the simulated point cloud data of the background of the target object model, and delete the partial point cloud.

[0115] In some embodiments, the determination unit includes:

[0116] A projection unit, configured to project the three-dimensionally labeled target object onto a two-dimensional plane to obtain a two-dimensionally labeled target object.

[0117] A statistics unit, configured to count the number of original point clouds in the area where the two-dimensionally labeled target object is located, so as to determine the terrain factors in the simulated point cloud data of the background of the target object model and the position where the point cloud of the target object model is to be fitted.

[0118] In some embodiments, the first fitting unit includes:

[0119] A moving unit, configured to move the lowest point of the point cloud of the target object model to the road surface if the two-dimensionally labeled target object includes a road surface.

[0120] A detection unit for performing a collision detection on the position where the target object model point cloud is intended to be fitted and the labeled target object;

[0121] A fitting subunit for, if there is no collision between the position where the target object model point cloud is intended to be fitted and the labeled target object, fitting the target object model point cloud to the simulated point cloud data of the target object model background according to the intended fitting position.

[0122] For the implementation processes of the functions and roles of each unit in the above device, refer to the implementation processes of the corresponding steps in the above method for details, which will not be elaborated here.

[0123] For the device embodiments, since they basically correspond to the method embodiments, refer to the partial descriptions of the method embodiments for the relevant parts. The device embodiments described above are only illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present disclosure solution. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0124] The system, device, module or unit illustrated in the above embodiments can be specifically implemented by a computer chip or entity, or by a product with certain functions. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver device, a game console, a tablet computer, a wearable device, or a combination of any several of these devices.

[0125] In a typical configuration, a computer includes one or more processors (CPUs), an input / output interface, a network interface, and a memory.

[0126] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM), and / or non-volatile memory in forms such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

[0127] A computer-readable medium includes both permanent and non-permanent, removable and non-removable media and can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transitory media that can be used to store information accessible by a computing device. As defined herein, a computer-readable medium does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0128] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data that have been authorized by the user or fully authorized by all parties. Moreover, the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or reject.

[0129] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, commodity or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, method, commodity or device comprising the said element.

[0130] The specific embodiments of the present disclosure have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0131] The terms used in one or more embodiments of the present disclosure are for the purpose of describing particular embodiments only and are not intended to limit one or more embodiments of the present disclosure. The singular forms "a", "the", and "said" used in one or more embodiments of the present disclosure and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0132] It should be understood that although the terms first, second, third, etc. may be used in one or more embodiments of the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0133] The above description is only a preferred embodiment of one or more embodiments of the present disclosure and is not intended to limit one or more embodiments of the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of one or more embodiments of the present disclosure shall be included within the scope of protection of one or more embodiments of the present disclosure.

Claims

1. A method for generating simulated point cloud data, characterized in that, The method includes: Collecting multi-view images of the background of the target object model to perform three-dimensional geometric structure reconstruction based on the multi-view images, and obtaining a three-dimensional point cloud model of the background of the target object model; Based on the scanning mechanism of the lidar, simulating the reflection process of laser beams with different directions and numbers of scan lines on the surface of the three-dimensional point cloud model of the background of the target object model, so as to obtain the simulated point cloud data of the background of the target object model; According to the terrain factors in the simulated point cloud data of the background of the target object model, fitting the pre-obtained point cloud of the target object model to the simulated point cloud data of the background of the target object model, so as to obtain the simulated point cloud data including the target object model and the background of the target object model.

2. The method according to claim 1, characterized in that, The step of, based on the scanning mechanism of the lidar, simulating the reflection process of laser beams with different directions and numbers of scan lines on the surface of the three-dimensional point cloud model of the background of the target object model, so as to obtain the simulated point cloud data of the background of the target object model, includes: Based on the scanning mechanism of the lidar, simulating and generating laser beams with different directions and numbers of scan lines; Using the ray-triangle intersection algorithm to calculate the intersection points of each generated laser beam with the surface of the three-dimensional point cloud model of the background of the target object model; Calculating the reflection direction of each generated laser beam according to the law of reflection to simulate the lidar receiving the reflected signal, and taking the received signal intensity and the intersection point of each laser beam as the simulated point cloud data of the background of the target object model.

3. The method according to claim 2, characterized in that, The step of, based on the scanning mechanism of the lidar, simulating and generating laser beams with different directions and numbers of scan lines, includes: Based on the scanning mechanism of the lidar, simulating and generating laser beams with different directions and numbers of scan lines, and adding Gaussian noise to the measurement distances of the laser beams; Simulating the multi-path reflection of the laser beams on the surface of the three-dimensional point cloud model of the background of the target object model to obtain additional noise points.

4. The method according to claim 1, characterized in that, After obtaining the simulated point cloud data of the background of the target object model, the method further includes: Extracting the point cloud data of the target object in the simulated point cloud data of the background of the target object model, calculating the main direction of the target object according to the normal vector distribution of the point cloud data of the target object; and calculating the three-dimensional coordinates of the target object according to the geometric center or centroid of the point cloud data of the target object; Performing three-dimensional annotation on the target object in the simulated point cloud data of the background of the target object model according to the three-dimensional coordinates and main direction of the target object, as well as the pre-defined size and category information of the target object.

5. The method according to claim 4, wherein The step of, according to the terrain factors in the simulated point cloud data of the background of the target object model, fitting the pre-obtained point cloud of the target object model to the simulated point cloud data of the background of the target object model, includes: Determining the terrain factors in the simulated point cloud data of the background of the target object model according to the three-dimensionally annotated target object; Fitting the point cloud of the target object model to the simulated point cloud data of the background of the target object model through a collision detection algorithm and the terrain factors in the simulated point cloud data of the background of the target object model; Determine the partial point cloud of the target object model point cloud after fitting that is blocked by the existing obstacles in the simulated point cloud data of the target object model background, and delete the partial point cloud.

6. The method according to claim 5, wherein The terrain factors in the simulated point cloud data of the target object model background determined according to the three-dimensionally annotated target object include: Project the three-dimensionally annotated target object onto a two-dimensional plane to obtain the two-dimensionally annotated target object; Count the number of original point clouds in the area where the two-dimensionally annotated target object is located to determine the terrain factors in the simulated point cloud data of the target object model background and the position where the target object model point cloud is to be fitted.

7. The method according to claim 6, characterized in that The method of fitting the target object model point cloud to the simulated point cloud data of the target object model background through the collision detection algorithm and the terrain factors in the simulated point cloud data of the target object model background includes: If the two-dimensionally annotated target object includes a road surface, move the lowest point of the target object model point cloud to the road surface; Perform collision detection on the position where the target object model point cloud is to be fitted and the annotated target object; If there is no collision between the position where the target object model point cloud is to be fitted and the annotated target object, fit the target object model point cloud to the simulated point cloud data of the target object model background according to the position to be fitted.

8. A simulation point cloud data generation device, characterized in that, The device includes: A reconstruction unit, configured to collect multi-view images of the target object model background, and perform three-dimensional geometric structure reconstruction according to the multi-view images to obtain a three-dimensional point cloud model of the target object model background; A simulation unit, configured to simulate the reflection process of laser beams emitted in different directions and densities on the surface of the three-dimensional point cloud model of the target object model background based on the scanning mechanism of a lidar, so as to obtain the simulated point cloud data of the target object model background; A fitting unit, configured to adapt and fit the pre-acquired target object model point cloud to the simulated point cloud data of the target object model background according to the terrain factors in the simulated point cloud data of the target object model background, so as to obtain the simulated point cloud data including the target object model and the target object model background.

9. An electronic device, characterized in that, It includes a communication interface, a processor, a memory, and a bus, and the communication interface, the processor, and the memory are interconnected through the bus; Machine-readable instructions are stored in the memory, and the processor executes the method according to any one of claims 1 to 7 by calling the machine-readable instructions.

10. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-readable instructions, and when the machine-readable instructions are called and executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images