Point cloud scanning simulation method for deep learning training

Abstract

The invention discloses a point cloud scanning simulation method for deep learning training, and the method comprises the following steps: 1, importing a to-be-scanned 3D model, setting a virtual scanner and scanning parameters, and constructing a point cloud scanning simulation model; 2, virtual light is calculated according to the point cloud scanning simulation model constructed in the step 1; 3, the virtual light and the 3D model to be scanned intersect, a point cloud attribute true value is obtained, and a scanning result point cloud is obtained; 4, noise is added to the scanning result point cloud according to the scanning parameters set in the step 1; 5, updating the normal direction of the point cloud added with the noise; and step 6, early warning of problematic noise points in the noise point cloud is carried out, and point cloud scanning simulation for deep learning training is completed. The method can significantly reduce the data collection cost, improves the robustness and precision of deep learning model training, and is suitable for the fields of automatic driving, robot perception and the like.

Description

Technical Field

[0001] The present invention relates to a point cloud simulation method, in particular to a point cloud scanning simulation method for deep learning training. Background Art

[0002] This section merely provides background information related to the present disclosure and is not necessarily prior art.

[0003] Point cloud data, an important representation of the surface information of objects in three-dimensional space, is widely used in fields such as autonomous driving, robotic perception, and architectural modeling. However, obtaining high-quality point cloud data typically relies on expensive LiDAR equipment or other high-precision scanning instruments. This is not only costly but also limited by the acquisition environment and equipment performance, making it difficult to meet the diverse point cloud data requirements of large-scale deep learning model training.

[0004] Currently, researchers are trying to synthesize point cloud data through point cloud simulation to make up for the lack of real data. However, existing simulation methods are insufficient in the realism, diversity and robustness of point cloud distribution, and it is difficult to accurately simulate key characteristics in real scans, such as noise, occlusion, distance and resolution limitations.

[0005] Existing point cloud simulation methods have significant shortcomings in reproducing the characteristics of real scans, making it difficult to generate data with realistic physical properties and diversity. These issues significantly limit the potential of point cloud simulation technology for deep learning applications.

[0006] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the Invention

[0007] Purpose of the invention: The technical problem to be solved by the present invention is to provide a point cloud scanning simulation method for deep learning training in response to the shortcomings of the existing technology.

[0008] In order to solve the above technical problems, the present invention discloses a point cloud scanning simulation method for deep learning training, comprising the following steps:

[0009] Step 1: Import the 3D model to be scanned, set the virtual scanner and scanning parameters, and build a point cloud scanning simulation model;

[0010] Step 2, calculating virtual light according to the point cloud scanning simulation model constructed in step 1;

[0011] Step 3: Intersect the virtual ray with the 3D model to be scanned to obtain the true value of the point cloud attribute and obtain the scan result point cloud;

[0012] Step 4: Add noise to the scan result point cloud according to the scanning parameters set in step 1;

[0013] Step 5, update the normal direction of the point cloud after adding noise;

[0014] Step 6: Warning of problematic noise points in the noise point cloud, and completing the point cloud scanning simulation for deep learning training.

[0015] Furthermore, the construction of the point cloud scanning simulation model described in step 1 includes:

[0016] Step 1-1, set up a virtual scanner around the imported 3D model to be scanned;

[0017] Step 1-2: Calculate the extrinsic parameter matrix and intrinsic parameter matrix of the virtual scanner according to the set scanning parameters.

[0018] Furthermore, the setting of scanning parameters described in step 1 includes:

[0019] The virtual scanner's focal point center, the virtual scanner's position eye, the virtual scanner's upward direction vector up, and the virtual scanner's scanning field of view FOV in the x and y directions x and FOV y (Or directly give the focal length f of the virtual scanner in the x and y directions x and f y ), the scanning angle resolution of the virtual scanner in the x and y directions res x and res y .

[0020] Furthermore, the calculation of the extrinsic parameter matrix and the intrinsic parameter matrix of the virtual scanner described in steps 1-2 includes:

[0021] Step 1-2-1, calculate the extrinsic parameter matrix T of the virtual scanner cam→world , as follows:

[0022]

[0023] Where R represents the virtual scanner rotation matrix, which is calculated as follows:

[0024] R=(right new_up fwd)

[0025] Among them, fwd and right represent the forward and right direction vectors of the virtual scanner respectively, and new_up represents the upward direction vector of the virtual scanner after update. The specific calculation methods are as follows:

[0026]

[0027] new_up=right×fwd

[0028] Among them, ‖‖ represents the modulus of the vector, and × represents the vector cross product;

[0029] Step 1-2-2, calculate the intrinsic parameter matrix K of the virtual scanner, as follows:

[0030]

[0031] Among them, f x and f y Represents the focal length of the virtual scanner in the x and y directions, c x and c y They represent the center point coordinates of the image scanned by the virtual scanner, and the specific calculation methods are as follows:

[0032]

[0033] in Indicates rounding down.

[0034] Furthermore, the calculation of virtual light in step 2 includes:

[0035] According to the virtual scanner orientation information, the camera basis vector of the virtual scanner is calculated. According to the virtual scanner viewing angle range, the x and y directions are divided equally according to the angular resolution, and combined according to the orientation information to obtain the direction ray of each virtual light of the virtual scanner. ij , as follows:

[0036] ray ij =R 2ij ·R 1ij ·(-1)·fwd

[0037] Among them, ray ij represents the virtual ray at row i and column j, R 2ij is the direction angle rotation transformation matrix of the forward direction vector fwd, R 1ij is the pitch angle rotation transformation matrix of the forward direction vector fwd, which is calculated as follows:

[0038]

[0039] R 2ij =Rot(θ j ,new_up)

[0040] Among them, Rot means the calculation is done using the Rodrigues formula. Indicates the angle that the virtual light of row i needs to rotate from the forward direction vector fwd around the right direction vector right, θ jIndicates the angle that the j-th virtual ray needs to rotate from the forward direction vector fwd around the new upward direction vector new_up. The specific calculation method is as follows:

[0041]

[0042] If FOV x and FOV y If not given, it is calculated as follows:

[0043]

[0044]

[0045] Furthermore, the intersection of the virtual light in step 3 and the 3D model to be scanned includes:

[0046] Step 3-1: Eliminate invalid virtual rays, as follows:

[0047] Using the virtual scanner, obtaining a depth map of the virtual scanner;

[0048] Find the depth value in the depth map is -1, that is, the area where there is no object in the scan scene, and cast the virtual ray ij Polar coordinates Convert to depth map pixel coordinates After filtering out invalid virtual rays, the coordinate transformation of a single ray is expressed as follows:

[0049]

[0050]

[0051] If the virtual ray ij Corresponding depth map pixel coordinates In the area where there is no object in the above-mentioned scanning scene, the virtual ray is deleted if and only if the following conditions are met at the same time:

[0052]

[0053] Among them, depth(i,j) represents the depth value of row i and column j. Indicates rounding up. Indicates rounding down;

[0054] Step 3-2, finding the intersection points of the 3D model to be scanned and the remaining virtual rays after deleting the invalid virtual rays, and obtaining the Cartesian coordinates of all intersection points and the triangle facet numbers of the 3D model where the intersection points are located;

[0055] Step 3-3, update the point cloud information and obtain the scanning result point cloud, as follows:

[0056] According to the Cartesian coordinates of the intersection point, a scanning result point cloud is generated, and according to the triangle facet number, the properties of the triangle facet are obtained from the original file of the 3D model to be scanned, and the properties of the scanning result point cloud are set accordingly.

[0057] Furthermore, the point cloud of the scanning result described in step 4 is added with noise, including:

[0058] Step 4: Add noise to the point cloud based on the scanning parameters, including: polar coordinate conversion, adding Gaussian noise, and Cartesian coordinate conversion:

[0059] Step 4-1: For each point in the scan result point cloud, calculate the polar coordinates in the virtual scanner coordinate system based on the coordinates and depth r1, as follows:

[0060]

[0061] θ1=tan -1 (Δy,Δx)

[0062]

[0063] Where Δx, Δy, and Δz are the offsets in the x, y, and z coordinate axes, respectively, and are expressed as follows:

[0064] Δx=x1-eye x

[0065] Δy=y1-eye y

[0066] Δz=z1-eye z

[0067] Among them, eye x 、eye y and eye z is the component of the virtual scanner's position eye on the x, y, and z axes, and x1, y1, and z1 represent the Cartesian coordinates of the point on the scan result point cloud;

[0068] Step 4-2, add Gaussian noise to get the updated polar coordinates and depth r2, expressed as follows:

[0069]

[0070] Among them, t r , t θ and is the preset Gaussian noise amplitude, Represents Gaussian noise with mean 0 and variance 1;

[0071] Step 4-3, update the polar coordinates The depth r2 is restored to the Cartesian coordinate system to obtain the point coordinates x2, y2 and z2 after adding noise, which are expressed as follows:

[0072]

[0073] Furthermore, the normal direction of the point cloud after adding noise as described in step 5 is updated, including:

[0074] Step 5-1, sampling s points on the 3D model to be scanned as high-density sampling points;

[0075] Step 5-2, use the k-NN clustering method to calculate the high-density sampling points closest to each point on the point cloud after adding noise;

[0076] In step 5-3, the normal direction of the patch of the 3D model where the high-density sampling point is located is used as the normal direction of the point on the point cloud with added noise.

[0077] Furthermore, the problematic noise points in the warning noise point cloud described in step 6 include:

[0078] For each point in the point cloud after adding noise, calculate the cosine value α of the angle between the updated normal direction and the virtual light direction. If the cosine value α is greater than the warning value α0 set by the user, an alert is issued for the point.

[0079] Beneficial effects:

[0080] 1. This invention uses polar coordinates to add noise, making the noise point cloud distribution more consistent with the real distribution

[0081] 2. The present invention adopts a simulation strategy of intersection-noise addition-update of true value, which can generate noise point clouds of various noise levels in batches.

[0082] 3. The present invention provides a method for synchronizing the true value of the normal direction to improve the accuracy of the normal direction of the noisy point cloud. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more apparent.

[0084] Figure 1 It is a schematic diagram of the overall process of the present invention.

[0085] Figure 2 FIG. 1 is a schematic diagram of the final point cloud scanning simulation result in an embodiment. DETAILED DESCRIPTION

[0086] This paper proposes a point cloud scanning simulation method for deep learning training. This method can flexibly simulate the various physical characteristics of real-world laser radar scans, generate diverse, high-quality simulated point cloud data, and provide accurate attribute information such as point cloud normals. This significantly reduces data acquisition costs and improves the robustness and accuracy of deep learning models. The laser point cloud scanning simulation method for deep learning training described in this paper includes the following steps:

[0087] Step 1: Build a point cloud scanning simulation model: Import the 3D model to be scanned, set the position and orientation information of the 3D model and the virtual scanner, set the virtual scanner's angular resolution, focal length, noise and other scanning parameters, and build a virtual scanning scene.

[0088] Step 2: Calculation of virtual scanner parameters and virtual light: Through the virtual scanner simulation module, the virtual scanner scanning internal and external parameters and the ray direction of the emitted virtual light are calculated according to the orientation information and observation information.

[0089] Step 3: Virtual ray intersection and point cloud attribute true value calculation: The intersection of the virtual ray and the 3D model is calculated in parallel, and the true value of the normal direction is obtained based on the facet information where the intersection point is located.

[0090] Step 4: Noise simulation: Based on the virtual rays and intersection points from steps 2 and 3, a noise-free point cloud in the polar coordinate system is obtained. Gaussian noise of specified scales is added in the two polar coordinate directions and depth dimensions to obtain a noisy point cloud.

[0091] Step 5: Update the true normal value: Update the true normal value of the noise point cloud according to the position of each point in the 3D model.

[0092] Step 6: Problem noise point warning: Based on the angle between the virtual light direction of the noise point cloud and the synchronized true value direction, a warning is issued for points with an angle less than 90°.

[0093] Example 1:

[0094] like Figure 1 As shown, a point cloud scanning simulation method for deep learning training specifically includes the following steps:

[0095] Step 1: Import the 3D model to be scanned, set the scanning parameters, and build a point cloud scanning simulation model; specifically, import the model to be scanned and the scanning parameters center, eye, up, FOV x FOV y 、res x 、res y (FOV x FOV yCan be replaced by the focal length f of the virtual scanner in the x and y directions x and f y ) and then calculate the virtual scanner internal and external parameter matrices K and T according to the scanning parameters. cam→world , set up virtual scanners. Layout and preview the visual range and imaging effects of each virtual scanner. Users fine-tune the simulation model settings based on the preview results. After repeated previews and adjustments, the simulation model configuration for scanning is determined.

[0096] Among them, center and eye are composed of three unranged real numbers, representing the focus point and position of the virtual scanner respectively; the upward direction vector up of the virtual scanner is composed of three real numbers dx, dy, and dz, satisfying dx 2 +dy 2 +dz 2 =1; FOV x and FOV y Represents the scanning field of view of the virtual scanner in the x and y directions, ranging from 0° to 180°; res x and res y Indicates the scanning angle resolution of the virtual scanner in the x and y directions, and its value range is a positive integer. The virtual scanner external parameter matrix T cam→world The calculation method is as follows:

[0097]

[0098] new_up=right×fwd

[0099] R=(right new_up fwd)

[0100]

[0101] Among them, fwd and right represent the forward and right directions of the virtual scanner respectively, new_up represents the updated upward direction vector of the virtual scanner, ‖‖ represents the vector modulus, × represents the vector cross product, and R represents the virtual scanner rotation matrix; the calculation method of the virtual scanner internal parameter matrix K is as follows:

[0102]

[0103] Among them, f x and f y Represents the focal length of the virtual scanner in the x and y directions, c x and c y They represent the center point coordinates of the image taken by the virtual scanner, Indicates rounding down.

[0104] Step 2: Calculate virtual scanner parameters and the direction of virtual light rays based on the scanning parameters of step 1; specifically,

[0105] According to the virtual scanner's orientation information, the camera basis vector of the virtual scanner is calculated. According to the virtual scanner's viewing angle range, the x and y directions are divided equally according to the angular resolution (the goal of this scheme is scanning simulation, because the working principle of the laser radar is to divide the angle equally, and this method fully simulates this feature during simulation). According to the orientation information, the direction ray of each virtual light of the virtual scanner is obtained. ij , satisfying the following formula:

[0106]

[0107] R 2ij =Rot(θ j ,new_up)

[0108] ray ij =R 2ij ·R 1ij ·(-1)·fwd

[0109] Among them, i and j represent the virtual ray row and column numbers, and their values ​​range from 0 to res. x -1 and 0 to res y -1, Rot is the Rodrigues formula (reference: https: / / zhuanlan.zhihu.com / p / 451579313), Indicates the angle that the virtual light of row i needs to rotate from the fwd vector around the right vector, θ j Indicates the angle that the jth column virtual ray needs to rotate from the fwd vector around the new_up vector. x and FOV y If not given, it is calculated as follows:

[0110]

[0111] Step 3: Intersect the virtual ray with the model to calculate the true value of the point cloud attribute; this mainly includes three parts: invalid virtual ray removal (to improve calculation efficiency and save time), virtual ray intersection, and point cloud information update:

[0112] Step 3.1: Use the virtual scanner set up in step 1 to perform high-precision rasterization (https: / / github.com / isl-org / Open3D / blob / main / cpp / open3d / visualization / rendering / Renderer.cpp#L94) to obtain the depth map of the virtual scanner. Use the flood fill algorithm (reference: https: / / blog.csdn.net / Eason_Y / article / details / 127782837) to find the area with a depth value of -1 (i.e., no objects in the scanned scene). ij θ j and Polar coordinates converted to depth map pixel coordinates After filtering out invalid virtual rays, the coordinate transformation of a single ray is expressed as follows:

[0113]

[0114] If ray ij Corresponding Within the range described by the above-mentioned object-free area of ​​the scanning scene, the virtual ray is deleted if and only if the following conditions are met at the same time:

[0115]

[0116] Among them, depth(i,j) represents the depth value of row i and column j. Indicates rounding up. Indicates rounding down.

[0117] Step 3.2: Use the mitsuba engine (reference: https: / / github.com / mitsuba-renderer / mitsuba3 / blob / main / include / mitsuba / render / scene.h#L76) to find the intersection point between the scanned 3D model and the remaining virtual rays after deleting the invalid virtual rays. Obtain the Cartesian coordinates of the intersection point and the triangle number (this information is included in the 3D model to be scanned).

[0118] Step 3.3: Generate a point cloud of the scanned result based on the Cartesian coordinates of the intersection point. Find the normal direction, material, semantic type, and other information corresponding to the triangle face from the model file to be scanned based on the triangle face number. Set the point cloud properties such as color, normal direction, and semantic type.

[0119] Step 4: Add noise to the point cloud based on the scanning parameters, including: polar coordinate conversion, adding Gaussian noise, and Cartesian coordinate conversion:

[0120] Step 4.1: For each point in the point cloud of step 3.3, calculate the polar coordinates θ1, and depth r1:

[0121] Δx=x1-eye x

[0122] Δy=y1-eye y

[0123] Δz=z1-eye z

[0124]

[0125] θ1=tan -1 (Δy,Δx)

[0126]

[0127] Among them, eye x 、eye y 、eye z is the component of the virtual scanner's spatial coordinate eye on the xyz axis, x1, y1, z1 represent the Cartesian coordinates of the point cloud, and Δx, Δy, Δz are the offsets in the x, y, and z coordinate axes.

[0128] Step 4.2: Add Gaussian noise to the polar coordinates and depth respectively to get the updated polar coordinate representation r2, θ2,

[0129]

[0130] where t r , t θ 、 The Gaussian noise amplitude set by the user, t r The value range is 10 -4 and 10 -2 Between θ and The value range is from 0° to 2°. represents Gaussian noise with mean 0 and variance 1.

[0131] Step 4.3: Restore the polar coordinates and depth from step 4.2 to the Cartesian coordinate system to obtain the point coordinates x2, y2, and z2 after adding noise:

[0132]

[0133] Step 5: Update the true value of the normal direction of the noise point cloud, including:

[0134] Use the Area-Weighted Sampling algorithm (reference: https: / / blog.csdn.net / weixin_45301653 / article / details / 143407035) to sample s points on the model to be scanned, where the value of s is in the range of 10 4 and 10 6 Use the k-NN algorithm (reference: https: / / www.runoob.com / ml / ml-knn.html) to calculate the nearest high-density sampling point to each point in the point cloud after adding noise, and use the normal direction of the patch where the high-density sampling point is located as the true value of the normal direction of the point where the noise is added;

[0135] Step 6: Warning of problematic noise points in the noise point cloud, including:

[0136] For each point in the noise point cloud, the cosine value α of the angle between the updated normal direction and the virtual light direction is calculated. If the cosine value is greater than the warning value α0 set by the user, where the warning value range is between -0.5 and 0.5, the point cloud may appear in an area that the original scanner cannot scan. The user is warned to filter or mark these points to improve the accuracy of the noise point cloud.

[0137] This step improves the accuracy of virtual simulation scanning and can avoid the problem of incorrect normal orientation of the point cloud generated when scanning 3D objects that are not watertight.

[0138] Example 2:

[0139] Taking the typical indoor scene dataset ScanNet++ as an example, assuming that the present invention is needed to perform point cloud scanning simulation on the Scene_002 indoor scene in the dataset for deep learning training, then:

[0140] Step 1: Import the 3D model of Scene_002 and set the scanning parameters center, eye, up, and FOV according to the deep learning training requirements. x FOV y 、res x 、res y In this embodiment, center can be set to the geometric center of Scene_002, and eye, as the virtual scanner, is positioned at a certain distance from center (preferably outside the model of Scene_002) to ensure that a sufficiently large area of ​​Scene_002 is scanned. x 、res ySet the visible range to the fully contained model of Scene_002; calculate the virtual scanner internal and external parameter matrices K and T based on the scanning parameters cam→world , set up virtual scanners. Layout and preview the visual range and imaging effects of each virtual scanner, fine-tune the simulation model settings based on the preview results, and finally determine the simulation model configuration for scanning after repeated previews and adjustments.

[0141] Step 2: Calculate virtual scanner parameters and the direction of virtual light rays based on the scanning parameters of step 1; specifically,

[0142] According to the virtual scanner's orientation information, the camera basis vector of the virtual scanner is calculated. According to the virtual scanner's viewing angle range, the x and y directions are equally divided according to the angular resolution, and combined according to the orientation information to obtain the direction ray of each virtual light of the virtual scanner. ij .

[0143] Step 3: Intersect the virtual ray obtained in step 2 with the Scene_002 model to calculate the true value of its point cloud attributes; it mainly includes three parts: invalid virtual ray removal, virtual ray intersection and point cloud information update.

[0144] Step 4: Add noise to the point cloud according to the scanning parameters, specifically including: polar coordinate system conversion, adding Gaussian noise, and Cartesian coordinate system conversion. In the step of adding Gaussian noise in this embodiment, the user sets the Gaussian noise amplitude t r , t θ 、 It can be set to 10 as a percentage based on the scene size of Scene_002. -3 , 1°, 1°.

[0145] Step 5: Update the true value of the normal direction of the noise point cloud, including:

[0146] Use the Area-Weighted Sampling algorithm to sample s points on the model to be scanned. According to the relatively large characteristics of Scene_002, the value of s can be taken as 10 5 A certain degree of downsampling is performed to improve the efficiency of the subsequent deep learning training process. The k-NN algorithm is used to calculate the nearest high-density sampling point to each point in the point cloud after adding noise, and the normal direction of the patch where the high-density sampling point is located is used as the true value of the normal direction of the point added with noise.

[0147] Step 6: Warning of problematic noise points in the noise point cloud, including:

[0148] For each point in the noise point cloud, the cosine value α of the angle between the updated normal direction and the virtual light direction is calculated. In this embodiment, it is assumed that the user believes that points with an angle between the normal direction and the virtual light direction less than 90° are problematic points. The user can set the warning value α0 to 0. If the cosine value is greater than the warning value α0 set by the user, the point cloud may appear in an area that the original scanner cannot scan. The user is warned to filter or mark these points to improve the accuracy of the noise point cloud.

[0149] The final point cloud result preview of the Scene_002 model obtained by the above steps is as follows: Figure 2 As shown, points of the same color represent that they come from the same virtual scanner.

[0150] In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit, wherein the computer storage medium is capable of storing a computer program that, when executed by the data processing unit, can execute the invention content of the point cloud scanning simulation method for deep learning training provided by the present invention and some or all of the steps in each embodiment. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0151] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented by means of computer programs and their corresponding general hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, in essence or in other words, the part that contributes to the prior art, can be embodied in the form of a computer program, i.e., a software product. The computer program software product can be stored in a storage medium and includes a number of instructions for enabling a device including a data processing unit (which can be a personal computer, server, single-chip microcomputer, MCU, or network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.

[0152] This invention provides a method and approach for point cloud scanning simulation for deep learning training. There are many methods and approaches for implementing this technical solution. The above is only a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the invention, and such improvements and modifications should also be considered within the scope of protection of the invention. Any components not specified in this embodiment may be implemented using existing technologies.

Claims

1. A point cloud scanning simulation method for deep learning training, characterized in that: The steps include: Step 1: Import the 3D model to be scanned, set the virtual scanner and scanning parameters, and build a point cloud scanning simulation model; Step 2, calculating virtual light according to the point cloud scanning simulation model constructed in step 1; Step 3: Intersect the virtual ray with the 3D model to be scanned to obtain the true value of the point cloud attribute and obtain the scan result point cloud; Step 4: Add noise to the scan result point cloud according to the scanning parameters set in step 1; Step 5, update the normal direction of the point cloud after adding noise; Step 6: Warning of problematic noise points in the noise point cloud, and completing the point cloud scanning simulation for deep learning training.

2. A point cloud scanning simulation method for deep learning training according to claim 1, characterized in that: The construction of the point cloud scanning simulation model described in step 1 includes: Step 1-1, set up a virtual scanner around the imported 3D model to be scanned; Step 1-2: Calculate the extrinsic parameter matrix and intrinsic parameter matrix of the virtual scanner according to the set scanning parameters.

3. The point cloud scanning simulation method for deep learning training according to claim 2, characterized in that: Set the scan parameters as described in step 1, including: The virtual scanner's focal point center, the virtual scanner's position eye, the virtual scanner's upward direction vector up, and the virtual scanner's scanning field of view FOV in the x and y directions x and FOV y or the focal length f of the virtual scanner in the x and y directions x and f y , the scanning angle resolution of the virtual scanner in the x and y directions res x and res y .

4. The point cloud scanning simulation method for deep learning training according to claim 3, characterized in that: The calculation of the extrinsic parameter matrix and the intrinsic parameter matrix of the virtual scanner described in steps 1-2 includes: Step 1-2-1, calculate the extrinsic parameter matrix T of the virtual scanner cam→world , as follows: Where R represents the virtual scanner rotation matrix, which is calculated as follows: R=(right new_up fwd) Among them, fwd and right represent the forward and right direction vectors of the virtual scanner respectively, and new_up represents the upward direction vector of the virtual scanner after update. The specific calculation methods are as follows: new_up=right×fwd Among them, ‖ ‖ represents the modulus of the vector, and × represents the vector cross product; Step 1-2-2, calculate the intrinsic parameter matrix K of the virtual scanner, as follows: Among them, f x and f y Represents the focal length of the virtual scanner in the x and y directions, c x and c y They represent the center point coordinates of the image scanned by the virtual scanner, and the specific calculation methods are as follows: in Indicates rounding down.

5. The point cloud scanning simulation method for deep learning training according to claim 4, characterized in that: The calculation of virtual light described in step 2 includes: According to the virtual scanner orientation information, the camera basis vector of the virtual scanner is calculated. According to the virtual scanner viewing angle range, the x and y directions are divided equally according to the angular resolution, and combined according to the orientation information to obtain the direction ray of each virtual light of the virtual scanner. ij , as follows: ray ij =R 2i ·R 1ij ·(-1)·fwd Among them, ray ij represents the virtual ray at row i and column j, R 2ij is the direction angle rotation transformation matrix of the forward direction vector fwd, R 1ij is the pitch angle rotation transformation matrix of the forward direction vector fwd, which is calculated as follows: R 2ij =Rot(θ j ,new_up) Among them, Rot means the calculation is done using the Rodrigues formula. Indicates the angle that the virtual light of row i needs to rotate from the forward direction vector fwd around the right direction vector right, θ j Indicates the angle that the j-th virtual ray needs to rotate from the forward direction vector fwd around the new upward direction vector new_up. The specific calculation method is as follows: If the virtual scanner's scanning field of view (FOV) in the x and y directions is not set when setting the scanning parameters in step 1 x and FOV y , the calculation method is as follows:

6. The point cloud scanning simulation method for deep learning training according to claim 5, characterized in that: The intersection of the virtual ray described in step 3 and the 3D model to be scanned includes: Step 3-1: Eliminate invalid virtual rays, as follows: Using the virtual scanner, obtaining a depth map of the virtual scanner; Find the depth value in the depth map is -1, that is, the area where there is no object in the scan scene, and cast the virtual ray ij Polar coordinates Convert to depth map pixel coordinates After filtering out invalid virtual rays, the coordinate transformation of a single ray is expressed as follows: If the virtual ray ij Corresponding depth map pixel coordinates In the area where there is no object in the above-mentioned scanning scene, the virtual ray is deleted if and only if the following conditions are met at the same time: Among them, depth(i,j) represents the depth value of row i and column j. Indicates rounding up. Indicates rounding down; Step 3-2, finding the intersection points of the 3D model to be scanned and the remaining virtual rays after deleting the invalid virtual rays, and obtaining the Cartesian coordinates of all intersection points and the triangle facet numbers of the 3D model where the intersection points are located; Step 3-3, update the point cloud information and obtain the scanning result point cloud, as follows: According to the Cartesian coordinates of the intersection point, a scanning result point cloud is generated, and according to the triangle facet number, the properties of the triangle facet are obtained from the original file of the 3D model to be scanned, and the properties of the scanning result point cloud are set accordingly.

7. The point cloud scanning simulation method for deep learning training according to claim 6, characterized in that: Add noise to the scan result point cloud described in step 4, including: Step 4: Add noise to the point cloud based on the scanning parameters, including: polar coordinate conversion, adding Gaussian noise, and Cartesian coordinate conversion: Step 4-1: For each point in the scan result point cloud, calculate the polar coordinates in the virtual scanner coordinate system based on the coordinates and depth r1, as follows: θ1=tan -1 (Δy,Δx) Where Δx, Δy, and Δz are the offsets in the x, y, and z coordinate axes, respectively, and are expressed as follows: Δx=x1-eye x Δy=y1-eye y Δz=z1-eye z Among them, eye x 、eye y and eye z is the component of the virtual scanner's position eye on the x, y, and z axes, and x1, y1, and z1 represent the Cartesian coordinates of the point on the scan result point cloud; Step 4-2, add Gaussian noise to get the updated polar coordinates and depth r2, expressed as follows: Among them, t r , t θ and is the preset Gaussian noise amplitude, Represents Gaussian noise with mean 0 and variance 1; Step 4-3, update the polar coordinates The depth r2 is restored to the Cartesian coordinate system to obtain the point coordinates x2, y2 and z2 after adding noise.

8. The point cloud scanning simulation method for deep learning training according to claim 7, characterized in that: The coordinates x2, y2, and z2 of the point after adding noise in step 4-3 are expressed as follows:

9. The point cloud scanning simulation method for deep learning training according to claim 8, characterized in that: The normal direction of the point cloud after adding noise as described in step 5 is updated, including: Step 5-1, sampling s points on the 3D model to be scanned as high-density sampling points; Step 5-2, use the k-NN clustering method to calculate the high-density sampling points closest to each point on the point cloud after adding noise; In step 5-3, the normal direction of the patch of the 3D model where the high-density sampling point is located is used as the normal direction of the point on the point cloud with added noise.

10. The point cloud scanning simulation method for deep learning training according to claim 9, characterized in that: The problematic noise points in the warning noise point cloud described in step 6 include: For each point in the point cloud after adding noise, calculate the cosine value α of the angle between the updated normal direction and the virtual light direction. If the cosine value α is greater than the warning value α0 set by the user, an alert is issued for the point.

Images