Virtual camera point cloud simulation method based on error mechanism modeling

Abstract

This invention relates to a virtual camera point cloud simulation method based on error mechanism modeling, encompassing the fields of virtual assembly simulation, 3D point cloud generation, and machine vision inspection. Existing methods for physically adjusting camera positions are inefficient and inaccurate. This method constructs a virtual camera and a model of the workpiece under test within a 3D simulation environment, setting virtual camera parameters. The virtual camera performs laser scanning on the workpiece model to obtain a set of pixels. This set is then filtered, and the filtered pixels are used as the ideal point cloud. Preset distortion coefficients and depth distortion coefficients are applied to the ideal point cloud to obtain a distorted point cloud. Preset random noise is then applied to the distorted point cloud, and the point cloud after applying random noise is filtered to obtain the final point cloud. The pose of the workpiece model is obtained from the final point cloud. If the pose is not the preset pose, the virtual camera position is adjusted. This invention is used to simulate point clouds formed by a virtual camera.

Description

Technical Field

[0001] This invention relates to the fields of virtual assembly simulation, 3D point cloud generation, and machine vision inspection technology. Background Technology

[0002] 3D point cloud data is widely used in complex industrial assembly tasks for workpiece docking, pose measurement, and assembly gap detection. Therefore, the choice of camera placement directly affects the quality of point cloud data and the recognition accuracy of subsequent algorithms. To obtain comprehensive and clearly defined point clouds, engineers typically need to repeatedly adjust the camera posture in real-world environments, collect point cloud data under different poses, and analyze the data to determine the optimal camera placement scheme. However, this physically-based adjustment method has the following drawbacks:

[0003] 1. Low adjustment efficiency: Each change in camera pose requires manual intervention, resulting in a long adjustment cycle.

[0004] Second, poor accuracy: Lighting and other conditions in the real environment are difficult to keep constant. Data of the same pose collected at different times may vary. In addition, the camera has uncertain error factors such as ranging error, field of view edge degradation, and random noise in real time. It is difficult to quantify the impact of error factors on point cloud quality, resulting in inaccurate camera position.

[0005] In summary, existing methods of physically adjusting camera position are inefficient and inaccurate. Summary of the Invention

[0006] The purpose of this invention is to solve the problems of low efficiency and poor accuracy of existing physical methods for adjusting camera position, and to propose a virtual camera point cloud simulation method based on error mechanism modeling.

[0007] A virtual camera point cloud simulation method based on positive voxel laser scanning and error mechanism modeling, the method comprising:

[0008] Step 1: Construct a virtual camera and a model of the workpiece to be tested in a 3D simulation environment, and set the virtual camera parameters;

[0009] Step 2: Adjust the position of the virtual camera for the nth time, and use the adjusted virtual camera to perform laser scanning on the workpiece model to obtain the pixel set of the workpiece model. The pixel set of the workpiece model is filtered according to the occlusion relationship of the real camera scanning the real workpiece. The filtered pixels are used as the ideal point cloud of the workpiece model. The initial value of n is 1.

[0010] Step 3: Apply the set distortion coefficient and depth distortion coefficient to the ideal point cloud of the workpiece model to be tested to obtain the distorted point cloud. Apply the set random noise to the distorted point cloud, filter the point cloud after applying random noise, and obtain the final point cloud. Obtain the pose of the workpiece model to be tested based on the final point cloud.

[0011] Step 4: Determine whether the pose of the workpiece model under test is the preset pose. If yes, output the virtual camera position corresponding to the pose of the workpiece model under test. If no, proceed to step 5.

[0012] Step 5: Set n = n + 1, then execute step 2.

[0013] Preferably, the virtual camera parameters include the virtual camera's resolution, field of view, near and far clipping planes, focal length, and depth of field.

[0014] Preferably, in step 2, the pixel set of the workpiece model to be tested is filtered according to the occlusion relationship of the real workpiece scanned by the real camera, specifically as follows:

[0015] Pixels that exceed the near-far clipping plane or depth of field range in the pixel set of the workpiece model to be tested are removed. The remaining pixels are further filtered as follows: if a ray emitted from the virtual camera exit point has multiple intersection points with the workpiece model to be tested, the intersection point corresponding to the shortest distance among the multiple intersection points is retained as the filtered pixel.

[0016] Preferably, each distorted point cloud consists of three-dimensional coordinates, the three-dimensional coordinates including the distorted... Plane coordinates, after distortion Planar coordinates and depth.

[0017] Preferably, after distortion Plane coordinates are represented as:

[0018] ,

[0019] In the formula, For the first After distortion at each point Plane coordinates and The lens tangential distortion coefficient. For the ideal point cloud points To coordinates, For the ideal point cloud points To coordinates, For normalized distance, , and All are lens radial distortion coefficients.

[0020] Preferably, after distortion Plane coordinates are represented as:

[0021] ,

[0022] In the formula, For the first After distortion at each point Planar coordinates.

[0023] Preferably, the depth is expressed as:

[0024] ,

[0025] In the formula, For the first Depth of each point For the ideal point cloud Depth of each point For the first Depth distortion deviation at each point and All are depth distortion coefficients. .

[0026] Preferably, the random noise includes Gaussian noise, uniform noise, or salt-and-pepper noise.

[0027] Preferably, the point cloud after applying random noise is screened, and the specific process is as follows:

[0028] The reflection intensity of each point is obtained based on the ambient light intensity of the simulation environment, the reflectivity of the workpiece material at each point in the point cloud after applying random noise, the incident angle at each point in the point cloud after applying random noise, and the depth at each point in the ideal point cloud or the depth at each point after applying random noise. The reflection intensity of each point is compared with a preset intensity threshold, and points with reflection intensities lower than the preset intensity threshold are deleted.

[0029] Preferably, the reflection intensity at each point is expressed as:

[0030] ,

[0031] In the formula, After applying random noise, the first The reflection intensity at each point To simulate ambient light intensity, After applying random noise, the first The reflectance of the workpiece material at each point is to be measured. After applying random noise, the first The angle of incidence at each point For the ideal point cloud The depth of the nth point or the nth point after applying random noise Depth of each point, It is a constant.

[0032] The beneficial effects of this invention are:

[0033] By replacing depth inversion with forward scanning, an ideal point cloud can be generated in a virtual environment with a clear mechanism. Then, by applying predefined distortion and depth distortion coefficients to the ideal point cloud, the generated distorted point cloud accurately simulates the image bending and shifting phenomena caused by imperfections in the optical system of a real camera, overcoming the deviation between the ideal pinhole model and the physical lens in existing technologies. Random noise is then applied to the distorted point cloud to simulate environmental interference and other factors, enhancing the diversity of the simulation data and making it closer to the measured data. Therefore, through distortion and noise, the generated point cloud maintains a high degree of consistency with the output of a real 3D camera. This allows for the accurate and efficient determination of a suitable camera position through simulation, enabling the deployment of the real camera at this position and ultimately obtaining a clear point cloud of the workpiece under test. This facilitates the detection of gaps at preset positions on the workpiece. Attached Figure Description

[0034] Figure 1 The flowchart shows a virtual camera point cloud simulation method based on error mechanism modeling.

[0035] Figure 2 This is a schematic diagram of a workpiece model being scanned using laser scanning.

[0036] Figure 3 The occlusion relationship diagram of the real workpiece to be tested is scanned by a real camera;

[0037] Figure 4 A schematic diagram of a virtual camera scanning another workpiece model to be tested. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the invention.

[0040] Example:

[0041] A virtual camera point cloud simulation method based on error mechanism modeling, characterized in that the method includes:

[0042] Step 1: Construct a virtual camera and a model of the workpiece to be tested in a 3D simulation environment, and set the virtual camera parameters;

[0043] Step 2: Adjust the position of the virtual camera for the nth time, and use the adjusted virtual camera to perform laser scanning on the workpiece model to obtain the pixel set of the workpiece model. The pixel set of the workpiece model is filtered according to the occlusion relationship of the real camera scanning the real workpiece. The filtered pixels are used as the ideal point cloud of the workpiece model. The initial value of n is 1.

[0044] Step 3: Apply the set distortion coefficient and depth distortion coefficient to the ideal point cloud of the workpiece model to be tested to obtain the distorted point cloud. Apply the set random noise to the distorted point cloud, filter the point cloud after applying random noise, and obtain the final point cloud. Obtain the pose of the workpiece model to be tested based on the final point cloud.

[0045] Step 4: Determine whether the pose of the workpiece model under test is the preset pose. If yes, output the virtual camera position corresponding to the pose of the workpiece model under test. If no, proceed to step 5.

[0046] Step 5: Set n = n + 1, then execute step 2.

[0047] Further specifying, the virtual camera parameters include the virtual camera's resolution, field of view, near and far clipping planes, focal length, and depth of field.

[0048] Further specifying, in step 2, the pixel set of the workpiece model to be tested is filtered according to the occlusion relationship of the real workpiece scanned by the actual camera, specifically as follows:

[0049] Pixels that exceed the near-far clipping plane or depth of field range in the pixel set of the workpiece model to be tested are removed. The remaining pixels are further filtered as follows: if a ray emitted from the virtual camera exit point has multiple intersection points with the workpiece model to be tested, the intersection point corresponding to the shortest distance among the multiple intersection points is retained as the filtered pixel.

[0050] Specifically, this embodiment is deployed in a 3D simulation environment, using Unity as an example below. The scene includes a model of the workpiece to be tested, a target model, and a background model. The virtual camera is installed in a designated position and maintains fixed extrinsic parameters with the scene. The virtual camera parameters are configured through the interactive interface, including horizontal and vertical resolution, field of view, near and far clipping planes, focal length, and depth of field. During the point cloud generation stage, the pixel set is traversed according to a set sampling step size each time a scan is triggered. Pixels that exceed the near and far clipping planes or the depth of field are discarded. The retained points are further filtered, and the coordinates of the further filtered retained points are transformed to the camera coordinate system to form an ideal point cloud. The normal and material reflectivity of the ideal point cloud are recorded simultaneously.

[0051] Figure 2 The frustum-shaped region formed by the yellow and purple planes in front of the camera is the virtual camera imaging area; the green dots on the workpiece are the ideal point cloud. Light emitted from the camera illuminates... Figure 2 At each green dot on the workpiece under test, light will penetrate the workpiece along each green dot and illuminate the other side of the workpiece. However, in this embodiment, for multiple green lines formed by a single light point, only the shortest green line is taken to form an ideal point cloud, such as... Figure 3 As shown, this embodiment is to simulate the point cloud formed on the actual workpiece when the actual camera illuminates it. Due to the occlusion relationship, the workpiece on the other side cannot be illuminated when the actual camera illuminates it.

[0052] This embodiment allows for flexible setting of the virtual camera's pose. It is suitable for verifying pose recognition algorithms in the early stages of related engineering projects, and for quickly finding suitable camera positions for depth cameras.

[0053] Further specifying, each distorted point cloud consists of three-dimensional coordinates, which include the distorted... Plane coordinates, after distortion Planar coordinates and depth.

[0054] Further restrictions, after distortion Plane coordinates are represented as:

[0055] ,

[0056] In the formula, For the first After distortion at each point Plane coordinates and The lens tangential distortion coefficient. For the ideal point cloud points To coordinates, For the ideal point cloud points To coordinates, For normalized distance, , and All are lens radial distortion coefficients.

[0057] Further restrictions, after distortion Plane coordinates are represented as:

[0058] ,

[0059] In the formula, For the first After distortion at each point Planar coordinates.

[0060] Distortion is achieved by projecting an ideal point onto the image plane to obtain normalized coordinates, obtaining distorted coordinates according to the distortion model, and then projecting it back into three-dimensional space.

[0061] Further specifying, depth is represented as:

[0062] ,

[0063] In the formula, For the first Depth of each point For the ideal point cloud Depth of each point For the first Depth distortion deviation at each point and All are depth distortion coefficients. .

[0064] To further define, random noise includes Gaussian noise, uniform noise, or salt-and-pepper noise.

[0065] Specifically, random noise can also be flexibly combined according to the actual sensor characteristics to simulate sensor circuit noise, environmental interference and other factors.

[0066] Further refinement involves filtering the point cloud after applying random noise. The specific process is as follows:

[0067] The reflection intensity of each point is obtained based on the ambient light intensity of the simulation environment, the reflectivity of the workpiece material at each point in the point cloud after applying random noise, the incident angle at each point in the point cloud after applying random noise, and the depth at each point in the ideal point cloud or the depth at each point after applying random noise. The reflection intensity of each point is compared with a preset intensity threshold, and points with reflection intensities lower than the preset intensity threshold are deleted.

[0068] Further specifying, the reflection intensity at each point is expressed as:

[0069] ,

[0070] In the formula, After applying random noise, the first The reflection intensity at each point To simulate ambient light intensity, After applying random noise, the first The reflectance of the workpiece material at each point is to be measured. After applying random noise, the first The angle of incidence at each point For the ideal point cloud The depth of the nth point or the nth point after applying random noise Depth of each point, It is a constant.

[0071] Specifically, the incident angle refers to the angle between the scanning ray and the normal.

[0072] The reflection intensity and missed detection stage calculates the echo intensity based on the material reflectivity, incident angle, and distance. Points below a threshold are identified as missed and removed to simulate missed detections in weak reflection areas. The probability of missed detection is increased at the obstruction boundary to simulate the sparse and fragmented phenomena of real boundary points. At the same time, different thresholds or different noise levels can be set for different material areas to form an error difference consistent with the surface condition of the project.

[0073] The distortion coefficient, noise parameters, and intensity threshold can all be adjusted independently, making it easy to generate point cloud samples under different error conditions.

[0074] When it is necessary to change the workpiece or target, the new model is loaded through the model management module and the template point cloud is re-exported, enabling rapid replacement of the point cloud content with the template while maintaining consistency with the algorithm interface. After model replacement, the camera pose control module can be used to repeatedly scan under different observation postures, forming a multi-pose template library or a multi-condition dataset.

[0075] This embodiment generates point clouds while simultaneously acquiring RGB images of the workpiece under test. The position of each sub-workpiece within the workpiece is obtained from the point cloud. For locations with a small number of point clouds in the workpiece under test, the limited number of points cannot be used to detect gaps. Therefore, for locations with fewer points, the RGB image is used to detect gaps, enabling the detection of alignment issues, assembly misalignments, etc., at specific points within the workpiece. Point clouds and RGB images can be generated in batches under different field of view angles, distances, point cloud densities, and combinations of noise parameters.

[0076] The accuracy of the point cloud obtained in this embodiment can be verified using the following methods:

[0077] The accuracy of point clouds is assessed using statistical point cloud error metrics, statistical gap measurement metrics, or preset image content. Statistical point cloud error metrics include RMSE, MAE, maximum error, false negative rate, and outlier ratio. Statistical gap measurement metrics include pixel gap, millimeter gap, average error, and maximum error. Different combinations of filtering, thresholding, and edge operators are compared. Preset image content includes point cloud RMSE curves under different noise standard deviations, false negative rate curves under different thresholds, gap error box plots under different threshold strategies, edge positioning deviation comparisons under different edge operators, and comparisons of the improvement in gap measurement stability brought about by local ROI encryption. Tables are constructed, including a list of parameter combinations and corresponding point counts, error statistics, and a comparison table of theoretical and measured gap values ​​at key locations.

[0078] While the invention has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the invention. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.

Claims

1. A virtual camera point cloud simulation method based on error mechanism modeling, characterized in that, The method includes: Step 1: Construct a virtual camera and a model of the workpiece to be tested in a 3D simulation environment, and set the virtual camera parameters; Step 2: Adjust the position of the virtual camera for the nth time, and use the adjusted virtual camera to perform laser scanning on the workpiece model to obtain the pixel set of the workpiece model. The pixel set of the workpiece model is filtered according to the occlusion relationship of the real camera scanning the real workpiece. The filtered pixels are used as the ideal point cloud of the workpiece model. The initial value of n is 1. Step 3: Apply the set distortion coefficient and depth distortion coefficient to the ideal point cloud of the workpiece model to be tested to obtain the distorted point cloud. Apply the set random noise to the distorted point cloud, filter the point cloud after applying random noise, and obtain the final point cloud. Obtain the pose of the workpiece model to be tested based on the final point cloud. Step 4: Determine whether the pose of the workpiece model under test is the preset pose. If yes, output the virtual camera position corresponding to the pose of the workpiece model under test. If no, proceed to step 5. Step 5: Set n = n + 1, then execute step 2.

2. The virtual camera point cloud simulation method based on error mechanism modeling according to claim 1, characterized in that, Virtual camera parameters include the virtual camera's resolution, field of view, near and far clipping planes, focal length, and depth of field.

3. The virtual camera point cloud simulation method based on error mechanism modeling according to claim 2, characterized in that, In step 2, the pixel set of the workpiece model to be tested is filtered according to the occlusion relationship of the real workpiece scanned by the real camera. Specifically: Pixels that exceed the near-far clipping plane or depth of field range in the pixel set of the workpiece model to be tested are removed. The remaining pixels are further filtered as follows: if a ray emitted from the virtual camera exit point has multiple intersection points with the workpiece model to be tested, the intersection point corresponding to the shortest distance among the multiple intersection points is retained as the filtered pixel.

4. The virtual camera point cloud simulation method based on error mechanism modeling according to claim 1, characterized in that, Each distorted point cloud consists of three-dimensional coordinates, which include the distorted... Plane coordinates, after distortion Planar coordinates and depth.

5. The virtual camera point cloud simulation method based on error mechanism modeling according to claim 4, characterized in that, After distortion Plane coordinates are represented as: ， In the formula, For the first After distortion at each point Plane coordinates and The lens tangential distortion coefficient. For the ideal point cloud points To coordinates, For the ideal point cloud points To coordinates, For normalized distance, , and All are lens radial distortion coefficients.

6. The virtual camera point cloud simulation method based on error mechanism modeling according to claim 5, characterized in that, After distortion Plane coordinates are represented as: ， In the formula, For the first After distortion at each point Planar coordinates.

7. The virtual camera point cloud simulation method based on error mechanism modeling according to claim 6, characterized in that, Depth is represented as: ， In the formula, For the first Depth of each point For the ideal point cloud Depth of each point For the first Depth distortion deviation at each point and All are depth distortion coefficients. .

8. The virtual camera point cloud simulation method based on error mechanism modeling according to claim 1, characterized in that, Random noise includes Gaussian noise, uniform noise, or salt and pepper noise.

9. The virtual camera point cloud simulation method based on error mechanism modeling according to claim 1, characterized in that, The specific process for filtering point clouds after applying random noise is as follows: The reflection intensity of each point is obtained based on the ambient light intensity of the simulation environment, the reflectivity of the workpiece material at each point in the point cloud after applying random noise, the incident angle at each point in the point cloud after applying random noise, and the depth at each point in the ideal point cloud or the depth at each point after applying random noise. The reflection intensity of each point is compared with a preset intensity threshold, and points with reflection intensities lower than the preset intensity threshold are deleted.

10. The virtual camera point cloud simulation method based on error mechanism modeling according to claim 9, characterized in that, The reflection intensity at each point is represented as: ， In the formula, After applying random noise, the first The reflection intensity at each point To simulate ambient light intensity, After applying random noise, the first The reflectance of the workpiece material at each point is to be measured. After applying random noise, the first The angle of incidence at each point For the ideal point cloud The depth of the nth point or the nth point after applying random noise Depth of each point, It is a constant.

Images