A method, apparatus, electronic device, and storage medium for repairing point cloud holes.

Abstract

This invention discloses a method, apparatus, electronic device, and storage medium for repairing point cloud holes. The method includes: acquiring three-dimensional outliers near the holes in the point cloud, wherein the three-dimensional outliers are noisy point clouds where the point cloud information exists; projecting an outlier plane formed by the outliers located on the same plane to obtain two-dimensional outlier point cloud information, and determining the length of the outlier plane based on the two-dimensional outlier point cloud information; acquiring the correlation between the length of the outlier plane and the intensity of line structured light, and determining the error distance between the outlier plane and the surface where the point cloud hole is located based on the correlation; and determining the position of the point cloud hole based on the error distance. This invention repairs holes based on the position information of outliers, realizing the filling of point cloud holes, greatly preserving the true shape of the object, providing low-noise point cloud data with complete object geometric features for subsequent registration, and having better accuracy and robustness.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional vision technology, specifically to a method, apparatus, electronic device, and storage medium for repairing holes in point clouds. Background Technology

[0002] In current industrial production applications, point clouds are collections of point data on the surface of a product obtained through measuring instruments. For 3D reconstruction of objects, structured light triangulation is mostly used to quickly acquire large amounts of high-precision point cloud data of the object's surface in a short period of time.

[0003] However, during actual point cloud data acquisition, the inherent errors of the acquisition equipment and the lighting or reflection properties of the scene object surface inevitably cause the point cloud acquired by the structured light 3D scanning system sensor to contain noise and outliers. This results in the loss of correct location information and outliers, causing the loss of point cloud information in some areas of the scanned object, forming point cloud holes. These problems greatly impair the image quality acquired by the imaging equipment, affecting the accuracy and completeness of the generated 3D model, and also significantly impacting subsequent surface reconstruction and point cloud classification. Summary of the Invention

[0004] The purpose of this invention is to overcome the above-mentioned technical deficiencies and provide a point cloud hole repair method, device, electronic device and storage medium to solve the technical problem in the prior art that point cloud holes are caused by strong reflection and structural reflection in the three-dimensional reconstruction of reflective areas.

[0005] To achieve the above-mentioned technical objectives, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides a method for repairing point cloud holes, comprising:

[0007] Obtain three-dimensional outliers near holes in the point cloud, wherein the three-dimensional outliers are noisy point clouds in which the point cloud information exists;

[0008] The outlier plane formed by the outlier points located on the same plane is projected to obtain two-dimensional outlier point cloud information, and the length of the outlier plane is determined based on the two-dimensional outlier point cloud information.

[0009] Obtain the correlation between the outlier plane length and the line structured light intensity, and determine the error distance between the outlier plane and the surface where the point cloud hole is located based on the correlation.

[0010] The location of the point cloud hole is determined based on the error distance.

[0011] In some embodiments, before acquiring the 3D outliers near the point cloud holes, the method further includes:

[0012] Based on the line structured light 3D reconstruction method and image acquisition system, the 3D point cloud information of the surface of the object under test is obtained, and the location of the hole in the point cloud is marked.

[0013] Based on the location of the point cloud holes, the three-dimensional outlier points are obtained.

[0014] In some embodiments, projecting the outlier plane formed by the outliers located in the same plane specifically involves projecting the outlier plane onto the direction perpendicular to the plane where the laser line and the field of view of the acquisition device are located; the projection of the outlier plane can be expressed by the following formula:

[0015] f xoz (P i (x,y,z)=P i (x,0,z)

[0016] Where: f xoz (·) represents the projection onto the xoz plane; P i (x,y,z) represents the location information of outliers.

[0017] In some embodiments, determining the length of the outlier plane based on the two-dimensional outlier point cloud information includes:

[0018] Obtain the positions of the end point and the start point of the outlier plane;

[0019] The length of the outlier plane is determined based on the positions of the endpoints and the starting point; the length of the outlier plane can be expressed by the following formula:

[0020]

[0021] Where, d o This represents the length of the outlier plane, with the end point P1(X1,0,Z1) and the starting point P2(X2,0,Z2).

[0022] In some embodiments, obtaining the correlation between the outlier plane length and the line structured light intensity, and determining the error distance between the outlier plane and the surface where the point cloud hole is located, includes:

[0023] Based on the principle of optical model, the slope of the outlier plane is determined according to the length of the outlier plane;

[0024] Gray-scale centroid extraction is performed on the laser line to obtain the surface gray-scale distribution information of the object under test;

[0025] Based on the slope of the outlier plane and the surface grayscale distribution information, the slope of the grayscale change of the linear structured light is determined;

[0026] Based on the outlier plane length, surface grayscale distribution information, and the slope of grayscale change of the linear structured light, the error distance between the outlier plane and the surface where the point cloud hole is located is determined.

[0027] In some embodiments, the error distance between the outlier plane and the surface where the point cloud holes are located can be expressed by the following formula: Where, d o Let d be the length of the outlier plane. l For error distance, This represents the surface grayscale distribution information.

[0028] In some embodiments, determining the location of the point cloud hole based on the error distance includes:

[0029] Obtain the angle between the CCD camera's field of view and the structured light;

[0030] Based on the plane perpendicular to the plane of the laser line and the field of view of the acquisition device, the coordinate transformation method is used to determine the true surface point cloud information according to the included angle.

[0031] Secondly, the present invention also provides a point cloud hole repair device, comprising:

[0032] The acquisition module is used to acquire three-dimensional outliers near holes in the point cloud, wherein the three-dimensional outliers are noisy point clouds in which the point cloud information exists;

[0033] The outlier plane length determination module is used to project the outlier plane formed by the outlier points located on the same plane to obtain two-dimensional outlier point cloud information, and determine the length of the outlier plane based on the two-dimensional outlier point cloud information.

[0034] An error distance determination module is used to obtain the correlation between the outlier plane length and the line structured light intensity, and to determine the error distance between the outlier plane and the surface where the point cloud hole is located based on the correlation.

[0035] The point cloud hole location determination module is used to determine the location of the point cloud hole based on the error distance.

[0036] Thirdly, the present invention also provides an electronic device, comprising: a processor and a memory;

[0037] The memory stores a computer-readable program that can be executed by the processor;

[0038] When the processor executes the computer-readable program, it implements the steps in the point cloud hole repair method described above.

[0039] Fourthly, the present invention also provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the point cloud hole repair method described above.

[0040] Compared with the prior art, the point cloud hole repair method, apparatus, electronic device and storage medium provided by the present invention first obtain three-dimensional outliers near the point cloud hole, obtain the corresponding two-dimensional outlier point cloud information by projecting the three-dimensional outliers onto the corresponding two-dimensional plane, thereby determining the length of the outlier plane, and determining the error distance between the outlier plane and the surface where the point cloud hole is located based on the correlation between the length of the outlier plane and the intensity of the line structured light, thereby determining the location of the point cloud hole and repairing the point cloud hole. The point cloud hole repair method provided by this invention can effectively fill point cloud holes, effectively solving the problem of point cloud holes caused by mixed reflections during line structured light scanning. Furthermore, compared to other point cloud hole repair methods that mostly start from the edge of the hole and use deep learning and other methods to repair the hole from the outside in, where the repair effect deteriorates towards the center of the hole, this method significantly reduces the workload of subsequent point cloud repair. Moreover, by repairing the positional information of outliers, the method achieves the filling of point cloud holes, greatly preserving the true shape of the object. This provides low-noise point cloud data with intact geometric features for subsequent registration, and offers better accuracy and robustness. Attached Figure Description

[0041] Figure 1 This is a flowchart of an embodiment of the point cloud hole repair method provided by the present invention;

[0042] Figure 2 This is a flowchart of an embodiment of the point cloud hole repair method provided by the present invention, which involves obtaining outliers;

[0043] Figure 3 This is a schematic diagram of the direct-fire laser triangulation image acquisition system in the point cloud hole repair method provided by the present invention;

[0044] Figure 4 This is a flowchart of an embodiment of step S102 in the point cloud hole repair method provided by the present invention;

[0045] Figure 5 This is a flowchart of an embodiment of step S103 in the point cloud hole repair method provided by the present invention;

[0046] Figure 6 This is a simplified diagram of the outlier planar structure under the hybrid reflection model in the point cloud hole repair method provided by the present invention;

[0047] Figure 7This is a flowchart of an embodiment of step S104 in the point cloud hole repair method provided by the present invention;

[0048] Figure 8 This is a schematic diagram of an embodiment of the point cloud hole repair device provided by the present invention;

[0049] Figure 9 This is a schematic diagram of the operating environment of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0051] The point cloud hole repair method, apparatus, device, or computer-readable storage medium involved in this invention can be used in OpenMP-based and CUDA-based point cloud hole repair algorithms to obtain clearer and more complete 3D models in 3D reconstruction. The method, apparatus, device, or computer-readable storage medium involved in this invention can be integrated with the aforementioned systems or operate relatively independently.

[0052] Figure 1 This is a flowchart of the point cloud hole repair method provided in the embodiments of the present invention. Please refer to it. Figure 1 Point cloud hole repair methods include:

[0053] S101. Obtain three-dimensional outliers near the holes in the point cloud, wherein the three-dimensional outliers are noisy point clouds in which the point cloud information exists;

[0054] S102. Project the outlier plane formed by the outlier points located on the same plane to obtain two-dimensional outlier point cloud information, and determine the length of the outlier plane based on the two-dimensional outlier point cloud information.

[0055] S103. Obtain the correlation between the length of the outlier plane and the intensity of the line structured light, and determine the error distance between the outlier plane and the surface where the point cloud hole is located based on the correlation.

[0056] S104. Determine the position of the point cloud hole based on the error distance.

[0057] In this embodiment of the invention, three-dimensional outliers near the point cloud holes are first obtained. By projecting the three-dimensional outliers onto the corresponding two-dimensional plane, the corresponding two-dimensional outlier point cloud information is obtained, thereby determining the length of the outlier plane. Based on the relationship between the length of the outlier plane and the intensity of the line structured light, the error distance between the outlier plane and the surface where the point cloud hole is located is determined, thereby determining the location of the point cloud hole and repairing the point cloud hole. The point cloud hole repair method provided by this invention can effectively fill point cloud holes, effectively solving the problem of point cloud holes caused by mixed reflections during line structured light scanning. Furthermore, compared to other point cloud hole repair methods that mostly start from the edge of the hole and use deep learning and other methods to repair the hole from the outside in, where the repair effect deteriorates towards the center of the hole, this method significantly reduces the workload of subsequent point cloud repair. Moreover, by repairing holes based on the location information of outliers, it achieves the filling of point cloud holes, greatly preserving the true shape of the object. This provides low-noise point cloud data with intact geometric features for subsequent registration, and offers better accuracy and robustness.

[0058] It should be noted that in this embodiment, outliers are extracted manually near the holes in the point cloud. Outliers are noise points in the captured image caused by factors such as the surface properties of the object being measured, the method of acquiring the object's 3D information, environmental factors, and lighting conditions. These noise points in the generated point cloud information of the object's surface are called outliers. The plane formed by outliers located on the same plane is called the outlier plane. In typical point cloud processing, outliers are considered erroneous point cloud data and are generally treated as noise points to be removed to ensure the accuracy of the model. However, the point cloud hole repair method of this invention relies on outlier information to repair point cloud holes, and therefore is not considered as point cloud noise removal.

[0059] Furthermore, a point cloud is specifically a dataset containing information about the surface of the object being measured. It consists of several points that reflect the surface information of the object, and each point contains X, Y, Z, and other information. Point cloud holes occur because the surface properties of the object being measured, the method of acquiring the object's three-dimensional information, environmental factors, etc., result in a certain degree of loss in the actual obtained three-dimensional point cloud information, making the entire three-dimensional point cloud data appear to have holes.

[0060] Specifically, the obtained three-dimensional point cloud information is used to obtain two-dimensional point cloud information by projecting the outlier plane onto the corresponding plane, thereby realizing the dimensionality reduction operation of the point cloud. The corresponding plane is the direction perpendicular to the laser line and the plane where the monitoring field of view is located, and its purpose is to facilitate the measurement of the length of the outlier plane formed by the outlier points.

[0061] In some embodiments, please refer to Figure 2Before acquiring the 3D outliers near the point cloud holes, the method further includes:

[0062] S201. Based on the line structured light 3D reconstruction method and image acquisition system, obtain the 3D point cloud information of the surface of the object under test, and mark the position of the hole in the point cloud;

[0063] S202. Based on the location of the point cloud holes, obtain the three-dimensional outlier points.

[0064] In this embodiment, based on line structured light 3D reconstruction technology, the 3D surface point cloud information of the measured object is acquired through a direct-fire laser triangulation image acquisition system. Specifically, the direct-fire laser triangulation image acquisition system includes a moving platform, a laser, a moving stage, a sensor bracket, and a CCD camera, as shown in the specific structure below. Figure 3 As shown, the specific workflow of the direct-fire laser triangulation image acquisition system is as follows: The laser is fixed perpendicular to the vision-moving platform. The laser beam illuminates the object being measured, which is placed on a unidirectional motion platform at a specific speed. A CCD camera at a certain angle to the laser acquires the surface information of the object using laser triangulation, ultimately generating a 3D point cloud of the object's surface. Further, the object is scanned by a line laser, and simultaneously photographed by a camera to obtain an image with structured light. The 3D coordinates of the points on the structured light are extracted. By scanning the entire object with the laser, the 3D coordinates of all points can be calculated, achieving a 3D reconstruction of the object's surface. This allows for the measurement of the distance between any two points on the object's surface.

[0065] In one specific embodiment, based on line structured light 3D reconstruction technology, a direct-fire laser triangulation image acquisition system is used to acquire the 3D surface point cloud information of the object under test. The direct-fire laser triangulation image acquisition system includes a line-shaped infrared laser (model FU650AB100-GD16-WLD) with a power of 100mW and a wavelength of 650nm red light. The CCD camera is model C2-2040-GigE with a resolution of 2048×1088 pixels. The moving platform is model ZM-LSCAN-M with a moving speed of 2.5cm / s; the workpiece is a WD-2 automotive connecting rod with distinct hybrid reflection characteristics.

[0066] Specifically, a linear infrared laser is mounted above the moving platform, vertically projecting the laser line onto the surface of the connecting rod workpiece on the platform's platform. A CCD camera is mounted at a certain angle above the moving platform to capture the infrared laser line in the field of view. During the operation of the moving platform, the CCD camera continuously acquires infrared structured light images of the car connecting rod surface, finally generating a corresponding 3D point cloud model of the WD-2 workpiece. The point cloud model is then preprocessed by removing planes and noise points.

[0067] In some embodiments, the projection of the outlier plane can be expressed by the following formula:

[0068] f xoz (P i (x,y,z)=P i (x,0,z)

[0069] Where: f xoz (·) represents the projection onto the xoz plane; P i (x,y,z) represents the location information of outliers.

[0070] In this embodiment, a two-dimensional outlier plane is obtained by projecting the three-dimensional outlier plane, and the length of the outlier plane can be calculated.

[0071] In some embodiments, please refer to Figure 4 The step of determining the length of the outlier plane based on the two-dimensional outlier point cloud information includes:

[0072] S401. Obtain the position of the end point and the position of the start point of the outlier plane;

[0073] S402. Determine the length of the outlier plane based on the position of the endpoint and the position of the starting point; the length of the outlier plane can be expressed by the following formula:

[0074]

[0075] Where, d o This represents the length of the outlier plane, with the end point P1(X1,0,Z1) and the starting point P2(X2,0,Z2).

[0076] In this embodiment, by manually extracting outliers, the positions of the first and last endpoints of the outlier plane can be obtained, and the length of the outlier plane can be determined according to the outlier plane length formula.

[0077] In some embodiments, please refer to Figure 5 The step of obtaining the correlation between the outlier plane length and the line structured light intensity, and determining the error distance between the outlier plane and the surface where the point cloud hole is located, includes:

[0078] S501. Based on the principle of optical model, determine the slope of the outlier plane according to the length of the outlier plane;

[0079] S502. Extract the gray-scale centroid of the laser line to obtain the surface gray-scale distribution information of the object being measured.

[0080] S503. Determine the slope of the gray-scale change of the linear structured light based on the slope of the outlier plane and the surface gray-scale distribution information;

[0081] S504. Based on the outlier plane length, surface grayscale distribution information, and the slope of the grayscale change of the linear structured light, determine the error distance between the outlier plane and the surface where the point cloud hole is located.

[0082] In this embodiment, based on the constructed triangular laser measurement equipment, the actual line structured light grayscale distribution information, and the simplified structural diagram of the optical principle, as shown below... Figure 6 As shown, the optical model principle is as follows: a laser source emits Gaussian laser stripes onto a surface, producing diffuse and specular reflections. The diffuse light (green line) still follows a Gaussian distribution, but only accounts for a portion of the light source intensity (represented by the length of the green stripe). The CCD sensor on the left captures the diffuse reflection, detects the peak value, and records a point (green dot) on the scanned surface. Due to the Gaussian distribution of the laser intensity, during the entire scanning process, there will be areas where the non-central laser light illuminates the edge regions. These edge regions are characterized by continuously changing surface normals. The camera receives this specularly reflected light. Although the light source intensity in this area is less than that of the central light, when this large portion of the light intensity is reflected by the specular surface, it increases the intensity received by the camera, forming a local peak. Therefore, the sensor and camera have two peak points mixed together: one is the diffusely reflected light from the side of the central light, and the other is the specularly reflected light from the edge region with a forbidden normal. When the light intensity is high enough, the peak value of specular reflection will also become a recorded data point. However, such outlier points (red) deviate significantly from the true value points (green), representing incorrect measurement data points, not data points on the scanned plane. As the laser moves to the left, the distance between the diffuse reflection region at the laser center and the specular reflection region outside the center gradually increases, leading to an increase in outlier points and a greater distance from the true scanned plane, forming a plane constructed from outlier points, i.e., the outlier plane. Because the light intensity is weaker on both sides of the Gaussian distribution intensity of the light source, the intensity of specular reflection will decrease to a level that is not captured by the camera, and the density of outlier points will become sparser as the distance moves away from the scanned plane. Specifically, the optical model relationship is:

[0083]

[0084] k o =tan(90°-θ)

[0085] Where θ is the angle between the CCD camera's field of view and the structured light; d o The length of the outlier plane; the platform displacement distance d p ;k o Let be the slope of the outlier plane.

[0086] It should be noted that the grayscale distribution information of the structured light is specifically obtained by extracting the grayscale centroid of the laser line when acquiring the surface information of the object being measured using the triangulation laser measurement method. This is achieved by capturing an image of the laser line scanning area using a CCD camera and then extracting the grayscale centroid of the laser line. In this process, since the laser beam emitted by the laser follows a Gaussian distribution, it is reflected in the image as the grayscale distribution of the beam. Specifically, this grayscale distribution uses black as the base color, with different saturations of black representing the image, white having a grayscale value of 0%, and black having a grayscale value of 100%.

[0087] Furthermore, in practice, due to the characteristics of lasers, the Gaussian distribution of structured light becomes more concentrated, making the actual gray-level distribution of line-structured light more linear. In this example, the gray-level distribution of line-structured light is considered to be a linear change. The gray levels of the line-structured light at times T1 and T2 are I1 and I2, respectively (obtained from the gray-level distribution diagram of the line-structured light). The relationship of the gray-level distribution of the line-structured light is:

[0088]

[0089] In the formula: I1 and I2 are the grayscale values ​​of the line structured light; k l The slope of the grayscale change in linear structured light.

[0090] Depend on Figure 7 As shown, we can obtain:

[0091]

[0092]

[0093] In the formula: d t d represents the distance between the maximum grayscale value of the line structured light and the grayscale value of the line structured light at time T2; l This is the distance between the end of the outlier plane and the real surface.

[0094] The distance d from the end of the outlier plane to the real surface can be obtained from the above formula. l :

[0095]

[0096] In some embodiments, please refer to Figure 7 Determining the position of the point cloud hole based on the error distance includes:

[0097] S701, Obtain the angle between the CCD camera's field of view and the structured light;

[0098] S702. Based on the plane perpendicular to the plane of the laser line and the field of view of the acquisition device, the coordinate transformation method is used to determine the true surface point cloud information according to the included angle.

[0099] In this embodiment, assuming the actual surface point cloud coordinates are P3(X3,Y3,Z3), then under this projection plane:

[0100] X3 = X1 + d l *sinθ

[0101] Y3=0

[0102] Z3=Z1+d l *cosθ

[0103] The actual surface point cloud information generated by this outlier plane is:

[0104] P3=(X3,Y1,Z3)∪(X3,Y2,Z3)∪(X3,Y3,Z3)…∪(X3,Y n ,Z3)

[0105] In the formula: Y1~Y n Let Y be the Y coordinates of all outliers on the outlier plane.

[0106] Furthermore, the distance from the ends of all outlier planes to the real surface is calculated using the method described above.

[0107] It should be noted that in this example, the angle θ between the CCD camera's field of view and the structured light is 60°; the length d of the outlier plane... o Extracted by point cloud visualization software; I1 and I2 can be extracted by image analysis software to obtain the structured light grayscale distribution of the cross section; I3 is the maximum value of the line structured light grayscale, which is 255.

[0108] Based on the above-described point cloud hole repair method, this invention also provides a corresponding point cloud hole repair device 800. Please refer to [link to device 800]. Figure 8 The point cloud hole repair device 800 includes an acquisition module 810, an outlier plane length determination module 820, an error distance determination module 830, and a point cloud hole location determination module 840, wherein:

[0109] The acquisition module 810 is used to acquire three-dimensional outliers near holes in the point cloud, wherein the three-dimensional outliers are noisy point clouds in which the point cloud information exists;

[0110] The outlier plane length determination module 820 is used to project the outlier plane formed by the outlier points located on the same plane to obtain two-dimensional outlier point cloud information, and determine the length of the outlier plane based on the two-dimensional outlier point cloud information.

[0111] Error distance determination module 830 is used to obtain the correlation between the outlier plane length and the line structured light intensity, and determine the error distance between the outlier plane and the surface where the point cloud hole is located based on the correlation.

[0112] The point cloud hole location determination module 840 is used to determine the location of the point cloud hole based on the error distance.

[0113] In this embodiment, three-dimensional outliers near point cloud holes are first acquired. These outliers are then projected onto corresponding two-dimensional planes to obtain the corresponding two-dimensional outlier point cloud information, thereby determining the length of the outlier plane. Based on the correlation between the outlier plane length and the intensity of the line structured light, the error distance between the outlier plane and the surface where the point cloud hole is located is determined, thus determining the location of the point cloud hole and repairing it. The point cloud hole repair method provided by this invention can effectively fill point cloud holes, effectively solving the problem of point cloud holes caused by mixed reflections during line structured light scanning. Furthermore, compared to other point cloud hole repair methods that mostly start from the edge of the point cloud hole and use deep learning and other methods to repair the hole from the outside in, where the repair effect deteriorates towards the center of the hole, this method greatly reduces the workload of subsequent point cloud repair. Moreover, by repairing the position information of outliers, the point cloud hole filling is achieved, greatly preserving the true shape of the object. This provides low-noise point cloud data with intact geometric features for subsequent registration, and offers better accuracy and robustness.

[0114] like Figure 9 As shown, based on the above-described point cloud hole repair method, the present invention also provides an electronic device, which can be a mobile terminal, desktop computer, laptop, handheld computer, server, or other computing device. The electronic device includes a processor 910, a memory 920, and a display 930. Figure 9 Only some components of the electronic device are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0115] In some embodiments, the memory 920 may be an internal storage unit of the electronic device, such as a hard disk or memory. In other embodiments, the memory 920 may be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Furthermore, the memory 920 may include both internal and external storage units. The memory 920 is used to store application software and various types of data installed on the electronic device, such as program code installed on the electronic device. The memory 920 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 920 stores a point cloud hole repair program 940, which can be executed by the processor 910 to implement the point cloud hole repair methods of the embodiments of this application.

[0116] In some embodiments, processor 910 may be a central processing unit (CPU), microprocessor or other data processing chip, used to run program code stored in memory 920 or process data, such as executing point cloud hole repair methods.

[0117] In some embodiments, display 930 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 930 is used to display information about the point cloud hole repair device and to display a user interface for visualization. Components 910-930 of the electronic device communicate with each other via a system bus.

[0118] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The storage medium can be a memory, magnetic disk, optical disk, etc.

[0119] The specific embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made in accordance with the technical concept of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for repairing point cloud holes, characterized in that, include: Obtain three-dimensional outliers near holes in the point cloud, wherein the three-dimensional outliers are noisy point clouds in which the point cloud information exists; The outlier plane formed by the three-dimensional outliers located on the same plane is projected to obtain two-dimensional outlier point cloud information, and the length of the outlier plane is determined based on the two-dimensional outlier point cloud information. The process involves obtaining the correlation between the length of the outlier plane and the intensity of the linear structured light, and determining the error distance between the outlier plane and the surface where the point cloud hole is located based on the correlation. This includes: determining the slope of the outlier plane based on the length of the outlier plane according to the principle of optical modeling; extracting the gray-scale centroid of the laser line to obtain the surface gray-scale distribution information of the object under test; determining the gray-scale change slope of the linear structured light based on the slope of the outlier plane and the surface gray-scale distribution information; and determining the error distance between the outlier plane and the surface where the point cloud hole is located based on the length of the outlier plane, the surface gray-scale distribution information, and the gray-scale change slope of the linear structured light. The error distance between the outlier plane and the surface where the point cloud holes are located can be expressed by the following formula: ,in, Let the length of the outlier plane be . For error distance, This refers to surface grayscale distribution information; Determining the location of the point cloud hole based on the error distance includes: obtaining the angle between the CCD camera's field of view and the structured light. Based on the plane perpendicular to the plane where the laser line is perpendicular to the field of view of the acquisition device, according to the included angle... The coordinate transformation method was used to determine the true surface point cloud information; Where the actual surface point cloud coordinates are P3(X3,Y3,Z3), then in Below the projection plane: ； The actual surface point cloud information generated by this outlier plane is: P3=(X3,Y1,Z3) (X3,Y2,Z3) (X3,Y3,Z3)… (X3,Y n ,Z3) In the formula: Y1~ Y n Let Y be the Y coordinates of all outliers on the outlier plane.

2. The point cloud hole repair method according to claim 1, characterized in that, Before acquiring the 3D outliers near the point cloud holes, the process also includes: Based on the line structured light 3D reconstruction method and image acquisition system, the 3D point cloud information of the surface of the object under test is obtained, and the location of the hole in the point cloud is marked. Based on the location of the point cloud holes, the three-dimensional outlier points are obtained.

3. The point cloud hole repair method according to claim 1, characterized in that, The projection of the outlier plane formed by the three-dimensional outliers located on the same plane specifically involves: The outlier plane is projected onto the plane perpendicular to the laser line and the field of view of the acquisition device; the projection of the outlier plane can be expressed by the following formula: in: In order to be in Projecting onto a plane; This is represented as outlier location information.

4. The point cloud hole repair method according to claim 1, characterized in that, Determining the length of the outlier plane based on the two-dimensional outlier point cloud information includes: Obtain the positions of the end point and the start point of the outlier plane; The length of the outlier plane is determined based on the positions of the endpoints and the starting point; the length of the outlier plane can be expressed by the following formula: in, This represents the length of the outlier plane, with the end point P1(X1,0,Z1) and the starting point P2(X2,0,Z2).

5. A point cloud hole repair device, used to implement the point cloud hole repair method as described in any one of claims 1-4, characterized in that, include: The acquisition module is used to acquire three-dimensional outliers near holes in the point cloud, wherein the three-dimensional outliers are noisy point clouds in which the point cloud information exists; The outlier plane length determination module is used to project the outlier plane formed by the three-dimensional outlier points located on the same plane to obtain two-dimensional outlier point cloud information, and determine the length of the outlier plane based on the two-dimensional outlier point cloud information. An error distance determination module is used to obtain the correlation between the outlier plane length and the line structured light intensity, and to determine the error distance between the outlier plane and the surface where the point cloud hole is located based on the correlation. The point cloud hole location determination module is used to determine the location of the point cloud hole based on the error distance.

6. An electronic device, characterized in that, include: Processor and memory; The memory stores a computer-readable program that can be executed by the processor; When the processor executes the computer-readable program, it implements the steps in the point cloud hole repair method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps in the point cloud hole repair method as described in any one of claims 1-4.

Images