Point cloud outlier elimination system and method for line structured light scanning

Abstract

The application discloses a point cloud outlier elimination system for line structured light scanning, and the grayscale image acquisition module collects multiple images of a measured object frame by frame, carries out filtering and denoising, image normalization and image grayscale preprocessing on each image of the measured object frame by frame, and obtains corresponding multiple two-dimensional grayscale images; the outlier elimination module converts each two-dimensional grayscale image into a binary image through a threshold filtering method, and eliminates a light spot area with a pixel less than a pixel threshold; the outlier elimination module eliminates a light spot area profile with a grayscale average value less than a grayscale threshold; the outlier elimination module calculates a ratio of a profile area of each light spot area to a convex point number of the light spot area, and eliminates two light spot area profiles with the maximum ratio. The application can solve the technical problem of point cloud outliers caused by strong reflection, structural reflection and error recognition of laser stripes in three-dimensional reconstruction of a reflection area in the prior art.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional vision technology, specifically to a system and method for removing outliers from point clouds using line structured light scanning. Background Technology

[0002] In current industrial production applications, point clouds are collections of point data on the surface of a product obtained through measuring instruments. For 3D reconstruction of objects, structured light triangulation is often used to rapidly acquire high-precision, massive amounts of point cloud data from the object's surface within a short time. However, during the data acquisition process, factors such as vibration of the acquisition equipment, inherent errors, the scanning environment, and the brightness or reflectivity of the object's surface inevitably cause the point clouds acquired by the structured light 3D scanning system's sensors to contain noise and anomalies. This results in the loss of correct location information and outliers, significantly impairing the image quality acquired by the imaging equipment, affecting the accuracy and completeness of the generated 3D model, and also significantly impacting subsequent surface reconstruction and point cloud classification. Summary of the Invention

[0003] The purpose of this invention is to provide a point cloud outlier removal system and method for line structured light scanning. This invention can solve the technical problem of point cloud outliers caused by strong reflection and structured reflection in the three-dimensional reconstruction of reflective areas in the prior art.

[0004] To achieve this objective, the point cloud outlier removal system designed for line structured light scanning in this invention is characterized by comprising a grayscale image acquisition module and an outlier removal module.

[0005] The grayscale image acquisition module is used to acquire multiple RGB images of the object under test frame by frame through the direct laser triangulation image acquisition system. Each RGB image of the object under test frame by frame is preprocessed by filtering and denoising, image normalization and image grayscale conversion to obtain the corresponding multiple two-dimensional grayscale images.

[0006] The outlier removal module is used to convert each two-dimensional grayscale image into a binary image through threshold filtering, obtain the spot regions in each binary image, and remove the spot regions with pixels smaller than the pixel threshold, thereby achieving the initial removal of abnormal spot regions in the spot region.

[0007] The outlier removal module uses a contour tracking algorithm to perform contour detection and recognition of the spot regions in each binary image after the initial removal of spot regions, and obtains the gray value of the spot region contour. For each spot region contour, the top A% of the pixels with the highest gray values ​​are selected, and the average gray value of the top A% of the pixels is calculated. Spot regions with average gray values ​​lower than the gray value threshold are removed, thus achieving further removal of abnormal spot regions in the spot region.

[0008] The outlier removal module traverses the outline of the light spot region in each binary image after further removal of the light spot region, obtains the area of ​​the light spot region outline and the number of convex points of the light spot region outline, calculates the ratio of the area of ​​each light spot region outline to the number of convex points of the light spot region outline, and removes the two light spot region outlines with the largest ratio, thereby realizing the further removal of abnormal light spot regions in the light spot region.

[0009] In this invention, the 3D point cloud model is established by calculating the spatial 3D coordinates of a series of 2D images using the gray-scale centroid method. If there are anomalous light spot regions in the 2D images (these regions are caused by strong reflections or structural reflections), they are reflected in the point cloud as outliers. Therefore, if the anomalous light spot regions in the 2D images are removed, outliers can be removed from the 3D point cloud.

[0010] The beneficial effects of this invention are:

[0011] Compared to other point cloud outlier removal methods, this method effectively removes outliers from point clouds, effectively addressing outliers caused by mixed reflections during line structured light scanning. Furthermore, unlike other outlier removal methods that rely on outlier identification algorithms and threshold filtering, which suffer from limited applicability and poor performance across various 3D point clouds as the object changes, this method offers a simpler and more universal approach to remove outliers from point cloud models of different scanned parts. It solves the problem of outlier generation in line structured light scanning systems for complex, smooth-surfaced objects. By processing frame-by-frame grayscale images acquired through line structured light scanning, irrelevant outliers are removed, significantly preserving the object's true shape. This provides low-noise point cloud data with intact geometric features for subsequent registration, exhibiting better accuracy and robustness. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the structure of the present invention;

[0013] Figure 2 This is a flowchart of the present invention;

[0014] Figure 3 It is based on the principle of direct laser triangulation;

[0015] Figure 4 This is a schematic diagram of the grayscale centroid method. Detailed Implementation

[0016] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0017] like Figure 1 and 2 The system shown is a point cloud outlier removal system for line structured light scanning, which includes a grayscale image acquisition module and an outlier removal module.

[0018] The grayscale image acquisition module is used to acquire multiple RGB images of the object under test frame by frame through a direct-fire laser triangulation image acquisition system. Each RGB image of the object under test frame is preprocessed by filtering and denoising, image normalization, and image grayscale conversion to obtain multiple corresponding two-dimensional grayscale images. The initial RGB images acquired by the camera may contain noise due to ambient light, platform motion, etc. Filtering and denoising can remove this noise during image preprocessing. However, the initial acquired images have a large resolution, occupying a large amount of memory, which is detrimental to the efficiency of subsequent algorithm processing. This invention reduces the image size through normalization and simplifies the image information through image grayscale conversion, facilitating subsequent processing.

[0019] The outlier removal module converts each 2D grayscale image into a binary image using threshold filtering. It then obtains the spot regions in each binary image (observing the binary image reveals the collected laser line stripe spots (normal spot regions) and the spots to be removed due to reflection (abnormal spot regions), both referred to as spot regions). Spot regions with pixels smaller than the pixel threshold (100) are removed, achieving initial removal of abnormal spot regions. Some images have obvious reflective areas, which can be observed in the binary image. By setting a threshold, these areas can be preliminarily processed. For images where the spot cannot be determined as a spot to be processed, the next step is performed. The 2D image regions generated by mixed reflection exhibit characteristics of mixed reflection and laser, including: 1. Most areas are small; 2. The overall average grayscale variance is large, and the average grayscale value is low; 3. The edges of the regions change drastically. This step removes some abnormal regions due to their small area.

[0020] The outlier removal module utilizes a contour tracking algorithm (opological structural analysis of digitized binary images by border following) to further detect and identify the contours of the light spot regions in each binary image after initial removal. It then obtains the grayscale values ​​of the light spot region contours, selects the top A% of pixels for each contour, and calculates the average grayscale value of these A% pixels. Contours of light spot regions with an average grayscale value below a grayscale threshold (200) are removed, further eliminating abnormal light spot regions (i.e., the image regions corresponding to outliers). These light spot regions are the set of normal light spot regions indicated by laser stripes and abnormal light spot regions indicated by mixed reflections in the image. This scheme is designed to address the characteristics of a large average grayscale variance and a low average grayscale value in the overall region. This process removes these abnormal light spot regions.

[0021] The outlier removal module iterates through the outline of the light spot region in each binary image after further removal, obtaining the area and number of convex points of the outline (the boundary formed by connecting the outermost pixels of each light spot in the image is the outline of the light spot, and the pixels at the inflection points of the boundary outline are the convex points; the number of inflections equals the number of convex points). It calculates the ratio of the outline area to the number of convex points for each light spot region, and removes the two light spot regions with the largest ratio, achieving further removal of the light spot regions. This step is designed to address the characteristics of a large average grayscale variance and a low average grayscale value in the overall region, thus removing this portion of abnormal light spot regions.

[0022] The above technical solution also includes a deep learning module. The deep learning module marks the contours of abnormal light spot regions in each binary image after the light spot regions are removed by manual marking. The deep learning model is used to learn and train the marked abnormal light spot region contours (the training set includes: 1. abnormal light spot regions with a pixel value less than the pixel threshold (100); 2. the top A% of the pixels in gray value of each light spot region contour, and the average gray value of the top A% of the pixels in gray value is calculated. The abnormal light spot regions with an average gray value lower than the gray value threshold (200); 3. the ratio of the area of ​​the light spot region contour to the number of convex points of the light spot region contour, and the two light spot regions with the largest ratio; 4. the manually marked abnormal light spot regions). The feature parameter model of the difficult-to-remove abnormal light spot regions is obtained. The feature parameter model is specifically the feature weights of the samples in the training set obtained by the deep learning network model, which are used for subsequent image recognition. This invention uses the identified abnormal light spot regions as a training set to generate a deep learning model through a deep learning image recognition network. The image recognition network then automatically removes abnormal light spot regions from a series of images, improving versatility and generalization ability, and enabling the removal of abnormal light spot regions missed in the previous steps. The feature parameters include the pixel values ​​of manually marked light spot region contours, the average grayscale value of the top A% of pixels in each light spot region contour, and the ratio of the light spot region contour area to the number of convex points in that contour.

[0023] The deep learning module utilizes the outlines of the removed spot regions and manually labeled spot regions to build a ResNet image recognition framework and create a training set. Based on the training set, it uses feature parameters to identify outlier spot regions in the outlines of the spot regions in each binary image after further spot region removal. Abnormal spot regions are removed from the spot regions to generate new image sequences. The image recognition network extracts features from abnormal spot regions, which is also applicable to other image sequences, removing abnormal spot regions to improve the method's versatility.

[0024] The above technical solution also includes a three-dimensional point cloud acquisition module, which is used to obtain a three-dimensional point cloud model of a new image sequence by removing outliers using the gray-scale centroid method.

[0025] In the above technical solution, the specific process of the direct laser triangulation measurement image acquisition system acquiring multiple RGB images of the object under test frame by frame is as follows: the laser is fixed at a position perpendicular to the visual moving platform, the laser beam is irradiated on the object under test placed on a unidirectional motion platform at a specific speed (currently set to 2.5cm / s), and the CCD camera at a certain angle to the laser acquires the surface information of the object under test through the triangulation laser measurement method, and finally generates the three-dimensional point cloud information of the surface of the object under test.

[0026] The direct-fire laser triangulation image acquisition system employs the principle of monocular structured light 3D measurement. Common monocular structured light scanning methods include direct-fire and oblique-fire methods. Both methods target the structured light beam. By extracting the center of the structured light stripe, the corresponding pixel coordinates are obtained. Combined with the corresponding camera parameters, triangulation is used to calculate the 3D coordinates of the corresponding point. This method is applied to all surface features on the workpiece to obtain all coordinates, acquiring the surface point cloud information of the object. This experimental environment uses a direct-fire laser triangulation image acquisition system. The following section details the principle of direct-fire laser triangulation.

[0027] Figure 3 This is a schematic diagram of a direct-light measurement method. In the diagram, a laser beam projected by a laser is perpendicularly incident on the outer surface of the object. The incident light undergoes diffuse reflection at a point on the surface, resulting in an image on the CCD photosensitive surface. The angle between the incident light l1l0 and the reflected light l0m0 is α. The angle between the reflected light l0m0 and the photosensitive surface m1m0 is β. Let the plane containing l0 represent the reference plane, where l1 is a point on the object surface, and m1 is the image point on the photosensitive surface formed by the lens. It can be seen from the diagram that points at different locations on the object surface have different positions on the imaging plane. The calculation of the three-dimensional spatial coordinates of points on the curved surface mainly utilizes the relevant parameters of the light plane and the camera to convert the pixel coordinates of the feature points at the location of the light beam into three-dimensional spatial coordinates. The specific formula is as follows:

[0028] Draw perpendicular lines l1c and m1d from l1 and m1 to l0m0 respectively. Based on the similarity of triangles Δl1cq and Δm1dq, the following formula can be obtained:

[0029]

[0030] Substituting the relevant parameters, we get:

[0031]

[0032] Further simplification yields:

[0033]

[0034] Where h represents the height of the object being measured from the laser point to the reference plane, x represents the pixel distance of the photosensitive plane, cq represents the line segment in triangle Δl1cq, and dq represents the line segment in triangle Δm1dq.

[0035] When using a direct-fire laser and a CCD camera to reconstruct the 3D structure of an object, this paper requires a 3D scanning instrument system based on line structured light. The main function of this system is to scan the 3D surface feature model of the object. Its workflow is as follows: the laser is fixed above a visual motion platform perpendicular to it; the CCD camera is positioned at the same horizontal level as the laser, at a certain angle; the object is placed on a unidirectional motion platform moving at a specific speed; the laser beam illuminates the object; and the surface information of the object is acquired using triangulation laser measurement to obtain the 3D point cloud of the object.

[0036] In the above technical solution, the specific method for preprocessing each frame of the RGB image of the object under test by filtering and denoising, image normalization, and image grayscale conversion is as follows:

[0037] For each frame of the RGB image of the object under test, noise is suppressed while preserving the image detail features by using adaptive median filtering, Gaussian filtering, bilateral filtering, and guided filtering algorithms.

[0038] The image normalization transforms the filtered image into a unique standard form. Its working principle is to use the invariance of the image to affine transformation to determine the parameters of the transformation function, and then use the transformation function determined by these parameters to transform the filtered image into a standard form.

[0039] Image grayscale conversion, specifically, is the process of converting a normalized image into a grayscale image. In a normalized image, the color of each pixel is determined by three components: R, G, and B. Each component has 255 possible values, resulting in a pixel having over 16 million (255*255*255) color variations. A grayscale image, on the other hand, is a special type of color image where all three R, G, and B components are equal, and its pixel has only 255 possible variations. Therefore, in digital image processing, images of various formats are generally converted to grayscale first to reduce the computational load in subsequent image processing. Like color images, grayscale images still reflect the overall and local distribution and characteristics of chromaticity and brightness levels within the entire image.

[0040] In the above technical solution, the contour detection and recognition specifically involves scanning line by line starting from the upper left corner of the binary image. When the following two situations are found, it is considered that the starting point of the boundary has been found.

[0041] row i:

[0042] G ij-1 =0; G ij =1 indicates that the outer boundary has been encountered;

[0043] G ij ≥1; G ij+1=0 indicates that a hole has been encountered, where G ij-1 G represents the value in the i-th row and (j-1)-th column of a binary image. ij G represents the value in the i-th row and j-th column of a binary image. ij+1 This represents the value in the i-th row and j+1-th column of the binary image;

[0044] This is used to determine whether it is the edge of the light spot area outline.

[0045] In the above technical solution, the specific method for obtaining a 3D point cloud model with outliers removed from a new image sequence using the gray-scale centroid method is as follows:

[0046] The gray-scale centroid method uses the weighted centroid coordinates of light intensity on each cross-section of the target region as the center point, such as... Figure 4 As shown, the maximum brightness Gmax is first obtained on each column cross section using the extreme value method, and a threshold K = Gmax - Δg is set. Δg is a threshold set according to the actual situation, generally 10-20. Pixels with a value greater than the threshold K are judged on both sides of the threshold K, and their centroid position is determined as the center of the laser stripe (the laser stripe is a strip-shaped area with a width in the image, and this area is regarded as a line by the gray-scale centroid method).

[0047] The formula for calculating the grayscale centroid using the grayscale centroid method is as follows:

[0048]

[0049]

[0050] Among them, u center v center Let U and V represent the centroids of mass in the u-direction (horizontal axis of the image) and v-direction (vertical axis of the image), respectively, and P(u, v) be the gray value of the pixel (u, v). It is the set of pixels in each row or column used to determine the centroid position. Combining this with the previous direct laser triangulation principle, assigning height information to this line allows it to be transformed into a point cloud model.

[0051] In the above technical solution, the deep learning module utilizes the outlines of the removed spot regions and the outlines of manually marked spot regions to build a ResNet image recognition framework, create a training set, and use feature parameters based on the training set to identify outlier spot regions in the outlines of the spot regions in each binary image after further spot region removal, thereby removing the spot regions and generating a new image sequence. The specific method is as follows:

[0052] Dataset creation: Extract and save each contour from all images individually, and in a preset order, generate a line of text for each contour image and write it to a text file. Each line of text consists of two parts: the first part is the path of the contour image, and the second part is the label. Setting 0 means that this contour is a reflective spot and needs to be removed, while 1 means that it is a normal laser line and needs to be retained.

[0053] Model training: Each contour image and its corresponding label are read line by line from the generated text file and loaded into the network model (an untrained ResNet18) for forward propagation in deep learning. This yields a trained parametric model. The parametric model calculates a judgment value, compares it with the label value, and calculates the gradient based on the difference. The weights of the feature parameters in the parametric model are then updated (faster gradient descent results in larger weights). This process is repeated n times (n is set to 50 in this method). The model with the smallest difference (i.e., the smallest error) is saved, which is the optimal parametric model.

[0054] Abnormal light spot region identification: Taking one image as an example, each region in the image is labeled with a sequence number and then extracted individually. The extracted contour images are transformed to a size that the network model can recognize, and then fed into a trained parametric model. The parametric model determines whether each contour image needs to be removed and outputs a judgment value (the parametric model is actually the weight of different features. After the image is passed in, the weights are used to calculate which feature the image is most similar to. The feature parameters are different each time they are trained, and the judgment probability is also different each time. The network calculates the probability of an outlier contour through the parametric model). Finally, based on the corresponding sequence number and judgment value, a masking operation is performed to obtain the image of the light spot to be removed. The above operation is performed on all images in sequence to generate a new image sequence of light spot regions.

[0055] A method for removing outliers from point clouds using line structured light scanning, comprising the following steps:

[0056] Step 1: Acquire multiple RGB images of the object under test frame by frame using a direct laser triangulation image acquisition system. Perform preprocessing such as filtering and denoising, image normalization, and image grayscale conversion on each RGB image of the object under test frame by frame to obtain multiple corresponding two-dimensional grayscale images.

[0057] Step 2: Convert each two-dimensional grayscale image into a binary image using a threshold filtering method, obtain the spot regions in each binary image, and remove spot regions with pixels smaller than the pixel threshold to achieve preliminary removal of spot regions;

[0058] After the initial removal of the spot regions, the contour tracking algorithm is used to further detect and identify the contours of the spot regions in each binary image. The coordinates of the contours of the spot regions are obtained (in order to determine the sequence number of each contour and avoid confusion after sorting) and the gray value. For each spot region contour, the top A% of the pixels with the highest gray values ​​are selected, and the average gray value of the top A% of the pixels is calculated. Spot regions with average gray values ​​lower than the gray value threshold are removed to achieve further removal of the spot regions.

[0059] Traverse the outline of the light spot region in each binary image after further removal of the light spot region, obtain the area of ​​the light spot region outline and the number of convex points of the light spot region outline, calculate the ratio of the area of ​​each light spot region outline to the number of convex points of the light spot region outline, and remove the two light spot region outlines with the largest ratio to achieve the second removal of the light spot region.

[0060] Step 3: Manually label the contours of the light spot regions in each binary image after further removal. Then, use a deep learning model to train the labeled contours to obtain a feature parameter model for the difficult-to-remove light spot regions. Specifically, this feature parameter model is a feature weight model of the samples in the training set obtained through the deep learning network model, used for subsequent image recognition. The model training process is as follows: each contour image and its corresponding label are read line by line from the generated text file, loaded into the network model for computation, and the model calculates a judgment value. The judgment value is compared with the label value, and the gradient is calculated based on this difference to update the feature parameters of each layer in the model. This process is repeated n times (n is set to 50 in this method), and the model with the smallest difference (i.e., the smallest error) is saved as the optimal parameter model.

[0061] Step 4: Use the outlines of the removed spot regions and the outlines of the manually marked spot regions to create a training set. Train the ResNet image recognition framework to generate a weight parameter model. Based on the feature parameter model, identify outlier spot regions in the outlines of the spot regions in each binary image after removing the spot regions again. Remove the spot regions to generate a new image sequence.

[0062] Dataset creation: Extract and save each contour from all images individually, and in a preset order, generate a line of text for each contour image and write it to a text file. Each line of text consists of two parts: the first part is the path of the contour image, and the second part is the label. Setting 0 means that this contour is a reflective spot and needs to be removed, while 1 means that it is a normal laser line and needs to be retained.

[0063] Model training: Each contour image and its corresponding label are read line by line from the generated text file and loaded into the network model (an untrained ResNet18) for forward propagation in deep learning. This yields a trained parametric model. The parametric model calculates a judgment value, compares it with the label value, and calculates the gradient based on the difference. The weights of the feature parameters in the parametric model are then updated (faster gradient descent results in larger weights). This process is repeated n times (n is set to 50 in this method). The model with the smallest difference (i.e., the smallest error) is saved, which is the optimal parametric model.

[0064] Abnormal light spot region identification: Taking one image as an example, each region in the image is marked with a sequence number and then extracted individually. The extracted contour images are transformed to a size that the network model can recognize, and then fed into a trained parametric model. The parametric model determines whether each contour image needs to be removed and outputs a judgment value (the parametric model is actually the weight of different features. After the image is passed in, the weight is used to calculate which feature the image is most similar to. The feature parameters are different each time they are trained, and the judgment probability is also different each time. The network calculates the probability of an outlier contour through the parametric model). Finally, based on the corresponding sequence number and judgment value, a masking operation is performed to obtain the image of the light spot to be removed. The above operation is performed on all images in sequence to generate a new image sequence of light spot regions.

[0065] Step 5: Used to obtain a 3D point cloud model of the new image sequence by removing outliers using the gray-scale centroid method.

[0066] In step 1 of the above technical solution, a direct-fire laser triangulation image acquisition system is used. The CCD camera is a C5-2040-GigE high-speed 3D camera with a frame rate of 5. Gaussian filtering and grayscale processing are enabled to acquire frame-by-frame images of the scanned object. The acquired images are then filtered and denoised using the camera's built-in Gaussian filtering algorithm (the filtering method can be determined based on the specific factors of the research object).

[0067] The formula for the Gaussian function is as follows:

[0068] G(x, y)=1 / (2πσ^2)exp{-(x^2+y^2) / (2σ^2)}

[0069] Where x and y are the horizontal and vertical coordinates, G(x, y) represents the weight at x and y, σ^2 is a set parameter that is set according to different situations. Here it is set to 1.0, x^2 represents the square of x, and y^2 represents the square of y.

[0070] The filtered and denoised image is then normalized, specifically by converting it into a standard image of the same form using a linear function transformation, as shown in the following expression:

[0071]

[0072] Where, x i The values ​​represent the pixel values ​​of the image. max(x) and min(x) represent the maximum and minimum values ​​of the image pixels, respectively. x′ represents the normalized pixel value.

[0073] The image is converted to grayscale using the camera's built-in algorithm. Specifically, a weighted average method is used, which calculates the weighted average of the three RGB components in the image based on their importance and indices, as shown in the following expression.

[0074] Gray(i,j)=0.299*R(i,j)+0.578*G(i,j)+0.114*B(i,j):

[0075] Where Gray(i,j) represents the gray value at position i,j in the image, R(i,j) represents the red component value at position i,j in the image, G(i,j) represents the green component value at position i,j in the image, and B(i,j) represents the blue component value at position i,j in the image.

[0076] In step 2 of the above technical solution, the grayscale image is converted into a binary image by threshold filtering. Specifically, the threshold for the grayscale value of the image is set to 160, the grayscale value of pixels with a grayscale value lower than 160 is set to zero, and the grayscale value of pixels with a grayscale value higher than 160 is set to 255.

[0077] In step 2 of the above technical solution, the contour of the image spot region is detected by a contour tracking algorithm, and the contour of the spot region with less than 100 pixels is removed. Specifically:

[0078] S1 = SS n

[0079] Among them, S n S represents the outline with a pixel count less than 100, S represents the outline of the entire image, and S1 represents the remaining outline after removing irrelevant outlines.

[0080] In step 2 of the above technical solution, the grayscale value of the light spot region contour is obtained. For each light spot region contour, the top 10% of the pixels with the highest grayscale values ​​are selected, and the average grayscale value of the top 10% of the pixels is calculated. Light spot region contours with an average grayscale value lower than 200 are discarded. The specific expression is as follows:

[0081]

[0082] Where P is the gray value of the pixel, P i(grey) represents the grayscale value of the current i-th pixel. It is the average grayscale value of the top 10% of the pixels.

[0083]

[0084] Note: S1 is the outline of the spot region where the average gray value of the top 10% of pixels is less than 200. S2 is the spot region that is retained, and S1 is the spot region retained in the previous step.

[0085] The outlines of the light spot regions in each binary image after further removal are traversed, and the area and number of convex points of each light spot region outline are obtained. The ratio of the area of ​​each light spot region outline to the number of convex points of that light spot region outline is calculated, and the two light spot region outlines with the largest ratio are removed. The specific expression is as follows:

[0086] B = L / T

[0087] Note: L is the contour area, T is the number of protrusions in the contour, and B is the ratio of the contour area to the number of protrusions in the contour.

[0088] S3 = S2 - S B

[0089] Among them, S B S3 is the region defined by the ratio of the outline area of ​​each spot region to the number of convex points of that spot region outline. S2 is the spot region that is retained in the previous step.

[0090] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

Claims

1. A point cloud outlier removal system for line structured light scanning, characterized in that, It includes a grayscale image acquisition module, an outlier removal module, a deep learning module, and a 3D point cloud acquisition module; The grayscale image acquisition module is used to acquire multiple RGB images of the object under test frame by frame through the direct laser triangulation image acquisition system. Each RGB image of the object under test frame by frame is preprocessed by filtering and denoising, image normalization and image grayscale conversion to obtain the corresponding multiple two-dimensional grayscale images. The outlier removal module is used to convert each two-dimensional grayscale image into a binary image through threshold filtering, obtain the spot regions in each binary image, and remove the spot regions with pixels smaller than the pixel threshold to achieve the initial removal of spot regions. The outlier removal module uses a contour tracking algorithm to perform contour detection and recognition of the light spot region in each binary image after the initial removal of the light spot region, and obtains the gray value of the light spot region contour. For each light spot region contour, the top A% of the pixels with the highest gray value are selected, and the average gray value of the top A% of the pixels with the highest gray value is calculated. The light spot region contours with an average gray value lower than the gray value threshold are removed to achieve further removal of the light spot region. The outlier removal module traverses the outline of the light spot region in each binary image after further removal of the light spot region, obtains the area of ​​the light spot region outline and the number of convex points of the light spot region outline, calculates the ratio of the area of ​​each light spot region outline to the number of convex points of the light spot region outline, and removes the two light spot region outlines with the largest ratio to achieve the second removal of the light spot region. The deep learning module manually marks the outlines of the light spot regions in each binary image after further removal of the light spot regions. It then uses a deep learning model to train on the marked light spot region outlines to obtain feature parameters of the light spot regions that are difficult to remove. The deep learning module uses the removed light spot region outlines and the manually marked light spot region outlines to build a ResNet image recognition framework, creates a training set, and uses the feature parameters to identify outlier light spot regions in the outlines of the light spot regions in each binary image after further removal of the light spot regions, removing the light spot regions and generating a new image sequence. The 3D point cloud acquisition module is used to obtain a 3D point cloud model of a new image sequence by removing outliers using the gray-scale centroid method.

2. The point cloud outlier removal system for line structured light scanning according to claim 1, characterized in that: The specific process of the direct-fire laser triangulation image acquisition system acquiring multiple RGB images of the object under test frame by frame is as follows: the laser is fixed at a position perpendicular to the visual moving platform, the laser beam is irradiated on the object under test placed on a unidirectional motion platform at a specific speed, and a CCD camera at a certain angle to the laser acquires the surface information of the object under test through triangulation laser measurement method, and finally generates three-dimensional point cloud information of the surface of the object under test.

3. The point cloud outlier removal system for line structured light scanning according to claim 1, characterized in that: The specific method for preprocessing each frame of the RGB image of the object under test, including filtering and denoising, image normalization, and image grayscale conversion, is as follows: For each frame of the RGB image of the object under test, noise is suppressed by applying adaptive median filtering, Gaussian filtering, bilateral filtering, and guided filtering algorithms while preserving image detail features. The image normalization transforms the filtered image into a unique standard form. Its working principle is to use the invariance of the image to affine transformation to determine the parameters of the transformation function, and then use the transformation function determined by these parameters to transform the filtered image into a standard form. Image grayscale conversion is specifically the process of converting a normalized image into a grayscale image. The color of each pixel in a normalized image is determined by three components: R, G, and B. Each component has 255 possible values. A grayscale image is a special type of color image where the three R, G, and B components are the same. The description of a grayscale image is the same as that of a color image, still reflecting the overall and local distribution and characteristics of the color and brightness levels of the entire image.

4. The point cloud outlier removal system for line structured light scanning according to claim 1, characterized in that: The contour detection and recognition process involves scanning line by line starting from the top left corner of the binary image. When the following two situations are found, the starting point of the boundary is considered to have been found. row i: This indicates that the outer boundary has been encountered; , indicating encountering a hole, where G ij-1 G represents the value in the i-th row and (j-1)-th column of a binary image. ij G represents the value in the i-th row and j-th column of a binary image. ij+1 This represents the value in the i-th row and j+1-th column of the binary image; This is used to determine whether it is the edge of the light spot area outline.

5. The point cloud outlier removal system for line structured light scanning according to claim 1, characterized in that: The specific method for obtaining a 3D point cloud model with outliers removed from a new image sequence using the gray-scale centroid method is as follows: The gray-scale centroid method uses the light intensity weight centroid coordinates on each column cross section as the center point in the target area. First, the maximum brightness Gmax is obtained on each column cross section using the extreme value method, and a threshold K = Gmax - Δg is set. Δg is a threshold set according to the actual situation. Pixels with values ​​greater than the threshold K are judged on both sides of the threshold K, and their centroid positions are obtained as the center of the laser stripe. The formula for calculating the grayscale centroid using the grayscale centroid method is as follows: in, , These represent the horizontal axis (u direction) and the vertical axis (v direction) of the image, respectively. For pixels grayscale value, It is a set of pixels in each row or column used to determine the centroid position. Combining this with the previous direct laser triangulation principle, the line is given height information, which can then be converted into a point cloud model.

6. The point cloud outlier removal system for line structured light scanning according to claim 1, characterized in that: The deep learning module utilizes the outlines of the removed spot regions and manually marked spot regions to build a ResNet image recognition framework, creating a training set. Based on the training set, it uses feature parameters to identify outlier spot regions in the outlines of the spot regions in each binary image after further spot region removal. The specific method for removing spot regions and generating a new image sequence is as follows: Dataset creation: Extract and save each contour from all images individually, and in a preset order, generate a line of text for each contour image and write it to a text file. Each line of text consists of two parts: the first part is the path of the contour image, and the second part is the label. 0 represents that the contour is a reflective spot and needs to be removed, while 1 represents a normal laser line and needs to be retained. Model training: Read each contour image and its corresponding label line by line from the generated text file, load them into the network model for forward propagation in deep learning, and obtain the trained parameter model. The parameter model calculates the judgment value, compares the judgment value with the label value, calculates the gradient based on the difference, and updates the weights of the feature parameters in the parameter model. Abnormal light spot region identification: Each region in the image is marked with a number and then extracted individually. The extracted contour images are transformed into a format that the network model can recognize, and then fed into a trained parameter model. The parameter model determines whether each contour image needs to be removed and outputs a judgment value. Finally, based on the corresponding sequence number and judgment value, the image of the removed light spot is obtained through a masking operation. The above operation is performed on all images in sequence to obtain a new image sequence of light spot regions.

7. A method for removing outliers from point clouds using line structured light scanning, characterized in that, It includes the following steps: Step 1: Acquire multiple RGB images of the object under test frame by frame using a direct laser triangulation image acquisition system. Perform preprocessing such as filtering and denoising, image normalization, and image grayscale conversion on each RGB image of the object under test frame by frame to obtain multiple corresponding two-dimensional grayscale images. Step 2: Convert each two-dimensional grayscale image into a binary image using a threshold filtering method, obtain the spot regions in each binary image, and remove spot regions with pixels smaller than the pixel threshold to achieve preliminary removal of spot regions; After the initial removal of the spot region, the contour tracking algorithm is used to further detect and identify the contour of the spot region in each binary image, and obtain the gray value of the spot region contour. For each spot region contour, the top A% of the pixels with the highest gray value are selected, and the average gray value of the top A% of the pixels is calculated. Spot regions with an average gray value lower than the gray value threshold are removed to achieve further removal of the spot region. Traverse the outline of the light spot region in each binary image after further removal of the light spot region, obtain the area of ​​the light spot region outline and the number of convex points of the light spot region outline, calculate the ratio of the area of ​​each light spot region outline to the number of convex points of the light spot region outline, and remove the two light spot region outlines with the largest ratio to achieve the second removal of the light spot region. Step 3: Manually mark the outline of the light spot region in each binary image after the light spot region is removed again. Then, use a deep learning model to learn and train the marked light spot region outline to obtain the feature parameters of the light spot region that is difficult to remove. The feature parameters include the pixel value of the manually marked light spot region outline, the average gray value of the first A% of the pixels of each manually marked light spot region outline, and the ratio of the area of ​​the manually marked light spot region outline to the number of convex points of the light spot region outline. Step 4: Using the outlines of the removed spot regions and the outlines of the manually marked spot regions, a ResNet image recognition framework is built to create a training set. Based on the training set, feature parameters are used to identify outlier spot regions in the outlines of the spot regions in each binary image after the spot regions are removed again. The spot regions are then removed to generate a new image sequence. Step 5: Used to obtain a 3D point cloud model of the new image sequence by removing outliers using the gray-scale centroid method.

Images