Methods for identifying and removing interference points in structured light projection images, and methods and systems for 3D reconstruction based on structured light projection.

Abstract

This invention proposes a method for identifying and removing interference points in structured light projection images, a three-dimensional reconstruction method based on structured light projection, and a system thereof, relating to the technical field of digital image processing. In the proposed interference point identification and removal method, an interference point identification model is trained and used to identify interference points. Compared to threshold setting methods, this method offers faster identification speed and higher accuracy. Furthermore, interference points are removed through an overlay operation, filling in the gaps where interference points were originally located, which improves the accuracy of subsequent three-dimensional reconstruction and facilitates detailed study of the target object. The proposed three-dimensional reconstruction method and system improve the accuracy of target object feature analysis and are suitable for fields requiring high surface precision or for measuring the target object, avoiding damage caused by contact measurement.

Description

Technical Field

[0001] This invention relates to the field of digital image processing technology, and more specifically, to a method for identifying and removing interference points in structured light projection images, and a three-dimensional reconstruction method and system based on structured light projection. Background Technology

[0002] Structured light 3D measurement technology projects grating stripes onto the object being measured using a projector, which modulates the shape of the object to form measurement stripes. The camera then captures the images of the measurement stripes, decodes and calculates the phase, and finally uses epipolar constraint criteria and stereo vision technology to obtain the 3D data of the measured surface. The 3D data can then be used to reconstruct the 3D model of the object, providing powerful tools and technical support for scientific research, industrial design, and cultural heritage protection.

[0003] Due to the limitations of the optical imaging mechanism of projectors and cameras, as well as the influence of image noise, the raster stripe patterns acquired and captured by cameras and projectors inevitably contain invalid points such as shadows and backgrounds. These invalid points affect the quality of 3D data and further affect the 3D reconstruction of objects.

[0004] Currently, setting or calibrating thresholds can remove invalid points in images. For example, a method for extracting coal lattice fringes is disclosed in the prior art. In the extraction process, a series of image preprocessing is first performed, and then binarization is performed using the gray value of the fringes as a threshold to remove noise and invalid edge lattice fringes, ensuring the purity of the fringed image. However, in structured light projection applications, if the structured light projection image is subsequently used for 3D reconstruction, the image quality requirements are high. This method of removing invalid points by setting or calibrating thresholds may result in poor output image quality due to insufficient processing accuracy, failing to meet the requirements of 3D reconstruction. Summary of the Invention

[0005] To address the issue of low accuracy in traditional image interference removal methods, this invention proposes a method for interference identification and removal in structured light projection images. This method improves the accuracy of interference removal and ensures the purity of the structured light projection image. Furthermore, this invention proposes a three-dimensional reconstruction method and system based on structured light projection. This method performs three-dimensional reconstruction on high-quality structured light projection images, facilitating detailed study of target objects and providing tools and technical support for multiple fields.

[0006] To solve the above problems, the technical solution adopted in this application is as follows:

[0007] On the one hand, this application proposes a method for identifying and removing interference points in structured light projection images, including the following steps:

[0008] S1: Obtain a structured light projection sample image, which includes interference points, and label the interference points.

[0009] S2: Train the interference point recognition model using the labeled structured light projection sample images to obtain the trained interference point recognition model;

[0010] S3: Acquire a structured light stripe surface image of the target object to be identified;

[0011] S4: Input the structured light projection image to be identified into the trained interference point identification model, and output the label information of the structured light projection image to be identified, thereby identifying interference points;

[0012] S5: Analyze the feature data of the normal area of ​​the structured light projection image to be identified, excluding interference points. Based on the feature data of the normal area, perform a coverage operation on the interference points to remove them.

[0013] In this technical solution, the method of using a trained interference point recognition model to identify interference points is faster and more accurate than the threshold setting method. Furthermore, the method of removing interference points by covering them and filling in the blanks where the original interference points were located is beneficial for subsequent detailed study of the target object.

[0014] Preferably, after S3 and before S4, the method further includes: calibrating and correcting the received structured light projection image of the target object; performing phase analysis and frequency analysis on the light stripes in the structured light stripe surface image of the target object to be identified, and extracting the structured light coding information to facilitate accurate calculation of the three-dimensional coordinates of the object surface.

[0015] Preferably, the feature data includes: the horizontal gradient, vertical gradient, and pixel value at a pixel (x, y); the feature data of the normal region is extracted, and the extraction process satisfies:

[0016] G x (x,y)=H(x+1,y)-H(x-1,y)

[0017] G y (x,y)=H(x,y+1)-H(x,y-1)

[0018] Among them, G x (x,y), G y H(x,y) and H(x,y) represent the horizontal gradient, vertical gradient and pixel value at pixel (x,y), respectively.

[0019] Preferably, the process of removing interference points by covering them based on the feature data of the normal area includes:

[0020] Let the interference point pixel be (x h ,y l ), and the pixel values ​​(x) of the four neighboring points (up, down, left, and right) of the interfering pixel. h-1 ,y l ), (x h+1 ,y l ), (x h ,y l-1 ), (x h ,y l+1 Extract them separately;

[0021] The average value of the column pixels among the pixel values ​​of the four neighboring points (top, bottom, left, and right) is used as the column pixel of the interference point, and the average value of the row pixels among the pixel values ​​of the four neighboring points (top, bottom, left, and right) is used as the row pixel of the interference point. The pixel values ​​of the interference point are then overwritten to remove the interference point.

[0022] Secondly, this application proposes a three-dimensional reconstruction method based on structured light projection, including the following steps:

[0023] Acquire a structured light projection image of the target object;

[0024] The interference points in the structured light projection image are identified and removed using the aforementioned method for identifying and removing interference points in the structured light projection image of the target object, thereby obtaining a structured light projection image of the target object with interference points removed.

[0025] Extract data feature information from the structured light projection image of the target object after removing interference points;

[0026] Based on data feature information, the target object is reconstructed in three dimensions to generate a three-dimensional reconstruction model of the target object.

[0027] Preferably, the three-dimensional reconstruction method based on structured light projection further includes:

[0028] The structured light projection image of the target object is calibrated and corrected.

[0029] Phase analysis and frequency analysis are performed on the light stripes in the structured light projection image of the target object to extract the encoded information of the structured light.

[0030] Preferably, the three-dimensional reconstruction method based on structured light projection further includes: correcting the generated three-dimensional reconstruction model of the target object; and outputting two-dimensional images of the target object from various angles based on the corrected three-dimensional model.

[0031] The above technical solutions improve the output quality of structured light projection images.

[0032] Thirdly, this application also proposes a three-dimensional reconstruction system based on structured light projection, comprising:

[0033] The image acquisition module is used to acquire structured light projection images of the target object;

[0034] The image processing module is used to process the structured light projection image of the target object, identify and remove interference points, and obtain the structured light projection image of the target object after removing interference points.

[0035] The feature extraction module is used to extract data feature information from the structured light projection image of the target object after removing interference points;

[0036] The 3D reconstruction module performs 3D reconstruction of the target object based on data feature information, generating a 3D reconstruction model of the target object.

[0037] Preferably, the image processing module includes:

[0038] The image calibration and correction module receives the structured light projection image of the target object transmitted by the image acquisition module, and calibrates and corrects the received structured light projection image of the target object.

[0039] The analysis module is used to perform phase analysis and frequency analysis on the light stripes in the structured light projection image of the target object, and extract the encoded information of the structured light.

[0040] The interference point identification and removal module is used to identify and remove interference points in the structured light projection image of the target object.

[0041] Preferably, the three-dimensional reconstruction system based on structured light projection further includes:

[0042] The feedback correction module is used to correct the generated 3D reconstruction model of the target object.

[0043] The image output module is used to output two-dimensional images of the target object from various angles based on the corrected three-dimensional model.

[0044] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0045] This invention proposes a method for identifying and removing interference points in structured light projection images, a 3D reconstruction method and system based on structured light projection, and trains an interference point identification model. This model is used to identify interference points, offering faster identification speed and higher accuracy compared to threshold-based methods. Furthermore, interference points are removed through an overlay operation, filling in the gaps where they were originally located, which improves the accuracy of subsequent 3D reconstruction and facilitates detailed study of the target object. The proposed 3D reconstruction method and system enable non-contact acquisition of structured light projection images and extraction of image features from the target object. This effectively avoids contact errors caused by traditional measurement methods, improving the accuracy of target object feature analysis. This method is also applicable to fields requiring high surface precision or to the measurement of the target object during use, and it avoids damage to the target object caused by contact measurements. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating the structured light projection image interference point identification and removal method proposed in this embodiment of the invention.

[0047] Figure 2 This is a flowchart illustrating the three-dimensional reconstruction method based on structured light projection proposed in this embodiment of the invention.

[0048] Figure 3 This diagram illustrates the first composition of the three-dimensional reconstruction system based on structured light projection proposed in this embodiment of the invention.

[0049] Figure 4 This diagram illustrates a second composition of the three-dimensional reconstruction system based on structured light projection proposed in this embodiment of the invention.

[0050] Figure 5 This diagram illustrates a third composition of the three-dimensional reconstruction system based on structured light projection proposed in this embodiment of the invention. Detailed Implementation

[0051] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this application.

[0052] To better illustrate this embodiment, some parts of the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions;

[0053] It is understandable to those skilled in the art that some well-known details may be omitted from the accompanying drawings.

[0054] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments;

[0055] Example 1

[0056] like Figure 1As shown, this embodiment proposes a method for identifying and removing interference points in structured light projection images. (See [link to relevant documentation]). Figure 1 This includes the following steps:

[0057] S1: Obtain a structured light projection sample image, which includes interference points, and label the interference points.

[0058] S2: Train the interference point recognition model using the labeled structured light projection sample images to obtain the trained interference point recognition model;

[0059] S3: Acquire a structured light stripe surface image of the target object to be identified;

[0060] S4: Input the structured light projection image to be identified into the trained interference point identification model, and output the label information of the structured light projection image to be identified, thereby identifying interference points;

[0061] S5: Analyze the feature data of the normal area of ​​the structured light projection image to be identified, excluding interference points. Based on the feature data of the normal area, perform a coverage operation on the interference points to remove them.

[0062] In this embodiment, the structured light projection sample image refers to the measurement stripes formed by projecting grating stripes onto an object using a projector, modulating the object's surface, and then acquiring the measurement stripe image using an image acquisition module. Therefore, when acquiring the structured light projection sample image, an image acquisition device such as a camera can be used. To ensure the effectiveness of the subsequent interference point recognition model, the amount of structured light projection sample images acquired is sufficient. Then, interference points in the structured light projection sample images are labeled. Labeling can be done manually or using machine intelligence. When training the interference point recognition model, the label information carried by the structured light projection sample images also participates in the model training process. This enables the interference point recognition model to distinguish interference points after training, which can then be used for subsequent interference point recognition.

[0063] In this embodiment, the interference point identification model is a neural network model based on depthwise separable convolution, such as Xception or MobileNet. During the training process, if the identification accuracy of the interference point identification model reaches the preset accuracy, the training is terminated, and at this time, the trained interference point identification model is obtained.

[0064] This embodiment uses a trained interference point recognition model to identify interference points. Compared with the traditional method of setting or calibrating thresholds, this method is faster and more accurate. In addition, the overlay operation removes interference points and fills in the blanks where the original interference points were, which is beneficial for subsequent detailed study of the target object.

[0065] In this embodiment, after S3 and before S4, the method further includes: performing phase analysis and frequency analysis on the light stripes in the structured light stripe surface image of the target object to be identified, and extracting the structured light encoding information to facilitate the subsequent accurate calculation of the three-dimensional coordinates of the object surface.

[0066] In this embodiment, the normal region refers to the area in the structured light projection image to be identified, excluding interference points. After identification, interference points can be marked with bounding boxes, and then feature data extraction is performed. The feature data includes the horizontal gradient, vertical gradient, and pixel value at a pixel (x, y). The feature data of the normal region is extracted, and the extraction process satisfies the following conditions:

[0067] G x (x,y)=H(x+1,y)-H(x-1,y)

[0068] G y (x,y)=H(x,y+1)-H(x,y-1)

[0069] Among them, G x (x,y), G y H(x,y) and H(x,y) represent the horizontal gradient, vertical gradient and pixel value at pixel (x,y), respectively.

[0070] In this embodiment, the process of removing interference points by covering them based on the feature data of the normal area includes:

[0071] Let the interference point pixel be (x h ,y l ), and the pixel values ​​(x) of the four neighboring points (up, down, left, and right) of the interfering pixel. h-1 ,y l ), (x h+1 ,y l ), (x h ,y l-1 ), (x h ,y l+1 Extract them separately;

[0072] The average value of the column pixels among the pixel values ​​of the four neighboring points (top, bottom, left, and right) is used as the column pixel of the interference point, and the average value of the row pixels among the pixel values ​​of the four neighboring points (top, bottom, left, and right) is used as the row pixel of the interference point. The pixel values ​​of the interference point are then overwritten to remove the interference point.

[0073] Example 2

[0074] See Figure 2 This embodiment proposes a three-dimensional reconstruction method based on structured light projection, including the following steps:

[0075] SA: Acquire structured light projection image of the target object;

[0076] SB: The interference points in the structured light projection image of the target object are identified and removed using the interference point identification and removal method, so as to obtain the structured light projection image of the target object with interference points removed.

[0077] SC: Extracts data feature information from the structured light projection image of the target object after removing interference points;

[0078] SD: Based on data feature information, perform three-dimensional reconstruction of the target object and generate a three-dimensional reconstruction model of the target object.

[0079] In this embodiment, the three-dimensional reconstruction method based on structured light projection further includes:

[0080] After the SA acquires the structured light projection image of the target object, before performing the interference point identification and removal described in the SB, the structured light projection image of the target object is calibrated and corrected; and the light stripes in the structured light projection image of the target object are subjected to phase analysis and frequency analysis to extract the structured light coding information.

[0081] In this embodiment, the process of reconstructing the target object in three dimensions based on feature data includes:

[0082] Using optical geometry and triangulation, three-dimensional coordinate data of the target object's surface are obtained, and three-dimensional reconstruction is performed by combining the data feature information.

[0083] After generating the 3D reconstruction model, the 3D reconstruction method based on structured light projection also includes: correcting the generated 3D reconstruction model of the target object; and outputting 2D images of the target object from various angles based on the corrected 3D model, thereby improving the output quality of the structured light projection image.

[0084] Example 3

[0085] like Figure 3 As shown, this embodiment proposes a three-dimensional reconstruction system based on structured light projection, including:

[0086] The image acquisition module is used to acquire structured light projection images of the target object;

[0087] The image processing module is used to process the structured light projection image of the target object, identify and remove interference points, and obtain the structured light projection image of the target object after removing interference points.

[0088] The feature extraction module is used to extract data feature information from the structured light projection image of the target object after removing interference points;

[0089] The 3D reconstruction module performs 3D reconstruction of the target object based on data feature information, generating a 3D reconstruction model of the target object.

[0090] like Figure 4 As shown, in the 3D reconstruction system based on structured light projection proposed in this embodiment, the image processing module includes:

[0091] The image calibration and correction module receives the structured light projection image of the target object transmitted by the image acquisition module, and calibrates and corrects the received structured light projection image of the target object.

[0092] The analysis module is used to perform phase analysis and frequency analysis on the light stripes in the structured light projection image of the target object, and extract the encoded information of the structured light.

[0093] The interference point identification and removal module is used to identify and remove interference points in the structured light projection image of the target object.

[0094] like Figure 5 As shown, this embodiment also proposes a three-dimensional reconstruction system based on structured light projection, in addition to Figure 4 In addition to the structure shown, the 3D reconstruction system based on structured light projection also includes:

[0095] The feedback correction module is used to correct the generated 3D reconstruction model of the target object.

[0096] The image output module is used to output two-dimensional images of the target object from various angles based on the corrected three-dimensional model.

[0097] In this embodiment, the image acquisition module includes hardware components such as a camera and an optical configuration unit. The optical configuration unit is used to configure the device hardware parameters, adjust the projection light in real time to ensure that it hits the surface of the target object within the camera's field of view, and ensure the clarity and stability of the projected image at different angles and distances. The image acquisition module captures the structured light projection image of the target object. Its transmitting end is connected to the receiving end of the image calibration and correction module. The image calibration and correction module receives the structured light projection image of the target object transmitted by the image acquisition module, calibrates and corrects the received structured light projection image of the target object. The transmitting end of the image calibration and correction module is connected to the receiving end of the parsing module. The image calibration and correction module calibrates and corrects the received structured light projection image of the target object, and after the calibration process is completed, it performs correction processing on it using its internal and external parameters. The calibration process may include steps such as head distortion correction and adjustment of camera internal and external parameters to improve the accuracy and reliability of subsequent processing. The correction process calibrates the camera internal and external parameters to ensure that the structured light projection image has accurate spatial position and geometry in subsequent processing. The analysis module performs phase analysis and frequency analysis on the light stripes in the calibrated and corrected light projection image, extracting the structured light encoding information to facilitate the subsequent accurate calculation of the three-dimensional coordinates of the object surface.

[0098] The sending end of the parsing module is connected to the receiving end of the interference point identification and removal module. The interference point identification and removal module identifies and removes interference points in the structured light projection image of the target object, and obtains the structured light projection image of the target object with interference points removed, effectively removing invalid points and interference in the image.

[0099] The sending end of the interference point identification and removal module is connected to the receiving end of the feature extraction module. The feature extraction module extracts and separates data feature information from the image. This extraction utilizes complex image processing algorithms, such as morphological processing and segmentation, to extract key data feature information from the image. This data feature information can include local or global features such as edges, corners, and textures, which are used for subsequent 3D reconstruction. Before feature extraction, these images may undergo filtering enhancement or other preprocessing steps to improve the accuracy and stability of feature extraction.

[0100] The 3D reconstruction module performs 3D reconstruction of the target object based on data feature information and structured light encoding information, generating a 3D reconstructed model of the target object. Specifically, the 3D reconstruction module uses the data feature information extracted by the feature extraction module to associate and match it with the parameter information corresponding to the stereo image. The stereo image can be multiple viewpoint images captured by hardware devices such as sensors and cameras in the image acquisition module. Therefore, its parameter information includes the relative positions and angles between viewpoints, used to establish the depth and structure model of the scene. By comparing and analyzing the image features under different viewpoints, the depth information or parallax of each pixel in the scene is determined, thereby reconstructing the 3D structure of the target object.

[0101] The feedback correction module corrects the generated 3D reconstructed model of the target object. During correction, it replaces and optimizes the corresponding parameters of the 3D reconstructed model based on the real-time parameter information of the target object. This may involve adjusting parallax, depth information, or other key parameters to ensure the accuracy and consistency of the 3D reconstructed model. The 3D simulation model is updated in real time, integrating the optimized parameter information into the model to reflect the current state and changes of the environment.

[0102] The image output module outputs two-dimensional images of the target object from various angles after correction, based on the 3D model after correction.

[0103] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for identifying and removing interference points in structured light projection images, characterized in that, Includes the following steps: S1: Obtain a structured light projection sample image, which includes interference points, and label the interference points. S2: Train the interference point recognition model using the labeled structured light projection sample images to obtain the trained interference point recognition model; S3: Acquire a structured light stripe surface image of the target object to be identified; S4: Input the structured light projection image to be identified into the trained interference point identification model, and output the label information of the structured light projection image to be identified, thereby identifying interference points; S5: Analyze the feature data of the normal area of ​​the structured light projection image to be identified, excluding interference points. Based on the feature data of the normal area, perform a coverage operation on the interference points to remove them. The feature data includes: the horizontal gradient, vertical gradient, and pixel value at a given pixel (x, y); the feature data of the normal region is extracted, and the extraction process satisfies the following: in, , , Representing pixels ( x , y The horizontal gradient, vertical gradient, and pixel value at point ( ); based on the feature data of the normal region, the process of removing interference points by covering them includes: Let the number of interference points be ( ) x h , y l ), and the pixel values ​​of the four neighboring points (upper, lower, left, and right) of the interfering pixel. x h-1 , y l ), ( x h+1 , y l ), ( x h , y l-1 ), ( x h , y l+1 Extract them separately; The average value of the column pixels among the pixel values ​​of the four neighboring points (top, bottom, left, and right) is used as the column pixel of the interference point, and the average value of the row pixels among the pixel values ​​of the four neighboring points (top, bottom, left, and right) is used as the row pixel of the interference point. The pixel values ​​of the interference point are then overwritten to remove the interference point.

2. The method for identifying and removing interference points in structured light projection images according to claim 1, characterized in that, The process after S3 and before S4 includes: calibrating and correcting the received structured light projection image of the target object; performing phase analysis and frequency analysis on the light stripes in the structured light stripe surface image of the target object to be identified, and extracting the structured light coding information.

3. A three-dimensional reconstruction method based on structured light projection, characterized in that, Includes the following steps: Acquire a structured light projection image of the target object; The interference points of the structured light projection image as described in claim 1 or 2 are identified and removed from the structured light projection image of the target object to obtain a structured light projection image of the target object with interference points removed. Extract data feature information from the structured light projection image of the target object after removing interference points; Based on data feature information, the target object is reconstructed in three dimensions to generate a three-dimensional reconstruction model of the target object.

4. The three-dimensional reconstruction method based on structured light projection according to claim 3, characterized in that, Also includes: The structured light projection image of the target object is calibrated and corrected. Phase analysis and frequency analysis are performed on the light stripes in the structured light projection image of the target object to extract the encoded information of the structured light.

5. The three-dimensional reconstruction method based on structured light projection according to claim 4, characterized in that, Also includes: The generated 3D reconstruction model of the target object is then corrected. Based on the corrected 3D model, output 2D images of the target object from various angles.

6. A three-dimensional reconstruction system based on structured light projection, characterized in that, include: The image acquisition module is used to acquire structured light projection images of the target object; The image processing module is used to process the structured light projection image of the target object, identify and remove interference points using the structured light projection image interference point identification and removal method described in claim 1 or 2, and obtain the structured light projection image of the target object after removing interference points. The feature extraction module is used to extract data feature information from the structured light projection image of the target object after removing interference points; The 3D reconstruction module performs 3D reconstruction of the target object based on data feature information, generating a 3D reconstruction model of the target object.

7. The three-dimensional reconstruction system based on structured light projection according to claim 6, characterized in that, The image processing module includes: The image calibration and correction module receives the structured light projection image of the target object transmitted by the image acquisition module, and calibrates and corrects the received structured light projection image of the target object. The analysis module is used to perform phase analysis and frequency analysis on the light stripes in the structured light projection image of the target object, and extract the encoded information of the structured light. The interference point identification and removal module is used to identify and remove interference points in the structured light projection image of the target object.

8. The three-dimensional reconstruction system based on structured light projection according to claim 7, characterized in that, Also includes: The feedback correction module is used to correct the generated 3D reconstruction model of the target object. The image output module is used to output two-dimensional images of the target object from various angles based on the corrected three-dimensional model.

Images