A method and system for 3D reconstruction of mechanical parts based on structured light technology

Abstract

This invention provides a method and system for 3D reconstruction of mechanical parts based on structured light technology, belonging to the field of 3D reconstruction technology for mechanical parts. The method includes: jointly calibrating a camera and projector based on Gray code images to obtain the intrinsic and extrinsic parameter matrices and distortion coefficients of the camera and projector; using a four-step phase-shifting method combined with Gray code images to solve for the absolute phase of the object under test to obtain the absolute phase map of the object under test; and performing matrix transformation on the absolute phase map of the object under test using the intrinsic and extrinsic parameter matrices of the camera and projector to draw a 3D point cloud map of the object under test. This invention, by combining Gray code, four-step phase-shifting method and other technologies to perform 3D modeling of the object under test, lays the foundation for subsequent 3D defect detection and solves the problem that ADC system models cannot detect corresponding defects because 2D images cannot present defect features such as depressions and protrusions with associated heights.

Description

Technical Field

[0001] This invention relates to the field of 3D reconstruction technology for mechanical parts, and more specifically, to a method and system for 3D reconstruction of mechanical parts based on structured light technology. Background Technology

[0002] Mechanical parts can develop various defects during different stages of manufacturing. Currently, many mechanical parts manufacturers have introduced intelligent defect detection systems such as AOI (Automatic Optical Inspection) and ADC (Automatic Defect Classification) to replace manual inspection of parts, achieving good results in actual production.

[0003] ADC systems primarily employ deep learning for defect detection, mainly based on 2D defect images for modeling. However, in actual production lines, the parts under test are three-dimensional objects, and the captured 2D images lack their height information. Consequently, the 2D images cannot represent defect features with associated heights, such as pits and protrusions, and therefore the ADC system model cannot detect the corresponding defects. Summary of the Invention

[0004] This invention provides a method and system for 3D reconstruction of mechanical parts based on structured light technology, which solves the problem that ADC system models cannot detect corresponding defects because 2D images cannot present defect features such as depressions and protrusions with associated heights.

[0005] In a first aspect, embodiments of the present invention provide a method for 3D reconstruction of mechanical parts based on structured light technology, the method comprising the following steps:

[0006] The camera and projector are jointly calibrated based on Gray code images to obtain the intrinsic and extrinsic parameter matrices and distortion coefficients of the camera and projector.

[0007] A four-step phase-shifting method combining Gray code images is used to solve for the absolute phase of the object under test in order to obtain the absolute phase map of the object under test.

[0008] The absolute phase map of the test object is transformed by combining the intrinsic and extrinsic parameter matrices of the camera and projector to draw a 3D point cloud map of the test object, and the distortion coefficients of the camera and projector are used to correct the 3D point cloud map of the test object.

[0009] In the above embodiments, the present invention first generates and projects horizontal and vertical positive and negative Gray codes onto a checkerboard calibration board, and acquires four sets of different projected images by changing positions. The Zhang Zhengyou calibration method is used to jointly calibrate the camera and projector, solving the problem that the 2D camera calibration scheme cannot calibrate the projector. Then, vertical Gray codes and a four-step cosine grating image are generated and projected onto the object under test, and corresponding projected images are acquired. The phase of each point in the image is solved and unfolded using the four-step phase-shifting method to obtain the absolute phase map of the image. This solves the problem of point-by-point error accumulation and phase unfolding errors caused by discontinuities on the surface of the object under test in the classic spatial phase unfolding algorithm. Finally, combining the intrinsic and extrinsic parameter matrices of the camera and projector with the absolute phase map of the image, the three-dimensional spatial coordinates corresponding to the pixel coordinates of each point in the image are calculated through matrix transformation to obtain a 3D point cloud map of the object under test. This solves the problem that 2D images cannot present some defect feature information, laying the foundation for subsequent 3D defect detection.

[0010] As some optional implementations of this application, the process of jointly calibrating the camera and projector based on Gray code images to obtain the intrinsic and extrinsic parameter matrices and distortion coefficients of the camera and projector is as follows:

[0011] Place the checkerboard calibration board within the field of view of the camera and projector, and generate a Gray code image;

[0012] Record the three-dimensional spatial coordinates of each corner point of the checkerboard calibration board at different positions. Project Gray code images, all-white images, and all-black images onto the checkerboard calibration board at different positions using a projector. Also, capture images of the checkerboard calibration board at different positions using a camera to obtain multiple sets of checkerboard images.

[0013] For each set of chessboard images, a corner detection algorithm is used to detect each corner of the chessboard calibration board in the chessboard image projected from a completely white image, so as to obtain the camera image coordinates of each corner.

[0014] For each set of chessboard images, a local homography matrix transformation is performed on each corner point of the chessboard calibration board in the chessboard image projected from the Gray code image to obtain the projector image coordinates of each corner point.

[0015] Based on the three-dimensional spatial coordinates of each corner point at multiple different locations, the camera image coordinates, and the projector image coordinates, the Zhang Zhengyou calibration method is used to calibrate the camera and the projector respectively, so as to obtain the intrinsic and extrinsic parameter matrices and distortion coefficients of the camera and the projector.

[0016] In the above embodiments, the present invention uses Zhang Zhengyou calibration method based on image projection to achieve joint calibration of camera and projector, which solves the problem that the camera 2D calibration scheme cannot calibrate the projector.

[0017] As some optional implementations of this application, the Gray code images generated when jointly calibrating the camera and projector based on the Gray code images include horizontal and vertical positive and negative Gray code images.

[0018] In the above embodiments, the projection of Gray code images can be combined with the regularity of Gray code images to perform joint calibration of the camera and projector, thereby improving calibration accuracy.

[0019] As some optional embodiments of this application, the absolute phase of the object under test is solved by a four-step phase-shifting method combined with Gray code images to obtain the absolute phase map of the object under test. The process is as follows:

[0020] Generate Gray code images and four-step cosine grating images;

[0021] When the joint calibration of the camera and projector is completed, keep the position of the checkerboard calibration plate unchanged, and place the object to be tested on the surface of the checkerboard calibration plate.

[0022] Gray code images and four-step cosine grating images are projected onto the object under test using a projector, and images of the checkerboard calibration board are acquired using a camera to obtain multiple Gray code images and multiple four-step cosine grating images captured by the camera.

[0023] For four-step cosine grating images captured by multiple cameras, a four-step phase-shifting method is used to solve the relative phase of the object under test in order to obtain the relative phase map of the object under test.

[0024] For Gray code images captured by multiple cameras, the cosine grating period of each point of the test object in the relative phase map is determined by using the binarized Gray code images, and the absolute phase map of the test object is obtained by restoring the cosine grating period.

[0025] In the above embodiments, the present invention uses the regularity of Gray code and cosine grating, as well as phase interpretability, to solve for the relative and absolute phases of the object under test, that is, it uses structured light technology to realize 3D reconstruction.

[0026] As some optional embodiments of this application, the Gray code image generated when the absolute phase of the object under test is solved using the four-step phase shift method combined with Gray code images includes a vertical Gray code image.

[0027] In the above embodiments, the image patterns of the vertical Gray code image and the four-step cosine grating image are similar, so the phase of the object under test can be solved based on the phase pattern of the two.

[0028] As some optional embodiments of this application, the absolute phase map of the object under test is transformed by combining the intrinsic and extrinsic parameter matrices of the camera and projector to draw a 3D point cloud map of the object under test, and the process of correcting the 3D point cloud map of the object under test by combining the distortion coefficients of the camera and projector is as follows:

[0029] For the absolute phase map of the object under test, the pixel coordinates of each point of the object under test in the projector image are calculated based on the phase invariance assumption and the cosine grating period.

[0030] Based on the pixel coordinates of each point of the object under test in the phase image and the corresponding pixel coordinates in the projector image, and combined with the intrinsic and extrinsic parameter matrices of the camera and the projector, the three-dimensional spatial coordinates of each point in the camera image are calculated by rectangular transformation.

[0031] A 3D point cloud map of the object under test is drawn based on three-dimensional coordinates, and the distortion coefficients of the camera and projector are used to correct the 3D point cloud map of the object under test.

[0032] In the above embodiments, the present invention, based on the relative phase map, combines the phase assumption of the cosine grating and the intrinsic and extrinsic parameter matrices of the camera and projector, and uses rectangular transformation calculation to solve the absolute phase map.

[0033] As one of the optional embodiments of this application, the object to be tested is a mechanical part.

[0034] In a second aspect, the present invention provides a 3D reconstruction system for mechanical parts based on structured light technology, the system comprising:

[0035] A joint calibration unit performs joint calibration of the camera and projector based on Gray code images to obtain the intrinsic and extrinsic parameter matrices and distortion coefficients of the camera and projector.

[0036] An absolute phase solving unit is used to solve the absolute phase of the test object by employing a four-step phase shift method combined with Gray code images to obtain the absolute phase map of the test object.

[0037] The 3D reconstruction unit combines the intrinsic and extrinsic parameter matrices of the camera and projector to perform matrix transformation on the absolute phase map of the object under test to draw a 3D point cloud map of the object under test, and combines the distortion coefficients of the camera and projector to correct the 3D point cloud map of the object under test.

[0038] In a third aspect, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor performing the aforementioned method for 3D reconstruction of mechanical parts based on structured light technology.

[0039] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for 3D reconstruction of mechanical parts based on structured light technology.

[0040] The beneficial effects of this invention are as follows: This invention combines Gray code, four-step phase shift method and other technologies to perform 3D modeling of the test object, laying the foundation for subsequent 3D defect detection, and solving the problem that the ADC system model cannot detect the corresponding defects because the defect features such as depressions and protrusions with associated heights cannot be presented in 2D images. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart of a 3D reconstruction method for mechanical parts based on structured light technology;

[0043] Figure 2 It is a schematic diagram of 22 vertical forward and reverse Gray code images of 11 bits;

[0044] Figure 3 These are schematic diagrams of 24 images of 11-bit Gray code, including positive and negative Gray code images, all-white Gray code images, and all-black Gray code images.

[0045] Figure 4 This is a schematic diagram of the camera capturing images after 22 vertical Gray code images are projected onto a checkerboard calibration board;

[0046] Figure 5 This is a schematic diagram of the camera capturing images after 24 horizontal Gray code images, all-white images, and all-black images are projected onto a checkerboard calibration board;

[0047] Figure 6 This is a schematic diagram of the detection results of each corner point of the checkerboard calibration board in a checkerboard image projected from a completely white image.

[0048] Figure 7 This is a schematic diagram of the local homography transformation from the corner region of a camera image to the corner region of a projector image;

[0049] Figure 8 This is a schematic diagram of 5 Gray code images;

[0050] Figure 9 This is a schematic diagram of four four-step cosine grating images;

[0051] Figure 10 This is a schematic diagram of the camera capturing images when five projected Gray code images are projected onto the object under test;

[0052] Figure 11 This is a schematic diagram of the camera capturing images when four four-step cosine grating images are projected onto the object under test;

[0053] Figure 12 This is a schematic diagram of the relative phase diagram of the object under test;

[0054] Figure 13 This is a schematic diagram of the absolute phase diagram of the analyte;

[0055] Figure 14 This is a schematic diagram of the 3D point cloud image of the object under test;

[0056] Figure 15 This is a structural block diagram of a 3D reconstruction system for mechanical parts based on structured light technology. Detailed Implementation

[0057] To better understand the above technical solutions, the technical solutions of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solutions of the present invention, rather than limitations on the technical solutions of the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0058] It should also be understood that, in order to simplify the description of the invention and thus aid in the understanding of at least one embodiment, multiple features may sometimes be grouped into a single embodiment, drawing, or description thereof in the foregoing description of the embodiments of the invention. However, this method of disclosure does not imply that the subject matter of the invention requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of the single embodiment disclosed above.

[0059] Example 1

[0060] This invention provides a method for 3D reconstruction of mechanical parts based on structured light technology, the method comprising:

[0061] (1) The camera and projector are jointly calibrated based on Gray code images to obtain the intrinsic and extrinsic parameter matrices and distortion coefficients of the camera and projector.

[0062] In this embodiment of the invention, the process for joint calibration of the camera and projector based on Gray code images is as follows:

[0063] (1.1) Place the checkerboard calibration plate within the field of view of the camera and projector, and generate a Gray code image. Specifically, the resolution of the camera and projector needs to be selected according to the actual situation, including but not limited to selecting a camera resolution of 3000*4096 and a projector resolution of 1080*1920. The size of the checkerboard calibration plate also needs to be selected according to the actual situation of the object to be measured. For small objects to be measured, such as screws, the size of the checkerboard calibration plate includes but is not limited to 100*100mm, with a grid size of 5*5mm.

[0064] In this embodiment of the invention, the Gray code image includes vertical and horizontal positive and negative Gray code images; preferably, the Gray code image includes 11-bit vertical and horizontal positive and negative Gray code images, a pure white image, and a pure black image, totaling 46 images; please refer to [link / reference]. Figure 2 and Figure 3 , Figure 2 This is a schematic diagram of 22 vertical, positive and negative Gray code images of 11 bits each. Figure 3 This is a schematic diagram of 24 11-bit horizontal forward and reverse Gray code images, a pure white image, and a pure black image.

[0065] (1.2) Record the three-dimensional spatial coordinates of each corner point of the checkerboard calibration board at different positions. Project the Gray code image, the all-white image and the all-black image onto the checkerboard calibration board at different positions using a projector. Also, acquire images of the checkerboard calibration board at different positions using a camera to obtain multiple sets of checkerboard images.

[0066] In this embodiment of the invention, the position transformation of the checkerboard calibration board includes, but is not limited to, three times. This means that the three-dimensional spatial coordinates of each corner point of the checkerboard calibration board at four different positions need to be recorded. Vertical forward and reverse Gray code images (22 images in total), horizontal forward and reverse Gray code images (22 images in total), a pure white image, and a pure black image are projected onto the checkerboard calibration board at the four different positions using a projector. Images are then acquired using a camera to obtain four sets of checkerboard images acquired by the camera. Each set of camera-acquired checkerboard images contains a total of 46 images. That is, the images acquired by the camera correspond one-to-one with the generated images. Please refer to [link / reference]. Figure 4 and Figure 5 , Figure 4 The camera captures images after projecting 22 vertical, positive and negative Gray code images onto a checkerboard calibration board. Figure 5 The camera captures images after projecting 24 horizontally projected positive and negative Gray code images, a pure white image, and a pure black image onto a checkerboard calibration board.

[0067] (1.3) For each set of chessboard images, a corner detection algorithm is used to detect each corner point of the chessboard calibration board in the chessboard image projected onto the all-white image, so as to obtain the camera image coordinates of each corner point. Please refer to [link to relevant documentation]. Figure 6 , Figure 6 The image shows the detection results of each corner point of the checkerboard calibration board in the checkerboard image projected from a completely white image.

[0068] In this embodiment of the invention, the corner detection algorithm uses a fixed window to slide in any direction on the image, compares the changes in pixel values ​​in the window area before and after the slide, and if there are large changes in pixel values ​​in any direction, then it can be considered that there are corners in the window. The coordinates of the corners are obtained based on the changes in pixel values, so as to obtain the camera image coordinates of the corners.

[0069] (1.4) For each set of chessboard images, perform local homography matrix transformation on each corner point of the chessboard calibration board in the chessboard image projected by the Gray code image to obtain the projector image coordinates of each corner point.

[0070] In this embodiment of the invention, the process of performing local homography matrix transformation on each corner point of the checkerboard calibration board in the checkerboard image projected from the Gray code image is as follows:

[0071] (1.41) The checkerboard image based on the projection of a completely black image is compensated for by the direct light-indirect light decomposition method to separate the influence of indirect light during imaging. Direct light is the light that shines directly onto the object through the light source of the projector. Meanwhile, the light that is reflected off an object and then projected onto the object is called indirect light. Therefore, the idea of ​​the direct light-indirect light decomposition method is to subtract the pixel values ​​of the checkerboard image projected from those of the completely black image from the pixel values ​​of the checkerboard image projected from those of the Gray code image to obtain the optimized checkerboard image projected from the Gray code image.

[0072] (1.42) The optimized Gray code image projection of the checkerboard image is binarized, and the corner detection algorithm is used to detect each corner of the checkerboard calibration board in the checkerboard image to obtain the projector image coordinates of each corner and several adjacent points in the local area.

[0073] (1.42) In the optimized white image projection of the checkerboard, the corner detection algorithm is used to detect each corner of the checkerboard calibration board in the checkerboard image to obtain the camera image coordinates of each corner and several adjacent points in the local area.

[0074] (1.43) A local homography matrix from camera to projector is calculated based on the camera image coordinates of each corner point and several adjacent points in the local neighborhood, as well as the projector image coordinates. Furthermore, based on the camera image coordinates of each corner point, the local homography matrix is ​​used to calculate more precise pixel coordinates of each corner point in the projector image, thus obtaining the final projector image coordinates. Please refer to [link to relevant documentation]. Figure 7 , Figure 7This diagram illustrates the local homography matrix transformation from the corner region of a camera image to the corner region of a projector image, involving the formula q. n =H n ·p n , where q n p represents the corner point of the nth projected image. n H represents the corner point of the nth camera image. n This represents the local homography matrix at the nth corner point.

[0075] In this embodiment of the invention, binarization is to set pixels with pixel values ​​greater than a certain threshold range to 255 and pixels with pixel values ​​less than a certain threshold range to 0.

[0076] In this embodiment of the invention, the number of times the position of the checkerboard calibration board changes includes, but is not limited to, 3 times. That is, it is necessary to record the three-dimensional spatial coordinates, camera image coordinates and projector image coordinates of each corner point of the checkerboard calibration board at 4 different positions. At the same time, about 40 adjacent points can be selected in the local area.

[0077] (1.5) Based on the three-dimensional spatial coordinates of each corner point at multiple different locations, the camera image coordinates, and the projector image coordinates, the Zhang Zhengyou calibration method is used to calibrate the camera and projector respectively, so as to obtain the intrinsic and extrinsic parameter matrices and distortion coefficients of the camera and projector. That is, the three-dimensional spatial coordinates of each corner point at four different locations are calibrated with the camera image coordinates and the projector image coordinates respectively, so as to obtain the intrinsic and extrinsic parameter matrices and distortion coefficients of the camera and projector.

[0078] In this embodiment of the invention, the idea of ​​Zhang Zhengyou calibration method is to calibrate the camera and projector by using the pixel coordinates of each corner point and the physical coordinates (i.e., three-dimensional spatial coordinates) of each corner point in the world coordinate system, so as to obtain the intrinsic and extrinsic parameter matrices and distortion parameters of the camera and projector. Since Zhang Zhengyou calibration method is existing technology, the calibration process of Zhang Zhengyou calibration method will not be described in detail in this invention.

[0079] (2) The absolute phase of the test object is solved by using a four-step phase shift method combined with Gray code image to obtain the absolute phase map of the test object.

[0080] In this embodiment of the invention, the process of solving the absolute phase of the object under test using the four-step phase-shifting method combined with Gray code images is as follows:

[0081] (2.1) Generate the vertical Gray code image and the four-step cosine grating image; specifically, generate 4-bit and 5-bit Gray code images by selecting the second-to-last image from all the 4-bit and 5-bit Gray code images, for a total of 5 Gray code images. Please refer to [link to relevant documentation]. Figure 8 and Figure 9 , Figure 8 This is a schematic diagram of 5 Gray code images. Figure 9 This is a schematic diagram of four four-step cosine grating images.

[0082] (2.2) Keep the position of the checkerboard calibration plate unchanged when the camera and projector are calibrated together, and place the object to be tested on the surface of the checkerboard calibration plate. The object to be tested is a three-dimensional mechanical part, including screws, bolts, etc.

[0083] (2.3) Project five Gray code images and four four-step cosine grating images onto the test object using a projector, and acquire images of the checkerboard calibration plate using a camera to obtain multiple Gray code images and multiple four-step cosine grating images, totaling nine images; please refer to [link to relevant documentation]. Figure 10 and Figure 11 , Figure 10 This is a schematic diagram illustrating the projection of five Gray code images onto images acquired by the phenological camera under test. Figure 11 This is a schematic diagram of projecting four four-step cosine grating images onto the image acquired by the phenological camera under test.

[0084] (2.4) For the four-step cosine grating images captured by the four cameras, the relative phase of the object under test is solved using the four-step phase-shifting method to obtain the relative phase map of the object under test. Please refer to [link to relevant documentation]. Figure 12 , Figure 12 This is the relative phase diagram of the analyte.

[0085] In this embodiment of the invention, the four-step cosine grating image is obtained by shifting the cosine structured light stripes by a specific phase, with each phase increment being π / 2, thereby obtaining four different cosine structured light stripe images. The four cosine structured light stripe images correspond to phases of 0, π / 2, π, and 3π / 2, respectively. Meanwhile, if the generated Gray code image is a horizontal Gray code image, then the four-step sine grating image needs to be used for corresponding subsequent processing.

[0086] In this embodiment of the invention, the principle of the four-step phase-shifting method is as follows: First, the intensity of four-step cosine grating images captured by four cameras is calculated, usually approximated by pixel values; then, by comparing the fringe intensity of two adjacent four-step cosine grating images captured by two cameras, the corresponding phase difference can be calculated; since the phase shift increment is π / 2 each time, the phase difference should be an integer multiple of π / 2; finally, the phase differences are accumulated or subtracted to obtain a complete phase distribution, and the relative phase map of the measured object is obtained based on the phase distribution.

[0087] (2.5) For the five Gray code images captured by the camera, the cosine grating period of each point of the object under test in the relative phase map is determined using the binarized Gray code images. The absolute phase map of the object under test is then reconstructed using the cosine grating period. Please refer to [link to relevant documentation]. Figure 13 , Figure 13 This is a schematic diagram of the absolute phase diagram of the object under test. Specifically, the conversion between relative phase and absolute phase can be performed with reference to the phase conversion formula, which will not be elaborated further in this embodiment of the invention.

[0088] (3) Perform matrix transformation on the absolute phase map of the object under test by combining the intrinsic and extrinsic parameter matrices of the camera and projector to draw a 3D point cloud map of the object under test. Then, correct the 3D point cloud map of the object under test by combining the distortion coefficients of the camera and projector. Please refer to [link to relevant documentation]. Figure 14 , Figure 14 This is a schematic diagram of the 3D point cloud image of the object under test.

[0089] In summary, this invention first generates and projects horizontal and vertical positive and negative Gray codes onto a checkerboard calibration board, then acquires four sets of different projected images by changing positions. The Zhang Zhengyou calibration method is used to jointly calibrate the camera and projector, solving the problem that the 2D camera calibration scheme cannot calibrate the projector. Next, vertical Gray codes and a four-step cosine grating image are generated and projected onto the object under test, acquiring the corresponding projected images. The phase of each point in the image is calculated and unfolded using the four-step phase-shifting method to obtain the absolute phase map of the image. This solves the problems of point-by-point error accumulation and discontinuities on the surface of the object under test leading to phase unfolding errors in the classic spatial phase unfolding algorithm. Finally, combining the intrinsic and extrinsic parameter matrices of the camera and projector with the absolute phase map of the image, matrix transformation is used to calculate the three-dimensional spatial coordinates corresponding to the pixel coordinates of each point in the image, obtaining a 3D point cloud map of the object under test. This solves the problem that 2D images cannot present some defect feature information, laying the foundation for subsequent 3D defect detection.

[0090] Example 2

[0091] This invention provides a 3D reconstruction system for mechanical parts based on structured light technology. Please refer to [link / reference]. Figure 15 , Figure 15 This is a structural block diagram of a 3D reconstruction system for mechanical parts, the system comprising:

[0092] A joint calibration unit performs joint calibration of the camera and projector based on Gray code images to obtain the intrinsic and extrinsic parameter matrices and distortion coefficients of the camera and projector.

[0093] An absolute phase solving unit is used to solve the absolute phase of the test object by employing a four-step phase shift method combined with Gray code images to obtain the absolute phase map of the test object.

[0094] The 3D reconstruction unit combines the intrinsic and extrinsic parameter matrices of the camera and projector to perform matrix transformation on the absolute phase map of the object under test to draw a 3D point cloud map of the object under test, and combines the distortion coefficients of the camera and projector to correct the 3D point cloud map of the object under test.

[0095] Example 3

[0096] This invention provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the computer program executes the 3D reconstruction method for mechanical parts based on structured light technology described in Embodiment 1 when the processor is running.

[0097] The computer device provided in this embodiment can implement the method described in Embodiment 1. To avoid repetition, it will not be described again here.

[0098] Example 4

[0099] This invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the 3D reconstruction method for mechanical parts based on structured light technology described in Embodiment 1.

[0100] The computer-readable storage medium provided in this embodiment can implement the method described in Embodiment 1. To avoid repetition, it will not be described again here.

[0101] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0102] The memory can be used to store the computer program and / or modules. The processor, by running or executing the data stored in the memory, realizes various functions of the 3D reconstruction system for mechanical parts based on structured light technology in the invention. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart memory card, secure digital card, flash memory card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0103] If a 3D reconstruction system for mechanical parts based on structured light technology is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program that can be stored in a computer-readable storage medium. When executed by a processor, this computer program can implement the steps of the various method embodiments described above. The computer program includes computer program code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory, random access memory, dot carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0104] The basic concepts of this invention have been described. It is obvious to those skilled in the art that the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0105] Computer storage media may contain a propagated data signal containing computer program code, for example, on baseband or as part of a carrier wave. This propagated signal may take various forms, including electromagnetic, optical, and suitable combinations thereof. Computer storage media can be any computer-readable medium other than a computer-readable storage medium, which can be connected to an instruction execution system, apparatus, or device to enable communication, propagation, or transmission of a program for use. The program code located on the computer storage medium can be propagated through any suitable medium, including radio, cable, fiber optic cable, RF, or similar media, or any combination of the above media.

Claims

1. A method for 3D reconstruction of mechanical parts based on structured light technology, characterized in that, The method includes the following steps: The camera and projector are jointly calibrated based on Gray code images to obtain the intrinsic and extrinsic parameter matrices and distortion coefficients of the camera and projector. A four-step phase-shifting method combining Gray code images is used to solve for the absolute phase of the object under test, in order to obtain the absolute phase map of the object under test, including: Place the checkerboard calibration board within the field of view of the camera and projector, and generate a Gray code image; Record the three-dimensional spatial coordinates of each corner point of the checkerboard calibration board at different positions. Project Gray code images, all-white images, and all-black images onto the checkerboard calibration board at different positions using a projector. Also, capture images of the checkerboard calibration board at different positions using a camera to obtain multiple sets of checkerboard images. For each set of chessboard images, a corner detection algorithm is used to detect each corner of the chessboard calibration board in the chessboard image projected from a completely white image, so as to obtain the camera image coordinates of each corner. For each set of chessboard images, a local homography matrix transformation is performed on each corner point of the chessboard calibration board in the chessboard image projected from the Gray code image to obtain the projector image coordinates of each corner point. Based on the three-dimensional spatial coordinates of each corner point, camera image coordinates, and projector image coordinates at multiple different locations, the Zhang Zhengyou calibration method is used to calibrate the camera and projector respectively, so as to obtain the intrinsic and extrinsic parameter matrices and distortion coefficients of the camera and projector. Generate Gray code images and four-step cosine grating images; Keep the position of the checkerboard calibration plate unchanged when the camera and projector are calibrated together, and place the object to be tested on the surface of the checkerboard calibration plate. Gray code images and four-step cosine grating images are projected onto the object under test using a projector, and images of the checkerboard calibration board are acquired using a camera to obtain multiple Gray code images and multiple four-step cosine grating images captured by the camera. For four-step cosine grating images captured by multiple cameras, a four-step phase-shifting method is used to solve the relative phase of the object under test in order to obtain the relative phase map of the object under test. For Gray code images captured by multiple cameras, the cosine grating period of each point of the test object in the relative phase map is determined by using the binarized Gray code images, and the absolute phase map of the test object is obtained by restoring the cosine grating period. A matrix transformation is performed on the absolute phase map of the object under test by combining the intrinsic and extrinsic parameter matrices of the camera and projector to draw a 3D point cloud map of the object under test. Furthermore, the distortion coefficients of the camera and projector are used to correct the 3D point cloud map of the object under test, including: For the absolute phase map of the object under test, the pixel coordinates of each point of the object under test in the projector image are calculated based on the phase invariance assumption and the cosine grating period. Based on the pixel coordinates of each point of the object under test in the phase image and the corresponding pixel coordinates in the projector image, and combined with the intrinsic and extrinsic parameter matrices of the camera and the projector, the three-dimensional spatial coordinates of each point in the camera image are calculated by rectangular transformation. A 3D point cloud map of the object under test is drawn based on three-dimensional coordinates, and the distortion coefficients of the camera and projector are used to correct the 3D point cloud map of the object under test.

2. The method for 3D reconstruction of mechanical parts based on structured light technology according to claim 1, characterized in that: The Gray code images generated during the joint calibration of the camera and projector based on Gray code images include horizontal and vertical positive and negative Gray code images.

3. The method for 3D reconstruction of mechanical parts based on structured light technology according to claim 1, characterized in that: The Gray code image generated when the absolute phase of the object under test is solved by the four-step phase shift method combined with Gray code image includes a vertical Gray code image.

4. The method for 3D reconstruction of mechanical parts based on structured light technology according to claim 1, characterized in that, The object to be tested is a mechanical part.

5. A 3D reconstruction system for mechanical parts based on structured light technology for implementing the method of claim 1, characterized in that, The system includes: A joint calibration unit performs joint calibration of the camera and projector based on Gray code images to obtain the intrinsic and extrinsic parameter matrices and distortion coefficients of the camera and projector. An absolute phase solving unit is used to solve the absolute phase of the test object by employing a four-step phase shift method combined with Gray code images to obtain the absolute phase map of the test object. The 3D reconstruction unit combines the intrinsic and extrinsic parameter matrices of the camera and projector to perform matrix transformation on the absolute phase map of the object under test to draw a 3D point cloud map of the object under test, and combines the distortion coefficients of the camera and projector to correct the 3D point cloud map of the object under test.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements a 3D reconstruction method for mechanical parts based on structured light technology as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the 3D reconstruction method for mechanical parts based on structured light technology as described in any one of claims 1-4.

Images