Fringe projection 3D measurement method, terminal device and computer-readable storage medium

Abstract

The present invention discloses a fringe projection 3D measurement method, terminal device, and computer-readable storage medium, comprising: generating at least four marker points, projecting at least four marker points and a phase-shifted fringe pattern; adjusting the calibration plate posture to capture multiple sets of calibration patterns, projecting and capturing multiple sets of calibration images, multiple sets of marker point patterns, and multiple sets of phase-shift patterns; obtaining at least four local area maps by wrapping the phase pattern, obtaining at least four starting calculation points based on the at least four marker points, and performing a series solution to obtain a phase solution result; establishing an inverse fifth-order complete polynomial distortion model to determine the positional relationship between the projector and the camera; dedistorting pixel coordinates and phase values ​​based on the inverse fifth-order complete polynomial distortion model, and substituting the dedistorted pixel coordinates and phase values ​​into the three-dimensional reconstruction equation to calculate the three-dimensional coordinates of the corresponding points. The inverse distortion polynomial model proposed by the present invention does not require nonlinear dedistortion operations, effectively improving the computational efficiency of three-dimensional reconstruction.

Description

Technical Field

[0001] The present invention relates to the field of optical technology, and in particular to a fringe projection 3D measurement method, a terminal device, and a computer-readable storage medium. Background Art

[0002] In the field of structured light measurement, phase measurement profilometry (PMP) is a mature method for obtaining the three-dimensional shape point cloud of an object's surface. PMP has the advantages of non-contact, full-field measurement, high precision, fast speed and large range. It has been widely used in industrial automation detection, biomedicine, quality control and other fields.

[0003] Key technologies in PMP include phase unwrapping and system calibration. However, stereo vision-based calibration methods treat the projector as an inverse camera, projecting phase-shifted fringes onto a calibration plate. Phase values ​​at corner points are interpolated, and traditional calibration methods are then used to calculate the projector's internal distortion. Camera calibration can use either the PnP method or the telecentric lens calibration method, depending on the lens type. This method requires dedistortion of the obtained phase values ​​and pixel coordinates during reconstruction, which is computationally complex and time-consuming. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a fringe projection 3D measurement method, a terminal device and a computer-readable storage medium, which can effectively improve the computational efficiency of three-dimensional reconstruction.

[0005] To achieve the above-mentioned purpose, an embodiment of the present invention provides a fringe projection 3D measurement method, which includes: generating at least four marking points, determining the fringe order of the marking points according to the positions of the marking points and a preset phase-shift fringe pattern, and projecting at least four of the marking points and the phase-shift fringe pattern; adjusting the posture of the calibration plate to shoot multiple sets of calibration images, and corresponding to the multiple sets of calibration images, projecting and shooting multiple sets of calibration images, multiple sets of marking point images, and multiple sets of phase-shift images; calculating the wrapping phase to obtain a wrapping phase map, and obtaining at least four local area maps through the wrapping phase map, wherein at least four of the marking points are located at the centers of at least four local areas. According to each of the at least four marking points, at least four starting calculation points are obtained to perform series solution to obtain a phase solution result; camera calibration is performed, and a calculated phase value is obtained by calculating the projection model. An inverse fifth-order complete polynomial distortion model is established through the calculated phase value, the phase solution result, and the original pixel value to determine the positional relationship between the projector and the camera; the marking point map and the phase shift map are projected and photographed on the test piece, and the pixel coordinates and phase values ​​are dedistorted according to the inverse fifth-order complete polynomial distortion model, and the dedistorted pixel coordinates and phase values ​​are substituted into the three-dimensional reconstruction equation to calculate the three-dimensional coordinates of the corresponding point.

[0006] As an improvement to the above solution, the step of "generating at least four marking points" includes: dividing the original image pre-projected by the projector into four equal parts, and generating the marking points in the centers of the four areas divided into four equal parts.

[0007] As an improvement of the above solution, the step of "projecting at least four of the marking points and the phase-shift fringe pattern" includes: before projecting the phase-shift fringe pattern, projecting at least four of the marking points.

[0008] As an improvement to the above scheme, after the step of "calculating the wrapping phase to obtain the wrapping phase map", it also includes: template matching the generated marker point bitmap with the marker point bitmap collected by shooting to obtain the actual pixel coordinates of at least four of the marker points.

[0009] As an improvement of the above scheme, the step of "obtaining at least four local area maps through the wrapped phase map, and at least four of the marked points are located in the centers of at least four local areas" includes: dividing the length and width of the wrapped phase map into two equal parts to obtain at least four local area maps, and each local area map is centered on the marked point.

[0010] As an improvement to the above scheme, the step of "obtaining at least four starting calculation points according to each of the at least four marking points to perform series solution to obtain the phase solution result" includes: determining the phase order of the three adjacent points on the left, above, and upper left corresponding to the marking point, obtaining four starting calculation points, and performing phase divergence expansion in two directions for each of the starting calculation points.

[0011] As an improvement to the above scheme, the step of "performing camera calibration and calculating the phase value through the projection model" includes: selecting a set of the calibration images, calibrating the camera through a pure white light image, extracting the pixel coordinates of the corner points of the calibration plate, and decomposing the pinhole model of the lens imaging according to the size information of the calibration plate to obtain the initial values ​​of the magnification, principal point, and rotation and translation matrix.

[0012] As an improvement to the above scheme, the step of "establishing an inverse fifth-order complete polynomial distortion model through the calculated phase value, the phase solution result, and the original pixel value" includes: for the pixel coordinates of the corner points on the calibration plate, the phase calculated by the pinhole model is used as the distorted value, the phase solution result and the original pixel value are used as the values ​​before distortion, and inverse fifth-order complete polynomial distortion fitting is performed to calculate the distortion coefficient, and the external parameter result of the same posture corresponding to the camera calibration in the projection calibration result is extracted as the system parameter.

[0013] An embodiment of the present invention further provides a terminal device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements any of the above-described fringe projection 3D measurement methods when executing the computer program.

[0014] An embodiment of the present invention further provides a computer-readable storage medium, which includes a stored computer program. When the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned fringe projection 3D measurement methods.

[0015] Compared to existing technologies, the fringe projection 3D measurement method, terminal device, and computer-readable storage medium provided by embodiments of the present invention offer the following advantages: Only one marker point map and four phase shift maps need to be projected during 3D reconstruction calculations, significantly reducing projection time. Sixteen-thread parallel computations across four marker points and four directions improve the efficiency of phase unwrapping and, to a certain extent, address the inability of spatial phase unwrapping methods to perform phase unwrapping when fringes are broken or discontinuous. The proposed inverse distortion polynomial model eliminates the need for nonlinear dedistortion operations, effectively improving the computational efficiency of 3D reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flow chart of a preferred embodiment of a fringe projection 3D measurement method provided by the present invention;

[0017] Figure 2 1 is a schematic structural diagram of a preferred embodiment of a fringe projection 3D measurement system provided by the present invention;

[0018] Figure 3 It is a structural diagram of a preferred embodiment of a terminal device provided by the present invention. DETAILED DESCRIPTION

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0020] See also Figure 1 and Figure 2 , Figure 1 This is a flow chart of a preferred embodiment of a fringe projection 3D measurement method provided by the present invention. Figure 2 yes Figure 2It is a structural schematic diagram of a preferred embodiment of a fringe projection 3D measurement system provided by the present invention.

[0021] like Figure 1 As shown, the fringe projection 3D measurement method includes:

[0022] Step S100, generating at least four marking points, determining the fringe level of the marking point according to the position of the marking point and a preset phase-shift fringe pattern, and projecting the at least four marking points and the phase-shift fringe pattern;

[0023] Step S200: Adjust the posture of the calibration plate to capture multiple sets of calibration images, and project and capture multiple sets of calibration images, multiple sets of marker point maps, and multiple sets of phase shift maps corresponding to the multiple sets of calibration images;

[0024] Step S300: Calculate the wrapped phase to obtain a wrapped phase diagram, obtain at least four local area diagrams from the wrapped phase diagram, and obtain at least four starting calculation points based on each of the at least four marked points to perform a series solution to obtain a phase solution result.

[0025] Step S400: calibrate the camera 202, calculate the calculated phase value through the projection model, establish an inverse fifth-order complete polynomial distortion model based on the calculated phase value, the phase solution result, and the original pixel value, and determine the positional relationship between the projector 201 and the camera 202;

[0026] In step S500, the marked positioning map and the phase shift map are projected onto the test piece and photographed, pixel coordinates and phase values ​​are dedistorted according to the inverse fifth-order complete polynomial distortion model, and the dedistorted pixel coordinates and phase values ​​are substituted into the three-dimensional reconstruction equation to calculate the three-dimensional coordinates of the corresponding points.

[0027] Specifically, the fringe projection 3D measurement system used in the present invention includes a projector 201 and a camera 202 , wherein the projector 201 uses an industrial lens (CCD camera) and the camera 202 uses a telecentric lens camera.

[0028] In step S100, which designs the projection map of the marker points, the original image projected by projector 201 is first divided into four equal parts. Four cross-shaped marker points are designed at the centers of the four regions. A positioning map of these four marker points is projected before projecting the phase-shifted fringes. The fringe level for each marker point is determined based on the central pixel position of each marker point and the designed phase-shifted fringe pattern.

[0029] For example, the aforementioned marking points may also be circular marking points, QR codes, or other characteristic points, but the present invention is not limited thereto.

[0030] In step S200, i.e., the step of taking calibration pictures, it also includes: step S201, placing a dot calibration plate with good diffuse reflection characteristics of the bottom plate in the common field of view of the projector 201 and the camera 202, first taking a calibration image of projecting pure white light, and then projecting and taking a positioning image of the marked points and a four-step phase shift image in two directions; step S202, changing the posture of the calibration plate, repeating step S201, and collecting 20 sets of left and right calibration images.

[0031] In step S300, i.e., the phase solution step, it also includes: step S301, calculating the wrapped phase to obtain the wrapped phase map in two directions, wherein this step is to obtain the wrapped phase map in two directions by performing differential, intensity, reflectivity, modulation and inverse tangent calculations on the four phase shift maps in each direction; step S302, performing template matching on the collected marked point map according to the designed marked point map, and obtaining the actual pixel coordinates of the four marked points; step S303, dividing the length and width of the wrapped phase map into two equal parts, and obtaining a total of four local area maps, and on each local map, the marks on it are used as the basis for the matching. The point is taken as the center, and the series is solved according to step S304; wherein, step S304 includes: according to the phase jump between adjacent points, first determine the phase series of the three adjacent points on the left, above, and upper left of the marked point in the local graph, and obtain a total of four starting calculation points. For each starting calculation point, only two directions of phase divergence expansion are required, and the left point is calculated point by point to the left and downward, and the upper left point is calculated to the left and upward, and the other points are calculated by analogy; it can be understood that the above embodiment opens up sixteen threads to perform parallel calculation of series expansion for each starting point in each local graph.

[0032] Step S400, i.e., the system calibration step, also includes selecting a set of calibration images and calibrating the telecentric lens camera 202 using pure white light images. The pixel coordinates of the calibration image corners are extracted. Based on the physical dimensions of the calibration image, the pinhole model of the telecentric lens imaging is decomposed to obtain initial values ​​for the magnification, principal point, and rotation and translation matrices. The values ​​calculated by the model are used as the distorted points, and the detected corner pixel coordinates are used as the undistorted points. Inverse distortion fitting is performed using a fifth-order complete polynomial, and the distortion coefficients are calculated using the least squares method.

[0033] It can be understood that the phase solution method of the present invention is used to solve the phase of each set of calibration images, and the phase at the corner points is obtained through bicubic B-spline interpolation. Since the world coordinates of the corner points are known, the traditional Zhang Zhengyou method can be used to solve the projector intrinsic parameters and the projector extrinsic parameters relative to each calibration posture. For the corner points on the calibration plate, the phase calculated using the pinhole model of projector 201 is used as the distorted value, and the interpolated phase is used as the undistorted value. Inverse fifth-order complete polynomial distortion fitting is performed to calculate the distortion coefficient. The extrinsic parameter results of the same posture corresponding to the camera 202 calibration in the projection calibration results are extracted as system parameters. At this point, the positional relationship between projector 201 and camera 202 relative to the calibration plate has been established.

[0034] In step S500, i.e., the 3D reconstruction step, it also includes: step S501, projecting and photographing a mark point map and four phase shift images in a single direction on the test piece, wherein the stripe direction of the phase shift image must be perpendicular to the line connecting the camera 202 and the projector 201; step S502, solving the phase, at this time, each integer pixel point on the camera 202 corresponds to a phase value, and according to the distortion results of the camera 202 and the lens, the values ​​of the pixel point and the phase value before distortion are calculated respectively. Since the inverse distortion model is adopted, the de-distortion calculation at this time only requires simple four arithmetic operations, which greatly improves the calculation efficiency; step S503, substituting the de-distorted pixel coordinates and phase values ​​into the 3D reconstruction equation to calculate the 3D coordinates of the corresponding point.

[0035] It should be noted that the above four marking points can perform phase unwrapping calculations at the same time, and each marking point performs series calculations in the four directions of upper left, lower left, upper right, and lower right at the same time, so that 16 threads can be used for parallel calculations; in addition, it is also mentioned in the above embodiment that this method reversely regards the value calculated by the projection model as the result after distortion, and regards the phase solution result and the original pixel value as the result before distortion, and establishes a reverse fifth-order complete polynomial distortion model, which only requires four arithmetic operations without the need for nonlinear dedistortion.

[0036] In addition, in the above embodiment, the solution is performed in parallel with at least four marking points and at least four directions of divergence. Other numbers of marking points and numbers of phase divergence directions (such as dividing the image into 8 areas, and each area is phase-unfolded in parallel along the marking points upward, downward, left, right, upper left, lower left, upper right, and lower right) are also regarded as alternatives to the present invention. The present invention is not limited to this, as long as it can achieve the same technical effect.

[0037] Correspondingly, the present invention also provides a fringe projection 3D measurement system, which can implement all the processes of the fringe projection 3D measurement method in the above embodiment.

[0038] In specific implementation, the working principle, control process and technical effects achieved by the fringe projection 3D measurement system provided by the embodiment of the present invention are the same as those of the fringe projection 3D measurement method in the above embodiment, and will not be repeated here.

[0039] See also Figure 3 , Figure 3 3 is a schematic diagram of a preferred embodiment of a terminal device provided by the present invention. The terminal device includes a processor 301, a memory 302, and a computer program stored in the memory 302 and configured to be executed by the processor 301. When the processor 301 executes the computer program, it implements the fringe projection 3D measurement method described in any of the above embodiments.

[0040] Preferably, the computer program can be divided into one or more modules / units (e.g., computer program 1, computer program 2, ...). These one or more modules / units are stored in the memory 302 and executed by the processor 301 to implement the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the terminal device.

[0041] The processor 301 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor, or the processor 301 can be any conventional processor. The processor 301 is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the terminal device.

[0042] The memory 302 primarily includes a program storage area and a data storage area. The program storage area can store an operating system, at least one application required for a function, and the data storage area can store related data. Furthermore, the memory 302 can be a high-speed random access memory or a non-volatile memory, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, or a flash card. Alternatively, the memory 302 can be other volatile solid-state storage devices.

[0043] It should be noted that the above terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that Figure 3 The structural diagram is only an example of the above-mentioned terminal device and does not constitute a limitation on the above-mentioned terminal device. It may include more or fewer components than shown in the figure, or combine certain components, or different components.

[0044] An embodiment of the present invention further provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the fringe projection 3D measurement method described in any of the above embodiments.

[0045] Embodiments of the present invention provide a fringe projection 3D measurement method, terminal device, and computer-readable storage medium. These methods propose more efficient algorithms for phase resolution, calibration, and 3D reconstruction, enabling rapid measurement. Specifically, the present invention requires only one marker point map and four phase shift maps for 3D reconstruction, significantly reducing projection time. Sixteen-thread parallel computation across four marker points and four directions improves the efficiency of phase unwrapping and, to a certain extent, addresses the inability of spatial phase unwrapping methods to perform phase unwrapping when fringes are broken or discontinuous. The proposed inverse distortion polynomial model eliminates the need for nonlinear dedistortion, effectively improving the computational efficiency of 3D reconstruction.

[0046] It should be noted that the system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the system embodiment provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement the present invention without inventive work.

[0047] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A fringe projection 3D measurement method, characterized in that: The method comprises: generating at least four marking points, and determining the fringe level at which the marking points are located according to the positions of the marking points and a preset phase-shift fringe pattern, so as to project the at least four marking points and the phase-shift fringe pattern; Adjusting the posture of the calibration plate to shoot multiple sets of calibration images, and corresponding to the multiple sets of calibration images, projecting and shooting multiple sets of calibration pictures, multiple sets of marking point maps and multiple sets of phase shift maps; Calculating the wrapped phase to obtain a wrapped phase diagram, obtaining at least four local area diagrams through the wrapped phase diagram, at least four of the marked points being located at the center of the at least four local area diagrams, and obtaining at least four starting calculation points based on each of the at least four marked points to perform a series solution to obtain a phase solution result; Perform camera calibration, calculate phase values ​​through projection model calculation, establish an inverse fifth-order complete polynomial distortion model through the calculated phase values, the phase solution results, and original pixel values, and determine the positional relationship between the projector and the camera; The mark point bitmap and the phase shift map are projected onto the test piece and photographed, the pixel coordinates and phase values ​​are dedistorted according to the inverse fifth-order complete polynomial distortion model, and the dedistorted pixel coordinates and phase values ​​are substituted into the three-dimensional reconstruction equation to calculate the three-dimensional coordinates of the corresponding points.

2. The fringe projection 3D measurement method according to claim 1, wherein: The “generating at least four marking points” includes: dividing the original image pre-projected by the projector into four equal parts, and generating the marking points in the centers of the four areas divided into four equal parts.

3. The fringe projection 3D measurement method according to claim 1, wherein: The “projecting at least four of the marking points and the phase-shift fringe pattern” includes: before projecting the phase-shift fringe pattern, projecting at least four of the marking points.

4. The fringe projection 3D measurement method according to claim 1, wherein: After the "calculating the wrapped phase to obtain the wrapped phase diagram", the method further includes: The generated marker point bitmap is template matched with the marker point bitmap collected by shooting to obtain the actual pixel coordinates of at least four marker points.

5. The fringe projection 3D measurement method according to claim 1, wherein: The “obtaining at least four local area maps by wrapping the phase map, wherein at least four of the marking points are located at the centers of the at least four local area maps” includes: The length and width of the wrapped phase image are divided into two equal parts to obtain at least four local area images, each of which is centered on the marked point.

6. The fringe projection 3D measurement method according to claim 5, wherein: The step of "obtaining at least four starting calculation points based on each of the at least four marking points to perform series solution to obtain a phase solution result" includes: The phase orders of the three adjacent points on the left, above, and upper left corresponding to the marked point are determined to obtain four starting calculation points. For each of the starting calculation points, phase divergence expansion in two directions is performed.

7. The fringe projection 3D measurement method according to claim 1, wherein: The "performing camera calibration and calculating the phase value by using the projection model" includes: A set of calibration images is selected, and the camera is calibrated using a pure white light image. The pixel coordinates of the corner points of the calibration plate are extracted. Based on the size information of the calibration plate, the pinhole model of the lens imaging is decomposed to obtain the initial values ​​of the magnification, principal point, and rotation and translation matrix.

8. The fringe projection 3D measurement method according to claim 7, wherein: The “establishing a reverse fifth-order complete polynomial distortion model by using the calculated phase value, the phase solution result, and the original pixel value” includes: For the pixel coordinates of the corner points on the calibration plate, the phase calculated by the pinhole model is used as the distorted value. The phase solution result and the original pixel value are used as the pre-distorted value. Inverse fifth-order complete polynomial distortion fitting is performed to calculate the distortion coefficient. The external parameter result of the same posture corresponding to the camera calibration in the projection calibration result is extracted as the system parameter.

9. A terminal device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the fringe projection 3D measurement method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the fringe projection 3D measurement method according to any one of claims 1 to 8.

Images