Structured light three-dimensional reconstruction method and system

Abstract

Embodiments of the present invention disclose a structured light three-dimensional reconstruction method and system. The method comprises: obtaining internal parameters of an industrial camera by using a checkerboard calibration method; obtaining stripe patterns adapted to a display screen and a projector; on the basis of the stripe patterns, obtaining correspondences among pixels of the industrial camera, the display screen, and the projector; on the basis of the internal parameters and the correspondences, obtaining distance information; on the basis of the internal parameters, the correspondences, and the distance information, obtaining three-dimensional coordinates; and on the basis of the three-dimensional coordinates, obtaining a three-dimensional image. Thus, by constructing a structured light three-dimensional reconstruction system, the calibration process between the industrial camera and the projector is simplified, and the geometric constraint complexity of a phase-height model is reduced.

Description

A structured light 3D reconstruction method and system

[0001] This application claims priority to the Chinese patent application filed on December 18, 2023, with application number 202311850986.7 and invention name “A structured light three-dimensional reconstruction method and system”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present invention relates to the field of three-dimensional vision technology, and in particular to a structured light three-dimensional reconstruction method and system. Background Art

[0003] Structured light 3D reconstruction technology can perform 3D reconstruction based on the 3D shape and texture information of an object's surface. In structured light 3D reconstruction, the projector needs to project, for example, a binary code image, a color fringe image, or a sinusoidal fringe image. The projected image is used to encode the object surface, thereby achieving 3D reconstruction of a single object surface. Among them, structured light 3D reconstruction technology is divided into binocular structured light and monocular structured light. Binocular structured light adds a projector to binocular stereo vision, while monocular structured light replaces one of the cameras with a projector. In monocular structured light, there are two main methods for 3D reconstruction of the object surface: one is to use the triangular relationship between the camera and the projector to reconstruct using a triangular stereo model; the other is to use a phase height model when the optical axes of the camera and projector are parallel.

[0004] Although existing technologies can achieve three-dimensional reconstruction of single objects with surface textures, there are still problems such as calibrating the internal and external parameters of industrial cameras and projection optical machines, as well as the cumbersome calibration procedures of phase models.

[0005] Summary of the Invention

[0006] In view of this, an object of an embodiment of the present invention is to provide a structured light 3D reconstruction method and system to simplify the calibration procedure between an industrial camera and a projection optical machine and reduce the difficulty of geometric constraints on relative height models.

[0007] In a first aspect, an embodiment of the present invention provides a structured light 3D reconstruction method, the method comprising:

[0008] Obtain the internal parameters of industrial cameras through the checkerboard calibration method;

[0009] Obtaining stripe patterns suitable for display screens and projector light machines;

[0010] Obtaining a correspondence between pixels of an industrial camera, a display screen, and a projection light machine according to the fringe pattern;

[0011] Acquiring distance information according to the internal parameters and the corresponding relationship;

[0012] Acquire three-dimensional coordinates according to the internal parameters, the corresponding relationship and the distance information;

[0013] A three-dimensional image is acquired according to the three-dimensional coordinates.

[0014] In some embodiments, the fringe pattern includes a Gray code pattern and a phase shift pattern;

[0015] The step of obtaining a stripe pattern adapted to a display screen or a projection light engine includes:

[0016] Get the predetermined rectangular area;

[0017] Divide the rectangular area into 2M equal parts in the horizontal or vertical direction, where M is a positive integer greater than or equal to 1;

[0018] Generate horizontal or vertical M-bit Gray code image;

[0019] The phase shift diagram is obtained according to the N-step phase shift method.

[0020] In some embodiments, the correspondence relationship includes a first correspondence relationship between industrial camera pixels and display screen pixels, a second correspondence relationship between industrial camera pixels and projection optical machine pixels, and a third correspondence relationship between display screen pixels and projection optical machine pixels;

[0021] The step of obtaining the corresponding relationship between the pixels of the industrial camera, the display screen, and the projector according to the stripe pattern includes:

[0022] Displaying the stripe pattern on the display screen, decoding the image of the display screen acquired by the industrial camera to obtain a first absolute phase value, decoding the image displayed on the display screen to obtain a second absolute phase value, and matching the first absolute phase value and the second absolute phase value to obtain the first corresponding relationship;

[0023] Projecting the fringe pattern through a projection optical machine, decoding an image of the projection optical machine acquired by the industrial camera to obtain a third absolute phase value, decoding the image projected by the projection optical machine to obtain a fourth absolute phase value, and matching the third absolute phase value and the fourth absolute phase value to obtain the second corresponding relationship;

[0024] The third corresponding relationship is acquired according to the first corresponding relationship and the second corresponding relationship.

[0025] In some embodiments, the distance information includes a first distance between the optical center of the industrial camera and the display screen, a second distance between the optical center of the projector and the display screen, and a third distance between the optical axis of the industrial camera and the optical axis of the projector;

[0026] The acquiring of distance information according to the internal parameters and the corresponding relationship includes:

[0027] Obtaining, based on the internal parameters, a focal length of the industrial camera and a first pixel corresponding to a principal point of the industrial camera, determining, based on the first corresponding relationship, a second pixel on the display screen corresponding to the first pixel, selecting a third pixel from the pixels of the industrial camera, determining, based on the first corresponding relationship, a fourth pixel on the display screen corresponding to the third pixel, and obtaining the first distance based on the first pixel, the second pixel, the third pixel, and the fourth pixel;

[0028] Obtaining, according to the internal parameters, a focal length of the projection light engine and a fifth pixel corresponding to the principal point of the projection light engine, determining, according to a third correspondence, a sixth pixel on the display screen corresponding to the fifth pixel, selecting a seventh pixel from the projection light engine pixels, determining, according to the third correspondence, an eighth pixel on the display screen corresponding to the seventh pixel, and obtaining the second distance based on the fifth, sixth, seventh, and eighth pixels;

[0029] The third distance is acquired according to the second pixel and the sixth pixel.

[0030] In some embodiments, the first distance is calculated using the following formula:

[0031] Among them, f c is the focal length of the industrial camera, [u c0 ,v c0 ] is the first pixel, [u sc ,v sc ] is the second pixel, [u c1 ,v c1 ] is the third pixel, [u s1 ,v s1 ] is the fourth pixel, H c and V c H is the length and width of the industrial camera pixel. s 、V s The length and width of the display pixel.

[0032] In some embodiments, the second distance is calculated using the following formula:

[0033] Among them, f p is the focal length of the projection machine, [u p0 ,v p0 ] is the fifth pixel, [u sp ,v sp ] is the sixth pixel, [u p1 ,v p1] is the seventh pixel, [u s2 ,v s2 ] is the eighth pixel, H s 、V s H is the length and width of the display pixel. p 、V p They are the length and width of the projection optical machine pixel respectively.

[0034] In some embodiments, the third distance is calculated using the following formula:

[0035] Among them, [u sc ,v sc ] is the second pixel, [u sp ,v sp ] is the sixth pixel, H s 、V s The length and width of the display pixel.

[0036] In some embodiments, obtaining three-dimensional coordinates according to the internal parameters, the corresponding relationship, and the distance information includes:

[0037] Determine the ninth pixel of the measurement point in the projector and the tenth pixel in the industrial camera;

[0038] Acquire, according to the corresponding relationship, an eleventh pixel on the display screen corresponding to the ninth pixel and a twelfth pixel on the display screen corresponding to the tenth pixel;

[0039] The three-dimensional coordinates of the point to be measured are acquired according to the distance information, the ninth pixel, the tenth pixel, the eleventh pixel, and the twelfth pixel, where the three-dimensional coordinates are the three-dimensional coordinates of the point to be measured relative to the display screen.

[0040] In some embodiments, the three-dimensional coordinates are obtained by the following formula:

[0041] Among them, [u p2 ,v p2 ] is the ninth pixel, [u c2 ,v c2 ] is the tenth pixel, [u s3 ,v s3 ] is the eleventh pixel, [u s4 ,v s4 ] is the twelfth pixel, [u s5 ,v s5 ] is the vertical mapping display screen pixel of the point to be measured, H s 、V s L is the length and width of the display pixel. c is the first distance, L p is the second distance.

[0042] In a second aspect, an embodiment of the present invention provides a structured light 3D reconstruction system, the system comprising:

[0043] Projection light machine, used to project structured light onto the display screen or the object to be measured;

[0044] Industrial cameras are used to obtain the structured light projected by the projector onto the display screen and the structured light projected by the projector onto the object to be measured;

[0045] Display screen for displaying stripe patterns or structured light projected by a light projection machine;

[0046] A slide rail, used to adjust the relative position of the projection light machine or industrial camera with respect to the display screen;

[0047] A data processing device comprises a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in any one of the first aspects.

[0048] The technical solution of the present invention uses a checkerboard calibration method to obtain the internal parameters of an industrial camera, obtain a fringe pattern adapted to the display screen and projector, and then, based on the fringe pattern, obtain the correspondence between the pixels of the industrial camera, display screen, and projector. Distance information is obtained based on the internal parameters and this correspondence, and three-dimensional coordinates are obtained based on the internal parameters, correspondence, and distance information. A three-dimensional image is then obtained based on these three-dimensional coordinates. Thus, by building a structured light 3D reconstruction system, the calibration process between the industrial camera and projector is simplified, and the geometric constraints of the relative height model are reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0050] FIG1 is a schematic diagram of a comparative phase height model;

[0051] FIG2 is a structural diagram of a structured light 3D reconstruction system according to an embodiment of the present invention;

[0052] 3 is a flow chart of a structured light 3D reconstruction method according to an embodiment of the present invention;

[0053] FIG4 is a schematic diagram of calibrating an industrial camera according to an embodiment of the present invention;

[0054] FIG5 is a schematic diagram of a rectangular area according to an embodiment of the present invention;

[0055] FIG6 is a schematic diagram of a longitudinal stripe pattern according to an embodiment of the present invention;

[0056] FIG7 is a schematic diagram of obtaining a corresponding relationship according to an embodiment of the present invention;

[0057] FIG8 is a flowchart of obtaining a corresponding relationship according to an embodiment of the present invention;

[0058] 9 is a flowchart of obtaining a first distance according to an embodiment of the present invention;

[0059] 10 is a flowchart of obtaining a second distance according to an embodiment of the present invention;

[0060] 11 is a structural diagram of a parallel-axis structured light vision system according to an embodiment of the present invention;

[0061] FIG12 is a schematic diagram of the structured light 3D reconstruction principle according to an embodiment of the present invention;

[0062] FIG13 is a schematic diagram of a structured light 3D reconstruction apparatus according to an embodiment of the present invention;

[0063] FIG14 is a schematic diagram of a data processing device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The present application is described below based on the following embodiments, but the present application is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without the description of these details. To avoid obscuring the essence of the present application, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0065] Furthermore, persons of ordinary skill in the art will appreciate that the figures provided herein are for illustration purposes only and are not necessarily drawn to scale.

[0066] Unless the context clearly requires otherwise, words like “include”, “comprising” and the like throughout this application should be interpreted as including rather than exclusive or exhaustive; that is, as meaning “including but not limited to”.

[0067] In the description of this application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood to indicate or imply relative importance. In addition, in the description of this application, unless otherwise specified, "plurality" means two or more.

[0068] The three-dimensional shape and texture information of an object's surface can provide important applications in fields such as industrial manufacturing, virtual reality, medical imaging, art sculpture, and cultural heritage preservation. Common methods for acquiring three-dimensional shape information include time-of-flight cameras, binocular stereo vision, laser scanning, and structured light 3D reconstruction technology, with measurement accuracy ranging from a few millimeters to several meters.

[0069] Structured light 3D reconstruction technology is categorized into binocular structured light and monocular structured light. Binocular structured light adds a projector to binocular stereo vision, while monocular structured light replaces one of the cameras in binocular stereo vision with a projector. Compared to binocular stereo vision, structured light 3D reconstruction technology can reconstruct objects with very simple surface textures, making it much more powerful than binocular stereo vision. In structured light 3D reconstruction, the projector projects images such as binary code, color fringe patterns, and sinusoidal fringe patterns. The projected image encodes the object surface, enabling 3D reconstruction of a single object surface.

[0070] In monocular structured light 3D reconstruction, 3D reconstruction of object surfaces falls into two main categories. The first utilizes the triangular relationship between the camera and projector, employing a triangular stereo model for 3D reconstruction. The second utilizes the phase height model that exists when the optical axes of the industrial camera and projector are parallel. For the first category, the intrinsic and extrinsic parameters of the industrial camera and projector, as well as the rotation and translation matrices between the two, must be calibrated. For the second category, the phase height model requires not only that the optical axes of the industrial camera and projector be parallel and perpendicular to the reference plane, but also that the distances d1 and d2 from the optical centers of the industrial camera and projector (i.e., Op and Oc) are equal from the reference plane, as shown in Figure 1. Here, Z is the pixel position of the projector, Y is the pixel position of the industrial camera, W is the pixel position of the industrial camera pixel Y relative to the reference plane, U is the pixel position of the projector pixel Z relative to the reference plane, and d is the height of the object under test perpendicular to the reference plane. It can be seen that the triangulated stereo model has the problem of complicated calibration procedures, and the traditional phase height model has the problem of difficulty in meeting strict geometric constraints.

[0071] FIG2 is a schematic diagram of a structured light 3D reconstruction system according to an embodiment of the present invention. In the embodiment shown in FIG2 , the structured light 3D reconstruction system includes a projector 11, an industrial camera 12, a display screen 13, a data processing device 14, and a slide rail, wherein the slide rail includes a first slide rail 15 and a second slide rail 16. The projector 11 is mounted on the first slide rail 15, and the industrial camera 12 is mounted on the second slide rail 16. The optical axes of the projector 11 and the industrial camera 12 are perpendicular to the display screen 13, and the optical axes of the projector 11 and the industrial camera 12 are parallel to each other. The data processing device 14 connects the projector 11 and the industrial camera 12. The data processing device 14 includes a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the structured light 3D reconstruction method according to an embodiment of the present invention.

[0072] The optical projector 11 is used to project structured light onto a display screen 13 or an object to be measured. Specifically, the optical projector 11 is mounted on a first slide rail 15, and the optical projector 11 generally includes a light source and a structured light generator. The structured light generator can modulate the light emitted by the light source into light with a specific structure, such as stripes, patterns, etc. The optical projector 11 projects the generated structured light onto the object to be measured or the display screen, and makes the area of ​​the stripe pattern projected by the optical projector 11 smaller than the area of ​​the display screen 13. During the projection process, the optical projector 11 generally focuses and adjusts the light through optical elements such as lenses to ensure that the structured light can be accurately projected onto the target surface. When the structured light is projected onto the object to be measured or the display screen, it interacts with the surface. For the object to be measured, the structured light will produce different reflections and scatterings according to the shape and surface characteristics of the object, thereby forming an image reflecting the shape of the object. For the display screen, the structured light will interact with the pixels on the display screen to form an image with a specific structure. The structured light image on the object to be measured or the display screen can be obtained through an image acquisition device. These images contain information such as the shape and distribution of structured light, which can be used for subsequent image processing or three-dimensional reconstruction.

[0073] The industrial camera 12 is used to obtain the structured light projected by the projector 11 onto the display screen 13 and the structured light projected by the projector 11 onto the object to be measured. Specifically, the industrial camera 12 is installed on the second slide rail 16 to ensure that the relative position between it and the projector 11, the display screen 13 or the object to be measured is accurate and the position of the industrial camera 12 is adjusted so that the display screen 13 is completely included in the field of view. According to the characteristics of the projector 11, the display screen 13 or the object to be measured, the parameters of the industrial camera 12, such as focal length, exposure time, gain, etc., are calibrated to ensure that the structured light image can be clearly obtained. The structured light image projected by the projector 11 onto the display screen 13 or the object to be measured is captured by the industrial camera 12.

[0074] Display screen 13 is used to display stripe patterns or structured light projected by projector 11. Specifically, a film is applied to the surface of display screen 13. When display screen 13 is turned on, the content displayed on display screen 13 is projected externally, allowing industrial camera 12 to capture the content displayed on display screen 13. Furthermore, when display screen 13 is turned off, when projector 11 emits structured light toward display screen 13, display screen 13 is able to reflect the structured light from projector 11.

[0075] The first slide rail 15 and the second slide rail 16 are used to adjust the relative position of the projection light machine 11 or the industrial camera 12 with respect to the display screen 13 .

[0076] Furthermore, the first slide rail 15 and the second slide rail 16 are arranged to be perpendicular to the plane where the display screen 13 is located.

[0077] The data processing device 14 can be implemented by various devices or terminals with data processing functions. For example, the data processing device 14 can be implemented by a server, a laptop computer, a desktop computer, a tablet computer, a mobile phone or other dedicated data processing terminals.

[0078] The present invention uses a checkerboard calibration method to obtain the internal parameters of an industrial camera, a fringe pattern adapted to the display screen and projector, and then uses the fringe pattern to determine the correspondence between the pixels of the industrial camera, display screen, and projector. Distance information is then obtained based on the internal parameters and the correspondence. Three-dimensional coordinates are then obtained based on the internal parameters, the correspondence, and the distance information. A three-dimensional image is then obtained based on the three-dimensional coordinates. Thus, by building a structured light 3D reconstruction system, the calibration process between the industrial camera and projector is simplified, and the geometric constraints of the relative height model are reduced.

[0079] FIG3 is a flow chart of a structured light 3D reconstruction method according to an embodiment of the present invention. As shown in FIG3 , the structured light 3D reconstruction method according to this embodiment is applicable to a data processing device 14 , which includes a memory and a processor. The memory is configured to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the following steps:

[0080] Step S100: Obtain internal parameters of the industrial camera through a checkerboard calibration method.

[0081] Figure 4 is a schematic diagram of calibrating an industrial camera according to an embodiment of the present invention. As shown in Figure 4, the industrial camera 12 calibrates its internal parameters by photographing a standard checkerboard. The standard checkerboard can be achieved by printing an image or displaying an image on a screen.

[0082] In this embodiment, a checkerboard image is displayed. The checkerboard image, as shown in FIG4 , is a black and white checkered square. The positions of the corner points in the checkerboard image are known. An industrial camera is used to capture the checkerboard image from different angles and distances. A data processing device acquires the checkerboard image captured by the industrial camera and detects the checkerboard corners using a corner extraction algorithm. Based on the detected corner point positions, a homography matrix between the checkerboard plane and the image plane is calculated. An analytical solution estimation method is used to calculate the camera's internal parameters based on the homography matrix and the camera's imaging model. The internal parameters include focal length, principal point coordinates, etc.

[0083] In an optional implementation, a checkerboard image is first printed on paper, the printed checkerboard image is attached to a stable plane, and then an industrial camera is used to photograph the checkerboard image from different angles and distances.

[0084] In another optional implementation, an electronic checkerboard image is set by a data processing device, the display screen 13 is controlled to display the checkerboard image, and then an industrial camera is used to shoot the checkerboard image from different angles and distances.

[0085] Step S200: Obtain a stripe pattern adapted to the display screen and the projection light engine.

[0086] In this embodiment, the data processing device obtains Gray code encoding data and phase shift encoding data, the display screen displays the stripe pattern according to the Gray code encoding data and phase shift encoding data, and the projection light engine projects the stripe pattern according to the Gray code encoding data and phase shift encoding data.

[0087] The fringe pattern includes a Gray code pattern and a phase-shift pattern. More specifically, the Gray code pattern and the phase-shift pattern include a horizontal Gray code pattern and a phase-shift pattern, as well as a vertical Gray code pattern and a phase-shift pattern. Taking the generation of the vertical Gray code pattern and the phase-shift pattern as an example, the data processing device obtains a predetermined rectangular area, divides the rectangular area into 2M equal parts vertically, where M is a positive integer greater than or equal to 1, generates an M-bit vertical Gray code pattern, and then obtains the vertical phase-shift pattern using an N-step phase shift method.

[0088] Specifically, the data processing device acquires a stripe pattern adapted to a display screen and a projection light machine by setting an area, and the specific implementation method includes the following steps:

[0089] Step S201: Acquire a predetermined rectangular area.

[0090] Figure 5 is a schematic diagram of a rectangular region according to an embodiment of the present invention. In the embodiment shown in Figure 5, each small rectangular region represents a pixel, with each pixel having a length of V and a width of H. Two pixel points, [u1, v1] and [u2, v2], are selected, and the gray region connected by the diagonal line between these two pixel points constitutes the rectangular region. A pixel, also known as a pixel point, is the smallest unit that composes a digitized image. In digital image processing, it serves as the sampling point when scanning and digitizing an analog image.

[0091] In some embodiments, the data processing device can import an image processing library, such as OpenCV (a cross-platform computer vision library released under an open source license) or PIL (Python Image Library, Python, a graphics processing library). The graphics processing library sets the rectangular area with the line connecting the pixel points [u1, v1] and [u2, v2] as the diagonal line as the stripe pattern appearance area, and sets the remaining pixels to 0.

[0092] Step S202: Divide the rectangular area into 2M equal parts in the horizontal direction or in the vertical direction, where M is a positive integer greater than or equal to 1.

[0093] In this embodiment, the data processing device can calculate the width and length of each portion through the graphics processing library, and divide the rectangular area into 2M equal portions in the longitudinal direction as required through the function of the graphics processing library, where M is a positive integer greater than or equal to 1.

[0094] Step S203: Generate a horizontal or vertical M-bit Gray code image.

[0095] In this embodiment, the generation of a vertical Gray code diagram is used as an example. Since a rectangular area is required to be divided vertically into 2M equal parts, a vertical Gray code is generated based on the rectangular area. Gray code is an absolute encoding scheme. A typical Gray code is a single-step self-complementary code with reflective and cyclic properties. In a Gray code, only one binary digit differs between two adjacent codes. When generating the vertical Gray code, a vertical Gray code sequence containing M elements is sequentially generated according to the Gray code generation algorithm.

[0096] Taking the vertical Gray code as an example, create an array of length 2M to store the vertical Gray code sequence. Initialize the array to 0, indicating that all grids are blank at the beginning. Then traverse the Gray code sequence in sequence. For each Gray code, plot its corresponding binary bit to the corresponding position of the one-dimensional array to generate an M-bit vertical Gray code diagram.

[0097] Taking M=3 as an example, the array with a length of 2M=8 is:

[0098] (0,0,0),(0,0,1),(0,1,0),(0,1,1),(1,0,0),(1,0,1),(1,1,0),(1,1,1).

[0099] When M=3, three Gray code images are generated. Each Gray code image is divided into 8 parts vertically. The codes corresponding to the 8 parts of the first Gray code image are the combination of the first elements in the above 8 arrays, that is, (0, 0, 0), (0, 0, 1), (0, 1, 0), the part shown by (0, 1, 1) is black, and the parts shown by (1, 0, 0), (1, 0, 1), (1, 1, 0), and (1, 1, 1) are white; the codes corresponding to the 8 parts of the second Gray code image are the combination of the second elements in the above 8 arrays, that is, (0, 0, 0), (0, 0, 1) are black, and the parts shown by (0, 1, 0), (1, 0, 1), (1, 1, 0), and (1, 1, 1) are white. The portion shown by (0, 1, 1) is white, the portions shown by (1, 0, 0) and (1, 0, 1) are black, and the portions shown by (1, 1, 0) and (1, 1, 1) are white. The eight copies of the third Gray code image correspond to the combination of the third elements in the above eight arrays, that is, the portion shown by (0, 0, 0) is black, the portion shown by (0, 0, 1) is white, the portion shown by (0, 1, 0) is black and the portion shown by (0, 1, 1) is white, the portion shown by (1, 0, 0) is black and the portion shown by (1, 0, 1) is white, and the portion shown by (1, 1, 0) is black and the portion shown by (1, 1, 1) is white. Each set of arrays differs by one bit, with 0 representing black and 1 representing white. The three Gray code images generated can be shown in the patterns P1-P3 as shown in Figure 6.

[0100] Based on the same principle, when M=5, five Gray code images P1-P5 shown in FIG6 can be generated.

[0101] Step S204: Obtain a phase shift diagram according to the N-step phase shift method.

[0102] Specifically, for each pixel point (u, v) within the rectangular area, the first light intensity value of the pixel point on the nth phase shift image is obtained according to the following formula:

[0103] Among them, A(u,v) is the background light intensity, B(u,v) is the modulation amplitude, is the phase shift step size, and n represents the current phase shift cycle number. By formula Get, where i is the coordinate value v of the pixel in the image coordinate system, corresponding to the vertical coordinate of the image pixel, and width represents the number of columns of pixels in the phase shift map.

[0104] In this embodiment, corresponding horizontal Gray code patterns and phase shift patterns can be generated based on the above method for generating the vertical Gray code patterns and phase shift patterns.

[0105] In this embodiment, the data processing device can transmit the generated horizontal and vertical Gray code patterns and phase shift patterns to the projector and display screen via a data transmission line or network, and simultaneously control parameters such as the power on / off, brightness, and contrast of the projector and display screen via control signals. Specifically, a stripe pattern adapted to the display screen can be obtained based on the resolution of the display screen, and a stripe pattern adapted to the projector can be obtained based on the resolution of the projector. For example, when the resolution of the display screen is 1920*1080, the size of the stripe pattern generated to adapt to the display screen is also 1920*1080. When the resolution of the pattern projected by the projector is 912*1140, the size of the adapted stripe pattern is also 912*1140.

[0106] Figure 6 is a schematic diagram of a longitudinal stripe pattern according to an embodiment of the present invention. As shown in Figure 6 , stripe patterns P1-P5 are longitudinal Gray code patterns, and stripe patterns P6-P9 are longitudinal phase shift patterns generated based on the longitudinal Gray code patterns.

[0107] Specifically, P1-P5 encode 32 pixels according to the 5-bit Gray code. Each pixel in the image is encoded with Gray code. The encoding of the first Gray code image is (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1 ... The encoding of the fourth Gray code image is (0,0,1,1,1,1,0,0,0,0,1,1,1,1,0,0,0,0,1,1,1,1,0,0,0,0,1,1,1,1,0,0); the encoding of the fifth Gray code image is (0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0). P6 is a phase-shifted diagram of Gray code diagram P1 shifted by a quarter of a cycle, P7 is a phase-shifted diagram of Gray code diagram P2 shifted by a quarter of a cycle, P8 is a phase-shifted diagram of Gray code diagram P3 shifted by a quarter of a cycle, P9 is a phase-shifted diagram of Gray code diagram P5 shifted by a quarter of a cycle, and P10 is a phase-shifted diagram of Gray code diagram P5 shifted by a quarter of a cycle, where a quarter of a cycle is π / 4.

[0108] Step S300: Obtain the corresponding relationship between the pixels of the industrial camera, the display screen and the projector according to the stripe pattern.

[0109] In this embodiment, the correspondence includes a first correspondence between industrial camera pixels and display screen pixels, a second correspondence between industrial camera pixels and projection optical machine pixels, and a third correspondence between display screen pixels and projection optical machine pixels.

[0110] Figure 7 is a schematic diagram of obtaining a corresponding relationship according to an embodiment of the present invention. As shown in Figure 7, a display screen displays 9 vertical stripe patterns, an industrial camera obtains 9 vertical stripe patterns, and a projection machine projects 9 vertical stripe patterns.

[0111] Specifically, the display screen displays the stripe pattern in the order of P1-P9 according to the acquired P1-P5 Gray code image and P6-P9 phase shift image, and the industrial camera acquires the P1-P5 Gray code image and P6-P9 phase shift image displayed on the display screen in sequence.

[0112] The projection optical machine adjusts its internal parameters according to the obtained P1-P5 Gray code diagram and P6-P9 phase shift diagram until it can correctly project the stripe pattern, and projects the stripe pattern in sequence according to the order of P1-P9. The industrial camera sequentially obtains the P1-P5 Gray code diagram and P6-P9 phase shift diagram projected by the projection optical machine.

[0113] In this embodiment, obtaining the corresponding relationship between the pixels of the industrial camera, the display screen, and the projector according to the stripe pattern includes:

[0114] The stripe pattern is displayed on the display screen, and a data processing device decodes the image of the display screen acquired by the industrial camera to obtain a first absolute phase value. The data processing device decodes the image displayed on the display screen to obtain a second absolute phase value. The data processing device matches the first absolute phase value and the second absolute phase value to obtain the first corresponding relationship.

[0115] The stripe pattern is projected by a projection light machine, and a data processing device decodes the image of the projection light machine obtained by the industrial camera to obtain a third absolute phase value. The data processing device decodes the image projected by the projection light machine to obtain a fourth absolute phase value. The data processing device matches the third absolute phase value and the fourth absolute phase value to obtain the second corresponding relationship.

[0116] The data processing device obtains the third corresponding relationship according to the first corresponding relationship and the second corresponding relationship.

[0117] In some embodiments, the acquired Gray code image is binarized. This typically involves selecting an appropriate threshold, comparing the grayscale value of each pixel with the threshold, and then assigning the pixel a value of 0 or 1 based on the comparison result. A similar binarization process is also required for the phase-shifted image. The grayscale value of each pixel is compared with a threshold, and then assigned a value of 0 or 1 based on the comparison result. The binarized Gray code image and the phase-shifted image are then pixel-matched.

[0118] The absolute phase value refers to the phase state of each pixel in the absolute phase image. Phase is a parameter that describes the periodic variation of a wave, while the absolute phase value is the phase value at a specific reference point. An absolute phase image is an image used to represent the phase information of a wave. In an absolute phase image, the phase value of each pixel is calculated based on a decoding algorithm and represents the position of that pixel in the periodic variation of the wave. This phase value is the relative position within a cycle and is usually expressed in radians or degrees. By matching the pixels in the absolute phase image, the corresponding relationship between them can be determined, allowing for further analysis of phenomena such as wave propagation and interference.

[0119] In some embodiments, the correspondence is obtained through pixel matching. For two absolute phase images, the phase values ​​of each pixel can be compared one by one to find pixels with identical or similar phase values, thereby determining the correspondence between them. If noise interference is present, direct pixel-by-pixel matching is difficult. In this case, a feature point matching method can be used. First, feature points (such as edges, corners, etc.) are extracted from the two absolute phase images. These feature points are then matched to find identical or similar feature points, thereby determining the correspondence between them. Alternatively, the correspondence can be obtained through template matching. A subregion is selected as a template, and then a region matching the template is searched in the other absolute phase image. If a matching region is found, the correspondence between the two regions can be determined. Alternatively, Fourier transform matching can be used. Fourier transform is a commonly used image processing method that converts images from the spatial domain to the frequency domain. In the frequency domain, the phase information of the image can be more easily compared and matched. Therefore, the two absolute phase images can be Fourier transformed first, and then matched in the frequency domain to find identical or similar frequency components, thereby determining the correspondence between them.

[0120] Step S400: Acquire distance information according to the internal parameters and the corresponding relationship.

[0121] In this embodiment, the distance information includes a first distance between the optical center of the industrial camera and the display screen, a second distance between the optical center of the projector and the display screen, and a third distance between the optical axis of the industrial camera and the optical axis of the projector.

[0122] The acquiring of distance information according to the internal parameters and the corresponding relationship includes:

[0123] Step S410: The data processing device obtains the focal length of the industrial camera and the first pixel corresponding to the principal point of the industrial camera according to the internal parameters, the data processing device determines the second pixel on the display screen corresponding to the first pixel according to the first correspondence, selects a third pixel from the industrial camera pixels, the data processing device determines the fourth pixel on the display screen corresponding to the third pixel according to the first correspondence, and the data processing device obtains the first distance based on the first pixel, the second pixel, the third pixel, and the fourth pixel.

[0124] In this embodiment, the data processing device obtains the first distance by calculating using the following formula:

[0125] Among them, f c is the focal length of the industrial camera, [u c0 ,v c0 ] is the first pixel, [u sc ,v sc ] is the second pixel, [u c1 ,v c1 ] is the third pixel, [u s1 ,v s1 ] is the fourth pixel, H c and V c The length and width of the pixel of the industrial camera.

[0126] Step S420: The data processing device obtains the focal length of the projection light machine and the fifth pixel corresponding to the principal point of the projection light machine according to the internal parameters, the data processing device determines the sixth pixel on the display screen corresponding to the fifth pixel according to the third corresponding relationship, the data processing device selects a seventh pixel from the projection light machine pixels, the data processing device determines the eighth pixel on the display screen corresponding to the seventh pixel according to the third corresponding relationship, and the data processing device obtains the second distance based on the fifth pixel, the sixth pixel, the seventh pixel and the eighth pixel.

[0127] In this embodiment, the data processing device obtains the second distance by calculating using the following formula:

[0128] Among them, f p is the focal length of the projection machine, [u p0 ,v p0 ] is the fifth pixel, [u sp ,v sp ] is the sixth pixel, [u p1 ,v p1] is the seventh pixel, [u s2 ,v s2 ] is the eighth pixel, H s 、V s H is the length and width of the display pixel. p 、V p They are respectively the length and width of the pixel of the projection optical machine.

[0129] Step S430: The data processing device obtains the third distance according to the second pixel and the sixth pixel.

[0130] In this embodiment, the data processing device obtains the third distance by calculating the following formula:

[0131] Among them, [u sc ,v sc ] is the second pixel, [u sp ,v sp ] is the sixth pixel, H s 、V s The length and width of the display pixel.

[0132] Step S500: Acquire three-dimensional coordinates according to the internal parameters, correspondence and distance information.

[0133] In this embodiment, internal parameters are obtained through step S100, and the structured light three-dimensional reconstruction system is calibrated through steps S200-S400 to obtain correspondence and distance information. After that, the object to be measured is placed within the coverage range of the industrial camera and the projector, and structured light is projected onto the object to be measured through the projector. The industrial camera captures an image of the object to be measured and sends it to a data processing device. The data processing device obtains the three-dimensional coordinates of the object to be measured based on the internal parameters, correspondence, distance information and image.

[0134] Specifically, the data processing device obtains a point to be measured from the image. Since a portion of the image is the object to be measured and another portion is not the object to be measured, the point to be measured is a point on the object to be measured. The ninth pixel of the projector and the tenth pixel of the industrial camera are determined for the point to be measured. The data processing device obtains the eleventh pixel on the display screen corresponding to the ninth pixel and the twelfth pixel on the display screen corresponding to the tenth pixel based on the corresponding relationship. The data processing device obtains the three-dimensional coordinates of the point to be measured based on the distance information, the ninth pixel, the tenth pixel, the eleventh pixel, and the twelfth pixel. The three-dimensional coordinates are the three-dimensional coordinates of the point to be measured relative to the display screen.

[0135] In this embodiment, the data processing device obtains the three-dimensional coordinates using the following formula:

[0136] Among them, [u p2 ,v p2 ] is the ninth pixel, [u c2 ,v c2 ] is the tenth pixel, [u s3 ,v s3 ] is the eleventh pixel, [u s4 ,v s4 ] is the twelfth pixel, [u s5 ,v s5 ] is the vertical mapping display screen pixel of the point to be measured, H s 、V s L is the length and width of the display pixel. c is the first distance, L p is the second distance.

[0137] Repeat the above steps to extract different points to be measured in the image and obtain the corresponding 3D coordinates. At the same time, by adjusting the direction of the object to be measured, continuously obtaining images in different directions, repeating the above steps to extract different points to be measured in the image and obtain the corresponding 3D coordinates, the 3D coordinates of each point on the object to be measured can be obtained, and then the 3D shape of the object to be measured can be obtained.

[0138] Step S600: Acquire a three-dimensional image according to the three-dimensional coordinates.

[0139] In some embodiments, the data processing device can import the acquired 3D data into 3D visualization software or a library, convert the data into a format supported by the software or library, and then call appropriate functions to load it. After importing the 3D data, the visualization software or library's functions are used to set properties such as view, color, and lighting to create a 3D scene. The 3D scene is then rendered into an image using a rendering engine provided by the visualization software or library, employing advanced rendering techniques such as ray tracing, shadows, and texturing.

[0140] The present invention uses a checkerboard calibration method to obtain the internal parameters of an industrial camera, a fringe pattern adapted to the display screen and projector, and then uses the fringe pattern to determine the correspondence between the pixels of the industrial camera, display screen, and projector. Distance information is then obtained based on the internal parameters and the correspondence. Three-dimensional coordinates are then obtained based on the internal parameters, the correspondence, and the distance information. A three-dimensional image is then obtained based on the three-dimensional coordinates. Thus, by building a structured light 3D reconstruction system, the calibration process between the industrial camera and projector is simplified, and the geometric constraints of the relative height model are reduced.

[0141] FIG8 is a flowchart of obtaining a correspondence relationship according to an embodiment of the present invention. As shown in FIG8 , the correspondence relationship between the pixels of the industrial camera, the display screen, and the projection light engine is obtained, and the correspondence relationship includes a first correspondence relationship between the pixels of the industrial camera and the pixels of the display screen, a second correspondence relationship between the pixels of the industrial camera and the pixels of the projection light engine, and a third correspondence relationship between the pixels of the display screen and the pixels of the projection light engine.

[0142] In this embodiment, obtaining the correspondence between the pixels of the industrial camera, the display screen, and the projector according to the fringe pattern includes the following steps:

[0143] Step S301: displaying the stripe pattern on the display screen, decoding the image of the display screen acquired by the industrial camera to obtain a first absolute phase value, and decoding the image displayed on the display screen to obtain a second absolute phase value.

[0144] Step S302: Match the first absolute phase value and the second absolute phase value to obtain the first corresponding relationship.

[0145] Step S303: project the fringe pattern through a projection optical machine, decode the image of the projection optical machine acquired by the industrial camera to obtain a third absolute phase value, and decode the image projected by the projection optical machine to obtain a fourth absolute phase value.

[0146] Step S304: Match the third absolute phase value and the fourth absolute phase value to obtain the second corresponding relationship.

[0147] Step S305: Acquire the third corresponding relationship according to the first corresponding relationship and the second corresponding relationship.

[0148] In some embodiments, the data processing device needs to pre-process the acquired image to remove noise and improve image quality to facilitate subsequent feature recognition and pattern matching. Pre-processing methods may include filtering, contrast enhancement, etc. In the pre-processed image, patterns or patterns with specific features are identified. These features may include specific shapes, colors, or textures. Methods for identifying features may include template matching, feature point detection, etc. The position and direction of features in the image are determined by pattern matching. Pattern matching methods may include feature point-based matching algorithms, template-based matching algorithms, etc. Based on the recognition and matching results, the absolute phase value of each pixel or area can be determined. The specific method may vary depending on the encoding method used. For example, for an encoding method based on Gray code, the absolute phase value of each pixel or area can be determined by table lookup or calculation.

[0149] In this embodiment, the distance information obtained between the industrial camera, the display screen and the projector light machine includes the first distance between the optical center of the industrial camera and the display screen, the second distance between the optical center of the projector light machine and the display screen, and the third distance between the optical axis of the industrial camera and the optical axis of the projector light machine.

[0150] FIG9 is a flow chart of obtaining a first distance according to an embodiment of the present invention. As shown in FIG9 , obtaining a first distance between the optical center of the industrial camera and the display screen includes the following steps:

[0151] Step S411: Acquire the focal length of the industrial camera and the first pixel corresponding to the principal point of the industrial camera according to the internal parameters.

[0152] In this embodiment, the data processing device determines the principal point of the industrial camera by obtaining the focal length of the industrial camera, and determines that the principal point of the industrial camera is located at the first pixel [u c0 ,v c0 ] place.

[0153] Step S412: Determine a second pixel on the display screen corresponding to the first pixel according to the first corresponding relationship.

[0154] In this embodiment, the data processing device determines the first pixel [u c0 ,v c0 ] corresponds to the second pixel of the display screen [u sc ,v sc ].

[0155] Step S413: Select a third pixel from the pixels of the industrial camera, and determine a fourth pixel on the display screen corresponding to the third pixel according to the first corresponding relationship.

[0156] In this embodiment, the data processing device takes the third pixel [u c1 ,v c1 ], according to the first corresponding relationship, determine the third pixel [u c1 ,v c1 ] corresponds to the fourth pixel of the display screen [u s1 ,v s1 ].

[0157] Step S414: Acquire the first distance according to the first pixel, the second pixel, the third pixel, and the fourth pixel.

[0158] In this embodiment, the data processing device obtains the first distance using the following formula:

[0159] Among them, f c is the focal length of the industrial camera, [u c0 ,vc0 ] is the first pixel, [u sc ,v sc ] is the second pixel, [u c1 ,v c1 ] is the third pixel, [u s1 ,v s1 ] is the fourth pixel, H c and V c H is the length and width of the industrial camera pixel. s 、V s The length and width of the display pixel.

[0160] FIG10 is a flow chart of obtaining the second distance according to an embodiment of the present invention. As shown in FIG10 , obtaining the second distance between the optical center of the projector and the display screen includes the following steps:

[0161] Step S421: Acquire the focal length of the projection light engine and the fifth pixel corresponding to the principal point of the projection light engine according to the internal parameters.

[0162] In this embodiment, the data processing device determines the principal point of the projection light machine by obtaining the focal length of the projection light machine, and determines that the principal point of the projection light machine is located at the fifth pixel [u p0 ,v p0 ] place.

[0163] Step S422: Determine the sixth pixel on the display screen corresponding to the fifth pixel according to the third corresponding relationship.

[0164] In this embodiment, the data processing device determines the fifth pixel [u p0 ,v p0 ] corresponds to the sixth pixel of the display screen [u sp ,v sp ].

[0165] Step S423: Select a seventh pixel from the projection light engine pixels, and determine an eighth pixel on the display screen corresponding to the seventh pixel according to the third corresponding relationship.

[0166] In this embodiment, the data processing device takes any seventh pixel [u p1 ,v p1 ], according to the first corresponding relationship, determine the seventh pixel [u p1 ,v p1 ] corresponds to the eighth pixel of the display screen [u s2 ,v s2 ].

[0167] Step S424: Acquire the second distance according to the fifth pixel, the sixth pixel, the seventh pixel, and the eighth pixel.

[0168] In this embodiment, the data processing device obtains the second distance using the following formula:

[0169] Among them, f p is the focal length of the projection machine, [u p0 ,v p0 ] is the fifth pixel, [u sp ,v sp ] is the sixth pixel, [u p1 ,v p1 ] is the seventh pixel, [u s2 ,v s2 ] is the eighth pixel, H s 、V s H is the length and width of the display pixel. p 、V p They are the length and width of the projection optical machine pixel respectively.

[0170] In this embodiment, the data processing device obtains the third distance between the optical axis of the industrial camera and the optical axis of the projection optical machine according to the second pixel and the sixth pixel.

[0171] Specifically, the data processing device determines the second pixel [u sc ,v sc ] corresponds to the sixth pixel of the display screen [u sp ,v sp ], and obtain the third distance using the following formula:

[0172] Among them, [u sc ,v sc ] is the second pixel, [u sp ,v sp ] is the sixth pixel, H s 、V s The length and width of the display pixel.

[0173] Figure 11 is a block diagram of a parallel-axis structured light vision system according to an embodiment of the present invention. As shown in Figure 11, the vision system according to an embodiment of the present invention includes a projector 11, an industrial camera 12, and a display screen 13. The projector 11 includes a lens 111, and the industrial camera 12 includes a lens 121. The projector and industrial camera are positioned in the same orientation and perpendicularly across from the display screen.

[0174] In the embodiment of the present invention, the main point position of the projection optical machine 11 is [u p0 ,v p0 ], that is, the fifth pixel; the main point of the industrial camera 12 is [u c0 ,v c0], that is, the first pixel; the main point [u p0 ,v p0 ] and the lens 111 is the focal length fp of the projection optical machine, and the main point [u c0 ,v c0 ] and the lens 121 is the focal length fc of the industrial camera. The optical axes of the lens 111 of the projector 11 and the lens 121 of the industrial camera 12 are parallel to each other and perpendicular to the display screen 13. After the display screen 13 and the projector 11 obtain the stripe pattern and the industrial camera 12 calibrates the internal parameters, the object to be measured is placed in the parallel axis structured light vision system, the projector projects the structured light onto the object to be measured 17, the industrial camera 12 obtains the structured light projected onto the object to be measured 17, and the data processing equipment decodes the structured light. The three-dimensional coordinates of the object to be measured are obtained based on the correspondence between the display screen pixels, the projector pixels and the industrial camera pixels and the distance information between the devices. A three-dimensional image of the object to be measured can be obtained based on the three-dimensional coordinates.

[0175] The present invention uses a checkerboard calibration method to obtain the internal parameters of an industrial camera, a fringe pattern adapted to the display screen and projector, and then uses the fringe pattern to determine the correspondence between the pixels of the industrial camera, display screen, and projector. Distance information is then obtained based on the internal parameters and the correspondence. Three-dimensional coordinates are then obtained based on the internal parameters, the correspondence, and the distance information. A three-dimensional image is then obtained based on the three-dimensional coordinates. Thus, by building a structured light 3D reconstruction system, the calibration process between the industrial camera and projector is simplified, and the geometric constraints of the relative height model are reduced.

[0176] FIG12 is a schematic diagram of the structured light 3D reconstruction principle of an embodiment of the present invention. As shown in FIG12 , A is the point to be measured, B is the pixel of the industrial camera [u c2 ,v c2 ], C is the projection optical pixel [u p2 ,v p2 ], D is the distance between the optical axis of the industrial camera and the projection optical machine, E is the pixel of the industrial camera [u c2 ,v c2 ] corresponds to the pixel [u s4 ,v s4 ], F is the main point pixel of the industrial camera [u c0 ,v c0 ] corresponds to the pixel of the display screen [u sc ,v sc ], J is the pixel of the vertically mapped display screen of point A [u s5 ,v s5 ], R is the optical center of the projection machine, I is the main point pixel of the projection machine [u p0 ,v p0]The corresponding display pixel is [u sp ,v sp ], G is the optical center of the industrial camera, K is the main point pixel of the industrial camera [u c0 ,v c0 ], L is the main point pixel of the projection optical machine [u p0 ,v p0 ], M is the pixel of industrial camera [u c2 ,v c2 ] corresponds to the pixel [u s3 ,v s3 ].

[0177] In this embodiment, the focal length f of the industrial camera is obtained according to the internal parameters. c , according to the focal length f c Get point G, K principal point is located at pixel [u c0 ,v c0 ], according to the first corresponding relationship pixel [u c0 ,v c0 ]The corresponding display pixel is [u sc ,v sc ]; Take any industrial camera pixel [u c1 ,v c1 ], according to the first correspondence, any industrial camera pixel [u c1 ,v c1 ]The corresponding display pixel is [u s1 ,v s1 ]. According to the focal length of the projection machine is f p Get point R, the main point of R is located at pixel [u p0 ,v p0 ], according to the third correspondence pixel [u p0 ,v p0 ]The corresponding display pixel is [u sp ,v sp ]; Take any projection optical pixel [u p1 ,v p1 ], according to the third corresponding relationship, any projection optical pixel [u p1 ,v p1 ]The corresponding display pixel is [u s2 ,v s2 ], point A in space corresponds to the projection optical pixel [u p2 ,v p2 ], corresponding to the industrial camera pixel [u c2 ,v c2 ], according to the first and third correspondence pixel [u p2 ,v p2 ]、[u c2 ,v c2] respectively correspond to the display pixels [u s3 ,v s3 ]、[u s4 ,v s4 ], while point A in space vertically maps the display screen pixel [u s5 ,v s5 ].

[0178] The present invention uses a checkerboard calibration method to obtain the internal parameters of an industrial camera, a fringe pattern adapted to the display screen and projector, and then uses the fringe pattern to determine the correspondence between the pixels of the industrial camera, display screen, and projector. Distance information is then obtained based on the internal parameters and the correspondence. Three-dimensional coordinates are then obtained based on the internal parameters, the correspondence, and the distance information. A three-dimensional image is then obtained based on the three-dimensional coordinates. Thus, by building a structured light 3D reconstruction system, the calibration process between the industrial camera and projector is simplified, and the geometric constraints of the relative height model are reduced.

[0179] Figure 13 is a schematic diagram of a structured light 3D reconstruction apparatus according to an embodiment of the present invention. As shown in Figure 13, the structured light 3D reconstruction apparatus according to an embodiment of the present invention includes a calibration module 81 for acquiring internal parameters of an industrial camera using a checkerboard calibration method; a fringe pattern acquisition module 82 for acquiring a fringe pattern adapted to a display screen and a projector; a correspondence acquisition module 83 for acquiring the correspondence between pixels of the industrial camera, display screen, and projector based on the fringe pattern; a distance information acquisition module 84 for acquiring distance information based on the internal parameters and the correspondence; a 3D data acquisition module 85 for acquiring 3D coordinates based on the internal parameters, the correspondence, and the distance information; and a 3D image acquisition module 86 for acquiring a 3D image based on the 3D coordinates.

[0180] In some embodiments, the fringe pattern acquisition module 82 includes:

[0181] The predetermined area acquisition module is used to acquire a predetermined rectangular area.

[0182] The area division module is used to divide the rectangular area into 2M equal parts in the horizontal direction or the vertical direction, where M is a positive integer greater than or equal to 1.

[0183] The Gray code image acquisition module is used to generate a horizontal or vertical M-bit Gray code image.

[0184] The phase shift map acquisition module is used to acquire the phase shift map through the N-step phase shift method.

[0185] In some embodiments, the correspondence relationship acquisition module 83 includes:

[0186] A first correspondence acquisition module is used to display the stripe pattern through the display screen, decode the image of the display screen acquired by the industrial camera to obtain a first absolute phase value, decode the image displayed on the display screen to obtain a second absolute phase value, and match the first absolute phase value and the second absolute phase value to obtain the first correspondence.

[0187] A second correspondence acquisition module is used to project the stripe pattern through a projection light machine, decode the image of the projection light machine acquired by the industrial camera to obtain a third absolute phase value, decode the image projected by the projection light machine to obtain a fourth absolute phase value, and match the third absolute phase value and the fourth absolute phase value to obtain the second correspondence.

[0188] The third correspondence relationship acquisition module is configured to acquire the third correspondence relationship according to the first correspondence relationship and the second correspondence relationship.

[0189] In some embodiments, the distance information acquisition module 84 includes:

[0190] A first distance acquisition module is used to obtain the focal length of the industrial camera and the first pixel corresponding to the principal point of the industrial camera according to the internal parameters, determine the second pixel on the display screen corresponding to the first pixel according to the first corresponding relationship, select a third pixel from the industrial camera pixels, determine the fourth pixel on the display screen corresponding to the third pixel according to the first corresponding relationship, and obtain the first distance based on the first pixel, the second pixel, the third pixel and the fourth pixel.

[0191] A second distance acquisition module is used to obtain the focal length of the projection light machine and the fifth pixel corresponding to the principal point of the projection light machine according to the internal parameters, determine the sixth pixel on the display screen corresponding to the fifth pixel according to the third corresponding relationship, select a seventh pixel from the projection light machine pixels, determine the eighth pixel on the display screen corresponding to the seventh pixel according to the third corresponding relationship, and obtain the second distance based on the fifth pixel, the sixth pixel, the seventh pixel and the eighth pixel.

[0192] The third distance acquisition module is configured to acquire the third distance according to the second pixel and the sixth pixel.

[0193] In some embodiments, the three-dimensional data acquisition module 85 includes:

[0194] The ninth pixel and tenth pixel acquisition modules are used to determine the ninth pixel of the measured point in the projection optical machine and the tenth pixel in the industrial camera;

[0195] An eleventh pixel and a twelfth pixel acquisition module, configured to acquire, according to the corresponding relationship, an eleventh pixel on the display screen corresponding to the ninth pixel and a twelfth pixel on the display screen corresponding to the tenth pixel;

[0196] A three-dimensional coordinate acquisition module is used to acquire the three-dimensional coordinates of the point to be measured based on the distance information, the ninth pixel, the tenth pixel, the eleventh pixel and the twelfth pixel, where the three-dimensional coordinates are the three-dimensional coordinates of the point to be measured relative to the display screen.

[0197] The present invention uses a checkerboard calibration method to obtain the internal parameters of an industrial camera, a fringe pattern adapted to the display screen and projector, and then uses the fringe pattern to determine the correspondence between the pixels of the industrial camera, display screen, and projector. Distance information is then obtained based on the internal parameters and the correspondence. Three-dimensional coordinates are then obtained based on the internal parameters, the correspondence, and the distance information. A three-dimensional image is then obtained based on the three-dimensional coordinates. Thus, by building a structured light 3D reconstruction system, the calibration process between the industrial camera and projector is simplified, and the geometric constraints of the relative height model are reduced.

[0198] Figure 14 is a schematic diagram of a data processing device according to an embodiment of the present invention. The electronic device shown in Figure 14 is a general-purpose address query device, comprising a general-purpose computer hardware structure, including at least a processor 91 and a memory 92. Processor 91 and memory 92 are connected via a bus 93. Memory 92 is adapted to store instructions or programs executable by processor 91. Processor 91 can be a standalone microprocessor or a collection of one or more microprocessors. Thus, processor 91 executes the instructions stored in memory 92, thereby executing the method flow of the embodiment of the present invention described above to process data and control other devices. Bus 93 connects the aforementioned components together and also connects them to a display controller 94 and a display device, as well as an input / output (I / O) device 95. The I / O device 95 can be a mouse, keyboard, modem, network interface, touch input device, somatosensory input device, printer, or other devices known in the art. Typically, the I / O device 95 is connected to the system via an I / O controller 96.

[0199] It will be understood by those skilled in the art that the embodiments of the present application may be provided as methods, devices (equipment), or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0200] The present application is described with reference to flowcharts of methods, apparatuses (devices), and computer program products according to embodiments of the present application. It should be understood that each process in the flowcharts can be implemented by computer program instructions.

[0201] These computer program instructions can be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart.

[0202] These computer program instructions can also be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart.

[0203] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program, wherein the computer-readable program is used to enable a computer to execute part or all of the above method embodiments.

[0204] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by specifying relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0205] The foregoing is merely a preferred embodiment of the present application and is not intended to limit the present application. Persons skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application are intended to be within the scope of protection of the present application.

Claims

1. A structured light three-dimensional reconstruction method, characterized in that, The method includes: Obtaining the internal parameters of the industrial camera through the checkerboard calibration method; Obtaining the stripe pattern adapted to the display screen and the projection optical machine; Obtaining the corresponding relationship between the pixels of the industrial camera, the display screen and the projection optical machine according to the stripe pattern; Obtaining the distance information according to the internal parameters and the corresponding relationship; Obtaining the three-dimensional coordinates according to the internal parameters, the corresponding relationship and the distance information; Obtaining the three-dimensional image according to the three-dimensional coordinates.

2. The method according to claim 1, wherein The stripe pattern includes a Gray code pattern and a phase-shifted pattern; Among them, the obtaining of the stripe pattern adapted to the display screen and the projection optical machine includes: Obtaining a predetermined rectangular area; Dividing the rectangular area into 2M equal parts horizontally or vertically, where M is a positive integer greater than or equal to 1; Generating a Gray code pattern of M bits horizontally or vertically; Obtaining the phase-shifted pattern according to the N-step phase-shifting method.

3. The method according to claim 1, wherein The corresponding relationship includes a first corresponding relationship between the pixels of the industrial camera and the pixels of the display screen, a second corresponding relationship between the pixels of the industrial camera and the pixels of the projection optical machine, and a third corresponding relationship between the pixels of the display screen and the pixels of the projection optical machine; Among them, the obtaining of the corresponding relationship between the pixels of the industrial camera, the display screen and the projection optical machine according to the stripe pattern includes: Displaying the stripe pattern through the display screen, decoding the image of the display screen obtained by the industrial camera to obtain a first absolute phase value, decoding the image displayed by the display screen to obtain a second absolute phase value, and matching the first absolute phase value and the second absolute phase value to obtain the first corresponding relationship; Projecting the stripe pattern through the projection optical machine, decoding the image of the projection optical machine obtained by the industrial camera to obtain a third absolute phase value, decoding the image projected by the projection optical machine to obtain a fourth absolute phase value, and matching the third absolute phase value and the fourth absolute phase value to obtain the second corresponding relationship; Obtaining the third corresponding relationship according to the first corresponding relationship and the second corresponding relationship.

4. The method according to claim 3, wherein The distance information includes a first distance between the optical center of the industrial camera and the display screen, a second distance between the optical center of the projection optical machine and the display screen, and a third distance between the optical axis of the industrial camera and the optical axis of the projection optical machine; Among them, the obtaining of the distance information according to the internal parameters and the corresponding relationship includes: Obtaining the focal length of the industrial camera and the first pixel corresponding to the principal point of the industrial camera according to the internal parameters, determining the second pixel in the display screen corresponding to the first pixel according to the first corresponding relationship, selecting a third pixel in the industrial camera pixels, determining the fourth pixel in the display screen corresponding to the third pixel according to the first corresponding relationship, and obtaining the first distance according to the first pixel, the second pixel, the third pixel and the fourth pixel; Obtain the focal length of the projection optical machine and the fifth pixel corresponding to the principal point of the projection optical machine according to the internal parameters, determine the sixth pixel in the display screen corresponding to the fifth pixel according to the third correspondence relationship, select a seventh pixel in the projection optical machine pixels, determine the eighth pixel in the display screen corresponding to the seventh pixel according to the third correspondence relationship, and obtain the second distance according to the fifth pixel, the sixth pixel, the seventh pixel, and the eighth pixel; Obtain the third distance according to the second pixel and the sixth pixel.

5. The method according to claim 4, characterized in that The first distance is calculated and obtained through the following formula: Among them, f c is the focal length of the industrial camera, [u c0 , v c0 is the first pixel, [u sc , v sc is the second pixel, [u c1 , v c1 is the third pixel, [u s1 , v s1 is the fourth pixel, H c and V c are the length and width dimensions of the industrial camera pixels, H s , V s are the length and width dimensions of the display screen pixels.

6. The method according to claim 4, characterized in that, The second distance is calculated and obtained through the following formula: Among them, f p is the focal length of the projection optical machine, [u p0 , v p0 is the fifth pixel, [u sp , v sp is the sixth pixel, [u p1 , v p1 is the seventh pixel, [u s2 , v s2 is the eighth pixel, H s , V s are the length and width dimensions of the display screen pixels, H p , V p are the length and width dimensions of the projection optical machine pixels respectively.

7. The method according to claim 4, wherein The third distance is obtained by calculation using the following formula: Among them, [u sc , v sc is the second pixel, and [u sp , v sp is the sixth pixel. H s , V s are the length and width dimensions of the display screen pixels.

8. The method according to claim 1, wherein The obtaining of the three-dimensional coordinates according to the internal parameters, the correspondence relationship, and the distance information includes: Determine the ninth pixel of the point to be measured in the projection optical machine and the tenth pixel in the industrial camera; Obtain the eleventh pixel in the display screen corresponding to the ninth pixel and the twelfth pixel in the display screen corresponding to the tenth pixel according to the correspondence relationship; Obtain the three-dimensional coordinates of the point to be measured according to the distance information, the ninth pixel, the tenth pixel, the eleventh pixel, and the twelfth pixel, where the three-dimensional coordinates are the three-dimensional coordinates of the point to be measured relative to the display screen.

9. The method according to claim 8, wherein The three-dimensional coordinates are obtained through the following formula: Among them, [u p2 , v p2 is the ninth pixel, [u c2 , v c2 is the tenth pixel, [u s3 , v s3 is the eleventh pixel, [u s4 , v s4 is the twelfth pixel, [u s5 , v s5 is the pixel on the display screen vertically mapped by the point to be measured, H s , V s are the length and width dimensions of the display screen pixel, L c is the first distance, L p is the second distance.

10. A structured light three-dimensional reconstruction system, characterized in that, The system includes: A projection optical machine for projecting structured light onto a display screen or an object to be measured; An industrial camera for obtaining the structured light projected by the projection optical machine onto the display screen and the structured light projected by the projection optical machine onto the object to be measured; A display screen for displaying a stripe pattern or the structured light projected by the projection optical machine; A slide rail for adjusting the relative positions of the projection optical machine or the industrial camera relative to the display screen; A data processing device including a memory and a processor, where the memory is used to store one or more computer program instructions, and wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1-9.

Images