A three-dimensional reconstruction method, device, equipment and storage medium

Abstract

A three-dimensional reconstruction method, device, equipment and storage medium, including projecting N phase-shifted fringe images and M self-encoding fringe images onto the outer surface of the object to be measured, and collecting N+M reflected fringe images modulated by the object to be measured. According to the reflected fringe images corresponding to the N phase-shifted fringe images, a wrapped phase image is calculated and a target neighborhood is constructed, and a first area and a second area are determined according to the target neighborhood. The sum of squared grayscale differences between the second area in the reflected fringe image corresponding to the M self-encoding fringe images and the first area in the reflected fringe image corresponding to the N phase-shifted fringe images are calculated to construct a grayscale difference squared sum matrix. According to the grayscale difference squared sum matrix, the code value sequence corresponding to the pixel point is determined and the fringe order of the pixel point is calculated, and an absolute phase map is calculated based on the wrapped phase image and the fringe order of each pixel point, and the absolute phase map is converted into the pixel coordinates of the projector and three-dimensional reconstruction is performed. The speed and accuracy of three-dimensional reconstruction are improved.

Description

Technical Field

[0001] The present invention relates to the field of industrial three-dimensional detection technology, and in particular to a three-dimensional reconstruction method, device, equipment and storage medium. Background Art

[0002] In the process of three-dimensional reconstruction using structured light three-dimensional measurement equipment, phase unwrapping is usually performed through traditional multi-frequency methods and Gray code schemes, which are used for the three-dimensional reconstruction process of structured light. At present, there are also many documents proposing different three-dimensional reconstruction methods, for example, document 1: A multi-frequency phase unwrapping method based on staggered ternary Gray code, which discloses a structured light phase unwrapping method, which introduces ternary Gray code encoding to assist in solving the order of multi-frequency wrapped phase. Although it improves the decoding accuracy and reduces the bit error rate to a certain extent compared with the traditional Gray code scheme, in some complex application scenarios with high reflectivity, errors will still be generated by detecting the Gray code stripe boundaries, resulting in bit errors, which leads to limited practical application scenarios, narrow application scope and poor feasibility of this method.

[0003] Reference 2: Optical three-dimensional measurement method based on phase order self-encoding, this document discloses a structured light phase unwrapping method with order self-encoding, which embeds the order information of the binary sequence of stripes into the conventional phase-shifted N-step stripes through the stripe and order construction formula, obtains the position at the truncated phase by calculating the gradient of the wrapped phase, and then obtains the order of the stripes of the current cycle by analyzing the binary sequence at the truncated phase position, so as to unwrap the wrapped phase according to the order information. The advantage of this method is that it does not require additional projection of phase order-encoded images, saving time, but because this method determines the truncation position by calculating the gradient method, it is sensitive to noise and has no correction measures, which easily causes bit errors and order disorder. At the same time, in the actual application scene of high reflectivity, the grayscale of the phase-shifted stripes in the overexposed area jumps with the grayscale value of the encoded sequence due to the high reflectivity, and the information phase of the two interferes, thereby affecting the accuracy of stripe order decoding and the accuracy of phase unwrapping. Therefore, the three-dimensional measurement scene of this method is ideal, the scope of application is narrow, and the three-dimensional reconstruction effect is low in accuracy and poor in robustness.

[0004] Document 3: In a phase-coding order correction method for bidirectional decoding, this document proposes a phase-order coding stripe, wherein the coding stripe is generated by embedding the step phase of the phase-shift stripe into the ideal phase-shift stripe, and further utilizes the wrapped phase of the coding stripe to obtain the misalignment order of the phase-shift stripe by bidirectional decoding, and combines the misalignment order and the wrapped phase of the phase-shift stripe to obtain the absolute phase. This method requires the projection of coding stripes with the same phase shift step number, and the number of coding stripes increases with the increase of the phase shift step number. For applications that require more phase shift steps, the decoding process takes a long time, and the phase-coding stripe is prone to grayscale jumps and other anomalies in noisy and highly reflective environments, which causes the decoding order to be disordered. Therefore, this method is also limited in complex practical application scenarios such as high reflectivity, and the phase unfolding process is more complicated, and the accuracy and robustness of the three-dimensional measurement effect are poor.

[0005] Reference 4: A structured light three-dimensional measurement method based on phase-shift encoded fringe order. This reference proposes a three-dimensional measurement method that embeds phase shift into phase-shift fringe to encode fringe order and obtain absolute phase. This method generates M additional coded phase fringes by constructing a lookup table. At the same time, it is necessary to use pre-calibrated system parameters to estimate the imaging area and width. If the fringe image is applied to a highly reflective surface with a large difference in reflectivity, the estimation process will have a large error, resulting in fringe order decoding errors. The decoding process requires a lookup table, and the solution process is more complicated, thus limiting the actual application scope of this method. Summary of the invention

[0006] The main technical problem solved by the present invention is to improve the speed and accuracy of three-dimensional reconstruction.

[0007] According to the first aspect, an embodiment provides a three-dimensional reconstruction method, which is applied in a structured light measurement system, wherein the structured light measurement system includes a projector and a camera, and the three-dimensional reconstruction method includes:

[0008] Acquire N phase-shifted fringe images generated according to the N-step phase-shifting method;

[0009] M self-encoding fringe images are generated according to the N phase-shifted fringe images, wherein the grayscale value of the f-th order fringe of the self-encoding fringe image with a serial number of m is copied from the grayscale value of the f-th order fringe of the self-encoding fringe image with a serial number of i m The f-th order fringe of the phase-shifted fringe image, and satisfies , where I m The range is [0, N-1];

[0010] The N phase-shifted fringe images and the M self-encoding fringe images are projected onto the outer surface of the object to be measured by the projector, and the N+M reflected fringe images modulated by the object to be measured are collected by the camera; wherein the N+M reflected fringe images include reflected fringe images corresponding to the N phase-shifted fringe images and reflected fringe images corresponding to the M self-encoding fringe images;

[0011] A wrapped phase image is calculated based on the reflected fringe images corresponding to the N phase-shifted fringe images, each pixel point in the wrapped phase image is traversed in a preset direction, and a target neighborhood is constructed with each pixel point as the center;

[0012] For each pixel point, a first area corresponding to a target neighborhood constructed by the pixel point is determined in the reflection fringe image corresponding to each phase-shifted fringe image, and a second area corresponding to the target neighborhood is determined in the reflection fringe image corresponding to each self-encoding fringe image, and the sum of squares of grayscale differences between the second area in the reflection fringe image corresponding to the M self-encoding fringe images and the first area in the reflection fringe image corresponding to the N phase-shifted fringe images are calculated to construct a grayscale difference square sum matrix;

[0013] The code value sequence corresponding to the pixel point is determined according to the grayscale difference square sum matrix, wherein the code value sequence includes M code values, the mth code value C m Indicates that the gray value of the stripe at the position of the pixel in the mth self-encoded stripe image is copied from the Cth m For the stripes of the same order in the phase-shifted stripe image, the stripe order of the pixel point is calculated according to the code value sequence;

[0014] An absolute phase map is calculated based on the wrapped phase image and the fringe order of each pixel, the absolute phase map is converted into pixel coordinates of the projector, and three-dimensional reconstruction is performed based on the pixel coordinates of the projector.

[0015] In some embodiments, the number of images M of the self-encoded fringe image is determined according to the number of phase shift steps N of the N-step phase shift method and the number of levels F in the phase shift fringe image, and satisfies .

[0016] In some embodiments, constructing a target neighborhood with each pixel as the center includes:

[0017] The coordinate components of each pixel point in the preset direction are determined, and a target neighborhood centered on each pixel point is constructed in the preset direction according to the coordinate components of each pixel point in the preset direction and a preset neighborhood radius.

[0018] In some embodiments, the expression for the sum of squares of grayscale differences between a first region in a reflection fringe image corresponding to the nth phase-shifted fringe image and a second region in a reflection fringe image corresponding to the ith self-encoding fringe image is as follows:

[0019] ,

[0020] in, represents the sum of squared grayscale differences between the first region in the reflected fringe image corresponding to the nth phase-shifted fringe image and the second region in the reflected fringe image corresponding to the ith self-encoded fringe image, u represents the coordinate component of the pixel point in the preset direction, W represents the radius of the target neighborhood, represents the coordinate component of the pixel point in the first area or the second area in the preset direction, The coordinate component of the second area in the reflected fringe image corresponding to the i-th self-encoded fringe image in the preset direction is The gray value of the pixel, The coordinate component of the first area in the reflection fringe image corresponding to the nth phase-shifted fringe image in the preset direction is The grayscale value of the pixel, the range of i is [1, M], and the range of n is [0, N-1].

[0021] In some embodiments, the grayscale difference square sum matrix is: ; Determining the code value sequence corresponding to the pixel point according to the grayscale difference square sum matrix includes:

[0022] Minimize the grayscale difference square sum matrix row by row to obtain the minimum value of each row;

[0023] The serial number of the phase-shifted fringe image corresponding to the minimum value of each row is used as the code value, and the code value sequence corresponding to the pixel point is constructed according to the code values ​​obtained from each row in the grayscale difference square sum matrix.

[0024] In some embodiments, the expression of the fringe level of the pixel point is as follows:

[0025] ,

[0026] in, represents the fringe order of the pixel point (x, y), M represents the number of the self-encoding fringe images, N represents the phase shift step number of the phase shift fringe image, Represents the mth code value in the code value sequence corresponding to the pixel point (x, y).

[0027] In some embodiments, the step of calculating the absolute phase map based on the wrapped phase image and the fringe order of each pixel point includes:

[0028] Correcting the jump level and the misaligned level in the fringe level of each pixel point to obtain the corrected fringe level of each pixel point;

[0029] The absolute phase image is calculated based on a preset absolute phase image calculation formula, the corrected fringe order of each pixel point and the wrapped phase image.

[0030] According to the second aspect, an embodiment provides a three-dimensional device, which is applied in a structured light measurement system, wherein the structured light measurement system includes a projector and a camera, and the three-dimensional reconstruction device includes:

[0031] A phase-shift fringe image acquisition module is used to acquire N phase-shift fringe images generated according to an N-step phase-shift method;

[0032] A self-encoding fringe image generation module is used to generate M self-encoding fringe images according to the N phase-shifted fringe images, wherein the grayscale value of the f-th level fringe of the self-encoding fringe image with a serial number of m is copied from the grayscale value of the f-th level fringe of the self-encoding fringe image with a serial number of I m The f-th order fringe of the phase-shifted fringe image, and satisfies , where I m The range is [0, N-1];

[0033] An image projection module, used to project the N phase-shifted fringe images and the M self-encoding fringe images onto the outer surface of the object to be measured by the projector, and to collect N+M reflected fringe images modulated by the object to be measured by the camera; wherein the N+M reflected fringe images include reflected fringe images corresponding to the N phase-shifted fringe images and reflected fringe images corresponding to the M self-encoding fringe images;

[0034] A neighborhood construction module is used to calculate a wrapped phase image according to the reflected fringe images corresponding to the N phase-shifted fringe images, traverse each pixel point in the wrapped phase image in a preset direction, and construct a target neighborhood with each pixel point as the center;

[0035] A grayscale difference square sum calculation module is used to determine, for each pixel point, a first area corresponding to a target neighborhood constructed by the pixel point in a reflection fringe image corresponding to each phase-shifted fringe image, determine a second area corresponding to the target neighborhood in a reflection fringe image corresponding to each self-encoding fringe image, and calculate the grayscale difference square sums between the second areas in the reflection fringe images corresponding to the M self-encoding fringe images and the first areas in the reflection fringe images corresponding to the N phase-shifted fringe images, so as to construct a grayscale difference square sum matrix;

[0036] A code value sequence determination module is used to determine the code value sequence corresponding to the pixel point according to the grayscale difference square sum matrix, wherein the code value sequence includes M code values, the mth code value C m Indicates that the gray value of the stripe at the position of the pixel in the mth self-encoded stripe image is copied from the Cth m For the stripes of the same order in the phase-shifted stripe image, the stripe order of the pixel point is calculated according to the code value sequence;

[0037] A three-dimensional reconstruction module is used to calculate an absolute phase map based on the wrapped phase image and the fringe order of each pixel point, convert the absolute phase map into the pixel point coordinates of the projector, and perform three-dimensional reconstruction according to the pixel point coordinates of the projector.

[0038] According to a third aspect, an embodiment provides a three-dimensional reconstruction device, including:

[0039] Memory, used to store programs;

[0040] The processor is used to implement the three-dimensional reconstruction method by executing the program stored in the memory.

[0041] According to a fourth aspect, an embodiment provides a computer-readable storage medium, on which a program is stored, and the program can be executed by a processor to implement a three-dimensional reconstruction method.

[0042] According to the three-dimensional reconstruction method, device, equipment and computer-readable storage medium of the above-mentioned embodiment, since N phase-shifted fringe images are used to generate M self-encoding fringe images, the noise introduced by shooting the fringe can be reduced, and the reduction of phase accuracy can be avoided. The N phase-shifted fringe images and the M self-encoding fringe images are projected onto the outer surface of the object to be measured, and the fringe order of each pixel point is determined according to the collected modulated N+M reflected fringe images, so as to perform phase unwrapping of high-frequency fringe. Since the projected M self-encoding fringe images are generated by N phase-shifted fringe images, and the number of generated self-encoding fringe images is smaller, it is not necessary to use fringes of other frequencies, and the time required for image acquisition is greatly reduced without reducing the unwrapping accuracy. At the same time, the M self-encoding fringe images can assist the N phase-shifted fringe images in obtaining absolute phase images, so the accuracy and speed of three-dimensional reconstruction can be improved compared with the traditional three-dimensional reconstruction method. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flowchart of a three-dimensional reconstruction method according to an embodiment of the present application;

[0044] Figure 2 A coding diagram of mutually mapping stripes of different orders in a self-encoding stripe image and stripes of different orders in a phase-shifted stripe image in an embodiment;

[0045] Figure 3 A stripe image of a self-encoded stripe image G1 according to an embodiment;

[0046] Figure 4 A stripe image of a self-encoded stripe image G2 according to an embodiment;

[0047] Figure 5 A stripe image of a self-encoded stripe image G3 according to an embodiment;

[0048] Figure 6 A flowchart of determining a code value sequence corresponding to a pixel point according to a grayscale difference square sum matrix in an embodiment;

[0049] Figure 7 A flowchart of an embodiment of the present invention for calculating an absolute phase image based on a wrapped phase image and a fringe order of each pixel point;

[0050] Figure 8 A reference diagram of the phase change of the 500th column of a single-frequency eight-step phase shift wrapped phase image according to an embodiment;

[0051] Fig. 9 A reference diagram of the change of the level of the 500th column of the corrected stripe level image in one embodiment;

[0052] Fig.10 A reference diagram of the phase change of the 500th column of the single-frequency eight-step phase shift absolute phase diagram of an embodiment;

[0053] Fig.11 A complete absolute phase diagram after phase unwrapping of an embodiment;

[0054] Fig.12 FIG. 4 is a schematic structural diagram of a three-dimensional reconstruction device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0056] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0057] The serial numbers assigned to the components herein, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0058] Please refer to Figure 1 In an embodiment of the present invention, a three-dimensional reconstruction method is proposed, which is applied to a structured light measurement system, wherein the structured light measurement system includes a projector and a camera. The three-dimensional reconstruction method includes steps S10 to S70, which are described in detail below.

[0059] Step S10: Acquire N phase-shifted fringe images generated according to the N-step phase-shifting method.

[0060] In some embodiments, when N=8, 8 phase-shifted fringe images generated according to the 8-step phase-shifting method are acquired, wherein the sequence numbers of the 8 phase-shifted fringe images are from 0 to 7, respectively.

[0061] Step S20: Generate M self-encoding fringe images according to N phase-shifted fringe images.

[0062] In some embodiments, the number of images M of the self-encoded fringe image is determined according to the number of phase shift steps N of the N-step phase shift method and the number of levels F in the phase shift fringe image, and satisfies Among them, in meeting Under this condition, M has multiple different values. Under this condition, the minimum value of M is taken as the number of images of the self-encoded stripe image.

[0063] In some embodiments, when N=8, F=90, the number of images M of the self-encoded stripe image needs to satisfy It can be seen that when M is greater than or equal to 3, all values ​​satisfy the above conditions, and the minimum value of M that satisfies the above conditions is taken, therefore, M=3.

[0064] In some embodiments, the number of images M of the self-encoded stripe image is flexibly set according to the number of phase shift steps N of the N-step phase shift method and the number of levels F in the phase shift stripe image. Compared with the traditional Gray code method and other auxiliary coding methods, the number of self-encoded stripe images is smaller, so the accuracy and robustness of subsequent decoding are higher.

[0065] In some embodiments, each phase-shifted fringe image includes multiple fringe stripes of different orders, and each self-encoded fringe image also includes multiple fringe stripes of different orders, wherein the grayscale value of the f-th order fringe of the self-encoded fringe image with serial number m is copied from the grayscale value of the f-th order fringe of the self-encoded fringe image with serial number i. m The f-th order fringe of the phase-shifted fringe image, and satisfies That is, f is represented by an M-bit N-ary number, I m That is, the number at the mth position. m The range is [0, N-1].

[0066] Please refer to Figure 2 In some embodiments, when F=90, N=8, M=3, according to the formula A table for mapping the stripes of different orders in the self-encoding stripe image and the stripes of different orders in the phase-shift stripe image is constructed, and the first 17 orders out of 90 orders are selected as examples. For the self-encoding stripe images G1, G2 and G3, and the phase-shift stripe images numbered 0 to 7, the grayscale values ​​of the stripes of the 7th order of the self-encoding stripe image G3 numbered 3 are copied from the stripes of the 7th order of the phase-shift stripe image numbered 7, and the grayscale values ​​of the stripes of the 8th order of the self-encoding stripe image G2 numbered 2 are copied from the stripes of the 8th order of the phase-shift stripe image numbered 1, and so on.

[0067] Please refer to Figure 3 , Figure 3 For the self-encoded stripe image G1, please refer to Figure 4 , Figure 4 For the self-encoded stripe image G2, please refer to Figure 5 , Figure 5 is the self-encoded stripe image G3.

[0068] Step S30: projecting N phase-shifted fringe images and M self-encoding fringe images onto the outer surface of the object under test by a projector, and collecting N+M reflected fringe images modulated by the object under test by a camera.

[0069] In some embodiments, N phase-shifted fringe images and M self-encoding fringe images are projected onto the outer surface of the object under test by a projector, and the reflected fringe images corresponding to the N phase-shifted fringe images modulated by the object under test and the reflected fringe images corresponding to the M self-encoding fringe images are captured by a camera.

[0070] Step S40: Calculate a wrapped phase image according to the reflected fringe images corresponding to the N phase-shifted fringe images, traverse each pixel point in the wrapped phase image in a preset direction, and construct a target neighborhood with each pixel point as the center.

[0071] In some embodiments, for the reflected fringe images corresponding to the N phase-shifted fringe images, a wrapped phase image is calculated using an inverse tangent formula, and the inverse tangent formula is expressed as follows:

[0072] ,

[0073] in, is the wrapped phase image, N is the number of phase shift steps, n represents the nth phase shift fringe image, Represents the reflected fringe image corresponding to the nth phase-shifted fringe image.

[0074] In some embodiments, each pixel point in the wrapped phase image is traversed in a preset direction, the coordinate components of each pixel point in the preset direction are determined, and a target neighborhood centered on each pixel point is constructed in the preset direction based on the coordinate components and a preset reference neighborhood.

[0075] In some embodiments, the preset direction is the direction of phase value growth, and each pixel in the wrapped phase image is traversed in turn in the direction of phase value growth, and the coordinate component u of each pixel in the direction of phase value growth is determined, and a target neighborhood [uW,u+W] centered on each pixel is constructed in the direction of phase value growth according to the coordinate component u of each pixel in the preset direction and the preset neighborhood radius W. Among them, the phase in the wrapped phase image is proportional to the order of the stripes, and the direction of phase value growth is the direction of increasing order.

[0076] For example, for a pixel point (x, y), assuming that the direction of phase value growth is the horizontal direction or X direction of the image, the constructed target neighborhood includes multiple pixel points whose X coordinates are in the interval [xW, x+W] and whose Y coordinates are y.

[0077] Step S50: For each pixel point, determine the first area corresponding to the target neighborhood constructed with the pixel point in the reflected fringe image corresponding to each phase-shifted fringe image, determine the second area corresponding to the target neighborhood in the reflected fringe image corresponding to each self-encoded fringe image, calculate the sum of squares of grayscale differences between the second area in the reflected fringe image corresponding to the M self-encoded fringe images and the first area in the reflected fringe image corresponding to the N phase-shifted fringe images, so as to construct a grayscale difference square sum matrix.

[0078] In some embodiments, the expression for the sum of squares of grayscale differences between a first region in a reflection fringe image corresponding to the nth phase-shifted fringe image and a second region in a reflection fringe image corresponding to the ith self-encoding fringe image is as follows:

[0079] ,

[0080] in, represents the sum of squared grayscale differences between the first region in the reflected fringe image corresponding to the nth phase-shifted fringe image and the second region in the reflected fringe image corresponding to the ith self-encoded fringe image, u represents the coordinate component of the pixel point in the preset direction, W represents the weight of the target neighborhood, represents the coordinate component of the pixel point in the first area or the second area in the preset direction, The coordinate component in the preset direction in the second area of ​​the reflected fringe image corresponding to the i-th self-encoding fringe image is The gray value of the pixel, The coordinate component in the preset direction in the first area of ​​the reflected fringe image corresponding to the nth phase-shifted fringe image is The grayscale value of the pixel, the range of i is [1, M], and the range of n is [0, N-1].

[0081] It can be understood that the sum of squares of grayscale differences between the first region and the second region can measure the similarity between the first region and the second region. The smaller the sum of squares of grayscale differences is, the greater the similarity is.

[0082] In some embodiments, the sum of squared grayscale differences between the second region in the reflected fringe image corresponding to the M self-encoded fringe images and the first region in the reflected fringe image corresponding to the N phase-shifted fringe images is calculated to construct a grayscale difference squared sum matrix. The grayscale difference squared sum matrix is:

[0083] ,

[0084] S60: Determine the code value sequence corresponding to the pixel point according to the grayscale difference square sum matrix, and calculate the fringe order of the pixel point according to the code value sequence.

[0085] Please refer to Figure 6 In some embodiments, step S60 determines the code value sequence corresponding to the pixel point according to the grayscale difference square sum matrix, including steps S61 to S62, which are described in detail below.

[0086] Step S61: Find the minimum value of the grayscale difference square sum matrix row by row to obtain the minimum value of each row.

[0087] In some embodiments, the expression for the minimum value of each row is ,in, Represents the i-th row vector of the grayscale difference square sum matrix E.

[0088] Step S62: taking the serial number of the phase-shifted fringe image corresponding to the minimum value of each row as the code value, and constructing the code value sequence corresponding to the pixel point according to the code values ​​obtained from each row in the grayscale difference square sum matrix.

[0089] In some embodiments, the expression of the code value sequence corresponding to the pixel point is: .

[0090] The code value sequence includes M code values, the mth code value C m Indicates that the gray value of the stripe at the position of the pixel in the mth self-encoded stripe image is copied from the Cth m The fringes of the same order in the phase-shifted fringe image.

[0091] In some embodiments, the mth code value C m is the sequence number of the phase-shifted fringe image corresponding to the minimum value of the m-th row.

[0092] In some embodiments, the stripe level of the pixel is calculated according to the code value sequence, and the expression of the stripe level of the pixel is as follows:

[0093] ,

[0094] in, represents the fringe order of the pixel point (x, y), M represents the number of self-encoded fringe images, and N represents the number of phase shift steps of the phase shift fringe image. Represents the mth code value in the code value sequence corresponding to the pixel point (x, y).

[0095] In some embodiments, the expression of the code value sequence at a single level is as follows:

[0096] ,

[0097] In some embodiments, the expression of the code value sequence composed of all levels is as follows:

[0098] ,

[0099] S70: Obtain an absolute phase map based on the wrapped phase image and the fringe order of each pixel point, convert the absolute phase map into pixel point coordinates of the projector, and perform three-dimensional reconstruction according to the pixel point coordinates of the projector.

[0100] Please refer to Figure 7 In some embodiments, step S70 calculates an absolute phase map based on the wrapped phase image and the fringe order of each pixel, including steps S71 to S72, which are described in detail below.

[0101] Step S71: correcting the jump level and the misaligned level in the fringe level of each pixel point to obtain the corrected fringe level of each pixel point.

[0102] In some embodiments, due to the influence of noise and defocus, the fringe order of each pixel point is prone to individual jump orders and misaligned orders at the 2π truncation of the wrapped phase image. The median filtering method is used to remove the individual jump orders, and then the neighborhood correction method is used to correct the misaligned orders.

[0103] In some embodiments, the process of neighborhood correction is shown in the following formula:

[0104] ,

[0105] Where r is the radius of the correction neighborhood.

[0106] Step S72: Calculate the absolute phase image based on the preset absolute phase image calculation formula, the corrected fringe order of each pixel point and the wrapped phase image.

[0107] In some embodiments, the absolute phase calculation formula is as follows:

[0108] ,

[0109] in, represents the absolute phase diagram, is the wrapped phase image, Represents the corrected fringe level of the pixel (x, y).

[0110] In some embodiments, the absolute phase map is converted into the pixel coordinates of the projector, and the coordinates of the three-dimensional points are calculated based on the pixel coordinates of the projector and the corresponding matching camera according to the principle of triangulation to complete the reconstruction process of the three-dimensional measurement.

[0111] In some embodiments, the expression of the pixel coordinate point of the projector is as follows:

[0112] ,

[0113] ,

[0114] Among them, the pixel coordinate point of the projector is , is the vertical absolute phase in the absolute phase diagram, is the horizontal absolute phase in the absolute phase image, F is the number of levels in the phase-shifted fringe image, Represents the pixel coordinate point of the camera in the pixel coordinate system.

[0115] Please refer to Figure 8 , Fig. 9 , Fig.10 and Fig.11In some embodiments, an industrial projector is used to sequentially project 8 phase-shifted fringe images and 3 self-encoded fringe images onto the surface of the object to be measured, an industrial camera is used to collect 11 reflected fringe images modulated by the object to be measured, and the method of steps S40 to S70 is used to obtain the wrapped phase image, the corrected fringe order, the absolute phase map, and the complete absolute phase map after phase unwrapping. Among them, Figure 8 The phase change of the 500th column of the single-frequency eight-step phase shift wrapped phase image is: Fig. 9 is the change of the 500th column level of the corrected fringe level image, Fig.10 The phase change of the 500th column of the absolute phase diagram of the single-frequency eight-step phase shift is: Fig.11 This is the complete absolute phase diagram after phase unwrapping.

[0116] In some embodiments, the N phase-shifted fringe images generated by directly using the N-step phase shift method are directly used as phase encoding units, and M self-encoding fringe images are generated based on the N phase-shifted fringe images. The code value sequence of each pixel is determined using the M self-encoding fringe images and the N phase-shifted fringe images, and the fringe order of the pixel is calculated based on the code value sequence. The order of the fringe is obtained by filtering and correcting the fringe jump at the truncated phase, so as to unfold the wrapped phase map of the phase-shifted fringe image, and finally complete the three-dimensional measurement process using the calibrated phase height model and system parameters. Since the embodiment of the present application uses the phase-shifted fringe image itself for self-encoding, it has stronger anti-interference ability to environmental noise, especially for surface measurements with large reflectivity differences, it can still perform high-precision three-dimensional point cloud reconstruction, so it has a wider range of use, which is of great significance for improving the reconstruction accuracy of a large field of view. At the same time, the decoding process of this embodiment only requires simple image matching, constructing a target neighborhood and matching the phase-shifted fringe image and the self-encoded fringe image within the target neighborhood to obtain the fringe order. Therefore, the order decoding process is simple. Compared with the traditional multi-frequency method and Gray code phase unwrapping process, this embodiment requires fewer projected images, and at the same time, the decoding accuracy is higher and the decoding robustness is stronger.

[0117] In some embodiments, the phase unwrapping of traditional multi-frequency stripes requires a large number of stripes to be projected to assist in obtaining the order, and the high-frequency phase unwrapping process is cumbersome, the unwrapping phase calculation is time-consuming, and the shooting of stripes will introduce more noise, resulting in reduced phase accuracy. In this embodiment, by introducing the coded image of the phase-shifted stripe self-encoding, the stripe order is determined by matching the M-frame self-encoded stripe image with the phase-shifted stripe, and then the accurate stripe order is obtained by the stripe order filtering and correction post-processing method, so as to perform phase unwrapping of high-frequency stripes. Since this method can still accurately decode stripes with higher frequencies and requires fewer projected images, it improves the accuracy and speed of three-dimensional reconstruction compared to traditional methods. At the same time, this embodiment only needs one high-frequency phase-shifted stripe, and does not need to rely on stripe information of other frequencies. It greatly reduces the time required for image acquisition without reducing the unwrapping accuracy, thereby improving efficiency. Compared with the traditional method of unwrapping combined with the Gray code method, the decoding accuracy of this embodiment is high, and it can be applied to surfaces with complex reflectivity such as high reflectivity, while the Gray code method is prone to bit errors in complex application scenarios due to the need to detect boundaries.

[0118] Please refer to Fig.12 In some embodiments, a three-dimensional reconstruction device is proposed, which is applied in a structured light measurement system, wherein the structured light measurement system includes a projector and a camera, and the three-dimensional reconstruction device includes a phase-shifted fringe image acquisition module 10, a self-encoding fringe image generation module 20, an image projection module 30, a neighborhood construction module 40, a grayscale difference square sum calculation module 50, a code value sequence determination module 60 and a three-dimensional reconstruction module 70, which are described in detail below.

[0119] The phase-shift fringe image acquisition module 10 is used to acquire N phase-shift fringe images generated according to the N-step phase shift method.

[0120] In some embodiments, when N=8, the phase-shift fringe image acquisition module 10 acquires 8 phase-shift fringe images generated according to the 8-step phase shift method, wherein the sequence numbers of the 8 phase-shift fringe images are from 0 to 7 respectively.

[0121] The self-encoding fringe image generation module 20 is used to generate M self-encoding fringe images according to N phase-shifted fringe images.

[0122] In some embodiments, the number of images M of the self-encoded fringe image is determined according to the number of phase shift steps N of the N-step phase shift method and the number of levels F in the phase shift fringe image, and satisfies Among them, in meeting Under this condition, M has multiple different values. Under this condition, the minimum value of M is taken as the number of images of the self-encoded stripe image.

[0123] In some embodiments, when N=8, F=90, the number of images M of the self-encoded stripe image needs to satisfy It can be seen that when M is greater than or equal to 3, all values ​​satisfy the above conditions, and the minimum value of M that satisfies the above conditions is taken, therefore, M=3.

[0124] In some embodiments, the number of images M of the self-encoded stripe image is flexibly set according to the number of phase shift steps N of the N-step phase shift method and the number of levels F in the phase shift stripe image. Compared with the traditional Gray code method and other auxiliary coding methods, the number of self-encoded stripe images is smaller, so the accuracy and robustness of subsequent decoding are higher.

[0125] In some embodiments, each phase-shifted fringe image includes multiple fringe stripes of different orders, and each self-encoded fringe image also includes multiple fringe stripes of different orders, wherein the grayscale value of the f-th order fringe of the self-encoded fringe image with serial number m is copied from the grayscale value of the f-th order fringe of the self-encoded fringe image with serial number i. m The f-th order fringe of the phase-shifted fringe image, and satisfies That is, f is represented by an M-bit N-ary number, I m That is, the number at the mth position. m The range is [0, N-1].

[0126] Please refer back to Figure 2 In some embodiments, when F=90, N=8, M=3, according to the formula A table for mapping the stripes of different orders in the self-encoding stripe image and the stripes of different orders in the phase-shift stripe image is constructed, and the first 17 orders out of 90 orders are selected as examples. For the self-encoding stripe images G1, G2 and G3, and the phase-shift stripe images numbered 0 to 7, the grayscale values ​​of the stripes of the 7th order of the self-encoding stripe image G3 numbered 3 are copied from the stripes of the 7th order of the phase-shift stripe image numbered 7, and the grayscale values ​​of the stripes of the 8th order of the self-encoding stripe image G2 numbered 2 are copied from the stripes of the 8th order of the phase-shift stripe image numbered 1, and so on.

[0127] Please refer back to Figure 3 , Figure 3 For the self-encoded stripe image G1, please refer to Figure 4 , Figure 4 For the self-encoded stripe image G2, please refer to Figure 5 , Figure 5 is the self-encoded stripe image G3.

[0128] The image projection module 30 is used to project N phase-shifted fringe images and M self-encoding fringe images onto the outer surface of the object under test through a projector, and to collect N+M reflected fringe images modulated by the object under test through a camera.

[0129] In some embodiments, the image projection module 30 projects N phase-shifted fringe images and M self-encoding fringe images onto the outer surface of the object under test through a projector, and captures the reflected fringe images corresponding to the N phase-shifted fringe images modulated by the object under test and the reflected fringe images corresponding to the M self-encoding fringe images through a camera.

[0130] The neighborhood construction module 40 is used to calculate the wrapped phase image according to the reflected fringe images corresponding to the N phase-shifted fringe images, traverse each pixel point in the wrapped phase image in a preset direction, and construct a target neighborhood with each pixel point as the center.

[0131] In some embodiments, the neighborhood construction module 40 calculates the reflected fringe images corresponding to the N phase-shifted fringe images by using the inverse tangent formula to obtain the wrapped phase image. The inverse tangent formula is expressed as follows:

[0132] ,

[0133] in, is the wrapped phase image, N is the number of phase shift steps, n represents the nth phase shift fringe image, Represents the reflected fringe image corresponding to the nth phase-shifted fringe image.

[0134] In some embodiments, the neighborhood construction module 40 traverses each pixel point in the wrapped phase image in a preset direction, determines the coordinate components of each pixel point in the preset direction, and constructs a target neighborhood centered on each pixel point in the preset direction based on the coordinate components and a preset reference neighborhood.

[0135] In some embodiments, the preset direction is the direction of phase value growth, and each pixel point in the wrapped phase image is traversed in turn along the direction of phase value growth to determine the coordinate component u of each pixel point in the direction of phase value growth. According to the coordinate component u of each pixel point in the preset direction and the preset neighborhood radius W, a target neighborhood [uW,u+W] centered on each pixel point is constructed in the direction of phase value growth.

[0136] For example, for a pixel point (x, y), assuming that the direction of phase value growth is the horizontal direction or X direction of the image, the constructed target neighborhood includes multiple pixel points whose X coordinates are in the interval [xW, x+W] and whose Y coordinates are y.

[0137] The grayscale difference square sum calculation module 50 is used to determine, for each pixel point, a first area corresponding to a target neighborhood constructed with the pixel point in the reflected fringe image corresponding to each phase-shifted fringe image, determine a second area corresponding to the target neighborhood in the reflected fringe image corresponding to each self-encoded fringe image, calculate the grayscale difference square sums between the second areas in the reflected fringe images corresponding to the M self-encoded fringe images and the first areas in the reflected fringe images corresponding to the N phase-shifted fringe images, so as to construct a grayscale difference square sum matrix.

[0138] In some embodiments, the expression for the sum of squares of grayscale differences between a first region in a reflection fringe image corresponding to the nth phase-shifted fringe image and a second region in a reflection fringe image corresponding to the ith self-encoding fringe image is as follows:

[0139] ,

[0140] in, represents the sum of squared grayscale differences between the first region in the reflected fringe image corresponding to the nth phase-shifted fringe image and the second region in the reflected fringe image corresponding to the ith self-encoded fringe image, u represents the coordinate component of the pixel point in the preset direction, W represents the weight of the target neighborhood, represents the coordinate component of the pixel point in the first area or the second area in the preset direction, The coordinate component in the preset direction in the second area of ​​the reflected fringe image corresponding to the i-th self-encoding fringe image is The gray value of the pixel, The coordinate component in the preset direction in the first area of ​​the reflected fringe image corresponding to the nth phase-shifted fringe image is The grayscale value of the pixel, the range of i is [1, M], and the range of n is [0, N-1].

[0141] It can be understood that the sum of squares of grayscale differences between the first region and the second region can measure the similarity between the first region and the second region. The smaller the sum of squares of grayscale differences is, the greater the similarity is.

[0142] In some embodiments, the grayscale difference square sum calculation module 50 calculates the grayscale difference square sums between the second regions in the reflection fringe images corresponding to the M self-encoding fringe images and the first regions in the reflection fringe images corresponding to the N phase-shifted fringe images, so as to construct a grayscale difference square sum matrix. The grayscale difference square sum matrix is:

[0143] ,

[0144] The code value sequence determination module 60 is used to determine the code value sequence corresponding to the pixel point according to the grayscale difference square sum matrix, and calculate the fringe order of the pixel point according to the code value sequence.

[0145] Please refer back to Figure 6 In some embodiments, the code value sequence determination module 60 determines the code value sequence corresponding to the pixel point according to the grayscale difference square sum matrix, which is described in detail below.

[0146] The code value sequence determination module 60 finds the minimum value of the grayscale difference square sum matrix row by row to obtain the minimum value of each row. The serial number of the phase-shifted fringe image corresponding to the minimum value of each row is used as the code value, and the code value sequence corresponding to the pixel point is constructed according to the code value obtained for each row in the grayscale difference square sum matrix.

[0147] In some embodiments, the expression for the minimum value of each row is ,in, represents the i-th row vector of the grayscale difference square sum matrix E. The expression of the code value sequence corresponding to the pixel point is The code value sequence includes M code values, the mth code value C m Indicates that the gray value of the stripe at the position of the pixel in the mth self-encoded stripe image is copied from the Cth m The fringes of the same order in the phase-shifted fringe image.

[0148] In some embodiments, the mth code value C m is the sequence number of the phase-shifted fringe image corresponding to the minimum value of the m-th row.

[0149] In some embodiments, the 3D reconstruction module 70 calculates the fringe level of the pixel point according to the code value sequence. The expression of the fringe level of the pixel point is as follows:

[0150] ,

[0151] in, represents the fringe order of the pixel point (x, y), M represents the number of self-encoded fringe images, and N represents the number of phase shift steps of the phase shift fringe image. Represents the mth code value in the code value sequence corresponding to the pixel point (x, y).

[0152] In some embodiments, the expression of the code value sequence at a single level is as follows:

[0153] ,

[0154] In some embodiments, the expression of the code value sequence composed of all levels is as follows:

[0155] ,

[0156] The 3D reconstruction module 70 is used to calculate an absolute phase map based on the wrapped phase image and the fringe order of each pixel, convert the absolute phase map into the pixel coordinates of the projector, and perform 3D reconstruction according to the pixel coordinates of the projector.

[0157] Please refer back to Figure 7 In some embodiments, the 3D reconstruction module 70 calculates an absolute phase map based on the wrapped phase image and the fringe order of each pixel, which is described in detail below.

[0158] The jump order and misalignment order in the fringe order of each pixel are corrected to obtain the corrected fringe order of each pixel. The absolute phase map is calculated based on the preset absolute phase map calculation formula, the corrected fringe order of each pixel and the wrapped phase image.

[0159] In some embodiments, due to the influence of noise and defocus, the fringe order of each pixel point is prone to individual jump orders and misaligned orders at the 2π truncation of the wrapped phase image. The median filtering method is used to remove the individual jump orders, and then the neighborhood correction method is used to correct the misaligned orders.

[0160] In some embodiments, the process of neighborhood correction is shown in the following formula:

[0161] ,

[0162] Where r is the radius of the correction neighborhood.

[0163] In some embodiments, the absolute phase calculation formula is as follows:

[0164] ,

[0165] in, represents the absolute phase diagram, is the wrapped phase image, Represents the corrected fringe level of the pixel (x, y).

[0166] In some embodiments, the three-dimensional reconstruction module 70 converts the absolute phase map into the pixel coordinates of the projector, and calculates the coordinates of the three-dimensional points based on the principle of triangulation according to the pixel coordinates of the projector and the pixel coordinates of the corresponding matching camera, thereby completing the reconstruction process of the three-dimensional measurement.

[0167] In some embodiments, the expression of the pixel coordinate point of the projector is as follows:

[0168] ,

[0169] ,

[0170] Among them, the pixel coordinate point of the projector is , is the vertical absolute phase in the absolute phase diagram, is the horizontal absolute phase in the absolute phase image, F is the number of levels in the phase-shifted fringe image, Represents the pixel coordinate point of the camera in the pixel coordinate system.

[0171] In some embodiments, the N phase-shifted fringe images generated by directly using the N-step phase shift method are directly used as phase encoding units, and M self-encoding fringe images are generated based on the N phase-shifted fringe images. The code value sequence of each pixel is determined using the M self-encoding fringe images and the N phase-shifted fringe images, and the fringe order of the pixel is calculated based on the code value sequence. The order of the fringe is obtained by filtering and correcting the fringe jump at the truncated phase, so as to unfold the wrapped phase map of the phase-shifted fringe image, and finally complete the three-dimensional measurement process using the calibrated phase height model and system parameters. Since the embodiment of the present application uses the phase-shifted fringe image itself for self-encoding, it has stronger anti-interference ability to environmental noise, especially for surface measurements with large reflectivity differences, it can still perform high-precision three-dimensional point cloud reconstruction, so it has a wider range of use, which is of great significance for improving the reconstruction accuracy of a large field of view. At the same time, the decoding process of this embodiment only requires simple image matching, constructing a target neighborhood and matching the phase-shifted fringe image and the self-encoded fringe image within the target neighborhood to obtain the fringe order. Therefore, the order decoding process is simple. Compared with the traditional multi-frequency method and Gray code phase unwrapping process, this embodiment requires fewer projected images, and at the same time, the decoding accuracy is higher and the decoding robustness is stronger.

[0172] In some embodiments, the phase unwrapping of traditional multi-frequency stripes requires a large number of stripes to be projected to assist in obtaining the order, and the high-frequency phase unwrapping process is cumbersome, the unwrapping phase calculation is time-consuming, and the shooting of stripes will introduce more noise, resulting in reduced phase accuracy. In this embodiment, by introducing the coded image of the phase-shifted stripe self-encoding, the stripe order is determined by matching the M-frame self-encoded stripe image with the phase-shifted stripe, and then the accurate stripe order is obtained by the stripe order filtering and correction post-processing method, so as to perform phase unwrapping of high-frequency stripes. Since this method can still accurately decode stripes with higher frequencies and requires fewer projected images, it improves the accuracy and speed of three-dimensional reconstruction compared to traditional methods. At the same time, this embodiment only needs one high-frequency phase-shifted stripe, and does not need to rely on stripe information of other frequencies. It greatly reduces the time required for image acquisition without reducing the unwrapping accuracy, thereby improving efficiency. Compared with the traditional method of unwrapping combined with the Gray code method, the decoding accuracy of this embodiment is high, and it can be applied to surfaces with complex reflectivity such as high reflectivity, while the Gray code method is prone to bit errors in complex application scenarios due to the need to detect boundaries.

[0173] In some embodiments, the embodiments of the present application provide a three-dimensional reconstruction device, including: a memory for storing programs; and a processor for implementing a three-dimensional reconstruction method by executing the programs stored in the memory.

[0174] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above-mentioned embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to implement the above-mentioned functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above-mentioned functions can be implemented. In addition, when all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash disk or mobile hard disk, and can be downloaded or copied and saved in the memory of the local device, or the system of the local device is updated, and when the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0175] The above specific examples are used to illustrate the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art, according to the concept of the present invention, some simple deductions, modifications or substitutions can be made.

Claims

1. A three-dimensional reconstruction method, applied in a structured light measurement system, wherein: The structured light measurement system comprises a projector and a camera, and is characterized in that the three-dimensional reconstruction method comprises: Acquire N phase-shifted fringe images generated according to the N-step phase-shifting method; M self-encoding fringe images are generated according to the N phase-shifted fringe images, wherein the grayscale value of the f-th order fringe of the self-encoding fringe image with a serial number of m is copied from the grayscale value of the f-th order fringe of the self-encoding fringe image with a serial number of i m The f-th order fringe of the phase-shifted fringe image, and satisfies , where I m The range is [0, N-1]; The N phase-shifted fringe images and the M self-encoding fringe images are projected onto the outer surface of the object to be measured by the projector, and the N+M reflected fringe images modulated by the object to be measured are collected by the camera; wherein the N+M reflected fringe images include reflected fringe images corresponding to the N phase-shifted fringe images and reflected fringe images corresponding to the M self-encoding fringe images; A wrapped phase image is calculated based on the reflected fringe images corresponding to the N phase-shifted fringe images, each pixel point in the wrapped phase image is traversed in a preset direction, and a target neighborhood is constructed with each pixel point as the center; For each pixel point, a first area corresponding to a target neighborhood constructed by the pixel point is determined in the reflection fringe image corresponding to each phase-shifted fringe image, and a second area corresponding to the target neighborhood is determined in the reflection fringe image corresponding to each self-encoding fringe image, and the sum of squares of grayscale differences between the second area in the reflection fringe image corresponding to the M self-encoding fringe images and the first area in the reflection fringe image corresponding to the N phase-shifted fringe images are calculated to construct a grayscale difference square sum matrix; The code value sequence corresponding to the pixel point is determined according to the grayscale difference square sum matrix, wherein the code value sequence includes M code values, the mth code value C m Indicates that the gray value of the stripe at the position of the pixel in the mth self-encoded stripe image is copied from the Cth m For the stripes of the same order in the phase-shifted stripe image, the stripe order of the pixel point is calculated according to the code value sequence; An absolute phase map is calculated based on the wrapped phase image and the fringe order of each pixel, the absolute phase map is converted into pixel coordinates of the projector, and three-dimensional reconstruction is performed based on the pixel coordinates of the projector.

2. The three-dimensional reconstruction method according to claim 1, characterized in that: The number of images M of the self-encoded fringe image is determined according to the number of phase shift steps N of the N-step phase shift method and the number of levels F in the phase shift fringe image, and satisfies .

3. The three-dimensional reconstruction method according to claim 1, characterized in that: The target neighborhood is constructed with each pixel as the center, including: The coordinate components of each pixel point in the preset direction are determined, and a target neighborhood centered on each pixel point is constructed in the preset direction according to the coordinate components of each pixel point in the preset direction and a preset neighborhood radius.

4. The three-dimensional reconstruction method according to claim 3, characterized in that: The expression for the sum of squares of grayscale differences between the first region in the reflection fringe image corresponding to the nth phase-shifted fringe image and the second region in the reflection fringe image corresponding to the ith self-encoding fringe image is as follows: ， in, represents the sum of squared grayscale differences between the first region in the reflected fringe image corresponding to the nth phase-shifted fringe image and the second region in the reflected fringe image corresponding to the ith self-encoded fringe image, u represents the coordinate component of the pixel point in the preset direction, W represents the radius of the target neighborhood, represents the coordinate component of the pixel point in the first area or the second area in the preset direction, The coordinate component of the second area in the reflected fringe image corresponding to the i-th self-encoded fringe image in the preset direction is The gray value of the pixel, The coordinate component of the first area in the reflection fringe image corresponding to the nth phase-shifted fringe image in the preset direction is The grayscale value of the pixel, the range of i is [1, M], and the range of n is [0, N-1].

5. The three-dimensional reconstruction method according to claim 4, characterized in that: The grayscale difference square sum matrix is: ; The determining the code value sequence corresponding to the pixel point according to the grayscale difference square sum matrix includes: Minimize the grayscale difference square sum matrix row by row to obtain the minimum value of each row; The serial number of the phase-shifted fringe image corresponding to the minimum value of each row is used as the code value, and the code value sequence corresponding to the pixel point is constructed according to the code values ​​obtained from each row in the grayscale difference square sum matrix.

6. The three-dimensional reconstruction method according to claim 1, characterized in that: The expression of the fringe order of the pixel point is as follows: ， in, represents the fringe order of the pixel point (x, y), M represents the number of the self-encoding fringe images, N represents the phase shift step number of the phase shift fringe image, Represents the mth code value in the code value sequence corresponding to the pixel point (x, y).

7. The three-dimensional reconstruction method according to claim 1, characterized in that: The absolute phase map is obtained by calculating based on the wrapped phase image and the fringe order of each pixel point, including: Correcting the jump level and the misaligned level in the fringe level of each pixel point to obtain the corrected fringe level of each pixel point; The absolute phase image is calculated based on a preset absolute phase image calculation formula, the corrected fringe order of each pixel point and the wrapped phase image.

8. A three-dimensional reconstruction device, used in a structured light measurement system, wherein: The structured light measurement system comprises a projector and a camera, and is characterized in that the three-dimensional reconstruction device comprises: A phase-shift fringe image acquisition module is used to acquire N phase-shift fringe images generated according to an N-step phase-shift method; A self-encoding fringe image generation module is used to generate M self-encoding fringe images according to the N phase-shifted fringe images, wherein the grayscale value of the f-th level fringe of the self-encoding fringe image with a serial number of m is copied from the grayscale value of the f-th level fringe of the self-encoding fringe image with a serial number of I m The f-th order fringe of the phase-shifted fringe image, and satisfies , where I m The range is [0, N-1]; An image projection module, used to project the N phase-shifted fringe images and the M self-encoding fringe images onto the outer surface of the object to be measured by the projector, and to collect N+M reflected fringe images modulated by the object to be measured by the camera; wherein the N+M reflected fringe images include reflected fringe images corresponding to the N phase-shifted fringe images and reflected fringe images corresponding to the M self-encoding fringe images; A neighborhood construction module is used to calculate a wrapped phase image according to the reflected fringe images corresponding to the N phase-shifted fringe images, traverse each pixel point in the wrapped phase image in a preset direction, and construct a target neighborhood with each pixel point as the center; A grayscale difference square sum calculation module is used to determine, for each pixel point, a first area corresponding to a target neighborhood constructed by the pixel point in a reflection fringe image corresponding to each phase-shifted fringe image, determine a second area corresponding to the target neighborhood in a reflection fringe image corresponding to each self-encoding fringe image, and calculate the grayscale difference square sums between the second areas in the reflection fringe images corresponding to the M self-encoding fringe images and the first areas in the reflection fringe images corresponding to the N phase-shifted fringe images, so as to construct a grayscale difference square sum matrix; A code value sequence determination module is used to determine the code value sequence corresponding to the pixel point according to the grayscale difference square sum matrix, wherein the code value sequence includes M code values, the mth code value C m Indicates that the gray value of the stripe at the position of the pixel in the mth self-encoded stripe image is copied from the Cth m For the stripes of the same order in the phase-shifted stripe image, the stripe order of the pixel point is calculated according to the code value sequence; A three-dimensional reconstruction module is used to calculate an absolute phase map based on the wrapped phase image and the fringe order of each pixel point, convert the absolute phase map into the pixel point coordinates of the projector, and perform three-dimensional reconstruction according to the pixel point coordinates of the projector.

9. A three-dimensional reconstruction device, characterized in that: include: Memory, used to store programs; A processor, configured to implement the three-dimensional reconstruction method according to any one of claims 1 to 7 by executing the program stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, and the program can be executed by a processor to implement the three-dimensional reconstruction method according to any one of claims 1 to 7.

Images