Single-frame halftone composite fringe three-dimensional measurement method and device based on deep learning

Abstract

The application discloses a single-frame Haar-like composite fringe three-dimensional measurement method and device based on deep learning, relates to the field of optical three-dimensional measurement, and designs a Haar-like composite gray fringe pattern (HCFP). The three-frequency sinusoidal fringe is converted into a square wave and is superimposed through Haar transformation, and a four-gray-level composite fringe with clear period boundaries is generated. A two-stage one-to-three network (TO-Net) is constructed. The frequency separation of a single HCFP to the first frame of the three-frequency phase shift fringe is completed in the first stage, the remaining three frames of the phase shift fringe are predicted in the second stage, high-precision decoding from a single gray image to 12 standard phase shift fringes is realized, and three-dimensional reconstruction is completed in combination with binocular phase matching and system calibration parameters. The application can realize high-precision three-dimensional measurement only by using a monochrome camera and single-frame acquisition, the measurement time is greatly shortened compared with the traditional three-frequency four-step phase shift method, the measurement of isolated, overlapped, large-slope and discontinuous surface objects can be adapted, the cost is reduced, and the application is suitable for high-speed three-dimensional measurement scenes.

Description

Technical Field

[0001] This invention relates to the field of optical three-dimensional measurement technology, and in particular to a method and apparatus for single-frame Haar composite fringe three-dimensional measurement based on deep learning. Background Technology

[0002] Fringe projection profilometry (FPP) is a core technology in active optical 3D measurement, offering advantages such as non-contact operation, high precision, and ease of deployment. It is widely used in industrial quality inspection, reverse engineering, and robot navigation. Currently, FPP is available in monocular and binocular architectures. Binocular systems can further improve measurement robustness through phase matching, addressing the occlusion problem of monocular systems.

[0003] Phase extraction and phase unwrapping are core components of FPP (Focus-Proof-of-Package): Fourier transform can calculate phase in a single frame, but it suffers from spectral aliasing and leakage, making it unsuitable for measuring discontinuous surfaces or multiple isolated objects; phase-shifting methods offer high phase accuracy and strong noise resistance, but require acquiring multiple frames, resulting in low measurement efficiency and difficulty meeting the needs of dynamic scenes. Phase unwrapping is divided into spatial unwrapping and temporal unwrapping. Spatial unwrapping requires a continuous surface and fails to measure isolated or overlapping objects; temporal unwrapping requires additional projection of coded patterns, further increasing the number of images acquired.

[0004] Existing single-frame measurement technologies have three major drawbacks: ① Color composite stripe encoding relies on RGB color cameras, resulting in high system costs and failure to measure objects with high red, green, and blue saturation, making it unsuitable for the diverse colors of parts in industrial scenarios; ② Traditional sinusoidal composite grayscale stripes (SCFP) have blurred periodic boundaries, making it difficult for neural networks to accurately separate stripe information of different frequencies, leading to severe amplitude and period deviations in the predicted phase, ultimately causing phase unwrapping failure; ③ The traditional three-frequency four-step phase shift method (TFPM) requires the acquisition of 12 frames of images, with a single-frame measurement time exceeding 11 seconds, which cannot meet the real-time requirements of industrial online inspection.

[0005] In summary, existing technologies suffer from poor color stripe adaptability, blurred boundaries of composite stripes, insufficient single-frame prediction accuracy, and difficulty in balancing measurement efficiency and accuracy. There is an urgent need for a binocular 3D measurement method that is resistant to color crosstalk, has high single-frame accuracy, and is low-cost. Summary of the Invention

[0006] To address the above problems, this invention proposes a deep learning-based single-frame Haar-like composite 3D measurement method and device. The method involves designing a Haar-like composite grayscale fringe pattern to project onto the sample and acquire the data. A two-stage, one-to-three-frame network is constructed: the first stage performs frequency separation, and the second stage predicts the remaining three phase-shifting fringes, achieving high-precision decoding from a single grayscale image to 12 standard phase-shifting fringes. The 3D shape of the object is then obtained through phase unwrapping and 3D reconstruction. High-precision 3D measurement can be achieved with only a monochrome camera and single-frame acquisition, significantly reducing the measurement time compared to the traditional three-frequency, four-step phase-shifting method. This method is suitable for measuring isolated, overlapping, high-slope, and discontinuous surface objects, and also reduces costs, making it applicable to high-speed 3D measurement scenarios.

[0007] On the one hand, the specific steps of the single-frame Haar-like composite fringe 3D measurement method based on deep learning are as follows:

[0008] S1. A Haar-like composite grayscale fringe pattern is generated by using Haar-like transform to generate three different frequency sinusoidal fringes. On different samples, the composite grayscale fringe deformation image acquired by projecting the Haar-like composite grayscale fringe pattern is used as measurement data. The four-frame phase-shifted fringes of each frequency acquired by projecting the three different frequency sinusoidal fringes through the four-step phase-shifting method are used as label data to construct a dataset.

[0009] S2, construct two-stage one-to-three networks and train them using the dataset to obtain trained two-stage one-to-three networks;

[0010] The two-stage one-to-three network includes a first convolutional neural network and a second convolutional neural network; the first convolutional neural network takes a composite grayscale stripe deformation image as input and outputs the first frame image of each of the three frequencies; the second convolutional neural network takes the first frame image of any frequency as input and outputs the corresponding remaining three phase-shifted images of the first frame image of that frequency.

[0011] S3, project the object to be measured using a grayscale composite stripe pattern to acquire a composite grayscale stripe deformation image; input the composite grayscale stripe deformation image into a trained two-stage one to three network to predict four frames of phase-shifted stripes for each of the three frequencies.

[0012] S4. Phase unwrapping is performed on the four-frame phase shift fringes of the three predicted frequencies to obtain an absolute phase map. The absolute phase map is then used for three-dimensional reconstruction to obtain the three-dimensional shape of the object to be measured.

[0013] Preferably, the step of generating a Haar-like composite grayscale stripe pattern from three sinusoidal stripes of different frequencies using a Haar-like transform specifically involves:

[0014] The sinusoidal fringes at each frequency are converted into binary square wave images using a Haar-like transform; the formula for the Haar-like transform is expressed as:

[0015] ;

[0016] in, Indicates the first A binary square wave image generated at a frequency. Represents the pixel spatial coordinates in the projected image. The coordinates are in the horizontal direction. The coordinates are in the vertical direction; Indicates the first One frequency; Indicates the total width of the projected image; Indicates the first The initial phase shift of the fringes at each frequency;

[0017] Square wave images generated at three frequencies are superimposed and normalized according to pixel gray levels to generate a Haar-like composite grayscale stripe pattern with discrete grayscale levels and clear periodic boundaries.

[0018] Preferably, the three sinusoidal fringes of different frequencies satisfy the uniqueness condition of multi-frequency heterodyne phase unwrapping, as expressed as:

[0019] ;

[0020] in, Indicates the synthesis frequency; , and These represent the three frequencies of the sine stripes. .

[0021] Preferably, the values ​​of the three different frequencies of sinusoidal stripes are: F1=80, F2=70 and F3=61; correspondingly, the Haar-like composite grayscale stripe pattern has four discrete grayscale levels, with grayscale values ​​of 50, 110, 170 and 230 respectively.

[0022] Preferably, both the first and second convolutional neural networks adopt an improved U-Net architecture; the encoding layer of the improved U-Net architecture adopts a convolution + batch normalization + Leaky ReLU structure, the decoding layer adopts a transposed convolution upsampling + cross-layer feature fusion structure, and the output layer adopts a ReLU activation function to ensure that the output grayscale value is non-negative.

[0023] Preferably, the step of unwrapping the phase shift fringes of the four frames for each of the three predicted frequencies to obtain the absolute phase map specifically involves:

[0024] The wrapping phase was calculated using the four-step phase-shifting method on the four-frame phase-shifted fringes of the three predicted frequencies, resulting in wrapping phase maps for the three frequencies. , and ;

[0025] The three-frequency heterodyne method is used to unwrap the wrapped phase, specifically as follows:

[0026] ;

[0027] ;

[0028] ;

[0029] in, express and The difference frequency phase; express and The difference frequency phase; Indicates a globally unique phase;

[0030] Guided by the difference frequency phase and the globally unique phase, the absolute phase diagram of the highest frequency is obtained by unwrapping step by step.

[0031] Preferably, the grayscale composite stripe pattern is projected using a projector, and the four-step phase-shifting method is used for projection; the measurement data and label data are acquired using two monochrome industrial cameras; the projector and the two monochrome industrial cameras form a binocular stereo vision measurement structure.

[0032] Preferably, the three-dimensional reconstruction specifically includes:

[0033] Two absolute phase maps were obtained from composite grayscale stripe deformation images acquired by two monochrome industrial cameras.

[0034] A disparity map is obtained by subpixel-level matching of two absolute phase maps. Combined with the calibration parameters of the binocular stereo vision measurement structure, the three-dimensional coordinates of the object surface are calculated using the triangulation principle to complete the three-dimensional shape reconstruction.

[0035] Preferably, the different samples include metal parts, complex curved surfaces, isolated objects, and stacked objects.

[0036] On the other hand, a deep learning-based single-frame Haar composite stripe 3D measurement device includes the following:

[0037] The dataset construction module is used to generate Haar-like composite grayscale stripe patterns from sinusoidal stripes of three different frequencies through Haar-like transform; on different samples, the composite grayscale stripe deformation images acquired by projecting the Haar-like composite grayscale stripe patterns are used as measurement data, and the four-frame phase-shifted stripes of each frequency acquired by projecting the sinusoidal stripes of three different frequencies through the four-step phase-shifting method are used as label data, thereby constructing the dataset.

[0038] The network construction and training module is used to construct two-stage one-to-three networks and train them using a dataset to obtain well-trained two-stage one-to-three networks.

[0039] The two-stage one-to-three network includes a first convolutional neural network and a second convolutional neural network; the first convolutional neural network takes a composite grayscale stripe deformation image as input and outputs the first frame image of each of the three frequencies; the second convolutional neural network takes the first frame image of any frequency as input and outputs the corresponding remaining three phase-shifted images of the first frame image of that frequency.

[0040] The network inference module is used to project the object to be measured using a gray-scale composite stripe pattern to acquire a composite gray-scale stripe deformation image; the composite gray-scale stripe deformation image is input into a trained two-stage one to three network to predict four frames of phase-shifted stripes for each of the three frequencies.

[0041] The phase calculation and 3D reconstruction module is used to unwrap the phase of the four-frame phase shift fringes of the three predicted frequencies to obtain an absolute phase map, and then use the absolute phase map to perform 3D reconstruction to obtain the 3D shape of the object to be measured.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] (1) The present invention uses grayscale stripe encoding, which eliminates the need for a color camera to collect data, thus reducing hardware costs; it avoids using RGB color encoding to obtain composite stripes, thus overcoming the problem of color crosstalk.

[0044] (2) The Haar composite grayscale stripe pattern (HCFP) of the present invention has clear periodic boundaries, and the accuracy of network frequency separation is significantly improved; the accuracy is close to that of traditional multi-frame methods.

[0045] (3) This invention only requires the acquisition of one composite stripe image, which reduces the number of image acquisitions by 11 compared with the traditional three-frequency four-step phase shift method, greatly improving the measurement efficiency and meeting the needs of industrial online detection.

[0046] (4) This invention can accurately measure isolated objects, stacked / overlapping objects, high-slope surfaces and discontinuous surfaces by acquiring a single HCFP image, two-stage one to three network inference prediction, phase unwrapping and three-dimensional reconstruction. It solves the problem of surface continuity limitation of traditional spatial phase unwrapping, has strong adaptability to complex scenes, and the accuracy of three-dimensional reconstruction meets the accuracy requirements of industrial precision testing. Attached Figure Description

[0047] The present invention will now be described in further detail with reference to the accompanying drawings;

[0048] Figure 1This is a flowchart illustrating the steps of a deep learning-based three-dimensional measurement method for single-frame Haar-like composite stripes according to an embodiment of the present invention.

[0049] Figure 2 This is a flowchart of a deep learning-based single-frame Haar composite stripe three-dimensional measurement method according to an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of the binocular composite grayscale fringe projection three-dimensional measurement system based on a single-frame Haar composite fringe three-dimensional measurement method according to an embodiment of the present invention.

[0051] Figure 4 The diagram shows the generation process of Haar composite grayscale fringes (HCFP) in the single-frame Haar composite fringes three-dimensional measurement method based on deep learning according to an embodiment of the present invention; wherein, (a) represents the spatial distribution diagram of three-frequency sinusoidal fringes; (b) represents the intensity curve diagram of three-frequency sinusoidal fringes; (c) represents the comparison diagram of Haar composite (square wave) fringes and sinusoidal fringes; and (d) represents the Haar composite grayscale fringes diagram.

[0052] Figure 5 This is a schematic diagram of the overall structure of the two-stage one-to-three network (TO-Net) of the deep learning-based single-frame Haar composite stripe three-dimensional measurement method according to an embodiment of the present invention.

[0053] Figure 6 This is a comparison of the grayscale distribution prediction results of the HCFP and the traditional SCFP of the single-frame Haar composite fringe 3D measurement method based on deep learning in this embodiment of the invention.

[0054] Figure 7 The images provided are the original image, HCFP image, predicted fringe image, wrapped phase map, calculated absolute phase map, standard absolute phase map, phase error map, and three-dimensional point cloud distribution map of the metal part for the single-frame Haar composite fringe three-dimensional measurement method based on deep learning in this embodiment of the invention.

[0055] Figure 8 The original image, HCFP image, predicted stripe image, wrapped phase map, calculated absolute phase map, standard absolute phase map, phase error map, and three-dimensional point cloud distribution map of the complex human portrait based on the deep learning-based single-frame Haar composite stripe three-dimensional measurement method of this invention are shown in the embodiments of the present invention.

[0056] Figure 9 The images provided are the original image, HCFP image, predicted fringe image, wrapped phase map, calculated absolute phase map, standard absolute phase map, phase error map, and three-dimensional point cloud distribution map of the isolated handle in the single-frame Haar composite fringe three-dimensional measurement method based on deep learning according to an embodiment of the present invention.

[0057] Figure 10The images provided are the original image, HCFP image, predicted fringe image, wrapped phase map, calculated absolute phase map, standard absolute phase map, phase error map, and three-dimensional point cloud distribution map of the stacked baffle in the single-frame Haar composite fringe three-dimensional measurement method based on deep learning according to an embodiment of the present invention.

[0058] Figure 11 The figure shows the accuracy verification result of a standard ceramic plane based on the single-frame Haar composite stripe three-dimensional measurement method according to the embodiment of the present invention; wherein, (a) represents the actual ceramic plane; (b) represents the three-dimensional point cloud map of the ceramic plane measurement; and (c) represents the measurement error RMSE.

[0059] Figure 12 The figure shows the accuracy verification result of a standard ceramic ball based on the single-frame Haar composite stripe three-dimensional measurement method based on deep learning in this embodiment of the invention; wherein, (a) represents the physical picture of the standard ceramic ball; (b) represents the three-dimensional point cloud map of the standard ceramic ball measurement; and (c) represents the measurement error RMSE.

[0060] Figure 13 This is a structural block diagram of a single-frame Haar composite stripe three-dimensional measurement device based on deep learning, according to an embodiment of the present invention. Detailed Implementation

[0061] The present invention will be further described below through specific embodiments.

[0062] like Figure 1 and Figure 2 As shown, the specific steps of the single-frame Haar-like composite fringe 3D measurement method based on deep learning are as follows:

[0063] The binocular structured light measurement system used in this method can be found in [link to relevant documentation]. Figure 3 As shown, its hardware composition is as follows:

[0064] Cameras: 2 Imaging Source DMK23U274 monochrome industrial cameras, 1600×1200 pixels resolution, 30fps frame rate;

[0065] Projector: V1i digital light processing projector, resolution 1600×1200 pixels;

[0066] Computer: Intel Core i7-7800X CPU, NVIDIA GeForce GTX 3060Ti 12GB GPU;

[0067] System parameters: field of view 240mm×180mm, working distance (from camera to the object being measured) approximately 500mm.

[0068] Calibrate the binocular vision system. Fix the relative positions of two monochrome industrial cameras and a digital projector, move the checkerboard calibration board to 15 different poses, and simultaneously acquire calibration images from the left and right cameras. Use Zhang Zhengyou's calibration method to solve for the camera's intrinsic parameters, extrinsic parameters, and distortion coefficients, complete the joint calibration of the projector and binocular cameras, and obtain the system projection matrix and 3D reconstruction parameters.

[0069] Fix the relative positions of the binocular camera and the projector, and print a 9×12 checkerboard calibration board (square side length 10mm); move the calibration board to 15 different poses (including different angles, distances, and tilts), and simultaneously acquire calibration images from the left and right cameras; use Zhang Zhengyou calibration method to solve for the camera's intrinsic parameters (focal length, principal point coordinates), extrinsic parameters (rotation matrix, translation vector), and radial distortion coefficients; project the checkerboard pattern through the projector to complete the joint calibration of the projector and the binocular camera, and obtain all the parameters required for the system's 3D reconstruction.

[0070] S1. Three different frequency sinusoidal stripes are transformed into Haar-like composite grayscale stripe patterns through Haar-like transformation. On different samples, the composite grayscale stripe deformation images obtained by projecting the Haar-like composite grayscale stripe patterns are used as measurement data. The sinusoidal stripes of the three frequencies are projected using a four-step phase-shifting method, and four frames of phase-shifted stripe images corresponding to each frequency are collected as label data to construct a dataset.

[0071] The process of generating Haar-like composite grayscale fringes (HCFP) can be found in [link to documentation]. Figure 4 As shown. Frequency selection criteria based on multi-frequency time-phase unwrapping ( Three sets of sinusoidal stripes with frequencies of 80, 70, and 61 were selected. Each sinusoidal stripe was converted into a binary square wave image by a Haar-like transform, preserving clear periodic boundaries. The three sets of binary square waves were superimposed according to pixel gray levels and normalized to the gray level range of [50, 230] to generate an HCFP pattern containing only four discrete gray level values ​​of 50, 110, 170, and 230.

[0072] Specifically, select three frequencies. , , Satisfying frequency selection criteria Generate basic sine fringes:

[0073] ;

[0074] The sinusoidal fringes are converted into a binary square wave using a Haar-like transform, as follows:

[0075] ;

[0076] Superposition and normalization to generate HCFP:

[0077] ;

[0078] Ultimately, the grayscale values ​​of HCFP are only four discrete values: 50, 110, 170, and 230, with obvious grayscale jumps at the periodic boundaries.

[0079] A high-quality training dataset was constructed. 400 sets of real data from different scenes, including metal parts, complex head images, isolated handles, and stacked baffles, were collected. For each scene, a single HCFP image was first projected and acquired by a binocular camera, followed by the projection of 12 stripe images required by the traditional three-frequency four-step phase-shifting method as labels. The dataset was divided into 320 training sets, 80 validation sets, and 80 test sets in a 4:1:1 ratio. Pixel normalization (dividing by 255 to normalize all image pixel values ​​to [0,1]) and size cropping (1028×968 pixels) were performed on all images to accelerate training convergence.

[0080] S2. Construct a two-stage one-to-three network and train it using the dataset to obtain a trained two-stage one-to-three network.

[0081] Based on the improved U-Net, a two-stage "one-to-three" deep learning network is constructed, resulting in the two-stage one-to-three network (TO-Net). See [link / reference]. Figure 5 As shown, this includes CNN1 and CNN2:

[0082] CNN1 (Frequency Separation Network): The input is a single HCFP image, and the output is the first four-step phase-shifted image corresponding to three frequencies, thus decoupling the stripes of different frequencies.

[0083] CNN2 (Phase Shift Prediction Network): The input is the output of CNN1 or the first real-world phase-shifted image, and the output is three more phase-shifted images corresponding to each frequency (with phase shifts of...). , , ).

[0084] Training was performed using the TensorFlow framework. The encoding layer employed a "convolution + batch normalization + Leaky ReLU" structure, while the decoding layer used transposed convolution upsampling + cross-layer feature fusion. The batch size was set to 1, and training lasted for 500 epochs, with the training loss converging to 2.253 × 10⁻⁻⁶. 5 The loss converged to 3×10⁻ 5 .

[0085] Specifically, the encoding network based on the improved U-Net consists of 5 layers, each consisting of two convolutional blocks and one pooling block, with a convolutional kernel size of 3×3, a stride of 1, and padding of 1; the decoding network consists of 4 layers, each consisting of transposed convolutional upsampling, two convolutional blocks, and a cross-layer feature fusion block, with a transposed convolutional kernel size of 2×2 and a stride of 2; the output layer uses the ReLU activation function to ensure that the output grayscale value is non-negative.

[0086] Mean squared error (MSE) is used as the loss function, and the Adam optimizer updates its parameters; the batch size is set to 1, and the initial learning rate is 1×10⁻. 4 The training was performed for 500 epochs; an early stopping strategy was used during training to prevent overfitting, and the training loss eventually converged to 2.253 × 10⁻⁻⁻⁶. 5 The loss converged to 3×10⁻ 5 .

[0087] S3. Project the object to be measured using a grayscale composite stripe pattern to acquire a composite grayscale stripe deformation image. Input the composite grayscale stripe deformation image into a trained two-stage one-to-three network to predict four frames of phase-shifted stripes for each of the three frequencies.

[0088] Single-frame image acquisition and fringe prediction are performed. A projector projects a single HCFP pattern onto the surface of the object being measured, and two monochrome cameras simultaneously acquire images of deformed fringe patterns modulated by the object's surface. The HCFP images acquired by the left and right cameras are input into the trained TO-Net. CNN1 first outputs the first phase-shifted images of three frequencies, and CNN2 then predicts the other three phase-shifted images of each frequency based on this output, outputting 12 standard three-frequency four-step phase-shifted fringe images on each side within 1.1 seconds.

[0089] S4. Phase unwrapping is performed on the four-frame phase shift fringes of the three predicted frequencies to obtain an absolute phase map; the absolute phase map is used for three-dimensional reconstruction to obtain the three-dimensional shape of the object to be measured.

[0090] Phase calculation and time unwrapping are performed.

[0091] The wrap-around phase at three frequencies is calculated using a four-step phase-shifting method, and is expressed as follows:

[0092] ;

[0093] in, This represents the grayscale intensity value of the phase-shifting fringes. An index representing the frequency of stripes. Indicates the index of the phase-shifted image.

[0094] Time phase unwrapping is performed using the three-frequency heterodyne method: first, the difference frequency phase is calculated. and Then calculate the globally unique phase. Guided by this, the highest frequency is obtained by unpacking the package step by step. The absolute phase diagram.

[0095] Perform binocular phase matching and 3D reconstruction.

[0096] Subpixel-level matching of the absolute phase maps of the left and right cameras is performed to obtain a disparity map. Combined with the calibrated system intrinsic and extrinsic parameters and projection matrix, the three-dimensional coordinates of each pixel on the object surface are calculated using the principle of triangulation. The three-dimensional point cloud is then filtered and denoised to obtain the final three-dimensional shape of the object, thus completing the three-dimensional point cloud reconstruction.

[0097] Perform accuracy verification.

[0098] Figure 6 A comparison chart showing the stripe grayscale distribution predicted by HCFP and traditional SCFP is presented.

[0099] The original image, HCFP image, predicted fringe image, wrap-around phase map, calculated absolute phase map, standard absolute phase map, phase error map, and 3D point cloud distribution map used in 3D reconstruction of metal parts, complex human figures, isolated handles, and stacked baffles using this method are shown below. Figure 7 , Figure 8 , Figure 9 and Figure 10 As shown.

[0100] For standard ceramic plane measurement, see [link / reference]. Figure 11 As shown: The standard ceramic plane was placed in 10 different locations for measurement. The average RMSE of the plane fitting was 0.0394 mm, and the maximum RMSE was 0.0412 mm.

[0101] For standard ceramic ball pair measurements, see [link / reference]. Figure 12 As shown: Two standard ceramic spheres (radii of 19.0396 mm and 19.0332 mm, center-to-center distance of 100.0968 mm) were measured. The RMSE values ​​for the sphere radii were 0.0699 mm and 0.0602 mm, respectively, and the RMSE value for the center-to-center distance was 0.0648 mm.

[0102] In summary, this method is resistant to color crosstalk and has low operating costs: it uses grayscale stripe encoding, eliminating the need for a color camera and reducing hardware costs by more than 30%; it avoids using RGB color encoding to obtain composite stripes, thus overcoming the color crosstalk problem.

[0103] The phase prediction accuracy of this method is high: the Haar composite grayscale fringe pattern (HCFP) has clear periodic boundaries, which significantly improves the accuracy of network frequency separation; the mean absolute error (MAE) of absolute phase is about 0.05 rad, and as low as 0.004 rad in local areas, which is close to the accuracy of traditional multi-frame methods.

[0104] The measurement efficiency of this method is greatly improved: only one composite stripe image needs to be acquired, which reduces the number of image acquisitions by 11 compared with the traditional three-frequency four-step phase-shifting method, shortens the total measurement time to 1.1s, and improves the measurement efficiency by 11 times, meeting the needs of industrial online inspection.

[0105] This method is highly adaptable to complex scenarios: it can accurately measure isolated objects, stacked / overlapping objects, surfaces with high slope and discontinuous surfaces, and solves the surface continuity limitation problem of traditional spatial phase unwrapping.

[0106] The method exhibits excellent 3D reconstruction accuracy: the RMSE for standard ceramic plane reconstruction is approximately 0.039 mm, the RMSE for standard ceramic sphere radius measurement is approximately 0.065 mm, and the RMSE for sphere center distance measurement is approximately 0.065 mm, meeting the accuracy requirements for industrial precision testing.

[0107] This method requires only a monochrome camera and single-frame acquisition to achieve high-precision 3D measurement. The average absolute phase error is about 0.05 rad. The measurement time is reduced by 90% compared with the traditional three-frequency four-step phase shift method. It can be adapted to the measurement of isolated, overlapping, steep, and discontinuous surface objects. The system cost is reduced by more than 30%. It is suitable for high-speed 3D measurement scenarios such as industrial online inspection, intelligent manufacturing, and reverse engineering.

[0108] like Figure 13 As shown, the present invention also discloses a single-frame Haar-like composite stripe three-dimensional measurement device based on deep learning, comprising:

[0109] The dataset construction module 1301 is used to generate Haar-like composite grayscale stripe patterns from sinusoidal stripes of three different frequencies through Haar-like transformation; on different samples, the composite grayscale stripe deformation images obtained by projecting the Haar-like composite grayscale stripe patterns are used as measurement data; the sinusoidal stripes of the three frequencies are projected using the four-step phase-shifting method, and four frames of phase-shifted stripe images corresponding to each frequency are collected as label data, thereby constructing the dataset.

[0110] The network construction and training module 1302 is used to construct two-stage one-to-three networks and train them using a dataset to obtain well-trained two-stage one-to-three networks.

[0111] The two-stage one-to-three network includes a first convolutional neural network and a second convolutional neural network; the first convolutional neural network takes a composite grayscale stripe deformation image as input and outputs the first frame image of each of the three frequencies; the second convolutional neural network takes the first frame image of any frequency output by the first convolutional neural network as input and outputs the other three phase-shifted images corresponding to that frequency.

[0112] The network inference module 1303 is used to project the object to be measured using a gray-scale composite stripe pattern, acquire a composite gray-scale stripe deformation image, input the composite gray-scale stripe deformation image into a trained two-stage one to three network, and predict four frames of phase-shifted stripes for each of the three frequencies.

[0113] The phase calculation and 3D reconstruction module 1304 is used to unwrap the phase of the four-frame phase shift fringes of the three predicted frequencies to obtain an absolute phase map; and to use the absolute phase map to perform 3D reconstruction to obtain the 3D shape of the object to be measured.

[0114] The specific implementation of the deep learning-based single-frame Haar composite fringe 3D measurement device is the same as the deep learning-based single-frame Haar composite fringe 3D measurement method, and will not be described again in this embodiment.

[0115] The above are merely specific embodiments of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial modifications made to the present invention using this concept shall be considered as infringing upon the protection scope of the present invention.

Claims

1. A single-frame Haar-like composite fringe three-dimensional measurement method based on deep learning, characterized in that, Includes the following steps: S1. Three different frequency sinusoidal stripes are transformed into Haar-like composite grayscale stripe patterns through Haar-like transformation. On different samples, the composite grayscale stripe deformation images obtained by projecting the Haar-like composite grayscale stripe patterns are used as measurement data. The three frequency sinusoidal stripes are projected using a four-step phase-shifting method, and four frames of phase-shifted stripe images corresponding to each frequency are collected as label data to construct a dataset. S2, construct a two-stage one-to-three network and train it using the dataset to obtain a trained two-stage one-to-three network; The two-stage one-to-three network includes a first convolutional neural network and a second convolutional neural network; the first convolutional neural network takes a composite grayscale stripe deformation image as input and outputs the first frame image of each of the three frequencies; the second convolutional neural network takes the first frame image of any frequency output by the first convolutional neural network as input and outputs the other three phase-shifted images corresponding to that frequency. S3, project the object to be measured using a grayscale composite stripe pattern, acquire the composite grayscale stripe deformation image, input the composite grayscale stripe deformation image into the trained two-stage one to three network, and predict four frames of phase-shifted stripes for each of the three frequencies. S4. Phase unwrapping is performed on the four-frame phase shift fringes of the three predicted frequencies to obtain an absolute phase map; the absolute phase map is used for three-dimensional reconstruction to obtain the three-dimensional shape of the object to be measured.

2. The deep learning-based single-frame Haar-like composite fringe three-dimensional measurement method according to claim 1, characterized in that, The process of generating a Haar-like composite grayscale stripe pattern from three different frequency sinusoidal stripes using a Haar-like transform is specifically as follows: The sinusoidal fringes at each frequency are converted into binary square wave images using a Haar-like transform; the formula for the Haar-like transform is expressed as: ； in, Indicates the first A binary square wave image generated at a frequency, Represents the pixel spatial coordinates in the projected image. The coordinates are in the horizontal direction. The coordinates are in the vertical direction; Indicates the first One frequency; Indicates the total width of the projected image; Indicates the first The initial phase shift of the fringes at each frequency; Square wave images generated at three frequencies are superimposed and normalized according to pixel gray levels to generate a Haar-like composite grayscale stripe pattern with discrete grayscale levels and clear periodic boundaries.

3. The deep learning-based single-frame Haar-like composite fringe three-dimensional measurement method according to claim 2, characterized in that, The three different frequency sinusoidal fringes satisfy the uniqueness condition of multi-frequency heterodyne phase unwrapping, as expressed as: ； in, Indicates the synthesis frequency; , and These represent the three frequencies of the sine stripes. .

4. The deep learning-based single-frame Haar-like composite fringe three-dimensional measurement method according to claim 3, characterized in that, The values ​​of the three different frequencies of sinusoidal stripes are: F1=80, F2=70 and F3=61; correspondingly, the Haar-like composite grayscale stripe pattern has four discrete grayscale levels, with grayscale values ​​of 50, 110, 170 and 230 respectively.

5. The deep learning-based single-frame Haar-like composite stripe three-dimensional measurement method according to claim 1, characterized in that, Both the first and second convolutional neural networks adopt an improved U-Net architecture. The encoding layer of the improved U-Net architecture adopts a convolution + batch normalization + Leaky ReLU structure, the decoding layer adopts a transposed convolution upsampling + cross-layer feature fusion structure, and the output layer adopts the ReLU activation function to ensure that the output grayscale value is non-negative.

6. The method for three-dimensional measurement of single-frame Haar-like composite stripes based on deep learning according to claim 1, characterized in that, The process of unwrapping the phase shift fringes of the three predicted frequencies for each of the four frames to obtain the absolute phase map is as follows: The wrapping phase was calculated using the four-step phase-shift method on the four-frame phase-shift fringes of the three predicted frequencies, resulting in wrapping phase maps for the three frequencies. , and ; The three-frequency heterodyne method is used to unwrap the wrapped phase, specifically as follows: ； ； ； in, express and The difference frequency phase; express and The difference frequency phase; Indicates a globally unique phase; The absolute phase diagram of the highest frequency is obtained by unwrapping the components step by step, guided by the difference frequency phase and the globally unique phase.

7. The deep learning-based single-frame Haar-like composite stripe three-dimensional measurement method according to claim 1, characterized in that, The grayscale composite stripe pattern is projected using a projector, and the four-step phase-shifting method is used for projection. The measurement data and label data are collected using two monochrome industrial cameras. The projector and the two monochrome industrial cameras form a binocular stereo vision measurement structure.

8. The deep learning-based single-frame Haar-like composite fringe three-dimensional measurement method according to claim 7, characterized in that, The three-dimensional reconstruction specifically includes: Two absolute phase maps were obtained from composite grayscale stripe deformation images acquired by two monochrome industrial cameras. A disparity map is obtained by subpixel-level matching of two absolute phase maps. Combined with the calibration parameters of the binocular stereo vision measurement structure, the three-dimensional coordinates of the object surface are calculated using the triangulation principle to complete the three-dimensional shape reconstruction.

9. The deep learning-based single-frame Haar-like composite stripe three-dimensional measurement method according to claim 1, characterized in that, The different samples include metal parts, complex curved surfaces, isolated objects, and stacked objects.

10. A single-frame Haar-like composite stripe three-dimensional measurement device based on deep learning, characterized in that, Including the following: The dataset construction module is used to generate Haar-like composite grayscale stripe patterns from sinusoidal stripes of three different frequencies through Haar-like transform. On different samples, the composite grayscale stripe deformation images obtained by projecting the Haar-like composite grayscale stripe patterns are used as measurement data. The sinusoidal stripes of the three frequencies are projected using a four-step phase-shifting method, and four frames of phase-shifted stripe images corresponding to each frequency are collected as label data to construct the dataset. The network construction and training module is used to construct two-stage one- to three-stage networks and train them using datasets to obtain well-trained two-stage one- to three-stage networks. The two-stage one-to-three network includes a first convolutional neural network and a second convolutional neural network; the first convolutional neural network takes a composite grayscale stripe deformation image as input and outputs the first frame image of each of the three frequencies; the second convolutional neural network takes the first frame image of any frequency output by the first convolutional neural network as input and outputs the other three phase-shifted images corresponding to that frequency. The network inference module is used to project the object to be measured using a gray-scale composite stripe pattern, acquire a composite gray-scale stripe deformation image, input the composite gray-scale stripe deformation image into a trained two-stage one to three network, and predict four frames of phase-shifted stripes for each of the three frequencies. The phase calculation and 3D reconstruction module is used to unwrap the phase of the four-frame phase shift fringes of the three predicted frequencies to obtain an absolute phase map; the absolute phase map is used for 3D reconstruction to obtain the 3D shape of the object to be measured.

Images