A Multi-Scale Feature Fusion Neural Network Phase Unfolding Method and System Based on DeBruijn Stripe Coding

Abstract

The application discloses a multi-scale feature fusion neural network phase unwrapping method and system based on De Bruijn stripe coding, first constructs a wavelength-De Bruijn element mapping table; then, the wrapped phase and the background intensity are obtained by using the collected deformation stripes and the background reference map, and the nonlinear cosine component and the phase gradient feature are calculated according to the wrapped phase and the background intensity, so that a multi-channel physical feature input tensor is constructed. Then, the tensor is input into a multi-scale feature fusion neural network with multiple parallel branches. In the training stage, a loss function based on symbol physical wavelength weighting is introduced for constraint. The application directly realizes high-precision prediction of phase order by a deep learning network, does not need additional auxiliary decoding sequence, effectively avoids the complex mathematical principle process and the tedious logic retrieval in the traditional phase unwrapping algorithm, and significantly improves the efficiency of three-dimensional measurement and the robustness to complex environment.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional shape measurement technology, and specifically relates to a multi-scale feature fusion neural network phase unfolding method and system based on De Bruijn stripe coding. Background Technology

[0002] With the rapid development of photoelectric imaging theory and computer technology, optical image processing has become the core support for three-dimensional visual perception. The increasing demand for higher precision and greater diversity in image information has propelled computer vision from traditional two-dimensional image processing to three-dimensional spatial data acquisition. This transformation not only requires measurement technology to be more intelligent and efficient, but also demands its ability to process three-dimensional data accurately and at high speed. Therefore, achieving comprehensive spatial perception and high-precision three-dimensional reconstruction has become a key research focus in the field of intelligent sensing technology.

[0003] Optical 3D measurement technology, with its high precision, low cost, and non-contact characteristics, has shown broad application prospects in various fields such as industry, medicine, and engineering. Currently, structured light measurement is one of the mainstream methods in optical 3D measurement. This method utilizes a projector to project a grating pattern with phase information, combined with simultaneous imaging and phase decoding by a camera, thereby acquiring the 3D shape information of an object's surface. This technology can stably acquire depth information under various lighting conditions, and is therefore highly valued. However, phase information has periodicity, which... Cyclic phases within a given range can easily lead to phase ambiguity. Therefore, phase unwrapping techniques are needed to obtain continuous absolute phases in order to recover the complete three-dimensional information of an object.

[0004] Current phase unwrapping techniques are mainly divided into spatial phase unwrapping and temporal phase unwrapping. Spatial phase unwrapping is based on the phase relationship between pixels, but it is prone to propagation errors when dealing with drastic depth changes or independent objects. Temporal phase unwrapping uniquely identifies the phase order through multiple frames of images, which solves the problem of disambiguation in discontinuous regions, but it requires projecting multiple frames of additional auxiliary coded patterns, resulting in temporal redundancy that is difficult to meet the requirements of dynamic real-time measurement. In addition, traditional isoperiodic fringes lack a unique physical identifier across the entire field, and relying solely on phase information cannot achieve absolute positioning in a single measurement, limiting the robustness of the system in complex scenarios.

[0005] In recent years, deep learning, with its superior nonlinear fitting and feature extraction capabilities, has provided a new path to overcome the technical bottlenecks of traditional phase unfolding. Learning the mapping relationship between the physical features of stripes and phase orders through deep neural networks can significantly simplify decoding logic and enhance noise resistance. However, current deep learning methods mostly rely on the "black box" mapping of pure image features, ignoring the physical constraints of the spatial distribution of stripes. Especially when dealing with non-uniform stripes with unique encoded features, existing network structures often suffer from limited receptive fields, making it difficult to balance the microscopic phase details with the macroscopic global encoding logic, leading to disambiguation errors at phase order edges. Therefore, developing an efficient and reliable phase unfolding method based on deep learning neural networks that deeply integrates the physical distribution characteristics of stripe space and requires no additional auxiliary sequences, to meet the application requirements for higher accuracy, robustness, and stability, is a key technical goal for improving the core competitiveness of optical 3D measurement systems. Summary of the Invention

[0006] The present invention proposes a multi-scale feature fusion neural network phase unfolding method and system based on De Bruijn stripe coding. This method deeply integrates the physical distribution characteristics of stripe space and does not require additional auxiliary sequences. By embedding the De Bruijn sequence encoding into the stripe period modulation, the accuracy of phase unfolding is greatly improved.

[0007] The technical solution provided by this invention is as follows:

[0008] Firstly, a phase unrolling method for a multi-scale feature fusion neural network based on De Bruijn stripe coding includes the following steps:

[0009] Step 1: Construct a mapping table based on wavelength-De Bruijn elements, and based on the non-uniform wavelength coding scheme of the mapping table, generate an N-step phase shift fringe pattern based on the N-step phase shift algorithm. The non-uniform wavelength coding scheme has global uniqueness.

[0010] Step 2: Project the generated N-step phase-shift fringe pattern and a background image with a fixed gray value onto the object under test. Simultaneously, the camera acquires the N-step deformed phase-shift fringe pattern modulated by the object under test and a background image with a fixed gray value. Extract the background intensity and calculate the encapsulated phase.

[0011] Step 3: Extract sine and cosine feature components from the obtained package phase through the solution, calculate the phase gradient features, and construct a four-channel physical feature input tensor by combining the background intensity features;

[0012] Step 4: Construct a multi-scale feature fusion neural network;

[0013] The multi-scale feature fusion neural network adopts a four-way parallel convolutional architecture, which is then connected to a feature cascade module to align and fuse the extracted multi-scale semantic information in spatial dimensions, and finally outputs a classification probability distribution map of pixel-level phase order.

[0014] The prediction is a pixel-level classification of the phase order, which is used to subsequently determine the phase order to achieve phase unrolling;

[0015] Step 5: Using the four-channel physical feature input tensor constructed in Step 3, train the multi-scale feature fusion neural network constructed in Step 4; during the inference stage, based on the wavelength-De Bruijn element mapping table, the actual N-step deformed phase shift fringe pattern and a background image with a fixed gray value, execute Step 2 and Step 3 in sequence to obtain the input tensor, input the input tensor into the trained multi-scale feature fusion neural network, predict the pixel-level phase order, and combine it with the wrapped phase obtained in Step 2 to realize phase unwrapping.

[0016] The technical solution of this invention adopts a multi-scale feature fusion neural network architecture, which uses four parallel branches with different receptive fields to simultaneously extract the global wavelength constraint logic and local morphological detail features of De Bruijn encoded fringes. By constructing a four-channel physical feature input tensor containing background intensity, sine and cosine components and phase gradient, and designing a loss function based on symbol physical wavelength weighting, high-precision prediction of pixel-level phase order is achieved under a single set of N-step phase-shifted fringes, without the need for additional auxiliary encoded patterns.

[0017] Furthermore, the generation process of the N-step phase-shifted fringe pattern is as follows:

[0018] Step 1.1: Select m wavelengths and De Bruijn elements to construct a wavelength-De Bruijn element mapping table;

[0019] Step 1.2: Based on the wavelength-De Bruijn element mapping table construction rules, generate an m-element n-order De Bruijn wavelength sequence, and then map each symbol in the De Bruijn wavelength sequence to a stripe segment of the corresponding wavelength, thereby generating an N-step phase-shifted stripe pattern with non-uniform wavelength encoding.

[0020] The m-element n-order De Bruijn wavelength sequence refers to a sequence composed of m different wavelength symbols, and the sequence satisfies the order property of the De Bruijn sequence, that is, any subsequence composed of n consecutive symbols (i.e. n wavelength values) is unique in the whole sequence; where m and n are both positive integers, m≥2, n≥2;

[0021] Furthermore, the wavelength-De Bruijn element mapping relationship is as follows: De Bruijn elements {db1, db2, ..., db...}m} Corresponding to wavelengths respectively , ,..., };

[0022] The mapping relationship in the wavelength-De Bruijn element mapping table is set automatically according to the physical constraints of the structured light system. Through the coding sequence with specific wavelength combinations, the different fringe frequencies are spatially distributed in such a way that the wavelength combination sequence has local window uniqueness within any sliding window of length n.

[0023] This provides physical constraints for subsequent neural networks to achieve unambiguous absolute phase calculation through global observation;

[0024] Furthermore, the generation of the non-uniform wavelength encoded N-step phase-shift fringe pattern is based on the standard N-step phase-shift intensity model encoding.

[0025] ;

[0026] ;

[0027] in, This represents the horizontal and vertical coordinates of each pixel in the projector coordinate system in the N-step phase-shifted fringe pattern. To represent the i-th N-step phase shift fringe pattern, For the average strength of the code, To encode the modulation intensity, N represents the number of N-step phase shift fringe patterns. X represents the normalized phase space coordinates of the N-step phase-shifted fringe pattern modulated by the De Bruijn wavelength sequence in the projector coordinate system. This represents the ordinate of a pixel on the N-step phase-shifted fringe plane; Represents the De Bruijn wavelength sequence. and These represent the physical wavelengths corresponding to the 1st and jth symbols in the DeBruijn wavelength sequence, respectively.

[0028] Furthermore, the four-channel physical feature input tensor is represented as follows:

[0029] ;

[0030] in, This represents the four-channel physical feature input tensor. This indicates a cascading operation at the channel level; This represents the sinusoidal feature components constructed based on the wrapper phase. This represents the cosine feature components constructed based on the wrapped phase. This represents the extracted phase gradient features, and A represents the background intensity features of the background image with a fixed gray value.

[0031] , ;

[0032] ;

[0033] in, Represents the horizontal and vertical coordinates of the camera plane. This represents the sinusoidal feature components constructed based on the wrapper phase. To obtain the package phase, This indicates the wrap-around phase of the next pixel that is laterally adjacent to the current pixel. The imaginary unit, Represents a complex exponential function. Represents the principal value function of the argument; Represents the camera pixel coordinates At this point, the original phase difference value between adjacent pixels along the horizontal direction is limited to the domain of the result. .

[0034] Furthermore, the working process of the multi-scale feature fusion neural network is as follows:

[0035] Step 4.1: The constructed four-channel physical feature input tensor is simultaneously input into four parallel branches. Each branch first passes through a standard convolutional layer for channel dimension mapping. Then, each branch performs spatial downsampling using max pooling operators with different sampling strides. The downsampling strides s of the four parallel branches are 1, 2, 4, and 8, respectively. When s=1, no downsampling operation is performed and no pooling is done, maintaining the original resolution. When s=2, 4, and 8, the corresponding max pooling operators are used for spatial downsampling. Each branch uses a pooling operator to obtain feature information under different receptive fields.

[0036] Step 4.2: Deploy four consecutively stacked residual blocks as core feature learning units within each parallel branch. The downsampled feature tensors directly enter the core feature learning units. Within each parallel branch, each residual block integrates sequentially connected convolutional layers, batch normalization layers, and linear activation layers to form a residual mapping path. The stacking of residual blocks achieves deep nonlinear mapping and feature enhancement of multi-scale stripe features.

[0037] s=1 means no pooling, no spatial downsampling, and the original resolution is maintained. Figure 6 The first branch in the middle does not have a pooling layer drawn;

[0038] Step 4.3: At the end of each parallel branch, the extracted multi-scale feature map is upsampled using the bilinear interpolation algorithm to restore its spatial resolution to the original input size, thereby achieving dimensional alignment with the detail feature extraction branch.

[0039] Step 4.4: Then, channel-level feature concatenation and fusion are performed to achieve probability prediction mapping from fused features to each De Bruijn coding order.

[0040] Furthermore, a physical wavelength-weighted method is used to train and optimize the multi-scale feature fusion neural network. The specific steps are as follows:

[0041] Step 5.1: Based on the non-uniform stripe coding characteristics of De Bruijn, a loss function based on symbol physical wavelength weighting is constructed, and the multi-scale feature fusion neural network is iteratively trained based on the loss function until a preset termination condition is met. The preset termination condition is one or more of the following: the loss function converges to a preset threshold, the verification error no longer decreases, or the number of training iterations reaches a preset upper limit.

[0042] Step 5.2: Determine the final predicted phase order for each pixel by finding the maximum index of the confidence tensor in the channel dimension;

[0043] Step 5.3: Combine the calculated wrapper phase with the predicted phase order to generate the absolute phase.

[0044] Furthermore, the mathematical expression for the loss function based on symbol physical wavelength weighting is:

[0045] ;

[0046] Where M is the total number of pixels in the N-step deformed phase-shifted fringe pattern, and K is the total number of phase orders. and Let represent the true label and predicted probability of pixel i belonging to order k, respectively. The weighting coefficient is the physical category weight set for the k-th phase order, which is preset according to the physical wavelength corresponding to each symbol in the De Bruijn sequence.

[0047] Since different symbols occupy different pixel widths in the N-step phase-shifted fringe pattern, this weight can balance the contribution of symbols of different scales in the network training process, thereby improving the robustness of the model to non-uniform feature extraction.

[0048] Furthermore, The calculation rules follow the inverse wavelength principle:

[0049] ;

[0050] in, This represents the k-th physical wavelength in the wavelength sequence, where k represents the phase order corresponding to the pixel in the N-step deformed phase-shifted fringe pattern. The phase order is predicted by a multi-scale feature fusion neural network. is the maximum physical wavelength in the wavelength sequence.

[0051] Secondly, a multi-scale feature fusion neural network phase unrolling system based on De Bruijn stripe coding includes:

[0052] Mapping table construction and encoding unit: Construct a mapping table based on wavelength-De Bruijn elements, design a globally unique non-uniform wavelength encoding scheme based on the mapping table, and then use the designed encoding to generate an N-step phase shift fringe pattern;

[0053] Wrapping phase calculation unit: Projects the generated N-step phase shift fringe pattern and background image with fixed gray value onto the object under test. The camera synchronously acquires the N-step deformed phase shift fringe pattern modulated by the object under test and the background image with fixed gray value, extracts the background intensity and calculates the wrapping phase.

[0054] Four-channel physical feature input tensor construction unit: Extract sine and cosine feature components from the wrapped phase obtained by solving, calculate the phase gradient feature, and construct a four-channel physical feature input tensor by combining it with the background intensity feature;

[0055] The multi-scale feature fusion neural network building unit adopts a four-way parallel convolutional architecture, and then connects to the feature cascade module to obtain the multi-scale feature fusion neural network. The multi-scale feature fusion neural network building unit aligns and fuses the extracted multi-scale semantic information in spatial dimensions, and finally outputs a classification probability distribution map of pixel-level phase order.

[0056] Network training unit: The feature input tensor constructed by the four-channel physical feature input tensor construction unit is used to train and build a multi-scale feature fusion neural network;

[0057] Phase Unfolding Unit: Utilizing a trained multi-scale feature fusion neural network, the unit generates corresponding input tensors from the N-step deformed phase-shift fringe pattern and the background image with fixed grayscale values ​​obtained by sequentially calling the mapping table construction and encoding unit, the wrapping phase solution unit, and the four-channel physical feature input tensor construction unit. This predicts the pixel-level phase order and achieves phase unfolding.

[0058] The beneficial effects of this invention are as follows:

[0059] 1. This invention proposes a phase unfolding method for a multi-scale feature fusion neural network based on De Bruijn stripe coding. By constructing an innovative architecture combining physical feature enhancement and multi-scale fusion, it fundamentally solves the pain points of traditional deep learning, such as the lack of physical semantics in the input and the difficulty in simultaneously considering global and local features. This invention abandons the single input mode and constructs a four-channel physical feature input tensor containing background intensity, sine and cosine components, and phase gradient. It utilizes sine and cosine mapping to provide a continuous and smooth input manifold for the network to eliminate periodic jumps, and combines this with the complex domain spatial difference method to extract the phase gradient. This transforms the abstract De Bruijn wavelength evolution law into supervised physical features, thereby achieving a deep perception of "phase state" and "evolution logic" at the input layer. Based on this, this invention designs a multi-scale feature fusion network with four parallel branches, utilizing diverse receptive fields to simultaneously extract the long-wavelength global constraint logic and short-wavelength local fine details of the De Bruijn encoding. By using max pooling operators of different phase lengths and a cascaded fusion mechanism, the model can achieve high-precision phase unwrapping with only a single set of N-step phase shift fringes without the need for additional projection auxiliary sequences. It exhibits strong robustness to traditionally failure-prone scenarios such as abrupt depth changes, isolated objects, and complex textures, effectively overcoming the edge misjudgment problem caused by limited receptive field, and achieving the optimal balance between global encoding uniqueness and local shape resolution.

[0060] 2. This invention addresses the physical characteristic of uneven information density in De Bruijn non-uniform stripe coding by proposing a loss function based on the physical wavelength of the symbols and establishing a physical prior-driven asymmetric supervision mechanism. This mechanism, by introducing wavelength-inverse weights, allocates greater loss gain to shorter wavelength regions with higher spatial frequencies and denser phase information, effectively solving the problem of insufficient supervision and feature learning in shorter wavelength regions due to their smaller pixel proportion under traditional equal-weighted loss functions. This design not only significantly improves the model's ability to identify key detailed features in the encoded sequence but also, through stringent optimization constraints, specifically suppresses boundary effects at the junctions of symbols of different wavelengths, effectively correcting the "pseudo-jumps" in phase order prediction induced by wavelength switching, fundamentally ensuring the continuity of global phase unfolding. Furthermore, this physically weighted loss function, as an implicit regularization constraint, guides the model to deeply capture highly discriminative encoded logical features rather than random noise distributions, greatly enhancing the system's ability to suppress complex noise such as low reflectivity, ambient light interference, and motion blur. By coupling the physical wavelength prior depth into the target optimization function, this invention significantly reduces the order misjudgment rate of non-uniform fringes at wavelength switching, enabling the 3D reconstruction system to stably lock the physically meaningful phase order even when facing industrial-grade measurement scenarios, thus significantly improving the reliability of measurement and the stability of results.

[0061] 3. This invention achieves pixel-level absolute phase recovery under a single set of non-uniform wavelength fringes, fundamentally eliminating the reliance on redundant auxiliary sequences such as Gray codes or multi-frequency heterodynes in traditional phase unwrapping methods, thus greatly simplifying the projection coding process. This technical characteristic not only significantly reduces the number of projection frames required for a single 3D reconstruction, multiplying the system's time sampling frequency and coding efficiency, but also significantly enhances the system's ability to capture moving objects, effectively avoiding motion artifacts and phase deviations caused by multi-frame time delays in high-speed measurement scenarios, providing a solid technical guarantee for achieving high spatiotemporal resolution 3D real-time topography measurement. Simultaneously, this robust phase unwrapping method, relying solely on the characteristics of N-step phase-shifting fringes, significantly reduces the hardware dependence on high-specification projection equipment and complex high-speed synchronous control systems. Without requiring additional hardware investment, this invention utilizes the powerful feature mining capabilities of deep learning to compensate for the simplification of the physical sequence, realizing a low-cost, lightweight, and highly real-time feasible 3D measurement solution. This has significant practical value for promoting the application of structured light imaging technology in time-delay-sensitive fields such as industrial online inspection, human-computer interaction, and dynamic visual monitoring. Attached Figure Description

[0062] Figure 1 This is a schematic diagram of the phase unrolling method of a multi-scale feature fusion neural network based on De Bruijn stripe coding in Embodiment 1 of the present invention;

[0063] Figure 2 This is a schematic diagram of the five-element, third-order non-uniform wavelength De Bruijn N-step (N=4) phase shift fringe pattern generated in Embodiment 1 of the present invention, wherein (a), (b), (c), and (d) correspond to a phase shift of 0, respectively. , , The first to the Nth N-step phase shift fringe patterns (specifically, in this embodiment, N=4, i.e., four-step phase shift);

[0064] Figure 3 The images shown are N-step deformed phase-shift fringe patterns modulated by the surface morphology of the object under test, acquired by a camera in Embodiment 1 of the present invention, and a background image with a fixed gray value. Among them, (a), (b), (c), and (d) are the first to Nth deformed phase-shift fringe patterns corresponding to the N-step phase shift (specifically, N=4 in this embodiment, i.e., four-step phase shift), and (e) is a background image with a fixed gray value.

[0065] Figure 4 The encapsulation phase diagram is obtained by using the phase shift method in Embodiment 1 of the present invention;

[0066] Figure 5This is a schematic diagram of the four-channel physical feature input tensor distribution extracted by wrapping phase in Embodiment 1 of the present invention. (a) is the sine feature component, (b) is the cosine feature component, (c) is the phase gradient feature, and (d) is the background intensity feature.

[0067] Figure 6 This is a schematic diagram of a multi-scale feature fusion neural network architecture with parallel branches in Embodiment 1 of the present invention;

[0068] Figure 7 This is the predicted phase order diagram in Embodiment 1 of the present invention;

[0069] Figure 8 This is the predicted absolute phase map in Embodiment 1 of the present invention;

[0070] Figure 9 This is a cross-sectional view of the wrapped phase and the unfolded absolute phase in Embodiment 1 of the present invention, shown in row 350. Detailed Implementation

[0071] The present invention will be further described below with reference to the accompanying drawings and embodiments. However, this should not be construed as limiting the scope of the above-described subject matter of the present invention to the following embodiments; all technologies implemented based on the content of the present invention fall within the scope of the present invention.

[0072] To better illustrate the method of this invention, this embodiment 1 uses a 5-ary 3rd-order De Bruijn sequence as an example, combined with a four-step phase-shifting encoding scheme, to illustrate the three-dimensional measurement of an industrial part.

[0073] like Figure 1 The diagram shown is a flowchart of the present invention. A phase unrolling method for a multi-scale feature fusion neural network based on De Bruijn stripe coding includes the following steps:

[0074] Step 1: Construct a mapping table based on wavelength-De Bruijn elements, and design a globally unique non-uniform wavelength coding scheme according to the mapping table. Then, use the designed coding to generate an N-step (N=4) phase shift fringe pattern.

[0075] To ensure the uniformity of the calculated phase, five preset physical wavelengths were selected for sinusoidal encoding: 6, 8, 10, 12, and 14. These wavelengths were then assigned to the five De Bruijn elements, constructing a wavelength-De Bruijn element mapping table, as shown in Table 1.

[0076] Table 1 De Bruijn Element-Wavelength Mapping Table

[0077] De Bruijn elements 1 2 3 4 5 wavelength 6 8 10 12 14

[0078] Next, a set of 5-element 3rd order De Bruijn wavelength sequences was constructed. Considering the projector resolution and spatial periodicity constraints, the first 80 elements of the sequence were taken to form the target encoding sequence, as shown in formula (1).

[0079] (1)

[0080] Among them, F DB This represents a 5-ary, 3rd-order De Bruijn sequence. Next, using F... DB The wavelength in the image is used to generate a four-step De Bruijn phase-shift fringe pattern, as shown below. Figure 2 As shown, the calculation formula is:

[0081] (2)

[0082] in, (3)

[0083] This represents the x and y coordinates of each pixel in the projector coordinate system in the four-step phase-shift fringe pattern. For the i-th four-step phase-shift fringe pattern, For the average strength of the code, To encode the modulation intensity, N represents the number of N-step phase shift patterns. X represents the normalized phase space coordinates of the N-step phase-shifted fringe pattern modulated by the DeBruijn wavelength sequence in the projector coordinate system. This represents the vertical coordinate of a pixel on the plane of a four-step phase-shift fringe pattern.

[0084] Represents the De Bruijn wavelength sequence. and These represent the physical wavelengths corresponding to the 1st and jth symbols in the DeBruijn wavelength sequence, respectively.

[0085] Step 2: Project the generated four-step phase-shift fringe pattern and a background image with a fixed grayscale value onto the object under test. Simultaneously, the camera acquires the deformed four-step phase-shift fringe pattern modulated by the object under test and a background image with a fixed grayscale value. Extract the background intensity and calculate the wrapping phase.

[0086] A fixed uniform grayscale pattern and a four-step phase-shifted fringe pattern with a pixel grayscale value of 127 (choosing a pixel grayscale value of 127 because this value is the median of 8-bit grayscale, which can effectively avoid saturation and low signal-to-noise ratio problems in the linear response region of the background intensity feature A acquired by the camera, and at the same time match the average intensity of the fringe to ensure the best phase solution accuracy) are projected onto the scene to be tested by a projector. Simultaneously, an industrial camera captures a background modulation image, which is used as the background intensity component. and four-step deformed stripe pattern ,like Figure 3 As shown, the nonlinear wrapping phase is further calculated using formula (4), as follows: Figure 4 As shown;

[0087] (4)

[0088] in, This indicates the calculation of the arctangent function. Represents the horizontal and vertical coordinates of the camera plane. This represents the i-th deformed fringe pattern, where i represents the phase shift index, and N=4 represents the total number of phase shift steps. This indicates the phase shift.

[0089] Step 3: Extract sine and cosine feature components from the obtained package phase through the solution, calculate the phase gradient features, and construct a four-channel physical feature input tensor by combining the background intensity features;

[0090] In this embodiment 1, based on the solution obtained in step 2, it includes 2 Periodically changing wrap phase First, its spatially continuous sinusoidal eigencomponents are extracted through nonlinear mapping. With cosine characteristic components ,like Figure 5 As shown in formula (5).

[0091] , (5)

[0092] in, The package phase obtained in step 2, This represents the sinusoidal feature components constructed based on the wrapper phase. This represents the cosine feature components constructed based on the wrapped phase.

[0093] The above mapping transforms discontinuous phase values ​​into continuous, smooth input manifolds, effectively eliminating spurious gradient interference caused by phase jumps in neural networks.

[0094] Subsequently, the phase gradient component reflecting the local frequency evolution characteristics was extracted using the spatial difference method. ,like Figure 5 As shown. In order to accurately characterize the wavelength evolution law of De Bruijn encoding in space, this embodiment adopts the phase difference calculation strategy under complex domain mapping, as shown in formula (6):

[0095] (6)

[0096] in, This indicates the wrap-around phase of the next pixel that is horizontally adjacent to the current pixel. The imaginary unit, Represents a complex exponential function. This represents the principal value function of the argument. This formula automatically performs a "de-jump" operation by mapping the original phase difference between adjacent pixels to the complex plane, thus improving the calculation results. Always limited to Within a certain range, the evolution trend of stripe density modulated by the object's depth can be realistically reproduced.

[0097] Finally, the background intensity feature A and the sinusoidal feature components obtained in step 2 are... Cosine characteristic components and gradient features Perform a concatenation operation along the channel dimension to construct a four-channel physical feature input tensor, as shown in formula (7):

[0098] (7)

[0099] in, This indicates a cascading operation at the channel dimension. This represents the sinusoidal feature components constructed based on the wrapper phase. This represents the cosine feature components constructed based on the wrapped phase. Represents the camera pixel coordinates The original phase difference between adjacent pixels along the horizontal direction is given by the tensor X, where A represents the background intensity feature. This tensor X serves as the input to the subsequent multi-scale feature fusion neural network, achieving deep coupling between the "static phase distribution" and the "dynamic evolution logic," and providing rich physical prior information for the network to directly predict the phase order in a single sequence.

[0100] Step 4: Construct a multi-scale feature fusion neural network;

[0101] This embodiment 1 constructs a multi-scale feature fusion neural network with four parallel branches, such as... Figure 6 As shown. First, the four-channel physical features constructed in step 3 are input into the tensor. Simultaneously, four parallel branches are input. Each branch uses a max-pooling operator with different step lengths for spatial downsampling to obtain feature information under different receptive fields. Specifically, the downsampling step lengths s of the first to fourth branches are set to 1, 2, 4, and 8, respectively, as shown in formula (8):

[0102] (8)

[0103] Where s is the downsampling step size, This represents a max pooling operation with a step size of s. This represents the four-channel physical feature input tensor. The input tensor is the four-channel physical feature after downsampling of the s-th branch. Through this operation, the network can simultaneously capture the globally unique constraint logic encoded by De Bruijn (large step-size branch) and the local topographic details of the surface of the tested industrial part (small step-size branch).

[0104] Within each parallel branch, four consecutively stacked residual blocks are deployed as core feature learning units to enhance the network's ability to abstractly represent the physical features of non-uniform stripes. The mathematical model of the residual block is shown in formula (9):

[0105] (9)

[0106] in, This represents the four-channel physical feature input tensor entering the l-th residual block. This represents the four-channel physical feature tensor output after processing by the l-th residual block. denoted by , where represents a non-linear activation function (such as ReLU), F represents a residual mapping function consisting of a standard convolutional layer, a batch normalization layer, and a linear activation layer, and S represents a shortcut connection.

[0107] After feature learning is completed, the outputs of each downsampling branch (s=2,4,8) are upsampled and aligned using bilinear interpolation to ensure their spatial dimensions match those of the first branch. The aligned multi-scale feature map is shown below. Perform cascading fusion at the channel level, as shown in formula (10):

[0108] (10)

[0109] in, This represents the feature map of each branch after bilinear interpolation upsampling and alignment. Indicates cascading. This represents the fused feature tensor obtained after performing the channel cascading operation.

[0110] Features after fusion Deep fusion is performed through feature refinement mapping g, and finally, the classification weight operator is used. Mapped to phase order confidence distribution tensor As shown in formula (11):

[0111] (11)

[0112] in, For feature refinement mapping, For classification weight operators, This represents the fused feature tensor obtained after performing the channel concatenation operation. The number of channels corresponds to the preset total number of phase orders K. This structure realizes a direct mapping from multi-scale physical priors to absolute phase orders.

[0113] Specifically, in order to train the neural network, this embodiment pre-prepared a ground truth dataset. A standard scene was captured and decoded using a traditional Gray code combined with complementary Gray code (CGC) algorithm to obtain pixel-level absolute phase order as the ground truth label.

[0114] Step 5: Training optimization and absolute phase generation based on physical constraints.

[0115] During the training phase of the neural network, to enable the model to fully learn the unique features in the De Bruijn non-uniform coding sequence, this embodiment constructs a cross-entropy loss function based on symbol physical wavelength weighting. According to the wavelength mapping relationship determined in step 1.1, the maximum wavelength benchmark is taken. For each phase order k, calculate the corresponding physical weight. The weighted loss function constructed in this way is shown in formula (12):

[0116] (12)

[0117] Where M is the total number of pixels in the four-step phase-shifting fringe pattern. and These represent the true label of pixel i belonging to order k and the predicted probability of the network output, respectively. This weighting mechanism guides the network to automatically focus on physical regions with higher information density by increasing the error penalty for narrow wavelength (high frequency) regions.

[0118] The calculation rules follow the inverse wavelength principle:

[0119]

[0120] in, This represents the k-th physical wavelength in the wavelength sequence, where k represents the phase order corresponding to the pixel in the four-step deformed phase-shift fringe pattern. The phase order is predicted by a multi-scale feature fusion neural network. is the maximum physical wavelength in the wavelength sequence.

[0121] Step 5.1: Based on the non-uniform stripe coding characteristics of De Bruijn, a loss function based on symbol physical wavelength weighting is constructed, and the multi-scale feature fusion neural network is iteratively trained based on the loss function until a preset termination condition is met. The preset termination condition is one or more of the following: the loss function converges to a preset threshold, the verification error no longer decreases, or the number of training iterations reaches a preset upper limit.

[0122] Step 5.2: Determine the final predicted phase order for each pixel by finding the maximum index of the confidence tensor in the channel dimension;

[0123] Step 5.3: Combine the calculated wrapper phase with the predicted phase order to generate the absolute phase.

[0124] During the inference testing phase, a multi-channel feature tensor containing the shape information of the object under test is input into a trained multi-scale feature fusion neural network to obtain its output pixel-level confidence distribution tensor. The channel dimension of this tensor corresponds to a preset total number of orders K, and the value represents the confidence probability of a pixel belonging to each order. Subsequently, an argmax operation is performed on the channel dimension to directly determine the discrete phase order to which each pixel belongs. As in formula (13):

[0125] (13)

[0126] in, This is the pixel-level confidence distribution tensor of the neural network output.

[0127] like Figure 7 As shown, the final phase order is determined by finding the index of the maximum value of the confidence tensor in the channel dimension. Combined with the wrapping phase in step 2, the final absolute phase is synthesized according to formula (14):

[0128] (14)

[0129] in, The package phase obtained from step 2, This represents the predicted phase order.

[0130] like Figure 8 As shown, through the above process, this embodiment achieves high-precision, fuzz-free reconstruction of the true surface morphology of the measured object (industrial parts), greatly shortening the execution time of complex algorithms while ensuring reconstruction accuracy, and significantly improving the efficiency of 3D reconstruction. Figure 9 The cross-sectional distribution of the wrapped phase and the unfolded absolute phase at row 350 of the phase diagram in Embodiment 1 of the present invention is given to visually verify the reconstruction effect. The red line represents the wrapped phase. The blue line represents the result after phase expansion: absolute phase. ).

[0131] Example 2

[0132] A multi-scale feature fusion neural network phase unrolling system based on De Bruijn stripe coding includes:

[0133] Mapping table construction and encoding unit: Construct a mapping table based on wavelength-De Bruijn elements, design a globally unique non-uniform wavelength encoding scheme based on the mapping table, and then use the designed encoding to generate an N-step phase shift fringe pattern;

[0134] Wrapping phase calculation unit: Projects the generated N-step phase shift fringe pattern and a background image with a fixed gray value onto the object under test. The camera synchronously acquires the N-step deformed phase shift fringe pattern modulated by the object under test and the background image with a fixed gray value, extracts the background intensity and calculates the wrapping phase.

[0135] Four-channel physical feature input tensor construction unit: Extract sine and cosine feature components from the wrapped phase obtained by solving, calculate the phase gradient feature, and construct a four-channel physical feature input tensor by combining it with the background intensity feature;

[0136] The multi-scale feature fusion neural network building unit adopts a four-way parallel convolutional architecture, and then connects to the feature cascade module to obtain the multi-scale feature fusion neural network. The multi-scale feature fusion neural network building unit aligns and fuses the extracted multi-scale semantic information in spatial dimensions, and finally outputs a classification probability distribution map of pixel-level phase order.

[0137] The prediction is a pixel-level classification of the phase order, which is used to subsequently determine the phase order to achieve phase unrolling;

[0138] Network training unit: The feature input tensor constructed by the four-channel physical feature input tensor construction unit is used to train and build a multi-scale feature fusion neural network;

[0139] Phase Unfolding Unit: Utilizing a trained multi-scale feature fusion neural network, the N-step deformed phase-shifting fringe pattern and a background image with a fixed grayscale value are processed sequentially by the mapping table construction and encoding unit, the wrapping phase solution unit, and the four-channel physical feature input tensor construction unit to generate the corresponding input tensor. The input tensor is then fed into the trained multi-scale feature fusion neural network construction unit to predict the pixel-level phase order, thereby achieving phase unfolding.

[0140] It should also be understood that the specific implementation process of each module is described in the above method. This invention will not repeat it here. The above division of functional modules is only for illustrative purposes. In some embodiments, some functional modules can be combined and some functional modules can be separated. Each functional module can be implemented in software, hardware, or a combination of software and hardware. The software and hardware devices include, but are not limited to, general-purpose computer equipment, programmable gate arrays, digital signal processors, microprocessors and their corresponding programming or burning software.

[0141] Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0142] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application refers to flowchart illustrations and / or instructions executed by a processor of a method, apparatus (system), and computer program product according to embodiments of this application to create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams. These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowchart illustrations and / or one or more block diagrams. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more blocks of a block diagram.

[0143] The above embodiments are preferred embodiments of this application. Those skilled in the art can make various changes or improvements based on them. Without departing from the overall concept of this application, these changes or improvements should fall within the scope of protection claimed in this application.

Claims

1. A phase unwrapping method for a multi-scale feature fusion neural network based on De Bruijn stripe coding, characterized in that, Includes the following steps: Step 1: Construct a mapping table based on wavelength-De Bruijn elements, and based on the non-uniform wavelength coding scheme of the mapping table, generate an N-step phase shift fringe pattern based on the N-step phase shift algorithm. The non-uniform wavelength coding scheme has global uniqueness. Step 2: Project the generated N-step phase-shift fringe pattern and a background image with a fixed gray value onto the object under test. Simultaneously, the camera acquires the N-step deformed phase-shift fringe pattern modulated by the object under test and a background image with a fixed gray value. Extract the background intensity and calculate the encapsulated phase. Step 3: Extract sine and cosine feature components from the obtained package phase through the solution, calculate the phase gradient features, and construct a four-channel physical feature input tensor by combining the background intensity features; Step 4: Construct a multi-scale feature fusion neural network; The multi-scale feature fusion neural network adopts a four-way parallel convolutional architecture, which is then connected to a feature cascade module to align and fuse the extracted multi-scale semantic information in spatial dimensions, and finally outputs a classification probability distribution map of pixel-level phase order. Step 5: Using the four-channel physical feature input tensor constructed in Step 3, train the multi-scale feature fusion neural network constructed in Step 4; during the inference stage, based on the wavelength-De Bruijn element mapping table, the actual N-step deformed phase shift fringe pattern and a background image with a fixed gray value, execute Step 2 and Step 3 in sequence to obtain the input tensor, input the input tensor into the trained multi-scale feature fusion neural network, predict the pixel-level phase order, and combine it with the wrapped phase obtained in Step 2 to realize phase unwrapping.

2. The method according to claim 1, characterized in that, The generation process of the N-step phase-shifted fringe pattern is as follows: Step 1.1: Select m wavelengths and De Bruijn elements to construct a wavelength-De Bruijn element mapping table; Step 1.2: Based on the wavelength-De Bruijn element mapping table construction rules, generate an m-element n-order De Bruijn wavelength sequence, and then map each symbol in the De Bruijn wavelength sequence to a wavelength fringe segment corresponding to the wavelength, thereby generating an N-step phase-shift fringe pattern with non-uniform wavelength encoding.

3. The method according to claim 2, characterized in that, The wavelength-De Bruijn element mapping relationship is as follows: DeBruijn elements {db1, db2, ..., db...} m } Corresponding to wavelengths respectively , ,..., }; The mapping relationship in the wavelength-De Bruijn element mapping table is set automatically according to the physical constraints of the structured light system. Through the coding sequence with specific wavelength combinations, the different fringe frequencies are spatially distributed in such a way that the wavelength combination sequence has local window uniqueness within any sliding window of length n.

4. The method according to claim 2, characterized in that, The generation of the N-step phase-shifted fringe pattern with non-uniform wavelength encoding is based on the standard N-step phase-shifted light intensity model encoding; ； ； in, This represents the horizontal and vertical coordinates of each pixel in the projector coordinate system in the N-step phase-shifted fringe pattern. To represent the i-th N-step phase shift fringe pattern, For the average strength of the code, To encode the modulation intensity, N represents the number of N-step phase shift fringe patterns. X represents the normalized phase space coordinates of the N-step phase-shifted fringe pattern modulated by the De Bruijn wavelength sequence in the projector coordinate system. This represents the ordinate of a pixel on the N-step phase-shifted fringe plane; Represents the De Bruijn wavelength sequence. and These represent the physical wavelengths corresponding to the 1st and jth symbols in the DeBruijn wavelength sequence, respectively.

5. The method according to claim 1, characterized in that, The four-channel physical feature input tensor is represented as follows: ； in, This represents the four-channel physical feature input tensor. This indicates a cascading operation at the channel level; This represents the sinusoidal feature components constructed based on the wrapper phase. This represents the cosine feature components constructed based on the wrapped phase. This represents the extracted phase gradient features, and A represents the background intensity features of the background image with a fixed gray value. ， ； ； in, Represents the horizontal and vertical coordinates of the camera plane. To obtain the package phase, This indicates the wrap-around phase of the next pixel that is laterally adjacent to the current pixel. The imaginary unit, Represents a complex exponential function. Represents the principal value function of the argument; Represents the camera pixel coordinates At this point, the original phase difference value between adjacent pixels along the horizontal direction is limited to the domain of the result. .

6. The method according to claim 1, characterized in that, The working process of a multi-scale feature fusion neural network is as follows: Step 4.1: The constructed four-channel physical feature input tensor is simultaneously input into four parallel branches. Each branch first passes through a standard convolutional layer for channel dimension mapping, and then each branch performs spatial downsampling using max pooling operators with different sampling strides. The downsampling strides s of the four parallel branches are 1, 2, 4, and 8, respectively. When s=1, no downsampling operation is performed and no pooling is done, maintaining the original resolution. When s=2, 4, and 8, the corresponding max pooling operators are used for spatial downsampling to obtain feature information under different receptive fields. Step 4.2: Deploy four consecutively stacked residual blocks as core feature learning units within each parallel branch. The downsampled feature tensors directly enter the core feature learning units. Within each parallel branch, each residual block integrates sequentially connected convolutional layers, batch normalization layers, and linear activation layers to form a residual mapping path. The stacking of residual blocks achieves deep nonlinear mapping and feature enhancement of multi-scale stripe features. Step 4.3: At the end of each parallel branch, the extracted multi-scale feature map is upsampled using the bilinear interpolation algorithm to restore its spatial resolution to the original input size, thereby achieving dimensional alignment with the detail feature extraction branch. Step 4.4: Then, channel-level feature concatenation and fusion are performed to achieve probability prediction mapping from fused features to each De Bruijn coding order.

7. The method according to claim 1, characterized in that, The multi-scale feature fusion neural network is trained and optimized using a physical wavelength weighting method. The specific steps are as follows: Step 5.1: Based on the non-uniform stripe coding characteristics of De Bruijn, a loss function based on symbol physical wavelength weighting is constructed, and the multi-scale feature fusion neural network is iteratively trained based on the loss function until a preset termination condition is met. The preset termination condition is one or more of the following: the loss function converges to a preset threshold, the verification error no longer decreases, or the number of training iterations reaches a preset upper limit. Step 5.2: Determine the final predicted phase order for each pixel by finding the maximum index of the confidence tensor in the channel dimension; Step 5.3: Combine the calculated wrapper phase with the predicted phase order to generate the absolute phase.

8. The method according to claim 7, characterized in that, The mathematical expression for the loss function based on symbol physical wavelength weighting is: ； Where M is the total number of pixels in the N-step deformed phase-shifted fringe pattern, and K is the total number of phase orders. and Let represent the true label and predicted probability of pixel i belonging to order k, respectively. The weighting coefficient is the physical category weight set for the k-th phase order, which is preset according to the physical wavelength corresponding to each symbol in the De Bruijn sequence.

9. The method according to claim 8, characterized in that, The calculation rules follow the inverse wavelength principle: ； in, This represents the k-th physical wavelength in the wavelength sequence, where k represents the phase order corresponding to the pixel in the N-step deformed phase-shifted fringe pattern. The phase order is predicted by a multi-scale feature fusion neural network. is the maximum physical wavelength in the wavelength sequence.

10. A phase unrolling system for a multi-scale feature fusion neural network based on De Bruijn stripe coding, characterized in that, include: Mapping table construction and encoding unit: Construct a mapping table based on wavelength-De Bruijn elements, design a globally unique non-uniform wavelength encoding scheme based on the mapping table, and then use the designed encoding to generate an N-step phase shift fringe pattern; Wrapping phase calculation unit: Projects the generated N-step phase shift fringe pattern and a background image with a fixed gray value onto the object under test. The camera synchronously acquires the N-step deformed phase shift fringe pattern modulated by the object under test and the background image with a fixed gray value, extracts the background intensity and calculates the wrapping phase. Four-channel physical feature input tensor construction unit: Extract sine and cosine feature components from the wrapped phase obtained by solving, calculate the phase gradient feature, and construct a four-channel physical feature input tensor by combining it with the background intensity feature; Multi-scale feature fusion neural network building unit: A four-way parallel convolutional architecture is adopted, which is then connected to the feature cascade module to obtain a multi-scale feature fusion neural network. The multi-scale feature fusion neural network building unit aligns and fuses the extracted multi-scale semantic information in spatial dimensions, and finally outputs a classification probability distribution map of pixel-level phase order. Network training unit: The feature input tensor obtained by constructing the unit using the four-channel physical feature input tensor is used to train and construct a multi-scale feature fusion neural network; Phase Unfolding Unit: Utilizing a trained multi-scale feature fusion neural network, the N-step deformed phase-shifting fringe pattern and a background image with a fixed grayscale value are processed sequentially by the mapping table construction and encoding unit, the wrapping phase solution unit, and the four-channel physical feature input tensor construction unit to generate the corresponding input tensor. The input tensor is then fed into the trained multi-scale feature fusion neural network construction unit to predict the pixel-level phase order, thereby achieving phase unfolding.

Images