Failure typing module guided multi-channel non-Lambert surface FPP-PS fusion depth measurement method

Abstract

The invention discloses a multi-channel non-Lambert surface FPP-PS fusion depth measurement method guided by a failure typing module, and aims to solve the problems that a failure region under a single failure type is generally judged on the basis of a fixed threshold value or an empirical rule in the conventional method, a real mixed failure form is difficult to accurately establish, and the effectiveness of a subsequent repair or fusion algorithm is limited. According to the invention, after the obtained multi-source physical features are used to construct a multi-source joint feature model, the multi-source joint feature model is processed by a failure feature encoder to respectively form prototype vectors; multi-source joint feature data of an actual to-be-detected object and each prototype vector are substituted into the failure weight calculation model to obtain weights of corresponding repair channels, and finally a calculation result of a repair phase is output for each repair channel in a weighted fusion mode; the overall process of the method comprises four stages of multi-source physical feature construction, failure feature coding and prototype modeling, failure weight calculation and multi-channel phase repair fusion, and all the stages are cooperatively completed in the same calculation framework.

Description

Technical Field

[0001] Specifically, this invention relates to a multi-channel non-Lambertian surface FPP-PS fusion measurement depth method guided by a failure classification module. Background Technology

[0002] Fringe projection profilometry (FPP) and photometric stereo (PS) are two commonly used optical measurement methods in the current 3D measurement of complex curved surfaces. FPP achieves high-precision depth recovery through phase calculation. This technique projects a periodic fringe-coded pattern onto the surface of the object being measured using a projector. The fringes deform due to modulation of the object's surface height. A camera captures the deformed fringe image, and a computer decodes and unfolds the phase information in the image. Combined with system calibration parameters and triangulation principles, the true 3D coordinates of the object's surface are finally mapped, achieving high-precision reconstruction of the object's 3D shape.

[0003] Photometric stereometry is a non-contact measurement technique based on changes in illumination and the reflectivity of an object's surface. It involves taking multiple images of an object illuminated by light sources from different directions using a single camera at a fixed viewing angle, and then reconstructing the surface normal vector and three-dimensional shape of the object. Photometric stereometry estimates the surface normal vector using brightness information under multiple illumination conditions, which is used to supplement geometric details and local shape features.

[0004] In practical engineering applications, the objects under test often have complex reflective characteristics, such as metal surfaces, coated surfaces, or non-Lambertian reflective surfaces. Under these conditions, the phase measurement results of FPP are easily affected by factors such as saturation, modulation attenuation, and non-Lambertian reflection, resulting in local or structural phase distortion. Similarly, the PS method may also experience unstable normal estimation in highly reflective areas. Existing methods typically determine failure regions based on fixed thresholds or empirical rules, making it difficult to accurately distinguish different failure mechanisms, thus limiting the effectiveness of subsequent repair or fusion algorithms. Furthermore, the fusion process of fringe projection profilometry and photometric stereo phase has the following detailed challenges, concentrated in three core areas: information incompatibility due to differences in measurement principles, complex system calibration coupling, and insufficient data registration accuracy. These are analyzed in detail below:

[0005] First, FPP calculates absolute 3D coordinates through fringe phase changes, which is sensitive to the macroscopic geometric contours of objects, but its resolution for microscopic surface textures is limited by the fringe period. Photometric stereometry calculates surface normals through illumination grayscale differences, which excels at capturing microscopic morphology, but lacks an absolute depth reference, and integration easily accumulates errors. The measurement scales and information focuses of the two are inherently misaligned, and direct fusion will result in a discontinuity between macroscopic contours and microscopic textures.

[0006] Secondly, FPP requires rigorous extrinsic parameter calibration combining the projector and camera to ensure the mapping relationship between fringe phase and three-dimensional coordinates, while photometric stereo method requires precise calibration of the direction and intensity of multiple light sources and a fixed camera angle. The fusion system needs to simultaneously perform joint calibration between the projector, camera, and multiple light sources. Errors in any one of these links will affect the measurement results of both methods, causing the complexity of the calibration model to increase exponentially.

[0007] Then, regarding spatial registration, the point cloud data from FPP and the normal vector data from photometric stereo are based on the same field of view, but pixel-level precise matching is difficult; even slight camera shake or object displacement can cause spatial misalignment between the two types of data. In terms of temporal synchronization, FPP requires projecting multiple frames of stripe images, while photometric stereo requires switching between multiple light sources to capture multiple frames of grayscale images. The image acquisition sequence of both must be strictly synchronized; otherwise, data mismatch will occur in dynamic scenes.

[0008] Finally, FPP requires stable diffuse reflection characteristics of the object surface to avoid overexposure or reflection in the fringe image; the photometric stereo method, based on the Lambertian reflection model, is sensitive to specular and specular reflection areas and is prone to failure in normal vector calculation. The actual surface reflection characteristics of the object being measured will interfere with the measurement accuracy of both methods, making it difficult to compensate for errors using a unified model.

[0009] In summary, existing methods typically determine the failure region under a single failure type based on fixed thresholds or empirical rules, making it difficult to accurately establish the true mixed failure forms, thus limiting the effectiveness of subsequent repair or fusion algorithms. Summary of the Invention

[0010] This invention provides a multi-channel non-Lambertian surface FPP-PS fusion measurement depth method guided by a failure classification module to solve the above-mentioned problems.

[0011] A multi-channel non-Lambertian surface FPP-PS fusion depth measurement method guided by a failure classification module is proposed. The FPP-PS phase repair method involves constructing a multi-source joint feature model from the acquired multi-source physical features, processing the multi-source joint feature model through a failure feature encoder to form prototype vectors, inputting the multi-source joint feature data of the actual object to be detected and each prototype vector into a failure weight calculation model to obtain the weights of the corresponding repair channels, and finally outputting the calculation results of the repair phase of each repair channel through a weighted fusion method.

[0012] As a preferred option: the multi-source physical features are multi-source physical information fused from FPP and PS. The process of obtaining the multi-source physical features is as follows:

[0013] First, raw measurement results are obtained using fringe projection profilometry and photometric stereometry. These raw results are then used to construct a multi-source physical feature vector to characterize the phase failure state. For each pixel position (x, y), a joint physical feature vector is constructed:

[0014]

[0015] In the above formula, This represents the multivariate physical feature vector at the corresponding pixel.

[0016] The damaged phase value is obtained by fringe projection profilometry.

[0017] The stripe tone is used to characterize stripe quality and contrast level;

[0018] The local gradient characteristics of the damaged phase in space;

[0019] The surface normal vector is estimated using the photometric stereo method;

[0020] This provides the image brightness information for the corresponding pixel location.

[0021] As a preferred approach: After constructing the multi-source joint feature model, the process of processing the multi-source joint feature model into prototype vector models through a failure feature encoder is as follows:

[0022] After obtaining the multi-source physical features, these features are input into the failure feature encoding module. The failure feature encoding module performs nonlinear mapping and feature compression on the original physical quantities in the multi-source physical features to obtain low-dimensional embedded features that can characterize the phase failure state. The failure feature encoding process is expressed as follows:

[0023]

[0024] In the above formula, The failure feature embedding vector at pixel (x, y);

[0025] For failure feature encoding function;

[0026] It is a multi-source physical feature vector;

[0027] Then, the typical distribution locations of different phase failure mechanisms are processed, and several failure prototype vectors are introduced. The calculation formula is as follows:

[0028]

[0029] In the above formula, This is the failure prototype vector corresponding to the i-th type of phase failure mechanism;

[0030] d represents the dimension of the failure feature embedding space;

[0031] N is the preset number of failure mechanisms;

[0032] By calculating the distance relationship between the embedded features and each failure prototype, the network establishes the relative membership relationship between the current pixel and different failure mechanisms in the feature space, thereby completing the continuous failure perception modeling process.

[0033] As a preferred approach: the process of calculating the corresponding repair channels for each prototype vector model through failure weights involves obtaining the failure feature embedding vector and the corresponding failure prototype, calculating the similarity between the embedded features and each failure prototype, establishing the weight relationship of the current pixel under different failure mechanism assumptions, characterizing the relative degree of multiple failure mechanisms coexisting at the same pixel position, and finally constructing a corresponding phase repair channel for each preset failure mechanism to perform targeted correction processing on the damaged phase. The calculation process is as follows:

[0034] Failure weights are obtained by normalizing the distances between the embedded features and each failure prototype, and the corresponding calculation formula is as follows:

[0035]

[0036] The weights obtained by the above formula Let (x, y) be the relative confidence level of pixel (x, y) under the i-th type of failure mechanism assumption;

[0037] In the phase repair phase, the network constructs a corresponding phase repair channel for each type of failure mechanism. Each repair channel independently calculates the phase compensation amount under the same input conditions. The phase repair function corresponding to the i-th type of failure mechanism is expressed as:

[0038]

[0039] In the above formula, This is the phase repair network corresponding to the i-th type of failure subspace.

[0040] As a preferred solution: the network obtains the final repaired phase by weighted fusion of the phase compensation amounts output from each repair channel, and the calculation formula is as follows:

[0041]

[0042] The aforementioned network structure is trained end-to-end under a unified phase repair objective function constraint, ensuring that the failure feature encoding, failure prototype location, and parameters of each repair channel are optimized collaboratively during the training process, thereby achieving adaptive modeling and repair of various phase failure mechanisms under complex reflective surface conditions.

[0043] As a preferred solution, the calculation process for outputting the repair phase through weighted fusion of each repair channel is as follows:

[0044] The loss is calculated by introducing a weight distribution constraint, and the corresponding formula is as follows:

[0045]

[0046] In the above formula, This is a smoothing loss term used to constrain the distribution of failed weights, preventing weight collapse in the early stages of network training. As an auxiliary constraint, it works together with the main loss of phase restoration in the network training process; considering both phase restoration accuracy and training stability, the formula for calculating the overall network loss function is:

[0047]

[0048] In the above formula, λ is a weighting coefficient used to balance the influence of different loss terms;

[0049] The overall network loss function takes the accuracy of the final repaired phase as the core optimization objective, and integrates failure classification, failure subspace modeling and phase repair process into the same optimization framework to complete the repair calculation process that takes convergence into account during the overall training process.

[0050] Compared with existing technologies, this invention provides a multi-channel non-Lambertian surface FPP-PS fusion measurement depth method guided by a failure classification module, which has the following advantages:

[0051] The multi-channel non-Lambertian surface FPP-PS fusion depth measurement method guided by the failure classification module in this invention is essentially a quantitative method for phase repair based on the combination of FPP and PS through failure feature self-encoding and multi-channel repair fusion. In the implementation of this invention, the hard-discriminatory classification of phase failures is omitted. Instead, a continuous weight model is performed on multiple potential failure mechanisms through the learned failure feature embedding space, and then a weighted fusion process is applied to multiple repair results. This process is accurate and standardized. This invention avoids explicit and discrete classification of failure types, ensuring that the phase repair process maintains continuity and stability even when multiple failure mechanisms coexist or have ambiguous boundaries. The specific advantages of this invention are:

[0052] First, this invention specifically uses multi-source physical information from FPP and PS as input, extracts high-dimensional feature representations related to phase failure through a failure feature encoding network, and introduces several learnable failure prototype vectors into this feature space to characterize the typical distribution positions of different phase failure mechanisms in the feature space. By calculating the similarity between the input features and each failure prototype, the weight coefficients of various failure mechanisms are obtained. For each type of failure mechanism, a corresponding phase repair channel is constructed, and finally, the repaired phase result is output through a weighted fusion method.

[0053] Second: This invention can simplify and standardize the accurate differentiation of different failure mechanisms, extend the effectiveness of subsequent repair or fusion algorithms, and has a clear overall process and accurate calculation results. Specifically, it includes four stages: multi-source physical feature construction, failure feature encoding and prototype modeling, failure weight calculation, and multi-channel phase repair fusion. Each stage is completed collaboratively within the same calculation framework.

[0054] Third: This invention can adaptively distinguish different phase failure mechanisms and select the corresponding repair strategy fusion processing method according to the failure characteristics in the absence of clear failure labels, thereby improving the overall accuracy and robustness of FPP-PS joint measurement on complex reflective surfaces. Attached Figure Description

[0055] Figure 1 This is a schematic diagram of the process of the present invention;

[0056] Figure 2 This is a flowchart of the overall algorithm of the present invention;

[0057] Figure 3 This is a schematic diagram illustrating the process of integrating FPP and PS to form multi-source physical information. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] Specific implementation method one: Combining Figure 1 , Figure 2 and Figure 3This embodiment describes the FPP-PS phase repair method based on failure feature self-encoding. The FPP-PS phase repair method involves constructing a multi-source joint feature model from the acquired multi-source physical features, processing the multi-source joint feature model through a failure feature encoder to form prototype vectors, inputting the multi-source joint feature data of the actual object to be detected and each prototype vector into the failure weight calculation model to obtain the weights of the corresponding repair channels, and finally outputting the calculation results of the repair phase through a weighted fusion method for each repair channel.

[0060] Combination Figure 1 As shown, the overall algorithm flow of this invention is as follows: First, a joint physical feature is constructed based on the multi-source measurement results of FPP and PS. After failure feature encoding, it is mapped to the failure embedding space. The weight coefficients of different failure mechanisms are obtained by similarity calculation with multiple failure prototypes. The repair results are calculated in multiple phase repair channels respectively. Finally, the repaired phase output is obtained by weighted fusion.

[0061] Specific Implementation Method Two: This implementation method further defines Specific Implementation Method One. In this method, the multi-source physical features are multi-source physical information fused from FPP and PS. The acquisition process of multi-source physical features is essentially the establishment process of the overall network architecture. The overall network architecture adopted in this method is based on failure perception modeling driven by multi-source physical information, and integrates the measurement results of fringe projection profilometry and photometric stereo method into the same end-to-end trainable computational framework. The network consists of four functional levels: failure feature encoding, failure prototype modeling, failure weight estimation, and multi-channel phase repair and fusion. Each level is sequentially connected in the same forward propagation path and jointly optimized through a unified repair objective function.

[0062] At the input layer, the network receives multi-source physical quantities calculated by fringe projection profilometry and photometric stereometry, and combines them into a joint feature tensor to characterize various physical factors that may cause failure during phase measurement. Specifically, before phase repair processing, a multi-source physical feature is constructed to characterize the phase failure state based on the original measurement results obtained by fringe projection profilometry and photometric stereometry. This feature is not directly based on a single measurement quantity, but comprehensively considers phase information, fringe quality indicators, and surface reflection and geometric information to ensure sufficient expressive power for different phase failure mechanisms. Specifically, for each pixel position (x, y), a joint physical feature vector is constructed:

[0063]

[0064] In the above formula, This represents the multivariate physical feature vector at the corresponding pixel.

[0065] The damaged phase is specifically the damaged phase value obtained by fringe projection profilometry.

[0066] The stripe tone is used to characterize stripe quality and contrast level;

[0067] This refers to the spatial variation characteristics of the phase, specifically the local gradient characteristics of the damaged phase in space.

[0068] The surface normal vector is estimated using the photometric stereo method;

[0069] This refers to brightness information, specifically the image brightness information at the corresponding pixel location.

[0070] Specific Implementation Method Three: This implementation method is a further limitation of Specific Implementation Method One or Two. In this implementation method, after constructing the multi-source joint feature model, the process of processing the multi-source joint feature model through the failure feature encoder to form prototype vector models is as follows:

[0071] After obtaining the multi-source physical features, they are input into the failure feature encoding module. The original physical quantities undergo nonlinear mapping and feature compression to obtain low-dimensional embedded features that characterize the phase failure state. These embedded features do not directly correspond to specific physical quantities but rather depict the distribution characteristics of phase failure in the feature space. The calculation formula for the failure feature encoding process is as follows:

[0072]

[0073] In the above formula, : Failure feature embedding vector at pixel (x, y). This embedding space does not directly correspond to a specific physical quantity, but is used to characterize the distribution structure of different failure mechanisms in the feature space.

[0074] Failure feature encoding function, used to map physical features to the embedding space;

[0075] : Multi-source physical feature vectors;

[0076] In the embedding space, several failure prototype vectors are introduced as learnable parameters to characterize the typical feature centers of different phase failure mechanisms. That is, in order to characterize the typical distribution locations of different phase failure mechanisms in this feature space, several failure prototype vectors are introduced:

[0077]

[0078] In the above formula, This is the failure prototype vector corresponding to the i-th type of phase failure mechanism;

[0079] d represents the dimension of the failure feature embedding space;

[0080] N is the preset number of failure subspaces, and also the preset number of failure mechanisms;

[0081] The failure prototypes, as learnable parameters, are optimized together with the failure feature encoding module and the phase repair module during training. That is, by calculating the distance relationship between the embedded features and each failure prototype, the network establishes the relative membership relationship of the current pixel to different failure mechanisms in the feature space, thereby realizing continuous failure perception modeling.

[0082] After obtaining the failure feature embedding vector and its corresponding failure prototype, the similarity between the embedded features and each failure prototype is calculated to establish a weighted relationship between the current pixel under different failure mechanism assumptions. This weight is not used to form a discrete failure category determination, but rather to characterize the relative degree to which multiple failure mechanisms may coexist at the same pixel location. Finally, for each preset failure mechanism, a corresponding phase repair channel is constructed to specifically correct the damaged phase. Each repair channel independently calculates the phase repair amount under the same input conditions to reflect the phase compensation results under different failure assumptions. Finally, the phase repair amounts output by each repair channel are weighted and fused to obtain the final phase repair result.

[0083] Specific Implementation Method Four: This implementation method is a further limitation of Specific Implementation Methods One, Two, or Three. In this implementation method, the process of obtaining the corresponding repair channels after calculating the failure weights of each prototype vector model is as follows: After obtaining the failure feature embedding vector and the corresponding failure prototype, the similarity between the embedding feature and each failure prototype is calculated to establish the weight relationship of the current pixel under different failure mechanism assumptions, characterizing the relative degree of multiple failure mechanisms in the coexistence state at the same pixel position. Finally, for each preset failure mechanism, a corresponding phase repair channel is constructed to perform targeted correction processing on the damaged phase. The calculation process is as follows:

[0084] Failure weights are obtained by normalizing the distances between the embedded features and each failure prototype, and the corresponding calculation formula is as follows:

[0085]

[0086] The weights obtained by the above formula This weight represents the relative confidence level of pixel (x,y) under the assumption of the i-th type of failure mechanism. This weight does not constitute a discrete classification result, but is used for continuous modulation of subsequent multi-channel repair results.

[0087] In the phase repair phase, the network constructs a corresponding phase repair channel for each type of failure mechanism. Each repair channel independently calculates the phase compensation amount under the same input conditions. The phase repair function corresponding to the i-th type of failure mechanism is expressed as follows:

[0088]

[0089] In the above formula, This is the phase repair network corresponding to the i-th type of failure subspace.

[0090] Specific Implementation Method Five: This implementation method is a further limitation of Specific Implementation Methods One, Two, Three, or Four. In this implementation method, the network obtains the final repaired phase by weighted fusion of the phase compensation amounts output by each repair channel. The calculation formula is as follows:

[0091]

[0092] The aforementioned network structure is trained end-to-end under a unified phase repair objective function constraint, ensuring that the failure feature encoding, failure prototype location, and parameters of each repair channel are optimized collaboratively during the training process, thereby achieving adaptive modeling and repair of various phase failure mechanisms under complex reflective surface conditions.

[0093] Specific Implementation Method Six: This implementation method is a further limitation of Specific Implementation Methods One, Two, Three, Four, or Five. In this implementation method, the calculation process for outputting the repair phase through weighted fusion of each repair channel is as follows:

[0094] The loss is calculated by introducing a weight distribution constraint, and the corresponding formula is as follows:

[0095]

[0096] In the above formula, This is a smoothing loss term used to constrain the distribution of failed weights, preventing weight collapse in the early stages of network training. As an auxiliary constraint, it works together with the main loss of phase restoration in the network training process; considering both phase restoration accuracy and training stability, the formula for calculating the overall network loss function is:

[0097]

[0098] In the above formula, λ is a weighting coefficient used to balance the influence of different loss terms;

[0099] The overall network loss function takes the accuracy of the final repaired phase as the core optimization objective, and integrates failure classification, failure subspace modeling and phase repair process into the same optimization framework to complete the repair calculation process that takes convergence into account during the overall training process.

[0100] This invention constructs a multi-source physical feature model by inputting multi-source physical information from FPP and PS. The multi-source physical feature model is then processed by a failure feature encoder to form prototype vector models. Each prototype vector model is then calculated with a failure weight to obtain a corresponding repair channel. Finally, the repair phase is output by weighted fusion of each repair channel.

[0101] Specific Implementation Method Seven: This implementation method is a further limitation of Specific Methods One, Two, Three, Four, Five, or Six. In this implementation method, the failure classification module is used to model the phase failure state generated during the fringe projection profilometry measurement process in a continuous form. Its core function is to adaptively characterize the relative weights of different phase failure mechanisms in the feature space based on multi-source physical features, rather than to perform discrete category determination of phase failures. This module achieves joint perception of multiple potential failure mechanisms by constructing a failure feature embedding space and a failure prototype vector, providing weight guidance for the subsequent phase repair module.

[0102] In the failure typing module, the multi-source physical features jointly constructed by fringe projection profilometry and photometric stereo method are first input into the failure feature encoding network. A low-dimensional embedding representation for characterizing the phase failure state is obtained through nonlinear mapping. This process can be represented as follows:

[0103]

[0104] in, This represents the joint physical feature vector at pixel (x, y). Let z(x,y)∈Rd be the failure feature encoding function, and z(x,y)∈Rd be the corresponding failure feature embedding vector. The embedding vector is used to uniformly characterize the phase failure features caused by factors such as saturation, modulation degradation, and non-Lambertian reflection. In the failure feature embedding space, several failure prototype vectors are introduced as reference centers for failure classification to characterize the typical distribution positions of different phase failure mechanisms in the feature space. Let the failure prototype vector corresponding to the i-th type of failure mechanism be...

[0105]

[0106] Here, N represents the preset number of failure subspaces, and each failure prototype vector serves as a learnable parameter, updated along with the network parameters during training. These failure prototypes do not rely on manual annotation; instead, they form an implicit model of the failure mechanism through the overall error inverse constraint of the phase repair task.

[0107] To achieve continuous characterization of the impact of different failure mechanisms, the failure typing module constructs the relative weight of the current pixel in each failure subspace by calculating the distance relationship between the failure feature embedding vector and each failure prototype. This weight calculation process, where the calculated weight wi is the core of the entire method, is specifically calculated using the following formula:

[0108]

[0109] in Represents pixels The weighting coefficients for the i-th type of failure mechanism. Through this weight distribution, the failure classification module can reflect the possibility of multiple failure mechanisms existing simultaneously at the same pixel location, avoiding the discontinuities and instabilities introduced by traditional hard classification methods in the failure boundary region.

[0110] The output of the failure classification module is a set of continuous weighting coefficients. This output does not directly participate in the phase result calculation, but serves as weight guidance information for subsequent multi-channel phase repair modules, enabling the phase repair process to adaptively adjust the repair strategy according to the relative contribution of different failure mechanisms.

[0111] Specific Implementation Method Eight: This implementation method is a further limitation of Specific Methods One, Two, Three, Four, Five, Six, or Seven. In this implementation method, the failure subspace repair network is used to perform targeted structured repair of damaged phases under the weight guidance of the failure classification module. This network adopts a multi-branch parallel structure design, with each branch corresponding to a preset failure subspace to model the mapping relationship between the damaged phase and the true phase under the dominant failure mechanism. Each failure subspace repair network maintains a consistent overall architecture to ensure the uniformity and feasibility of the network structure, but its network parameters are independent during training, thereby learning the phase repair rules under different failure mechanisms.

[0112] In terms of network structure, each failure subspace repair network uses the damaged phase and the surface normal estimated by the photometric stereo method as the main inputs, and extracts repair features related to the failure mechanism through several layers of nonlinear mapping. Let the repair network corresponding to the i-th type of failure subspace be... Its input-output relationship can be expressed as

[0113]

[0114] in, Represents pixels Damaged phase at the location, This indicates the surface normal information at the corresponding location. This is the phase repair amount calculated under the assumption of the i-th type of failure subspace.

[0115] In the specific implementation, each failure subspace repair network adopts a deep network structure consisting of an input feature mapping layer, a feature transformation layer, and an output mapping layer connected sequentially. The input feature mapping layer is used to jointly encode the damaged phase and normal information, expressing them in a unified feature space; the feature transformation layer gradually extracts high-order features related to the phase error distribution through multiple layers of nonlinear operations; the output mapping layer maps the extracted repair features into phase compensation quantities, realizing quantitative repair of the damaged phase. The above network structure remains consistent across different failure subspaces, but their parameter weights are updated independently during training, thereby ensuring that each repair network can focus on feature learning of its corresponding failure mechanism.

[0116] During the phase repair phase, the repair networks in each failure subspace run in parallel, each outputting its corresponding phase repair value. The repair results are then weighted and fused using weights provided by the failure typing module. The final repaired phase is determined by the following relationship:

[0117]

[0118] in, Indicates the first Class failure subspace in pixels The weight coefficients at each point are given, and N is the number of failure subspaces. Through this multi-branch parallel and weighted fusion network structure design, the phase repair process can achieve smooth transition and adaptive adjustment under the combined effects of different failure mechanisms.

[0119] The failure subspace repair network, together with the failure classification module and the failure feature encoding module, constitutes a unified end-to-end trainable structure. Under the constraint of the phase repair objective function, the structural parameters of each repair subnetwork can be continuously optimized around its corresponding failure subspace, thereby improving the overall stability and accuracy of phase repair under complex reflective surface conditions.

Claims

1. A method for depth measurement using multi-channel non-Lambertian surface FPP-PS fusion guided by a failure classification module, characterized in that: After constructing a multi-source joint feature model from the acquired multi-source physical features, the multi-source joint feature model is processed by a failure feature encoder to form prototype vectors. The multi-source joint feature data of the actual object to be detected and each prototype vector are then fed into the failure weight calculation model to obtain the weights of the corresponding repair channels. Finally, the calculation results of the repair phase are output by weighted fusion of each repair channel.

2. The method for depth measurement of multi-channel non-Lambertian surfaces guided by a failure classification module according to claim 1, characterized in that: Multi-source physical features are multi-source physical information fused from FPP and PS. The process of acquiring multi-source physical features is as follows: First, raw measurement results are obtained using fringe projection profilometry and photometric stereometry. These raw results are then used to construct a multi-source physical feature vector to characterize the phase failure state. For each pixel position (x, y), a joint physical feature vector is constructed: In the above formula, This represents the multivariate physical feature vector at the corresponding pixel. The damaged phase value is obtained by fringe projection profilometry. The fringe tone is used to characterize fringe quality and contrast level; The local gradient characteristics of the damaged phase in space; The surface normal vector is estimated using the photometric stereo method; This provides the image brightness information for the corresponding pixel location.

3. The multi-channel non-Lambertian surface FPP-PS fusion measurement depth method guided by a failure classification module according to claim 1 or 2, characterized in that: After constructing the multi-source joint feature model, the process of processing the multi-source joint feature model into prototype vector models through the failure feature encoder is as follows: After obtaining the multi-source physical features, these features are input into the failure feature encoding module. The failure feature encoding module performs nonlinear mapping and feature compression on the original physical quantities in the multi-source physical features to obtain low-dimensional embedded features that can characterize the phase failure state. The failure feature encoding process is represented as follows: In the above formula, The failure feature embedding vector at pixel (x, y); For failure feature encoding function; It is a multi-source physical feature vector; Then, the typical distribution locations of different phase failure mechanisms are processed, and several failure prototype vectors are introduced. The calculation formula is as follows: In the above formula, This is the failure prototype vector corresponding to the i-th type of phase failure mechanism; d represents the dimension of the failure feature embedding space; N is the preset number of failure mechanisms; By calculating the distance relationship between the embedded features and each failure prototype, the network establishes the relative membership relationship between the current pixel and different failure mechanisms in the feature space, thereby completing the continuous failure perception modeling process.

4. The multi-channel non-Lambertian surface FPP-PS fusion measurement depth method guided by a failure classification module according to claim 3, characterized in that: The process of deriving corresponding repair channels from each prototype vector model through failure weight calculation is as follows: After obtaining the failure feature embedding vector and the corresponding failure prototype, the similarity between the embedded features and each failure prototype is calculated to establish the weight relationship of the current pixel under different failure mechanism assumptions, characterizing the relative degree of multiple failure mechanisms in the coexistence state at the same pixel position. Finally, for each preset failure mechanism, a corresponding phase repair channel is constructed to perform targeted correction processing on the damaged phase. The calculation process is as follows: Failure weights are obtained by normalizing the distances between the embedded features and each failure prototype, and the corresponding calculation formula is as follows: The weights obtained by the above formula Let (x, y) be the relative confidence level of pixel (x, y) under the i-th type of failure mechanism assumption; In the phase repair phase, the network constructs a corresponding phase repair channel for each type of failure mechanism. Each repair channel independently calculates the phase compensation amount under the same input conditions. The phase repair function corresponding to the i-th type of failure mechanism is expressed as: In the above formula, This is the phase repair network corresponding to the i-th type of failure subspace.

5. The method for depth measurement of multi-channel non-Lambertian surfaces guided by a failure classification module according to claim 1, characterized in that: The network obtains the final repaired phase by weighted fusion of the phase compensation amounts output from each repair channel. The calculation formula is as follows: The aforementioned network structure is trained end-to-end under a unified phase repair objective function constraint, ensuring that the failure feature encoding, failure prototype location, and parameters of each repair channel are optimized collaboratively during the training process, thereby achieving adaptive modeling and repair of various phase failure mechanisms under complex reflective surface conditions.

6. The method for depth measurement of multi-channel non-Lambertian surfaces guided by a failure classification module according to claim 5, characterized in that: The calculation process for outputting the repair phase from each repair channel using a weighted fusion method is as follows: The loss is calculated by introducing a weight distribution constraint, and the corresponding formula is as follows: In the above formula, This is a smoothness loss term used to constrain the distribution of failure weights, preventing weight collapse in the early stages of network training. As an auxiliary constraint, it works together with the main loss of phase restoration in the network training process; considering both phase restoration accuracy and training stability, the formula for calculating the overall network loss function is: In the above formula, λ is a weighting coefficient used to balance the influence of different loss terms; The overall network loss function takes the accuracy of the final repaired phase as the core optimization objective, and integrates failure classification, failure subspace modeling and phase repair process into the same optimization framework to complete the repair calculation process that takes convergence into account during the overall training process.

Images