Phase unwrapping method and system based on event camera correction and risk assessment

Abstract

The present application relates to the technical field of structured light three-dimensional topography measurement, and particularly relates to a phase unwrapping method and system based on event camera correction and risk assessment. The present unwrapping method relies on local circular domain redistribution correction to obtain high-quality wrapped phase, and then obtains pixel risk indicators through multi-dimensional residual risk assessment; comprehensive reliability is constructed by fusing modulation degree, two types of confidence, direction consistency and residual risk, and seed pixels are screened relying on triple thresholds of comprehensive reliability, residual risk and modulation degree. The neighborhood edge weight is set in combination with reliability and stripe prior, a global energy function containing seed constraints is established, and the absolute phase is converted after the phase order is optimized and solved. In the unwrapping method, multi-parameter constraints are realized relying on pre-correction and risk assessment, the disadvantages of single-index screening are broken through, distortion error diffusion is suppressed, complex working conditions such as reflection and high-speed motion are adapted, and the phase calculation and three-dimensional measurement precision of dynamic scenes are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of structured light three-dimensional topography measurement technology, specifically to a pixel-wrapped phase local circular domain redistribution correction method, a pixel residual risk assessment method, and a pixel phase unwrapping method and system. Background Technology

[0002] In the field of dynamic structured light 3D measurement, the event camera-based measurement scheme is widely used in high-speed dynamic scene measurement due to its advantages of responding only to changes in pixel brightness, small data volume, and high response frame rate, effectively overcoming the disadvantages of high cost and redundant data of traditional high-speed frame cameras.

[0003] Patent CN114659466B discloses a measurement scheme using an event camera and edge-shifting structured light. This scheme generates a wrapped phase by projecting periodic edge-shifting fringes and grouping event streams, and then obtains a 3D point cloud through spatial unwrapping. Additionally, there is a scheme that uses coaxial fusion acquisition of frame and event cameras. This involves cross-modal fusion of the acquired frame images, event time-series representation maps, and theoretical fringe prior maps to obtain an optimized fringe map. An N-step phase-shifting algorithm is then used to solve for the initial wrapped phase in the frame image coordinate system. Simultaneously, the fusion process can output frame confidence maps, event confidence maps, phase quality prior maps, and fusion modulation maps, and all image and phase data are aligned to the frame camera pixel coordinate system.

[0004] However, both traditional edge-shifting event structured light schemes and existing frame-event fusion phase solving schemes generally use global phase histogram equalization to correct the wrapped phase after obtaining the initial wrapped phase. This method applies a uniform phase distribution correction strategy to the entire image. However, when the object under test has high-speed motion, local reflections, sudden brightness changes, or optical gamma nonlinearity, the image will produce local non-uniform phase distortion. The globally uniform correction strategy cannot specifically correct the differential distortion in different pixel neighborhoods, which easily leads to problems such as residual local distortion or excessive smoothing of normal phase texture, resulting in reduced accuracy of subsequent phase unwrapping and phase jump distortion.

[0005] After phase correction, the existing edge-shifting structured light measurement system and frame event fusion measurement system based on event cameras rely solely on the traditional fringe modulation index to evaluate phase solution quality and on fringe brightness contrast to judge local phase reliability. They lack a multi-dimensional quantitative evaluation mechanism for local phase distribution stability, closed-loop residual anomalies, and fringe direction matching. Specifically, patent CN114659466B generates phase maps using fixed encoding rules, without designing phase quality assessment and risk identification mechanisms for dynamic scene distortion. This makes it impossible to accurately characterize the magnitude of local phase distortion in dynamic scenes, effectively identify phase singularities, error propagation boundaries, and potential solution risk areas, and hinders the construction of a refined phase distortion measurement and residual risk characterization system. Ultimately, this results in the inability to effectively distinguish between low-quality distortion areas and high-risk anomaly areas, failing to provide accurate risk prior constraints for subsequent phase solutions, and easily leading to problems such as unpredictable local phase anomalies and persistent latent distortions.

[0006] In the phase unwrapping stage, existing publicly available edge-shifting event structured light technologies generally employ fixed spatial phase unwrapping algorithms. These algorithms rely solely on a single modulation threshold to select seed pixels and use mechanical region growing, fixed-weight expansion, or conventional global graph optimization strategies to complete phase expansion. This results in a single dimension for seed pixel selection and a lack of multi-constraint guidance for phase propagation. Such methods fail to integrate reliable information from multiple dimensions, including frame confidence, event confidence, phase distortion degree, directional consistency, and residual risk. Consequently, they cannot accurately select highly reliable seed regions, nor can they adaptively adjust propagation edge weights and unwrapping priorities based on local phase quality.

[0007] In summary, existing event-frame fusion structured light measurement technology has inherent shortcomings in correction methods, quality assessment, and unpacking strategies, making it difficult to adapt to the high-precision phase solution requirements in complex dynamic scenarios. Summary of the Invention

[0008] To address the technical problems of traditional global phase equalization, such as its difficulty in specifically correcting local non-uniform distortion, the tendency to retain distortion, or the over-smoothing of effective phase, this invention provides a pixel-wrapped phase local circular domain redistribution correction method. To address the technical problem that relying solely on modulation index cannot comprehensively quantify phase distortion or accurately identify phase anomalies and error propagation regions, this invention provides a pixel residual risk assessment method. To address the technical problem that the single seed pixel selection criterion and lack of multi-dimensional constraints easily lead to large-scale phase error propagation, this invention provides a pixel phase unwrapping method and system.

[0009] To achieve the above objectives, the present invention provides the following technical solution: A pixel-wrapped phase local circular domain redistribution correction method includes the following correction steps: First, for the pixel (x,y) that needs to be corrected in the frame image coordinate system, based on its local neighborhood Ω x,y The frame confidence C corresponding to each pixel (u,v) within the frame f (u,v), event confidence C e (u,v), phase quality prior Q p (u,v), the fusion modulation scheme M0(u,v), and the weighted summation to obtain the corresponding local weight ω(u,v); Then, the initial wrap phase of each pixel (u,v) is used. The cosine component c(x,y) and sine component s(x,y) of pixel (x,y) are calculated as follows: ; ; Next, the local circular mean phase is calculated using the four-quadrant arctangent function atan2(·,·). : ; Next, the Ω is processed by the differentiable resampling operator wrap(·). x,y All of them Perform a rotation transformation to obtain the rotated phase. And in combination with ω(u,v), a weighted empirical distribution function F is constructed. x,y (θ): ; ; Where θ represents the phase independent variable, 1(·) is the indicator function, and ε is the zero constant; Finally, through F x,y (θ) Perform local homogenization mapping, and then perform inverse rotation transformation using wrap(·) to restore the original frame image coordinate system to obtain the corrected wrap phase. .

[0010] As a further improvement to the above scheme, the formula for calculating ω(u,v) is as follows: ; Wherein, λ1, λ2, λ3, and λ4 are the weight coefficients for frame confidence, event confidence, phase quality prior, and fusion modulation, respectively.

[0011] A pixel residual risk assessment method includes the following assessment steps: S1. Based on the cosine component, sine component, local weight, and corrected wrapping phase obtained from the pixel wrapping phase local circular domain redistribution correction method, in Ω x,yThe phase distortion of pixel (x,y) is calculated internally. and phase residual ρ(x,y); S2, using a weighted fusion strategy for coupling And ρ(x,y), and after mapping constraints, generate pixel residual risk values.

[0012] As a further improvement to the above scheme: phase distortion component It is expressed as follows: ; ; Where Γ(x,y) represents the pixel (x,y) in Ω x,y Local circle consistency index within; The phase residual ρ(x,y) is expressed as follows: ; Where, Δ x Δ y This represents the phase change along the X and Y axes of the frame image coordinate system; , The wrapped phase of pixels (x+1,y) and (x,y+1) is corrected by a local circular domain redistribution correction method for pixel wrapped phase.

[0013] As a further improvement to the above scheme, the pixel residual risk value is expressed as follows: ; Among them, R r (x,y) represents the pixel residual risk value of pixel (x,y); σ(·) is the Sigmoid function; C f (x,y), C e M0(x,y) and M0(x,y) represent the frame confidence, event confidence, and fusion modulation of pixel (x,y); a1~a5 are the corresponding positive weight coefficients.

[0014] A pixel phase unwrapping method, the steps of which are as follows: A1. The pixel residual risk value obtained by introducing the pixel residual risk assessment method is combined with the multi-source imaging quality parameters to calculate the pixel comprehensive reliability corresponding to each pixel; A2. Based on the pixel overall reliability, pixel residual risk value, and fusion modulation, a screening threshold is set to obtain the unwrapping seed pixel set. A3. Construct pixel neighborhood connection weights based on pixel comprehensive reliability, perform phase unwrapping using constraint optimization, and output absolute phase.

[0015] As a further improvement to the above scheme, the pixel-level overall reliability is expressed as follows: ; ; Where Q(x,y) is the pixel-wide reliability of pixel (x,y); C d (x,y) is the stripe direction consistency coefficient of pixel (x,y); for The gradient of ; t(x,y) is the unit vector of the fringe theory tangent direction at pixel (x,y); η is the scale adjustment constant; exp(·) is the exponential function with the natural constant as the base; b1~b5 are the corresponding positive weight coefficients.

[0016] As a further improvement to the above scheme, the seed pixel set S is represented as follows: ; in, , , These are the reliability threshold, risk threshold, and regulation threshold, respectively.

[0017] As a further improvement to the above scheme, the process of obtaining the absolute phase is as follows: For any pair of adjacent pixels i and j, define the weight w of the connecting edge between them. ij as follows: ; Among them, Q i Q j The pixel reliability of pixels i and j are respectively; κ is the preset scaling factor. , These are the corrected wrapper phases for pixels i and j, respectively. The expected phase difference between pixels i and j is obtained based on the prior knowledge of the fringe direction; Set an integer variable k i k j Given the phase integer correction orders for pixels i and j respectively, the optimal phase integer correction order distribution {k} is obtained by minimizing the following global energy function. i}: ; Where min represents the minimization operation; M is the set of all adjacent pixel edges; The baseline integer order is given in advance for the seed pixel; λ is the seed constraint weight. {k i Substituting the optimal phase integer correction order for each pixel in the formula into the following equation, the absolute phase for each pixel is calculated: ; in, Let be the absolute phase of pixel i.

[0018] A phase unwrapping processing system for multi-source imaging parameter constraints includes a phase correction module, a residual risk calculation module, and a constraint solution module; The phase correction module is used to perform the pixel-wrapped phase local circular domain redistribution correction method, which receives the initial wrapped phase and multi-source imaging parameters, and outputs the corrected wrapped phase. The residual risk calculation module is electrically connected to the phase correction module and is used to solve the residual risk value of each pixel according to the pixel residual risk assessment method, combined with the corrected wrapping phase and multi-source imaging parameters. The constraint solution module is connected to the residual risk calculation module and the phase correction module respectively. It is used to calculate the pixel comprehensive reliability and screen the unwrapping seed based on the pixel phase unwrapping method using pixel residual risk and multi-source imaging parameters, and output the absolute phase through weighted constraint optimization calculation.

[0019] Compared with the prior art, the beneficial effects of the present invention are: 1. In the pixel-wrapped phase local circular domain redistribution correction method, the traditional global equalization approach of uniform correction of the entire image is abandoned. Instead, a local neighborhood of a single pixel is used as an independent processing unit. Adaptive local weights are constructed by fusing multiple dimensions of indicators, including frame confidence, event confidence, phase quality prior, and fusion modulation. Based on these weights, the weighted sine and cosine components of the neighborhood are solved to obtain the local circular mean phase. First, a circular rotation transformation is performed on the phase within the neighborhood. Then, the phase uniformity mapping within the neighborhood is achieved by relying on the weighted empirical distribution function. Finally, the corrected phase is obtained through inverse rotation, realizing regional differential phase equalization. The correction can be completed adaptively according to the degree of distortion at each location, effectively solving the drawbacks of local distortion residue or excessive smoothing of normal phase texture caused by global uniform correction.

[0020] 2. In the pixel residual risk assessment method, this invention is no longer limited to quality evaluation based on a single fusion modulation. It relies on the weighted components obtained from the aforementioned correction to solve the local circular consistency index and convert the phase distortion component to quantify the degree of local phase discrete distortion. The closed-loop phase residual is obtained through the closed-loop differential operation of the four-neighborhood of the phase after correction to characterize the phase singularity and error propagation risk. The phase residual and phase distortion component are then weighted and fused with the fusion modulation, event confidence, and frame confidence, and the residual risk value is obtained through function mapping. The quantitative assessment is achieved from multiple dimensions such as phase distribution characteristics, closed-loop residual anomalies, and multi-source imaging reliability, comprehensively quantifying phase distortion and accurately identifying phase anomalies and error propagation areas.

[0021] 3. In the pixel phase unwrapping method, this invention no longer relies solely on a single modulation degree to select seed pixels. Instead, it integrates the fusion modulation degree, event confidence, frame confidence, fringe direction consistency coefficient, and residual risk value to calculate a multi-dimensional comprehensive reliability. Seed pixels are selected using a triple threshold of comprehensive reliability, residual risk, and fusion modulation degree. At the same time, the connection edge weights between adjacent pixels are adaptively defined by combining comprehensive reliability and fringe direction prior. The phase integer order is optimized by minimizing the global energy through the introduction of seed benchmark order constraints. By relying on multiple index constraints on seed selection and phase propagation path, the large-scale spread of errors in low-quality areas is effectively avoided. Attached Figure Description

[0022] Figure 1 This is a flowchart of the unpacking method.

[0023] Figure 2 This is a schematic diagram illustrating the working principle of the measurement system.

[0024] Figure 3 This is a schematic diagram illustrating the process of synchronously acquiring event streams and frame images, and capturing event time windows.

[0025] Figure 4 This is a schematic diagram of the cross-modal phase enhancement principle. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] like Figure 1 As shown, this invention first constructs a near-coaxial acquisition system of an event camera and a frame camera using a beam splitter. A projector projects phase-shifted fringes to synchronously acquire frame images and asynchronous events. Matching events are captured at the frame exposure time and event representations are generated. Frame images, event representations, and fringe priors are fused to obtain optimized fringe data and various confidence maps. Based on the optimized fringes, the initial wrapping phase is calculated. Combined with multi-source imaging parameters, phase distortion caused by motion and illumination is eliminated through local circular domain redistribution. Then, the pixel reliability and risk coefficient are constructed by integrating phase distortion and confidence information. High-quality seed pixels are selected, and phase unwrapping is completed pixel by pixel under confidence constraints. The missing periodic terms of each pixel's phase are filled in, and a continuous and complete absolute phase of the object surface is output.

[0028] I. System Composition and Working Principle

[0029] like Figure 2As shown, the measurement system in this embodiment includes a projector, a beam splitter, an event camera, and a frame camera. The projector projects an N-step phase-shifted sinusoidal fringe and Gray code pattern onto the target surface; the light reflected from the target surface passes through the imaging lens and enters the beam splitter, which splits the same beam into two paths: one path enters the frame camera to form a grayscale image, and the other path enters the event camera to form an event stream. Since the two light paths originate from the same incident channel, the two sensors have a near-coaxial observation relationship of the target's field of view.

[0030] The frame camera uses hardware-triggered exposure, while the event camera records the corresponding moment of the trigger edge to establish a unified time reference. First, the time synchronization and spatial alignment of the frame images and the event stream are completed, and then the fused fringe response, initial wrap phase, and absolute phase map are calculated.

[0031] II. Event Stream Time Window Extraction and Construction of Original Event Confidence Representation

[0032] like Figure 3 As shown, this step performs temporal registration and feature representation preprocessing on the original asynchronous events output by the event camera. The preprocessing generates the original event features, and the event confidence and frame confidence are then calculated based on the obtained original event features.

[0033] Let e ​​be the kth event output by the event camera. k =(x k ,y k ,t k ,p k ), where x k y k Let t be the pixel coordinates corresponding to the k-th event. k Let p be the timestamp of the k-th event, and p be the polarity of the k-th event. k ∈{-1,+1}.

[0034] For the nth phase-shifted frame, take its exposure center time as Define a subset of events that match the temporal domain of that frame: ΔT is the event integration time window, and δt is the time offset correction between the frame camera and the event camera. This subset of events allows for a one-to-one temporal correspondence between the asynchronous event stream and the discrete frame image.

[0035] This section provides two parallel options for generating the original event features: Option 1: Original characteristics of voxel raster format events The filtered event subset E n Converting to voxel raster format event original features: Divide the time window ΔT evenly into B consecutive time sub-intervals, each time sub-interval corresponding to an independent voxel channel; first, process event e... k timestamp tk Normalization is performed within the corresponding time window to obtain the normalized time index. Then, the single-channel features are solved through weighted summation. The formula for calculating the channel features corresponding to the b-th voxel channel is as follows: ; Among them, V b (x,y) is the channel voxel feature value corresponding to the b-th voxel channel at pixel (x,y), which is the original feature of the event under this scheme.

[0036] Option 2: Time-domain attenuated filter for original event features

[0037] The first step, to optimize the event feature representation corresponding to the phase-shifting fringes, is to construct an event-driven temporal decay surface: ; Wherein, R(x,y,t) is the temporal attenuation response surface of pixel (x,y) at sampling time t, which is the intermediate characterization data of this scheme. This is a preset time decay constant used to control the decay rate of the event response over time. 1 (t≥t k The characteristic function is defined only if the sampling time t is not earlier than the event occurrence time t. k The value is 1 if the event occurs, and 0 otherwise, ensuring that the event decays only in the forward time domain and has no reverse effect. exp(·) is an exponential function with the natural constant as the time since the event occurred; the response amplitude decays exponentially as time goes on.

[0038] The second step is to apply frequency domain filtering to the time-domain decay surface along the time dimension to extract the original event features at the matched phase shift frequency: ; Among them, S n (x,y) represents the pixel event fringe feature corresponding to the phase shift frequency at step n, which is the original time-domain attenuated filter event feature output by this scheme. F t (·) represents the positive Fourier transform along the time dimension, transforming the time-domain decay surface R to the frequency domain. With a center frequency of ω n The frequency domain filter, ω n Let be the theoretical angular frequency of the nth group of projected phase-shifted fringes, retaining only the frequency components corresponding to the target fringes.

[0039] The two schemes generate two different types of event raw features, and either type of event raw feature can be used in subsequent calculations to provide basic parameters for the subsequent phase correction steps.

[0040] III. Cross-modal phase enhancement processing

[0041] like Figure 4As shown, this step takes the original event features, original frame images, and various stripe prior information obtained in the previous step and performs cross-modal fusion processing. After enhancement operations, a unified output is given: a fused stripe intensity map. , Stripe pattern D n Event confidence level C e Frame confidence C f and phase quality prior Q p These five types of parameters provide all the basic input data for the subsequent calculation of local weights. The processing includes two parallel optional implementation methods: deep learning network fusion and non-learning weighted fusion.

[0042] Let I denote the original frame image of the nth frame. n The original event features used for matching are V. n or S n P n The fringe prior data for the corresponding frame can be selected from any one or more combinations of fringe orientation gradient, Gabor response, bandpass filter response, centerline probability map, historical phase estimation, and projection template prior.

[0043] Implementation Method 1: Cross-modal Feature Fusion Based on Deep Neural Networks

[0044] Since the imaging physical mechanisms of frame images and original event features differ, frame images are good at characterizing static brightness and global stripe texture, while original event features focus on recording pixel brightness abrupt changes and dynamic changes in stripe edges. Therefore, a multi-branch structure is used to complete feature extraction separately.

[0045] In the Lth layer of the deep neural network, a frame feature encoder is used. Event Feature Encoder Extract the corresponding two types of modal features respectively , .

[0046] Simultaneously, a priori branch is set, relying on the striped prior data P. n Output the corresponding prior features .

[0047] Due to equipment assembly deviations, lens distortion, and optical path errors, even with near-coaxial beam splitting imaging, the pixel coordinates of the frame image and the event image still exhibit local positional discrepancies. Therefore, an offset field Δ is introduced. L The pixel offset is represented by the event features, and the original event features are spatially aligned using the differentiable resampling operator warp(·) to obtain the coordinate-matched event features. .

[0048] After completing the spatial alignment of features across modalities, an attention-weighted fusion mechanism is used to achieve adaptive fusion of multiple features. First, frame feature gating weights are generated through convolution operations and the Sigmoid function. σ(·) is the Sigmoid function, which constrains the weights to the interval [0,1]; Conv[·] represents the convolution operation.

[0049] α L A higher value indicates that the current region prioritizes frame image information; conversely, a lower value indicates that event features are prioritized. Static, clear regions are weighted towards 1, while regions with sudden motion changes or overexposure are weighted towards 0.

[0050] Based on frame feature gating weight α L Prior guiding weight β L Event characteristics By using ⊙, representing element-wise weighted operations, the three-source features are weighted and fused to obtain the multi-scale fused feature F. L : ; Multi-scale fusion feature F L The data is fed into a phase-enhanced decoder, and the fused fringe intensity map is finally obtained. , Stripe pattern D n Event confidence level C e Frame confidence C f and phase quality prior Q p .

[0051] For parameter optimization of deep neural networks, a multi-constraint joint loss function is used to complete model training: ; Where: L int Constrained fused fringe intensity accuracy, representing the intensity loss term, based on the fused fringe intensity map. The pixel grayscale difference between the reference standard fringe image and the network output is used to construct the loss, which is used to constrain the fused fringe intensity to be consistent with the true reference intensity, thereby improving the accuracy of fringe amplitude calculation.

[0052] L phase Constrained phase error, which is the phase loss term, is calculated based on the wrapped phase obtained from the fused fringes and the reference true phase. Construct loss, preferably using This eliminates the periodic ambiguity caused by the 2π phase jump and constrains the phase solution error.

[0053] L dir The effect of constrained fringe orientation prediction is represented by the orientation loss term, with the network output fringe pattern D. nThe deviation from the theoretical fringe direction given by the fringe prior is calculated, and the fringe direction prediction result of constrained pixel position fits the real fringe direction.

[0054] L qual The matching relationship between the constrained mass prior and the actual phase error is represented by the mass loss term, and the constrained phase mass prior is Q. p The magnitude of Q is monotonically negatively correlated with the actual phase error at that position; the larger the phase error, the greater the corresponding Q value. p The lower the value, the more accurately the quality prior can characterize the reliability of the local phase.

[0055] L cons Constraining the consistency of frame and event feature distributions is a feature consistency loss term. It ensures that the feature distribution trends of features extracted from frame branches and events branches remain consistent in the same source scene, thereby reducing the deviation in cross-modal feature space distribution.

[0056] L reg This is a regularization term for network parameters, which applies regularization constraints to all weight parameters of the network to suppress overfitting of model parameters and improve the generalization performance of the network.

[0057] d1~d6 are the manually preset balancing weight coefficients for each type of loss, used to adjust the proportion of each loss in the total loss.

[0058] Implementation Method 2: Explicit Weighted Fusion Without Network

[0059] This solution requires no deep neural network training; it relies on manually set weights to complete the fusion calculation, making it easy to implement in engineering. A stripe change response map G is generated from the original event features. n The original frame and stripe change response map G are analyzed using three weighted coefficients. n Weighted superposition of fringe priors generates a fused fringe intensity map. : ; in, is the fused stripe intensity map at pixel (x,y) in the nth frame, which is the final output of this formula. It replaces the original single-source frame image and is used for subsequent modulation and confidence calculations. f (x,y) represents the original frame image weight at pixel (x,y), and is taken independently for each pixel. n (x,y) represents the original grayscale value at pixel (x,y) of the nth original frame image. e (x,y) represents the event feature weight at pixel (x,y), which is also taken independently for each pixel. G n (x,y) represents the pixel value at pixel (x,y) in the stripe change response map of the nth frame, calculated based on the original features of the event, and characterizes the dynamic stripe change information. p(x,y) represents the prior weight of the stripe at pixel (x,y), which is also taken independently for each pixel. P n (x,y) represents the value of the stripe prior data corresponding to the nth frame at pixel (x,y).

[0060] All weights satisfy the normalization constraint: ; All weight coefficients in the formula can be determined through calibration tests, grid search, or engineering experience: the weight w corresponding to the lifting event under high-speed motion conditions. e (x,y), In static high signal-to-noise ratio scenes, increase frame weight w f (x,y), increasing the prior weight w for projection brightness distortion scenarios. p (x,y).

[0061] Based on the obtained fused fringe intensity map and fringe prior information, the fringe orientation pattern D can be directly calculated using existing mature image processing algorithms. n And simultaneously solve for the event confidence C. e Frame confidence C f Phase quality prior Q p .

[0062] The fused stripe intensity map output in this step , Stripe pattern D n Event confidence level C e Frame confidence C f and phase quality prior Q p It is used for modulation solving in subsequent stages and local circular domain redistribution correction of pixel-wrapped phase.

[0063] IV. Solving the Initial Wrap Phase and Fusion Modulation

[0064] This step, based on the multi-frame fused fringe intensity map output from the cross-modal phase enhancement processing described above, uses an N-step phase shift algorithm to solve for the basic phase and modulation parameters, obtaining the initial wrapped phase and fused modulation, providing basic raw data for subsequent phase correction, residual risk assessment, and phase unwrapping operations in this invention.

[0065] The standard N-step phase-shift imaging model used in this embodiment is as follows: ; Where A(x,y) is the background light intensity at pixel (x,y), and B(x,y) is the basic stripe tone at pixel (x,y). Let n be the true phase to be solved at pixel (x,y), n be the phase-shifted image frame number, and N be the total number of phase shift steps.

[0066] Based on the fused stripe intensity map of N consecutive frames The initial wrapping phase corresponding to pixel (x,y) is obtained by solving the N-step phase shift method. The calculation formula is: ; Where atan2(·,·) is the arctangent function in the four quadrants.

[0067] The core principle of this initial wrapping phase calculation is first-order Fourier phase estimation. The fused fringe intensity map of each frame is projected onto the sine and cosine component dimensions, respectively. The phase value of the periodic wrapping is obtained by solving the arctangent function, ultimately yielding the global initial wrapping phase. It should be noted that the sign of the phase calculation can be adjusted equivalently according to the preset rules of the device's projection phase shift direction without affecting the overall accuracy of the phase calculation.

[0068] The fusion modulation index M0(x,y) at pixel (x,y) is calculated synchronously based on the fused stripe intensity map of N frames. The calculation formula is as follows: ; The fusion modulation index, calculated from the amplitude of the first harmonic of the fringes, is a fundamental indicator characterizing the local fringe imaging quality and the reliability of phase solution. A higher fusion modulation index value at a pixel location indicates significant fringe brightness variations and excellent fringe imaging quality, corresponding to a higher reliability of the initial wrapping phase solution. Conversely, a low fusion modulation index value suggests problems such as underexposure, overexposure, image blurring, shadow occlusion, low object reflectivity, or sparse original event features in the area, resulting in poor phase solution accuracy and low reliability.

[0069] While relying solely on fused modulation index can provide a preliminary assessment of the reliability of the phase foundation, in complex dynamic scenarios such as high-speed motion, local distortion, and fringe blurring, a single modulation index cannot comprehensively and accurately characterize the distortion defects and unwrapping error risks of the wrapped phase. Therefore, this invention, based on existing fundamental phase shift solving techniques, further introduces innovative steps such as subsequent local circular domain redistribution phase correction, multi-dimensional residual risk modeling, and constrained phase unwrapping to overcome the shortcomings of traditional phase shift algorithms in terms of insufficient phase solving accuracy in dynamic and complex scenarios.

[0070] The initial wrap phase is the final output of this step. The fusion modulation scheme M0(x,y) will be combined with the event confidence C obtained in the previous step. e Frame confidence C f and phase quality prior Q p Together, they serve as all the input parameters for the next step: pixel-wrapped phase local circular domain redistribution correction.

[0071] V. Correction of Local Circular Domain Redistribution of Wrapped Phase

[0072] This step uses the initial wrapping phase, fusion modulation, frame confidence, event confidence, and phase quality prior obtained in the preceding steps as all input data, and obtains the corrected wrapping phase through local circular domain redistribution correction processing.

[0073] Traditional phase equalization schemes for suppressing gamma nonlinear errors mostly employ global polar coordinate equalization to process the wrapped phase. Such schemes can only achieve effective correction in static imaging and single sensor acquisition applications. When the measured area is accompanied by multiple interferences such as local image blurring, stripe occlusion, sparse original event features, and pixel brightness jumps, the global equalization method ignores the spatial non-uniform distribution characteristics of phase distortion, resulting in poor correction. Therefore, this invention adopts a local circular domain redistribution correction mechanism to complete the optimized correction of the wrapped phase.

[0074] For any pixel (x, y) to be corrected, define the local neighborhood Ω corresponding to that pixel. x,y Then, based on the multi-source reliability index, a local neighborhood Ω of pixel (x,y) is constructed. x,y Local weights within: ; Where λ1, λ2, λ3, and λ4 are the weight coefficients for frame confidence, event confidence, phase quality prior, and fusion modulation, respectively. ω(u,v) represents the local neighborhood Ω. x,y Local weights corresponding to the inner pixel (u,v). C f (u,v) is the local neighborhood Ω x,y Frame confidence level corresponding to the inner pixel (u,v). C e (u,v) is the local neighborhood Ω x,y The event confidence level corresponding to the inner pixel (u,v). Q p (u,v) is the local neighborhood Ω x,y The phase quality prior for the inner pixel (u,v). M0(u,v) is the local neighborhood Ω. x,y The fusion modulation level corresponding to the inner pixel (u,v).

[0075] Based on the aforementioned local weights, in the local neighborhood Ω x,y The initial wrap phase of each pixel (u,v) is obtained. The cosine component c(x,y) and sine component s(x,y) of pixel (x,y) are calculated as follows: ; ; Using the obtained sine and cosine components, the local circular mean phase is calculated through the four-quadrant arctangent function atan2(·,·). : ; Based on the local circular mean phase, the Ω is adjusted using the differentiable resampling operator wrap(·). x,y All initial wrap phases within By performing a circular rotation transformation, the neighborhood phase is uniformly mapped to the local circular coordinate system, resulting in the rotated phase. : ; Based on the phase after rotation transformation and local weights, a locally weighted empirical distribution function is constructed: ; Where θ represents the phase independent variable, 1(·) is the indicator function, and ε is the zero constant; Finally, through F x,y (θ) for the rotated phase Perform local homogenization mapping: ; Then, an inverse rotation transformation is performed using wrap(·) to restore the original frame image coordinate system, thus obtaining the corrected wrap phase: .

[0076] Compared to conventional phase equalization schemes that directly perform phase equalization in the real number domain, this scheme adapts to the inherent 0 and 2π periodic transition characteristics of the phase by relying on the operation form of circular mapping, which can properly handle the equalization distortion caused by phase boundary transitions. Compared to global phase equalization schemes, this scheme achieves partitioned correction based on local pixel neighborhoods, which fully preserves the original phase topology of each region and can adapt to the correction scenario of non-uniform phase distortion on the surface of dynamically measured objects.

[0077] The corrected wrap-around phase output in this step This serves as the core input parameter for subsequent pixel residual risk assessment.

[0078] VI. Phase Distortion Measurement and Pixel Residual Risk Construction

[0079] This step involves obtaining the corrected wrapper phase from the local circular region redistribution correction in the previous step. The local neighborhood includes the cosine component c(x,y), the sine component s(x,y), the local neighborhood weight ω(u,v), and the previously solved fusion modulation index M0 and event confidence C. e Frame confidence C f Using the input data, the calculations for the local circular consistency index, phase distortion component, and four-neighbor closed-loop phase residual are performed sequentially, ultimately generating the residual risk map R. r (x,y) is used to define a reliable propagation path for subsequent phase unwrapping.

[0080] First, a local circular consistency index Γ(x,y) is constructed based on the neighborhood weighted components (pixel (x,y) in Ω). x,y The local circle consistency index (within the pixel area) is used to quantify the concentration and stability of phase distribution within a local area of ​​a pixel. The calculation formula is as follows: ; Here, ε is a minimal constant used to avoid calculation failures caused by the summation term in the denominator being zero.

[0081] The range of Γ(x,y) is fixed at [0,1]. When the local phase arrangement is concentrated and uniform, the value of Γ(x,y) approaches 1; when affected by noise interference or phase multi-peak discrete distortion, the value of Γ(x,y) decreases accordingly.

[0082] The phase distortion component is further solved based on the local circular consistency index. The calculation formula is as follows: ; The higher the value, the greater the degree of phase distortion in the corresponding pixel area.

[0083] Secondly, a four-neighborhood closed-loop difference operation is performed on the corrected wrapped phase to solve for the closed-loop phase residual ρ(x,y), which is used to identify phase singularities and phase error propagation boundaries. ; Where, Δ x Δ y This represents the phase change along the X and Y axes of the frame image coordinate system; , The wrapped phase is the corrected phase for pixels (x+1,y) and (x,y+1).

[0084] If the calculated result of ρ(x,y) approaches 2π and its integer multiples, it indicates that there is a phase singularity at the pixel position, which has the potential for phase calculation errors to propagate outward.

[0085] By integrating multiple quantitative indicators such as closed-loop phase residual, phase distortion analysis, fusion modulation, event confidence, and frame confidence, a weighted pixel residual risk R is constructed. r (x,y) (Pixel residual risk value for pixel (x,y): ; Among them, R r (x,y) represents the pixel residual risk value of pixel (x,y); σ(·) is the Sigmoid function; a1~a5 are the corresponding positive weight coefficients.

[0086] The pixel residual risk R output in this step r(x,y) along with the corrected wrap phase , which serves as the input parameter for the next stage of the constraint-based phase unwrapping step.

[0087] VII. Fringe Direction Consistency and Overall Reliability Modeling

[0088] This step uses the corrected package phase obtained earlier. , Stripe pattern D n , fusion regulation M0(x,y), event confidence C e (x,y), frame confidence C f (x,y) and pixel residual risk R r Using (x,y) as input parameters, the direction consistency index is solved and the comprehensive reliability model is built in sequence. The resulting comprehensive reliability is used for seed pixel selection, phase propagation priority allocation and edge weight assignment in subsequent phase unwrapping.

[0089] From the inherent variation law of the fringes, the fringe phase changes rapidly along the fringe normal direction and gradually along the fringe tangent direction. Based on the obtained fringe pattern D... n Or via corrected wrapping phase gradient The unit vector t(x,y) of the fringe theoretical tangent direction corresponding to the pixel position is obtained by solving the problem, and the orientation consistency index C of the pixel position is constructed accordingly. d (x, y), the corresponding calculation formula is: ; Where η is the scaling constant and ε is the minimum constant, used to avoid the denominator being zero.

[0090] C d The larger the value of (x,y), the higher the matching degree between the phase gradient change pattern and the fringe normal change feature at the current position, and the better the reliability of the local phase solution.

[0091] Based on the directional consistency index, the pixel comprehensive reliability Q(x,y) is constructed by integrating the fusion modulation index, event confidence, frame confidence, directional consistency index, and residual risk index, as shown in the following expression: ; Where b1~b5 are the corresponding positive weight coefficients.

[0092] Q(x,y) integrates multi-dimensional reliability information and serves as a unified benchmark for seed pixel selection, phase propagation priority determination, and edge weight assignment for unwrapping graph optimization during the unwrapping operation.

[0093] The overall reliability Q(x,y) output in this step is related to the corrected package phase. Pixel residual risk R r(x,y) together serve as the input data for subsequent constraint phase unwrapping.

[0094] 8. Confidence-guided seed selection and unwrapping optimization

[0095] This step, based on the corrected wrapping phase, overall reliability, pixel residual risk, and fusion modulation obtained in the previous steps, constructs a multi-constraint adaptive phase unwrapping optimization framework. Traditional phase unwrapping methods often employ fixed line-by-line scanning, conventional quality-guided region growing, minimum cost flow, or minimum spanning tree strategies, resulting in poor adaptability and a tendency for phase error propagation. This implementation is not limited to a single unwrapping solver; it constructs a general optimization architecture and achieves confidence-guided accurate phase unwrapping computation through high-reliability seed pixel selection and adaptive neighborhood edge weight configuration, effectively suppressing the propagation of phase solution errors in dynamic scenes.

[0096] First, high-confidence seed pixel regions are selected based on multi-dimensional reliability thresholds to construct an unpacking seed pixel set, which provides accurate baseline constraints for subsequent unpacking operations. In this embodiment, pixels that meet the triple threshold conditions of comprehensive reliability, pixel residual risk, and fusion modulation are considered valid seed pixels. The unpacking seed pixel set S is defined as follows: ; in, , , These are the reliability threshold, risk threshold, and modulation threshold, respectively. Seed pixels can be generated using a globally optimal selection method, generating a single core seed region from the region with the highest overall global reliability; alternatively, multiple locally highly reliable seed clusters can be generated based on the connected component criterion, adapting to test scenarios with complex topography and multiple independent stripes in multiple regions.

[0097] After seed pixel selection, an adaptive neighborhood propagation connection edge weight w is constructed for any two adjacent pixels i and j in the image. ij Combining pixel reliability with the prior constraint of stripe direction to solve the reasonableness of the propagation path, the edge weights are connected. ; where Q i Q j The pixel reliability of pixels i and j are respectively; κ is the preset scaling factor. , These are the corrected wrapper phases for pixels i and j, respectively. The expected phase difference between pixels i and j is obtained a priori based on the fringe direction.

[0098] When the overall reliability of adjacent dual pixels is high and the matching degree between the actual phase difference and the prior expected phase difference in the fringe direction is high, the corresponding connecting edge weight is larger, indicating that the neighborhood edge is a highly reliable unwrapping propagation path and is given priority for phase extension transmission.

[0099] Based on the selected seed pixels and the constructed connection edge weights, this step provides two parallel optional unwrapping optimization methods: graph optimization and priority propagation. Implementation Method 1: Graph-based Decompression Define the integer phase integer correction order k i , used to characterize the phase integer correction of pixel i. To simultaneously satisfy local phase continuity, fringe direction consistency, and seed pixel reference constraints, the optimal phase order distribution {k} is solved by minimizing the global energy function. i The global energy function expression is: ; Where min represents the minimization operation; M is the set of all adjacent pixel edges; The seed pixel is given a reference integer order in advance; λ is the seed constraint weight. It is used to adjust the constraint strength of the seed reference.

[0100] This global energy function can simultaneously constrain the smooth continuity of the neighborhood phase, the prior matching of the fringe direction, and the effectiveness of the highly reliable seed pixel reference, thus ensuring the accuracy of the global absolute phase solution.

[0101] {k i Substituting the optimal phase integer correction order for each pixel in the formula into the following equation, the absolute phase for each pixel is calculated: ; in, Let be the absolute phase of pixel i.

[0102] Implementation Method 2: Priority Propagation Unpacking

[0103] This implementation constructs a priority queue based on pixel-level overall reliability, and gradually expands the unwrapped area outward in order of overall reliability from high to low to achieve pixel-by-pixel phase calculation.

[0104] For any candidate pixel j to be unwrapped, relying on the set of neighboring pixels U that have already been unwrapped, traverse the optimal integer order to find the optimal phase order of the candidate pixel: ; Where: N j Let m be the set of neighboring pixels of candidate pixel j, and m be the integer order of traversal. Simultaneously, a fault tolerance constraint is applied based on pixel residual risk; if the pixel residual risk value R of candidate pixel j... r (j) If the threshold is exceeded, the phase propagation of the pixel is delayed, or it is incorporated into subsequent local map optimization for fine correction, so as to avoid the early spread of phase errors in low-reliability areas from the root.

[0105] This step uses confidence-guided seed selection and adaptive optimization unwrapping to ultimately output a high-precision, error-free global absolute phase result, thus completing the accurate solution for the overall dynamic scene fringe phase.

[0106] IX. Industrial Application Prospects

[0107] This invention addresses the challenge of structured light phase determination under complex conditions such as dynamic interference, local distortion, and reflection from special materials. It proposes an adaptive phase correction and unwrapping scheme based on event-frame cross-modal fusion, possessing strong scene adaptability and engineering feasibility, and can be widely applied to various high-precision dynamic vision measurement scenarios. Specifically, it includes online inspection of high-speed conveyor belts, precision measurement of workpiece contours under mechanical vibration conditions, target pose perception before robot grasping operations, precision imaging of medical surfaces, fringe phase recovery of transparent and semi-reflective special targets, and various dynamic structured light vision measurement systems equipped with event cameras.

[0108] Meanwhile, the technical solution of this invention possesses dual-mode deployability. It can achieve high-precision cross-modal feature fusion and phase enhancement through deep learning networks, adapting to precision measurement scenarios of high-end computing power equipment; alternatively, it can achieve lightweight deployment using a non-learning architecture of traditional graph optimization and weighted fusion, adapting to low-computing-power, embedded industrial equipment. Due to its flexible deployment form, this invention can adapt to different hardware platforms and complex application conditions, exhibiting strong versatility and portability, and possessing broad prospects for industrial application and promotional value.

[0109] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A pixel-wrapped phase local circular domain redistribution correction method, characterized in that, The following calibration steps are included: First, for the pixel (x,y) that needs to be corrected in the frame image coordinate system, based on its local neighborhood Ω x,y The frame confidence C corresponding to each pixel (u,v) within the frame f (u,v), event confidence C e (u,v), phase quality prior Q p (u,v), the fusion modulation scheme M0(u,v), and the weighted summation to obtain the corresponding local weight ω(u,v); Then, the initial wrap phase of each pixel (u,v) is used. The cosine component c(x,y) and sine component s(x,y) of pixel (x,y) are calculated as follows: ； ； Next, the local circular mean phase is calculated using the four-quadrant arctangent function atan2(·,·). : ； Next, the Ω is processed by the differentiable resampling operator wrap(·). x,y All of them Perform a rotation transformation to obtain the rotated phase. And in conjunction with ω(u,v), a weighted empirical distribution function F is constructed. x,y (θ): ； ； Where θ represents the phase independent variable, 1(·) is the indicator function, and ε is the zero constant; Finally, through F x,y (θ) Perform local homogenization mapping, and then perform inverse rotation transformation using wrap(·) to restore the original frame image coordinate system to obtain the corrected wrap phase. .

2. The pixel-wrapped phase local circular domain redistribution correction method according to claim 1, characterized in that, The formula for calculating ω(u,v) is as follows: ； Wherein, λ1, λ2, λ3, and λ4 are the weight coefficients for frame confidence, event confidence, phase quality prior, and fusion modulation, respectively.

3. A method for assessing pixel residual risk, characterized in that, The following evaluation steps are included: S1. The cosine component, sine component, local weight, and corrected wrapping phase obtained in the pixel wrapping phase local circular domain redistribution correction method according to claim 1 or 2, in Ω x,y The phase distortion of pixel (x,y) is calculated internally. and phase residual ρ(x,y); S2, using a weighted fusion strategy for coupling And ρ(x,y), and after mapping constraints, generate pixel residual risk values.

4. The pixel residual risk assessment method according to claim 3, characterized in that, Phase distortion It is expressed as follows: ； ； Where Γ(x,y) represents the pixel (x,y) in Ω x,y Local circle consistency index within; The phase residual ρ(x,y) is expressed as follows: ； Where, Δ x Δ y This represents the phase change along the X and Y axes of the frame image coordinate system; , The wrapped phase of pixels (x+1,y) and (x,y+1) is corrected by the local circular domain redistribution correction method for pixel wrapped phase as described in claim 1 or 2.

5. The pixel residual risk assessment method according to claim 3, characterized in that, The pixel residual risk value is expressed as follows: ； Among them, R r (x,y) represents the pixel residual risk value of pixel (x,y); σ(·) is the Sigmoid function; C f (x,y), C e M0(x,y) and M0(x,y) represent the frame confidence, event confidence, and fusion modulation of pixel (x,y); a1~a5 are the corresponding positive weight coefficients.

6. A pixel phase unwrapping method, characterized in that, The steps are as follows: A1. The pixel residual risk value obtained by the pixel residual risk assessment method described in claim 5 is introduced, and the pixel comprehensive reliability corresponding to each pixel is calculated in conjunction with the multi-source imaging quality parameters. A2. Based on the pixel overall reliability, pixel residual risk value, and fusion modulation, a screening threshold is set to obtain the unwrapping seed pixel set. A3. Construct pixel neighborhood connection weights based on pixel comprehensive reliability, perform phase unwrapping using constraint optimization, and output absolute phase.

7. A pixel phase unwrapping method according to claim 6, characterized in that, The overall reliability of pixels is expressed as follows: ； ； Where Q(x,y) is the pixel-wide reliability of pixel (x,y); C d (x,y) is the stripe direction consistency coefficient of pixel (x,y); for The gradient of ; t(x,y) is the unit vector of the fringe theory tangent direction at pixel (x,y); η is the scale adjustment constant; exp(·) is the exponential function with the natural constant as the base; b1~b5 are the corresponding positive weight coefficients.

8. A pixel phase unwrapping method according to claim 7, characterized in that, The seed pixel set S is represented as follows: ； in, , , These are the reliability threshold, risk threshold, and regulation threshold, respectively.

9. A pixel phase unwrapping method according to claim 8, characterized in that, The process of obtaining the absolute phase is as follows: For any pair of adjacent pixels i and j, define the weight w of the connecting edge between them. ij as follows: ； Among them, Q i Q j The pixel reliability of pixels i and j are respectively; κ is the preset scaling factor. , These are the corrected wrapper phases for pixels i and j, respectively. The expected phase difference between pixels i and j is obtained based on the prior knowledge of the fringe direction; Set an integer variable k i k j Given the phase integer correction orders for pixels i and j respectively, the optimal phase integer correction order distribution {k} is obtained by minimizing the following global energy function. i }: ； Where min represents the minimization operation; M is the set of all adjacent pixel edges; The baseline integer order is given in advance for the seed pixel; λ is the seed constraint weight. {k i Substituting the optimal phase integer correction order for each pixel in the formula into the following equation, the absolute phase for each pixel is calculated: ； in, Let be the absolute phase of pixel i.

10. A phase unwrapping processing system for multi-source imaging parameter constraints, characterized in that, It includes a phase correction module, a residual risk calculation module, and a constraint solution module; The phase correction module is used to execute the pixel-wrapped phase local circular domain redistribution correction method as described in claim 1 or 2, receive the initial wrapped phase and multi-source imaging parameters, and output the corrected wrapped phase; The residual risk calculation module is electrically connected to the phase correction module and is used to solve the residual risk value of each pixel in accordance with the pixel residual risk assessment method according to any one of claims 3 to 5, combined with the corrected wrapping phase and multi-source imaging parameters. The constraint solution module is connected to the residual risk calculation module and the phase correction module respectively. It is used to calculate the pixel comprehensive reliability and screen the unwrapping seed by using the pixel residual risk and multi-source imaging parameters based on the pixel phase unwrapping method of any one of claims 6 to 9, and output the absolute phase through weighted constraint optimization calculation.

Images