Phase unwrapping method based on space-time diagram cut algorithm
The space-time graph is constructed and energy optimization is performed through the space-time graph cutting algorithm, which solves the problem of phase expansion error in complex and dynamic scenarios, and achieves higher phase expansion accuracy, which is suitable for dynamic scenarios.
Patent Information
- Application Number
- CN202510308502.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-17
- Publication Date
- 2025-07-04
AI Technical Summary
When the prior art deals with complex and dynamically changing scenarios, the phase expansion method is prone to phase resolution errors, and it is difficult to ensure the accuracy of phase expansion.
The space-time graph cutting algorithm is used to obtain the wrapping phase through structured light stripe projection, build the space-time graph, and define the energy function. The space-time graph cutting algorithm is used to minimize energy optimization, and finally phase de-wrap to ensure the continuity of phase expansion.
The accuracy of phase expansion is improved, and it can adapt to complex and dynamically changing scenes, reduce error accumulation, especially when the surface of the object is deformed or moved in a dynamic scene, and the time continuity information of multi-frame images is used for more accurate phase expansion.
Smart Images
Figure CN120259326A_ABST
Abstract
Description
Technical Field
[0001] The field of the present invention is the field of image processing technology, and specifically relates to a phase unwrapping method based on a spatio-temporal graph cut algorithm. Background Art
[0002] Phase maps are usually affected by various noises, such as environmental noise, instrument noise, etc., which can cause errors in phase data. In addition, phase jumps (i.e., sudden discontinuous changes in phase) are also a common problem, especially when the phase value approaches or exceeds the boundaries of π and -π. The methods in the prior art use a graph cut algorithm and utilize only the spatial dimension to participate in constructing the graph and unwrapping. The core idea is to construct a graph where nodes represent phase values, edges connect adjacent phases, and find the optimal cut of the graph by minimizing an energy function, and then perform unwrapping according to the cut result. However, in dealing with complex and dynamically changing scenarios in the prior art, due to the excessive complexity of the scenario, phase unwrapping errors are likely to occur. Summary of the Invention
[0003] The object of the present invention is to provide a phase unwrapping method based on a spatio-temporal graph cut algorithm. The present invention can adapt to complex and dynamically changing scenarios and improve the accuracy of phase unwrapping.
[0004] The technical solution of the present invention: A phase unwrapping method based on a spatio-temporal graph cut algorithm, comprising the following steps:
[0005] Step 1: Through the analysis of the fringe pattern based on structured light fringe projection, according to the obtained analysis results, extract the wrapped phase;
[0006] Step 2: Based on the wrapped phase in Step 1, construct a spatio-temporal graph; according to the constructed spatio-temporal graph, define an energy function, and use the spatio-temporal graph cut algorithm to minimize the energy to obtain the optimal cut label;
[0007] Step 3: According to the result obtained by the graph cut, perform phase unwrapping, construct a continuous unwrapped phase map, and complete phase unwrapping.
[0008] In the foregoing phase unwrapping method based on a spatio-temporal graph cut algorithm, in Step 1, the steps for extracting the wrapped phase include:
[0009] Step 2-1: Determine the surface area of the object to be scanned, and calibrate the internal and external parameters of the camera and the projector;
[0010] Step 2-2: Generate a structured light fringe pattern through the projector, and project the generated structured light fringe pattern onto the surface of the object to be scanned;
[0011] Step 2-3: Capture the structured light stripe pattern projected on the object surface through a camera to obtain a captured image, and ensure that the captured image is aligned with the projected structured light stripe pattern;
[0012] Step 2-4: Perform fringe analysis on the captured image to calculate the wrapped phase.
[0013] In the foregoing phase unwrapping method based on the spatio-temporal graph cut algorithm, in Step 2-4, the expression of the fringe analysis is:
[0014]
[0015] In the formula, a is the average background light intensity, b is the modulation value, is the wrapped phase, (x, y) are the image coordinates, n is the number of phase shift steps, and N is the total number of phase shift steps;
[0016] The calculation formula of the average background light intensity a is:
[0017]
[0018] The wrapped phase The calculation formula of is:
[0019]
[0020] In the foregoing phase unwrapping method based on the spatio-temporal graph cut algorithm, in Step 2, the specific process of constructing the spatio-temporal graph is that for each spatial position (x, y), take its wrapped phase changing with time As the longitudinal data of the spatio-temporal graph, the horizontal direction is the time axis t, representing the phase change on different time frames, and the vertical direction is the spatial position (x, y), representing the phase change with time.
[0021] Each pixel in the constructed spatio-temporal graph corresponds to a node in each frame, and the edge connections are divided into spatial edges and temporal edges. The spatial edges represent the edges connecting adjacent pixels in the same frame, reflecting the spatial continuity; while the temporal edges represent the edges connecting pixels at the same position or adjacent positions in consecutive frames, reflecting the temporal continuity.
[0022] In the foregoing phase unwrapping method based on the spatio-temporal graph cut algorithm, the specific formula of the energy function is:
[0023] E(k) = ∑ (p，q)∈Q V(k p , k q ) + Σ p∈P D(k p );
[0024] Among them, k is the cut label of all pixel points, p and q are the spatial positions (x, y) of the pixel points, Q is the neighborhood connection of the pixel points, and k p is the cut label at pixel p, and k q is the cut label at pixel q, and V(k p , k q ) is the smooth term, which is used to ensure the consistency of phase unwrapping of adjacent pixels; D(k p ) is the data term, which is used to ensure that the unwrapped phase is as close as possible to the measured phase.
[0025] In the foregoing phase unwrapping method based on the spatio-temporal graph cut algorithm, the calculation formula of the data term D(k p ) is as follows:
[0026]
[0027] Among them, is the wrapped phase, is the initially estimated unwrapped phase, and k p is such that the unwrapped phase is closest to the true phase;
[0028] The calculation formula of the smooth term V(k p , k q ) is as follows:
[0029] V(k p , k q ) = W pq *|k p - k q |;
[0030] Among them, W pq is the weight between neighborhoods. If k p and k q are the same, no cost is added. If they are different, a higher cost will be incurred to prevent unnecessary jumps.
[0031] In the foregoing phase unwrapping method based on the spatio-temporal graph cut algorithm, the specific process of minimizing the energy using the graph cut algorithm is to find a cut to divide the nodes in the spatio-temporal graph into two subsets, so that the cost of the cut is minimized.
[0032] In the foregoing phase unwrapping method based on the spatio-temporal graph cut algorithm, in step three, the process of phase unwrapping is to label each pixel as foreground or background, ensure the overall phase continuity, start from the source point, traverse all nodes, and accumulate the necessary 2π adjustments.
[0033] In the foregoing phase unwrapping method based on the spatio-temporal graph cut algorithm, the calculation formula of the phase unwrapping is as follows:
[0034]
[0035] Among them, φ(x, y) is the expanded phase, which is the wrapped phase; k(x, y) is the grating order.
[0036] Compared with the prior art, the present invention has the following beneficial effects:
[0037] The present invention obtains depth information through structured light stripe projection and calculates the wrapped phase. Based on the physical optics principle, it can relatively accurately obtain the phase information basis of the object surface. And in the subsequent spatio-temporal map construction, energy function optimization, and phase unwrapping processes, various factors such as the spatial and temporal continuity of pixels are considered, which can effectively avoid error accumulation in the phase unwrapping process, thereby improving the accuracy of phase unwrapping. In addition, since the spatio-temporal map constructed by the present invention contains time edges, connecting pixels at the same position or adjacent positions in consecutive frames, this phase unwrapping method can handle the phase unwrapping problem in dynamic scenes. In a dynamic scene, the object surface may deform or move. This method can utilize the temporal continuity information between multiple frames of images to perform phase unwrapping more accurately, and has better adaptability compared to methods that only consider single-frame images. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 It is a flowchart of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0039] The present invention will be further described below in conjunction with the drawings and embodiments, but it is not used as a basis for limiting the present invention.
[0040] Embodiment
[0041] A phase unwrapping method based on a spatio-temporal graph cut algorithm, as Figure 1 shown, specifically includes the following steps:
[0042] Step 1: Through the analysis of the fringe pattern based on structured light fringe projection, according to the obtained results, extract the wrapped phase;
[0043] In this embodiment, first, the designed fringe pattern is burned into the projector. After adjusting the distances between the camera, the projector, and the object, the projector is regarded as an inverse camera using the triangular stereo model. By matching the phase corresponding points of the camera and the projector, the triangular ranging method is used for reconstruction.
[0044] Structured light projection is a common 3D scanning technology, usually used to obtain the 3D shape of the object surface. The steps for extracting the wrapped phase include:
[0045] Step 2-1: Determine the object surface area to be scanned, and calibrate the internal and external parameters of the camera and the projector.
[0046] Specifically, install the camera and the projector, and ensure that they can be accurately aligned with the scanning area.
[0047] Step 2-2: Generate a structured light fringe pattern through the projector, and project the generated structured light fringe pattern onto the surface of the object to be scanned;
[0048] Specifically, an encoder or a computer is usually used to generate the structured light fringe pattern, which can be a sine wave, a Gray code or other patterns, and the projector projects these fringe patterns onto the surface of the object to be scanned.
[0049] Step 2-3: Capture the structured light fringe pattern projected on the surface of the object through the camera to obtain a captured image, and ensure that the captured image is aligned with the projected structured light fringe pattern;
[0050] Specifically, the camera captures an image of the surface of the object containing the structured light fringes, and ensures that the parameters of the camera and the projector are correctly calibrated.
[0051] Step 2-4: Perform fringe analysis on the captured image to calculate the wrapped phase.
[0052] Specifically, the expression of the fringe analysis is:
[0053]
[0054] In the formula, a is the average background light intensity, b is the modulation value, is the wrapped phase, (x, y) are the image coordinates, n is the number of steps of phase shift (n = 0, 2,... N-1), and N is the total number of phase shift steps.
[0055] The calculation formula for the average background light intensity a is:
[0056]
[0057] Furthermore, according to the result of the fringe analysis, the calculation of the wrapped phase is
[0058] The formula is:
[0059]
[0060] Step two: Based on the wrapped phase in step one, construct a spatio-temporal graph; according to the constructed spatio-temporal graph, define an energy function, and use the spatio-temporal graph cut algorithm to minimize the energy;
[0061] In this example, the specific process of constructing the spatio-temporal graph is that for each spatial position (x, y), take its wrapped phase changing with time As the longitudinal data of the spatio-temporal graph, the horizontal direction is the time axis t, representing the phase change on different time frames, and the vertical direction is the spatial position (x, y), representing the change of phase over time.
[0062] Each pixel in the constructed spatio-temporal graph corresponds to a node in each frame. The edge connections are divided into spatial edges and temporal edges. Spatial edges represent the edges connecting adjacent pixels in the same frame, reflecting spatial continuity; while temporal edges represent the edges connecting pixels at the same position or adjacent positions in consecutive frames, reflecting temporal continuity. The weight of the temporal edge is determined based on the gradient magnitude of the phase change between adjacent frames.
[0063] The energy function is the core component for optimizing the problem, defining the objective to be minimized. The specific formula of the energy function is:
[0064] E(k) = ∑ (p，q)∈Q V(k p , k q ) + ∑ p∈P D(k p );
[0065] where k is the cut label of all pixel points, p and q are the spatial positions (x, y) of the pixel points, Q is the neighborhood connection of the pixel points, k p is the cut label at pixel p, k q is the cut label at pixel q, V(k p , k q ) is the smoothing term, used to ensure the consistency of phase unwrapping for adjacent pixels; D(k p ) is the data term, used to ensure that the unwrapped phase is as close as possible to the measured phase;
[0066]
[0067] where is the wrapped phase, is the initially estimated unwrapped phase, k p is such that the unwrapped phase is closest to the true phase;
[0068] V(k p , k q ) = W pq *|k p - k q |;
[0069] where W pq is the weight between neighborhoods. If k p and k q are the same, no cost is added. If they are different, a higher cost is incurred to prevent unnecessary jumps.
[0070] The data term usually evaluates the probability that a pixel belongs to the foreground or background based on the difference between the phase value of the pixel and the expected phase value. The data term usually refers to the cost associated with a single pixel or node, which is directly calculated based on the image values. In phase unwrapping, the data term can be constructed based on the difference between the actually observed phase value and the theoretical or expected phase value. Its purpose is to evaluate the probability or cost that each pixel is classified into different states (such as adding or subtracting 2π) according to the current phase value. The smooth term takes into account the phase continuity between spatially and temporally adjacent pixels and penalizes the drastic change in the phase values of adjacent pixels. The smooth term involves the relationship between the nodes in the graph and is mainly used to penalize the discontinuity of the phase between adjacent nodes. This term encourages the phase values of adjacent pixels in the graph to be similar, thus ensuring that the overall phase map is continuous both visually and physically. Specifically, if the phase difference between adjacent pixels is large, the cost of the smooth term will increase, and vice versa.
[0071] Furthermore, in the graph cut algorithm, each pixel node is assigned a binary label (usually 0 or 1), indicating which of the two subsets of the graph the pixel belongs to (for example, foreground or background). The assignment of the label determines the way of dividing the graph (i.e., the "cut"). The specific process of minimizing the energy using the graph cut algorithm is to find a cut that divides the nodes in the spatio-temporal graph into two subsets, such that the cost of the cut is minimized, and obtain the optimal cut label. The spatio-temporal graph cut algorithm can effectively utilize the combination of time information and space information, thereby effectively processing the phase data and obtaining the continuous phase of the target object.
[0072] The present invention not only utilizes the spatial information of the target object scene but also the time information. The time edge connects the pixel points at the same spatial position but between consecutive time frames; and for the definition of the time edge weight, it is defined based on the difference between the expected and actually observed phase changes, and the energy function of the entire graph is defined by combining the weights of the spatial and time edges, so as to be able to well handle complex and dynamically changing scenes.
[0073] Step 3: According to the result obtained by the graph cut, perform phase unwrapping to construct a continuous unwrapped phase map and complete the phase unwrapping.
[0074] In this embodiment, the process of phase unwrapping is to label each pixel as foreground or background, ensure the overall phase continuity, start from the source point, traverse all nodes, and accumulate the necessary 2π adjustments.
[0075] The obtained phase is wrapped and there are 2π phase jumps. In order to obtain the continuous absolute phase, it is necessary to perform phase unwrapping. The calculation formula for phase unwrapping is:
[0076]
[0077] Among them, φ(x, y) is the expanded phase, which is the wrapped phase; k(x, y) is the grating order.
[0078] The present invention obtains depth information by structured light stripe projection and calculates the wrapped phase. Based on the physical optics principle, the phase information of the object surface can be obtained more accurately. And in the subsequent spatio-temporal map construction, energy function optimization, and phase unwrapping process, multiple factors such as the spatial and temporal continuity of pixels are considered, which can effectively avoid error accumulation in the phase unwrapping process, thereby improving the accuracy of phase unwrapping. In addition, since the spatio-temporal map constructed by the present invention contains time edges, connecting pixels at the same or adjacent positions in consecutive frames, this phase unwrapping method can handle the phase unwrapping problem in dynamic scenes. In dynamic scenes, the object surface may deform or move. This method can utilize the temporal continuity information between multiple frames of images to perform phase unwrapping more accurately, and has better adaptability compared to the method that only considers a single frame of image.
[0079] In summary, the present invention can adapt to complex and dynamically changing scenes and improve the accuracy of phase unwrapping.
Claims
1. A phase unwrapping method based on a spatio-temporal graph cut algorithm, characterized in that, It includes the following steps: Step 1: Through the fringe pattern analysis based on structured light fringe projection, according to the obtained analysis results, extract the wrapped phase; Step 2: Based on the wrapped phase in Step 1, construct a spatio-temporal graph; according to the constructed spatio-temporal graph, define an energy function, and use the spatio-temporal graph cut algorithm to minimize the energy to obtain the optimal cut label; Step 3: According to the results obtained by graph cut, perform phase unwrapping, construct a continuous unwrapped phase map, and complete phase unfolding.
2. The phase unwrapping method based on the spatio-temporal graph cut algorithm according to claim 1, wherein: In Step 1, the extraction steps of the wrapped phase include: Step 2-1: Determine the surface area of the object to be scanned, and calibrate the internal and external parameters of the camera and the projector; Step 2-2: Generate a structured light fringe pattern through the projector, and project the generated structured light fringe pattern onto the surface of the object to be scanned; Step 2-3: Capture the structured light fringe pattern projected on the object surface through the camera to obtain a captured image, and ensure that the captured image is aligned with the projected structured light fringe pattern; Step 2-4: Perform fringe analysis on the captured image to calculate the wrapped phase.
3. The phase unwrapping method based on the spatio-temporal graph cut algorithm according to claim 2, characterized in that: In Step 2-4, the expression of the fringe analysis is: where a is the average background light intensity, b is the modulation value, is the wrapped phase, (x, y) are the image coordinates, n is the number of phase shift steps, and N is the total number of phase shift steps; The calculation formula of the average background light intensity a is: The wrapped phase is calculated by the formula:
4. The phase unwrapping method based on the spatio-temporal graph cut algorithm according to claim 1, characterized in that: In step 2, the specific process of constructing the spatio-temporal graph is that for each spatial position (x, y), the wrapped phase that changes with time is taken as the longitudinal data of the spatio-temporal graph. The horizontal direction is the time axis t, representing the phase change on different time frames, and the vertical direction is the spatial position (x, y), representing the change of the phase with time; Each pixel in the constructed spatio-temporal graph corresponds to a node in each frame. The edge connections are divided into spatial edges and temporal edges. The spatial edge represents the edge connecting adjacent pixels in the same frame, reflecting spatial continuity; while the temporal edge represents the edge connecting pixels at the same position or adjacent positions in consecutive frames, reflecting temporal continuity.
5. The phase unwrapping method based on the spatio-temporal graph cut algorithm according to claim 1, wherein: The specific formula of the energy function is: E(k) = ∑ (p，q)∈Q V(k p , k q ) + ∑ p∈PD (k p ); where k is the cut label of all pixels, p and q are the spatial positions (x, y) of the pixels, Q is the neighborhood connection of the pixels, k p is the cut label at pixel p, k q is the cut label at pixel q, V(k p , k q ) is the smooth term, which is used to ensure the consistency of phase unwrapping of adjacent pixels; D(k p ) is the data term, which is used to ensure that the unwrapped phase is as close as possible to the measured phase.
6. The phase unwrapping method based on the spatio-temporal graph cut algorithm according to claim 5, characterized in that: The data item D(k p ) is calculated as follows: Among them, is the wrapped phase, is the initially estimated unwrapped phase, and k p is such that the unwrapped phase is closest to the true phase; The smoothing term V(k p , k q ) is calculated as follows: V(k p , k q ) = V pq * |k p - k q |; Among them, W pq is the weight between neighborhoods. If k p and k q are the same, the cost is not increased. If they are different, a higher cost is incurred to prevent unnecessary jumps.
7. The phase unwrapping method based on the spatio-temporal graph cut algorithm according to claim 1, characterized in that: The specific process of using the graph cut algorithm to minimize the energy is to find a cut that divides the nodes in the spatio-temporal graph into two subsets, so that the cost of the cut is minimized.
8. The phase unwrapping method based on the spatio-temporal graph cut algorithm according to claim 1, characterized in that: In Step 3, the process of phase unwrapping is to label each pixel as foreground or background, ensure the overall phase continuity, start from the source point, traverse all nodes, and accumulate the necessary 2π adjustments.
9. The phase unwrapping method based on the spatio-temporal graph cut algorithm according to claim 8, wherein: The calculation formula of the phase unwrapping is as follows: Among them, φ(x, y) is the expanded phase, which is the wrapped phase; k(x, y) is the grating order.