A structured light three-dimensional reconstruction method and system based on gradient prior and edge attention

Abstract

This invention relates to a method and system for 3D reconstruction of structured light based on gradient prior and edge attention. The method includes a training phase and an inference phase. In the training phase, the gradient of the initial prediction loss of the phase retrieval network with respect to the input structured light stripe image is calculated to generate a loss gradient map representing the difficulty of regression. Using this map, on the one hand, a predicted edge probability map is obtained through an edge prediction network to construct an edge loss; on the other hand, it is fused with the intermediate feature map of the phase retrieval network to generate a spatial attention weight map. After weighted enhancement of the features, an optimized predicted phase map is obtained, and a second loss is calculated. Finally, a joint loss function is constructed by combining the first loss, the second loss, and the edge loss to update the network parameters. In the inference phase, the trained network is used to recover the phase from a single image and complete the 3D reconstruction. This invention can adaptively focus on difficult-to-regress regions, significantly improving the quality of reconstructed edges.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and optical 3D measurement technology, and in particular to a structured light 3D reconstruction method and system based on gradient prior and edge attention. Background Technology

[0002] Structured light 3D measurement is an important non-contact optical measurement technique. It projects a specific pattern of light stripes onto the object under test, and a camera captures the deformed structured light stripe image modulated by the object's surface topography. Then, through steps such as phase solving and phase-to-3D coordinate mapping, it achieves high-precision reconstruction of the object's 3D shape. Due to its high precision and efficiency, this technology is widely used in industrial inspection, medical imaging, and robotic vision perception.

[0003] Traditional structured light 3D measurement methods, such as phase-shifting profilometry and Fourier transform profilometry, typically require projecting and acquiring multiple structured light fringe images to calculate a high-precision phase distribution. This limits their application in dynamic scenes or situations with extremely high measurement speed requirements. In recent years, deep learning techniques have been introduced to replace part or all of the phase retrieval process, aiming to reduce the number of pattern frames required for projection and thus improve measurement efficiency. These deep learning methods typically employ end-to-end regression networks to directly predict wrapper phase, absolute phase, or other phase-related intermediate representations from a single or a few structured light fringe images.

[0004] However, existing deep learning-based structured light phase retrieval methods generally suffer from problems such as overly smooth reconstruction results and edge distortion in areas with abrupt depth changes and complex local textures, particularly around object edges. The root cause lies in the fact that commonly used encoder-decoder network structures tend to lose subtle edge details during downsampling. Furthermore, when supervision relies solely on global pixel-level loss functions (such as L1 or L2 loss), the network optimization process tends to prioritize fitting smooth regions that occupy the majority of the image area, neglecting edge regions that contribute significantly to the error but have a smaller number of pixels. This results in larger phase regression errors in these critical areas, ultimately affecting the reconstruction quality of the 3D point cloud and causing blurred boundary contours and geometric distortion.

[0005] To improve edge quality, existing technologies have proposed several edge enhancement strategies, but most rely on explicit edge masks (such as those using Sobel operators) pre-computed and extracted from the original structured light fringe image or real phase map to guide the network. These pre-constructed edge masks are easily affected by image noise, object surface reflection characteristics, complex textures, and algorithm parameter settings, making it difficult to stably and accurately characterize the critical regions that are truly difficult to regress and most sensitive to overall error under the current network training state. Therefore, existing technologies lack prior information that can adapt to the network's own training state and directly reflect its difficult-to-regress regions. Furthermore, existing edge supervision mechanisms are often not tightly coupled with the optimization process of the main phase regression task, making it difficult to achieve a continuous, stable, and effective enhancement effect on edge regions.

[0006] In summary, how to establish a technical solution that can adaptively identify and focus on difficult regions (especially edge regions) during training without significantly increasing system complexity, thereby simultaneously improving the global accuracy of phase recovery and the quality of edge reconstruction, remains a technical problem that urgently needs to be solved in this field.

[0007] It should be noted that the information disclosed in the background section above is only for understanding the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0008] The technical problem to be solved by this application is "how to provide a structured light 3D reconstruction scheme that can adaptively focus on the hard-to-regress regions in the training of the phase retrieval network, thereby effectively overcoming the reconstruction distortion of object edges and detail regions, and significantly improving the quality of boundary reconstruction while ensuring global accuracy".

[0009] The technical solution adopted in this application to solve the above-mentioned technical problems is as follows.

[0010] This application proposes a structured light 3D reconstruction method based on gradient prior and edge attention, including a training phase and an inference phase; The training phase includes: Obtain the training dataset, which includes structured light stripe images and their corresponding ground phase maps; The structured light stripe image is input into a phase retrieval network to obtain a predicted phase map; the first loss between the predicted phase map and the corresponding true phase map is calculated; the gradient of the first loss with respect to the structured light stripe image is calculated to generate a loss gradient map, which characterizes the influence of different pixel regions in the structured light stripe image on the difficulty of phase regression. The loss gradient map is input into an edge prediction network to obtain a predicted edge probability map, which represents the possibility of the existence of discontinuous regions in the true phase map. The loss gradient map is fused with an intermediate feature map of the phase retrieval network to generate a spatial attention weight map. The intermediate feature map is then weighted and enhanced using the spatial attention weight map, and the enhanced intermediate feature map is fed back into the phase retrieval network for subsequent calculations to obtain an optimized predicted phase map. Calculate the second loss between the optimized predicted phase map and the true phase map; construct a joint loss function based on the first loss, the second loss, and the edge loss constructed by the predicted edge probability map, and update the phase recovery network based on the joint loss function; The inference phase includes: inputting the structured light fringe image of the object under test into the trained phase retrieval network to obtain its phase information; and performing three-dimensional reconstruction based on the phase information.

[0011] In some embodiments, the edge loss is obtained by calculating the difference between a predicted edge probability map and a ground truth edge label map.

[0012] In some embodiments, the ground truth edge map is obtained by edge detection of the ground truth phase map.

[0013] In some embodiments, the fusion method for fusing the loss gradient map with an intermediate feature map of the phase recovery network is to concatenate or weightedly add them along the channel dimension.

[0014] In some embodiments, generating a spatial attention weight map includes: inputting a fused feature map obtained by fusing the loss gradient map and the intermediate feature map into a convolutional neural network for processing to generate a spatial attention weight map.

[0015] In some embodiments, weighting the intermediate feature map using the spatial attention weight map includes: multiplying the spatial attention weight map and the intermediate feature map element by element.

[0016] In some embodiments, the joint loss function is a weighted sum of the first loss, the second loss, and the edge loss.

[0017] In some embodiments, the phase retrieval network is a neural network with an encoder-decoder structure; the edge prediction network is a network with an encoder-decoder structure.

[0018] In some embodiments, a structured light 3D reconstruction system based on gradient prior and edge attention is also provided, comprising: The training module is used to perform operations during the training phase; The inference module is used to perform operations during the inference phase; The training module includes: a data acquisition unit for acquiring a training dataset, which includes structured light fringe images and their corresponding ground truth phase maps; an initial prediction unit for inputting the structured light fringe images into a phase retrieval network to obtain a predicted phase map; a first loss calculation unit for calculating the first loss between the predicted phase map and the corresponding ground truth phase map; a gradient map generation unit for calculating the gradient of the first loss with respect to the structured light fringe images to generate a loss gradient map, which characterizes the influence of different pixel regions in the structured light fringe images on the difficulty of phase regression; and an edge prediction unit for inputting the loss gradient map into an edge prediction network to obtain a predicted edge probability map, which characterizes the difference between the predicted and ground truth phase maps. The probability of discontinuous regions existing in the real phase map; the attention weight generation unit, which fuses the loss gradient map with an intermediate feature map of the phase retrieval network to generate a spatial attention weight map; the feature enhancement and optimization unit, which uses the spatial attention weight map to weight and enhance the intermediate feature map, and feeds the enhanced intermediate feature map back to the phase retrieval network for subsequent calculations to obtain an optimized predicted phase map; the second loss and joint update unit, which calculates the second loss between the optimized predicted phase map and the real phase map, and constructs a joint loss function based on the first loss, the second loss, and the edge loss constructed by the predicted edge probability map, and updates the phase retrieval network based on the joint loss function; The inference module includes: a phase retrieval unit, which inputs the structured light fringe image of the object under test into the trained phase retrieval network to obtain its phase information; and a 3D reconstruction unit, which performs 3D reconstruction based on the phase information.

[0019] In some embodiments, a computer-readable storage medium is also provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the structured light three-dimensional reconstruction method of the present invention.

[0020] The present invention has the following beneficial effects: This invention utilizes the technical feature of "calculating the gradient of the first loss on the structured light stripe image to generate a loss gradient map" to quantify the contribution of each pixel region in the structured light stripe image to the phase prediction error. This can automatically and accurately locate pixel regions that are difficult to predict accurately in phase regression tasks (these regions usually correspond to the edges of objects, texture details, or phase transitions), thus providing accurate gradient prior information for subsequent targeted optimization. Secondly, through the technical feature of "inputting the loss gradient map into an edge prediction network to obtain a predicted edge probability map", the loss gradient map, which represents the regression difficulty, is used as the input of a dedicated edge prediction network to learn and predict the distribution probability of discontinuities (i.e., edges) in the real phase image. This approach explicitly transforms implicit regression difficulty information into a predicted edge probability map of the phase edge structure, providing a direct target for explicitly strengthening constraints on edge regions in the loss function. By fusing the loss gradient map with an intermediate feature map of the phase retrieval network to generate a spatial attention weight map, and then using this spatial attention weight map to weight and enhance the intermediate feature map, the loss gradient map is introduced into the phase retrieval network. This is fused with the intermediate feature map, and the network's attention to different spatial locations is dynamically adjusted through a spatial attention mechanism. At the internal feature level of the phase retrieval network, this adaptively enhances the feature representation of difficult-to-regress regions, guiding the phase retrieval network to... More computational resources are allocated to these key regions, thus laying the foundation for improving the accuracy of these regions during the feature extraction stage. By constructing a joint loss function based on the first loss, the second loss, and the edge loss constructed by the predicted edge probability map, a multi-task learning framework is built. This framework simultaneously optimizes the global accuracy of phase retrieval (through the first and second losses) and the prediction accuracy of edge regions (through the edge loss). From the perspective of optimization objectives, the phase retrieval network and the edge prediction network are forced to pursue the overall phase accuracy while also taking into account the fidelity of the edge structure during training, thus achieving synergistic optimization of global accuracy and local detail reconstruction quality.

[0021] In summary, this invention organically links the training processes of the "phase retrieval network" and the "edge prediction network" through the core bridge of the "loss gradient map," and utilizes the "spatial attention weight map" to achieve feature enhancement within the "phase retrieval network." Finally, it performs unified optimization through a "joint loss function." This close collaboration of technical features enables adaptive focusing on hard-to-regress regions and effectively improves the quality of boundary reconstruction.

[0022] Other beneficial effects of the present invention will be further described below. Attached Figure Description

[0023] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a framework for structured light reconstruction methods based on gradient priors and edge attention. Figure 2 This is a component diagram of a structured light reconstruction method based on gradient priors and edge attention. Figure 3 Here is a structural diagram of the structured light measurement system; Figure 4 The experimental results are shown in the figure, which includes two types of samples: gears and plaster dolls.

[0024] Figure 5 Comparison of phase error and 3D point cloud error for different phase recovery networks and module combinations on metal datasets and plaster datasets. Detailed Implementation

[0025] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention.

[0026] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0027] Existing technologies address the problem of insufficient edge precision in the following main ways.

[0028] The first category involves directly improving the deep learning regression objective or loss function. For example, in tasks related to depth estimation and phase regression, gradient loss is introduced. [1] Edge loss, ranking loss [2] Depth / Occlusion Prior [3,4] and supervised edge attention mechanism [5] These measures aim to reduce boundary blurring and enhance the resilience of edge areas.

[0029] The second category involves introducing smoothing constraints and edge preservation constraints into structured light phase retrieval. For example, through gradient smoothing... [6] Total variational model [7] These methods strike a trade-off between overall smoothness and edge preservation. While they can improve edge quality to some extent, they typically rely on edge masks explicitly generated from structured light fringe images or phase gradients, which are often unstable in complex scenes.

[0030] Furthermore, existing deep learning methods for structured light mainly focus on direct regression of different phase representations, such as regressing sine / cosine components. [8] , wrap phase [9] Phase order

[10] Or directly return to three-dimensional depth

[11] To improve measurement speed and reconstruction efficiency, these methods generally still do not systematically address the issues of adaptive localization of difficult-to-reach edge regions, edge feature enhancement, and the closed-loop coupling between edge supervision and main task optimization. Existing technologies also review typical technical approaches such as sine and cosine regression, wrapped phase joint prediction, phase unfolding-assisted prediction, and direct depth regression.

[0031] However, existing methods generally lack adaptive identification and closed-loop enhancement mechanisms for difficult edge regions, making it difficult to simultaneously ensure global accuracy and edge reconstruction quality.

[0032] This invention aims to solve the problems of excessively smooth edges, large phase regression errors, and distorted 3D reconstruction boundaries in existing structured light deep learning phase retrieval methods, such as object edges, abrupt depth changes, and complex texture regions.

[0033] Furthermore, the problems this invention aims to address include: existing methods lack adaptive priors that can directly reflect the difficult-to-regress regions of the current network; existing edge supervision is usually not tightly coupled with the backbone phase retrieval process, making it difficult to form a stable and effective enhancement mechanism for key edge regions; existing edge preservation methods often rely on explicit edge masks or phase gradient construction, but such information is not always easy to obtain accurately in structured light scenarios. Related literature has clearly pointed out that existing end-to-end convolutional networks are prone to oversmoothing in depth discontinuous regions, and many edge preservation methods rely on explicit edge masks or phase gradients, with insufficient training priors and key region weighting mechanisms.

[0034] In some embodiments, the present invention provides a structured light 3D reconstruction method based on gradient prior and edge attention, as shown in Figure 1. The method generally includes: structured light stripe image input, trunk phase information regression, loss gradient prior generation, edge attention enhancement, edge detection loss, edge perception loss, and 3D mapping after phase recovery.

[0035] In some embodiments, the loss gradient prior refers to: during training, calculating the gradient of the loss function with respect to the input structured light stripe image x using an automatic differentiation mechanism based on the global L1 loss constructed between the phase retrieval network output and the ground truth. The gradient response is then used as prior information characterizing the currently difficult-to-regress region. Preferably, the loss gradient prior is calculated separately for each training sample. Let the scale of the input structured light stripe image be ( Then, the gradient of the loss function with respect to the input image is obtained through automatic differentiation. The scale is also ( Furthermore, since the gradient may have a large and unstable numerical range, we perform Min-Max normalization after taking the absolute value of each batch to obtain a loss gradient prior map that corresponds one-to-one with the spatial location of the input image. .

[0036] in, To prevent the use of tiny constants with a denominator of 0, regions in the gradient prior map G with larger response values ​​indicate that these locations are more sensitive to the current regression error and harder for the network to fit. These typically correspond to object edges, regions of abrupt depth changes, and areas with localized detailed textures.

[0037] In some embodiments, during training, the loss gradient prior map can be calculated in real time based on the current network state with each parameter update, thereby dynamically reflecting the region that the network is most difficult to fit. Preferably, the loss gradient prior map is used to simultaneously drive the edge attention module (for feature fusion and weighted enhancement) and the edge detection module (edge ​​prediction network) to form joint enhancement and supervision of difficult regions. To improve training stability, in the early stages of training (the first 50 epochs), only L1 loss is used to optimize the phase retrieval network, and the loss gradient prior is introduced after the phase retrieval network reaches initial stability.

[0038] The loss gradient prior map was then used in the edge attention module and the edge detection module to improve the edge accuracy of structured light phase retrieval and 3D reconstruction.

[0039] In some embodiments, the phase retrieval network is used to receive at least one set of structured light fringe images and output phase information. The phase information is not limited to sine / cosine components, but can also be a wrapped phase, absolute phase, intermediate phase encoded representation, or other continuous or piecewise continuous representations that can be used for phase retrieval. Four-frequency single-step input is a preferred embodiment, but is not a necessary limitation in the independent claims of this invention.

[0040] In some embodiments, during training, based on the global loss (L1 loss) between the end-to-end phase regression network output and the ground truth, as shown in Figures 2 and 1, the gradient of the loss with respect to the input structured light stripe image is calculated (as shown in Figures 2 and 1, loss gradient), resulting in a loss gradient prior map at the same scale as the input space. The loss gradient prior is used to characterize the sensitivity of each input location to the current regression error. Regions with larger gradient magnitudes correspond to key regions that are more difficult for the current model to fit, typically including object edges, abrupt depth changes, and detailed texture regions.

[0041] In some embodiments, the edge attention module receives the intermediate feature map F( from the phase retrieval network). ) and loss gradient prior graph G( Preferably, the loss gradient prior map and the intermediate feature map are first concatenated along the channel dimension to obtain the fused features; then, an attention weight map A is generated through three layers of convolutional mapping. Preferably, all three convolutional layers employ... The convolutional kernel consists of two layers, the first two of which are used to extract and fuse edge-related features, and the last layer is used to generate a single-channel attention response map. Further, the attention weight map is mapped to the [0,1] interval using a sigmoid activation function, thus obtaining spatial attention weights that enhance key edge regions. Preferably, the first two convolutional layers are followed by a ReLU activation function.

[0042] The enhanced feature map can be represented as: in, This indicates element-wise multiplication. By using the above method, the phase retrieval network can allocate more representational power to regions with high error contribution during training, thereby improving the phase retrieval accuracy of key edge regions.

[0043] In some embodiments, the edge detection module takes the loss gradient prior map G as input. In another embodiment, the edge detection module takes the backbone feature map F and the loss gradient prior map G as input, and preferably generates a predicted edge probability map through feature concatenation and convolutional mapping. The values ​​of the predicted marginal probability map are preferably located in the interval [0,1].

[0044] In some embodiments, during the training phase, the marginal probability map is predicted. An edge supervision mask M constructed based on the true phase information is constrained using the MAE mask loss to form an edge-assisted task. Preferably, the edge supervision mask is generated by extracting gradient edges from the true phase map using the Sobel operator and then performing Min-Max processing; in other embodiments, the supervision signal can also be constructed using Scharr, Canny, Laplacian, or continuous edge response maps.

[0045] Subsequently, the edge weight map is predicted. Furthermore, these edge-weighted regions are used as pixel-level weights in the phase regression loss calculation, giving them higher weights during the optimization process and reducing the dominant role of smooth regions in the overall loss. Preferably, the edge-weighted loss can be expressed as...

[0046] in, This represents the phase correlation output predicted by the network. Indicates the corresponding truth value. This represents the total number of pixels. Meanwhile, the auxiliary supervision loss of the edge detection branch can be expressed as:

[0047] Preferably, the overall loss function includes global regression loss. Edge detection-assisted supervision loss and edge-weighted loss Its union form is: in, , This is the weighting coefficient. Preferably, , A pre-defined constant is selected based on the training process loss. (Example) , We select (1, 0.5, 0.5) respectively. Through the joint constraint of the above multiple losses, while maintaining the global phase accuracy, we improve the phase continuity, transition sharpness and 3D reconstruction edge quality of the boundary region.

[0048] Preferably, the overall loss function includes auxiliary task mask loss, global loss, and edge loss. By jointly constraining the above multiple losses, the phase continuity, transition sharpness, and 3D reconstruction edge quality of the boundary region are improved while maintaining global phase accuracy.

[0049] In some embodiments, during the inference phase, a structured light fringe image of the object under test is input, and the trained network obtains the phase information result; then, phase solution or inversion is performed according to the phase representation form adopted to obtain the phase information of the object under test; finally, the three-dimensional coordinate mapping is completed by combining the system calibration parameters, and the three-dimensional point cloud or three-dimensional shape result is output.

[0050] In some embodiments, the working principle of the present invention is as follows: First, a structured light measurement system is constructed. A projector projects phase-coded fringes onto the object under test, and a camera captures images of the deformed fringes modulated on the object's surface. The essence of structured light 3D reconstruction is to establish the correspondence between phase and 3D coordinates; through system calibration, the recovered target phase can be mapped to 3D coordinate space.

[0051] Secondly, during the training phase, the acquired multi-frequency structured light images are input into the phase retrieval network, which outputs a target phase correlation representation. This output can be a sine / cosine component, or it can be a representation of the wrapped phase, absolute phase, or other information that can recover the target phase. Preferably, the sine / cosine representation is suitable as a regression target due to its continuity and smoothness, but this is not the only output method that this invention must limit.

[0052] Next, the derivative of the current loss of the phase retrieval network with respect to the input stripe image is obtained to obtain the loss gradient prior. This gradient prior reflects the sensitivity of different input locations to the current error, thus characterizing the region where the network is currently most difficult to fit, typically corresponding to edges, fine textures, and regions with abrupt changes in depth. Existing techniques have clearly shown that this gradient prior has good consistency with edge regions and can dynamically guide the network to focus on high-error regions.

[0053] Then, the loss gradient prior and intermediate features are jointly input into the edge attention module to generate a spatial attention weight map, which enhances the intermediate features, enabling the phase retrieval network to allocate more representational power to hard-to-regress edge regions during training. This invention presents the basic form of the EA module, which generates an attention map jointly with the gradient prior and features, and enhances the features using residual modulation.

[0054] Simultaneously, the loss gradient prior is input into the edge detection module, and combined with edge supervision information extracted from the true phase, the edge detection branch is trained to output an edge probability map or an edge weight map. This output is further used as pixel-level weights in the phase regression loss calculation, thereby increasing the proportion of edge region errors in the total loss.

[0055] Finally, the global regression loss, edge mask loss, and edge weighted loss are jointly optimized. After training, the target phase is recovered based on the network output during the testing phase, and the three-dimensional point cloud of the target object is obtained through the pre-established calibration mapping relationship, thus achieving high edge accuracy structured light three-dimensional reconstruction.

[0056] The following will further describe specific embodiments of the present invention. These embodiments are merely illustrative and do not mean that the present invention is limited to the following examples.

[0057] Example 1: Constructing a structured light measurement system consisting of a DLP projector and an industrial camera, such as... Figure 3As shown. First, calibration samples are collected at multiple spatial locations using a calibration board to establish a pixel-level mapping relationship between phase and three-dimensional coordinates. Training data is then collected based on this system. During the training phase, a high-precision phase-shifting method is used to calculate phase reference information at multiple frequencies as the supervisory ground truth; in this embodiment, phase representation components at four frequencies are preferably used as supervisory information. During the inference phase, only multi-frequency single-step structured light fringe images need to be collected as network input.

[0058] The phase retrieval network employs an encoder-decoder backbone structure, incorporating an edge attention module and an edge detection module in its intermediate feature layer. During training, the global regression loss, edge masking loss, and edge-weighted loss are jointly optimized to enhance the network's ability to recover edge regions and areas with abrupt depth changes. In the testing phase, the wrapper phase and absolute phase are calculated based on the phase representation predicted by the network, and combined with a pre-established calibration mapping relationship to reconstruct the 3D point cloud of the target object.

[0059] This embodiment is applicable to measurement scenarios with obvious geometric abrupt changes, such as the edges of industrial parts, gears, bearings, and highly reflective metal surfaces, and can improve the phase recovery accuracy and 3D reconstruction quality of edge areas.

[0060] II. Comparative Examples and Transitional Examples Example 2: Only the edge attention module is retained, without the edge detection module. This scheme can enhance the network's attention to hard-to-regress regions by using loss gradient priors, and its edge performance is better than that of the unenhanced phase retrieval network, but its overall performance is generally weaker than the combined edge attention and edge detection scheme.

[0061] Example 3: Only the edge detection module is retained, without setting an edge attention module. This scheme improves boundary accuracy through real edge supervision and edge-weighted regression, but its adaptive focusing ability for difficult samples is not as good as the joint scheme.

[0062] Example 4: Edge supervision does not directly use a binary edge mask of the true phase, but uses a continuous edge response map or a multi-threshold edge level map for supervision, in order to enhance the fitting stability under different edge intensities.

[0063] Example 5: The input structured light image is not limited to a four-frequency single-step structured light fringe image; it can also be a two-frequency, three-frequency, or more-frequency single-step structured light fringe image, or a combination of multi-step phase-shifted structured light fringe images at the same frequency. The network output format is not limited to sine and cosine components; it can also be a wrapped phase, absolute phase, or other phase-related representations that can be used to recover the target's 3D information, and can even be further mapped to a 3D point cloud result. This scheme still guides the difficult-to-regress regions based on the loss gradient prior, and combines edge enhancement and edge supervision optimization to achieve phase recovery and 3D reconstruction. Therefore, it is suitable for structured light measurement scenarios with different requirements for measurement speed, reconstruction accuracy, and output format.

[0064] III. Possible Modified Embodiments Example 6: Expanding the source of gradient prior from the input image gradient to the loss gradient of intermediate feature maps, output phase maps, or multi-layer features, as long as it is still essentially used to indicate hard-to-regress regions and participate in edge enhancement training, should be included in the protection scope of this invention.

[0065] Example 7: Replace the edge attention module with channel attention, spatial attention, Transformer attention or a combination thereof. As long as it uses gradient prior to generate weights and enhances the backbone features, it is an equivalent solution.

[0066] Example 8: Replace the edge detection module with a boundary segmentation module, a saliency detection module, a contour prediction module, or other edge probability estimation modules. As long as their output is used to construct the edge loss, they should be included in the protection scope of this invention.

[0067] Example 9: Replace the edge operator from Sobel with Scharr, Laplacian, Canny, or a learning edge extractor, as long as it essentially generates edge supervision information from the real phase, it is a coverable implementation variation.

[0068] Example 10: Expanding the main output from sine / cosine components to wrapping phase, multi-frequency auxiliary features, or phase-related intermediate quantities, as long as the edge phase recovery accuracy is ultimately improved through gradient priors and edge enhancement mechanisms, should be included in the protection scope of this invention.

[0069] To verify the effectiveness of the method of this invention, a structured light 3D measurement system was constructed for experimentation. The system includes a projector and an industrial camera, wherein the projector is a DLP LightCrafter 6500, and the camera is a Basler acA2040-120μm with a 15 mm focal length lens. Both the projector and camera have a resolution of [missing information]. The system uses Zhang Zhengyou's calibration method for calibration. Calibration samples are collected at more than 20 different spatial locations using a black dot calibration plate. Spatial coordinates are obtained through the camera calibration process, and phase information is obtained based on a 12-step phase-shifting profilometry. The least squares method is then used to fit a mapping model between the phase and the three-dimensional coordinates.

[0070] The experimental data includes two typical structured light 3D reconstruction scenarios. Scenario 1 is a metal dataset containing 900 samples, mainly covering materials with different reflectivities such as stainless steel, aluminum alloy, and brass. Representative objects include gears, bearings, and other machined parts, with their main edge features being cut edges and standard mechanical structure boundaries. Scenario 2 is a plaster dataset containing 800 samples, primarily for 3D modeling scenarios, including samples of faces, teeth, and dolls. Its edge features are mainly reflected in clothing wrinkles, hair texture, and complex surface undulations. Both datasets are divided into training, validation, and test sets in an 8:1:1 ratio.

[0071] In the experiment, the sine / cosine components calculated at four frequencies using a 12-step phase-shift profilometry technique were used as the ground truth for network training. Training was implemented using Python and PyTorch, and the platform consisted of dual Intel Xeon CPUs, 64 GB of memory, and four Huawei Ascend AI NPUs. The batch size was set to 8, the initial learning rate to 0.001, and a cosine annealing strategy was employed, with a training duration of 400 epochs.

[0072] To verify the effectiveness of the edge attention module and edge detection module in this invention, UNet, ResNet18, ResNet34, ResNet50, ResNet101, and FCN were selected as the backbone baseline networks, and the following comparison scheme was set: (1) Baseline scheme using only phase retrieval network; (2) Phase recovery network combined with edge attention module (EA); (3) Phase retrieval network combined with edge detection module (ED); (4) The phase recovery network combines the edge attention module and the edge detection module (EA+ED).

[0073] The evaluation metrics include phase error MAE and 3D point cloud error MAE, and the error performance of the global region and the edge region are statistically analyzed separately.

[0074] The experimental results are partially visualized as follows: Figure 4 and Figure 5 As shown, where Figure 4The experimental subjects included two types of samples: gears and plaster dolls. (a) shows the test object and its actual 3D point cloud; (b)–(e) show the phase error map and 3D point cloud error map under different module combinations, where (b) shows the phase retrieval network alone, (c) shows the phase retrieval network combined with the edge attention module (EA), (d) shows the phase retrieval network combined with the edge detection module (ED), and (e) shows the phase retrieval network combined with both the edge attention module and the edge detection module (EA+ED).

[0075] The results show that the proposed solution achieves stable improvements on both metal and plaster datasets, and for multiple phase retrieval networks. Overall, introducing either the edge attention module or the edge detection module alone typically reduces global phase error and 3D point cloud error by approximately 10%–25%, with more significant improvements in edge region errors, typically reaching approximately 15%–30%. Furthermore, when the edge attention module and the edge detection module are used in combination, the overall performance is superior to the solution using only either module. On some typical phase retrieval networks, edge phase errors can be further reduced to approximately 1.85 rad and 0.93 rad, respectively, and edge 3D point cloud errors can be further reduced by approximately 5%–7% compared to using a single module. These results demonstrate that the proposed solution can significantly improve the retrieval quality of structured light 3D reconstruction in edge regions and regions with abrupt depth changes while maintaining global reconstruction accuracy.

[0076] In summary, compared with the prior art, the present invention has at least the following beneficial effects: 1. It can adaptively identify regions that are difficult to regress.

[0077] This invention constructs a loss gradient prior map by calculating the gradient of the loss function with respect to the input structured light stripe image, which is used to characterize the error sensitivity of the current network at different spatial locations. Compared with existing methods that rely on explicit edge masks or manual hard sample screening, this approach can more directly locate edge regions, regions of abrupt depth changes, and regions of detailed texture.

[0078] 2. It can effectively reduce the problem of overly smooth edges.

[0079] This invention uses an edge attention module to jointly model the loss gradient prior and intermediate features, and enhances the features of high error contribution areas. This allows the phase retrieval network to pay more attention to difficult edge areas during training, thereby improving the phase retrieval accuracy and 3D reconstruction quality of key boundary areas.

[0080] 3. It can form a closed-loop optimization through edge supervision and edge weighting.

[0081] This invention generates a predicted edge probability map through an edge detection module and uses it simultaneously for edge-assisted supervision and main task loss weighting, so that the edge region receives higher weight in the optimization process, thereby further improving the phase recovery accuracy and 3D reconstruction quality of the boundary region.

[0082] 4. It can significantly improve edge quality while maintaining global accuracy.

[0083] When the edge attention module and the edge detection module are used in combination, the overall performance is better than that of a scheme using only one of the modules. Experimental results show that on a typical phase retrieval network, the edge phase error can be reduced by about 24% to 35%, and the edge 3D point cloud error can be reduced by about 19% to 24%, indicating that the present invention can significantly improve the quality of edge region restoration while maintaining global reconstruction accuracy.

[0084] 5. It has good versatility and portability.

[0085] The edge attention module and edge detection module proposed in this invention are decoupled from the phase retrieval network, which can be easily integrated into different end-to-end phase retrieval networks such as UNet, ResNet series, and FCN. It also shows stable improvement in structured light measurement scenarios with different materials and texture complexities, indicating that this invention has good generalization ability and engineering application value.

[0086] Clearly, this invention not only improves the overall quality of structured light 3D reconstruction, but more importantly, it significantly reduces phase error and point cloud error in edge regions, thereby solving the problem of insufficient accuracy in edge regions in existing deep learning structured light methods.

[0087] It should be noted that the reason why the existing technology has not proposed the solution of this application is mainly because: on the one hand, the loss gradient is usually regarded as an interpretive analysis signal rather than a structural prior that can be directly used in training; on the other hand, using the gradient prior for edge attention construction, edge detection supervision and edge weighted regression at the same time requires unifying the network interpretation information, edge supervision mechanism and phase recovery objective into the same training loop, which is more complicated in design. Therefore, existing methods mostly remain at the level of a single loss or a single auxiliary module.

[0088] Compared to existing methods, this invention does not merely add an edge loss or edge branch, but rather unifies "adaptive identification of hard-to-regress regions," "edge feature enhancement," "edge-supervised generation," and "edge-weighted regression" into a single structured light phase retrieval training framework. By constructing the gradient of the loss function with respect to the input structured light stripe image as a training prior, it characterizes the hard-to-regress regions in the current network. This simultaneously drives edge attention enhancement and edge detection-assisted supervision. Furthermore, the predicted edge weights are further incorporated into the pixel-level weighting of the phase regression loss, thus forming a closed-loop optimization mechanism for hard-to-regress regions.

[0089] It should be noted that this invention can be widely applied to the following technical fields: (1) Industrial online inspection and dimensional measurement, including precision boundary measurement of complex edge parts, stamped parts, machined parts, mold parts, etc.; (2) Robot visual perception and grasping localization, especially suitable for three-dimensional perception of targets with obvious geometric boundaries; (3) Medical and oral 3D imaging, such as surface reconstruction of objects with rich details such as teeth, face, and organ models; (4) Digitalization of cultural relics, reconstruction of sculpture and complex texture models, used for high-quality 3D acquisition in scenes where boundary details and surface textures coexist; (5) Three-dimensional inspection tasks that are sensitive to boundary quality, such as consumer electronics, precision manufacturing, semiconductor packaging and microstructure measurement; (6) Other vision systems that use phase coding, stripe coding or similar optical coding methods for three-dimensional reconstruction, depth recovery or surface measurement.

[0090] The key documents and corresponding technical terms cited in the specific implementation process of this invention are listed below: Gradient loss [1] ; ranking loss [2] Depth / Occlusion Prior [3,4] Supervisory marginal attention mechanism [5] Gradient smoothing [6] Total variational model [7] ; Regression sine / cosine components [8] ; Wrap phase [9] Phase order

[10] Direct regression of 3D depth

[11] .

[0091] The corresponding reference list is as follows: [1] C. Chen, X. Chen, and H. Cheng, “On the over-smoothing problem ofcnn based disparity estimation,” in Proceedings of the IEEE / CVF InternationalConference on Computer Vision, 2019, pp. 8997–9005. [2] K. Xian, J. Zhang, O. Wang, L. Mai, Z. Lin, and Z. Cao,“Structureguided ranking loss for single image depth prediction,” inProceedings of the IEEE / CVF conference on computer vision and patternrecognition, 2020, pp. 611–620. [3] L. Xu, Y. Wang, and C. Wang, “Occlusion boundary and depth:Mutual enhancement via multi-task learning,” in Proceedings of the IEEE / CVFWinter Conference on Applications of Computer Vision, 2026, pp. 4427–4437. [4] S. Xie and Z. Tu, “Holistically-nested edge detection,” inProceedings of the IEEE international conference on computer vision, 2015,pp. 1395–1403. [5] J. Jiao, Y. Cao, Y. Song, and R. Lau, “Look deeper into depth:Monocular depth estimation with semantic booster and attention-driven loss,”in Proceedings of the European conference on computer vision (ECCV), 2018,pp. 53–69. [6] H. Liu, N. Yan, B. Shao, S. Yuan, and X. Zhang, “Deep learning infringe projection: A review,” Neurocomputing, vol. 581, p. 127493, 2024 [7] I. May-Cen, R. Legarda-Saenz, and C. Brito-Loeza, “A variationalmodel for wrapped phase denoising,” Mathematics, vol. 11, no. 12, p. 2618,2023. [8] S. Feng, Q. Chen, G. Gu, T. Tao, L. Zhang, Y. Hu, W. Yin, and C.Zuo, “Fringe pattern analysis using deep learning,” Advanced photonics, vol.1, no. 2, pp. 025 001–025 001, 2019. [9] H. Nguyen, E. Novak, and Z. Wang, “Accurate 3d reconstruction viafringe-to-phase network,” Measurement, vol. 190, p. 110663, 2022.

[10] H. Tan, Y. Xu, C. Zhang, Z. Xu, C. Kong, D. Tang, and B. Guo, “Ay-shaped network based single-shot absolute phase recovery method for fringeprojection profilometry,” Measurement Science and Technology, vol. 35, no. 3, p. 035203, 2023.

[11] S. Van der Jeught and JJ Dirckx, “Deep neural networks for single shot structured light profilometry,” Optics express, vol. 27, no. 12, pp. 17 091–17 101, 2019. Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0092] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0093] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0094] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0095] The background section of this invention may include background information about the problems or environment in which the invention is being developed, and is not necessarily a description of prior art. Therefore, the content included in the background section does not constitute an admission of prior art by the applicant.

[0096] The above description provides a further detailed explanation of the present invention in conjunction with specific / preferred embodiments, and it should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various substitutions or modifications can be made to these described embodiments without departing from the concept of the present invention, and all such substitutions or modifications should be considered within the scope of protection of the present invention. In the description of this specification, the reference to terms such as "an embodiment," "some embodiments," "preferred embodiment," "example," "specific example," or "some examples," etc., indicates that the specific features, structures, materials, or characteristics described in connection with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples. Although the embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions, and modifications can be made herein without departing from the scope of protection of the patent application.

Claims

1. A structured light 3D reconstruction method based on gradient prior and edge attention, characterized in that, Includes a training phase and an inference phase; The training phase includes: Obtain a training dataset, which includes structured light fringe images and corresponding ground phase maps; The structured light stripe image is input into a phase retrieval network to obtain a predicted phase map; a first loss between the predicted phase map and the corresponding true phase map is calculated; the gradient of the first loss with respect to the structured light stripe image is calculated to generate a loss gradient map, which characterizes the influence of different pixel regions in the structured light stripe image on the difficulty of phase regression. The loss gradient map is input into an edge prediction network to obtain a predicted edge probability map, which represents the possibility of the existence of discontinuous regions in the true phase map. The loss gradient map is fused with an intermediate feature map of the phase retrieval network to generate a spatial attention weight map; the intermediate feature map is then weighted and enhanced using the spatial attention weight map, and the enhanced intermediate feature map is fed back into the phase retrieval network for subsequent calculations to obtain an optimized predicted phase map; Calculate the second loss between the optimized predicted phase map and the true phase map; construct a joint loss function based on the first loss, the second loss, and the edge loss constructed by the predicted edge probability map, and update the phase recovery network based on the joint loss function; The inference stage includes: inputting the structured light fringe image of the object to be tested into the trained phase retrieval network to obtain its phase information; and performing three-dimensional reconstruction based on the phase information.

2. The structured light three-dimensional reconstruction method of claim 1, wherein, The edge loss is obtained by calculating the difference between the predicted edge probability map and a true edge label map.

3. The structured light three-dimensional reconstruction method of claim 2, wherein, The true edge label map is obtained by performing edge detection on the true phase map.

4. The structured light three-dimensional reconstruction method of claim 1, wherein, The fusion method for fusing the loss gradient map with an intermediate feature map of the phase recovery network is either concatenation or weighted addition along the channel dimension.

5. The structured light three-dimensional reconstruction method of claim 1, wherein, Generating a spatial attention weight map includes: inputting the fused feature map obtained by fusing the loss gradient map and the intermediate feature map into a convolutional neural network for processing, so as to generate the spatial attention weight map.

6. The structured light three-dimensional reconstruction method of claim 1, wherein, The method of weighting and enhancing the intermediate feature map using the spatial attention weight map includes: multiplying the spatial attention weight map and the intermediate feature map element by element.

7. The structured light three-dimensional reconstruction method of claim 1, wherein, The joint loss function is a weighted sum of the first loss, the second loss, and the edge loss.

8. The structured light three-dimensional reconstruction method of claim 1, wherein, The phase retrieval network is a neural network with an encoder-decoder structure; the edge prediction network is a network with an encoder-decoder structure. 9.A structured light 3D reconstruction system based on gradient prior and edge attention, characterized in that, include: The training module is used to perform operations during the training phase; The inference module is used to perform operations during the inference phase; The training module includes: a data acquisition unit for acquiring a training dataset, the training dataset including a structured light stripe image and a corresponding ground truth phase map; an initial prediction unit for inputting the structured light stripe image into a phase retrieval network to obtain a predicted phase map; a first loss calculation unit for calculating a first loss between the predicted phase map and the corresponding ground truth phase map; a gradient map generation unit for calculating the gradient of the first loss with respect to the structured light stripe image to generate a loss gradient map, the loss gradient map representing the influence of different pixel regions in the structured light stripe image on the difficulty of phase regression; and an edge prediction unit for inputting the loss gradient map into an edge prediction network to obtain a predicted edge probability map, the predicted edge probability map representing the... The probability of discontinuous regions existing in the true phase map; an attention weight generation unit, used to fuse the loss gradient map with an intermediate feature map of the phase retrieval network to generate a spatial attention weight map; a feature enhancement and optimization unit, used to use the spatial attention weight map to weight and enhance the intermediate feature map, and send the enhanced intermediate feature map back to the phase retrieval network for subsequent calculations to obtain an optimized predicted phase map; a second loss and joint update unit, used to calculate the second loss between the optimized predicted phase map and the true phase map, and construct a joint loss function based on the first loss, the second loss, and the edge loss constructed by the predicted edge probability map, and update the phase retrieval network based on the joint loss function; The inference module includes: a phase retrieval unit, used to input the structured light fringe image of the object under test into the trained phase retrieval network to obtain its phase information; and a three-dimensional reconstruction unit, used to perform three-dimensional reconstruction based on the phase information.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the structured light three-dimensional reconstruction method as described in any one of claims 1 to 8.

Images