A method, apparatus, and product for 3D reconstruction based on structured light projection images.

Abstract

This disclosure relates to the field of image processing technology, and proposes a method, apparatus, and product for 3D reconstruction based on structured light projection images. The method includes: acquiring a target projection image; generating a quality signal corresponding to the target projection image, wherein the quality signal is a reliability map used to indicate low-confidence regions; constructing a guiding signal related to the surface morphology changes of the object under test, and injecting the guiding signal into the reconstruction network or solution process of a deep learning model to enhance the foreground representation of the object under test; dividing the field of view of the target projection image into regions based on the quality signal, and using a deep learning model to select different inference paths or output strategies for different regions of the target projection image, fusing and outputting the 3D reconstruction data of the object under test. The technical solutions provided by one or more embodiments of this disclosure can achieve highly robust and high-precision 3D reconstruction of structured light projection images in complex scenes.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, specifically to a method, apparatus, and product for three-dimensional reconstruction based on structured light projection images. Background Technology

[0002] In the fields of industrial precision measurement and machine vision, single-shot fringe projection 3D reconstruction (SS-FPP) technology based on deep learning has attracted much attention due to its non-contact and high-efficiency characteristics. This technology projects a single sinusoidal or coded fringe pattern onto the surface of the object under test, and then a camera simultaneously acquires images of the fringe patterns that are deformed after being highly modulated by the surface of the object under test. Then, analytical models or data-driven methods (such as deep learning) can be used to calculate or predict the three-dimensional spatial coordinate information of the object under test from the single-frame deformed fringe image.

[0003] However, in real-world industrial scenarios, modulated stripe images often contain complex physical components. Existing 3D reconstruction algorithms, whether based on a fringe-to-Numerator-and-Denominator (F2ND) prediction path or an end-to-end prediction path that maps directly from the grating to the absolute phase, typically treat the entire image as an equivalent computational target, failing to effectively decouple effective information from noise information in the scene.

[0004] Therefore, traditional deep learning models struggle to maintain high reconstruction accuracy when faced with shadow areas caused by object occlusion, background interference from non-objects, and subtle surface contour features of the object being measured. This limits the application of the SS-FPP method in applications requiring high precision (e.g., micrometer-level). Summary of the Invention

[0005] In view of this, one or more embodiments of this disclosure provide a method, apparatus and product for three-dimensional reconstruction based on structured light projection images, which can achieve highly robust and high-precision three-dimensional reconstruction of structured light projection images in complex scenes, and ensure the flexibility of the output results.

[0006] In a first aspect, this disclosure provides a three-dimensional reconstruction method based on structured light projection images. The method includes: acquiring a target projection image, wherein the target projection image is an image acquired after projecting structured light onto an object under test; generating a quality signal corresponding to the target projection image, wherein the quality signal is a reliability map used to indicate low-confidence regions; constructing a guiding signal related to the surface morphology changes of the object under test, and injecting the guiding signal into the reconstruction network or solution process of a deep learning model to enhance the foreground representation of the object under test; dividing the field of view of the target projection image into regions based on the quality signal, and using the deep learning model to select different inference paths or output strategies for different regions of the target projection image, and fusing and outputting the three-dimensional reconstruction data of the object under test.

[0007] Secondly, this disclosure provides a three-dimensional reconstruction device based on structured light projection images. The device includes: an image acquisition unit for acquiring a target projection image, wherein the target projection image is an image acquired after projecting structured light onto an object under test; a first calculation unit for generating a quality signal corresponding to the target projection image, wherein the quality signal is a reliability map used to indicate low-confidence regions; a second calculation unit for constructing a guiding signal related to the surface morphology changes of the object under test, and injecting the guiding signal into the reconstruction network or solution process of a deep learning model to enhance the foreground representation of the object under test; and a three-dimensional reconstruction unit for dividing the field of view of the target projection image into regions based on the quality signal, and using the deep learning model to select different inference paths or output strategies for different regions of the target projection image, and fusing and outputting the three-dimensional reconstruction data of the object under test.

[0008] Thirdly, this disclosure provides a computer program product, which includes computer instructions for causing a computer to execute the above-described three-dimensional reconstruction method based on structured light projection images.

[0009] This disclosure provides a technical solution through one or more embodiments. By generating quality signals, the field of view of the target projection image is divided into regions, which can accurately identify low-confidence regions and suppress their negative impact on model learning and output results, avoiding local anomalies and error propagation. By constructing guiding signals related to the surface morphology changes of the object under test and injecting them into the deep learning model, the foreground representation of the object under test during the 3D reconstruction process is effectively enhanced, and the weights of background and interference regions are reduced, thereby improving the detail fidelity of the reconstruction. Based on the quality signals, differentiated inference paths or output strategies are executed for different regions of the target projection image, and by fusing the output 3D reconstruction data, the reconstruction accuracy and robustness in complex scenes are guaranteed. Furthermore, the method can adaptively switch between end-to-end fast output and intermediate quantity solving modes according to actual engineering needs (such as accuracy, latency, and computing power), significantly improving the practicality and deployment flexibility of the method. Attached Figure Description

[0010] The features and advantages of the embodiments of this disclosure will be more clearly understood by referring to the accompanying drawings, which are illustrative and should not be construed as limiting the scope of this disclosure in any way. In the drawings: Figure 1 A schematic diagram illustrating the steps of a three-dimensional reconstruction method based on structured light projection images in one embodiment of this disclosure is shown. Figure 2 A schematic diagram of the functional modules of a three-dimensional reconstruction device based on structured light projection images is shown in one embodiment of this disclosure; Figure 3 A schematic diagram of the structure of an electronic device according to one embodiment of the present disclosure is shown. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0012] In the embodiments of this disclosure, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0013] The single-frame 3D reconstruction technology in related technologies has the following three main shortcomings.

[0014] First, there is backpropagation interference from invalid gradients: during fringe projection, the shadow regions, due to their extremely low light intensity, essentially contain no phase information. Existing training criteria (such as simple L1 or MSE loss) still force the network to fit the shadow regions, causing invalid noise gradients to contaminate the weights of the effective foreground regions during backpropagation, thus reducing the overall reconstruction accuracy.

[0015] Secondly, insufficient attention is paid to foreground features: there are significant differences between the test object and its background in terms of spatial frequency and texture gradient, but traditional network structures lack targeted feature extraction branches, which causes irrelevant information such as background to distract the model's attention, resulting in smoothing or loss of details in the reconstruction results of object edges and complex surfaces.

[0016] Furthermore, the output mode is limited: existing technologies are often fixed to specific predictive pipelines, making it difficult to flexibly balance and switch between different tasks such as "real-time obstacle avoidance" and "precision quality inspection" according to the accuracy and speed requirements of different tasks in industrial settings.

[0017] In view of this, the three-dimensional reconstruction method based on structured light projection images provided by one embodiment of this disclosure can solve the problems of decreased reconstruction accuracy, boundary contamination and error propagation caused by interference from low confidence areas (such as shadows, strong reflections, etc.) in structured light three-dimensional reconstruction in complex scenes. It can achieve highly robust and high-precision three-dimensional reconstruction, while taking into account the flexibility of output strategies in engineering deployment.

[0018] Please see Figure 1 The present disclosure provides a three-dimensional reconstruction method based on structured light projection images, which may include the following steps.

[0019] S1: Acquire the target projection image, which is an image obtained by projecting structured light onto the object to be tested.

[0020] In this embodiment, the target projection image can refer to any single-frame, double-frame, or multi-frame image acquired by projecting structured light (including stripes, speckles, rectangular waves, sinusoidal gratings, etc.) onto the object under test.

[0021] Optionally, while acquiring the target projection image, auxiliary information such as background reference image (for background discrimination or background removal), camera / projector calibration parameters (for mapping phase / depth to 3D point cloud), task mode parameters or resource constraint parameters (e.g., frame rate / latency / computing budget) can also be acquired to better achieve the 3D reconstruction target.

[0022] In this embodiment, various data / image preprocessing techniques can be performed on the acquired target projection image, including but not limited to normalization, scale unification, lightweight denoising, and dynamic range processing, to improve the stability of region discrimination and network inference in subsequent steps. If a background reference image or camera calibration information is obtained, it can be read in and the necessary alignment or parameter parsing can be completed simultaneously during data / image preprocessing.

[0023] S2: Generate a quality signal corresponding to the target projection image, the quality signal being a reliability map used to indicate low-confidence regions.

[0024] In this embodiment, a quality signal is introduced to model the credibility of each region of the target projection image, which can explicitly distinguish low-credibility regions such as shadows from the target's credible regions. The quality signal can be used for both the constraint training of the deep learning model (the deep learning model specifically designed for 3D reconstruction tasks in this disclosure) and its inference process, thereby focusing the learning and output of the deep learning model on the reliably reconstructable feasible target region and preventing errors from low-credibility regions from dominating model optimization. The reliability map includes, but is not limited to, masks, weight maps, and confidence maps.

[0025] In some implementations, the low-confidence region includes one or more of the following: shadow occlusion region, low-contrast region, overexposed region, underexposed region, reflection saturation region, and background region; generating a quality signal corresponding to the target projection image includes generating the quality signal using at least one of threshold discrimination, statistical feature analysis, frequency domain feature analysis, learning segmentation, confidence network prediction, or multi-clue fusion.

[0026] Specifically, low-confidence regions of the target projection image may include, but are not limited to, areas unfavorable to phase / depth recovery, such as shadows, low contrast, low modulation, saturation, and underexposure. Low-confidence regions of the target projection image may also include background regions, such as background panels, fixtures, platforms, or non-measurement target areas. The initial estimation method for low-confidence regions is not limited and can be obtained using threshold discrimination (e.g., brightness / saturation detection), statistical features (e.g., local variance / contrast), frequency domain features (e.g., dominant frequency energy / modulation amplitude), learned segmentation, confidence prediction networks, or multi-cue fusion. Preferably, reliability maps of each target confidence region can be generated simultaneously with the generation of reliability maps for each low-confidence region.

[0027] In a practical application example, to improve the stability of reliability maps (such as masks) in complex scenes, morphological processing, connectivity filtering, region aggregation, or boundary smoothing can be performed on the reliability maps of low-confidence regions. In some preferred methods, statistics related to prediction residuals or consistency errors can also be introduced to iteratively update and correct the reliability maps of low-confidence regions, so that the reliability maps gradually conform to the real low-confidence distribution as training progresses, thereby improving the stability of the reliability maps under complex lighting and material conditions.

[0028] In some implementations, the method includes: during the training phase of the deep learning model, using the quality signal to perform regional control on the model optimization terms of the deep learning model, so as to suppress the impact of the low-confidence region on the model update.

[0029] Specifically, introducing a region-selective suppression mechanism during the training phase can prevent low-confidence regions from dominating network parameter updates. This region-selective suppression mechanism is not limited to multiplication operations between the network output and the reliability map (e.g., a mask), but can be summarized as regionalized control of model optimization terms such as loss contribution, gradient propagation, and sample selection, reducing or eliminating the contribution of low-confidence regions to parameter updates in model optimization terms.

[0030] In some implementations, the step of using the quality signal to perform regionalized control on the model optimization terms of the deep learning model includes at least one of the following: multiplying and weighting the quality signal with the network output or error term of the deep learning model; calculating the loss or participating in the summation only for the pixel set within the target confidence region, ignoring or setting the loss of the low confidence region to zero; setting differentiated loss weights for different regions, or using threshold gating, to significantly reduce the loss contribution of the low confidence region; gating, scaling, or directly stopping the propagation of the gradient in the low confidence region to reduce or eliminate the impact of the low confidence region on backpropagation updates; using robust loss or error truncation mechanisms for the low confidence region to limit the impact of outliers on optimization; and increasing the proportion of samples trained in the target confidence region through importance sampling or regionalized sampling strategies.

[0031] Specifically, the region-selective suppression mechanism is not limited to multiplicatively weighting the reliability map (e.g., mask) with the network output or error term, but can be implemented using any of the following or an equivalent combination of methods: First, calculate the loss or participate in the summation only for the pixel set of the target reliable region, ignoring or zeroing the loss of low-reliability regions; Second, set different loss weights for different regions or use threshold gating to significantly reduce the contribution of low-reliability region loss; Third, gate, scale, or stop gradient propagation for the gradient of low-reliability regions to reduce or eliminate its impact on backpropagation updates; Fourth, use robust loss or error truncation mechanisms for low-reliability regions to limit the impact of outliers on optimization; Fifth, increase the proportion of samples trained in the target reliable region through importance sampling or regional sampling strategies. The above suppression mechanisms can not only be applied to the master supervision loss of the deep learning model, but also to the consistency loss, regularization term, distillation loss, or other training objectives of the deep learning model, thereby achieving the effect of "shadow-free backpropagation / weak backpropagation" under different training paradigms, and concentrating the model capacity on learning the structured light patterns and target surface morphology of reliable regions.

[0032] S3: Construct a guiding signal related to the surface morphology changes of the object under test, and inject the guiding signal into the reconstruction network or solution process of the deep learning model to enhance the foreground representation of the object under test.

[0033] In this embodiment, a target enhancement guidance mechanism is introduced to reduce background interference, improve the detail recovery capability of the target foreground, and enhance robustness to complex scenes (such as shadows, reflections, weak textures, and strong background textures). During the 3D reconstruction of the structured light projection image, regression is performed not only based on the target projection image itself but also by utilizing guiding cues (including but not limited to gradients, edges, contour differences, frequency domain energy differences, structural tensors, or prediction residuals) related to changes in the surface morphology of the object under test (including geometric and phase changes) to generate guiding signals. Injecting these guiding signals into the recovery network or solution process of the deep learning model in a controllable manner can continuously enhance the representation of the target region, suppress background and low-confidence region features, and improve the quality of boundary and detail reconstruction. It should be noted that the guiding signals can be used to enhance the response of the target region and also to suppress, exclude, or weaken the background and low-confidence regions, thereby achieving bidirectional control of enhancement and suppression.

[0034] In some implementations, constructing a guiding signal related to the surface morphology changes of the object under test includes: the guiding signal originating from a single information source or a combination of multiple information sources, wherein the information source includes at least one of features directly calculated from the input image, intermediate results within the network, and external or auxiliary information; and the guiding signal is expressed in at least one of pixel-level weight maps, region-level gating signals, boundary or structural saliency maps, vector fields, and multi-channel guiding feature maps.

[0035] Specifically, the guiding signal can originate from a single information source or a combination of multiple information sources, and its specific form is not limited. The guiding signal can be directly calculated from the input image, such as brightness variations, local gradients, edges / contours, stripe directionality, stripe contrast or modulation information, spatial frequency domain energy and dominant frequency amplitude, structural tensor response, local coherence indices, etc. The guiding signal can also be generated from intermediate results within the network, such as gradient fields, residual fields, confidence / uncertainty maps, attention response maps, cross-scale consistency error maps, etc., of intermediate phase / depth predictions. The guiding signal can also be generated from external or auxiliary information, such as background reference maps, calibration information, prior geometric models, teacher model outputs, or difference maps and anomaly region indication maps obtained through consistency constraints (e.g., multi-scale consistency, geometric consistency, periodic consistency, physical consistency). The above multi-source information can be fused using fusion strategies to generate a unified guiding signal. For example, through normalization, thresholding, morphological processing, region aggregation, or lightweight networks for encoding and fusion, a target enhancement map or weight map can be obtained.

[0036] The guiding signal can be expressed as a pixel-level weight map, a region-level gating signal, a boundary / structure saliency map, a vector field (e.g., gradient direction and magnitude), or a multi-channel guiding feature map, and can be constructed at a single scale or multiple scales. Processing the guiding signal using a guiding branch, guiding module, or guiding encoder can output attention maps, gating weights, enhanced features, or dynamic parameters to enhance the backbone recovery process of a deep learning model. The guiding signal can act on the network input, feature representation, cross-layer skip connections, attention weights, fusion weights, convolution kernel selection, normalization parameters, or output layer results of the deep learning model. In other words, the guiding signal can play a role in the data layer, feature layer, structure layer, or output layer of the deep learning model. Optionally, jointly outputting quality-related auxiliary results (including but not limited to masks, confidence scores, uncertainties, or residual maps) and using them for training constraints, path selection, and post-processing control of the deep learning model can form a "quality-driven" controllable enhancement framework.

[0037] In some implementations, injecting the guiding signal into the reconstruction network or solution process of the deep learning model includes: the location of the guiding signal injection includes at least one of the encoding stage, decoding stage, multi-scale fusion module, and skip connection of the deep learning model; the injection method of the guiding signal includes at least one of feature concatenation, additive fusion, gating control, attention modulation, conditional normalization, dynamic convolution or dynamic operator selection, and guided regional reweighting or regional fusion.

[0038] Specifically, guiding signals can be injected into the encoding stage of a deep learning model to suppress background responses in advance, or into the decoding stage to enhance detail and boundary recovery. They can also be injected at multi-scale fusion or skip connections to achieve consistent enhancement across scales. Injection methods include, but are not limited to, feature concatenation, summation fusion, gating control, attention modulation, conditional normalization (e.g., conditional scale / bias modulation of features), dynamic convolution or dynamic operator selection, and guided regional reweighting or regional fusion. To ensure the effectiveness and stability of the guiding signal during model training, auxiliary constraints can be applied to the guiding branch or output, such as guiding sparsity / smoothness constraints, alignment constraints with target boundaries / structures, consistency constraints with backbone predictions, or suppression constraints on background / low-confidence regions. This allows the guiding signal to have generalizable enhancement effects in different scenarios.

[0039] During the inference phase of deep learning models, guidance mechanisms can operate in multiple ways: they can directly use guidance signals calculated from the input for online enhancement; they can dynamically generate guidance using confidence and structural responses estimated internally by the network; and they can adaptively adjust guidance strength and injection location based on quality assessment results. For example, they can increase suppression strength in scenes with stronger background interference or larger low-confidence regions, and enhance boundary cue weights in scenes with more complex target boundaries or richer details. Through the above guidance signal construction, expression, and injection mechanisms, the network can continuously strengthen the expression of the target's reliable regions and suppress background and low-confidence region features, thereby improving the clarity of the target boundary, the fidelity of detailed textures, and the accuracy of local shape reconstruction of the object under test.

[0040] S4: Based on the quality signal, the field of view of the target projection image is divided into regions, and the deep learning model is used to select different inference paths or output strategies for different regions of the target projection image, and the three-dimensional reconstruction data of the object under test is fused and output.

[0041] This implementation provides an output path selection strategy, allowing different reconstruction paths to be chosen based on quality assessment results or resource constraints, and even enabling regionalized hybrid inference. This can ensure the accuracy of 3D reconstruction while meeting the latency and throughput requirements of online detection. For example, when computing power is limited and real-time performance is extremely critical, the input deformed fringe pattern can be directly mapped to the final 3D coordinates or depth map. Alternatively, when computing power is sufficient, latency is acceptable, and high accuracy is required, the final 3D result is not directly output. Instead, some intermediate variables with clear physical meaning (such as wrapping phase, numerator / denominator, modulation, background light intensity, etc.) are output. Then, the final 3D information is calculated from these intermediate quantities using traditional phase unwrapping algorithms and phase-height mapping relationships.

[0042] In this implementation, not only can a single path output be selected at the global level, but different inference strategies, different output heads, or different back-end solution methods can also be used for different regions at the pixel level, region level, or block level, and the region results can be fused to obtain better reconstruction quality under the condition of limited total computing resources.

[0043] In this embodiment, the fused output 3D reconstruction data may include, but is not limited to, 3D reconstruction results such as absolute phase maps, depth maps, and point clouds. The fused output 3D reconstruction data may also include, but is not limited to, intermediate 3D reconstruction information such as shadow / low confidence masks, background masks, target confidence region weight maps, pixel-level confidence / uncertainty maps, and fusion weights or residual statistics, for engineering quality control and interpretability output.

[0044] In this implementation, the output results from each region are fused to obtain the final hybrid output. Fusion methods include, but are not limited to: weighted fusion based on weight graphs, selective replacement based on confidence levels, stitching fusion under boundary consistency constraints, or smooth transition and consistency correction in overlapping areas. To avoid discontinuities at the boundaries of different path results, boundary bands can be introduced at the region boundaries and a boundary-preserving fusion strategy can be adopted, or local consistency constraints can be applied before and after fusion. Through the above-mentioned regionalized hybrid inference and hybrid output mechanism, while maintaining high throughput in high-confidence regions, a more robust recovery method can be adopted for low-confidence regions, thereby improving the overall quality and engineering usability of single-frame 3D reconstruction in complex scenes under conditions of limited overall computing power and latency.

[0045] In this embodiment, to support regionalized hybrid inference in the inference phase of the deep learning model, multiple output heads or multiple computational paths (e.g., fast paths and precision paths) and the generation method of fusion weights can be trained simultaneously during the training phase. For example, during training, supervision strategies for different regions can be constructed based on confidence statistics or residual statistics, so that the fast path can obtain good fitting ability in high confidence regions, the precision path can have stronger robustness in low confidence regions, and a fusion / selection strategy can be learned to avoid boundary discontinuities.

[0046] In some implementations, the step of dividing the field of view of the target projection image into regions based on the quality signal and using the deep learning model to select different inference paths or output strategies for different regions of the target projection image includes: masking or adding confidence labels to the output of the low-confidence regions; adopting differentiated post-processing strategies for different regions, the post-processing strategies including one or more of smoothing, hole filling, or boundary preservation; and outputting the quality signal as a region confidence weight in the point cloud generation or subsequent detection stages to improve the engineering usability, anomaly interpretability, and quality traceability of the reconstruction results.

[0047] Specifically, during the inference phase of the deep learning model, the low-confidence region mask / weight map generated in step S2 can be further used for result output control and post-processing. For example, the low-confidence region output can be masked or labeled with confidence; different strengths of smoothing, hole-filling, or boundary-preserving strategies can be adopted for different regions; and the confidence level can be used as a quality weight output in point cloud generation or subsequent detection stages, thereby improving engineering usability, anomaly interpretability, and quality traceability.

[0048] In some implementations, the step of dividing the field of view of the target projection image into regions based on the quality signal and using the deep learning model to select different inference paths or output strategies for different regions of the target projection image includes at least one of the following: using an end-to-end fast inference path to generate absolute phase or depth information as a first result; using a precision-first path to output intermediate physical quantities and combine them with back-end constraints or solution methods to generate a second result.

[0049] Specifically, based on the low-confidence region mask / weight map generated in step S2 and the auxiliary quality information output in step S3 (such as confidence level / uncertainty / residual statistics), the input field of view can be divided into regions, resulting in at least two types of regions: high-confidence regions and low-confidence regions (or further subdivided into multiple quality level regions). Different inference and output strategies can be selected for different quality regions, including but not limited to: using an end-to-end fast inference path to directly output absolute phase / depth in high-confidence regions; using a precision-first path to output intermediate physical quantities and combining backend constraints or solutions to obtain more robust results in low-confidence regions; or enabling stronger guided injection, higher-resolution branches, stricter consistency constraints, or more conservative post-processing strategies in low-confidence regions. The above regionalization strategies are not limited to two-type region divisions, nor are they limited to fixed thresholds; the division threshold can be adaptively determined by the quality assessment results and can be dynamically adjusted according to scene changes.

[0050] In some practical application scenarios, the expected application scenarios of the technical solutions provided by one or more embodiments of this disclosure after implementation can be: high reflectivity workpiece detection, used for precise morphology measurement of the surface of metal parts, using gradient guidance to solve the edge recognition problem caused by metal reflection; dynamic robot grasping, using a fast mapping mode on high-speed production lines to provide millisecond-level object 3D positioning for industrial robots; quality traceability under complex working conditions, in warehouse environments with uneven lighting or cluttered backgrounds, using a noise suppression mechanism to achieve accurate 3D modeling and size discrimination of target objects.

[0051] This disclosure provides a technical solution through one or more embodiments. By generating quality signals, the field of view of the target projection image is divided into regions, which can accurately identify low-confidence regions and suppress their negative impact on model learning and output results, avoiding local anomalies and error propagation. By constructing guiding signals related to the surface morphology changes of the object under test and injecting them into the deep learning model, the foreground representation of the object under test during the 3D reconstruction process is effectively enhanced, and the weights of background and interference regions are reduced, thereby improving the detail fidelity of the reconstruction. Based on the quality signals, differentiated inference paths or output strategies are executed for different regions of the target projection image, and by fusing the output 3D reconstruction data, the reconstruction accuracy and robustness in complex scenes are guaranteed. Furthermore, the method can adaptively switch between end-to-end fast output and intermediate quantity solving modes according to actual engineering needs (such as accuracy, latency, and computing power), significantly improving the practicality and deployment flexibility of the method.

[0052] Please see Figure 2 This disclosure also provides a three-dimensional reconstruction apparatus based on structured light projection images, the apparatus comprising: The image acquisition unit 100 is used to acquire a target projection image, wherein the target projection image is an image acquired after projecting structured light onto the object to be tested; The first computing unit 200 is used to generate a quality signal corresponding to the target projection image, wherein the quality signal is a reliability map used to indicate low confidence regions; The second computing unit 300 is used to construct a guiding signal related to the surface morphology change of the object under test, and inject the guiding signal into the reconstruction network or solution process of the deep learning model to enhance the foreground representation of the object under test. The 3D reconstruction unit 400 is used to divide the field of view of the target projection image into regions based on the quality signal, and to use the deep learning model to select different inference paths or output strategies for different regions of the target projection image, and to fuse and output the 3D reconstruction data of the object under test.

[0053] In one embodiment, the low-confidence region includes one or more of the following: shadow occlusion region, low contrast region, overexposed region, underexposed region, reflection saturation region, and background region; the first computing unit 200 is specifically used to generate the quality signal by using at least one of threshold discrimination, statistical feature analysis, frequency domain feature analysis, learning segmentation, confidence network prediction, or multi-clue fusion.

[0054] In one implementation, during the training phase of the deep learning model, the quality signal is used to perform regional control on the model optimization terms of the deep learning model to suppress the impact of the low-confidence region on model updates.

[0055] In one embodiment, the 3D reconstruction unit 400 is specifically used to: mask or add confidence labels to the output of the low-confidence region; adopt differentiated post-processing strategies for different regions, the post-processing strategies including one or more of smoothing, hole filling or boundary preservation; and output the quality signal as a region confidence weight in the point cloud generation or subsequent detection stage to improve the engineering usability, anomaly interpretability and quality traceability of the reconstruction results.

[0056] In one embodiment, the three-dimensional reconstruction unit 400 is further configured to: generate absolute phase or depth information as a first result using an end-to-end fast inference path; and generate a second result by outputting intermediate physical quantities and combining them with backend constraints or solution methods using a precision-first path.

[0057] In one embodiment, the guiding signal originates from a single information source or a combination of multiple information sources, the information source including at least one of features directly calculated from the input image, intermediate results within the network, and external or auxiliary information; the expression form of the guiding signal includes at least one of pixel-level weight map, region-level gating signal, boundary or structural saliency map, vector field, and multi-channel guiding feature map.

[0058] In one embodiment, the location of the guiding signal injection includes at least one of the encoding stage, decoding stage, multi-scale fusion module, and skip connection of the deep learning model; the method of injecting the guiding signal includes at least one of feature concatenation, additive fusion, gating control, attention modulation, conditional normalization, dynamic convolution or dynamic operator selection, and guided regional reweighting or regional fusion.

[0059] The various units described in the above embodiments can be implemented by a computer chip or by a product with a certain function. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0060] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0061] Please see Figure 3 This disclosure also provides an electronic device, which includes a memory and a processor. The memory is used to store a computer program, and when the computer program is executed by the processor, it implements the above-described three-dimensional reconstruction method based on structured light projection images.

[0062] This disclosure also provides a computer-readable storage medium for storing a computer program that, when executed by a processor, implements the above-described three-dimensional reconstruction method based on structured light projection images.

[0063] The processor can be a central processing unit (CPU). It can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof.

[0064] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the methods in the above-described embodiments.

[0065] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0066] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0067] This disclosure also provides a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions of this disclosure through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, and installation package files. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions; or the computer compiling the instructions and then executing the corresponding compiled program; or the computer reading and executing the instructions; or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0068] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, embodiments of apparatus, devices, computer storage media, and program products are basically similar to method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0069] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

[0070] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A three-dimensional reconstruction method based on structured light projection images, characterized in that, The method includes: Acquire a target projection image, which is an image obtained by projecting structured light onto the object under test; Generate a quality signal corresponding to the target projection image, wherein the quality signal is a reliability map used to indicate low-confidence regions; Construct a guiding signal related to the surface morphology changes of the object under test, and inject the guiding signal into the reconstruction network or solution process of the deep learning model to enhance the foreground representation of the object under test; Based on the quality signal, the field of view of the target projection image is divided into regions, and the deep learning model is used to select different inference paths or output strategies for different regions of the target projection image, and the three-dimensional reconstruction data of the object under test is fused and output.

2. The method according to claim 1, characterized in that, The low-confidence region includes one or more of the following: shadow occlusion region, low contrast region, overexposed region, underexposed region, reflection saturation region, and background region. The generation of a quality signal corresponding to the target projection image includes: The quality signal is generated using at least one of threshold discrimination, statistical feature analysis, frequency domain feature analysis, learning segmentation, confidence network prediction, or multi-clue fusion.

3. The method according to claim 2, characterized in that, The method includes: During the training phase of the deep learning model, the quality signal is used to perform regional control on the model optimization terms of the deep learning model in order to suppress the impact of the low-confidence region on the model update.

4. The method according to claim 3, characterized in that, The method of using the quality signal to perform regionalized control of the model optimization terms of the deep learning model includes at least one of the following: The quality signal is multiplied and weighted with the network output or error term of the deep learning model; Loss is calculated or summed only for the set of pixels within the target trusted region, while the loss in the low-confidence region is ignored or set to zero; Differentiated loss weights are set for different regions, or threshold gating is used to significantly reduce the loss contribution of the low-confidence region. The gradients in the low-confidence regions can be gated, scaled, or propagated directly stopped to reduce or eliminate the impact of the low-confidence regions on backpropagation updates. A robust loss or error truncation mechanism is applied to the low-confidence region to limit the impact of outliers on optimization; By employing importance sampling or regional sampling strategies, the proportion of samples trained in the target trust region can be increased.

5. The method according to claim 1, characterized in that, The step of dividing the field of view of the target projection image into regions based on the quality signal, and using the deep learning model to select different inference paths or output strategies for different regions of the target projection image, includes: The output of the low-confidence region is masked or labeled with confidence level. Different post-processing strategies are adopted for different regions, and the post-processing strategies include one or more of smoothing, hole filling or boundary preservation; In point cloud generation or subsequent detection, the quality signal is output as a region confidence weight to improve the engineering usability, anomaly interpretability, and quality traceability of the reconstruction results.

6. The method according to claim 5, characterized in that, The step of dividing the field of view of the target projection image into regions based on the quality signal, and using the deep learning model to select different inference paths or output strategies for different regions of the target projection image, includes at least one of the following: An end-to-end fast inference path is adopted to generate absolute phase or depth information as the first result; By adopting a precision-first approach, intermediate physical quantities are output and combined with backend constraints or solution methods to generate a second result.

7. The method according to claim 1, characterized in that, The construction of a guiding signal related to the surface morphology changes of the object under test includes: The guiding signal originates from a single information source or a combination of multiple information sources, and the information source includes at least one of the following: features directly calculated from the input image, intermediate results within the network, and external or auxiliary information. The guiding signal can be expressed in at least one of the following forms: pixel-level weight map, region-level gating signal, boundary or structural saliency map, vector field, and multi-channel guiding feature map.

8. The method according to claim 7, characterized in that, The process of injecting the guiding signal into the reconstruction network or solution of the deep learning model includes: The location where the guiding signal is injected includes at least one of the encoding stage, decoding stage, multi-scale fusion module, and skip connection of the deep learning model; The injection method of the guiding signal includes at least one of feature splicing, addition fusion, gating control, attention modulation, conditional normalization, dynamic convolution or dynamic operator selection, and guided regional reweighting or regional fusion.

9. A three-dimensional reconstruction device based on structured light projection images, characterized in that, The device includes: An image acquisition unit is used to acquire a target projection image, wherein the target projection image is an image acquired after projecting structured light onto the object to be tested; A first computing unit is configured to generate a quality signal corresponding to the target projection image, wherein the quality signal is a reliability map used to indicate low-confidence regions. The second computing unit is used to construct a guiding signal related to the surface morphology changes of the object under test, and to inject the guiding signal into the reconstruction network or solution process of the deep learning model to enhance the foreground representation of the object under test. The three-dimensional reconstruction unit is used to divide the field of view of the target projection image into regions based on the quality signal, and to use the deep learning model to select different inference paths or output strategies for different regions of the target projection image, and to fuse and output the three-dimensional reconstruction data of the object under test.

10. A computer program product, characterized in that, The computer program product includes computer instructions for causing the computer to perform the method described in any one of claims 1 to 8.

Images