A structured light shadow region recovery reconstruction method based on kernel density estimation

Abstract

The present application belongs to the technical field of computer vision three-dimensional reconstruction, and particularly relates to a structured light shadow area recovery and reconstruction method based on kernel density estimation. The present application aims at the problem of shadow area generated by structured light when measuring objects with sharp height changes, and divides the measurement results into effective area and invalid area according to modulation intensity information; phase calculation and unwrapping are performed on the effective area, and for the invalid area, a kernel density estimation method is used to calculate a target vector set and construct a probability density function according to the phase information of the effective context pixels in the neighborhood, and the phase value of the shadow area is recovered by optimizing the objective function to minimize the context pixel phase prediction error and the local phase recovery error; the phase information is converted into three-dimensional coordinates in combination with the system calibration parameters, and three-dimensional topography measurement of the object is realized. The present application can effectively solve the problems of occlusion and shadow generated when measuring objects with large height difference, and improve the integrity of three-dimensional reconstruction.

Description

Technical Field

[0001] This invention belongs to the field of computer vision 3D reconstruction technology, specifically relating to a method for restoring and reconstructing structured light shadow regions based on kernel density estimation. Background Technology

[0002] Structured light-based 3D reconstruction technology is widely used in intelligent manufacturing and industrial inspection due to its simple system structure, non-contact operation, and high accuracy. It projects multi-step phase-shifting sinusoidal fringes onto the surface of the object being measured using a projector, and captures the deformed fringes modulated by the object's height using a camera. Absolute phase is obtained through phase calculation and phase unwrapping, establishing a phase-height relationship. Alternatively, the projector can be considered an inverse camera, using a stereo vision model to obtain the object's 3D shape. However, due to the triangular relationship between the camera and projector, when measuring surfaces with drastic height changes, such as objects with protrusions or depressions, occlusion or shadow problems occur. This prevents 3D reconstruction of shadowed areas, resulting in a significant loss of effective data. Furthermore, shadowed areas affect subsequent phase and point cloud computing, impacting the quality of the reconstructed data. To address occlusion and shadow issues, some research has explored measuring the same object from multiple perspectives, using the results to complement each other. While this solves the occlusion problem to some extent, introducing additional equipment increases costs and makes it difficult to apply in confined measurement environments.

[0003] Another solution is to identify and extract shadow areas or invalid points to mitigate the phase error caused by shadow areas. For example, some studies have used Otsu's method to adaptively set the shadow threshold based on the histogram of the background image or modulated image by designing different weighting coefficients. However, after extracting the shadow areas, directly deleting the erroneous phase of the shadow areas will create holes in the reconstructed object surface and ignore the effective 3D information contained in the shadow areas. Therefore, it is necessary to fill and restore the shadow areas.

[0004] This invention proposes a structured light shadow region restoration and reconstruction method based on kernel density estimation. After performing modulation-based region segmentation based on the measurement results, the kernel density estimation method fully utilizes the correct phase information of the neighborhood to repair the phase of the invalid shadow region, so as to obtain complete three-dimensional topographic information of the object surface. Summary of the Invention

[0005] The purpose of this invention is to provide a structured light shadow region recovery and reconstruction method based on kernel density estimation, so as to solve the problem that traditional structured light systems based on triangulation leave invalid shadow regions with holes in the reconstruction results due to shadows and occlusion when measuring objects with drastic height changes.

[0006] The present invention provides a structured light shadow region recovery and reconstruction method based on kernel density estimation. The optical system involved includes: a digital projector that projects a sinusoidal fringe pattern of a specific frequency onto the surface of the object under test; and a camera that synchronously acquires the height-modulated deformed fringes of the object under test via hard triggering. The object under test, such as one with features like protrusions or depressions and drastic height variations, often contains invalid regions with voids in the measurement results due to occlusion and shadowing issues. The specific steps of the method for recovering and reconstructing invalid shadow regions in structured light measurements are as follows:

[0007] Step 1: Region segmentation based on modulation information; Project a unit frequency and a high-frequency sine pattern onto the surface of the object to be measured, and use a camera to synchronously acquire deformed fringes. Calculate the modulation of the fringes and combine it with the object's BK background image to perform region segmentation of the measurement results, i.e., the effective region with high modulation and containing phase information, and the invalid region with low modulation and caused by shadows and occlusion.

[0008] Step 2: Effective region phase calculation and demodulation; calculate the phase value according to the multi-step phase shift algorithm. Since the tangent operation phase is wrapped in... Therefore, a continuous phase distribution can be obtained by using a dual-frequency phase demodulation algorithm.

[0009] Step 3: Phase recovery of invalid regions based on neighborhood information: First, a reliability coefficient is calculated based on the number of valid pixels in the neighborhood of the invalid pixel to determine the recovery order. The larger the reliability coefficient, the earlier the recovery order. Recovery is performed sequentially from the outer layer of the invalid region to its center, and the reliability coefficient is updated after the invalid pixel is repaired. Second, recovery is performed based on kernel density estimation. That is, the invalid pixel and the surrounding valid context pixels are constructed into a target vector. All vectors with the same size and shape as this target vector are found in a certain valid region and combined to construct a probability kernel density function. The objective function is iteratively optimized to minimize the context pixel phase prediction error and the local phase recovery error, thereby achieving phase recovery of the invalid region.

[0010] Step 4: 3D shape reconstruction; Based on the system parameters such as the camera's intrinsic and extrinsic parameters and the projector's extrinsic parameters, the continuous phase distribution, including the recovered invalid region, is converted into 3D coordinates to achieve 3D shape reconstruction of objects with drastic height changes.

[0011] Furthermore:

[0012] Step 1 describes using region segmentation based on modulation information to divide the measurement results into valid and invalid regions. The specific process is as follows: First, calculate the object mask based on the acquired object BK image. If the mask value of the corresponding pixel is 0 and the modulation is lower than the threshold, it is considered to belong to the background region and is not considered. Calculate the stripe modulation through the acquired high-frequency multi-step phase-shift image. When the modulation is greater than the empirical threshold Th (i.e., the threshold determined based on experience), it is considered to belong to the valid region containing effective phase information. Otherwise, it belongs to the invalid region caused by shadows and occlusions and requires subsequent phase repair.

[0013] Step 3, which calculates the reliability coefficient based on the number of valid pixels in the neighborhood of an invalid pixel to determine the recovery order, involves the following process: Traversing the pixels in the invalid region, considering the neighborhood centered on the invalid pixel. The size of this neighborhood is determined by the size of the invalid region. For example, for smaller invalid regions, such as isolated holes with a diameter less than 10 pixels, a smaller neighborhood (e.g., 5×5) is used to avoid introducing irrelevant distant pixels that interfere with the local structure. For larger invalid regions (e.g., continuous missing regions caused by large-scale occlusion), a larger neighborhood (e.g., 9×9 or 11×11) is used to ensure sufficient valid pixel information is captured, maintaining the continuity of the repair. A reliability coefficient is calculated based on the number of valid pixels in the neighborhood; the larger the coefficient, the earlier the recovery order. Once an invalid pixel is recovered, it is marked as a valid pixel, and the reliability coefficient is automatically updated for the recovery estimation of the next invalid pixel, ultimately achieving complete recovery from the outer layer of the invalid region to its center.

[0014] For larger invalid regions (such as continuous missing regions caused by large-scale occlusion), a larger neighborhood (such as 9×9 or 11×11) should be used to ensure that enough effective pixel information can be captured and the continuity of the repair can be maintained.

[0015] In step 3, the recovery based on kernel density estimation is performed as follows: A target vector is formed by assembling a neighborhood of valid context pixels of a certain size defined above around the invalid pixel. The effective region is selected based on the characteristics of the object under test. For objects with obvious segmented geometric features, such as convex cylinders or concave structures, the corresponding structural region is selected as the effective range based on the geometric location of the invalid region. For objects with complex geometric structures, a local expansion method centered on the neighborhood block of the pixel to be repaired is used to determine the effective range. Within this effective region, a set of all observed vectors of the same size and shape as the target vector is searched. The probability density function of the target vector is estimated based on the set of vectors using kernel density estimation. Since it has the form of a Gaussian mixture model, the phase of the invalid pixel to be recovered can be expressed as a sum of two parts: the weighted value of the phase of pixels at the same position as the invalid pixel in the vector set, and the correction amount related to the bandwidth matrix calculated based on the valid context pixels. An objective function is constructed to minimize the context pixel phase prediction error and the local phase recovery error to determine the bandwidth parameters and prediction weight factors, ultimately achieving direct recovery estimation of the phase within the invalid region.

[0016] The main advantages of this invention are:

[0017] This method can effectively measure the 3D topography of objects with drastic height variations, overcoming the limitation of traditional structured light methods that are prone to data gaps in the measurement of such objects. By using a nonparametric probability density function estimation method based on kernel density estimation, it fully utilizes the effective phase information of the neighborhood to construct the objective function, and achieves robust phase recovery of the shadow region through iterative optimization, thereby improving the integrity of structured light 3D measurement and showing broad application prospects in the field of intelligent 3D measurement. Attached Figure Description

[0018] Figure 1 This is a block diagram for the structured light shadow region recovery and reconstruction based on kernel density estimation in this invention.

[0019] Figure 2 This is an illustration of the structured light shadow region recovery and reconstruction algorithm based on kernel density estimation of the present invention.

[0020] Figure 3 This is a diagram showing the reconstruction results of an object with a protruding structure according to the present invention.

[0021] Figure 4 This is a diagram showing the restoration and reconstruction results of an object with a concave structure according to the present invention.

[0022] Figure 5 This is a diagram showing the reconstruction result of a human portrait model with a complex structure, based on the present invention. Detailed Implementation

[0023] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0024] Figure 1 This is a block diagram for the structured light shadow region recovery and reconstruction based on kernel density estimation in this invention. The specific steps are as follows:

[0025] Step 1: Region segmentation based on modulation information. First, calculate the object mask using the object's background (BK) image. If the mask value of the corresponding pixel is 0, it is considered to belong to the background region and is not considered. Acquire an N-step phase-shifted sinusoidal fringe image that is modulated by the object's height. , It means equal to The phase shift, therefore the modulation intensity of the fringes. The following formula can be represented as:

[0026] (1)

[0027] By modulating the modulation intensity with a suitable threshold When comparing, If the obtained phase information is considered correct and valid, it is regarded as a valid region; when The phase value of the pixel was deemed unreliable and belonged to an invalid region, requiring further repair.

[0028] Step 2: Effective Region Phase Calculation and Demodulation. The phase value is calculated using a multi-step phase shift algorithm. It can be expressed as the following formula:

[0029] (2)

[0030] However, because the phase of the tangent operation is wrapped in... Therefore, a dual-frequency phase demodulation algorithm is needed to obtain a continuous phase distribution. It can be expressed as the following formula:

[0031] (3)

[0032] (4)

[0033] in, For stripe order, This is a high-frequency phase diagram. This is a low-frequency phase diagram per unit frequency. This is the ratio of their fringe frequencies.

[0034] Step 3: Phase recovery of invalid regions based on neighborhood information; For invalid pixels within the invalid region, the repair order must first be determined to ensure that recovery proceeds sequentially from the outer layer to the center, and that repaired invalid pixels can be used for the recovery process of the next adjacent invalid pixel. The invalid pixel and surrounding valid context pixels are used to construct a target vector. Within a certain region, a set of all vectors with the same size and shape as this target vector is found, and a probability density function is constructed. The objective function is iteratively optimized to minimize the context pixel phase prediction error and the local phase recovery error, thereby recovering the phase of the invalid region. Specifically, the invalid shadow region recovery process is as follows:

[0035] (1) Determine the recovery order based on the number of valid pixels in the neighborhood, traverse all pixels in the neighborhood of the current invalid pixel, and calculate the reliability coefficient. Represented as:

[0036] (5)

[0037] The calculation process is as follows: Consider the neighborhood range centered on the invalid pixel. For example, sizes like 3x3 or 5x5, when the pixels are valid pixels. When the pixel is an invalid pixel When the reliability coefficient The larger the value, the higher the repair priority. Furthermore, once a pixel is repaired and becomes a valid pixel, the reliability coefficient is dynamically adjusted. ,in As the attenuation factor, Its neighborhood Total number of pixels.

[0038] (2) The structured light shadow region restoration and reconstruction algorithm based on kernel density estimation in this invention, such as Figure 2 As shown, consider the pixel phase value to be recovered. its surrounding neighborhood The effective context pixel phase value is denoted as Combine them into a target vector Within a certain range (Including a sufficient effective pixel area), find the target vector The set of all observation vectors of the same size and shape Consider a given set of vectors. Target vector probability density function It can be represented as:

[0039] , (6)

[0040] (7)

[0041] in, The bandwidth matrix controls the smoothness of the probability density function; It is a multivariate Gaussian function. Because its probability density function has the form of a Gaussian mixture model, it can be used to recover the phase of invalid pixels. Express in the following forms:

[0042] (8)

[0043] (9)

[0044] (10)

[0045] The above formula can be considered as the result of the repair. Consider it as being related to the invalid pixel in the set of observation vectors. Pixel phase at the same position The weighted value with a weighting factor, and the phase of the surrounding valid context pixels. The calculated correction values ​​related to the bandwidth matrix. Among them, The prediction weighting factor is expressed as:

[0046] (11)

[0047] (3) As can be seen from the above expression, it is necessary to adjust the bandwidth matrix. Make an estimate to obtain the repair results Consider a full-bandwidth matrix that can capture the correlation between neighboring data samples. Since the matrix contains a large number of parameters, its estimation complexity is high; therefore, [the following is omitted as it is not relevant to the main text]. Simplified to ,in Scaling factor The structure matrix representing the correlation between the observed data can be the covariance matrix calculated from the observed vector dataset. As a structure matrix. Therefore, the bandwidth matrix. It can be decomposed into:

[0048] (12)

[0049] Considering the symmetry of the bandwidth matrix, Substituting the above formula into the phase... The recovery formula (8) is further expressed as:

[0050] (13)

[0051] in, And from the predicted weighting factors The expression is visible, only Unknown, that is, what needs to be done (simplify Therefore, the complex bandwidth estimation is transformed into an estimation of the parameters. and Perform optimization estimation.

[0052] (4) By minimizing the prediction error of the context pixel phase To the parameters Optimization is performed, and then the prediction weight factors are calculated. :

[0053] (14)

[0054] In obtaining After adding the corresponding prediction weight factors, the current invalid pixels to be recovered are... Effective context pixels within the neighborhood Centered on the set of all observed vectors Selecting from and European style closest Context pixels This constitutes a set of locally similar samples. Defined based on parameters Local phase recovery error:

[0055] (15)

[0056] in, For sample set Medium and to be estimated Known phases at the same location, and For based on The calculated weighted prediction value. (By...) Considered By taking the function and setting its derivative to 0, we can obtain the optimal solution in closed form, which minimizes the squared error between the predicted phase and the corresponding true phase on a locally similar sample set.

[0057] (5) The estimated parameters , With the calculated , Substituting these values ​​into the final phase recovery formula, we can achieve adaptive recovery of the phase value of the current invalid shadow region based on neighborhood information.

[0058] Step 4: 3D topography reconstruction; consider the 3D points to be reconstructed. At the same time, by camera pixels and projector pixels Capture. Establish the following relationship:

[0059]

[0060]

[0061] in, and These are the parameter matrices for the camera and projector, respectively. By using pre-calibrated system calibration parameters, the continuous phase distribution, including recovered invalid regions, can be converted into three-dimensional coordinates, enabling the reconstruction of the three-dimensional shape of objects with drastic height changes.

[0062] Specific implementation examples demonstrate the 3D topography measurement and repair of objects with protruding and concave structures. Due to occlusion caused by abrupt changes in height, invalid areas with missing data appear in the original measurement results. For example... Figure 3 As shown, the protruding structure is a cylinder with a diameter of 80mm and a height of 30mm; Figure 4 The image shows a recessed structure with a diameter of 80 mm and a depth of 20 mm. Using the method described in this invention to repair invalid regions, the 3D reconstruction results are complete, and the height information of the invalid regions can be recovered relatively well. It is particularly important to note that when repairing neighborhood information for objects with protrusions or recesses, a corresponding effective phase region must be selected as a reference based on the geometric location of the invalid region. To this end, masks for the upper and lower surfaces of the object (or the top of the protrusion and the bottom of the recess) can be pre-calculated to guide neighborhood selection, thereby ensuring that the repair process conforms to local geometric continuity and improving the fidelity of the reconstructed structure. When using the method of this invention, the region segmentation threshold based on modulation information is 10, and a 9×9 neighborhood is used to recover the invalid shadow region. Figure 5 This paper demonstrates the restoration and 3D reconstruction of a human portrait model with a complex structure. Due to the dramatic changes in surface height and rich geometric features of the portrait, the original measurement results contained several areas of missing data caused by occlusion. Using the method proposed in this invention, the point cloud data of the missing parts can be effectively recovered, while preserving complex geometric features such as facial contours.

[0063] The structured light shadow region restoration and reconstruction method based on kernel density estimation disclosed in the embodiments of the present invention has been described in detail above. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for restoring and reconstructing structured light shadow regions based on kernel density estimation, characterized in that, The optical system involved includes: a digital projector that projects a sinusoidal fringe pattern of a specific frequency onto the surface of the object under test; and a camera that synchronously acquires the height-modulated deformed fringes of the object under test via hard triggering. The objects under test include those with drastic height variations and features such as protrusions or depressions, where occlusion or shadowing issues result in invalid areas with voids in the measurement results. The specific steps are as follows: Step 1: Region segmentation based on modulation information; Project the unit frequency and high-frequency sine pattern onto the surface of the object to be measured, and use a camera to synchronously acquire deformed fringes. Calculate the modulation intensity of the fringes and combine it with the BK background image of the object to perform region segmentation of the measurement results, that is, the effective region with high modulation intensity and containing phase information, and the invalid region with low modulation intensity due to shadows and occlusion. Step 2: Effective region phase calculation and demodulation; calculate the phase value according to the multi-step phase shift algorithm. Since the tangent operation phase is wrapped in... Between these points, a continuous phase distribution is obtained using a dual-frequency phase demodulation algorithm; Step 3: Phase recovery of invalid regions based on neighborhood information: First, a reliability coefficient is calculated based on the number of valid pixels in the neighborhood of the invalid pixel to determine the recovery order. The larger the reliability coefficient, the earlier the recovery order. Recovery is performed sequentially from the outer layer of the invalid region to its center, and the reliability coefficient is updated after the invalid pixel is repaired. Second, recovery is performed based on kernel density estimation. That is, the invalid pixel and the surrounding valid context pixels are constructed into a target vector. All vectors with the same size and shape as this target vector are found within a certain valid region and combined to construct a probability kernel density function. The objective function is iteratively optimized to minimize the context pixel phase prediction error and the local phase recovery error, thereby achieving phase recovery of the invalid region. Step 4: 3D topography reconstruction; Based on the system parameters such as the camera's intrinsic and extrinsic parameters and the projector's extrinsic parameters, the continuous phase distribution, including the recovered invalid regions, is converted into 3D coordinates to achieve 3D topography reconstruction of objects with drastic height changes.

2. The method for restoring and reconstructing structured light shadow regions according to claim 1, characterized in that, The method described in step 1, which uses region segmentation based on modulation intensity information to divide the measurement results into valid and invalid regions, is as follows: First, calculate the object mask based on the acquired object BK image. If the mask value of the corresponding pixel is 0 and the modulation intensity is lower than the threshold, it is considered to belong to the background region and is not considered. The stripe modulation intensity is calculated by acquiring high-frequency multi-step phase-shift images. When the modulation intensity is greater than the empirical threshold Th, it is considered to be a valid region containing effective phase information. Otherwise, it is considered to be an invalid region caused by shadows and occlusions, and subsequent phase repair is required.

3. The method for restoring and reconstructing structured light shadow regions according to claim 2, characterized in that, Step 3 involves calculating a reliability coefficient based on the number of valid pixels in the neighborhood of an invalid pixel to determine the recovery order. The specific process is as follows: traverse the pixels in the invalid region, consider the neighborhood centered on the invalid pixel, and determine the size of the neighborhood based on the size of the invalid region to ensure that enough valid pixel information can be captured and the continuity of the repair is maintained; calculate the reliability coefficient based on the number of valid pixels in the neighborhood, with a larger coefficient indicating a higher recovery order; once the invalid pixel is recovered, mark it as a valid pixel and automatically update the reliability coefficient for the recovery estimation of the next invalid pixel, ultimately achieving complete recovery from the outer layer of the invalid region to its center.

4. The method for restoring and reconstructing structured light shadow regions according to claim 3, characterized in that, The recovery based on kernel density estimation described in step 3 is as follows: A target vector is formed by assembling the effective context pixels within the neighborhood of the invalid pixel. An effective region is selected based on the characteristics of the object to be measured. Within this effective region, a set of all observed vectors with the same size and shape as the target vector is found. Kernel density estimation is used to estimate the probability density function of the target vector based on the vector set. Since it has the form of a Gaussian mixture model, the phase of the invalid pixel to be recovered is expressed as a sum of two parts: a weighted value of the phase of pixels at the same position as the invalid pixel in the vector set, and a correction amount related to the bandwidth matrix calculated based on the effective context pixels. An objective function is constructed to minimize the context pixel phase prediction error and the local phase recovery error to determine the bandwidth parameters and prediction weight factors, ultimately achieving direct phase recovery estimation within the invalid region.

5. The method for restoring and reconstructing structured light shadow regions according to claim 4, characterized in that, The step 1, which involves calculating the fringe modulation intensity of the acquired high-frequency multi-step phase-shift image, specifically involves acquiring an N-step phase-shift sinusoidal fringe image that is deformed by the object's height modulation. , It means equal to The phase shift amount, and therefore, the modulation intensity of the fringes. The following formula is represented: （1） By modulating the intensity with a suitable threshold When comparing, If the obtained phase information is considered correct and valid, it is regarded as a valid region; when The phase value of the pixel was deemed unreliable and belonged to an invalid region, requiring further repair.

6. The method for restoring and reconstructing structured light shadow regions according to claim 5, characterized in that, The effective region phase calculation and demodulation described in step 2 specifically involves calculating the phase value using a multi-step phase shift algorithm. It can be expressed as the following formula: （2） Because the phase of the tangent operation is wrapped in Between these points, a dual-frequency phase demodulation algorithm is needed to obtain a continuous phase distribution. It can be expressed as the following formula: （3） （4） in, For stripe order, This is a high-frequency phase diagram. This is a low-frequency phase diagram per unit frequency. This is the ratio of their fringe frequencies.

7. The method for restoring and reconstructing structured light shadow regions according to claim 6, characterized in that, The phase recovery of invalid regions based on neighborhood information in step 3 is specifically performed as follows: (1) Determine the recovery order based on the number of valid pixels in the neighborhood, traverse all pixels in the neighborhood of the current invalid pixel, and calculate the reliability coefficient. Represented as: （5） The calculation process is as follows: Consider the neighborhood range centered on the invalid pixel. When the pixel is a valid pixel When the pixel is an invalid pixel ; Reliability coefficient The larger the value, the higher the repair priority; and once the pixel is repaired and becomes a valid pixel, the reliability coefficient is dynamically adjusted. ,in As the attenuation factor, Its neighborhood Total number of pixels; (2) Structured light shadow region restoration and reconstruction based on kernel density estimation, considering the pixel phase value to be restored. its surrounding neighborhood The effective context pixel phase value is denoted as Combine them into a target vector In a sufficient number of effective pixel areas Inside, find the target vector. The set of all observation vectors of the same size and shape Consider a given set of vectors. Target vector probability density function Represented as: ， （6） （7） in, The bandwidth matrix controls the smoothness of the probability density function; It is a multivariate Gaussian function; since its probability density function has the form of a Gaussian mixture model, the phase of the invalid pixel to be recovered is... Express in the following forms: （8） （9） （10） The above formula can be considered as the result of the repair. Consider it as being related to the invalid pixel in the set of observation vectors. Pixel phase at the same position The weighted value with a weighting factor, and the phase of the surrounding valid context pixels. The calculated correction quantities related to the bandwidth matrix; where, The prediction weighting factor is expressed as: （11） (3) For the bandwidth matrix Make an estimate to obtain the repair results Consider a full-bandwidth matrix that can capture the correlation between neighboring data samples; Simplified to ,in Scaling factor The structure matrix representing the correlation between the observed data is the covariance matrix calculated from the observed vector dataset. As the structure matrix, the bandwidth matrix is ​​thus formed. It can be decomposed into: （12） Considering the symmetry of the bandwidth matrix, Substituting the above formula into the phase... The recovery formula is further expressed as: （13） in, ; and from the prediction weighting factors The expression is visible, only Unknown, that is, what needs to be done (brief notes) To perform estimation, the complex bandwidth estimation is thus transformed into an estimation of parameters. and Perform optimization estimation; (4) By minimizing the prediction error of the context pixel phase To the parameters Optimization is performed, and then the prediction weight factors are calculated. : （14） In obtaining After adding the corresponding prediction weight factors, the current invalid pixels to be recovered are... Effective context pixels within the neighborhood Centered on the set of all observed vectors Selecting from and European style closest Context pixels This constitutes a set of locally similar samples. Defined based on parameters Local phase recovery error: （15） in, For sample set Medium and to be estimated Known phases at the same location, and For based on Calculated weighted predicted values; Considered By finding the function and setting its derivative to 0, we obtain the optimal solution in closed form, which minimizes the squared error between the predicted phase and the corresponding true phase on a locally similar sample set. (5) The estimated parameters , With the calculated , Substituting these values ​​into the final phase recovery formula, we can achieve adaptive recovery of the phase value of the current invalid shadow region based on neighborhood information.

8. The method for restoring and reconstructing structured light shadow regions according to claim 7, characterized in that, The three-dimensional topography reconstruction described in step 4 specifically considers the 3D points that need to be reconstructed. At the same time, by camera pixels and projector pixels Capture and establish the following relationship: ； ； in, and These are the parameter matrices for the camera and the projector, respectively. By using pre-calibrated system parameters, the continuous phase distribution, including the recovered invalid region, is converted into three-dimensional coordinates, enabling the reconstruction of the three-dimensional shape of objects with drastic height changes.

Images