A three-dimensional measurement method for non-Lambertian metallic surfaces

Abstract

This invention belongs to the field of three-dimensional shape measurement technology. It discloses a three-dimensional measurement method for non-Lambertian metal surfaces. The method involves projecting polarization-encoded structured light onto the non-Lambertian metal surface and simultaneously acquiring reflection images and ambient light noise data. Noise is removed from each reflection image, and the light intensity, polarization, and polarization angle of each pixel under different polarization states are extracted and calculated. A grayscale image is constructed to distinguish between high and low polarization points, and specular and diffuse reflection regions are marked. Based on the processed reflection images, the phase field is calculated and phase jumps are corrected. A dual-branch network model is input using the continuous phase field and polarization reflectivity features. The model is then fused and optimized, outputting control point coordinate matrices and weight matrices. A NURBS surface is constructed for defect detection, reconstructing a high-precision three-dimensional surface shape. This method systematically solves the problems of reflection interference, dynamic efficiency, and industrial adaptability in three-dimensional measurement of non-Lambertian metal surfaces, providing a solution for high-precision industrial inspection.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional shape measurement technology, and more specifically, to a three-dimensional measurement method for non-Lambertian metallic surfaces. Background Technology

[0002] Non-Lambertian metallic surfaces (such as polished aluminum and chrome-plated parts) present significant challenges in 3D topography measurement due to their complex reflective properties (including mixed specular and diffuse reflection, anisotropic textures, etc.). Traditional methods, such as structured light projection and laser scanning, are typically designed based on the Lambertian assumption and struggle to effectively handle overexposure, phase jumps, and data gaps on highly reflective surfaces. These methods perform poorly with real metallic surfaces because they fail to adequately account for the effects of anisotropic reflection, limiting the accuracy and reliability of the measurement results.

[0003] Existing 3D measurement techniques for non-Lambertian metals have significant limitations: traditional methods mainly rely on Lambertian bodies or simple specular reflection models based on single-modal data (such as grayscale stripes or single-polarization information), which cannot adapt to the anisotropic reflection characteristics of metal surfaces (such as the directionality of grinding textures), resulting in jump errors in phase calculation; the hardware systems are complex, for example, phase measurement deflection (PMD) requires multiple projection units and sub-millimeter-level calibration, which is difficult to deploy and hard to adapt to dynamic scenes (moving workpieces cause motion blur); the ambient light suppression capability is weak, and traditional methods (such as narrowband filters and multi-exposure HDR imaging) are inflexible and inefficient, with a high data loss rate in specular reflection areas; in addition, the output format is limited to redundant 3D point clouds, making it difficult to directly interface with CAD systems or support process optimization feedback, which seriously restricts industrial online inspection applications. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art and to achieve the above objectives, the present invention provides the following technical solution: a three-dimensional measurement method for non-Lambertian metallic surfaces, comprising:

[0005] By projecting polarization-coded structured light onto a non-Lambertian metal surface, polarization reflection images and ambient light noise data are simultaneously acquired to generate a multimodal dataset.

[0006] Based on a multimodal dataset, a fused background image is generated for each reflection image using a weighted average method, and ambient light noise is removed using pixel-by-pixel subtraction. For each polarization state of the reflection image, the light intensity values ​​of each pixel under different polarization states are extracted, and the total light intensity value, the first light intensity component, the second light intensity component, and the third light intensity component are calculated. Based on these light intensity components, the light polarization value and light polarization angle of each pixel are calculated, and a grayscale image is constructed using the light polarization value. A light polarization threshold is set to distinguish between high-polarization points and low-polarization points, and high-polarization and low-polarization regions are marked using connected component analysis. For each high-polarization region, specular reflection regions and diffuse reflection regions are determined by comparing the standard deviation of the light polarization angle.

[0007] Based on the specular reflection region and diffuse reflection region in the reflection image, the reflection image after specular reflection elimination is obtained; based on the reflection image after specular reflection elimination, the multi-channel phase is calculated to obtain the preliminary absolute phase field; based on the preliminary absolute phase field, the phase jump caused by anisotropic reflection is corrected to generate the corrected continuous phase field and polarization reflectivity matrix.

[0008] A dual-branch network model combining phase and polarization branches is constructed. The continuous phase field and polarization reflectivity features are input into the dual-branch network model, the phase and polarization information are fused, and the dual-branch network model is optimized by combining physical constraint loss and GAN loss. The output is the control point coordinate matrix and weight matrix of the NURBS surface, and then the NURBS surface is constructed.

[0009] Defect detection is performed on NURBS surfaces, surface roughness and radius of curvature are extracted, and then a high-precision three-dimensional surface shape of the non-Lambertian metal object under test is reconstructed.

[0010] Furthermore, the multimodal dataset is generated in the following ways:

[0011] Using a high-resolution projector with built-in polarization function, structured light patterns with different polarization states are projected onto a non-Lambertian metal surface according to a preset time sequence. The polarization states include horizontal linear polarization, vertical linear polarization, left-tilt linear polarization, right-tilt linear polarization, left-hand circular polarization, and right-hand circular polarization. The structured light patterns are composed of Gray code and phase-shifting fringes.

[0012] Whenever a structured light pattern is projected, a camera is synchronously triggered to capture the polarized reflection image of the non-Lambertian metal surface at the corresponding moment. The reflection images of different polarization states in each polarization channel are obtained through polarization channel sub-channel technology. The polarization channels include circular polarization and linear polarization. Based on the captured polarized reflection images, the phase-shifting method and Gray code decoding algorithm are used to parse and obtain the phase-encoded information.

[0013] Meanwhile, before and after each projection of structured light, images of the non-Lambertian metal surface in the rg frame without projection were captured under the same lighting conditions and used as background images.

[0014] All the obtained reflection and background images are denoised and contrast-enhanced. The processed reflection and background images, along with the phase coding information, are then integrated into a multimodal dataset.

[0015] Furthermore, the specular reflection region and the diffuse reflection region are obtained in the following ways:

[0016] Based on the multimodal dataset, for each reflection image, a weighted average method is applied to each pixel position in all corresponding background images to obtain the weighted average value of each pixel position, and the weighted average value of each pixel position is integrated into a fused background image.

[0017] The corresponding pixel value of the fused background image is subtracted from each pixel value of the reflected image using pixel-by-pixel subtraction to obtain the reflected image after removing ambient light noise.

[0018] Based on the reflection images of each polarization channel after removing ambient light noise, all reflection images under each polarization state are taken as a set of reflection images;

[0019] For each set of reflection images, for each pixel at each pixel location, the light intensity value of the corresponding pixel in the horizontal linear polarization state, vertical linear polarization state, left-tilted linear polarization state, and right-tilted linear polarization state is extracted from the reflection images of different deflection states, as well as the light intensity value of each pixel in the right-hand circular polarization state and left-hand circular polarization state.

[0020] In the same set of reflection images, for each pixel, the average value of the light intensity under all polarization states is taken as the total light intensity value, the difference between the light intensity values ​​of the horizontal linear polarization state and the vertical linear polarization state is taken as the first light intensity component, the difference between the light intensity values ​​of the left-tilted linear polarization state and the right-tilted linear polarization state is taken as the second light intensity component, and the difference between the light intensity values ​​of the right-hand circular polarization state and the left-hand circular polarization state is taken as the third light intensity component.

[0021] Calculate the sum of squares of the first light intensity component, the second light intensity component, and the third light intensity component, take the square root of the sum of squares, and calculate the ratio of the result to the total light intensity value to obtain the light deviation value of each pixel.

[0022] Calculate the ratio of the first light intensity component to the second light intensity component, and calculate the arctangent of the ratio. Take half of the arctangent as the light deflection angle of each pixel.

[0023] Based on the light bias value of each pixel, a grayscale image is constructed using the light bias value as the grayscale value.

[0024] Based on the grayscale image, a light polarization threshold is set according to the light polarization value of each pixel. Pixels with light polarization values ​​greater than the light polarization threshold are recorded as high polarization points, and pixels with light polarization values ​​less than the light polarization threshold are recorded as low polarization points.

[0025] Based on high-skewness points and low-skewness points, connected component analysis is used to identify and label connected high-skewness points, forming different connected components, which are labeled as high-skewness regions. The remaining non-high-skewness regions are labeled as low-skewness regions.

[0026] For each high-bias region, take the standard deviation of the optical deflection angle of all pixels in the high-bias region and set the optical deflection angle threshold.

[0027] If the standard deviation of the optical polarization angle in the high polarization region is less than the optical polarization angle threshold, the high polarization region is determined to be a specular reflection region.

[0028] If the standard deviation of the light polarization angle in the high polarization region is greater than or equal to the light polarization angle threshold, it is determined to be a diffuse reflection region.

[0029] Furthermore, the method for obtaining the reflected image after specular reflection elimination includes:

[0030] Based on the specular reflection region of the grayscale image, the opening operation algorithm is used to remove noise and isolated points, and the closing operation algorithm is used to fill the holes in the specular reflection region after the opening operation.

[0031] The specular reflection region after the closing operation is subjected to dilation and erosion operations respectively. The dilated specular reflection region is subtracted from the eroded specular reflection region to obtain the specular reflection region with enhanced edge contour.

[0032] Based on the edge-enhanced specular reflection region, skeleton extraction technology is used to refine the boundary of the segmented specular reflection region, and morphological reconstruction technology is used to fill and repair the specular reflection region to obtain a complete specular reflection region.

[0033] Based on the complete specular reflection area within the grayscale image, create a binary mask, and mark the specular reflection area as 0 and other areas as 1 on the binary mask;

[0034] For each set of reflection images, the labeled binary mask is multiplied with all reflection images respectively to retain the diffuse reflection area and remove the specular reflection area, resulting in a set of reflection images after specular reflection elimination.

[0035] Furthermore, the preliminary absolute phase field is obtained in the following ways:

[0036] Based on the set of reflection images after specular reflection elimination, Fourier transform is performed on the reflection images of horizontal linear polarization, vertical linear polarization, +45 degree linear polarization, -45 degree linear polarization, right-hand circular polarization, and left-hand circular polarization to extract the fundamental frequency component and filter it. Then, the initial phase estimate of each polarization state is obtained through inverse Fourier transform.

[0037] Based on the initial phase estimate of each polarization state, the initial phase estimates of all polarization states are arranged into an initial phase estimate sequence;

[0038] Based on the initial phase estimate of any polarization state in the initial phase estimate sequence, calculate the sum of the initial phase estimate differences between all adjacent polarization states.

[0039] The weight of each initial phase estimate is determined based on the sum of the initial phase estimate differences between adjacent polarization states.

[0040] Based on the initial phase estimate of each polarization state and the corresponding weight, the preliminary absolute phase field is calculated using the weighted average method.

[0041] Furthermore, the generation methods of the continuous phase field and polarization reflectivity matrix include:

[0042] A lightweight residual network is used as the basic network structure. The absolute phase field and the light polarization value and light polarization angle of each pixel are taken as input. The loss function of the residual network is defined as the sum of the gradient smoothing term and the true phase supervision term. The residual network is trained using the defined loss function, and the output is the phase jump correction amount, which represents the amount of phase change that needs to be adjusted at each pixel.

[0043] The gradient smoothing term is defined as the L2 norm of the gradient difference between the predicted phase field and the input absolute phase field; the true phase supervision term is defined as the L1 norm error between the predicted phase field and the true label corresponding to the training sample.

[0044] Based on the output phase jump residual, the sum of the phase jump residual and the absolute phase field is taken to obtain the corrected continuous phase field;

[0045] Based on the light polarization value and light polarization angle of each pixel obtained from the reflection images under different polarization states, the light polarization value of each pixel is normalized.

[0046] The polarization reflectance of each pixel is calculated by multiplying the normalized optical polarization value with the cosine of the optical polarization angle. The polarization reflectance of all pixels is then integrated to construct a polarization reflectance matrix.

[0047] Furthermore, the construction method of the dual-branch network model combining phase branch and polarization branch includes:

[0048] A dual-branch network architecture is constructed, including an input layer and feature extraction, a cross-membrane attention mechanism, and feature fusion and decoding;

[0049] The input layer is defined as a dual-branch independent input, with the dual branches being a phase branch and a polarization branch. The input of the phase branch is a continuous phase field. Multi-scale convolution is used to extract the relevant features of the continuous phase field and output the feature map of the continuous phase field.

[0050] The polarization branch takes the polarization reflectance matrix as input, uses lightweight convolution to extract the features of polarization reflectance, and outputs a feature map of polarization reflectance.

[0051] Based on the feature map of the continuous phase field and the feature map of polarization reflectivity, the feature maps of the two branches are fused using a cross-attention mechanism. The feature map of the phase branch is used as the query, and the feature map of the polarization branch is used as the key and value. The attention weight is calculated using the attention weight formula of the cross-attention mechanism to obtain the attention-enhanced feature map.

[0052] Among them, Query, Key, and Value represent the core components for calculating attention weights in the cross-membrane attention mechanism;

[0053] The attention-enhanced feature map and the phase branch feature map are concatenated along the channel dimension of the multi-scale convolution to obtain a fused feature map. Then, the decoder is used to decode the fused feature map and output the control point coordinate matrix and weight matrix of the NURBS surface.

[0054] The decoder is constructed by combining upsampling and convolution. First, upsampling is used to increase the size of the fused feature map, and then convolutional layers are applied to refine the details of the fused feature map.

[0055] Furthermore, the training method of the dual-branch network model includes:

[0056] Collect training datasets, including continuous phase field and polarization reflectivity matrix, and corresponding ground truth labels;

[0057] The training dataset is divided into a training set and a validation set according to a certain ratio;

[0058] The loss function of a two-branch network is defined as the joint loss function, which is composed of physical constraint loss and GAN loss.

[0059] Physical constraint losses include phase smoothness loss, control point coordinate loss, and weight loss;

[0060] The phase smoothness loss is defined as follows: For each pixel, calculate the gradient of the pixel in the horizontal and vertical directions, take the absolute value of all gradients and add them together, and then take the sum of the sums of all pixels to obtain the total phase smoothness loss.

[0061] Control point loss is defined as follows: calculate the difference between the predicted control point coordinate matrix and the true control point coordinate matrix, and use the L2 norm to calculate the sum of these differences to obtain the value of control point coordinate loss;

[0062] The weight loss is defined as follows: calculate the difference between the predicted control point weight matrix and the true control point weight matrix, and use the L2 norm to calculate the sum of these differences to obtain the value of the weight loss;

[0063] The parameters of the two-branch network are initialized using a random initialization method. The training set is then input into the two-branch network for forward propagation to obtain the control point coordinate matrix and weight matrix of the NURBS surface. After each forward propagation of the model, the phase smoothness loss, control point coordinate loss, and weight loss are calculated based on the predicted control point coordinate matrix and weight matrix of the NURBS surface to obtain the values ​​of the three losses.

[0064] The discriminator in the GAN loss is used to score the values ​​of the three losses with the real labels, and then the GAN loss value is calculated using the GAN loss.

[0065] The joint loss function is obtained by weighted summing of the values ​​of phase smoothness loss, control point loss, weight loss, and GAN loss.

[0066] Based on the joint loss function, the gradient of each network parameter is calculated using the backpropagation algorithm, and the network parameters are updated using the Adam optimization algorithm to minimize the loss function;

[0067] For each iteration, AUC is used as the evaluation metric, and the AUC value is calculated on the validation set.

[0068] Based on the AUC values ​​on the validation set, calculate the difference between the AUC value after the current iteration and the AUC value after the previous iteration, and denot it as the iteration difference;

[0069] Set an iteration difference threshold. If the iteration difference is greater than the iteration difference threshold, it is determined that the model's performance has improved.

[0070] If the iteration difference is less than or equal to the iteration difference threshold, it is determined that the model's performance has not improved.

[0071] If the model's performance on the validation set does not improve in consecutive DC iterations, training is stopped, and a trained dual-branch network model is obtained.

[0072] Furthermore, the NURBS surface is constructed in the following ways:

[0073] Based on the control point coordinate matrix and weight matrix output by the dual-branch network model, the order of the two directions on the NURBS surface is set, and the number of control points and the corresponding uniformly distributed node vectors of the NURBS surface are set. The control point coordinate matrix and weight matrix are merged into a four-dimensional vector, and then a parameterized NURBS surface is constructed using NURBS tools.

[0074] Furthermore, the high-precision three-dimensional surface reconstruction method for the measured non-Lambertian metal object includes:

[0075] Based on the generated NURBS surface, the maximum and minimum principal curvature of each point on the NURBS surface are calculated using differential geometry. A high curvature threshold is set. If the maximum curvature of any point on the NURBS surface is greater than the high curvature, the point is determined to have a defect and is marked as an abnormal area on the NURBS surface.

[0076] Take the average height of all points on the NURBS surface as the average plane, and calculate the root mean square value of the height deviation of all points on the NURBS surface relative to the average plane as the surface roughness.

[0077] Based on the maximum and minimum principal curvature of each point on the NURBS surface, the radius of curvature of each point in both directions on the NURBS surface is calculated using the radius of curvature conversion formula.

[0078] Based on the radius of curvature of each point on the NURBS surface, as well as the abnormal regions and surface roughness, a smoothing algorithm is applied to improve the NURBS surface and reconstruct the three-dimensional surface shape, resulting in a high-precision three-dimensional surface shape of the non-Lambertian metal object being measured.

[0079] The technical effects and advantages of the three-dimensional measurement method for non-Lambertian metallic surfaces of this invention are as follows:

[0080] This invention projects structured light patterns with different polarization states onto a non-Lambertian metal surface and simultaneously collects reflection images and ambient light noise data. This comprehensively captures complex reflection characteristics (including specular and diffuse reflection) and the influence of ambient light noise, providing accurate basic data for subsequent processing. Secondly, a weighted average method is used to generate a fused background image and remove ambient light noise, improving the quality of the reflection image. Subsequently, based on the light intensity values ​​of each pixel under different polarization states, the light polarization value and angle are calculated to distinguish between specular and diffuse reflection areas, thereby reducing phase jump problems and improving measurement accuracy. Finally, processing is applied to the specular reflection area to ensure the authenticity and reliability of the reflection image, and multi-channel phase is calculated to obtain a continuous phase field, solving the phase jump problem and ensuring the continuity of the phase field. The method provides reliable data support for 3D topography reconstruction by combining phase information and polarization reflectivity features. Then, a dual-branch network model is constructed by combining phase information and polarization reflectivity features, and a joint optimization model using physical constraint loss and GAN loss is applied to output the control point coordinate matrix and weight matrix of the NURBS surface. This improves the robustness and generalization ability of the model, enhancing the accuracy of 3D topography reconstruction. Finally, a NURBS surface is constructed based on the control point coordinate matrix and weight matrix to perform defect detection and extract surface roughness and radius of curvature, thereby constructing a high-precision 3D surface. This achieves efficient and accurate measurement of the 3D topography of non-Lambertian metal surfaces. This method not only improves measurement accuracy and reliability but also has wide applicability and flexibility, effectively addressing complex industrial inspection needs and ensuring system stability and reliability. Attached Figure Description

[0081] Figure 1 This is a schematic diagram of a three-dimensional measurement method for a non-Lambertian metallic surface according to the present invention;

[0082] Figure 2 This is a schematic diagram of a three-dimensional measurement system for a non-Lambertian metallic surface according to the present invention;

[0083] Figure 3 This is a structural diagram of a three-dimensional measurement method for non-Lambertian metallic surfaces according to the present invention. Detailed Implementation

[0084] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0085] Example 1

[0086] Please see Figure 1 and Figure 3 As shown in this embodiment, a three-dimensional measurement method for non-Lambertian metallic surfaces includes:

[0087] By projecting polarization-coded structured light onto a non-Lambertian metal surface, polarization reflection images and ambient light noise data are simultaneously acquired to generate a multimodal dataset.

[0088] Based on a multimodal dataset, a fused background image is generated for each reflection image using a weighted average method, and ambient light noise is removed using pixel-by-pixel subtraction. For each polarization state of the reflection image, the light intensity value of each pixel under different polarization states is extracted, and the total light intensity value, first, second, and third light intensity components are calculated. The light polarization value and light polarization angle of each pixel are calculated based on these light intensity components, and a grayscale image is constructed using the light polarization value. A light polarization threshold is set to distinguish between high-polarization points and low-polarization points, and high-polarization and low-polarization regions are marked using connected component analysis. For each high-polarization region, specular reflection regions and diffuse reflection regions are determined by comparing the standard deviation of the light polarization angle.

[0089] Based on the specular reflection region and diffuse reflection region in the reflection image, the reflection image after specular reflection elimination is obtained; based on the reflection image after specular reflection elimination, the multi-channel phase is calculated to obtain the preliminary absolute phase field; based on the preliminary absolute phase field, the phase jump caused by anisotropic reflection is corrected to generate the corrected continuous phase field and polarization reflectivity matrix.

[0090] A dual-branch network model combining phase and polarization branches is constructed. The continuous phase field and polarization reflectivity features are input into the dual-branch network model, the phase and polarization information are fused, and the dual-branch network model is optimized by combining physical constraint loss and GAN loss. The output is the control point coordinate matrix and weight matrix of the NURBS surface, and then the NURBS surface is constructed.

[0091] Defect detection is performed on NURBS surfaces, surface roughness and radius of curvature are extracted, and then a high-precision three-dimensional surface shape of the non-Lambertian metal object under test is reconstructed.

[0092] Methods for generating multimodal datasets include:

[0093] Using a high-resolution projector with built-in polarization function, structured light patterns with different polarization states are projected onto a non-Lambertian metal surface according to a preset time sequence (such as dividing time into different moments according to a set time interval and projecting structured light once at each moment). (Through this encoding method, the three-dimensional information and reflection characteristics of the target surface can be effectively captured). The polarization states include horizontal (0 degrees), vertical (90 degrees), left tilt (e.g., +45 degrees), right tilt (e.g., -45 degrees) linear polarization, as well as left-handed and right-handed circular polarization. The structured light pattern is composed of Gray code and phase-shifted fringes.

[0094] Whenever a structured light pattern is projected, a camera is synchronously triggered to capture the polarized reflection image of the non-Lambertian metal surface at the corresponding moment. Polarization channel separation technology is used to obtain reflection images of different polarization states in each polarization channel. The polarization channels include circular polarization and linear polarization (when using linear polarization, when light strikes the surface at a certain angle, the specular reflection portion usually maintains its original polarization direction, while the diffuse reflection component causes a change in polarization direction; using circularly polarized light can further help distinguish different reflection types, because specular reflection usually maintains a circular polarization state, while diffuse reflection may lead to a transition from circular polarization to linear polarization or complete depolarization). Based on the captured polarized reflection images, phase-shifting and Gray code decoding algorithms are used to obtain phase-encoded information (specifically, by analyzing the deformation of the structured light pattern on the surface, the actual phase value of each pixel is calculated, thereby obtaining the three-dimensional morphology information of the object's surface. These polarized reflection images contain important information about the surface geometry and material properties, i.e., the so-called phase-encoded information, which originates from the deformation of the structured light pattern on the surface).

[0095] Meanwhile, before and after each projection of structured light, images of the non-Lambertian metal surface in the unprojected state of the rg frame are captured under the same lighting conditions as background images (i.e., background images, which are captured without active light source interference and can reflect various light sources in the environment (such as workshop lights and natural light), are used as references for subsequent removal of ambient light noise. These background images can help establish a dynamic background noise model for real-time reduction of ambient light interference. Specifically, for each frame of actual measured reflection image, the light intensity value in the corresponding background image is subtracted from the image to eliminate or reduce the interference of ambient light on the measurement results).

[0096] All the obtained reflection and background images are denoised and contrast enhanced. The processed reflection and background images, along with the phase coding information, are then integrated into a multimodal dataset.

[0097] For example, integrating into a multimodal dataset involves defining a unified data structure or framework to store information from different modalities. For instance, a three-dimensional array or tensor can be used, where each dimension represents a different type of information (such as spatial location, polarization state, time series, etc.).

[0098] The processed reflection images are stored in data frames according to their corresponding polarization states (circular polarization and linear polarization) to ensure that each image has a clear location representation, which facilitates subsequent access and processing.

[0099] The phase encoding information obtained from the analysis is also integrated into this data framework. Typically, the phase information corresponds to the spatial coordinates of the reflected image, so it can be directly mapped to the same spatial location.

[0100] Add necessary metadata to the entire dataset, including but not limited to shooting time, camera and projector parameter settings, experimental conditions, etc. This metadata helps to better understand and interpret the results during subsequent data analysis.

[0101] The methods for obtaining specular reflection and diffuse reflection areas include:

[0102] Based on the multimodal dataset, for each reflection image, a weighted average method is applied to each pixel position in all corresponding background images to obtain the weighted average value of each pixel position (that is, for all corresponding background images before and after, the pixel points of each pixel position in all background images are matched one by one, the pixel value of each pixel position in different background images is taken, and the weighted average value of each pixel position is calculated using the weighted average method). The weighted average value of each pixel position is then integrated into a fused background image.

[0103] The corresponding pixel value of the fused background image is subtracted from each pixel value of the reflected image using pixel-by-pixel subtraction to obtain the reflected image after removing ambient light noise.

[0104] Based on the reflection images of each polarization channel after removing ambient light noise, all reflection images under each polarization state are taken as a set of reflection images;

[0105] For each set of reflection images, for each pixel at each pixel location, the light intensity value of the corresponding pixel in the horizontal, vertical, left-tilted and right-tilted linear polarization states is extracted from the reflection images of different deflection states, as well as the light intensity value of each pixel in the right-hand circular polarization state and the left-hand circular polarization state.

[0106] In the same set of reflection images, for each pixel, the average value of the light intensity under all polarization states is taken as the total light intensity value, the difference between the light intensity values ​​of the horizontal linear polarization state and the vertical linear polarization state is taken as the first light intensity component, the difference between the light intensity values ​​of the left-tilted linear polarization state and the right-tilted linear polarization state is taken as the second light intensity component, and the difference between the light intensity values ​​of the right-hand circular polarization state and the left-hand circular polarization state is taken as the third light intensity component.

[0107] Calculate the sum of squares of the first, second, and third light intensity components, take the square root of the sum, and then calculate the ratio of the result to the total light intensity value to obtain the light offset value for each pixel. ,in, This represents the total light intensity value. Represents the first light intensity component. This represents the second light intensity component. Indicates the third light intensity component;

[0108] Calculate the ratio of the first light intensity component to the second light intensity component, and then calculate the arctangent of this ratio. Take half of the arctangent as the optical deflection angle for each pixel. ;

[0109] Based on the light bias value of each pixel, a grayscale image is constructed using the light bias value as the grayscale value.

[0110] Based on the grayscale image, a light deviation threshold (e.g., 0.5, the specific value of which is set by relevant personnel or experts in the industry based on experience) is set according to the light deviation value of each pixel. Pixels with a light deviation value greater than the light deviation threshold are recorded as high deviation points, and pixels with a light deviation value less than the light deviation threshold are recorded as low deviation points.

[0111] Based on high-skewness points and low-skewness points, connected component analysis is used to identify and label connected high-skewness points, forming different connected components, which are labeled as high-skewness regions. The remaining non-high-skewness regions are labeled as low-skewness regions.

[0112] For each high-bias region, the standard deviation of the optical polarization angle of all pixels within the high-bias region is taken, and an optical polarization angle threshold is set (e.g., 10 degrees, the specific value of which is set by relevant industry personnel or experts based on experience). If the standard deviation of the optical polarization angle within the high-bias region is less than the optical polarization angle threshold, the high-bias region is determined to be a specular reflection region (because specular reflection often maintains the polarization state of the incident light, the optical polarization value of the specular reflection region is higher, and under the condition of a fixed light source and viewing angle, the polarization direction of the specular reflection part will tend to be consistent, because they directly reflect the polarization characteristics of the incident light, and the resulting situation is that the angle of light deflection will be more regular, that is, the standard deviation of the optical polarization angle is smaller); if the standard deviation of the optical polarization angle within the high-bias region is greater than or equal to the optical polarization angle threshold, it is determined to be a diffuse reflection region (because diffuse reflection will cause the polarization direction to become irregular, and the resulting situation is that the angle of light deflection will be more dispersed, that is, the deviation of the optical polarization angle is larger).

[0113] Methods for obtaining the reflected image after specular reflection removal include:

[0114] Based on the specular reflection region of the grayscale image, the opening operation algorithm is used to remove noise and isolated points (the purpose is to remove small isolated points or noise points to prevent misjudgment as a specular reflection region), and the closing operation algorithm is used to fill the holes in the specular reflection region after the opening operation (the purpose is to fill the holes in the opening operation to ensure the continuity of the target region).

[0115] The specular reflection region after the closing operation is subjected to dilation and erosion operations respectively. The dilated specular reflection region is subtracted from the eroded specular reflection region to obtain the specular reflection region with enhanced edge contour.

[0116] Based on the edge-enhanced specular reflection region, skeleton extraction technology is used to refine the boundary of the segmented specular reflection region, and morphological reconstruction technology is used to fill and repair the specular reflection region to obtain a complete specular reflection region.

[0117] Based on the complete specular reflection area within the grayscale image, create a binary mask, and mark the specular reflection area as 0 and other areas as 1 on the binary mask;

[0118] For each set of reflection images, the labeled binary mask is multiplied with all reflection images respectively to retain the diffuse reflection area and remove the specular reflection area, resulting in a set of reflection images after specular reflection removal (directly multiplying the mask with the original area will turn the specular reflection area black, while the diffuse reflection area remains unchanged, and the resulting image retains both the diffuse reflection area and removes the specular reflection area).

[0119] The preliminary methods for obtaining the absolute phase field include:

[0120] Based on the set of reflection images after specular reflection elimination, Fourier transform is performed on the reflection images of horizontal, vertical, +45° and -45° linear polarization states, as well as right-hand circular polarization and left-hand circular polarization states, to extract the fundamental frequency component and perform filtering. Then, the initial phase estimate of each polarization state is obtained through inverse Fourier transform.

[0121] Based on the initial phase estimate for each polarization state, a weight is set for each initial phase estimate. ;

[0122] in, Indicates the first Initial phase estimation weights for each polarization state, This represents the sum of the initial phase estimates between adjacent polarization states; this part indicates the summation portion, which calculates the difference between the initial phase estimates and the values ​​of the adjacent polarization states. The sum of phase differences between adjacent polarization state pairs directly related to each polarization state; specifically, for the first polarization state... Only consider For the last polarization state Only consider For the intermediate polarization state ,consider and The result obtained by summing the parts is used as the denominator to calculate the first part. The weights of the polarization states are determined by the phase difference. Larger phase differences imply more inconsistencies and noise, therefore these regions should be assigned smaller weights. Conversely, smaller phase differences represent more stable phase information and should be assigned larger weights. u represents the index variable used to iterate over the polarization state. A pair of polarization states related to a polarization state. This indicates the end index of the summation. For the last polarization state, there is no subsequent state, so it ends at... End; for other states, proceed to... Finish, This indicates the starting index for determining the summation, for the first polarization state. It does not have a previous polarization state, so from To begin, for other states, start from start, Indicates the number of polarization states. (That is, four linear polarization states and two circular polarization states). Indicates the first and the The difference in initial phase estimation between polarization states is defined as follows: , Indicates the first Initial phase estimation for each polarization state, Represents a non-zero constant, for example This is used to prevent errors caused by the denominator being zero in the formula;

[0123] By basing the phase difference between each polarization channel and its immediate neighboring channels, a more reasonable weight allocation can be achieved, thereby effectively distinguishing specular reflection and diffuse reflection regions and improving the quality of phase solution.

[0124] Based on the initial phase estimate and corresponding weight of each polarization state, a preliminary absolute phase field is calculated using the weighted average method. By utilizing the differences in reflection characteristics of different polarization channels, the phase information of each channel is fused by weighting to compensate for the problem of insufficient data in a single channel. Furthermore, by weighting and averaging the phase field of each polarization state, it can also be used to suppress noise in the specular reflection region.

[0125] The methods for generating the continuous phase field and polarization reflectivity matrix include:

[0126] After joint phase calculation through multiple polarization states (i.e., multi-channel joint), a preliminary absolute phase field is obtained. However, due to various factors in actual application scenarios (such as surface texture, noise, etc.), this phase field may have discontinuities or jumps. Therefore, further phase correction is required to obtain smoother and more accurate results.

[0127] A lightweight residual network is used as the basic network structure. The absolute phase field, the optical bias value and the optical bias angle of each pixel are taken as inputs. The loss function of the residual network is defined as the sum of the gradient smoothing term and the true phase supervision term (the purpose of the loss function is to ensure that the predicted phase field conforms to the physical constraints and is close to the true value). The residual network is trained using the defined loss function, and the output is the phase jump correction amount, which represents the amount of phase change that needs to be adjusted at each pixel.

[0128] The gradient smoothing term is defined as the L2 norm of the gradient difference between the predicted phase field and the input absolute phase field; the true phase supervision term is defined as the L1 norm error between the predicted phase field and the true label corresponding to the training sample.

[0129] That is, the loss function is defined as:

[0130] ,in, This represents the predicted phase field. Represents the absolute phase field. This indicates the true phase field referred to by the true label in the training samples. and This represents the influence factor used to balance the smoothness and absolute accuracy of different loss terms (e.g.) , (The specific values ​​are set by relevant industry personnel and experts based on experience). This represents the gradient smoothing term. This part measures the similarity between the predicted phase field and the input fused phase field by comparing the gradient difference between the two. The role of this part is to ensure that the predicted phase field is as consistent as possible with the original fused phase field in its gradient direction, thereby maintaining the overall trend of the phase field and suppressing local noise and discontinuities.

[0131] This represents the true phase supervision term, which directly measures the difference between the predicted phase field and the true phase in the calibration sample. The purpose of this part is to make the prediction result as close as possible to the actual measured or known true phase value, so as to improve the accuracy of the prediction.

[0132] The entire loss function combines gradient smoothing and true phase supervision terms to ensure the physical rationality of the predicted phase field (such as smoothness and consistency) and to make it as close as possible to the real situation. The choice of weight coefficients determines the relative importance of these two factors in the optimization process. This design helps the model learn more accurate and physically reasonable phase information.

[0133] Based on the output phase jump residual, the sum of the phase jump residual and the absolute phase field is taken to obtain the corrected continuous phase field (the continuous phase field is a 2D matrix representing the absolute phase value of each pixel. The corrected continuous phase field is obtained by directly correcting the phase jump region, such as the phase jump caused by the metal brush texture).

[0134] Based on the light polarization value and light polarization angle of each pixel obtained from the reflection images under different polarization states, the light polarization value of each pixel is normalized.

[0135] The polarization reflectance of each pixel is calculated by multiplying the normalized optical polarization value with the cosine of the optical polarization angle. The polarization reflectance of all pixels is then integrated to construct a polarization reflectance matrix (a 2D matrix containing optical polarization value and optical polarization angle information).

[0136] By fusing linear and circular polarization phase fields, multi-channel phase complementarity effectively suppresses residual errors from specular reflection. By combining a residual network with a defined physically constrained loss function, the phase jump problem caused by processing textures is solved. By suppressing residual specular reflection and anisotropic interference, high-precision phase calculation is achieved. Furthermore, by combining polarization reflectivity, physical-data co-optimization is realized.

[0137] The construction methods for a dual-branch network model combining phase branching and polarization branching include:

[0138] A dual-branch network architecture is constructed, including an input layer and feature extraction, a cross-membrane attention mechanism, and feature fusion and decoding;

[0139] The input layer is defined as a dual-branch independent input, with the dual branches being a phase branch and a polarization branch. The phase branch input is a continuous phase field. Multi-scale convolution (such as the ResNet module) is used to extract relevant features of the continuous phase field (including local gradient features and global continuity features), and the output is a feature map of the continuous phase field.

[0140] The polarization branch takes the polarization reflectance matrix as input, uses lightweight convolution (such as MobileNet modules) to extract features of polarization reflectance (including anisotropic features, such as texture directionality), and outputs a feature map of polarization reflectance.

[0141] Based on the feature maps of the continuous phase field and polarization reflectance, a cross-attention mechanism is used to fuse the feature maps of the two branches. The feature map of the phase branch is used as the query, and the feature map of the polarization branch is used as the key and value. The attention weight is calculated using the attention weight formula of the cross-attention mechanism to obtain the attention-enhanced feature map, which enhances the sensitivity of the phase field to anisotropic regions. Here, query, key, and value represent the core part of calculating the attention weight in the cross-membrane attention mechanism.

[0142] Query: Here, Query is a feature vector or feature map extracted from a branch (such as a phase branch), and its role is to act as a "query" to find other related information;

[0143] Key: Key is a feature vector or feature map extracted from another branch (such as a polarization branch). They are used to respond to the "query" from the Query, that is, they help determine which parts of the information are most relevant to the Query.

[0144] Value: Value is also extracted from another branch (usually the branch that provides the Key, i.e., the polarization branch). Value contains the actual information content. Once the relevance is determined through Query and Key, Value will be used to generate the final output feature map.

[0145] The attention-enhanced feature map and the phase branch feature map are concatenated along the channel dimension of the multi-scale convolution to obtain a fused feature map. The decoder is then used to decode the fused feature map and output the control point coordinate matrix and weight matrix of the NURBS surface. The decoder is constructed by upsampling combined with convolution. First, upsampling is used to increase the size of the fused feature map, and then convolutional layers are applied to refine the details of the fused feature map.

[0146] Training methods for dual-branch network models include:

[0147] Collect training samples, including continuous phase field and polarization reflectivity matrix, and corresponding real labels;

[0148] The training samples are divided into a training set and a validation set according to a certain ratio;

[0149] The loss function of the dual-branch network is defined as the joint loss function, which is obtained by weighted summation of the values ​​of phase smoothness loss, control point coordinate loss, weight loss, and GAN loss.

[0150] The phase smoothness loss is defined as follows: For each pixel, calculate the gradient of the pixel in the horizontal and vertical directions, take the absolute value of all gradients and add them together, and then take the sum of the sums of all pixels to obtain the total phase smoothness loss.

[0151] Right now ,in, This represents the phase smoothness loss value. Indicates the position of a pixel. Represents the phase field in the horizontal direction In position The absolute value of the gradient at that point. Represents the phase field in the vertical direction In position The absolute value of the gradient at a given point is obtained by calculating the gradient using the finite difference method. and ;

[0152] Control point loss is defined as: calculating the difference between the predicted control point coordinate matrix and the true control point coordinate matrix, and then using... The norm is used to calculate the sum of these differences, thus obtaining the value of the control point coordinate loss.

[0153] Right now ,in, Indicates the predicted control point coordinates. This represents the actual control point coordinates within the actual label. express Norm;

[0154] Weight loss is defined as: calculating the difference between the predicted control point weight matrix and the true control point weight matrix, and using... The norm is used to calculate the sum of these differences, thus obtaining the value of the weight loss.

[0155] Right now ,in, Indicates the predicted control point weights. This represents the actual control point weights corresponding to the actual labels. express Norm;

[0156] The parameters of the two-branch network are initialized using random initialization. The training set is then input into the two-branch network for forward propagation to obtain the control point coordinate matrix and weight matrix of the NURBS surface. After each forward propagation, the phase smoothness loss, control point coordinate loss, and weight loss are calculated based on the predicted control point coordinate matrix and weight matrix of the NURBS surface, yielding the values ​​of the three losses. The discriminator in the GAN loss is used to score the values ​​of the three losses against the true label. Then, the adversarial loss of the GAN is calculated using the GAN loss, and finally, the joint loss function is obtained by weighted summation.

[0157] Based on the joint loss function, the gradient of each network parameter is calculated using the backpropagation algorithm, and the network parameters are updated using the Adam optimization algorithm to minimize the loss function;

[0158] For each iteration, AUC is used as the evaluation metric, and the AUC value is calculated on the validation set.

[0159] Based on the AUC values ​​on the validation set, calculate the difference between the AUC value after the current iteration and the AUC value after the previous iteration, and denot it as the iteration difference;

[0160] Set an iteration difference threshold. If the iteration difference is greater than the iteration difference threshold, it is determined that the model's performance has improved.

[0161] If the iteration difference is less than or equal to the iteration difference threshold, it is determined that the model's performance has not improved.

[0162] If the model's performance on the validation set does not improve in consecutive DC iterations, then training is stopped, and a well-trained dual-branch network model is obtained.

[0163] The methods for constructing NURBS surfaces include:

[0164] Based on the control point coordinate matrix and weight matrix output by the dual-branch network model, the parameters of the NURBS surface are set, and then the NURBS surface is generated through the control point coordinate matrix and weight matrix.

[0165] Methods for high-precision 3D surface reconstruction of non-Lambertian metal objects include:

[0166] Based on the generated NURBS surface, the maximum and minimum principal curvature of each point on the NURBS surface are calculated using differential geometry. A high curvature threshold is set. If the maximum curvature of any point on the NURBS surface is greater than the high curvature, the point is determined to have a defect and is marked as an abnormal area on the NURBS surface.

[0167] Take the average height of all points on the NURBS surface as the average plane, and calculate the root mean square value of the height deviation of all points on the NURBS surface relative to the average plane as the surface roughness.

[0168] Based on the maximum and minimum principal curvature of each point on the NURBS surface, the radius of curvature of each point in both directions on the NURBS surface is calculated using the radius of curvature conversion formula.

[0169] Based on the radius of curvature of each point on the NURBS surface, as well as the abnormal regions and surface roughness, a smoothing algorithm is applied to improve the NURBS surface and reconstruct the three-dimensional surface shape, thus obtaining the reconstructed three-dimensional surface shape of the non-Lambertian metal object being measured.

[0170] In this embodiment, by projecting structured light patterns with different polarization states onto a non-Lambertian metal surface and simultaneously collecting reflection images and ambient light noise data, it is possible to comprehensively capture complex reflection characteristics (including specular reflection and diffuse reflection) and the influence of ambient light noise, providing accurate basic data for subsequent processing. Secondly, a weighted average method is used to generate a fused background image and remove ambient light noise, improving the quality of the reflection image. Subsequently, based on the light intensity values ​​of each pixel under different polarization states, the light polarization value and angle are calculated to distinguish between specular reflection and diffuse reflection areas, thereby reducing phase jump problems and improving measurement accuracy. Then, processing is performed on the specular reflection area to ensure the authenticity and reliability of the reflection image, and multi-channel phase is calculated to obtain a continuous phase field, solving the phase jump problem and ensuring the continuity of the phase field. The method ensures high accuracy and reliability, providing reliable data support for 3D topography reconstruction. Next, a dual-branch network model is constructed by combining phase information and polarization reflectivity characteristics. A joint optimization model using physical constraint loss and GAN loss is applied to output the control point coordinate matrix and weight matrix of the NURBS surface, improving the model's robustness and generalization ability, and enhancing the accuracy of 3D topography reconstruction. Finally, a NURBS surface is constructed based on the control point coordinate matrix and weight matrix. Defect detection is performed, and surface roughness and radius of curvature are extracted, thereby constructing a high-precision 3D surface. This achieves efficient and accurate measurement of the 3D topography of non-Lambertian metal surfaces. This method not only improves measurement accuracy and reliability but also has wide applicability and flexibility, effectively addressing complex industrial inspection needs and ensuring system stability and reliability.

[0171] Example 2

[0172] Please see Figure 2 As shown, the parts not described in detail in this embodiment are described in Embodiment 1. A three-dimensional measurement system for non-Lambertian metallic surfaces is provided, including: a spectral data processing area, a polarization feature analysis area, a phase correction and feature extraction area, a dual-branch network optimization area, and a surface reconstruction and detection area.

[0173] The spectral data processing area includes a polarization projection unit, a dataset generation unit, and a noise removal unit. The polarization projection unit is responsible for projecting structured light patterns with different polarization states onto the non-Lambertian metal surface and simultaneously triggering the camera to capture images. The dataset generation unit collects and processes data from the polarization projection workshop to generate a multimodal dataset containing polarized reflection images and ambient light noise. The noise removal unit uses a weighted average method to generate a fused background image and removes ambient light noise by pixel-by-pixel subtraction to prepare a clean reflection image for subsequent analysis.

[0174] The polarization feature analysis area includes a light intensity calculation unit, a light polarization analysis unit, and a reflection differentiation unit. The light intensity calculation unit calculates the light intensity value of each pixel under different polarization states, as well as its total light intensity value, and the first, second, and third light intensity components, for the reflection image under each polarization state. The light polarization analysis unit calculates the light polarization value and light polarization angle of each pixel based on the light intensity components, and constructs a grayscale image using the light polarization value. The reflection differentiation unit uses the light polarization threshold to distinguish between high-polarization points and low-polarization points, and marks the specular reflection area and the diffuse reflection area.

[0175] The phase correction and feature extraction area includes a specular removal unit and a phase calculation unit. The specular removal unit processes the specular reflection area in the grayscale image to remove specular reflection and ensure the accuracy of the reflected image. The phase calculation unit calculates the multi-channel phase to obtain a preliminary absolute phase field and further corrects the phase jump caused by anisotropic reflection to generate a continuous phase field and a polarization reflectivity matrix.

[0176] The dual-branch network optimization region includes dual-branch units and loss optimization units. The dual-branch units construct a dual-branch network model that combines phase information and polarization reflectivity features to fuse the two types of information. The loss optimization unit defines and applies physical constraint loss and GAN loss to optimize the dual-branch network model and outputs the control point coordinate matrix and weight matrix of the NURBS surface.

[0177] The surface reconstruction and detection area includes a surface generation unit, a defect and effect exploration unit, and a 3D morphology reconstruction unit. The surface generation unit constructs a NURBS surface based on the optimized control point coordinate matrix and weight matrix. The defect and effect exploration unit performs defect detection on the constructed NURBS surface and extracts characteristics such as surface roughness and radius of curvature. The 3D morphology reconstruction unit applies algorithms to improve the NURBS surface based on the extracted surface features and reconstructs the high-precision 3D morphology of the non-Lambertian metal object being tested.

[0178] Example 3

[0179] This embodiment discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the operation mode of the three-dimensional measurement method for non-Lambertian metallic surfaces described above.

[0180] Since the electronic device described in this embodiment is used to implement the three-dimensional measurement method for non-Lambertian metallic surfaces described in this application, those skilled in the art can understand the specific implementation and various variations of the electronic device in this embodiment based on the three-dimensional measurement method for non-Lambertian metallic surfaces described in this application. Therefore, how the electronic device implements the method in this application will not be described in detail here. Any electronic device used by those skilled in the art to implement the three-dimensional measurement method for non-Lambertian metallic surfaces described in this application falls within the scope of protection of this application.

[0181] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0182] The above description is merely a preferred embodiment of the present invention, and the scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for users of ordinary technical skills, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A three-dimensional measurement method for non-Lambertian metallic surfaces, characterized in that, include: S1: Polarized structured light is projected onto a non-Lambertian metal surface, and polarized reflection images and ambient light noise data are acquired simultaneously to generate a multimodal dataset; S2: Based on the multimodal dataset, a fused background image is generated for each reflection image using a weighted average method, and ambient light noise is removed by pixel-by-pixel subtraction; for each polarization state of the reflection image, the light intensity value of each pixel under different polarization states is extracted and the total light intensity value, the first light intensity component, the second light intensity component and the third light intensity component are calculated. The light polarization value and angle of each pixel are calculated based on these light intensity components, and a grayscale image is constructed using the light polarization value. A light polarization threshold is set to distinguish between high-polarity and low-polarity points, and high-polarity and low-polarity regions are marked using connected component analysis. For each high-polarity region, specular reflection and diffuse reflection regions are determined by comparing the standard deviation of the light polarization angle. S3: Based on the specular reflection region and diffuse reflection region in the reflection image, obtain the reflection image after specular reflection elimination; based on the reflection image after specular reflection elimination, solve the multi-channel phase to obtain the preliminary absolute phase field; based on the preliminary absolute phase field, correct the phase jump caused by anisotropic reflection, and generate the corrected continuous phase field and polarization reflectivity matrix. S4: Construct a dual-branch network model that combines phase branch and polarization branch. Input the continuous phase field and polarization reflectivity features into the dual-branch network model, fuse phase and polarization information, and optimize the dual-branch network model by combining physical constraint loss and GAN loss. Output the control point coordinate matrix and weight matrix of the NURBS surface, and then construct the NURBS surface. S5: Defect detection is performed on NURBS surfaces, surface roughness and radius of curvature are extracted, and then a high-precision three-dimensional surface shape of the non-Lambertian metal object under test is reconstructed. The specular reflection region and diffuse reflection region are obtained in the following ways: Based on the multimodal dataset, for each reflection image, a weighted average method is applied to each pixel position in all corresponding background images to obtain the weighted average value of each pixel position, and the weighted average value of each pixel position is integrated into a fused background image. The corresponding pixel value of the fused background image is subtracted from each pixel value of the reflected image using pixel-by-pixel subtraction to obtain the reflected image after removing ambient light noise. Based on the reflection images of each polarization channel after removing ambient light noise, all reflection images under each polarization state are taken as a set of reflection images; For each set of reflection images, for each pixel at each pixel location, the light intensity value of the corresponding pixel in the horizontal linear polarization state, vertical linear polarization state, left-tilted linear polarization state, and right-tilted linear polarization state is extracted from the reflection images of different deflection states, as well as the light intensity value of each pixel in the right-hand circular polarization state and left-hand circular polarization state. In the same set of reflection images, for each pixel, the average value of the light intensity under all polarization states is taken as the total light intensity value, the difference between the light intensity values ​​of the horizontal linear polarization state and the vertical linear polarization state is taken as the first light intensity component, the difference between the light intensity values ​​of the left-tilted linear polarization state and the right-tilted linear polarization state is taken as the second light intensity component, and the difference between the light intensity values ​​of the right-hand circular polarization state and the left-hand circular polarization state is taken as the third light intensity component. Calculate the sum of squares of the first light intensity component, the second light intensity component, and the third light intensity component, take the square root of the sum of squares, and calculate the ratio of the result to the total light intensity value to obtain the light deviation value of each pixel. Calculate the ratio of the first light intensity component to the second light intensity component, and calculate the arctangent of the ratio. Take half of the arctangent as the light deflection angle of each pixel. Based on the light bias value of each pixel, a grayscale image is constructed using the light bias value as the grayscale value. Based on the grayscale image, a light polarization threshold is set according to the light polarization value of each pixel. Pixels with light polarization values ​​greater than the light polarization threshold are recorded as high polarization points, and pixels with light polarization values ​​less than the light polarization threshold are recorded as low polarization points. Based on high-skewness points and low-skewness points, connected component analysis is used to identify and label connected high-skewness points, forming different connected components, which are labeled as high-skewness regions. The remaining non-high-skewness regions are labeled as low-skewness regions. For each high-bias region, take the standard deviation of the optical deflection angle of all pixels in the high-bias region and set the optical deflection angle threshold. If the standard deviation of the optical polarization angle in the high polarization region is less than the optical polarization angle threshold, the high polarization region is determined to be a specular reflection region. If the standard deviation of the light polarization angle in the high polarization region is greater than or equal to the light polarization angle threshold, it is determined to be a diffuse reflection region.

2. The three-dimensional measurement method for non-Lambertian metallic surfaces according to claim 1, characterized in that, The methods for generating the multimodal dataset include: Using a high-resolution projector with built-in polarization function, structured light patterns with different polarization states are projected onto a non-Lambertian metal surface according to a preset time sequence. The polarization states include horizontal linear polarization, vertical linear polarization, left-tilted linear polarization, right-tilted linear polarization, left-hand circular polarization, and right-hand circular polarization. The structured light patterns are composed of Gray code and phase-shifting fringes. Whenever a structured light pattern is projected, a camera is synchronously triggered to capture the polarized reflection image of the non-Lambertian metal surface at the corresponding moment. The reflection images of different polarization states in each polarization channel are obtained through polarization channel sub-channel technology. The polarization channels include circular polarization and linear polarization. Based on the captured polarized reflection images, the phase-shifting method and Gray code decoding algorithm are used to parse and obtain the phase-encoded information. Meanwhile, before and after each projection of structured light, images of the non-Lambertian metal surface in the rg frame without projection were captured under the same lighting conditions and used as background images. All the obtained reflection and background images are denoised and contrast-enhanced. The processed reflection and background images, along with the phase coding information, are then integrated into a multimodal dataset.

3. The three-dimensional measurement method for non-Lambertian metallic surfaces according to claim 2, characterized in that, The methods for obtaining the reflected image after specular reflection elimination include: Based on the specular reflection region of the grayscale image, the opening operation algorithm is used to remove noise and isolated points, and the closing operation algorithm is used to fill the holes in the specular reflection region after the opening operation. The specular reflection region after the closing operation is subjected to dilation and erosion operations respectively. The dilated specular reflection region is subtracted from the eroded specular reflection region to obtain the specular reflection region with enhanced edge contour. Based on the edge-enhanced specular reflection region, skeleton extraction technology is used to refine the boundary of the segmented specular reflection region, and morphological reconstruction technology is used to fill and repair the specular reflection region to obtain a complete specular reflection region. Based on the complete specular reflection area within the grayscale image, create a binary mask, and mark the specular reflection area as 0 and other areas as 1 on the binary mask; For each set of reflection images, the labeled binary mask is multiplied with all reflection images respectively to retain the diffuse reflection area and remove the specular reflection area, resulting in a set of reflection images after specular reflection elimination.

4. The three-dimensional measurement method for a non-Lambertian metallic surface according to claim 3, characterized in that, The preliminary absolute phase field is obtained in the following ways: Based on the set of reflection images after specular reflection elimination, Fourier transform is performed on the reflection images of horizontal linear polarization, vertical linear polarization, +45 degree linear polarization, -45 degree linear polarization, right-hand circular polarization, and left-hand circular polarization to extract the fundamental frequency component and filter it. Then, the initial phase estimate of each polarization state is obtained through inverse Fourier transform. Based on the initial phase estimate of each polarization state, the initial phase estimates of all polarization states are arranged into an initial phase estimate sequence; Based on the initial phase estimate of any polarization state in the initial phase estimate sequence, calculate the sum of the initial phase estimate differences between all adjacent polarization states. The weight of each initial phase estimate is determined based on the sum of the initial phase estimate differences between adjacent polarization states. Based on the initial phase estimate of each polarization state and the corresponding weight, the preliminary absolute phase field is calculated using the weighted average method.

5. The three-dimensional measurement method for a non-Lambertian metallic surface according to claim 4, characterized in that, The methods for generating the continuous phase field and polarization reflectivity matrix include: A lightweight residual network is used as the basic network structure. The absolute phase field and the light polarization value and light polarization angle of each pixel are taken as input. The loss function of the residual network is defined as the sum of the gradient smoothing term and the true phase supervision term. The residual network is trained using the defined loss function, and the output is the phase jump correction amount, which represents the amount of phase change that needs to be adjusted at each pixel. The gradient smoothing term is defined as the L2 norm of the gradient difference between the predicted phase field and the input absolute phase field; the true phase supervision term is defined as the L1 norm error between the predicted phase field and the true label corresponding to the training sample. Based on the output phase jump residual, the sum of the phase jump residual and the absolute phase field is taken to obtain the corrected continuous phase field; Based on the light polarization value and light polarization angle of each pixel obtained from the reflection images under different polarization states, the light polarization value of each pixel is normalized. The polarization reflectance of each pixel is calculated by multiplying the normalized optical polarization value with the cosine of the optical polarization angle. The polarization reflectance of all pixels is then integrated to construct a polarization reflectance matrix.

6. The three-dimensional measurement method for a non-Lambertian metallic surface according to claim 5, characterized in that, The construction methods of the dual-branch network model combining phase branching and polarization branching include: A dual-branch network architecture is constructed, including an input layer and feature extraction, a cross-membrane attention mechanism, and feature fusion and decoding; The input layer is defined as a dual-branch independent input, with the dual branches being a phase branch and a polarization branch. The input of the phase branch is a continuous phase field. Multi-scale convolution is used to extract the relevant features of the continuous phase field and output the feature map of the continuous phase field. The polarization branch takes the polarization reflectance matrix as input, uses lightweight convolution to extract the features of polarization reflectance, and outputs a feature map of polarization reflectance. Based on the feature map of the continuous phase field and the feature map of polarization reflectivity, the feature maps of the two branches are fused using a cross-attention mechanism. The feature map of the phase branch is used as the query, and the feature map of the polarization branch is used as the key and value. The attention weight is calculated using the attention weight formula of the cross-attention mechanism to obtain the attention-enhanced feature map. Among them, Query, Key, and Value represent the core components for calculating attention weights in the cross-membrane attention mechanism; The attention-enhanced feature map and the phase branch feature map are concatenated along the channel dimension of the multi-scale convolution to obtain a fused feature map. Then, the decoder is used to decode the fused feature map and output the control point coordinate matrix and weight matrix of the NURBS surface. The decoder is constructed by combining upsampling and convolution. First, upsampling is used to increase the size of the fused feature map, and then convolutional layers are applied to refine the details of the fused feature map.

7. A three-dimensional measurement method for a non-Lambertian metallic surface according to claim 6, characterized in that, The training methods for the dual-branch network model include: Collect training datasets, including continuous phase field and polarization reflectivity matrix, and corresponding ground truth labels; The training dataset is divided into a training set and a validation set according to a certain ratio; The loss function of a two-branch network is defined as the joint loss function, which is composed of physical constraint loss and GAN loss. Physical constraint losses include phase smoothness loss, control point coordinate loss, and weight loss; The phase smoothness loss is defined as follows: For each pixel, calculate the gradient of the pixel in the horizontal and vertical directions, take the absolute value of all gradients and add them together, and then take the sum of the sums of all pixels to obtain the total phase smoothness loss. Control point loss is defined as follows: calculate the difference between the predicted control point coordinate matrix and the true control point coordinate matrix, and use the L2 norm to calculate the sum of these differences to obtain the value of control point coordinate loss; The weight loss is defined as follows: calculate the difference between the predicted control point weight matrix and the true control point weight matrix, and use the L2 norm to calculate the sum of these differences to obtain the value of the weight loss; The parameters of the two-branch network are initialized using a random initialization method. The training set is then input into the two-branch network for forward propagation to obtain the control point coordinate matrix and weight matrix of the NURBS surface. After each forward propagation of the model, the phase smoothness loss, control point coordinate loss, and weight loss are calculated based on the predicted control point coordinate matrix and weight matrix of the NURBS surface to obtain the values ​​of the three losses. The discriminator in the GAN loss is used to score the values ​​of the three losses with the real labels, and then the GAN loss value is calculated using the GAN loss. The joint loss function is obtained by weighted summing of the values ​​of phase smoothness loss, control point loss, weight loss, and GAN loss. Based on the joint loss function, the gradient of each network parameter is calculated using the backpropagation algorithm, and the network parameters are updated using the Adam optimization algorithm to minimize the loss function; For each iteration, AUC is used as the evaluation metric, and the AUC value is calculated on the validation set. Based on the AUC values ​​on the validation set, calculate the difference between the AUC value after the current iteration and the AUC value after the previous iteration, and denot it as the iteration difference; Set an iteration difference threshold. If the iteration difference is greater than the iteration difference threshold, it is determined that the model's performance has improved. If the iteration difference is less than or equal to the iteration difference threshold, it is determined that the model's performance has not improved. If the model's performance on the validation set does not improve in consecutive DC iterations, training is stopped, and a trained dual-branch network model is obtained.

8. The three-dimensional measurement method for a non-Lambertian metallic surface according to claim 7, characterized in that, The methods for constructing the NURBS surface include: Based on the control point coordinate matrix and weight matrix output by the dual-branch network model, the order of the two directions on the NURBS surface is set, and the number of control points and the corresponding uniformly distributed node vectors of the NURBS surface are set. The control point coordinate matrix and weight matrix are merged into a four-dimensional vector, and then a parameterized NURBS surface is constructed using NURBS tools.

9. A three-dimensional measurement method for a non-Lambertian metallic surface according to claim 8, characterized in that, The high-precision three-dimensional surface reconstruction method for the measured non-Lambertian metal object includes: Based on the generated NURBS surface, the maximum and minimum principal curvature of each point on the NURBS surface are calculated using differential geometry. A high curvature threshold is set. If the maximum curvature of any point on the NURBS surface is greater than the high curvature, the point is determined to have a defect and is marked as an abnormal area on the NURBS surface. Take the average height of all points on the NURBS surface as the average plane, and calculate the root mean square value of the height deviation of all points on the NURBS surface relative to the average plane as the surface roughness. Based on the maximum and minimum principal curvature of each point on the NURBS surface, the radius of curvature of each point in both directions on the NURBS surface is calculated using the radius of curvature conversion formula. Based on the radius of curvature of each point on the NURBS surface, as well as the abnormal regions and surface roughness, a smoothing algorithm is applied to improve the NURBS surface and reconstruct the three-dimensional surface shape, resulting in a high-precision three-dimensional surface shape of the non-Lambertian metal object being measured.

Images