A high-precision 3D defect measurement method and device

Abstract

This invention discloses a high-precision 3D defect measurement method and apparatus. The method includes the following steps: based on the original optical signals acquired by a 3D camera, an initial depth map and confidence map are generated through phase demodulation and depth reconstruction; the Poisson equation is discretized using the finite difference method to construct a weighted gradient field and generate a sparse linear equation; graph structure features are extracted using a graph convolutional neural network to predict a dynamic preprocessor matrix, improving solution stability; combined with the generalized minimum residual algorithm, matrix block partitioning, mixed precision, and parallel computing are used on a graphics processor to accelerate the solution, obtaining a high-precision optimized depth map; abnormal regions are located through gradient analysis, threshold segmentation, and morphological filtering, contours are extracted and mapped to three-dimensional space, generating a complete defect detection result including location, size, and visual annotations. This invention improves the accuracy of 3D camera defect detection.

Description

Technical Field

[0001] This invention relates to the field of 3D, and more particularly to a high-precision 3D defect measurement method and apparatus. Background Technology

[0002] With the widespread application of 3D cameras in 3D scanning, augmented reality, and intelligent sensing, the depth maps they output are often affected by noise, missing values, and reconstruction artifacts, leading to the easy concealment or false detection of minute defects, severely impacting the reliability of subsequent defect identification. Existing depth optimization methods based on the Poisson equation rely on traditional iterative solvers, resulting in low computational efficiency and slow convergence speed, making it difficult to meet the real-time processing requirements of 3D cameras. Furthermore, fixed precondition strategies and low-precision acceleration techniques have poor generalization ability in complex scenes, easily introducing numerical errors and further reducing the accuracy of defect detection. Therefore, this invention proposes a high-precision 3D defect measurement method and device. Summary of the Invention

[0003] This invention provides a high-precision 3D defect measurement method, characterized by comprising: S10. Based on the original optical signals collected by the 3D camera, the initial depth map and pixel confidence map are obtained through phase demodulation 3D imaging algorithm, analog-to-digital conversion and depth reconstruction process. S20. Based on the initial depth map and pixel confidence map, the Poisson equation is discretized using the finite difference method. After the process of constructing a weighted gradient field and generating the Laplace matrix, a sparse linear equation for numerical solution is obtained. S30. Based on the discretized matrix and right-hand vector, the neural preprocessor, constructed by the graph convolutional neural network, goes through the processes of graph structure feature extraction, dynamic non-zero padding pattern prediction and stability constraint optimization to obtain a preprocessor matrix that satisfies the accelerated convergence condition. S40. Based on the preprocessor matrix and sparse linear equations, a high-precision optimized depth map is obtained by performing matrix block parallel computation, mixed-precision operation and Krylov subspace orthogonalization on the graphics processor through an improved generalized minimum residual iterative algorithm. S50. Based on the optimized depth map, the defect detection algorithm, through gradient analysis, threshold segmentation and morphological filtering, goes through the abnormal region localization, contour extraction and three-dimensional space mapping process to obtain the final three-dimensional defect detection result including position, size and visual annotation.

[0004] The high-precision 3D defect measurement method described above involves obtaining an initial depth map and a pixel confidence map by using a phase demodulation 3D imaging algorithm based on the raw optical signals acquired by a 3D camera, followed by analog-to-digital conversion and depth reconstruction. Specifically, it comprises the following sub-steps: Four-phase sinusoidal stripes are projected by a structured light 3D camera, and four-step phase-shifted images are simultaneously acquired by a high frame rate camera. The images are then time-aligned and noise-reduced by a field-programmable gate array to generate a high-quality original optical signal matrix. Phase adjustment is performed on the four-step phase-shifted image sequence to extract the wrapped phase. Multi-frequency heterodyne technology combined with Gray code-assisted verification is used to achieve highly robust absolute phase unfolding, and an initial depth map is generated based on the phase height mapping relationship. After digitizing the analog optical signal through analog-to-digital conversion, and combining imaging quality indicators such as signal-to-noise ratio, fringe contrast, and edge blur, a confidence map for each pixel is calculated using a weighted S-function, and then compared with the initial depth... Figure 1 Same as output.

[0005] The high-precision 3D defect measurement method described above involves discretizing the Poisson equation using the finite difference method based on an initial depth map and pixel confidence map. This process includes constructing a weighted gradient field and generating the Laplace matrix to obtain a sparse linear equation for numerical solution. Specifically, it comprises the following sub-steps: Based on the confidence-guided weighted gradient field method, pixel-level confidence maps are used as weights to fuse the original and smooth gradients, thereby improving the stability and accuracy of depth map optimization. By using the finite difference discretization of the Laplacian operator, the weighted gradient field is transformed into a large-scale sparse linear equation, thereby achieving global optimization and reconstruction of the depth map, preserving key geometric features while effectively suppressing noise. By vectorizing the divergence field and incorporating known depth constraints, the process encapsulates the discrete equations into a standard linear algebra structure, forming a sparse matrix and right-hand vector pair for depth map optimization.

[0006] The high-precision 3D defect measurement method described above involves, based on the discretized matrix and right-hand vector, a neural preprocessor constructed from a graph convolutional neural network. This preprocessor undergoes graph structure feature extraction, dynamic non-zero padding pattern prediction, and stability constraint optimization to obtain a preprocessor matrix that satisfies the accelerated convergence condition. Specifically, it comprises the following sub-steps: The discretized matrix is ​​treated as the adjacency matrix of the graph and the node features are initialized. A three-layer graph convolutional network is used to extract the topological structure features and output a high-dimensional graph embedding containing global context information. By introducing dynamic filling prediction based on attention mechanism and utilizing the importance relationship between computing nodes in graph embedding, the sparse connection pattern of the preprocessor is adaptively generated. While preserving the original structure, key non-zero terms are intelligently added, effectively improving the adaptability to complex defect regions. Based on the dynamic filling mask predicted by the attention mechanism, an end-to-end preprocessor generation that balances structure guidance and numerical stability is achieved by learning adjustable weights at non-zero positions and jointly optimizing residual approximation, spectral radius constraints and low-rank regularization.

[0007] The high-precision 3D defect measurement method described above involves, based on a preprocessor matrix and sparse linear equations, a modified generalized minimum residual iterative algorithm, and matrix block parallel computation, mixed-precision computation, and Krylov subspace orthogonalization processes performed on a graphics processor to obtain a high-precision optimized depth map. Specifically, it comprises the following sub-steps: To address the different sparsity characteristics of sparse matrices and neural preprocessors, parallel strategies of spatial partitioning and compressed storage are adopted respectively to achieve efficient matrix-vector computation on graphics processors. Accelerated matrix operations are performed based on different levels of precision, and the accuracy of key steps is guaranteed. A residual correction mechanism is introduced to achieve efficient and stable mixed-precision solutions. The Krylov subspace is constructed through the Arnoldi iterative process, and the solution is transformed into a low-dimensional least squares problem using the Shanghai Senberg matrix. The optimal update direction is obtained through orthogonal triangular decomposition, and a refined two-dimensional depth map is output.

[0008] The high-precision 3D defect measurement method described above involves, based on an optimized depth map, employing gradient analysis, threshold segmentation, and morphological filtering defect detection algorithms. This process includes anomaly region localization, contour extraction, and 3D spatial mapping to obtain the final 3D defect detection result, which includes location, size, and visual annotations. Specifically, it comprises the following sub-steps: By analyzing the normal vector, curvature, and gradient features of the depth map, and combining RGB image auxiliary information, non-physical abnormal regions in the depth map are identified and located. Candidate regions are processed using adaptive threshold segmentation and morphological operations to remove artifacts and accurately extract meaningful anomaly regions. The detected abnormal areas are mapped to three-dimensional space to generate a defect report that includes precise location, size, shape, and visual annotations.

[0009] The high-precision 3D defect measurement method described above involves processing candidate regions through adaptive threshold segmentation and morphological operations to remove artifacts and accurately extract meaningful anomaly regions. Specifically, it consists of the following sub-steps: The gradient magnitude map is calculated based on the optimized depth map, and the Otsu algorithm is used to analyze the overall distribution of the gradient magnitude to automatically determine the global optimal segmentation threshold. A local variance compensation mechanism is introduced to generate an initial anomaly region mask; Morphological closing operations are performed on the binary mask to obtain more complete and connected anomalous regions. Isolated regions with too small an area are analyzed and removed to accurately extract anomalous regions.

[0010] The present invention also provides a high-precision 3D defect measurement device, comprising: Depth confidence module: Based on the raw optical signals acquired by the 3D camera, the module uses a phase demodulation 3D imaging algorithm to perform analog-to-digital conversion and depth reconstruction to obtain an initial depth map and a pixel confidence map. Sparse Equation Module: Based on the initial depth map and pixel confidence map, the Poisson equation is discretized using the finite difference method. After the weighted gradient field construction and Laplace matrix generation process, a sparse linear equation for numerical solution is obtained. Preprocessor module: Based on the discretized matrix and right-hand vector, a neural preprocessor composed of a graph convolutional neural network goes through graph structure feature extraction, dynamic non-zero padding pattern prediction and stability constraint optimization processes to obtain a preprocessor matrix that meets the accelerated convergence condition. Depth map optimization module: Based on the preprocessor matrix and sparse linear equations, the improved generalized minimum residual iterative algorithm is used to perform matrix block parallel computation, mixed precision operation and Krylov subspace orthogonalization on the graphics processor to obtain a high-precision optimized depth map. Defect detection module: Based on the optimized depth map, the module uses gradient analysis, threshold segmentation and morphological filtering defect detection algorithms to go through the abnormal region localization, contour extraction and 3D spatial mapping process to obtain the final 3D defect detection result including position, size and visual annotation.

[0011] The beneficial effects achieved by this invention are as follows: Starting from the original signal of the 3D camera, this invention uses phase demodulation and confidence modeling to generate a high-quality initial depth map. Through Poisson solution guided by a neural preprocessor, noise is effectively suppressed and missing regions are filled, fundamentally reducing artifacts and false detection sources. This enables accurate extraction and quantification of real defects in the depth map, thereby improving the accuracy and reliability of defect detection under complex imaging conditions. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0013] Figure 1 This is a flowchart of a high-precision 3D defect measurement method provided in Embodiment 1 of this application.

[0014] Figure 2 This is a schematic diagram of a high-precision 3D defect measurement device provided in Embodiment 2 of this application. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Example 1

[0017] like Figure 1 As shown, Embodiment 1 of this application provides a high-precision 3D defect measurement method, including: S10. Based on the original optical signals acquired by the 3D camera, the initial depth map and confidence map are obtained through phase demodulation 3D imaging algorithm, analog-to-digital conversion and depth reconstruction process.

[0018] S11. Four-phase sinusoidal stripes are projected by a structured light 3D camera, and four-step phase-shifted images are synchronously acquired by a high frame rate camera. The images are then time-aligned and noise-reduced by a field-programmable gate array to generate a high-quality original optical signal matrix.

[0019] To accurately capture the 3D information of an object's surface, a structured light 3D camera emits periodic sinusoidal phase-shifted grating patterns. By changing the phase of these patterns, the intensity variations of light reflected from the object's surface are captured. Specifically, a four-step phase-shifting method is employed: sinusoidal fringes with phase shifts of 0°, 90°, 180°, and 270° are projected sequentially, and a high frame rate camera module simultaneously captures image sequences at these four different phases. Each image records the grayscale information of its corresponding spatial location (x, y). To ensure temporal alignment of the image sequence and reduce noise interference, a field-programmable gate array (FPGA) is used to perform pixel-level temporal alignment on the acquired image sequence, ensuring that each pixel in each frame corresponds to the lighting conditions at the same moment. A bilateral filtering advanced noise reduction algorithm is then applied to effectively remove noise while preserving edge details. The processed images are then organized into an original optical signal matrix. , The set of real numbers is denoted by H and W, which represent the height and width of the image, respectively. The third dimension represents the set of images corresponding to the four phase shifts.

[0020] S12. Phase adjustment is performed on the four-step phase-shifted image sequence to extract the wrapped phase. Multi-frequency heterodyne technology combined with Gray code-assisted verification is used to achieve highly robust absolute phase unfolding, and an initial depth map is generated based on the phase height mapping relationship.

[0021] Phase adjustment is performed using the arctangent function formula to calculate the wrap-around phase from a four-step phase-shifted image sequence. The formula is as follows: The phase value is limited to Within the range, the periodically wrapped phase map cannot directly reflect the true height. To unfold the wrapped phase, a multi-frequency heterodyne method is used to make it an absolute phase. This method projects a set of low-frequency phase and a set of high-frequency phase sinusoidal fringe patterns, and calculates the corresponding wrapped phase field for each. and By utilizing the beat frequency effect between high and low frequencies, an equivalent heterodyne phase is constructed, as shown in the formula: ,in, and These represent the wavelengths corresponding to low and high frequencies, respectively. For pi, the mod 1 operation is used to handle the periodicity of the phase, thus accurately unfolding the phase.

[0022] Based on the principles of triangulation and the phase-height mapping relationship, the formula is used. The unfolded absolute phase is converted into actual height information, where... This is the equivalent heterodyne phase, where c represents the speed of light. The spatial frequency of the high-frequency stripe modulation. The height values ​​of all pixels are reorganized into a two-dimensional matrix to generate the initial depth map. It represents the three-dimensional shape of an object's surface.

[0023] To ensure real-time performance, a field-programmable gate array (FPGA) is employed to parallelize the entire process of phase demodulation and absolute phase unwrapping. Key steps such as four-step phase-shift demodulation, multi-frequency heterodyne calculation, and phase height mapping are completed within sub-milliseconds, meeting the requirements of high frame rate 3D imaging. Simultaneously, a Gray code-assisted verification mechanism is introduced. A set of binary Gray code patterns is superimposed while projecting sinusoidal fringes. The single-bit change characteristics of Gray code are used to uniquely encode the spatial region. The decoding result provides global topological constraints for phase unwrapping, effectively overcoming phase ambiguity and misunwrapping caused by texture loss, surface reflection, or abrupt geometric changes. This significantly improves the robustness and integrity of phase demodulation under complex conditions.

[0024] S13. After digitizing the analog optical signal through analog-to-digital conversion, and combining imaging quality indicators such as signal-to-noise ratio, fringe contrast, and edge blur, the confidence map of each pixel is calculated using a weighted S-function and compared with the initial depth. Figure 1 Same as output.

[0025] The analog optical signals from the 3D camera need to undergo an analog-to-digital conversion (ADC) process to convert them into digital signals for subsequent processing. The sampling precision of the ADC is set to 12 bits, and the quantization step size is [value missing]. While ensuring dynamic range, high-precision digitization is achieved, ensuring that subtle changes in light intensity are not drowned out by quantization noise.

[0026] The imaging quality of pixels in the four-step phase-shifted images is calculated as the basis for constructing the confidence map. Stripe contrast is a key parameter for measuring the sharpness of structured light stripes, defined as... , and These represent the maximum and minimum pixel intensities acquired at the same spatial location (x, y) as the phase-shift pattern changes periodically. A value closer to 1 indicates clearer fringes and more reliable phase demodulation results; however, in reflective, absorbent, or out-of-focus areas, contrast decreases significantly, leading to increased phase error. The second derivative of the initial depth map is calculated using the Laplacian operator. It is used to detect the degree of edge blurring. Blurred areas correspond to smaller curvature responses, indicating that the image is not clear. At the same time, it analyzes the uniformity of illumination to identify overexposed or underexposed areas, and combines the occlusion detection algorithm to determine whether there are parallax blind spots or shadow interference.

[0027] A weighted nonlinear combination method is used to fuse multi-dimensional quality indicators into a unified pixel-level confidence map, as shown in the formula: ,in The sigmoid function is used to compress the output to the (0,1) interval, enhancing smoothness. The weighting coefficients... These are empirical parameters, determined through calibration experiments to balance the contributions of various indicators. and The term acts as a positive incentive, increasing the weight of the high-reliability region. This term, acting as a penalty, proactively reduces confidence in regions with blurred edges or significant noise, preventing erroneous gradients from misleading subsequent optimization processes. The signal-to-noise ratio. This confidence plot... Compared with the preliminary depth map Output together.

[0028] S20. Based on the initial depth map and pixel confidence map, the Poisson equation is discretized using the finite difference method. After the weighted gradient field construction and Laplace matrix generation process, a sparse linear equation for numerical solution is obtained.

[0029] S21. Based on the confidence-guided weighted gradient field method, the pixel-level confidence map is used as the weight to fuse the original and smooth gradients, thereby improving the stability and accuracy of depth map optimization.

[0030] The initial depth map contains non-uniform noise and unreliable regions. Directly reconstructing its gradient using Poisson gradients amplifies the noise and errors, severely impacting the accuracy and reliability of defect detection. To overcome the limitations of traditional Poisson depth map optimization methods that directly use the initial depth map gradient... To address the issues that the gradient field can amplify noise, lead to artifacts and blurred edges when used as a right-hand side term in the Poisson equation, a confidence-guided weighted gradient field construction method is proposed: based on the measurement reliability of each pixel, the original gradient and the smoothed prior gradient are adaptively fused to construct an optimized guided field that preserves details in high-confidence regions while suppressing noise in low-confidence regions.

[0031] Specifically, pixel-level confidence maps are used. As a weighting guide signal This indicates that the depth measurement of this pixel is highly reliable. This indicates the presence of unreliable conditions such as occlusion, reflection, or blurred fringes. Based on this, the weighted gradient field is defined as follows: This formula achieves intelligent gradient fusion through two complementary terms: the first term The original gradient information of high-confidence regions is directly preserved to ensure that key geometric features such as object edges and indentations are accurately preserved; the second item Then a smooth prior is introduced for the low-confidence region, where The standard deviation is A Gaussian kernel is used to spatially smooth the original gradient to suppress noise propagation, while the coefficients... Ensure that the smoothing term is activated only in regions of low confidence to avoid blurring reliable edges; smoothing intensity coefficient. Adjustable prior strength control is provided. The generated weighted gradient field serves as the divergence source term for the subsequent Poisson equation, which not only significantly improves the stability of the optimization process but also ensures that high-precision depth maps with clear boundaries and good noise suppression can still be reconstructed in complex real-world scenarios.

[0032] S22. By using finite difference discretization of the Laplacian operator, the weighted gradient field is transformed into a large-scale sparse linear equation, thereby achieving global optimization and reconstruction of the depth map, preserving key geometric features while effectively suppressing noise.

[0033] After constructing the weighted gradient field, substitute it into the Poisson equation. , Let u be the second-order spatial rate of change of the depth map to be optimized. For divergence operators, To construct a weighted gradient field, the partial differential equation is discretized on a regular grid using the finite difference method, thus establishing a large-scale sparse linear equation. This transforms the continuous physical modeling problem into a computer-solvable discrete algebraic problem, thereby enabling accurate numerical reconstruction of the optimized depth map. The specific process is as follows: On an H×W regular pixel grid, for each internal pixel, an algebraic equation about the Laplacian operator is established using the relationship between the height values ​​of its four neighboring pixels (up, down, left, and right) and its own height value, following a five-point difference template. Then, based on the weighted gradient field... The divergence value at each pixel is calculated using the central difference method. This serves as the discrete input to the right-hand side of the Poisson equation, ensuring accurate injection of geometric information. Next, to guarantee the equation's closure, boundary conditions are selected based on the actual scenario. If there is no external height reference, the boundary normal gradient is set to zero, employing a von Neumann boundary. If the edge height information is known, a Dirichlet boundary is used, and a fixed value is set through interpolation, adjusting the coefficients and constant terms of the linear equation accordingly.

[0034] Expand all grid points into a one-dimensional vector in row-major order. And construct the corresponding sparse coefficient matrix. Each row of the matrix A corresponds to a discretized equation. Since each equation involves at most five variables, A is a typical pentagonal sparse matrix. To save storage space and improve subsequent solution efficiency, the coefficient matrix is ​​stored in a compressed sparse row format, recording only the non-zero elements and their positions, thus completing the full conversion from continuous partial differential equations to discrete sparse algebra problems.

[0035] S23. By vectorizing the divergence field and incorporating known depth constraints, the encapsulation from discrete equations to a standard linear algebra structure is completed, forming a sparse matrix and right-hand vector pair for depth map optimization.

[0036] To transform the discretized Poisson equation into a standard linear algebraic solution, the two-dimensional divergence field is first... Vectorization is performed: Each pixel coordinate (i,j) is mapped to a one-dimensional index k = i.W + j in row-major order, and then rearranged into a column vector of length N = HW. Each element Corresponding position in the original image divergence value at Simultaneously, known depth prior information is incorporated as a strong constraint. For pixels with known depth values, the corresponding equations are modified: the diagonal elements in the coefficient matrix A are set to 1, the off-diagonal elements are set to zero, and the right-hand vector is set... This forces the solution to be exactly equal to the reliable values ​​of the initial depth map at these locations, while the remaining unknown points retain the Laplace structure constructed by the five-point difference template. Finally, the entire algebraic structure is encapsulated, forming a standard form. , where u is the one-dimensional vector representation of the optimized depth map to be solved, A is the sparse coefficient matrix, and b is the right-hand side containing geometric guidance and prior constraints.

[0037] S30. Based on the discretized matrix and right-hand vector, a neural preprocessor composed of a graph convolutional neural network is used to extract graph structure features, predict dynamic non-zero padding patterns, and optimize stability constraints to obtain a preprocessor matrix that meets the accelerated convergence condition.

[0038] S31. Treat the discretized matrix as the adjacency matrix of the graph and initialize the node features. Use a three-layer graph convolutional network to extract topological features and output a high-dimensional graph embedding containing global context information.

[0039] To extract topological information from a sparse matrix A using graph convolutional networks, A is first considered as an adjacency matrix of a graph G=(V,E), where the vertex set... Represents all pixels in the image. Let A represent the k-th pixel node in the image. The edge set E is defined by the non-zero elements in A, representing the connections between pixels. Then, the feature vector of each node is initialized. That is, it contains all non-zero elements in the row corresponding to the k-th pixel in matrix A. and the right-hand item The value of is used to capture local geometric and physical constraints: represents taking all non-zero elements in the corresponding row. The eigenvectors of all nodes are arranged row-wise into a matrix. As the input layer, the three-layer graph convolutional network performs feature transformation and aggregation. The calculation formula for each layer is as follows: , Indicates after the first Node feature representation after layer graph convolution operation It is a general expression used to describe the forward propagation process of each layer. The values ​​are 0, 1, and 2. Indicates after the first Node feature representation after layer graph convolution operation It is the adjacency matrix after adding self-loops. It is its degree matrix. The weight matrix is ​​a learnable matrix. This represents the activation function. The high-dimensional graph embedding is output after three layers of graph convolutional network. N is the total number of nodes, and d is the embedding dimension. It encodes the complex interactions between pixels and global context information, providing strong feature support for further optimization of the depth map.

[0040] S32. By introducing dynamic filling prediction based on attention mechanism, and utilizing the importance relationship between nodes in graph embedding, the sparse connection pattern of the preprocessor is adaptively generated. While preserving the original structure, key non-zero terms are intelligently added, effectively improving the adaptability to complex defect regions.

[0041] To overcome the limitations of traditional ILU preprocessors, which require manual setting of fill levels and thus suffer from poor generalization and difficulty adapting to complex surface structures, a dynamic fill prediction method based on an attention mechanism is proposed. This method adaptively generates the sparse connection pattern of the preprocessor based on the optimized structural characteristics of the current depth map. Specifically, the method calculates the structural similarity score for each pair of nodes using the extracted high-dimensional graph embeddings and employs a learnable attention mechanism formula. , are the transposes of the high-dimensional graph embedding vectors of the i-th and j-th nodes, respectively, where For learnable query and key projection matrices, The scaling factor is used to stabilize the attention weights. Then, softmax normalization is applied to all similarity scores for each node i to obtain normalized attention weights, reflecting the importance of node j to node i in information propagation. Next, a dynamic threshold is set. If the attention weight between a pair of nodes is greater than Even if there are no direct connections in the original sparse matrix, non-zero elements are allowed at corresponding positions in the preprocessor matrix. Simultaneously, to maintain the original physical connection structure, all positions of non-zero elements in the original matrix are preserved. This ultimately generates a binary mask matrix. This mechanism explicitly indicates the locations of non-zero items that should be retained or added in the preprocessor. It can adaptively capture the anisotropic diffusion requirements in complex defect regions, maintain sparsity in flat areas to improve efficiency, and actively add key connections in edge, recessed, or textured areas, thereby constructing a more efficient preprocessor sparsity pattern that conforms to the actual geometry.

[0042] S33. Based on the dynamic filling mask predicted by the attention mechanism, an end-to-end preprocessor generation that balances structure guidance and numerical stability is achieved by learning adjustable weights at non-zero positions and jointly optimizing residual approximation, spectral radius constraints and low-rank regularization.

[0043] Sparse patterns alone are insufficient to guarantee numerical stability; arbitrarily assigning values ​​directly to this structure can still lead to divergence in the iterative solution process. A joint optimization mechanism integrating physical constraints and structural regularization is used to achieve a high-precision, stable preprocessor, moving from interpretable sparse patterns. The specific implementation process is as follows: Initialize a learnable matrix P on the defined support set, in the form of: , As a learnable weight matrix, based on high-dimensional graph embedding Z, a lightweight edge prediction head is used for each pair. nodes splicing its features into Output scalar value This represents the weight that the preprocessor matrix P should be assigned at position (i,j), and all of them... Fill in the corresponding positions to form a sparse learnable matrix. . This indicates element-wise multiplication.

[0044] P is taken as the approximate inverse matrix The initial estimate is obtained and structured optimization is performed on it through an objective function to ensure that it not only approximates the true inverse matrix but also possesses good numerical stability and computational efficiency. This objective function contains three key components: the first is the residual minimization term. This forces P to approach This effectively reduces systematic errors. U is the identity matrix, and A is the sparse coefficient matrix. The Frobenius norm; the second term introduces the spectral radius function. , Represents the modulus of the largest eigenvalue of the matrix, explicitly controls the convergence boundary of the iterative operator, and ensures the spectral radius function This fundamentally eliminates the risk of divergence; the third item adopts the nuclear norm. This promotes a low-rank structure for P, which helps suppress noise amplification, enhances generalization ability, and reduces storage overhead. The entire optimization objective is determined by the hyperparameters. and By balancing the weights among the three factors, the final training criterion is formed: This optimization problem is implemented within an automatic differentiation framework, using end-to-end backpropagation training on P to enable the network to learn and generate a preprocessor that is both efficient and stable. The final output is a preprocessor matrix subject to strict stability constraints. It can not only significantly accelerate the solution speed of linear systems, but also maintain robust operation when facing complex, ill-conditioned or noisy depth graph optimization problems.

[0045] S40. Based on the preprocessor matrix and sparse linear equations, a high-precision optimized depth map is obtained by performing matrix block parallel computation, mixed-precision operation and Krylov subspace orthogonalization on the graphics processor through an improved generalized minimum residual iterative algorithm.

[0046] S41. To achieve efficient matrix and vector computation on a graphics processor, parallel strategies of spatial partitioning and compressed storage are adopted to address the different sparsity characteristics of sparse matrices and neural preprocessors.

[0047] To support the efficient operation of the neural preprocessor in iterative solutions and fully leverage the massive parallelism of the graphics processing unit (GPU), matrix A is divided into B×B regular sub-blocks according to the image space. Each CUDA thread block is responsible for the sparse matrix-vector multiplication calculation of one sub-block, and non-zero elements are cached in shared memory to reduce memory access latency. For the neural preprocessor... Its sparse pattern is dynamically generated by an attention mechanism, includes non-local connections, and uses a compressed sparse line format combined with a warp-level parallel strategy to ensure that any sparse structure can efficiently perform computational tasks. To achieve seamless integration between data preparation on the CPU side and computation on the GPU side, unified memory management is used to achieve transparent data migration between the CPU and GPU, avoiding the performance loss and development burden caused by explicit copying.

[0048] S42. Accelerate matrix operations and ensure the accuracy of key steps based on different levels of precision, and introduce a residual correction mechanism to achieve efficient and stable mixed-precision solution.

[0049] Building upon matrix block parallel computation, a hybrid precision strategy is introduced to optimize the balance between computational efficiency and numerical stability at the data representation level. This strategy employs a dynamic precision management mechanism that coordinates FP16 and FP32. FP16 half-precision floating-point numbers are used in the sparse matrix-vector multiplication, which has the highest computational cost, to fully utilize the GPU's tensor cores and high bandwidth, significantly improving throughput. Conversely, in the orthogonalization process, where numerical stability is paramount, FP32 single-precision numbers are used to ensure the accuracy of inner product and projection calculations between vectors. To address the residual drift problem introduced by FP16 in long iterations, an adaptive residual correction mechanism is designed: after each iteration, a high-precision residual copy is maintained and compared with the residual from the low-precision path computation. The deviation between the two is extracted as a cumulative error compensation term, fed back to the next round of residual updates at a certain attenuation ratio, effectively suppressing error propagation. This hybrid strategy achieves convergence reliability close to FP32 while maintaining performance close to FP16, realizing efficient and robust solutions for large-scale linear systems.

[0050] S43. Construct the Krylov subspace through the Arnoldi iteration process, transform the solution into a low-dimensional least squares problem using the Shanghai Senberg matrix, obtain the optimal update direction through orthogonal triangular decomposition, and output a refined two-dimensional depth map.

[0051] Supported by a mixed-precision computation strategy, the core steps of the generalized minimum residual method are executed: Krylov subspace orthogonalization and solution update. Without explicit inversion, the optimal approximate solution is found in the Krylov subspace spanned by the initial residuals, minimizing the corresponding residuals. Specifically, the initial residuals are first calculated. Let b be the true right-hand side vector, and A be a sparse matrix. This is the initial depth map. It is then normalized to obtain the first basis vector. Subsequently, through the Arnoldi iterative process, the m basis vectors are mapped to the space of m+1 basis vectors, gradually constructing a set of orthonormal bases. Simultaneously generate Shanghai Senberg Matrix , used to represent the projection of matrix A onto this subspace. Then, the least squares problem of dimension m is solved through orthogonal trigonometric decomposition. In order to determine the optimal combination coefficients , This indicates that y is a vector of length m. Let be the norm of the initial residual. This is the first standard unit vector. The final solution will be... Mapping back from the subspace to the original space and updating the depth map as follows: ,in It is a matrix composed of orthogonal bases. This process iterates until the relative residual is lower than a preset threshold, at which point the optimized depth map vector is output and reshaped into a two-dimensional image. This completes the fine-grained restoration of the depth map.

[0052] S50. Based on the optimized depth map, the defect detection algorithm, through gradient analysis, threshold segmentation and morphological filtering, goes through the abnormal region localization, contour extraction and three-dimensional space mapping process to obtain the final three-dimensional defect detection result including position, size and visual annotation.

[0053] S51. By analyzing the normal vector, curvature, and gradient features of the depth map, and combining RGB image auxiliary information, non-physical abnormal regions in the depth map are identified and located.

[0054] Geometric feature analysis is performed on the optimized depth map to automatically identify candidate anomaly regions. By calculating the depth change rate of each pixel in the horizontal and vertical directions, the local surface normal direction is estimated, and geometric properties reflecting surface curvature, including Gaussian curvature and depth gradient magnitude, are derived. Gaussian curvature is used to determine whether the point is in a convex, concave, or saddle-shaped region, while the depth gradient magnitude characterizes the drasticness of depth change; together, they constitute a comprehensive description of the local geometric structure. To achieve accurate localization of anomaly regions, a geometric anomaly response index is proposed as the core criterion. , in, and These are the standard deviations of the Gaussian curvature and depth gradient in the local neighborhood centered on the current pixel, respectively, used to achieve adaptive normalization, enabling the criterion to adapt to regions with different texture densities and noise levels; the tanh() function compresses the joint response to the [0,1) interval, generating a smooth anomaly confidence heatmap. Let be the absolute value of the Gaussian curvature. This represents the depth gradient magnitude. To reduce noise in the RGB images captured by the camera using Gaussian filtering, the Sobel operator is used to calculate the image gradient. Then, non-maximum suppression and double thresholding are combined to extract the image edge confidence, which is used to exclude real object boundaries. The shadow detection results obtained by the multi-light condition comparison method are used to mask the pseudo-gradient regions caused by uneven lighting. The product term of the two ensures that the abnormal response is activated only in non-edge and non-shadow regions. Regions with higher values ​​are identified as abnormal candidate regions.

[0055] S52. The candidate region is processed by adaptive threshold segmentation and morphological operations to remove artifacts and accurately extract the abnormal regions with practical significance.

[0056] The identified anomalous candidate regions undergo refined processing to accurately extract and purify anomalous structures. First, a gradient magnitude map is calculated based on the optimized depth map to reflect the severity of depth changes in each region. Next, the Otsu algorithm is used to analyze the overall distribution of gradient magnitudes and automatically determine the globally optimal segmentation threshold to initially distinguish between significantly changing and smooth regions. To enhance adaptability to local features, a local variance compensation mechanism is introduced: the variance of depth values ​​is calculated within a 5×5 neighborhood of each pixel and weighted and superimposed onto the global threshold, forming an adaptive threshold that varies spatially, thus maintaining good segmentation capabilities even in regions with complex textures or uneven lighting. Subsequently, this dynamic threshold is applied to the gradient map for binarization, generating a preliminary anomalous region mask, where locations with gradients exceeding the threshold are marked as 1, and others as 0. Since noise or sampling discontinuities can cause breaks or holes in anomalous regions, morphological closing operations are performed on the binary mask. This involves first performing dilation to connect adjacent broken parts, and then performing erosion to restore the original boundary shape, thereby obtaining a more complete and connected anomalous region. Finally, by using connected component analysis to eliminate isolated regions with excessively small areas, we can accurately extract outlier regions with real impact.

[0057] S53. Map the detected abnormal areas to three-dimensional space to generate a defect report containing precise location, size, shape and visual annotations.

[0058] This method accurately maps abnormal regions detected in 2D images back to 3D space, achieving a transition from planar recognition to 3D quantization. First, using depth maps and camera calibration parameters, the coordinates of each pixel are back-projected to the corresponding 3D point cloud position, thus locating the spatial region containing the defect on the original 3D model. Next, the moving square algorithm is used to extract the boundary contour of this region, generating a high-precision closed polygon representation. Based on the extracted defect contour, the minimum bounding box is calculated using the convex hull and rotating caliper methods. The maximum circumscribed diameter is obtained by finding the extreme values ​​of point-to-point distances, the area is calculated using the shoelace formula, and the maximum depth deviation is obtained through reference plane fitting and residual analysis, achieving comprehensive quantification of the defect's size and shape.

[0059] Considering operators' cognitive habits regarding real-world visual scenes, the 3D inspection results are back-projected onto the corresponding original RGB image plane. Using calibrated camera intrinsic and extrinsic parameters, information such as defect contours, dimensions, and maximum depth deviation in 3D space is precisely overlaid onto a textured and colored real-world image, clearly marked with red highlight boxes, dimension lines, and text labels. Simultaneously, a standardized JSON-formatted inspection report is automatically generated, recording each defect's unique ID, 3D coordinates, depth error, area, and contour point set, supporting subsequent quality traceability, statistical analysis, and automated decision-making.

[0060] Example 2

[0061] like Figure 2 As shown, Embodiment 2 of this application provides a high-precision 3D defect measurement device, comprising: Depth Confidence Module: Based on the raw optical signals acquired by the 3D camera, this module uses a phase demodulation 3D imaging algorithm, undergoing analog-to-digital conversion and depth reconstruction, to obtain an initial depth map and a pixel confidence map. It is specifically divided into the following sub-modules: Original matrix submodule: Four-phase sinusoidal fringes are projected by a structured light 3D camera, and four-step phase-shifted images are synchronously acquired by a high frame rate camera. The images are then time-aligned and noise-reduced by a field-programmable gate array to generate a high-quality original optical signal matrix.

[0062] Initial Depth Submodule: Phase adjustment is performed on the four-step phase-shifted image sequence to extract the wrapped phase. Multi-frequency heterodyne technology combined with Gray code-assisted verification is used to achieve highly robust absolute phase unwrapping, and an initial depth map is generated based on the phase height mapping relationship.

[0063] Confidence Map Submodule: After digitizing the analog optical signal through analog-to-digital conversion, it combines imaging quality indicators such as signal-to-noise ratio, fringe contrast, and edge blur to calculate the confidence map of each pixel using a weighted S-function, and then compares it with the initial depth... Figure 1 Same as output.

[0064] The sparse equation module: Based on the initial depth map and pixel confidence map, the Poisson equation is discretized using the finite difference method. This involves constructing a weighted gradient field and generating the Laplacian matrix to obtain a sparse linear equation for numerical solution. Specifically, it is divided into the following sub-modules: Weighted gradient submodule: Based on the confidence-guided weighted gradient field method, it uses pixel-level confidence maps as weights to fuse the original and smooth gradients, thereby improving the stability and accuracy of depth map optimization.

[0065] The optimization and reconstruction submodule transforms the weighted gradient field into a large-scale sparse linear equation by using the finite difference discretization of the Laplacian operator, thereby achieving global optimization and reconstruction of the depth map, preserving key geometric features while effectively suppressing noise.

[0066] Encapsulation Submodule: By vectorizing the divergence field and fusing known depth constraints, the module completes the encapsulation from discrete equations to a standard linear algebra structure, forming a sparse matrix and right-hand vector pair for depth map optimization.

[0067] The preprocessor module, based on the discretized matrix and right-hand vector, uses a neural preprocessor constructed from a graph convolutional neural network. This preprocessor undergoes graph structure feature extraction, dynamic non-zero padding pattern prediction, and stability constraint optimization to obtain a preprocessor matrix that satisfies the accelerated convergence condition. Specifically, it is divided into the following sub-modules: The high-dimensional graph submodule treats the discretized matrix as an adjacency matrix of a graph and initializes node features. It then uses a three-layer graph convolutional network to extract topological features and outputs a high-dimensional graph embedding containing global context information.

[0068] Sparse Connection Submodule: By introducing dynamic filling prediction based on attention mechanism, it utilizes graph embedding to compute the importance relationship between nodes and adaptively generates the sparse connection pattern of the preprocessor. While preserving the original structure, it intelligently adds key non-zero terms, effectively improving the adaptability to complex defect regions.

[0069] The generation submodule is based on a dynamic filling mask predicted by an attention mechanism. It learns adjustable weights at non-zero positions and jointly optimizes residual approximation, spectral radius constraints and low-rank regularization to achieve an end-to-end preprocessor generation that balances structure guidance and numerical stability.

[0070] Depth Map Optimization Module: Based on the preprocessor matrix and sparse linear equations, a high-precision optimized depth map is obtained through a modified generalized minimum residual iterative algorithm on the graphics processor, involving matrix block parallel computation, mixed-precision arithmetic, and Krylov subspace orthogonalization. Specifically, it consists of the following sub-modules: Parallel Strategy Submodule: Based on the different sparsity characteristics of sparse matrices and neural preprocessors, parallel strategies of spatial partitioning and compressed storage are adopted respectively to achieve efficient matrix and vector computation on the graphics processor.

[0071] Mixed precision submodule: Accelerates matrix operations and ensures the accuracy of key steps based on different precision levels, and introduces a residual correction mechanism to achieve efficient and stable mixed precision solution.

[0072] The fine-grained depth submodule constructs a Krylov subspace through the Arnoldi iterative process, transforms the solution into a low-dimensional least squares problem using the Shanghai Senberg matrix, obtains the optimal update direction through orthogonal triangular decomposition, and outputs a fine-grained two-dimensional depth map.

[0073] Defect Detection Module: Based on the optimized depth map, this module employs gradient analysis, threshold segmentation, and morphological filtering algorithms to perform anomaly region localization, contour extraction, and 3D spatial mapping, ultimately obtaining the final 3D defect detection result, including location, size, and visual annotations. Specifically, it is divided into the following sub-modules: Anomaly Candidate Submodule: By analyzing the normal vector, curvature, and gradient features of the depth map, and combining RGB image auxiliary information, it identifies and locates non-physical anomaly regions in the depth map.

[0074] The anomaly identification submodule processes candidate regions through adaptive threshold segmentation and morphological operations to remove artifacts and accurately extract meaningful anomaly regions.

[0075] Defect Identification Submodule: Maps detected abnormal areas to 3D space and generates a defect report containing precise location, size, shape, and visual annotations.

[0076] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A high precision 3D defect measurement method, characterized by, include: S10. Based on the original optical signals collected by the 3D camera, the initial depth map and pixel confidence map are obtained through phase demodulation 3D imaging algorithm, analog-to-digital conversion and depth reconstruction process. S20. Based on the initial depth map and pixel confidence map, the Poisson equation is discretized using the finite difference method. After the process of constructing a weighted gradient field and generating the Laplace matrix, a sparse linear equation for numerical solution is obtained. S20 is specifically divided into the following sub-steps: Based on the confidence-guided weighted gradient field method, pixel-level confidence maps are used as weights to fuse the original and smooth gradients, thereby improving the stability and accuracy of depth map optimization. By using the finite difference discretized Laplacian operator, the weighted gradient field is transformed into a large-scale sparse linear equation, enabling global optimization and reconstruction of the depth map while preserving key geometric features and effectively suppressing noise. By vectorizing the divergence field and fusing known depth constraints, the encapsulation from discrete equations to a standard linear algebra structure is completed, forming a sparse matrix and right-hand vector pair for depth map optimization. S30. Based on the sparse matrix and the right-hand vector pair, the neural preprocessor, constructed by the graph convolutional neural network, goes through the process of graph structure feature extraction, dynamic non-zero padding pattern prediction and stability constraint optimization to obtain a preprocessor matrix that can significantly accelerate convergence. S30 is specifically divided into the following sub-steps: The discretized matrix is ​​treated as the adjacency matrix of the graph and the node features are initialized. A three-layer graph convolutional network is used to extract the topological structure features and output a high-dimensional graph embedding containing global context information. By introducing dynamic filling prediction based on attention mechanism and utilizing the importance relationship between nodes in graph embedding, the sparse connection pattern of the preprocessor is adaptively generated. While preserving the original structure, key non-zero terms are intelligently added, which effectively improves the adaptability to complex defect regions. Specifically, the intelligent addition of key non-zero terms involves preserving the positions of all non-zero elements in the original matrix and generating a binary mask matrix to indicate the positions of non-zero terms that should be retained or added in the preprocessor. Based on the dynamic filling mask predicted by the attention mechanism, an end-to-end preprocessor generation that balances structure guidance and numerical stability is achieved by learning adjustable weights at non-zero positions and jointly optimizing residual approximation, spectral radius constraints and low-rank regularization. S40. Based on the preprocessor matrix and sparse linear equations, a high-precision optimized depth map is obtained by performing matrix block parallel computation, mixed-precision operation and Krylov subspace orthogonalization on the graphics processor through an improved generalized minimum residual iterative algorithm. S50. Based on the optimized depth map, the defect detection algorithm, through gradient analysis, threshold segmentation and morphological filtering, goes through the abnormal region localization, contour extraction and three-dimensional space mapping process to obtain the final three-dimensional defect detection result including position, size and visual annotation.

2. The high-precision 3D defect measurement method as described in claim 1, characterized in that, Based on the raw optical signals acquired by the 3D camera, an initial depth map and pixel confidence map are obtained through a phase demodulation 3D imaging algorithm, undergoing analog-to-digital conversion and depth reconstruction. This process is divided into the following sub-steps: Four-phase sinusoidal stripes are projected by a structured light 3D camera, and four-step phase-shifted images are simultaneously acquired by a high frame rate camera. The images are then time-aligned and noise-reduced by a field-programmable gate array to generate a high-quality original optical signal matrix. Phase adjustment is performed on the four-step phase-shifted image sequence to extract the wrapped phase. Multi-frequency heterodyne technology combined with Gray code-assisted verification is used to achieve highly robust absolute phase unfolding, and an initial depth map is generated based on the phase height mapping relationship. After digitizing the analog optical signal through analog-to-digital conversion, the confidence map of each pixel is calculated by combining the signal-to-noise ratio, stripe contrast, and edge blur, and output together with the initial depth map using a weighted nonlinear combination method.

3. The high-precision 3D defect measurement method as described in claim 1, characterized in that, Based on the preprocessor matrix and sparse linear equations, a high-precision optimized depth map is obtained through an improved generalized minimum residual iterative algorithm, involving matrix block parallel computation, mixed-precision arithmetic, and Krylov subspace orthogonalization on a graphics processor. The process is divided into the following sub-steps: To address the different sparsity characteristics of sparse matrices and neural preprocessors, parallel strategies of spatial partitioning and compressed storage are adopted respectively to achieve efficient matrix-vector computation on graphics processors. Accelerated matrix operations are performed based on different levels of precision, and the accuracy of key steps is guaranteed. A residual correction mechanism is introduced to achieve efficient and stable mixed-precision solutions. The Krylov subspace is constructed through the Arnoldi iterative process, and the solution is transformed into a low-dimensional least squares problem using the Shanghai Senberg matrix. The optimal update direction is obtained through orthogonal triangular decomposition, and a refined two-dimensional depth map is output.

4. The high-precision 3D defect measurement method as described in claim 1, characterized in that, Based on the optimized depth map, the defect detection algorithm employs gradient analysis, threshold segmentation, and morphological filtering. Through anomaly region localization, contour extraction, and 3D spatial mapping, the final 3D defect detection result, including location, size, and visual annotations, is obtained. This process is divided into the following sub-steps: By analyzing the normal vector, curvature, and gradient features of the depth map, and combining RGB image auxiliary information, non-physical abnormal regions in the depth map are identified and located. Candidate regions are processed using adaptive threshold segmentation and morphological operations to remove artifacts and accurately extract meaningful anomaly regions. The detected abnormal areas are mapped to three-dimensional space to generate a defect report that includes precise location, size, shape, and visual annotations.

5. The high-precision 3D defect measurement method as described in claim 4, characterized in that, Candidate regions are processed using adaptive thresholding and morphological operations to remove artifacts and accurately extract meaningful anomaly regions. The process involves the following sub-steps: The gradient magnitude map is calculated based on the optimized depth map, and the Otsu algorithm is used to analyze the overall distribution of the gradient magnitude to automatically determine the global optimal segmentation threshold. A local variance compensation mechanism is introduced to generate an initial anomaly region mask; Morphological closing operations are performed on the binary mask to obtain more complete and connected anomalous regions. Isolated regions with too small an area are analyzed and removed to accurately extract anomalous regions.

6. A high-precision 3D defect measurement device, performing a high-precision 3D defect measurement method as described in any one of claims 1-5, characterized in that, include: Depth confidence module: Based on the raw optical signals acquired by the 3D camera, the module uses a phase demodulation 3D imaging algorithm to perform analog-to-digital conversion and depth reconstruction to obtain an initial depth map and a pixel confidence map. Sparse Equation Module: Based on the initial depth map and pixel confidence map, the Poisson equation is discretized using the finite difference method. After the weighted gradient field construction and Laplace matrix generation process, a sparse linear equation for numerical solution is obtained. Preprocessor module: Based on the discretized matrix and right-hand vector, a neural preprocessor composed of a graph convolutional neural network goes through graph structure feature extraction, dynamic non-zero padding pattern prediction and stability constraint optimization processes to obtain a preprocessor matrix that can significantly accelerate convergence. Depth map optimization module: Based on the preprocessor matrix and sparse linear equations, the improved generalized minimum residual iterative algorithm is used to perform matrix block parallel computation, mixed precision operation and Krylov subspace orthogonalization on the graphics processor to obtain a high-precision optimized depth map. Defect detection module: Based on the optimized depth map, the module uses gradient analysis, threshold segmentation and morphological filtering defect detection algorithms to go through the abnormal region localization, contour extraction and 3D spatial mapping process to obtain the final 3D defect detection result including position, size and visual annotation.

Images