A Method and System for Product Appearance Defect Identification Based on Multimodal Imaging

Abstract

This invention relates to the field of automated optical inspection technology, and discloses a method and system for identifying product appearance defects based on multimodal imaging. One method for identifying product appearance defects based on multimodal imaging includes: acquiring multi-time-step interferometric imaging, polarization imaging, and scattering imaging data sequences; extracting the instantaneous phase and instantaneous frequency at each time step using Hilbert transform to generate a time-series phase-frequency feature map; processing the interferometric fringe data at each time step using a phase unwrapping algorithm to output a time-series phase evolution matrix; learning the interlayer phase propagation law using a spatiotemporal graph convolutional network to generate a hierarchical defect evolution representation; and predicting the spatiotemporal evolution trajectory of the defect through four-dimensional gradient calculation, outputting the four-dimensional morphological parameters and development trend of the defect. This invention overcomes the limitation of traditional methods in simultaneously processing the spatial distribution and temporal evolution of phase information through a deep coupling mechanism between multi-scale phase evolution kernel functions and spatiotemporal graph networks.

Description

Technical Field

[0001] This invention relates to the field of automated optical inspection technology, and more specifically, to a method and system for identifying product appearance defects based on multimodal imaging. Background Technology

[0002] In the continuous production process of high-precision optical components, stress defects within transparent materials dynamically evolve with changes in temperature gradients and mechanical stress. These defects not only exhibit complex distribution characteristics in three-dimensional space but also change in morphology and location over time. Existing defect detection methods mainly suffer from the following technical problems: traditional static detection methods can only obtain three-dimensional defect reconstruction results at a specific moment or can only analyze temporal variation characteristics within a two-dimensional plane, failing to achieve a complete four-dimensional characterization of defects in three-dimensional space and time. This limitation makes it impossible to accurately predict the development trend of defects and assess the cumulative impact of defect evolution on the performance of optical components, severely restricting the quality control capabilities of high-precision optical components. Summary of the Invention

[0003] This invention provides a product appearance defect identification method and system based on multimodal imaging, which solves the technical problem in related technologies that can only process static three-dimensional data or two-dimensional time-series data in a separate processing manner, and cannot perform four-dimensional complete characterization and dynamic prediction of internal defects in transparent materials.

[0004] This invention provides a product appearance defect identification method based on multimodal imaging, comprising:

[0005] Multimodal imaging equipment is used to acquire imaging data of transparent materials at continuous time points, and the acquired imaging data is preprocessed to extract temporal phase feature maps.

[0006] Process time-series phase data and calculate the phase evolution matrix, which represents the rate of change of the phase over time;

[0007] Based on the voxelization algorithm, the three-dimensional phase field data at each time moment is discretized into spatial grid nodes, and a four-dimensional spatiotemporal graph network that integrates spatial topology and temporal evolution relationship is constructed.

[0008] The multi-scale phase evolution kernel function is applied to the four-dimensional spatiotemporal graph network, and spatiotemporal pattern features of phase change are extracted through convolution operation;

[0009] Based on the hierarchical graph pooling algorithm, subgraph structures of different depths are constructed, and a spatiotemporal graph convolutional network is used to generate defect evolution representations.

[0010] Calculate the four-dimensional gradient field and predict the spatiotemporal evolution trajectory of defects by streamline tracing.

[0011] In a preferred embodiment, the process of acquiring imaging data of a transparent material at consecutive time points using a multimodal imaging device, and extracting a temporal phase feature atlas after preprocessing the acquired imaging data includes:

[0012] Imaging data sequences of transparent materials at continuous time points were obtained using interferometric imaging equipment, polarization imaging equipment, and scattering imaging equipment, respectively.

[0013] The acquired raw imaging data is subjected to image intensity normalization, spatial registration, and temporal synchronization correction.

[0014] The Hilbert transform algorithm is applied to the interferometric imaging data at each time step to calculate the instantaneous phase and instantaneous frequency, generating a time-series feature map containing phase and frequency information.

[0015] In a preferred embodiment, the spatial registration process is implemented using a phase cross-correlation algorithm, which includes:

[0016] Interferometric imaging data was selected as the reference image;

[0017] The polarization imaging and scattering imaging data are cross-correlated with the reference image to obtain the cross-correlation matrix;

[0018] Find the peak position in the cross-correlation matrix to determine the optimal matching displacement;

[0019] Subpixel-level registration is performed using quadratic interpolation.

[0020] In a preferred embodiment, processing the time-series phase data and calculating the phase evolution matrix includes:

[0021] The interference fringe data at each time step are processed based on the phase unwrapping algorithm, and the wrapped phase is converted into a continuous phase distribution.

[0022] Perform differential operations on continuous phase data at adjacent time points to output a time-series phase evolution matrix that characterizes the rate of change of phase over time.

[0023] In a preferred embodiment, the three-dimensional phase field data at each time step are discretized into spatial grid nodes based on a voxelization algorithm, and a four-dimensional spatiotemporal graph network integrating spatial topology and temporal evolution relationships is constructed, including:

[0024] The three-dimensional phase field data at each time moment are discretized into spatial grid nodes using a voxelization algorithm;

[0025] Establish edge connection relationships between nodes based on phase continuity constraints;

[0026] Connect the corresponding spatial nodes in the time series through time edges to generate a four-dimensional spatiotemporal graph network.

[0027] In a preferred embodiment, the mesh size in the voxelization algorithm is determined according to the following formula:

[0028] The grid size is calculated by dividing the wavelength of the light source by eight times the refractive index of the medium, ensuring that there are at least eight sampling points in the region with the fastest phase change, thus satisfying the Nyquist sampling theorem's requirement for spatial frequency sampling.

[0029] In a preferred embodiment, applying a multi-scale phase evolution kernel function to the four-dimensional spatiotemporal graph network and extracting spatiotemporal pattern features of phase changes through convolution operations includes:

[0030] Apply the multi-scale phase evolution kernel function to the nodes of the four-dimensional spatiotemporal graph network;

[0031] Spatiotemporal pattern features of phase change are extracted through convolution operations at different scales;

[0032] The output is a feature tensor containing information at multiple scales.

[0033] In a preferred embodiment, constructing subgraph structures of different depths based on a hierarchical graph pooling algorithm, and generating defect evolution representations using a spatiotemporal graph convolutional network includes:

[0034] Based on the hierarchical graph pooling algorithm, nodes at different depth levels are constructed into subgraph structures;

[0035] The phase propagation rules between layers are learned using a spatiotemporal graph convolutional network;

[0036] A hierarchical defect evolution representation vector is generated by aggregating multi-layer features.

[0037] In a preferred embodiment, calculating the four-dimensional gradient field and predicting the spatiotemporal evolution trajectory of the defect by streamline tracing includes:

[0038] Calculate the spatial gradient components at each time step to generate a three-dimensional gradient vector field;

[0039] Calculate the time partial derivative of the phase field to obtain the time evolution rate;

[0040] Integrate spatial and temporal gradient components to construct a four-dimensional gradient vector field;

[0041] Starting from the initial position of the defect, numerical integration is performed along the four-dimensional gradient direction to predict the evolution trajectory of the defect in four-dimensional spacetime.

[0042] This invention provides a product appearance defect recognition system based on multimodal imaging, comprising:

[0043] A multimodal imaging module is used to acquire imaging data of transparent materials at continuous time points;

[0044] The data preprocessing module is used to preprocess the acquired imaging data and extract temporal phase feature maps;

[0045] The phase calculation module is used to process time-series phase data and calculate the phase evolution matrix;

[0046] The graph network construction module is used to build four-dimensional spatiotemporal graph networks.

[0047] The feature extraction module is used to extract spatiotemporal pattern features of phase changes;

[0048] The defect representation module is used to generate hierarchical defect evolution representations;

[0049] The trajectory prediction module is used to calculate the four-dimensional gradient field and predict the spatiotemporal evolution trajectory of the defect.

[0050] The beneficial effects of this invention are as follows:

[0051] By employing a coupled processing mechanism of multi-scale phase evolution kernel functions and a four-dimensional spatiotemporal graph network, this method overcomes the limitations of traditional methods that can only handle static three-dimensional data or two-dimensional time-series data. Therefore, it solves the technical problem of the inability to perform complete four-dimensional characterization and dynamic prediction of internal defects in transparent materials. By extending the phase field gradient propagation algorithm from three-dimensional space to four-dimensional space including the time dimension, accurate prediction of defect evolution trajectories is achieved, providing effective technical support for the quality control of high-precision optical components. Attached Figure Description

[0052] Figure 1 This is a flowchart of a product appearance defect identification method based on multimodal imaging according to the present invention;

[0053] Figure 2 This is a time-series line graph of the multimodal imaging data acquisition process of the present invention;

[0054] Figure 3 This is a heatmap of the defect evolution prediction of the present invention;

[0055] Figure 4 This is a bar chart showing the effect of the multi-scale phase evolution kernel function of this invention;

[0056] Figure 5 This is a scatter plot of the defect detection accuracy of the present invention; Detailed Implementation

[0057] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, some features described in the examples may be combined in other examples.

[0058] At least one embodiment of the present invention discloses a method for identifying product appearance defects based on multimodal imaging, such as... Figure 1 As shown, it includes the following steps:

[0059] Step 1: Acquire multi-time-phase imaging data and extract temporal phase feature maps;

[0060] Imaging data sequences of transparent materials at continuous time points were acquired using interferometric imaging, polarization imaging, and scattering imaging devices. The acquired raw imaging data underwent preprocessing. Further spatial registration was achieved using a phase cross-correlation algorithm. The specific steps were as follows: First, interferometric imaging data was selected as a reference image. Then, the polarization imaging and scattering imaging data were cross-correlated with the reference image, respectively. ,in For reference image, For the target image, For relative displacement; then in the cross-correlation matrix Finding the peak position This is the optimal matching displacement; finally, a quadratic interpolation method is used for sub-pixel level registration, achieving an accuracy of 0.1 pixels. Time synchronization correction uses the difference in timestamps recorded by the device trigger signal to perform linear interpolation on the acquired data, aligning the time sampling points of different modes to a unified time axis, with a sampling synchronization accuracy better than 1ms.

[0061] It should be noted that the Hilbert transform algorithm refers to the analytical construction of the interference fringe signal, and the calculation... Calculate the phase evolution matrix from the ordered phase data.

[0062] Step 2: Process the time-series phase data and calculate the phase evolution matrix;

[0063] Based on the phase unwrapping algorithm, the interference fringe data obtained in step 1 are processed to convert the wrapped phase into a continuous phase distribution. The continuous phase data at adjacent times are differentially processed to output a time-series phase evolution matrix that represents the phase time change rate.

[0064] Furthermore, the continuous phase distribution output by the phase unwrapping algorithm is no longer limited by... The range is not defined as a fixed interval, but rather can be extended to a wider range based on the actual physical processes. Scope, of which This is an integer representing the number of phase cycles. The unwrapping algorithm employs a phase continuity constraint, meaning the phase difference between adjacent pixels should not exceed [a certain value]. When a phase transition is detected, by increasing or decreasing To eliminate discontinuities, use integer multiples of the given value.

[0065] Step 3: Construct a four-dimensional spatiotemporal graph network structure;

[0066] The three-dimensional phase field data obtained in step 2 are discretized into spatial grid nodes using a voxelization algorithm. Based on the phase continuity constraint, edge connections between nodes are established, and corresponding spatial nodes in the time series are connected through temporal edges to generate a four-dimensional spatiotemporal graph network that integrates spatial topology and temporal evolution. ,in For a set of nodes, For the set of spatial edges, It is a time edge set.

[0067] The aforementioned voxelization algorithm takes continuous three-dimensional phase field data as input. The output is a set of discretized voxel nodes. Each node Includes spatial coordinates and phase value Voxelization involves dividing space into a regular grid, with the center of each grid serving as a node, and the grid size... Determined based on the spatial resolution of the phase field.

[0068] Furthermore, the mesh size in the voxelization algorithm The specific determination method is as follows: ,in To use the wavelength of the light source, Let be the refractive index of the medium. This size setting ensures at least 8 sampling points in the region of fastest phase change, satisfying the Nyquist sampling theorem's sampling requirement for spatial frequencies and avoiding aliasing effects of spatial information. When the spatial size of the target region is... At that time, the number of grids Round up to ensure coverage of the entire target area.

[0069] Step 4: Extract spatiotemporal features using a multi-scale phase evolution kernel function;

[0070] The multi-scale phase evolution kernel function is applied to the nodes of the four-dimensional spatiotemporal graph network constructed in step 3, and the phase evolution is achieved through different scales.

[0071] Convolution operations extract spatiotemporal pattern features of phase transitions, outputting a feature tensor containing information at multiple scales. ,in For the first Multiscale phase evolution kernel function at multiple scales For spacetime phase field, This indicates a convolution operation.

[0072] Furthermore, convolution operations follow the principle of separability of spatial and temporal dimensions;

[0073] in For nodes in a four-dimensional spacetime graph network, For a point in time, Represents a node Spatial neighborhood, The width of the time window is half (with a value of 2 to 3). Let be the spatial distance between nodes. The computational complexity of this convolution operation is O(n log n). ,in The number of nodes in the four-dimensional spatiotemporal graph network. This represents the number of time steps. To reduce computational complexity, a fast convolution optimization algorithm is used, reducing the complexity to [value missing]. .

[0074] in For spatial distance, For time intervals, For the first Spatial bandwidth parameters at each scale For time and frequency parameters, This is the phase offset parameter. To ensure dimensional consistency, the spatial distance... and time interval Standardization process: Divide by the feature space scale Obtain dimensionless spatial variables , Multiply by characteristic angular frequency Obtain the dimensionless time variable This ensures that the parameters of the exponential and trigonometric function terms in the kernel function are dimensionless, thus guaranteeing that the operations between different physical quantities conform to physical laws.

[0075] Furthermore, the value ranges and determination methods of each parameter in the multi-scale phase evolution kernel function are as follows: Spatial bandwidth parameter ,in The grid size of the voxels. For the characteristic scale of the space observation area, The specific values ​​follow a geometric series distribution, that is... , , This ensures effective capture of features at different spatial scales; time-frequency parameters ,in Total observation time The time sampling interval is... The values ​​of also follow a geometric progression distribution; phase offset parameter They are uniformly distributed to cover the entire phase period. Typically, the number of scales... The value ranges from 3 to 5 to balance computational complexity and feature extraction capability.

[0076] Step 5: Perform hierarchical feature learning to generate defect evolution representations;

[0077] Based on the hierarchical graph pooling algorithm, nodes at different depths are constructed into subgraph structures. The phase propagation law between layers is learned by using the spatiotemporal graph convolutional network. By aggregating multi-layer features, a hierarchical defect evolution representation vector is generated.

[0078] The aforementioned hierarchical graph pooling algorithm takes the node feature set and topology of a four-dimensional spatiotemporal graph network as input and outputs subgraph structures at different depths. The algorithm merges adjacent nodes through node clustering or sampling methods to form a hierarchical graph representation. To ensure the comparability of node features at different depths, node features are normalized using a min-max normalization method to map each node's feature value to the [0,1] interval.

[0079] Furthermore, the hierarchical graph pooling algorithm is specifically implemented using the differential pooling method. This method performs clustering based on node similarity and sets a similarity threshold. Nodes with similarity higher than a threshold are aggregated into a supernode, whose characteristics are a weighted average of the aggregated nodes. ,in This represents the set of clustered nodes; to preserve the graph's topology, the connection weights between supernodes are set to the sum of the connection weights between the original nodes. ,in These are the elements of the original adjacency matrix. These are the elements of the adjacency matrix after pooling. and This is the set of original nodes contained in the two supernodes.

[0080] The aforementioned spatiotemporal graph convolutional network defines its input layer as receiving the node feature matrix of a four-dimensional spatiotemporal graph network. and adjacency matrix ,in For the number of nodes, For time steps, The feature dimension is used; the output layer is a defect evolution representation vector. ,in This is the output feature dimension. The defect evolution representation vector output by the network is converted into specific defect physical parameters, including defect spatial coordinates, through a decoding mapping function. Geometric dimensions Morphological types (Classification coding) and time evolution rate The decoding process uses a combined mapping method, which uses linear mapping to recover the range of physical quantity values ​​for continuous parameters, and uses the softmax function for probabilistic decoding for discrete parameters.

[0081] Furthermore, the specific structure of the spatiotemporal graph convolutional network consists of three main modules: (1) a spatial graph convolution module, which uses Chebyshev polynomial approximation spectral graph convolution operation. ,in The normalized Laplace matrix, for Chebyshev polynomials For the first Learnable parameters of the layer (2) Temporal convolution module, using causal convolution structure. ,in This represents the temporal convolution kernel size (with a value of 3). (3) Spatiotemporal fusion module, through gating mechanism, is a learnable weight parameter; Integrating spatial features and time characteristics ,in For adaptive weights, This represents element-wise product. The network has 3 layers, with feature dimensions of 64, 128, and 256 for each layer, to balance network expressiveness and computational efficiency.

[0082] The training method for the aforementioned spatiotemporal graph convolutional network includes: end-to-end supervised learning for training, the Adam optimizer for optimization, and a learning rate set to... ;

[0083] Furthermore, to enhance the accuracy of the loss function in identifying temporal defect evolution, the loss function is extended to a weighted spatiotemporal joint version:

[0084]

[0085] The second item specifically constrains the speed of defect evolution over time. This is the weighting coefficient for the temporal velocity term, with a value ranging from [0.1, 0.5], to balance spatial precision and temporal continuity. This temporal constraint ensures that the modulus decays to 0.9 times its original value every 10 training epochs until a preset minimum learning rate is reached. .

[0086] Step 6: Calculate the four-dimensional gradient field to predict the defect evolution trajectory;

[0087] The phase field gradient propagation algorithm is extended from three-dimensional space to four-dimensional space including the time dimension by calculating the four-dimensional gradient. Based on streamline tracing of a four-dimensional gradient field, the spatiotemporal evolution trajectory of defects is predicted, outputting a set of four-dimensional morphological parameters of the defects, including spatial location, size, shape, and rate of change over time. To address the inconsistency between the spatial and temporal gradient dimensions in the four-dimensional gradient, a characteristic time scale is introduced. Standardize the time gradient: This ensures that each component of the four-dimensional gradient has the same physical dimensions, thus guaranteeing the accuracy of the four-dimensional gradient field streamline tracing calculation.

[0088] Furthermore, the characteristic time scale The method for determining it is as follows: ,in The characteristic length of the defect spatial distribution is taken as 1 / 10 of the observation area size; The characteristic velocity of defect evolution is obtained through historical data statistics, specifically calculated as the average rate of change of defect location within the observation period. When historical data is insufficient... It can be determined through physical estimation: ,in Dimensionless coefficient (range of values ​​is) ), The characteristic thickness of transparent materials, Let be the thermal diffusivity of the material. This method ensures the comparability of spatial and temporal gradients on the order of magnitude, guaranteeing the weight balance of each component in the four-dimensional gradient field.

[0089] The aforementioned phase field gradient propagation algorithm, extended from three-dimensional space to four-dimensional space including the time dimension, includes the following sub-steps:

[0090] Step 6.1: Calculate the spatial gradient components at each time step to generate a three-dimensional gradient vector field; the input is the spatiotemporal phase field data. Calculated using the finite difference method , , Output three-dimensional spatial gradient field .

[0091] Step 6.2: Calculate the time partial derivative of the phase field to obtain the time evolution rate; perform difference operations on the phase data at adjacent time points: Output the time gradient component.

[0092] Step 6.3: Integrate spatial and temporal gradient components to construct a four-dimensional gradient vector field; combine the results of steps 6.1 and 6.2 into a four-dimensional gradient vector. It outputs a complete four-dimensional gradient field.

[0093] Furthermore, the construction of the four-dimensional gradient vector field needs to satisfy the following constraints: (1) The spatial gradient field must satisfy the irrotational condition, that is... This can be verified by the independence of the integral path in gradient calculation; (2) the spatiotemporal hybrid gradient must satisfy the energy conservation constraint, expressed as ,in (3) The gradient field should satisfy the defect boundary conditions at the defect boundary. There are ,in Let be the boundary normal vector. To ensure these constraints are satisfied, the Jacobi iterative method is used to regularize the gradient field in the numerical calculation. The iterative formula is:

[0094]

[0095] in This is the relaxation factor (with a value of 0.1). Let be the number of iterations, and let be the termination condition. ,in It should be set to The convergence threshold.

[0096] Step 6.4: Perform four-dimensional streamline tracing to predict the defect propagation path; starting from the initial position of the defect, perform numerical integration along the four-dimensional gradient direction. ,in It is a four-dimensional position vector. The arc length parameter is used to output the evolution trajectory of the defect in four-dimensional spacetime.

[0097] Furthermore, the numerical integration in four-dimensional streamline tracing is implemented using the fourth-order Runge-Kutta (RK4) method. The specific calculation steps are as follows: given the current position... Calculate the next position:

[0098]

[0099] in:

[0100] ;

[0101] ;

[0102] ;

[0103] ;

[0104] For a normalized four-dimensional gradient field, This is the integration step size.

[0105] Step length The selection adopts an adaptive strategy: ,in The maximum step size is set to 1 / 10 of the voxel size. This is the precision control parameter (value 0.01). Let be the spatial derivative of the four-dimensional gradient field. Estimated using finite difference. To ensure the stability of streamline tracing, when the magnitude of the four-dimensional gradient field... Less than the threshold When the time is reached, the tracking process is terminated, and it is determined that the defect has reached a stable position.

[0106] This embodiment describes the specific process of applying the multimodal imaging method for identifying product appearance defects during the manufacturing of aerospace-grade optical lenses. A certain aerospace company produces high-precision optical lenses that require extremely high uniformity; the refractive index variation caused by internal stress distribution cannot exceed 5 × 10⁻⁶. -6 Furthermore, stability is required during temperature changes. Traditional single-moment detection methods struggle to predict the evolution of internal stress in lenses under temperature variations in aerospace environments, leading to severe optical performance degradation in some lenses during actual use.

[0107] In practical applications, the following method is used to detect and predict the quality of a batch of fused silica lenses with a diameter of 120 mm and a thickness of 22 mm:

[0108] Implementation Step 1: Acquire multi-time-phase imaging data and extract temporal phase feature maps;

[0109] Interferometric imaging, polarization imaging, and scattering imaging devices equipped with a 532nm laser source were used to acquire images of the lens samples. Five temperature control points (22℃, 35℃, 60℃, 80℃, and 100℃) were set for controlled heating, and each temperature point was maintained for 30 minutes to ensure temperature stability before data acquisition. The acquired raw imaging data underwent image intensity normalization, spatial registration, and temporal synchronization correction. A Hilbert transform algorithm was then applied to extract the temporal phase feature map.

[0110] After preprocessing the collected data according to step 1, the instantaneous phase is calculated using the Hilbert transform algorithm;

[0111] Step 2: Process the time-series phase data and calculate the phase evolution matrix;

[0112] Based on the phase unwrapping algorithm, the interference fringe data obtained in step 1 are processed, and the wrapped phase in the range of [-π,π] is converted into a continuous phase distribution. The continuous phase data of adjacent time points are differentially processed, and the time-series phase evolution matrix representing the phase time change rate is output.

[0113] Figure 2 This study demonstrates the temporal variation trends of phase data acquired by three imaging modalities (interferometric imaging, polarization imaging, and scattering imaging) under different temperature conditions. The effectiveness of the Hilbert transform algorithm in extracting temporal phase features is verified. By comparing the response characteristics of different modalities during temperature changes, the sensitivity of each imaging method to internal stress defects can be evaluated.

[0114] Figure 3 This study demonstrates the risk level distribution of different regions of an optical lens under a high-temperature condition of 150℃. It also verifies the effectiveness of extending the phase field gradient propagation algorithm from three dimensions to four-dimensional spacetime, providing an intuitive risk assessment tool for the quality control of high-precision optical components.

[0115] Implementation Step 3: Construct a four-dimensional spatiotemporal graph network structure;

[0116] The three-dimensional phase field data obtained in step 2 are discretized into spatial grid nodes using a voxelization algorithm. =532nm, refractive index The parameter setting is 1.46, according to... Calculate the mesh size =0.045mm. The entire lens region was discretized into approximately 3.6 million voxel nodes. Edge connections between nodes were established based on phase continuity constraints. Corresponding spatial nodes in the time series were connected via temporal edges to generate a four-dimensional spatiotemporal graph network. The constructed four-dimensional spatiotemporal graph network data;

[0117] Figure 5 The detection accuracy performance of the patent application method under different temperature conditions and defect sizes is demonstrated. The effectiveness of the spatiotemporal graph convolutional network and hierarchical feature learning (step 5) is verified. The stability and reliability of the method under various working conditions are shown through scatter distribution, providing a performance evaluation basis for practical engineering applications.

[0118] Step 4: Extract spatiotemporal features using a multi-scale phase evolution kernel function;

[0119] The multi-scale phase evolution kernel function is applied to the nodes of the four-dimensional spatiotemporal graph network constructed in step 3, and spatiotemporal pattern features of phase change are extracted through convolution operations at different scales. In this embodiment, K=4 scales are selected, and the spatial bandwidth parameter is... The thicknesses are 0.09mm, 0.27mm, 0.81mm, and 2.43mm, respectively, with time and frequency parameters. The phase offset parameters are 0.52 rad / min, 0.26 rad / min, 0.13 rad / min, and 0.065 rad / min, respectively. They are uniformly distributed in the interval [0, 2π).

[0120] Figure 4 This section demonstrates the feature extraction capabilities of the multi-scale phase evolution kernel function at different scales in step 4. The effectiveness of the multi-scale kernel function design is verified, and the advantages of this method in capturing defect features at different spatial frequencies are showcased by comparing the feature extraction results at different scales.

[0121] Step 5: Perform hierarchical feature learning to generate defect evolution representations;

[0122] Based on a hierarchical graph pooling algorithm, the lens voxels are divided into 5 layers along the depth direction to construct a subgraph structure. A spatiotemporal graph convolutional network is used to learn the phase propagation law between each layer. By aggregating multi-layer features, a hierarchical defect evolution representation vector is generated, and finally, 3 potential stress defect regions are identified.

[0123] Step 6: Calculate the four-dimensional gradient field and predict the defect evolution trajectory;

[0124] The phase field gradient propagation algorithm was extended from three-dimensional space to four-dimensional space including the time dimension. The four-dimensional gradient was calculated, and a streamline tracing algorithm was applied to predict the evolution trend of these three defects under higher temperature environments (up to 150℃). Characteristic time scales. use Formula calculation, where =0.5, =22mm is the lens thickness is the thermal diffusivity of fused silica.

[0125] Based on the prediction results, defect D3 was identified as having a high risk. This sheet-like stress defect located in the lens edge region will have a phase value exceeding 1.5π at 150℃ and a geometric size increase of over 30%, potentially leading to significant degradation of optical performance. Defect D1 has a medium risk level, and defect D2 has a low risk level.

[0126] Furthermore, to ensure the consistency of the physical meaning of the phase values, a quality map-guided path planning method is employed in the phase unwrapping algorithm. This method effectively avoids unwrapping errors caused by noise regions and phase discontinuities, ensuring the physical continuity of the phase field. In the four-dimensional gradient field calculation, to ensure numerical stability, a central difference scheme is used to calculate the spatial gradient. This scheme has second-order accuracy and can effectively suppress numerical oscillations; for the time gradient, considering the large sampling interval, a forward difference scheme is adopted:

[0127] ;

[0128] Furthermore, the time gradient is smoothed by combining a Savitzky-Golay filter with a filter window size of 5 and a polynomial order of 2. This processing method not only preserves the trend characteristics of phase change over time, but also effectively suppresses the influence of measurement noise.

[0129] In practical applications, companies have adjusted their production processes accordingly, particularly optimizing annealing parameters to address the formation of sheet-like stress defects. By extending the annealing time in the 550℃-500℃ range and reducing the annealing cooling rate from 5℃ / hour to 2℃ / hour, the probability of stress defects forming in the edge region was successfully reduced, and the pass rate of lenses in high-temperature environment testing increased by 23.8%.

[0130] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. A product appearance defect recognition method based on multi-modal imaging, characterized by, The method comprises the following steps: acquiring imaging data of the transparent material at continuous time points by using a multi-modal imaging device, and extracting a time-series phase feature set after pre-processing the acquired imaging data; processing the time-series phase data, and calculating a phase evolution matrix, wherein the phase evolution matrix represents a time change rate of the phase; the calculation of the phase evolution matrix comprises: processing the interference fringe data at each time point by using a phase unwrapping algorithm, and converting the wrapped phase into a continuous phase distribution; performing a difference operation on the continuous phase data at adjacent time points, and outputting a time-series phase evolution matrix representing the time change rate of the phase; discretizing the three-dimensional phase field data at each time point into spatial grid nodes by using a voxelization algorithm, and constructing a four-dimensional space-time graph network fusing spatial topology and time evolution relationship; applying a multi-scale phase evolution kernel function to the four-dimensional space-time graph network, and extracting space-time pattern features of the phase change by convolution operation; constructing subgraph structures at different depth layers based on a hierarchical graph pooling algorithm, and generating defect evolution representation by using a space-time graph convolution network; calculating a four-dimensional gradient field, and predicting a space-time evolution trajectory of the defect by flow line tracking comprises: calculating spatial gradient components at each time point, and generating a three-dimensional gradient vector field; calculating a time partial derivative of the phase field, and obtaining a time evolution rate; integrating the spatial and time gradient components, and constructing a four-dimensional gradient vector field; starting from the initial position of the defect, performing numerical integration along the four-dimensional gradient direction, and predicting the evolution trajectory of the defect in the four-dimensional space-time. 2.The product appearance defect recognition method based on multi-modal imaging of claim 1, wherein, The acquiring imaging data of the transparent material at continuous time points by using a multi-modal imaging device, and extracting a time-series phase feature set after pre-processing the acquired imaging data comprises: acquiring imaging data sequences of the transparent material at continuous time points by using an interference imaging device, a polarization imaging device and a scattering imaging device respectively; performing image intensity normalization processing, spatial registration processing and time synchronization correction on the acquired original imaging data; applying a Hilbert transform algorithm to the interference imaging data at each time point, calculating instantaneous phase and instantaneous frequency, and generating a time-series feature set containing phase and frequency information. 3.The product appearance defect recognition method based on multi-modal imaging of claim 2, characterized in that, The spatial registration processing is implemented by using a phase cross-correlation algorithm, and the phase cross-correlation algorithm comprises: selecting the interference imaging data as a reference image; performing cross-correlation calculation on the polarization imaging data and the scattering imaging data with the reference image respectively, and obtaining cross-correlation matrices; finding peak positions in the cross-correlation matrices, and determining the best matching displacement; performing sub-pixel level registration by using a quadratic interpolation method. 4.The product appearance defect recognition method based on multi-modal imaging of claim 1, wherein, The discretizing the three-dimensional phase field data at each time point into spatial grid nodes by using a voxelization algorithm, and constructing a four-dimensional space-time graph network fusing spatial topology and time evolution relationship comprises: discretizing the three-dimensional phase field data at each time point into spatial grid nodes by using a voxelization algorithm; establishing edge connection relationship between nodes based on phase continuity constraint conditions; connecting corresponding spatial nodes in the time sequence through time edges, and generating a four-dimensional space-time graph network. 5.The product appearance defect recognition method based on multi-modal imaging of claim 4, wherein, The grid size in the voxelization algorithm is determined according to the following formula: the grid size is the wavelength of the light source divided by eight times the refractive index of the medium, which ensures that there are at least eight sampling points in the area with the fastest phase change, and meets the sampling requirement of the Nyquist sampling theorem on the spatial frequency. 6.The product appearance defect recognition method based on multi-modal imaging of claim 1, wherein, The application of the multi-scale phase evolution kernel function to the four-dimensional spatiotemporal graph network includes: Applying a multi-scale phase evolution kernel function to a four-dimensional spatiotemporal graph network node; Extracting phase-changing spatiotemporal pattern features through different scale convolution operations; Outputting a feature tensor containing multiple scale information. 7.The product appearance defect recognition method based on multi-modal imaging of claim 1, wherein, The construction of subgraph structures of different depth layers based on the hierarchical graph pooling algorithm and the generation of defect evolution representations using a spatiotemporal graph convolution network include: Constructing nodes of different depth layers into subgraph structures based on the hierarchical graph pooling algorithm; Learning phase propagation rules between layers using a spatiotemporal graph convolution network; Generating hierarchical defect evolution representation vectors by aggregating multi-layer features.

8. A multi-modal imaging based product appearance defect identification system configured to perform the method of any one of claims 1-7, wherein, It includes: A multi-modal imaging module for acquiring imaging data of a transparent material at consecutive time points; A data preprocessing module for preprocessing the acquired imaging data and extracting a set of time-series phase feature maps; A phase calculation module for processing time-series phase data and calculating a phase evolution matrix; A graph network construction module for constructing a four-dimensional spatiotemporal graph network; A feature extraction module for extracting phase-changing spatiotemporal pattern features; A defect representation module for generating hierarchical defect evolution representations; A trajectory prediction module for calculating a four-dimensional gradient field and predicting the spatiotemporal evolution trajectory of a defect.

Images