Transparent coating surface detection method based on space-spectrum separation imaging

Abstract

The invention provides a transparent coating surface detection method based on space-spectrum separation imaging, and the method comprises the steps: setting an imaging system which comprises a wide-spectrum illumination unit, a light path separation unit, a space position information obtaining unit and a spectrum information obtaining unit, and the spectrum information obtaining unit comprises a dispersion subunit; the light path separation unit divides light reflected after the wide-spectrum illumination unit irradiates the surface of an object into two paths and then sends the two paths of light to the spatial position information acquisition unit and the spectral information acquisition unit which are jointly calibrated; the spatial position information acquisition unit and the spectral information acquisition unit perform data acquisition on a to-be-measured object; the two types of data acquired by the spatial position information acquisition unit and the spectral information acquisition unit are accurately calibrated; and extracting space image features, feature wavebands and spectral curve features from the two types of data after accurate calibration, fusing the features, and detecting the surface of the transparent coating by adopting a teacher-student neural network.

Description

Technical Field

[0001] This invention relates to a surface inspection method, and more particularly to a transparent coating surface inspection method based on spatial-spectral separation imaging. Background Technology

[0002] The principle of surface inspection with transparent coating for detecting defects and coating thickness is as follows. (1) Thickness detection by the interference principle of transparent coating: The transparent coating will exhibit interference under light irradiation, and its reflection spectrum and coating thickness satisfy the following relationship.

[0003] ,

[0004] in, R1 and R2 are the reflection coefficients of air-film and film-substrate, respectively, n is the refractive index of the coating, d is the coating thickness, θ is the incident angle, and λ is the wavelength. The reflection spectrum will show periodic oscillations, and the oscillation frequency is proportional to the thickness d. By analyzing the reflection spectrum, the coating thickness can be deduced. (2) Defect detection principle: Defects on the surface of transparent coatings, such as scratches, bubbles, and particles, will change the local optical properties. For example, scratches cause local scattering and reflectivity changes, bubbles change the local thickness and refractive index, causing changes in the interference pattern, and particles introduce additional scattering and reflection. These changes will be reflected in spatial images and spectral data. In spatial images, defective areas show contrast changes and texture abnormalities. In spectral data, the spectral curve of defective areas is different from that of normal areas. By analyzing spatial images and reflection spectra, defects can be detected. The existing detection method is to use a single-wavelength laser to partially penetrate the coating and form reflections on both the coating surface and the substrate surface. However, the light spot received by the CMOS sensor is a mixed light spot, which causes the calculated position to be neither the coating surface nor the substrate surface, resulting in serious errors. Neural networks offer the possibility of accurately detecting transparent coated surfaces. Patent application number 202411181992.2 discloses a method and system for detecting surface defects in power adapters based on image processing. This method acquires images of the power adapter surface and performs spectral reflectance analysis on suspicious areas with defects. The method improves the comprehensiveness and accuracy of surface defect detection through convolutional neural networks. However, the spectral analysis in this patent is intended to further verify whether there are defects in the suspicious areas; it does not provide clear guidance for detecting surfaces with transparent coatings, especially for coating thickness detection. Summary of the Invention

[0005] The purpose of this invention is to provide a method for detecting the surface of transparent coatings based on spatial-spectral separation imaging, comprising:

[0006] Step S100: Set up the imaging system. The imaging system includes a broadband illumination unit, an optical path separation unit, a spatial position information acquisition unit, and a spectral information acquisition unit. The spectral information acquisition unit includes a dispersion sub-unit. The optical path separation unit splits the light reflected from the object surface after the broadband illumination unit illuminates it into two paths and sends them to the jointly calibrated spatial position information acquisition unit and spectral information acquisition unit.

[0007] Step S200: The spatial location information acquisition unit and the spectral information acquisition unit collect data from the object under test.

[0008] Step S300: Accurately calibrate the two types of data acquired by the spatial location information acquisition unit and the spectral information acquisition unit;

[0009] In step S400, spatial image features, feature bands, and spectral curve features are extracted and fused from the two types of data after precise calibration, and a teacher-student neural network is used to detect the transparent coating surface.

[0010] Furthermore, in step S100, the spatial location information acquisition unit and the spectral information acquisition unit are jointly calibrated. The calibration process includes:

[0011] Step S101: In the direction of motion of the object being measured, at the edge of the field of view or at a specific position within the field of view, set several physical reference marks with unique spatial and spectral characteristics as calibration objects.

[0012] Step S102: Scan the object under test at different locations to acquire spatial images and complete spectral data of the calibration object;

[0013] Step S103: Extract the coordinates (x, y) of the marker points of the calibration object from the spatial image. spatial , y spatial );

[0014] Step S104: Query the spectral features corresponding to the spectral data in the spectral data to determine the coordinates (X, Y, X) of the marker point in the spectral data cube. spectral , λ);

[0015] Step S105, based on multiple sets of data (x) at different scanning positions spatial , y spatial (X) spectral The perspective transformation matrix H is calculated from λ).

[0016] ,

[0017] Among them, Y dummy This is a placeholder.

[0018] Furthermore, in step S200, the spatial location information acquisition unit generates a high-resolution two-dimensional spatial image; the spectral information acquisition unit obtains a monochromatic light array separated by the dispersion subunit.

[0019] Furthermore, the specific process of step S300 includes:

[0020] Step S301, the three-dimensional data cube C obtained by the spectral information acquisition unit is... h Each spatial pixel (x) h ,y h The perspective transformation matrix H obtained through joint calibration yields the true-color image I acquired by the spatial location information acquisition unit. c The corresponding coordinates (x) c ,y c );

[0021] Step S302, from the three-dimensional data cube C h A hyperspectral pseudo-RGB image is synthesized by selecting several spectral bands with distinct characteristics. h,rgb ;

[0022] Step S303: Obtain the hyperspectral pseudo-RGB image I h,rgb and true color image I c The corresponding key feature point pair {P h ,P c};

[0023] Step S304, from the key feature point pair {P h ,P c Randomly select the transformation model H in} refined Minimum required sample set;

[0024] Step S305, use the current transformation matrix H i Test all key feature point pairs {P c ,P h}, calculate I for each hyperspectral pseudo-RGB image h,rgb The pixels are processed by H i After transformation and compared with true color image I c The distance between pixels, where i is the randomly selected round; if the error is less than the set threshold, the point is marked as an interior point;

[0025] Step S306: Count the number of interior points selected each time, and determine the round with the most interior points corresponding to H. i Let H be the transformation matrix with precise calibration. refined .

[0026] Furthermore, the teacher-student network described in step S400 is trained using the training set, including training the teacher network using the training set and the teacher network guiding the student network training. The feature fusion process during the teacher model training includes:

[0027] Step S411: Obtain the spatial image feature tensor F s,teacher Characteristic band tensor F b,teacher Spectral curve characteristic tensor F c,teacher Adjust the feature tensors of the three branches to have the same spatial size and number of channels;

[0028] Step S412: Generate queries, keys, and values ​​for the features of each branch, and perform preliminary fusion of the three feature tensors using multi-head self-attention and cross-attention to obtain... ,in ;

[0029] Step S413: Using a combination of channel attention and spatial attention, calculate the channel weights of the preliminary fused features. and spatial weights ;

[0030] Step S414: Apply attention weights to each branch feature using element-wise multiplication to obtain...

[0031] ;

[0032] Step S415, for The fused feature F is obtained by splicing. fusion

[0033] .

[0034] Furthermore, in step S411, the last three layers of the ResNet50 network are used to extract the spatial feature tensor F from the spatial image. s,teacher Spatial features include {texture features, shape features, color features, gradient features, and structural features}.

[0035] Furthermore, in step S411, multiple convolutional and pooling layers are used to extract spatial-spectral features from the dispersed spectral image. Spatial attention is then applied to these features using a spatial attention module to obtain the feature band tensor F. b,teacherSpatial-spectral features include interference features and thickness-sensitive features. Interference features include extremum distribution and interference modes. Extremum distribution includes {peak wavelength position, valley wavelength position, extremum spacing, and extremum intensity}. Interference modes include {oscillation frequency, oscillation amplitude, phase information, and envelope shape}. Thickness-sensitive features include wavelength shift features and intensity variation features. Wavelength shift features include {peak drift, spacing variation, and shape distortion}. Intensity variation features include {reflectivity variation, contrast variation, and slope variation}.

[0036] Furthermore, in step S411, 1D convolution is used to process the spectrum of each point along the spectral dimension to obtain the spectral curve features, and 2D convolution is used for spatial dimension processing to obtain the spectral curve feature tensor F. c,teacher The spectral curve features include {peak value, valley value, peak spacing, oscillation amplitude, and overall spectral shape}.

[0037] Furthermore, the specific process of step S412 includes:

[0038] Step S4121: For the features of each branch, generate query Q, key K and value V vectors respectively using linear transformation;

[0039] Step S4122: Divide the Q, K, and V of each branch into multiple heads along the channel dimension, and perform attention calculation for each head;

[0040] Step S4123: Using the spatial image branch as the query, perform cross-attention calculation for the i-th head.

[0041]

[0042]

[0043]

[0044] Among them, D k The number of channels for each head, This indicates the concatenation of the keys of the three branches. This indicates the concatenation of values ​​from the three branches;

[0045] In step S4124, each branch obtains two sets of features: one set is the features obtained after the spatial image interacts with the other two branches as a query, and the other set is its own value features. These features are weighted and summed to obtain the preliminary fusion features of each branch.

[0046] Furthermore, the teacher provides online guidance for the student's online training. After training, the student's network generates a defect probability map P based on the precisely calibrated two types of data. student and thickness map T student .

[0047] Compared with the prior art, the present invention has the following advantages: (1) It uses spatial location information and spectral information to detect the surface of the transparent coating; (2) The spatial image and the spectral image are calibrated twice to ensure that the physical points on the spatial image and the hyperspectral data are completely corresponding; (3) The transparent coating surface is detected by a trained teacher-student network. The teacher network guides the student network to train, and the student network is more suitable for edge devices with limited computing resources; (4) The spatial image features, feature bands and spectral curve features are fused by feature fusion in the neural network. The feature bands and spectral curve features have spatial attributes, which makes it convenient to accurately locate defects and areas with different thicknesses.

[0048] The present invention will now be further described with reference to the accompanying drawings. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0050] Combination Figure 1 A method for detecting transparent coating surfaces based on spatial-spectral separation imaging, comprising:

[0051] Step S100: The imaging system is configured to split the light reflected from the object surface into two paths and send them to the subsequent spatial position information acquisition unit and spectral information acquisition unit. The imaging system includes a broadband illumination unit, a light path separation unit, a spatial position information acquisition unit, and a spectral information acquisition unit.

[0052] Step S200: The spatial location information acquisition unit and the spectral information acquisition unit collect data from the object under test.

[0053] Step S300: Accurately calibrate the two types of data acquired by the spatial location information acquisition unit and the spectral information acquisition unit;

[0054] In step S400, spatial image features, feature bands, and spectral curve features are extracted and fused from the two types of data after precise calibration, and a teacher-student neural network is used to detect the transparent coating surface.

[0055] In step S100, the broadband illumination unit uses a light source with a continuous spectrum, such as a halogen lamp, xenon lamp, or supercontinuum laser, typically covering the visible to near-infrared region (e.g., 400nm - 1000nm). The optical path separation unit includes a beam splitter and lenses located on the two optical paths after beam splitting, splitting the composite reflected light (containing spatial and spectral information) returned from the surface of the object under test into two beams. The spatial position information acquisition unit is a high-resolution, high dynamic range area array CCD or CMOS camera that captures a high-definition two-dimensional image of the surface of the object under test; this image provides the precise spatial coordinates (x,y) of each pixel; the contrast changes in the image can be directly used to identify macroscopic, high-contrast defects, such as scratches, dirt, and bubbles. The spectral information acquisition unit is an imaging spectrometer, which internally includes an entrance slit, a collimating lens, a grating (dispersive element), an imaging mirror, and an area array detector. The object under test is imaged onto the slit through a lens. The light from each point on the slit is dispersed by the grating, forming a three-dimensional data cube (x, y, λ) on the detector, where λ is the wavelength. The row pixel direction of the area array CCD / CMOS camera, the slit direction of the imaging spectrometer, and the motion direction of the object under test are set parallel to each other. The motion direction of the object is perpendicular to the row direction of the camera, and the dispersion direction (wavelength unfolding direction) within the imaging spectrometer is perpendicular to the slit direction. The sensor plane of the area array CCD / CMOS camera and the entrance slit plane of the imaging spectrometer must be located on the imaging plane of their respective lens barrels.

[0056] For imaging systems, joint system calibration is required. The calibration process is as follows:

[0057] Step S101: On the direction of motion (scanning direction) of the object being measured, set several known physical reference marks with unique spatial and spectral characteristics as calibration objects at the edge of the field of view or at specific positions within the field of view; in this embodiment, a set of marks with specific fine geometric shapes (such as crosshairs, checkerboard patterns) and known reflectance spectra are used.

[0058] Step S102: Scan the object under test at different locations to acquire spatial images and complete spectral data of the calibration object;

[0059] Step S103: Extract the coordinates (x, y) of the marker points of the calibration object from the spatial image. spatial , y spatial For example, the geometric center (crosshairs) or the feature corner (chessboard).

[0060] Step S104: Query the spectral features corresponding to the spectral data in the spectral data to determine the coordinates (X, Y, X) of the marker point in the spectral data cube. spectral , λ);

[0061] Step S105, based on multiple sets of data (x) at different scanning positions spatial , y spatial (X) spectral The perspective transformation matrix H is calculated using the given information (λ), which establishes the mapping relationship between the space camera pixel coordinate system and the spectral detector pixel coordinate system.

[0062] ,

[0063] Among them, Y dummy This is a placeholder.

[0064] In step S200, once the velocity of the object under test stabilizes, the spatial position information acquisition unit and the spectral information acquisition unit simultaneously acquire data. The area array CCD / CMOS camera captures a global, high-resolution two-dimensional image, i.e., a spatial image; the imaging spectrometer performs line scanning according to a preset line frequency. All images and spectral data are transmitted in real time to the industrial control computer for data processing via Camera Link or CoaXPress interface.

[0065] In step S300, the data acquired by the spatial location information acquisition unit and the spectral information acquisition unit are fused to obtain the spatial and spectral fused data features. In reality, a tiny scratch may only occupy a few pixels in a spatial image. If there is a 1-2 pixel error in the calibration of the data collected by the spatial location information acquisition unit and the spectral information acquisition unit from the object under test, the fusion algorithm may fuse the edge texture of the scratch with the spectrum of the adjacent normal area, causing the network to fail to learn the correct "scratch features," resulting in missed or false detections. Therefore, before fusion, it is necessary to recalculate the mapping relationship between the spatial camera pixel coordinate system and the spectral detector pixel coordinate system to obtain a more accurate transformation model H suitable for the actual object under test. refined The specific process is as follows:

[0066] Step S301, the three-dimensional data cube C obtained by the spectral information acquisition unit is... h Each spatial pixel (x) h ,y h The true-color image I obtained by the spatial location information acquisition unit is obtained through the perspective transformation matrix H. c The corresponding coordinates (x) c ,y c );

[0067] Step S302, from the three-dimensional data cube C h A hyperspectral pseudo-RGB image is synthesized by selecting several spectral bands with distinct characteristics. h,rgb ;

[0068] Step S303: Obtain the hyperspectral pseudo-RGB image I h,rgb and true color image I c The corresponding key feature point pair {P h ,P c};

[0069] Step S304, from the key feature point pair {P h ,P c Randomly select the transformation model H in} refined Minimum required sample set;

[0070] Step S305, use the current transformation matrix H i Test all key feature point pairs {P c ,P h}, calculate I for each hyperspectral pseudo-RGB image h,rgb The pixel (source point) passes through H i After transformation and compared with true color image I c The distance to the target pixel is i, where i is the randomly selected round; if the error is less than the set threshold, such as 2 pixels, then the point is marked as an inner point.

[0071] Step S306: Count the number of interior points selected each time, and determine the round with the most interior points corresponding to H. i Let it be H refined .

[0072] Through H refined The transformation ensures that (x,y) in the spatial image and (x,y) in the hyperspectral data represent exactly the same physical points.

[0073] The key feature points in step S303 are preferably physical features that are stable in shape under different spectral bands, such as high-contrast marks, edges, and corners on the object under test, or micro-reference marks (such as laser-etched cross lines) that are deliberately left during calibration and will not interfere with the detection area.

[0074] Using only a single feature to detect surfaces with transparent coatings cannot yield accurate results, but each feature contributes to surface detection. Therefore, it is necessary to use fused features to obtain surface information. In step S400, the recalibrated spatial image and the scanning image from the imaging spectrometer are used to obtain spatial image features, feature bands, and spectral curve features through spatial image branching, feature band branching, and spectral curve branching, respectively.

[0075] The spatial image branch processes the first reflected light to generate a high-resolution two-dimensional spatial image. The extracted multi-scale spatial features, including surface texture, edges, shape context, and local contrast, capture macroscopic and microscopic structural information about the surface, such as anomalous texture or shape variations in defective areas. The spatial image branch directly provides spatial location information for defects. Since defects (such as scratches or spots) typically appear as local anomalies (such as brightness variations or texture breaks) in spatial images, analyzing spatial features allows for precise identification of the defect's coordinates, size, and shape.

[0076] The characteristic band branch processes the spectral information of the second reflected light. After grating decomposition, specific characteristic bands (such as key absorption or reflection bands of the coating material) are selected for analysis. Extracted features include intensity values, band ratios, spectral derivatives, or statistical characteristics (such as mean and variance) within the characteristic bands. The characteristic band branch is used for thickness measurement of transparent coatings. Based on the principle of thin-film interference, changes in coating thickness cause reflectivity oscillations at specific wavelengths. By analyzing the intensity changes in the characteristic bands, a correspondence between thickness and spectrum can be established, providing initial clues for thickness estimation.

[0077] The spectral curve branch processes the complete spectral information of the second reflected light, acquiring a continuous spectral curve for each pixel. Extracted features include shape features of the spectral curve (such as peaks, valleys, and slopes), global statistical features (such as curve integrals and variance), or frequency domain features obtained through transformations (such as Fourier transforms). The spectral curve branch detects defects by analyzing spectral anomalies at each pixel. Defective regions may exhibit spectral curves different from normal regions (such as absorption peak shifts or shape distortions), thus aiding in defect localization.

[0078] Due to the complexity of interference models, such as multilayer films and changes in incident angle, traditional model fitting methods may not be accurate enough, while neural networks can learn more complex mappings. For coating thickness identification, neural networks learn the mapping relationship between interference spectra and thickness through spectral curves and characteristic bands: for spectral curves, neural networks can learn the entire oscillation mode, thus accurately estimating the thickness; for several characteristic bands that are sensitive to thickness, these bands may be located near the extreme points of oscillation and are sensitive to thickness changes. For defect identification, neural networks detect defects by fusing spatial images and cross-branch attention: for spatial images, CNNs are used to extract spatial image features, capturing local texture and shape changes caused by defects; for cross-branch attention, spectral features and spatial image features are fused, enabling the network to simultaneously utilize spectral anomalies and spatial anomalies to locate defects.

[0079] This embodiment uses a teacher-student neural network to obtain the detection results of the transparent coating surface. The teacher-student network employs different methods for acquiring and fusing the three features during training. The purpose of choosing a teacher-student neural network in this embodiment is that the detection of defects and coating thickness in the transparent coating surface detection process requires more refined features, necessitating a larger and more complex model. Edge devices, such as area-array CCD or CMOS cameras, have limited computing resources and require smaller and faster models. Therefore, a teacher-student network is chosen, using the teacher model to guide the student model during training. The student model, by mimicking the output of the teacher model, significantly improves inference speed while maintaining accuracy.

[0080] The teacher network structure comprises three feature acquisition branches, a self-attention mechanism module, a multi-head output concatenation module, a residual connection layer, a normalization layer, a channel attention module, a spatial attention module, a multiplication module, and a concatenation module. The three feature acquisition branches are: a spatial feature acquisition branch, a feature band acquisition branch, and a spectral curve feature branch. The spatial feature acquisition branch employs a deep convolutional neural network, such as ResNet50. The feature band acquisition branch includes multiple convolutional layers and pooling layers, with an attention module following the pooling layers. The spectral curve feature branch includes one 1D convolutional layer and one 2D convolutional layer. The self-attention mechanism module fuses the three feature tensors through multi-head self-attention and cross-attention. The residual connection layer adds the attention output of each branch to the original features. The normalization layer normalizes the output of the residual connection layer. The feedforward network includes two linear transformations and a non-linear activation function (such as ReLU), with a hidden layer in between. The second linear transformation layer is followed by the residual connection layer and the normalization layer. The channel attention module includes a global average pooling layer and a fully connected layer. The spatial attention module includes convolutional layers and fully connected layers. The teacher network uses H... refined The labels generated by the physical model are used for training, and the loss function includes segmentation loss and regression loss, with physical consistency loss also included.

[0081] Feature fusion during teacher model training includes:

[0082] Step S411: Use a deep CNN, such as ResNet50, to extract the spatial feature tensor F. s,teacher Specifically, the spatial image is input into the ResNet50 network. The feature maps of the last three stages of the ResNet50 network are upsampled and stitched together to obtain the multi-scale spatial feature tensor F. s,teacher ;

[0083] In step S412, after grating decomposition, the second path reflected light image is processed through multiple convolutional and pooling layers to extract the spatial-spectral features of the feature bands. Spatial attention is then applied to the spatial-spectral features using a spatial attention module to obtain the feature band tensor F. b,teacher ;

[0084] Step S413: The spectral curve is processed along the spectral dimension using 1D convolution to obtain the spectral curve features at each point, and then processed in the spatial dimension using 2D convolution to obtain the spectral curve feature tensor F. c,teacher ;

[0085] Step S414: Adjust the feature maps of the three branches to have the same spatial size and number of channels;

[0086] Step S415: Generate queries, keys, and values ​​for the features of each branch, and perform preliminary fusion of the three feature tensors using multi-head self-attention and cross-attention to obtain... ,in ;

[0087] Step S416: Calculate the channel weights of the preliminary fused features using a combination of channel attention and spatial attention. and spatial weights ;

[0088] Step S417: Apply attention weights to each branch feature using element-wise multiplication to obtain...

[0089] ;

[0090] Step S418, for F is obtained by splicing. fusion

[0091] ;

[0092] The final fused feature is obtained by reducing the number of channels through a 1×1 convolution.

[0093] The spatial features mentioned in step S411 include {texture features, shape features, color features, gradient features, and structural features}. Texture features are the microstructure of the surface, such as roughness, graininess, and texture. These features can help distinguish normal surfaces from areas with defects (such as scratches, pits, etc.). Shape features are the macroscopic geometry of the surface, such as edges, contours, and curvature. These features help detect surface deformation, depressions, or protrusions. Color features are the color distribution and intensity of the surface. In transparent coating detection, color features provide useful information. Gradient features are the direction and magnitude of changes in image intensity. Structural features are the regular or repetitive patterns of the surface, such as periodic or random textures. ResNet50 contains four layers, namely layer 1 to layer 4. Layers 2-4 output spatial map features. Layer 2 outputs shallow features, such as low-level features like texture, color, and gradient. Layers 3 and 4 output high-level semantic features including shape and structure.

[0094] The characteristic bands in step S412 are key wavelength regions selected from the complete spectrum. These bands are highly sensitive to variations in the thickness of the transparent coating and surface defects. The resulting spectral data cube (X) is obtained after dispersion. spectral The region (λ) contains detailed spatial and spectral information, thus spatial-spectral features can be obtained through convolutional and pooling layers. These features include interference characteristics and thickness-sensitive characteristics. Interference characteristics include extremum distribution and interference modes. Extremum distribution includes {peak wavelength position, valley wavelength position, extremum spacing, and extremum intensity}, where peak wavelength position is the wavelength at which reflectivity is maximum, valley wavelength position is the wavelength at which reflectivity is minimum, extremum spacing is the wavelength interval between adjacent peaks / valleys, and extremum intensity is the reflectivity of the peak / valley. Interference modes include {oscillation frequency, oscillation amplitude, phase information, and envelope shape}, where oscillation frequency is the speed of spectral oscillation, oscillation amplitude is the intensity range of reflectivity variation, phase information is the phase relationship of the interference fringes, and envelope shape is the overall contour of the oscillation. Thickness-sensitive features include wavelength shift features and intensity variation features; wavelength shift features include {peak drift, spacing variation, shape distortion}, where peak drift is the shift of peak wavelength caused by thickness variation, spacing variation is the change in the spacing between extreme points with thickness, and shape distortion is the change in the shape of the interference pattern; intensity variation features include {reflectivity variation, contrast variation, slope variation}, where reflectivity variation is the reflectivity intensity at a specific wavelength, contrast variation is the contrast between peaks and valleys, and slope variation is the steepness of the spectral curve.

[0095] Each band responds differently to thickness variations, necessitating the extraction of band-specific features. Therefore, step S412 incorporates multiple convolutional layers, performing convolution operations on each band individually to extract local spatial features. Furthermore, correlations exist between different bands, potentially including thickness information such as peak shifts and amplitude variations. Consequently, step S412 includes a single convolutional layer that uses multiple bands as channels for cross-band convolution to capture these inter-band relationships. The pooling layer in step S412 reduces spatial resolution, increases the receptive field, and preserves important features. Finally, the spatial attention module in step S412 applies spatial attention to the feature map, directing the network to focus on spatial regions related to thickness or defects.

[0096] The spectral curve features described in step S413 include {peak value, valley value, peak spacing, oscillation amplitude, and overall spectral shape}. The 1D convolutional kernels in step S413 are learned to act as derivative operators to detect change points, where the first derivative of the peak and valley values ​​is zero, and the second derivative is negative or positive. Multiple convolutional kernels can detect peaks and valleys at different scales, thereby capturing the peak spacing. The oscillation amplitude is obtained by detecting the response intensity of the convolutional kernels through the envelope. A deep 1D convolutional network is used to obtain shallow {peak value, valley value} features, mid-layer {spacing, amplitude} features, and deep {overall trend, curvature} features, resulting in the overall spectral shape. These features ({peak value, valley value, peak spacing, oscillation amplitude, and overall spectral shape}) are rearranged into a spatial grid, and 2D convolution is performed on these feature maps. The spatial convolution model can integrate information from adjacent pixels, thereby capturing patterns such as the regionality of imperfections and spatial variations in thickness.

[0097] The specific process of step S415 includes:

[0098] Step S4151: For the features of each branch, generate query, key, and value vectors using linear transformations respectively;

[0099] Step S4152: Divide the Q, K, and V of each branch into multiple heads along the channel dimension, perform attention calculation for each head, use the spatial image branch as the query, and perform cross-attention calculation for the i-th head.

[0100]

[0101]

[0102]

[0103] Among them, D k The number of channels for each head, This indicates the concatenation of the keys of the three branches. This indicates that the values ​​of the three branches are concatenated, so that the query of each branch will interact with the keys and values ​​of all branches, thereby achieving cross-branch information fusion;

[0104] In step S4153, each branch will obtain two sets of features: one set is the features obtained after the spatial image interacts with the other two branches as a query, and the other set is its own value features. These features are weighted and summed to obtain the preliminary fusion features of each branch.

[0105] In step S416, channel attention is calculated. The channel weights are calculated by using global average pooling (GAP) to compress the spatial dimension of each channel, and then generating a channel weight vector through a fully connected layer (containing an activation function such as sigmoid).

[0106] ,

[0107] In step S416, spatial attention is calculated. Spatial weight Specifically, for Spatial weights are generated through 1×1 convolution and Sigmoid.

[0108] .

[0109] The student network structure is similar to the teacher network structure. When training the student network, the probabilistic and thickness maps generated by the teacher network are used to guide its training. Specifically, the student network's loss functions include hard-label loss, distillation loss, and MSE loss. Distillation loss is calculated using the probabilistic and thickness maps generated by the teacher network and the student network's output. MSE loss uses KL divergence to match the probability distributions of the teacher and student networks.

[0110] In the student network, the spatial image branch, one of the three feature extraction branches, uses a lightweight CNN, such as MobileNetV2, to extract spatial features. Early layers of MobileNetV2 obtain shallow features, including texture, color, and gradient features, while intermediate layers obtain deep features, including shape and structure features. These features are then fused to obtain the spatial feature tensor F. s,student Compared to the teacher network, which uses ResNet50, the teacher network extracts richer and more refined features, especially deep features with stronger semantic information. The student network, due to its lightweight model and simplified structure, is slightly weaker in feature richness and abstraction capabilities, but through multi-scale fusion, it still retains sufficient information for surface defect detection and thickness measurement.

[0111] The feature band branch in the student network uses fewer convolutional layers and fewer filters to obtain F. b,student The characteristic band branches of the student network and the teacher network may differ structurally, and therefore the richness and abstraction of the extracted features may differ. However, through knowledge distillation, the student network can learn to imitate the features of the teacher network. Although the representation may be different, the features extracted by the student network and the features extracted by the teacher network are functionally similar.

[0112] The spectral curve branch uses a simple 1D convolution and fully connected layer to obtain the spectral curve feature tensor F. c,student The student network employs lightweight channel attention, lightweight spatial attention, and single-head attention to fuse features. Lightweight channel attention uses global average pooling and two fully connected layers, but with a higher channel compression ratio to reduce parameters. Lightweight spatial attention uses only average pooling and a small convolutional kernel (3x3) to generate the spatial attention map. Single-head attention uses cosine similarity to calculate attention weights, avoiding complex matrix operations.

[0113] The student network includes a defect segmentation head and a thickness estimation head as output heads. The defect segmentation head uses a decoder composed of several upsampling convolutional layers to upsample the fused feature maps to the original image size, outputting a defect probability map P. student The thickness estimation head also uses several upsampling convolutional layers to construct the decoder output thickness map T. student .

[0114] Defect probability diagram P student It is a two-dimensional matrix image, with its spatial dimensions consistent with the spatial image. Each pixel value in the matrix is ​​a floating-point number between 0 and 1. The closer the floating-point value of a pixel is to 1, the more confident the network is that the pixel belongs to a defective area, such as scratches, dents, or blemishes; the closer the floating-point value of a pixel is to 0, the more confident the network is that the pixel belongs to a normal, defect-free background area. A threshold is set, and the two-dimensional matrix image is compared with the threshold to generate a binary mask, which is directly used to calculate the number, area, and location of defects.

[0115] Thickness map T student It is a two-dimensional matrix corresponding to the size of the input spatial image. Each pixel value in the matrix is ​​a non-negative real number, which represents the physical thickness of the transparent coating at that pixel point estimated by the network. It is used to calculate statistical indicators such as the average thickness, maximum / minimum thickness, and thickness uniformity (standard deviation) of the entire workpiece.

Claims

1. A method for detecting the surface of a transparent coating based on spatial-spectral separation imaging, characterized in that, include: Step S100: Set up the imaging system. The imaging system includes a broadband illumination unit, an optical path separation unit, a spatial position information acquisition unit, and a spectral information acquisition unit. The spectral information acquisition unit includes a dispersion sub-unit. The optical path separation unit splits the light reflected from the object surface after the broadband illumination unit illuminates it into two paths and sends them to the jointly calibrated spatial position information acquisition unit and spectral information acquisition unit. Step S200: The spatial location information acquisition unit and the spectral information acquisition unit collect data from the object under test. Step S300: Accurately calibrate the two types of data acquired by the spatial location information acquisition unit and the spectral information acquisition unit; In step S400, spatial image features, feature bands, and spectral curve features are extracted and fused from the two types of data after precise calibration, and a teacher-student neural network is used to detect the transparent coating surface.

2. The method according to claim 1, characterized in that, In step S100, the spatial location information acquisition unit and the spectral information acquisition unit are jointly calibrated. The calibration process includes: Step S101: In the direction of motion of the object being measured, at the edge of the field of view or at a specific position within the field of view, set several physical reference marks with unique spatial and spectral characteristics as calibration objects. Step S102: Scan the object under test at different locations to acquire spatial images and complete spectral data of the calibration object; Step S103: Extract the coordinates (x, y) of the marker points of the calibration object from the spatial image. spatial , y spatial ); Step S104: Query the spectral features corresponding to the spectral data in the spectral data to determine the coordinates (X, Y, X) of the marker point in the spectral data cube. spectral , λ); Step S105, based on multiple sets of data (x) at different scanning positions spatial , y spatial (X) spectral The perspective transformation matrix H is calculated from λ). ， Among them, Y dummy This is a placeholder.

3. The method according to claim 1, characterized in that, In step S200, the spatial location information acquisition unit generates a high-resolution two-dimensional spatial image; the spectral information acquisition unit obtains a monochromatic light array separated by the dispersion subunit.

4. The method according to claim 1, characterized in that, The specific process of step S300 includes: Step S301, the three-dimensional data cube C obtained by the spectral information acquisition unit is... h Each spatial pixel (x) h ,y h The perspective transformation matrix H obtained through joint calibration yields the true-color image I acquired by the spatial location information acquisition unit. c The corresponding coordinates (x) c ,y c ); Step S302, from the three-dimensional data cube C h A hyperspectral pseudo-RGB image is synthesized by selecting several spectral bands with distinct characteristics. h,rgb ; Step S303: Obtain the hyperspectral pseudo-RGB image I h,rgb and true color image I c The corresponding key feature point pair {P h ,P c }; Step S304, from the key feature point pair {P h ,P c Randomly select the transformation model H in} refined Minimum required sample set; Step S305, use the current transformation matrix H i Test all key feature point pairs {P c ,P h }, calculate I for each hyperspectral pseudo-RGB image h,rgb The pixels are processed by H i After transformation and compared with true color image I c The distance between pixels, where i is the randomly selected round; if the error is less than the set threshold, the point is marked as an interior point; Step S306: Count the number of interior points selected each time, and determine the round with the most interior points corresponding to H. i Let H be the transformation matrix with precise calibration. refined .

5. The method according to claim 1, characterized in that, The teacher-student network described in step S400 is trained using the training set, including training the teacher network using the training set and the teacher network guiding the student network training. The feature fusion process during the teacher model training includes: Step S411: Obtain the spatial image feature tensor F s,teacher Characteristic band tensor F b,teacher Spectral curve characteristic tensor F c,teacher Adjust the feature tensors of the three branches to have the same spatial size and number of channels; Step S412: Generate queries, keys, and values ​​for the features of each branch, and perform preliminary fusion of the three feature tensors using multi-head self-attention and cross-attention to obtain... ,in ; Step S413: Using a combination of channel attention and spatial attention, calculate the channel weights of the preliminary fused features. and spatial weights ; Step S414: Apply attention weights to each branch feature using element-wise multiplication to obtain... ； Step S415, for The fused feature F is obtained by splicing. fusion 。 6. The method according to claim 5, characterized in that, In step S411, the last three layers of the ResNet50 network are used to extract the spatial feature tensor F from the spatial image. s,teacher Spatial features include {texture features, shape features, color features, gradient features, and structural features}.

7. The method according to claim 5, characterized in that, In step S411, multiple convolutional and pooling layers are used to extract spatial-spectral features from the dispersed spectral image. Spatial attention is then applied to these features using a spatial attention module to obtain the feature band tensor F. b,teacher Spatial-spectral features include interference features and thickness-sensitive features. Interference features include extremum distribution and interference modes. Extremum distribution includes {peak wavelength position, valley wavelength position, extremum spacing, and extremum intensity}. Interference modes include {oscillation frequency, oscillation amplitude, phase information, and envelope shape}. Thickness-sensitive features include wavelength shift features and intensity variation features. Wavelength shift features include {peak drift, spacing variation, and shape distortion}. Intensity variation features include {reflectivity variation, contrast variation, and slope variation}.

8. The method according to claim 6, characterized in that, In step S411, 1D convolution is used to process the spectrum of each point along the spectral dimension to obtain the spectral curve features, and 2D convolution is used for spatial dimension processing to obtain the spectral curve feature tensor F. c,teacher The spectral curve features include {peak value, valley value, peak spacing, oscillation amplitude, and overall spectral shape}.

9. The method according to claim 5, 6, 7, or 8, characterized in that, The specific process of step S412 includes: Step S4121: For the features of each branch, generate query Q, key K and value V vectors respectively using linear transformation; Step S4122: Divide the Q, K, and V of each branch into multiple heads along the channel dimension, and perform attention calculation for each head; Step S4123: Using the spatial image branch as the query, perform cross-attention calculation for the i-th head. Among them, D k The number of channels for each head, This indicates the concatenation of the keys of the three branches. This indicates the concatenation of values ​​from the three branches; In step S4124, each branch obtains two sets of features: one set is the features obtained after the spatial image interacts with the other two branches as a query, and the other set is its own value features. These features are weighted and summed to obtain the preliminary fusion features of each branch.

10. The method according to claim 9, characterized in that, Teachers provide online guidance for students' online training. After training, the student networks generate a defect probability map P based on precisely calibrated data from two categories. student and thickness map T student .

Images