Defect detection system and method based on multispectral analysis

Abstract

The invention discloses a defect detection system and method based on multispectral analysis, and particularly relates to the technical field of industrial nondestructive testing, and the system comprises a spectrum modulation excitation module, a high-dimensional data acquisition module, an intelligent fusion processing module, a self-adaptive decision module and a closed-loop feedback control module, a customizable high-intensity narrow-band excitation light beam is used for irradiating a detected object, multi-channel images are synchronously collected to construct a high-dimensional data cube including space, spectrum and time sequence information, a deep learning network with a spectrum-space attention mechanism is used for adaptively fusing multi-spectral features, a feature map with highlighted defects is generated, and the feature map with highlighted defects is obtained. An online incremental learning classifier is adopted for recognition and classification, a model is dynamically updated, an excitation strategy is reversely optimized in real time according to a decision result, a self-adaptive detection closed loop is formed, high-sensitivity, high-precision and self-adaptive detection of complex defects is achieved, and the intelligent level and the continuous optimization capacity of a detection system are improved.

Description

Technical Field

[0001] This invention relates to the field of industrial nondestructive testing technology, and more specifically, to a defect detection system and method based on multispectral analysis. Background Technology

[0002] In the field of industrial manufacturing and quality control, defect detection is a key link in ensuring product reliability and safety. Traditional manual visual inspection methods are inefficient, subjective, and prone to fatigue, making it difficult to meet the requirements of modern industry for high precision and high efficiency. Therefore, automated and intelligent non-destructive testing technology has become an inevitable trend.

[0003] Existing technologies employ visual inspection systems with fixed-band or multispectral imaging. These systems typically use several pre-set broadband light sources or LEDs with fixed wavelengths to illuminate the object under test. An industrial camera captures reflected or transmitted images, and then traditional image processing algorithms (such as threshold segmentation and edge detection) or shallow machine learning models are used to identify and classify defects in the images.

[0004] However, in practical use, it still has some drawbacks. For example, the spectrum of the excitation source is fixed and the bandwidth is wide, making it difficult to excite the most significant optical response for specific materials or tiny defects, resulting in a low detection signal-to-noise ratio. It is particularly insensitive to internal defects or complex contaminants. The processing method for multispectral data is simple, failing to deeply mine and adaptively weight and fuse the most critical defect features from multiple dimensions of space, spectrum and time. It lacks self-optimization ability. Once the detection model is deployed, it remains fixed and cannot be learned and updated online based on newly emerging defect samples. Furthermore, the detection parameters cannot be adjusted in a closed loop based on real-time results, resulting in poor adaptability and a rapid decline in detection accuracy when facing new materials or defect types. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a defect detection system and method based on multispectral analysis, which solves the problems mentioned in the background art through the following solutions.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a defect detection system and method based on multispectral analysis, comprising a spectral modulation excitation module: a programmable multi-band LED light source array and an acousto-optic tunable filter, generating a time-alternating, spectrally customized high-intensity narrowband excitation beam according to the material properties of the object under test, and pre-matching the excitation spectrum combination through a built-in material spectral library; High-dimensional data acquisition module: Based on a scientific-grade camera and high-speed image buffer unit with a mercury cadmium telluride array detector, it synchronously captures multi-channel image sequences of the test object excited by the spectral modulation excitation module under different narrowband spectra, and constructs a high-dimensional data cube including spatial, spectral and temporal dimensions in real time. Intelligent fusion processing module: It has a built-in deep convolutional neural network with attention mechanism to perform end-to-end feature extraction and information fusion on high-dimensional data cubes. Through a dedicated spectral-spatial attention sub-network, it adaptively learns and fuses the features most sensitive to defects in different spectral channels to generate a fusion feature map that highlights defects. Adaptive Decision Module: Receives fused feature maps, uses an online incremental learning classifier based on transfer learning to identify and classify defects, dynamically updates its classification model based on newly emerging defect samples, and outputs a structured report; Closed-loop feedback control module: Connected to the adaptive decision module and the spectral modulation excitation module, it adjusts the excitation spectrum strategy and parameters of the spectral modulation excitation module in real time according to the decision results, forming a self-optimizing detection closed loop.

[0007] S1: Spectral modulation excitation step, based on the material properties of the test object, matches and drives a programmable light source and filter from a preset spectral library to generate a high-intensity narrowband excitation beam with alternating time and customized spectrum to irradiate the test object; S2: High-dimensional data acquisition step, synchronously capturing multi-channel image sequences of the analyte under different customized narrowband spectra, and constructing a high-dimensional data cube including spatial, spectral and temporal dimensions in real time; S3: Intelligent fusion processing step, inputting the high-dimensional data cube into a deep learning model with a spectral-spatial attention mechanism, adaptively weighting and fusing features from different spectral channels to generate a fusion feature map that highlights defects; S4: Adaptive decision-making step, which uses an online incremental learning classifier to identify and classify defects in the fused feature map, and dynamically updates the model to output structured information about the defects; S5: Closed-loop feedback step. Based on the structured information of the defect, the excitation spectrum strategy and parameters in the spectral modulation excitation are adjusted in real time to optimize the detection sensitivity for subsequent similar defects, forming a self-optimizing closed loop.

[0008] The technical effects and advantages of this invention are as follows: 1. This invention uses a programmable spectral modulation excitation module to dynamically match the optimal narrowband excitation spectrum for different materials and defect types, thereby enhancing the optical response signal of defects from the source. Combined with high-dimensional data cube construction and deep learning model based on attention mechanism, it adaptively integrates multi-dimensional features most sensitive to defects, thereby improving the identification ability and signal-to-noise ratio of micron-sized defects, internal defects and complex defects. 2. This invention uses a built-in online incremental learning classifier to dynamically update the model based on newly emerging defect samples, effectively solving the problem that traditional detection systems cannot identify new defect types due to model fixation. At the same time, it optimizes the activation strategy in real time based on the decision results, enabling the system to continuously evolve. 3. This invention improves the accuracy of single detection by integrating specific spectral excitation, multi-dimensional information fusion, intelligent decision-making and parameter feedback into an automatically optimized closed loop. Through a hierarchical feedback strategy, it achieves a dynamic balance between system detection accuracy, efficiency and energy consumption, fundamentally reducing the reliance on external manual intervention and parameter adjustment, and realizing the intelligentization and systematization of the detection process. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of the overall structure of the present invention; Figure 2 This is a flowchart illustrating the intelligent fusion processing module of the present invention; Figure 3 This is a flowchart illustrating the closed-loop feedback control module of the present invention. Detailed Implementation

[0010] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0011] As attached Figure 1 Appendix Figure 2 and appendix Figure 3 The defect detection system shown includes a spectral modulation excitation module: a programmable multi-band LED light source array and an acousto-optic tunable filter, which generates a time-alternating, spectrum-customized high-intensity narrowband excitation beam according to the material properties of the test object, and pre-matches the excitation spectrum combination through a built-in material spectrum library.

[0012] It should be specifically noted that the data processing method of the material spectral library includes data preprocessing, data classification and indexing, and data updating and optimization. The generation process of the high-intensity narrowband excitation beam includes excitation spectrum combination matching, multi-band LED light source array control, and acousto-optic tunable filter adjustment.

[0013] It should be further explained that the multi-band LED light source array is selected according to the following standard: covering the ultraviolet-visible-near-infrared band (200nm-1100nm). The LED light sources of different bands are independently controlled in terms of switching and light intensity. Among them, the ultraviolet band (200nm-400nm) LEDs are used to excite the defect energy level transition in semiconductor materials, the visible band (400nm-760nm) LEDs are used to excite the oxidation defects on the surface of metal materials, and the near-infrared band (760nm-1100nm) LEDs are used to penetrate and excite the internal structural defects of composite materials. The selection criteria for the acousto-optic tunable filter are: the center wavelength is adjustable in the range of 200nm-1100nm and the bandwidth is adjustable in the range of 0.5nm-20nm. It features fast wavelength switching speed (microsecond level), flexible bandwidth adjustment, and high reliability without mechanical moving parts. It converts the broadband beam output from the multi-band LED light source array into a high-intensity narrowband excitation beam and adjusts the bandwidth of the output beam according to the spectral resolution required for detecting defects in different materials.

[0014] The material spectral library data was collected through numerous experiments, including absorption and emission spectra of various commonly used industrial materials under different defect states. Data acquisition was conducted using a high-precision spectrometer in a standard experimental environment. Historical spectral data from actual industrial production processes were also collected, and these data were screened and preprocessed to remove outliers before being added to the material spectral library.

[0015] The material spectral library data processing method includes data preprocessing: baseline correction is performed on the collected raw spectral data, polynomial fitting is used to remove baseline drift in the spectrum, smoothing is then performed, Savitzky-Golay filtering algorithm (window size 5-15 points, polynomial degree 2-4) is used to reduce spectral noise and retain spectral feature information, and finally normalization is performed to normalize the spectral data to the 0-1 range, eliminating the influence of light intensity fluctuations on the spectral data, so that spectral data collected under different conditions are comparable. Data Classification and Indexing: The preprocessed spectral data is classified and stored according to material type and defect type. A multi-dimensional indexing system based on material name, defect type, and characteristic spectral peak position is established to facilitate the spectral modulation excitation module to quickly retrieve and match the corresponding excitation spectrum combination during detection.

[0016] Data Update and Optimization: New material spectral data and new defect state spectral data are collected regularly. The material spectral library is updated using incremental learning methods. At the same time, clustering algorithms are used to perform cluster analysis on the data in the library to optimize the data storage structure.

[0017] Once the object under test enters the detection area, the system uses the image recognition module to initially obtain the material type information of the object and transmits it to the spectral modulation and excitation module. The spectral modulation and excitation module searches the built-in material spectrum library based on the material type information and matches the optimal excitation spectrum combination for the material under different defect detection scenarios. If the system cannot determine the material type through preliminary image recognition, it can match the material type by manually inputting the material information. The spectral modulation excitation module sends control signals to the multi-band LED light source array based on the matched excitation spectrum combination, controlling the LED light source of the corresponding band to turn on and adjust its light intensity. The light intensity adjustment adopts pulse width modulation (PWM) technology, and precisely controls the duty cycle of the PWM signal according to the excitation light intensity required for different defect detections. The LED light intensity quantification analysis formula is as follows:

[0018] in, For target light intensity, Given the current light intensity, Retrieve standard data for the corresponding defect type from the material spectrum library. This is for converting the grayscale values ​​of images acquired in real time by the high-dimensional data acquisition module (the grayscale value is proportional to the feature response intensity, and the conversion coefficient is determined through standard calibration experiments).

[0019] Given that LED luminous intensity has a linear relationship with PWM duty cycle (D), the PWM duty cycle conversion... K is the light intensity coefficient, determined by the LED hardware parameters. The formula for analyzing the target duty cycle is:

[0020] Meanwhile, according to the preset timing cycle (the timing cycle is adjusted according to the detection efficiency requirements, ranging from 1 to 10 ms), the LED light sources of different wavelengths are controlled to turn on and off in sequence, forming a broadband beam output with alternating timing. While the multi-band LED light source array outputs a broadband beam, the spectral modulation excitation module sends control signals to the AOTF according to the center wavelength and bandwidth requirements of each narrowband spectrum in the matched excitation spectral combination. This adjusts the AOTF's radio frequency drive signal (RF frequency range 50-200MHz) and the center wavelength of the AOTF. With radio frequency It satisfies a linear relationship, and its formula is:

[0021] in, The calibration coefficients for AOTF are provided by the AOTF manufacturer or obtained through experimental calibration (e.g., selecting 3 known center wavelengths). Measure the corresponding radio frequency Calculated through linear regression and ).

[0022] The module converts a broadband beam into a narrowband beam with a corresponding center wavelength and bandwidth. For example, for a narrowband spectrum requirement with a center wavelength of 280 nm and a bandwidth of 1 nm, the module controls the radio frequency of the AOTF to allow only light with a wavelength of 280 nm ± 0.5 nm to pass through, thereby generating a high-intensity narrowband excitation beam.

[0023] High-dimensional data acquisition module: Based on a scientific-grade camera and high-speed image buffer unit with a mercury cadmium telluride array detector, it synchronously captures multi-channel image sequences of the test object excited by the spectral modulation excitation module under different narrowband spectra, and constructs a high-dimensional data cube including spatial, spectral and temporal dimensions in real time.

[0024] It should be specifically noted that the high-dimensional data acquisition module includes quality control during the data acquisition process, specifically including time synchronization control, detection area and device environmental control, and image data integrity check.

[0025] The construction of the high-dimensional data cube involves data preprocessing, including image denoising, image registration, and spectral data standardization. Based on the preprocessed image data, the constructed high-dimensional data cube includes spatial, spectral, and temporal dimensions.

[0026] It should be further explained that the HgCdTe area array detector scientific-grade camera uses a resolution of 2048×2048 pixels, a pixel size of 15μm×15μm, a spectral response range of 200nm-1100nm, and a frame rate of up to 100fps. The high-speed image cache unit uses a storage capacity of 1TB and a data transmission rate of up to 10Gbps. The high-speed image cache unit is directly connected to the HgCdTe area array detector scientific-grade camera, which can receive multi-channel image sequence data output by the camera in real time and temporarily store it to avoid data loss due to slow data transmission speed or processing delay. At the same time, it provides sufficient data cache space for the subsequent construction of high-dimensional data cubes.

[0027] While the high-intensity narrowband excitation beam with alternating timing and customized spectrum output from the spectral modulation excitation module irradiates the test object, the scientific-grade camera of the mercury cadmium telluride array detector acquires image sequence data of the test object under different narrowband spectra at a collection frequency (100fps) synchronized with the timing period of the excitation beam.

[0028] The collected data includes: two-dimensional spatial images of the object under test at different time points (corresponding to different moments within the time series) under each excitation band (resolution 2048×2048 pixels, grayscale value range of 0-4095 for each pixel), as well as excitation band information (center wavelength, bandwidth) and acquisition time information corresponding to each image.

[0029] The acquired raw image sequence is preprocessed, including noise reduction: an adaptive median filtering algorithm is used to automatically adjust the size of the filtering window (3×3-7×7) according to the noise intensity of different regions in the image. While effectively removing image noise, it preserves the defect details in the image to the greatest extent, avoiding the image detail blurring problem caused by traditional fixed window filtering algorithms. Image registration: A feature-point-based image registration algorithm is employed. Feature points (such as edge points and corner points) are extracted from each image using the SIFT (Scale Invariant Feature Transform) algorithm. Then, the matching relationship of feature points between different images is calculated. Based on the matched feature points, the image transformation matrix (translation, rotation, and scaling transformation) is solved. Finally, geometric correction is performed on the images based on the transformation matrix to ensure a one-to-one correspondence between pixels at the corresponding object locations in different images. This ensures the consistency of spatial dimension data within the constructed high-dimensional data cube. Root mean square error is used. The specific analysis formula for using this as an indicator to evaluate registration accuracy is as follows:

[0030] in, The corresponding feature points of the registered image and the reference image are defined by n, where n is the number of feature point pairs. It is recommended that n ≥ 50. The smaller the value, the higher the registration accuracy; preset threshold. Pixels (adjusted according to detection accuracy requirements); if the threshold is exceeded, re-registration is performed.

[0031] Spectral data standardization: The gray values ​​of images in different spectral bands are converted into relative reflectance or relative emissivity data to eliminate the influence of differences in LED light intensity in different bands on the image gray values. Using the reflectance data of a standard white board in each band as a benchmark, the ratio of the gray value of the tested object's image in a certain band to the gray value of the standard white board in that band is calculated to obtain the relative reflectance data of the tested object. The specific formula for analyzing the relative reflectance of the tested object is as follows:

[0032] in, The grayscale value of the image of the object being measured. The grayscale values ​​are for a dark field image (without excitation light). The grayscale values ​​are standard whiteboard images, all three of which were acquired under the same excitation band.

[0033] For emission spectrum images, background noise is removed by acquiring dark-field images to obtain the relative emissivity data of the analyte. The specific analysis formula is as follows:

[0034] in, The image grayscale value of a standard blackbody at the same temperature is obtained through a blackbody calibration source.

[0035] Based on the preprocessed image data, a four-dimensional high-dimensional data cube is constructed, which includes spatial dimension (X-axis: pixel coordinates in the horizontal direction of the image, Y-axis: pixel coordinates in the vertical direction of the image), spectral dimension (λ-axis: center wavelength of the excitation spectrum), and temporal dimension (T-axis: image acquisition time).

[0036] The spatial dimension is 2048×2048 (corresponding to the camera resolution), the spectral dimension is determined by the number of bands in the matched excitation spectrum combination (e.g., if there are 3 bands, the spectral dimension is 3), and the temporal dimension is determined by the detection duration (e.g., if the detection duration is 1 second and the acquisition frequency is 100 fps, the temporal dimension is 100).

[0037] Each data point (X,Y,λ,T) in the high-dimensional data cube corresponds to the relative reflectance or relative emissivity data of the test object at its spatial location (X,Y), spectral wavelength λ, and time T. This comprehensively integrates the spatial, spectral, and temporal information of the test object, providing a structured data carrier for feature extraction and information fusion in the subsequent intelligent fusion processing module.

[0038] Intelligent fusion processing module: It has a built-in deep convolutional neural network with attention mechanism to perform end-to-end feature extraction and information fusion on high-dimensional data cubes. Through a dedicated spectral-spatial attention sub-network, it adaptively learns and fuses the features most sensitive to defects in different spectral channels to generate a fusion feature map that highlights defects.

[0039] It should be specifically noted that the feature extraction includes spatial feature extraction and spectral feature extraction, and the information fusion adopts a cross-dimensional feature splicing and channel attention fusion method to fuse the spatial features, spectral features and time-series compressed features after being filtered by spectral and spatial attention.

[0040] It should be further explained that the module uses a high-dimensional data cube as input data, and the core analysis object is the relative reflectance / emissivity data of each data point.

[0041] To reduce data dimensionality and improve network computation efficiency, the input data is preprocessed, and the high-dimensional data cube undergoes temporal dimension compression. A temporal attention pooling algorithm is employed, and the contribution of data at different time points to defect features is analyzed based on the correlation coefficient between each time-series data point and the defect label. The specific analysis formula is as follows:

[0042] Where n is the number of samples, the t-th time series data is calculated first. Pearson correlation coefficient with defect label Y (1 for defective, 0 for non-defective) Then the correlation coefficient is converted into attention weights. The Softmax function is used for normalization, and the specific analysis formula is as follows:

[0043] Where T is the total number of temporal dimensions, and the final temporal compression feature The weighted sum of the time series data is calculated using the following formula:

[0044] The data from T time-series dimensions are compressed into a single comprehensive time-series feature dimension, forming a three-dimensional feature tensor with spatial dimension X×Y, spectral dimension λ, and time-series feature 1.

[0045] Spatial feature extraction employs three convolutional layers (Conv1-Conv3) to extract spatial features from the three-dimensional feature tensor. The Conv1 layer uses 64 3×3×1 convolutional kernels (3×3 spatial dimension, 1 spectral dimension) with a stride of 1, focusing on extracting basic spatial features such as the edges and textures of the object under test. The Conv2 layer uses 128 3×3×1 convolutional kernels to increase the network depth and capture more complex spatial structural features (such as the shape and contour of defects). The Conv3 layer uses 256 3×3×1 convolutional kernels to further enhance the ability to extract spatial features of minute defects (such as 1μm-level scratches on the surface of semiconductor wafers). After each convolutional layer, a ReLU activation function is used to introduce nonlinearity, and a max pooling layer (2×2 pooling kernel, stride of 2) is used to reduce the feature map size and reduce the computational cost. Based on spatial feature extraction, spectral feature extraction is performed by extracting spectral dimension features through two spectral convolutional layers (SpecConv1-SpecConv2). The SpecConv1 layer uses 128 1×1×3 convolutional kernels (1×1 in spatial dimension and 3 in spectral dimension) with a stride of 1 to capture the feature differences between adjacent spectral bands. The SpecConv2 layer uses 256 1×1×5 convolutional kernels to expand the spectral sensing range and extract global spectral features across multiple bands (such as the combination of spectral absorption peaks unique to defects).

[0046] A spectral attention weight generation network is constructed. The input is the spectral feature map output by the SpecConv2 layer (dimension X / 4×Y / 4×256×λ). The spatial dimension is compressed to 1×1 by global average pooling to obtain λ spectral feature vectors. Then, λ spectral attention weights are generated by passing two fully connected layers (FC1: 256→128, FC2: 128→λ) and the Sigmoid activation function. The weights are multiplied by the original spectral feature map by band to enhance the high contribution spectral features and suppress the low contribution spectral features.

[0047] Based on the feature map after spectral attention processing, the spatial attention mechanism constructs a spatial attention weight generation network. It compresses the spectral dimension to 1 through global average pooling to obtain an X / 4×Y / 4 spatial feature map. Then, it generates X / 4×Y / 4 spatial attention weights through a single convolutional layer (3×3 convolutional kernel, 1 output channel) and a sigmoid activation function. The weights are multiplied with the spatial feature map pixel by pixel to focus on strengthening the features of the spatial region where the defect is located.

[0048] Multi-dimensional feature fusion involves concatenating spatial feature maps (X / 4×Y / 4×256), spectral feature maps (X / 4×Y / 4×256), and temporal feature maps (X / 4×Y / 4×64) along the channel dimension to form a concatenated feature map of X / 4×Y / 4×576. Then, a channel attention module (composed of global average pooling, two fully connected layers, and a sigmoid function) generates 576 channel attention weights to optimize the channel features of the concatenated feature map, highlighting the channel features that contribute the most to defect identification. The fused feature map generation process compresses the fused feature map into a single-channel feature map using a single convolutional layer (1×1 kernel, 1 output channel). Then, a sigmoid activation function maps the feature values ​​to the 0-1 range, generating the fused feature map. The closer the feature value is to 1, the higher the probability of a defect in that region. The feature value for defective regions is 0.8-1.0, while that for normal regions is 0.0-0.2, thus clearly highlighting defective regions. The specific analysis formula is as follows:

[0049] in, To calculate the average feature value of the defect region in the fused feature map, C represents the average feature value of the normal region, and C represents the contrast of the defect region. The larger C is, the more obvious the distinction between the defect and the normal region. The preset threshold C≥2 (adjusted according to the defect type) is used. If it is lower than the threshold, the attention subnetwork parameters are optimized.

[0050] Adaptive Decision Module: Receives fused feature maps, uses an online incremental learning classifier based on transfer learning to identify and classify defects, dynamically updates its classification model based on newly emerging defect samples, and outputs a structured report.

[0051] It should be noted that the structured report includes five parts: basic detection information, defect identification results, defect classification details, spectral feature analysis, and model performance feedback.

[0052] It should be further explained that the classifier uses the fused feature map output by the intelligent fusion processing module as the core input data, and combines the original spectral features in the high-dimensional data cube as auxiliary input.

[0053] The labeling system is constructed as a two-level labeling system including "presence or absence of defects" and "defect type". The first-level label is a two-category system (0: no defects, 1: defects). The second-level label is a multi-category system (based on industrial testing needs, covering 10 typical defect types, including surface scratches, internal cracks, impurities, oxidation corrosion, delamination defects, weld joints, bubble defects, excessive wear, coating peeling, and lattice defects, labeled with labels 1-10). The label data comes from two parts: first, standard defect samples labeled in the laboratory (10,000+ sets, no less than 1,000 sets for each defect type); second, actual test samples after manual verification in the industrial field (5,000+ sets), ensuring the completeness and accuracy of the labeling system.

[0054] Threshold segmentation is performed on the fused feature map, and an adaptive thresholding algorithm is used to calculate the defect judgment threshold. The specific analysis method is as follows: The adaptive thresholding algorithm maximizes the inter-class variance. Determine the threshold M, assuming the image grayscale value range is... The total number of pixels is N, and the grayscale value is The number of pixels is ,but: Gray-scale probability distribution:

[0055] Between-class variance:

[0056] in, The percentage of pixels below threshold M. The percentage of pixels above threshold M. The average gray level of pixels below threshold M. Given the average gray level of pixels above threshold M, iterate through M from 0 to L, such that... The largest M is the optimal threshold.

[0057] If the area of ​​a pixel in the feature map whose feature value is higher than the threshold accounts for more than 0.01% of the total area, it is judged as "defective" (label 1); otherwise, it is judged as "no defect" (label 0).

[0058] To reduce the false positive rate, regional morphological verification is introduced. Morphological analysis is performed on the suspected defective regions after threshold segmentation (the area, perimeter, and roundness of the region are calculated). If the region parameters match the morphological characteristics of a certain type of defect, it is confirmed as "defective". Otherwise, it is judged as noise interference and corrected to "no defect".

[0059] For samples identified as "defective", the spatial features (area, shape, and location of the defect) of the defect region in the fusion feature map and the spectral features (center wavelength of the feature peak, peak intensity, and half-width at half-maximum) of the defect region in the original high-dimensional data are extracted to form a 20-dimensional defect feature vector. The defect feature vector is input into a transfer learning-based classifier. The feature vector is deep encoded through a pre-trained ResNet-50 feature extraction layer to obtain a 2048-dimensional deep feature. Then, it is input into a fully connected classification layer (the output dimension is the number of defect types). The probability of a sample belonging to each defect type is calculated through the Softmax activation function. Finally, the type with the highest probability is selected as the final classification result.

[0060] Incremental learning is triggered when the system detects a classification error discovered during manual review, and the cumulative number of erroneous samples exceeds 5% of the total number of training samples in the current model, or when a new defect type is detected.

[0061] The incremental learning process includes sample selection and labeling: samples that trigger incremental learning (erroneous samples or new defective samples) are manually reviewed and labeled, and then added to the incremental sample set (each incremental sample set has a size of no less than 200 groups to ensure learning effectiveness). The specific analysis formula is as follows:

[0062] in, The sample size is based on statistical confidence. The z-value corresponds to the confidence level. The expected defect incidence rate (based on statistics from historical inspection data) ), To allow for error, set .

[0063] Model parameter update: The improved EWC-Plus algorithm is used to update the classifier parameters. The importance weight of the existing model parameters to old samples is calculated by Fisher information matrix. When updating the parameters, constraints are imposed on the parameters with high importance to avoid "catastrophic forgetting" (i.e., the original defect type cannot be identified after the update). Then, the incremental sample set is mixed with a small number of old samples (10% of the old samples are randomly selected) to fine-tune the classifier (the learning rate is 1 / 10 of the initial training, and the training rounds are 5-10). This achieves the ability to learn new defect types and retain the ability to identify old defect types. Model deployment and validation: Replace the original model with the updated model and test the model performance using a validation set (including old samples and incremental samples). If the model's recognition accuracy on old samples decreases by no more than 2% and its accuracy on incremental samples exceeds 90%, the model update is confirmed to be successful. Otherwise, readjust the incremental learning parameters (increase the proportion of old samples or adjust the learning rate) and train again.

[0064] The structured report includes basic information such as the name of the object being tested, material type, testing time, and area of ​​the testing area. Defect identification results include the presence or absence of defects, the number of defect areas, the location coordinates of each defect area (presented in both image pixel coordinates and actual physical coordinates), and the area of ​​the defect area. Defect classification details include the type of each defect, confidence level (classification probability), morphological parameters (such as the length and width of scratches, and the estimated depth of cracks). Spectral feature analysis includes a chart of characteristic spectral peaks of the defect area (the horizontal axis is wavelength, and the vertical axis is relative reflectance / emissivity), a comparative analysis with standard defect spectra, and model performance feedback including the model's recognition accuracy, classification accuracy, and whether incremental learning suggestions need to be triggered in this test. Report generation and export: The system uses a template-based report generation engine to automatically populate the detection data into the preset report template. It supports export in PDF and Excel formats. At the same time, the report data is synchronously stored in the system database for easy traceability and statistical analysis.

[0065] Closed-loop feedback control module: Connected to the adaptive decision module and the spectral modulation excitation module, it adjusts the excitation spectrum strategy and parameters of the spectral modulation excitation module in real time according to the decision results, forming a self-optimizing detection closed loop.

[0066] It should be noted that the closed-loop feedback control module adopts a hierarchical feedback control strategy, which is divided into three levels: real-time fine-tuning control, batch optimization control, and long-term iterative control, each corresponding to different control cycles and adjustment targets.

[0067] It should be further noted that, during a single detection process, if the defect identification result output by the adaptive decision module meets any of the following conditions, real-time fine-tuning will be triggered: The intensity of the characteristic spectral peaks in the defect region is below the threshold, indicating that the current excitation light intensity is insufficient and the defect characteristics are not fully manifested.

[0068] The defect classification confidence level is below 0.7 (the preset classification reliability threshold), and the spectral feature analysis shows that the spectral signal-to-noise ratio of the defect area is below 10:1, indicating that the current excitation spectral band or bandwidth is not properly selected, resulting in unclear defect spectral features.

[0069] Implementation of adjustment strategy: If the problem is insufficient light intensity, the closed-loop feedback control module sends a light intensity adjustment command to the spectral modulation excitation module to increase the light intensity of the LED light source in the corresponding excitation band by 20%-50% on the original basis (dynamically determined according to the difference between the intensity of the characteristic spectral peak and the threshold; the larger the difference, the higher the increase ratio), while keeping the spectral band, bandwidth and timing period unchanged. If the problem is due to improper spectral parameters, the module searches for the optimal excitation spectral parameters corresponding to the defect type in the material spectral library based on the defect type output by the adaptive decision module. After comparing the current excitation parameters and calculating the parameter differences, it sends a band and bandwidth adjustment command to the spectral modulation excitation module to adjust the center wavelength of the AOTF to the target value, compress or broaden the bandwidth to the target bandwidth, and at the same time fine-tune the LED light intensity to match the optimal excitation intensity of the new band.

[0070] Adjustment effect verification: After the adjustment command is executed, the excitation beam is regenerated, a new image sequence is acquired and a high-dimensional data cube is constructed, and feature extraction and defect judgment are performed again. If the intensity of the characteristic spectral peak of the defect area reaches the threshold or above and the classification confidence is improved to 0.7 or above after adjustment, the real-time fine-tuning is completed. If the target is not achieved, the above adjustment process is repeated (up to 3 times to avoid over-adjustment leading to a decrease in detection efficiency).

[0071] Once a testing batch is completed, the closed-loop feedback control module performs statistical analysis on the testing data of that batch. If any of the following conditions are met, batch optimization is triggered: The rate of missed defects within a batch exceeds 3%.

[0072] The classification accuracy (number of correctly classified samples / total number of samples of this type) for the same defect type within a batch is less than 90%.

[0073] The average energy consumption of the spectral modulation excitation module (energy consumption during the detection process of each analyte) exceeds the preset energy consumption threshold (set according to equipment power and industrial energy-saving requirements).

[0074] The formula for calculating the defect missed detection rate is:

[0075] In the formula, This represents the defect missed detection rate. This represents the number of defects that were missed in the sampled data. The actual number of defects present in the sample (determined through comprehensive manual inspection).

[0076] Sampling sample size Based on the binomial distribution, the sample size formula is:

[0077] In the formula, The maximum allowable false negative rate is set to 0.03. The allowable deviation for the false negative rate is set to 0.01, and the 95% confidence level is set to 1.96.

[0078] The energy consumption threshold is calculated using the following formula:

[0079] In the formula, Energy consumption threshold, is The system average power is calculated as the sum of the power of each module. The standard testing time for a single item is set according to the production line cycle time requirements.

[0080] Implementation of the adjusted strategy: If the problem is a high rate of missed detections, analyze the type and spectral characteristics of the missed defects. If most of the missed defects are found to be "micro scratches" (size <5μm), then update the excitation parameters of this defect type in the material spectral library, increase the excitation weight of the ultraviolet band (extend the excitation time of this band in the time period from the original 1ms to 1.5ms), and at the same time compress the AOTF bandwidth from 2nm to 1nm. If the problem is low classification accuracy, with more than 15% of scratches being misclassified as impurities, then the standard spectral characteristics of the two defects are compared, and the excitation spectrum combination is adjusted. On the original basis, the excitation of the 550nm band is added to enhance the characteristic spectral signal of the impurities. At the same time, the timing period is optimized, so that the excitation interval of the 450nm and 550nm bands is shortened from 2ms to 1ms. If the problem is excessive energy consumption, analyze the energy consumption ratio of each excitation band. If the contribution of this band to defect identification in the current batch of testing is low (by calculating the correlation coefficient between the spectral characteristics of this band and the defect label; a coefficient <0.3 is considered a low contribution), then appropriately reduce the excitation light intensity of this band (reduced by 10%-20%) or shorten the excitation time (shortened by 10%-15%).

[0081] Evaluation of optimization effect: The next batch of tests will be conducted using the optimized excitation spectroscopy strategy. After the batch is completed, the false negative rate, classification accuracy and energy consumption will be recalculated. If the false negative rate drops below 3%, the classification accuracy increases to above 90% and the energy consumption is below the threshold, the batch optimization is successful. If the target is not met, the reasons will be further analyzed (such as whether there are new interference factors), the optimization strategy will be adjusted and verified again.

[0082] After each month's inspection tasks are completed, the module performs a comprehensive analysis of all inspection data for that month (including the defect type distribution of each batch, excitation parameter adjustment records, inspection accuracy and energy consumption data). If the following situations occur, a long-term iteration will be triggered: The addition of more than three new defect types indicates that the existing material spectral library and excitation strategies cannot cover the new defect requirements. The spectral data of some materials in the material spectral library deviates by more than 15% from the spectral data collected in actual testing. The overall detection efficiency of the system (the number of objects detected per unit time) has decreased by more than 10% compared to the initial state.

[0083] Implementation of the adjusted strategy: If it is a new defect type, the standard spectral data (after manual annotation) of the new defect will be added to the material spectral library. Based on all the detection data of the month, machine learning algorithms will be used to analyze the differences in spectral characteristics between the new defect and the existing defect, optimize the indexing system of the material spectral library, add the "new defect type" search category, and at the same time, adjust the basic parameter library of excitation spectrum combination to include the optimal excitation parameters of the new defect (determined through experimental screening) to ensure that subsequent detection can be directly matched. If the deviation is due to spectral data, the spectral data of materials with large deviations will be re-acquired (50 sets of new sample data will be acquired using a high-precision spectrometer under standard experimental conditions). The corresponding data in the material spectral library will be updated using a data fusion algorithm. At the same time, the recommended model for excitation parameters for defect detection of the material will be corrected to ensure that the excitation parameters match the latest spectral characteristics of the material. If the detection efficiency decreases, the complexity of the existing excitation spectroscopy strategy should be evaluated. If the number of excitation bands exceeds 5 and the average number of adjustments per analyte exceeds 2, the excitation strategy should be simplified by merging excitation bands with similar spectral characteristics, optimizing the timing cycle, and reducing unnecessary parameter adjustment steps.

[0084] Long-term iteration effect verification: Based on the detection data of the following month, evaluate the detection accuracy (missed detection rate, classification accuracy), detection efficiency, and adaptability (ability to identify new defects) of the system after long-term iteration. If all indicators are better than before the iteration and meet the industrial detection requirements, the long-term iteration control is completed. If there are shortcomings (such as improved detection efficiency but decreased accuracy in identifying new defects), the indicators will be further balanced in the next long-term iteration.

[0085] A defect detection method based on multispectral analysis includes S1: a spectral modulation excitation step, in which a programmable light source and filter are matched and driven from a preset spectral library according to the material properties of the test object to generate a high-intensity narrowband excitation beam with alternating time and customized spectrum to irradiate the test object.

[0086] S2: High-dimensional data acquisition step, simultaneously capturing multi-channel image sequences of the analyte under different customized narrowband spectra, and constructing a high-dimensional data cube including spatial, spectral and temporal dimensions in real time.

[0087] S3: Intelligent fusion processing step, which inputs a high-dimensional data cube into a deep learning model with a spectral-spatial attention mechanism, adaptively weights and fuses features from different spectral channels to generate a fusion feature map that highlights defects.

[0088] S4: Adaptive decision-making step, which uses an online incremental learning classifier to identify and classify defects in the fused feature map, and dynamically updates the model to output structured information about the defects.

[0089] S5: Closed-loop feedback step. Based on the structured information of the defect, the excitation spectrum strategy and parameters in the spectral modulation excitation are adjusted in real time to optimize the detection sensitivity for subsequent similar defects, forming a self-optimizing closed loop.

[0090] Secondly: The accompanying drawings of the embodiments disclosed in this invention only involve the structures involved in the embodiments disclosed in this invention. Other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of this invention can be combined with each other. In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A defect detection system based on multispectral analysis, characterized in that, include: Spectral modulation excitation module: Programmable multi-band LED light source array and acousto-optic tunable filter to generate a time-alternating, spectrum-customized high-intensity narrowband excitation beam according to the material properties of the test object, and pre-match the excitation spectrum combination through the built-in material spectrum library; High-dimensional data acquisition module: Based on a scientific-grade camera and high-speed image buffer unit with a mercury cadmium telluride array detector, it synchronously captures multi-channel image sequences of the test object excited by the spectral modulation excitation module under different narrowband spectra, and constructs a high-dimensional data cube including spatial, spectral and temporal dimensions in real time. Intelligent fusion processing module: It has a built-in deep convolutional neural network with attention mechanism to perform end-to-end feature extraction and information fusion on high-dimensional data cubes. Through a dedicated spectral-spatial attention sub-network, it adaptively learns and fuses the features most sensitive to defects in different spectral channels to generate a fusion feature map that highlights defects. Adaptive Decision Module: Receives fused feature maps, uses an online incremental learning classifier based on transfer learning to identify and classify defects, dynamically updates its classification model based on newly emerging defect samples, and outputs a structured report; Closed-loop feedback control module: Connected to the adaptive decision module and the spectral modulation excitation module, it adjusts the excitation spectrum strategy and parameters of the spectral modulation excitation module in real time according to the decision results, forming a self-optimizing detection closed loop.

2. The defect detection system based on multispectral analysis according to claim 1, characterized in that: The data processing method of the material spectral library includes data preprocessing, data classification and indexing, and data updating and optimization. The generation process of the high-intensity narrowband excitation beam includes excitation spectrum combination matching, multi-band LED light source array control, and acousto-optic tunable filter adjustment.

3. The defect detection system based on multispectral analysis according to claim 1, characterized in that: The high-dimensional data acquisition module includes quality control during the data acquisition process, specifically including time synchronization control, detection area and device environmental control, and image data integrity check.

4. The defect detection system based on multispectral analysis according to claim 1, characterized in that: The construction of the high-dimensional data cube involves data preprocessing, including image denoising, image registration, and spectral data standardization. Based on the preprocessed image data, the constructed high-dimensional data cube includes spatial, spectral, and temporal dimensions.

5. The defect detection system based on multispectral analysis according to claim 1, characterized in that: The feature extraction includes spatial feature extraction and spectral feature extraction. The information fusion adopts a cross-dimensional feature splicing and channel attention fusion method to fuse spatial features, spectral features and time-series compressed features after spectral and spatial attention filtering.

6. The defect detection system based on multispectral analysis according to claim 1, characterized in that: The structured report includes five parts: basic detection information, defect identification results, defect classification details, spectral feature analysis, and model performance feedback.

7. The defect detection system based on multispectral analysis according to claim 1, characterized in that: The closed-loop feedback control module adopts a hierarchical feedback control strategy, which is divided into three levels: real-time fine-tuning control, batch optimization control, and long-term iterative control, each corresponding to different control cycles and adjustment targets.

8. A defect detection method based on multispectral analysis, used to implement the defect detection system based on multispectral analysis as described in any one of claims 1-7, characterized in that, include: S1: Spectral modulation excitation step, based on the material properties of the test object, matches and drives a programmable light source and filter from a preset spectral library to generate a high-intensity narrowband excitation beam with alternating time and customized spectrum to irradiate the test object; S2: High-dimensional data acquisition step, synchronously capturing multi-channel image sequences of the analyte under different customized narrowband spectra, and constructing a high-dimensional data cube including spatial, spectral and temporal dimensions in real time; S3: Intelligent fusion processing step, inputting the high-dimensional data cube into a deep learning model with a spectral-spatial attention mechanism, adaptively weighting and fusing features from different spectral channels to generate a fusion feature map that highlights defects; S4: Adaptive decision-making step, which uses an online incremental learning classifier to identify and classify defects in the fused feature map, and dynamically updates the model to output structured information about the defects; S5: Closed-loop feedback step. Based on the structured information of the defect, the excitation spectrum strategy and parameters in the spectral modulation excitation are adjusted in real time to optimize the detection sensitivity for subsequent similar defects, forming a self-optimizing closed loop.

Images