A machine vision-based resistance sheet defect online detection system and grading method

Abstract

The application discloses a kind of resistance sheet defect online detection system and grading method based on machine vision, it is related to industrial machine vision detection technical field, solve the defect problem of prior art in multi-scale defect dynamic perception, cross-level feature interaction and process adaptive optimization;The scheme suppresses metal reflection interference by Retinex illumination correction and joint denoising model;Variable convolution kernel and hollow spatial pyramid pooling are used to realize gradient entropy driven dynamic perception of multi-scale defects;Bidirectional cross-layer attention network is constructed to realize early fusion of high-resolution details and high-level semantics;Local-global feature physical association is modeled based on graph attention network and self-supervised message passing mechanism;Integrate reinforcement learning and memristor random computing unit to form closed-loop parameter optimization system;The application significantly improves multi-scale defect detection precision, cross-modal feature fusion efficiency and system adaptive ability under complex working conditions.

Description

Technical Field

[0001] This invention relates to the field of industrial machine vision inspection technology, and more specifically to an online detection system and grading method for resistor defects based on machine vision. Background Technology

[0002] This machine vision-based online defect detection and grading system for resistor wafers acquires surface images using a high-precision industrial camera, optimizes imaging quality using multimodal light sources, and utilizes deep learning algorithms and feature engineering to achieve defect segmentation and classification. The system can detect defects such as scratches, oxide layer peeling, and microcracks in real time, and automatically grades them based on parameters such as defect size and density. It is widely used in quality control during electronic component manufacturing, significantly improving detection efficiency and accuracy.

[0003] Existing technologies primarily achieve defect detection through a combination of image acquisition hardware optimization and algorithm processing. For example, patent CN105973912A uses a combination of a linear industrial camera and a linear light source to capture surface texture, extracts defect areas through grayscale difference analysis, and combines morphological filtering to segment features such as scratches and spots, constructing area and density parameters for classification. Patent CN108537772A proposes a multi-angle continuous shooting strategy for printing defects in the positive conductor of chip resistors, comparing standard templates and measured samples through image registration, and determining the defect type by combining contour recognition and the number of pixel changes. The improved Hough transform algorithm proposed by Zhu Zhengtao et al. uses dual-circle detection and low-pass filtering difference method to locate notches in circular resistor sheets and extract spot features, combining the inscribed circle radius parameter for classification. US patent US20140198974A1 uses high-resolution three-dimensional imaging technology, extracts micro-crack features through a convolutional neural network (CNN), and combines transfer learning to solve the problem of small sample training. Furthermore, Springer's paper introduces a fusion of the ResNet model and the XGBoost algorithm to optimize multi-label classification performance through data augmentation; however, it relies on manually labeled data and is sensitive to noise. While existing technologies improve detection accuracy through hardware optimization and algorithmic improvements, they still inevitably reveal some shortcomings in multi-scale defect feature extraction and fusion during practical applications.

[0004] First, traditional feature extraction algorithms (such as improved Hough transform and SIFT feature matching) rely on fixed-level morphological parameters (such as area and perimeter), which cannot dynamically adapt to defect features of different scales. For example, patent CN105973912A segments scratches and spots through morphological filtering, but lacks multi-scale representation capabilities for scenarios where sub-millimeter-level cracks and oxide spots coexist. High-frequency gradient information of small defects is easily filtered out by downsampling operations. Although US patent US20140198974A1 uses CNN to extract features, its fixed convolutional kernel design makes it difficult to adaptively adjust the receptive field. When the same defect (such as scratches) presents different scales due to slurry diffusion, the network depth or parameters need to be manually adjusted, which limits real-time performance and generalization. Second, although ResNet-50 in Springer's literature extracts multi-scale features through hierarchical convolution, its encoder-decoder architecture adopts a "delayed fusion" strategy, where high-level semantic features and low-level details are only merged at the end of the network, resulting in the loss of high-resolution information of small defects (such as crack edges) during early downsampling. While the dynamic threshold segmentation of patent CN108447051B improves adaptability, its K-means clustering relies on manually preset feature weights, which cannot achieve dynamic interaction and weight allocation of cross-scale features (such as modeling the correlation between local texture and global deformation). The misjudgment rate increases significantly with the increase of defect scale difference. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention discloses an online detection system and grading method for resistor sheet defects based on machine vision, aiming to solve the problems mentioned in the background technology.

[0006] To achieve the above-mentioned technical effects, the present invention adopts the following technical solution:

[0007] A machine vision-based online defect detection system for resistor sheets includes:

[0008] The dynamic compensation module is used to collect RGB and near-infrared dual-modal data based on a linear array camera and a multispectral light source. It uses the Retinex multi-scale illumination compensation algorithm to eliminate reflections and suppresses high-frequency noise through bilateral filtering and non-local mean denoising to obtain a standardized image.

[0009] The multi-scale perception module is used to dynamically adjust the sampling offset of the convolution kernel based on the standardized image through the local gradient entropy driving mechanism of the deformable convolutional network, and to embed multi-dilation rate convolution kernels in Conv1 to Conv4 using dilated spatial pyramid pooling to generate multi-scale feature tensors of high-frequency edges, mid-level textures and global semantics, which are then output to the cross-layer fusion module.

[0010] The cross-layer fusion module is used to construct a spatial and channel collaborative attention mechanism between Conv2 and Conv4 layers using a bidirectional cross-layer attention network. It dynamically weights feature association through a multi-head self-attention model and generates enhanced multi-scale feature maps through depthwise separable convolution dimensionality reduction.

[0011] The graph network classification module maps the enhanced multi-scale feature map into a heterogeneous feature relationship graph. Nodes represent multi-scale defect attributes, including edge gradient, texture density, and deformation curvature. A self-supervised message passing mechanism is constructed based on a graph attention network and the Node2Vec embedding method, outputting dynamic weight features to an improved XGBoost multi-label classifier. Combined with prior parameters, defect types and defect levels are generated. The defect types include surface defects and structural defects; the defect levels are four.

[0012] The parameter optimization module is used to optimize the online iterative strategy function of the model based on the production line parameters collected in real time from the edge nodes and the defect data output by the improved XGBoost multi-label classifier. It dynamically adjusts the convolution kernel size and classification threshold, and uses a memristor random calculation method to suppress transmission noise with random resonant coding. The optimized parameters are then fed back to the multi-scale perception module and the graph network classification module.

[0013] As a further technical solution of the present invention, a method for online classification of defects in resistor sheets based on machine vision includes:

[0014] Step 1: Simultaneously acquire RGB images and near-infrared reflection data streams of the resistive sheet surface using a linear industrial camera and a multispectral ring light source. Use the Retinex multi-scale illumination compensation algorithm to eliminate metal foil reflection interference. Combine non-local mean and wavelet threshold joint denoising model to suppress high-frequency noise and generate standardized dual-modal image data.

[0015] Step 2: Based on the local gradient entropy driving mechanism, a deformable convolutional network is used to dynamically adjust the sampling point offset of the convolutional kernel. Through multi-dilation rate convolutional kernels with hollow space pyramid pooling, the features of cracks below 0.2mm, slurry diffusion deformation and overall tilting are captured at Conv1 to Conv4 levels to generate multi-scale feature tensors.

[0016] Step 3: Construct a spatial and channel collaborative attention mechanism using a bidirectional cross-layer attention network with 112×112 Conv2 layers and 14×14 Conv4 layers. Calculate the association weights between high-resolution details and low-resolution semantics through a multi-head self-attention model. After dynamic weighted fusion, the enhanced multi-scale feature map is generated by depthwise separable convolution dimensionality reduction.

[0017] Step 4: Map the enhanced multi-scale feature map to a heterogeneous feature relationship graph. Nodes represent multi-scale defect attributes, including edge gradient, texture density and deformation curvature. Initialize edge connection rules based on graph attention network and Node2Vec embedding algorithm. Model the physical mapping between exposed foil spots and overall warping through self-supervised message passing mechanism. Output dynamic weight features to the improved XGBoost classifier. Combine prior parameters to generate defect type and defect level.

[0018] Step 5: Real-time acquisition of production line process parameters and historical defect database through edge computing nodes; online iteration of strategy function through near-end strategy optimization model; dynamic adjustment of convolution kernel size and classification threshold; and memristor random calculation method with random resonant coding to suppress transmission noise. The optimized parameters are then transmitted back to steps 2 and 4 via probabilistic pulse fault-tolerant transmission.

[0019] Based on the above technical solution, the positive and beneficial effects of the present invention are as follows:

[0020] 1. Deformable convolutional kernels dynamically adjust the sampling offset direction and density based on local gradient entropy, accurately capturing high-frequency edge details of sub-millimeter cracks; hollow spatial pyramid pooling extracts features under different receptive fields in parallel using multi-dilation ratio (1 / 6 / 12) convolutional kernel groups, covering the full scale range from micro-cracks to macro-deformations. Combined with a reinforcement learning-driven parameter optimization module to adjust the convolutional kernel size and classification threshold in real time, the system can adapt to defect scale changes caused by process fluctuations such as slurry diffusion and printing offset, completely solving the problems of missed detection and misjudgment caused by fixed parameters in traditional methods, and significantly improving the detection robustness under complex working conditions.

[0021] 2. The bidirectional cross-layer attention network introduces a multi-head self-attention mechanism in the shallow encoder stage (Conv2), dynamically fusing high-resolution edge features and high-level semantic features to preserve sub-pixel-level geometric information of cracks. The graph neural network models the physical relationship between local texture and global deformation through self-supervised message passing, and dynamically generates edge weights using the Node2Vec embedding algorithm, replacing the manually preset rules of traditional K-means clustering. This technology achieves coherent representation from pixel-level details to semantic-level context, solving the misjudgment problem caused by feature fragmentation or weight solidification, and improving the classification accuracy of complex defects.

[0022] 3. The Retinex multi-scale illumination compensation algorithm, combined with a bilateral filtering-nonlocal means denoising model, suppresses high-frequency noise while preserving the edges of micro-defects. The memristor unit employs stochastic resonant coding technology to convert the parameter optimization signal into an anti-noise pulse sequence, which is then transmitted to the front-end module via probabilistic modulation. Combined with a reinforcement learning real-time iterative strategy, the system can dynamically adjust the noise suppression intensity and feature extraction strategy based on production line process parameters (such as slurry viscosity and printing speed), forming a "perception-decision-feedback" closed loop. This solves the problems of detail loss and insufficient adaptability caused by static filtering or manual parameter tuning in traditional methods, ensuring continuous high-precision detection in complex production environments. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0024] Figure 1 This is a schematic diagram of the online defect detection system for resistor sheets based on machine vision according to the present invention.

[0025] Figure 2 This is a flowchart of the online defect classification method for resistive sheets based on machine vision according to the present invention.

[0026] Figure 3 This is a schematic diagram illustrating the working principle of the Retinex multi-scale illumination compensation algorithm of this invention.

[0027] Figure 4 This is a diagram illustrating the working principle of the bidirectional cross-layer attention network of the present invention.

[0028] Figure 5 This is a schematic diagram of the principle framework of the improved XGBoost multi-label classifier of the present invention;

[0029] Figure 6 This is a schematic diagram of the principle framework of the multi-head self-attention model of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] In one embodiment, a machine vision-based online defect detection system for resistor sheets, such as... Figure 1 As shown, it includes:

[0032] The dynamic compensation module is used to collect RGB and near-infrared dual-modal data from a linear array camera and multispectral light sources. It employs the Retinex multi-scale illumination compensation algorithm to eliminate reflections and uses bilateral filtering and non-local mean denoising to suppress high-frequency noise, thereby obtaining a standardized image. Figure 3 As shown, the working principle of the Retinex multi-scale illumination compensation algorithm is as follows: The Retinex multi-scale illumination compensation algorithm performs multi-channel decomposition on the original RGB image using Gaussian convolution kernels of 15, 80, and 200 scales to extract the illumination component and reflection component, and eliminates illumination unevenness through logarithmic domain transformation. For the reflective area of ​​the metal foil, the Retinex multi-scale illumination compensation algorithm detects abrupt changes in the gradient of highlight pixels through Gaussian difference filtering, generates a reflective mask matrix, and uses an adaptive weighting function to nonlinearly attenuate the reflection component of the highlight area to generate a reflectivity-corrected image. After the dynamic compensation module eliminates the reflection, it uses a bilateral filter kernel to calculate the weighted mean of the pixel neighborhood, and uses a nonlocal mean algorithm to construct a 15×15 search window. It filters 7×7 similar blocks through block similarity measurement, and eliminates salt-and-pepper noise and Gaussian noise using a weighted average iteration method. After denoising, the data is encapsulated into a dual-channel pixel stream through the AXI-Stream protocol, and aligned and converted through a parallel multiply-accumulator array in the FPGA to output a standardized dual-modal image.

[0033] The multi-scale perception module is used to dynamically adjust the sampling offset of the convolution kernel based on the standardized image through the local gradient entropy driving mechanism of the deformable convolutional network. It employs dilated spatial pyramid pooling to embed multi-dilation rate convolution kernels in Conv1 to Conv4, generating multi-scale feature tensors for high-frequency edges, mid-level textures, and global semantics, which are then output to the cross-layer fusion module. The working method of the multi-scale perception module is as follows:

[0034] s1. The gradient magnitude of each pixel neighborhood in the input normalized image is statistically analyzed using a local gradient entropy calculation algorithm to generate a gradient entropy distribution map to quantify the region complexity. The calculation formula is as follows:

[0035] (1)

[0036] In formula (1), Indicates position The local gradient entropy at a given point is used to quantify the complexity of the 5×5 window pixel neighborhood; Gradient magnitude within the neighborhood The probability distribution; This represents the total number of pixels in the neighborhood.

[0037] s2, when When the value is ≥1.2, the dynamic adjustment mechanism of the sampling point offset of the deformable convolutional network is triggered. The offset is calculated based on the gradient direction field, and the calculation formula is as follows:

[0038] (2)

[0039] In formula (2), , These represent the spatial offset of the sampling points of the convolution kernel; The local gradient direction angle is calculated using the following formula: ,in Representing coordinates and The image gradient field; for high-entropy regions, the 3×3 convolution kernel is driven to shift in the direction of gradient abrupt change, using the dilation rate. Dense convolutional kernels are used to extract high-frequency edge features;

[0040] s3. Using hollow spatial pyramid pooling, embed a group of two-branch parallel convolutional kernels with dilation rates r=1 and r=6 in Conv2 layer; embed a group of two-branch parallel convolutional kernels with dilation rates r=6 and r=12 in Conv3 layer; embed single convolutional kernels with dilation rates 1 and 12 in Conv1 and Conv4 layers respectively, wherein the receptive field of Conv4 layer is expanded as follows:

[0041] (3)

[0042] In formula (3), This is the effective receptive field size of the convolution kernel; The expansion rate is used to control the sampling point interval; The basic convolution kernel size;

[0043] s4. The output of the multi-dilation rate convolution is weighted and fused through the channel attention mechanism to generate a cross-scale feature tensor; and the high-resolution features of the Conv2 layer and the semantic features of the Conv4 layer are aligned by bilinear interpolation through cross-level skip connections to output a multi-scale fused feature tensor.

[0044] The cross-layer fusion module employs a bidirectional cross-layer attention network to construct a spatial and channel-based collaborative attention mechanism between Conv2 and Conv4 layers. It dynamically weights feature associations through a multi-head self-attention model, and generates enhanced multi-scale feature maps through depthwise separable convolutional dimensionality reduction. Figure 4 As shown: The working principle of a bidirectional cross-layer attention network is as follows:

[0045] The 112×112 high-resolution feature map of Conv2 layer is upsampled to 14×14 resolution using a bilinear interpolation algorithm to generate low-level detail features aligned with the semantic feature space of Conv4 layer.

[0046] A cross-layer relationship matrix is ​​constructed using a multi-head self-attention model. The query vector Q of the cross-layer relationship matrix is ​​generated by 1×1 convolution of Conv4 layer features, and the key vector K and value vector V are extracted by grouped convolution of interpolated Conv2 layer features.

[0047] In the spatial dimension, the cosine similarity matrix between Q and K is calculated, and a spatial attention weight map is generated by Softmax normalization. The edge details of Conv2 layer and semantic context of Conv4 layer are dynamically weighted and fused. At the same time, in the channel dimension, a squeeze-excitation network is used to calculate channel attention weights, and the fused features are recalibrated by channel after activation by the Sigmoid function.

[0048] Feature dimensionality reduction is achieved through depthwise separable convolution, where depthwise convolution uses a 3×3 kernel to extract local spatial relationships, and pointwise convolution uses a 1×1 kernel to achieve cross-channel information interaction.

[0049] The resulting multi-scale enhanced feature map is added to the original Conv4 layer features via skip connections, and a residual learning mechanism is used to avoid gradient vanishing.

[0050] Furthermore, such as Figure 6 As shown, the multi-head self-attention model includes a cross-modal projection layer, a spatial relationship modeling layer, a multi-head decomposition layer, a dynamic weight fusion layer, a residual connection layer, and a regularized output layer. The cross-modal projection layer upsamples the Conv2 layer features to 14×14 resolution using bilinear interpolation, and performs linear projection on the bimodal features using grouped convolutions to generate a query vector Q, a key vector K, and a value vector V. The projection weight matrix has a dimension of 512×512. The spatial relationship modeling layer calculates the cosine similarity matrix between Q and K using scaled dot product attention, where the similarity score is normalized using Softmax to generate a spatial attention weight map, with a weight matrix dimension of 14×10. 4×14×14; The multi-head decomposition layer decomposes the weight matrix into 8 independent attention heads along the channel dimension. Each head extracts local spatial correlation features through an independent fully connected layer, outputting a multi-head attention feature group; The dynamic weight fusion layer merges multi-head features through channel concatenation and uses gated recurrent units to dynamically learn the contribution weights of each attention head, generating fused global attention features; The residual connection layer fuses the attention enhancement features with the original features through element-wise addition, triggering a residual learning mechanism to preserve underlying semantic information; The regularized output layer uses layer normalization and depthwise separable convolution for feature dimensionality reduction, outputting an enhanced multi-scale feature map;

[0051] The graph network classification module maps the enhanced multi-scale feature map into a heterogeneous feature relationship graph. Nodes represent multi-scale defect attributes, including edge gradient, texture density, and deformation curvature. A self-supervised message passing mechanism is constructed based on a graph attention network and the Node2Vec embedding method, outputting dynamic weight features to an improved XGBoost multi-label classifier. This, combined with prior parameters, generates defect types and defect levels. The working method is as follows:

[0052] u1. The multi-scale feature tensor of the input is initialized with nodes based on the Node2Vec embedding algorithm. A random walk strategy is used to generate a node sequence, and a 128-dimensional node feature vector is learned using the Skip-gram model. The walk length is set to 10, and the window size is 5 to capture local structural information; the initialized node set The initial vertices that form the graph;

[0053] u2. Based on cosine similarity, the correlation between node features is calculated. If the similarity threshold exceeds a preset threshold, an initial graph topology is constructed through an improved edge connection generation mechanism. The improved edge connection generation mechanism calculates the attention coefficients between nodes through the multi-head attention layer of the graph attention network, and uses the LeakyReLU activation function to perform a nonlinear transformation on the features of adjacent nodes to generate dynamic edge weights. The calculation formula is as follows:

[0054] (4)

[0055] In formula (4), For nodes and The dynamic edge weights are used to characterize the correlation strength of defect features; and This represents the node embedding vector initialized by Node2Vec, with dimension d=128; This represents the LeakyReLU activation function with a negative slope of 0.2; This is a vector concatenation operation; Represents the learnable parameter vector The transpose of the matrix; Represents a node The set of neighboring nodes; For the iteration variable in the summation operation;

[0056] u3. Utilize a self-supervised message passing mechanism to iteratively update node features, performing three-layer iterative message propagation. Residual connections are used between layers to maintain gradient flow. Each layer updates the state of the central node through a message passing function. The formula expression is:

[0057] (5)

[0058] In formula (5), For the first Node at the next iteration eigenvectors; The message transformation matrix; This is a gated loop unit used to filter valid neighborhood information;

[0059] u4. After three message passes, the node features are mapped to dynamic weight vectors via a fully connected layer and output to the feature input layer of the improved XGBoost classifier. Simultaneously, an online hard example mining strategy is used to filter samples with classification confidence below 0.85, triggering the Node2Vec random walk sequence generation module to incrementally update the node embedding representation based on the Skip-Gram model. Among these, such as... Figure 5 As shown: The improved XGBoost multi-label classifier receives the dynamic weight feature vector output by the graph neural network through a dynamic feature weighting mechanism. It then constructs a parallel decision tree group using a multi-label tree structure expansion algorithm, where each subtree corresponds to a specific defect type, including scratches, oxidation, and delamination. Next, based on FL reconstruction of the objective function, the gradient weight of high-frequency defects is reduced through a class balancing factor, while the loss contribution of composite defects is increased. For the dynamic weight features, the node weight vector is concatenated with the original features through a feature grouping embedding strategy, compressed by a sparse matrix, and then input to the tree split point selection module. During the tree growth stage, the improved XGBoost multi-label classifier uses a proximal gradient descent algorithm to optimize the split gain calculation. If the feature weight is lower than a preset threshold, a pruning mechanism is triggered to skip irrelevant features. The classification result is transmitted to the reinforcement learning module through an online incremental learning interface, triggering a model parameter update signal. Simultaneously, a Shapley value interpreter is integrated to dynamically analyze feature contribution. When the resistivity feature contribution of local delamination deviates from the baseline by 20%, the tree node weights are corrected through an online calibration method.

[0060] The parameter optimization module, based on production line parameters collected in real-time from edge nodes and defect data output by an improved XGBoost multi-label classifier, employs a near-end strategy optimization model to iterate the policy function online, dynamically adjusting the convolution kernel size and classification threshold. It also utilizes a memristor stochastic computation method and random resonance coding to suppress transmission noise. The optimized parameters are then fed back to the multi-scale sensing module and the graph network classification module. The parameter optimization module collects production line process parameters and defect statistical features output by the XGBoost classifier in real-time from edge computing nodes, constructing a state observation vector which is input to the near-end strategy optimization model. This model uses an Actor-Critic architecture to perform parallel policy evaluation and optimization. The Actor network calculates the convolution kernel size, classification threshold, and morphological filter intensity based on the dominance function, while the Critic network uses TD error estimates to constrain the policy update amplitude. To address transmission noise interference, the parameter optimization module encodes the policy parameters into probability pulse sequences using a memristor stochastic computation unit and employs random resonance coding. The coding mechanism enhances signal features through a nonlinear stochastic resonance method when the pulse amplitude is lower than the noise floor. The optimized parameters are encapsulated into data packets using a probabilistic pulse amplitude modulation protocol and transmitted back to the deformable convolution controller of the multi-scale sensing module and the threshold adjuster of the graph network classification module via the SPI interface. If the standard deviation of slurry viscosity is detected to exceed 20% of the historical baseline, the online fine-tuning mechanism of the policy network is triggered, and the network weights are updated based on the dual-delay deep deterministic policy gradient algorithm. At the same time, a parameter version management queue is established. When three consecutive policy updates cause the classifier F1 score to fluctuate by more than 5%, the version rollback mechanism is triggered to load the historical best parameter snapshot.

[0061] In practice, a machine vision-based online defect classification method for resistor sheets, such as... Figure 2 As shown, it includes the following steps:

[0062] Step 1: Simultaneously acquire RGB images and near-infrared reflection data streams of the resistive sheet surface using a linear industrial camera and a multispectral ring light source. Use the Retinex multi-scale illumination compensation algorithm to eliminate metal foil reflection interference. Combine non-local mean and wavelet threshold joint denoising model to suppress high-frequency noise and generate standardized dual-modal image data.

[0063] Step 2: Based on the local gradient entropy driving mechanism, a deformable convolutional network is used to dynamically adjust the sampling point offset of the convolutional kernel. Through multi-dilation rate convolutional kernels with hollow space pyramid pooling, the features of cracks below 0.2mm, slurry diffusion deformation and overall tilting are captured at Conv1 to Conv4 levels to generate multi-scale feature tensors.

[0064] Step 3: Construct a spatial and channel collaborative attention mechanism using a bidirectional cross-layer attention network with 112×112 Conv2 layers and 14×14 Conv4 layers. Calculate the association weights between high-resolution details and low-resolution semantics through a multi-head self-attention model. After dynamic weighted fusion, the enhanced multi-scale feature map is generated by depthwise separable convolution dimensionality reduction.

[0065] Step 4: Map the enhanced multi-scale feature map to a heterogeneous feature relationship graph. Nodes represent multi-scale defect attributes, including edge gradient, texture density and deformation curvature. Initialize edge connection rules based on graph attention network and Node2Vec embedding algorithm. Model the physical mapping between exposed foil spots and overall warping through self-supervised message passing mechanism. Output dynamic weight features to the improved XGBoost classifier. Combine prior parameters to generate defect type and defect level.

[0066] Step 5: Real-time acquisition of production line process parameters and historical defect database through edge computing nodes; online iteration of strategy function through near-end strategy optimization model; dynamic adjustment of convolution kernel size and classification threshold; and memristor random calculation method with random resonant coding to suppress transmission noise. The optimized parameters are then transmitted back to steps 2 and 4 via probabilistic pulse fault-tolerant transmission.

[0067] In the dynamic compensation module described above, the Retinex multi-scale illumination compensation algorithm uses a three-scale Gaussian convolution kernel (σ=15 / 80 / 200) to perform frequency domain decomposition on the original RGB image: the large-scale kernel (σ=200) extracts the low-frequency illumination component to represent the global illumination distribution, the medium-scale kernel (σ=80) captures the mid-frequency reflection component corresponding to the material surface texture, and the small-scale kernel (σ=15) separates high-frequency noise and micro-defect features. Linear decoupling of the illumination and reflection components is achieved through multi-channel logarithmic domain transformation. The illumination component estimation is optimized using constrained least squares, while edge-preserving constraints are introduced during reflection component reconstruction to suppress halo artifacts. For high reflectivity interference from metal foil, a reflective region detection model based on difference of Gaussian (DoG) filtering is designed: first, a normalized gradient magnitude map is generated; then, the highlight edge contour is extracted using a double-threshold hysteresis method; finally, morphological closing operations are used to fill the region and generate a binary mask matrix. In the reflectivity correction stage, an adaptive attenuation function is applied to the mask-covered area, with its weight coefficients dynamically adjusted by local contrast to achieve nonlinear specular suppression. In the joint denoising stage, a hybrid domain filtering strategy is employed: in the spatial domain, a weighted mean of the pixel neighborhood is calculated based on a bilateral filter kernel (σ_d=2, σ_r=0.1), preserving sub-pixel-level edges through color similarity constraints; in the frequency domain, a non-local mean model is constructed, calculating the SSIM structural similarity of 7×7 similar blocks within a 15×15 search window, and using an exponential attenuation function for weighted averaging to eliminate salt-and-pepper noise. Finally, hardware-level synchronization of the dual-channel pixel stream is achieved through the AXI-Stream protocol, utilizing a heterogeneous computing architecture within the FPGA to complete data format conversion and alignment. The RGB channel uses a color space conversion from YUV420 to RGB888, while the near-infrared channel achieves dynamic range compression through 16-bit fixed-point mapping.

[0068] In implementation, this module uses a Basler raL2048-48gm linear array camera and a multispectral ring light source (wavelength 450nm / 850nm / 1050nm) to form an optical acquisition unit. The camera is connected to the Xilinx Zynq UltraScale+ MPSoC FPGA via a CoaXPress 2.0 interface, with a transmission rate of 6.25Gbps. The light source control module integrates a PWM dimming circuit to dynamically adjust the flash frequency (50-200Hz) according to the production line speed. A parallel pipeline processing architecture is built inside the FPGA: first, the raw image data (4096×512×12bit) is cached through a DDR4 controller; the Retinex algorithm is implemented in the programmable logic unit; the three-scale Gaussian convolution uses a split filter structure (horizontal / vertical convolution kernels); and a Systolic array is used to accelerate matrix operations. The reflection detection module is configured with dual-port Block RAM to store gradient magnitude maps, and morphological operations use a 3×3 structuring element pipeline architecture. In the joint denoising module, bilateral filtering pre-calculates spatial-color weights using a lookup table (LUT), and the nonlocal mean algorithm employs a sliding window buffer to reduce repeated data reads. The AXI-Stream interface is configured with a 4-channel, 128-bit width, and uses a cross-switch matrix to achieve time-division multiplexing transmission of RGB and near-infrared data. On the software side, Retinex parameters are optimized using Vivado HLS: the Gaussian kernel coefficients use a Q8.8 fixed-point format, and the logarithmic transformation is approximated using the CORDIC algorithm; the reflectivity attenuation function is set to a piecewise linear model, triggering exponential decay when the local contrast exceeds 30%. During the production line debugging phase, an offline calibration method is used to determine the optimal parameter combination: a standard grayscale board is used to calibrate illumination uniformity, and a defect sample library (containing defects ranging from 0.05 to 2 mm) is used to optimize the bilateral filtering parameters. Finally, the configuration is fixed to the FPGA bitstream file.

[0069] In practice, traditional metal surface inspection systems are limited by a single visible light imaging mode, making it difficult to overcome the dual interference of specular reflection and coating masking. This module, however, establishes a cross-band correction mechanism for the reflectivity characteristics of metal foil surfaces through visible-near-infrared dual-band collaborative imaging, combined with a Retinex multi-scale decomposition physical reflection model. This overcomes the bottleneck of single-mode data in perceiving internal defects. At the algorithm level, existing illumination compensation methods often employ fixed-threshold high-light suppression strategies, which cannot adapt to the dynamic changes in metal foil reflectivity under different process conditions. This module achieves precise nonlinear control of the reflection component through a gradient-driven adaptive attenuation function, solving the edge blurring problem caused by traditional linear compression. Regarding noise suppression, traditional methods often use spatial or frequency domain filtering in isolation. This module innovatively combines the local edge-preserving characteristics of bilateral filtering with the global similarity measurement of non-local means to construct a hybrid denoising model, effectively suppressing composite noise while preserving sub-pixel-level crack edges. At the hardware architecture level, it breaks through the speed bottleneck of traditional CPU serial processing. Through deep optimization of FPGA pipeline and parallel computing unit, it realizes real-time alignment and preprocessing of dual-modal data streams, providing a high signal-to-noise ratio input guarantee for subsequent feature extraction modules.

[0070] In the multi-scale perception module described above, the gradient magnitude distribution of the pixel neighborhood (5×5 window) is quantized based on Shannon information entropy theory during local gradient entropy calculation. The gradient magnitude is calculated using the Sobel operator to generate a gradient magnitude probability density function. When the entropy value exceeds the threshold (H≥1.2), the geometric deformation mechanism of the deformable convolution kernel is triggered: the sampling point offset vector is calculated based on the gradient direction field, where the gradient direction is obtained through a discrete differential operator (Scharr filter), and the offset amplitude is dynamically adjusted by the normalized entropy value to achieve dense sampling of high-frequency defect regions. In the hierarchical design of the expansion rate, the Conv2 layer adopts a dual-branch parallel convolution architecture (r=1 / r=6), where the r=1 branch extracts sub-millimeter-level crack edge features through a regular sampling interval, and the r=6 branch captures local deformation caused by slurry diffusion by expanding the sampling interval; the dual-branch structure of the Conv3 layer (r=6 / r=12) establishes a transition mapping from meso- to macro-level features, and the r=12 branch covers the global structure of 5mm-level defects through sparse sampling. The receptive field expansion formula shows that when using a base kernel of k=3, the effective receptive range of a convolutional kernel of r=12 can reach 25×25 pixels, breaking through the scale limitation of traditional convolutional kernels. The channel attention mechanism achieves dynamic weighting of feature channels through a squeeze-activation network (SENet). A global average pooling layer generates channel statistical descriptors, two fully connected layers learn the nonlinear relationship between channels, and the sigmoid activation function outputs normalized weights, strengthening the feature correlation between high-frequency edges and global semantics. Cross-layer skip connections use bilinear interpolation to align feature map resolution, and depthwise separable convolutions reduce computational complexity while preserving the spatial accuracy of the 112×112 high-resolution features in Conv2 layers.

[0071] In implementation, the gradient entropy calculation unit adopts a parallel pipeline design: Sobel gradient calculation uses a dual-path 5×5 convolutional kernel pipeline, the gradient magnitude statistics module integrates a 32-bit accumulator array, and the entropy threshold comparator triggers deformation control signals in real time. The dilated convolutional kernel group is implemented using parameterized IP cores, where the r=1 branch configures a standard convolutional unit, and the r=6 / r=12 branches use an interval sampling address generator, caching intermediate feature maps via BRAM. The channel attention mechanism is implemented on an ARM Cortex-A53 processor, utilizing the NEON instruction set to accelerate matrix operations, and the weight coefficients are written back to the FPGA feature fusion unit via the AXI bus.

[0072] In actual deployment, the gradient entropy threshold is set to 1.2 (an empirically optimized value), and the inflation rate configuration follows a progressive expansion strategy of Conv1(r=1)→Conv2(r=1 / 6)→Conv3(r=6 / 12)→Conv4(r=12). During the production line debugging phase, a gradient orientation field visualization tool is used to calibrate the accuracy of offset vector calculation, and the initial values ​​of channel attention weights are optimized using a defect sample library (containing defects ranging from 0.1 to 5 mm). During system operation, the Conv2 layer feature map is mapped to the DDR4 memory pool through the HP port, and cross-layer skip connections use the AXI-Stream protocol to achieve low-latency data transmission. The 3×3 depthwise convolutional kernels with depthwise separable convolutions are accelerated using the Winograd algorithm.

[0073] This module first proposes a gradient entropy-driven convolution kernel deformation algorithm, combining information entropy theory with deformable convolution. By quantifying local complexity, it achieves adaptive adjustment of sampling density, significantly improving the ability to capture sub-millimeter-level microcracks. Second, it constructs a hierarchical dilation rate feature pyramid, embedding a multi-branch parallel convolutional structure into the shallow network. Through a progressive receptive field expansion strategy, it establishes a continuous mapping relationship between micro and macro features, solving the feature fragmentation problem caused by skip sampling in traditional multi-scale methods. Finally, it designs a cross-level feature preservation architecture, fusing high-resolution detailed features with high-level semantic features after dimensionality reduction through depthwise separable convolution, reducing computational complexity while maintaining spatial positioning accuracy.

[0074] To verify the effectiveness of the multi-scale sensing module (Group A) during the design process, a traditional fixed-kernel ResNet-50 (Group B) was selected for comparison. The experimental platform used the same hardware configuration (Basler camera + FPGA + industrial computer), and the test samples were a standard library of 1000 resistor sheets containing defects ranging from 0.05 to 5 mm. Group A deployed deformable convolutional kernels and a multi-rate hierarchical design, while Group B used a standard ResNet-50 with a fixed 3×3 convolutional kernel and max pooling. The experiment simulated production line illumination fluctuations (500~2000 Lux) and process fluctuations (slurry viscosity ±15%). Each group underwent five repeated experiments, and the experimental data are recorded in Table 1.

[0075] Table 1 Experimental Record of Multi-Scale Sensing Module

[0076] experimental group parameter Experiment 1 Experiment 2 Experiment 3 Experiment 4 Experiment 5 Group A Microcrack detection rate (%) 100 100 100 100 100 Large-size defect detection rate (%) 98.6 99.2 98.9 99.5 99.1 Feature extraction latency (ms) 3.2 3.1 3.3 3.0 3.2 False detection rate due to light interference (%) 1.8 2.1 1.5 1.9 2.0 Cross-batch recognition rate (%) 96.4 95.8 97.2 96.1 96.9 Group B Microcrack detection rate (%) 72 78 85 80 75 Large-size defect detection rate (%) 92.3 93.7 91.5 94.0 93.2 Feature extraction latency (ms) 3.0 3.1 2.9 3.0 3.1 False detection rate due to light interference (%) 12.6 15.3 10.9 14.7 13.8 Cross-batch recognition rate (%) 82.1 79.4 84.6 81.3 83.5

[0077] As shown in Table 1, the multi-scale sensing module (Group A) achieved 100% stable detection of micro-cracks, improving performance by more than 20% compared to the traditional method (Group B). Furthermore, the false detection rate due to illumination interference was reduced to below 2%, validating the effectiveness of the gradient entropy-driven hierarchical design of deformation and expansion rate. The cross-batch recognition rate improved by approximately 14%, indicating the module's strong generalization ability to process fluctuations. Although the feature extraction latency was slightly higher by 0.2ms, its overall performance was significantly better than the fixed convolution kernel scheme, meeting the real-time requirements of high-speed production lines (100m / min).

[0078] In the aforementioned cross-layer fusion module, the bidirectional cross-layer attention network is based on spatial-channel collaborative attention and multi-scale feature reconstruction theory. Between the Conv2 layer (112×112 resolution) and the Conv4 layer (14×14 resolution), a bilinear interpolation algorithm is used to spatially align the features of the Conv2 layer. A bicubic interpolation kernel function is then used to perform weighted interpolation on local neighboring pixels, generating a low-level feature map that matches the resolution of the Conv4 layer, preserving sub-pixel-level edge details such as micro-cracks and oxidation spots. This process avoids the jagged edge effect caused by traditional nearest-neighbor interpolation by optimizing the weight distribution of the sub-pixel convolution kernel.

[0079] Subsequently, a cross-layer relationship matrix is ​​constructed. The query vector (Q) is projected from Conv4 layer features through a 1×1 convolution to a 512-dimensional semantic space. The key vector (K) and value vector (V) are extracted from the interpolated Conv2 layer features through grouped convolution (Group=8). The grouping strategy reduces computational complexity by minimizing parameter sharing between channels. In the spatial dimension, the cosine similarity matrix between Q and K is calculated using scaled dot product attention, and a temperature coefficient is introduced ( The attention distribution is adjusted, and a spatial attention weight map is generated by normalization using the Softmax function, dynamically representing the association strength between Conv2 layer pixels and Conv4 layer semantic categories. In the channel dimension, a squeeze-excitation network (SENet) is used to generate channel attention weights: spatial information is compressed through global average pooling to generate channel descriptors, and the nonlinear relationship between channels is learned through two fully connected layers. The sigmoid function outputs normalized weights to enhance the response strength of defect-sensitive channels. The spatial and channel attention outputs are fused using Hadamard Product, and then dimensionality is reduced through depthwise separable convolution: depthwise convolution (3×3 kernels) extracts local spatial patterns, and pointwise convolution (1×1 kernels) achieves cross-channel information interaction, generating enhanced multi-scale feature maps. A residual learning mechanism adds the fused features element-wise to the original Conv4 layer features, preserving underlying semantic information through identity mapping and suppressing the risk of gradient vanishing.

[0080] The multi-head self-attention model comprises four core stages: cross-modal projection, multi-head decomposition, dynamic aggregation, and regularized output. The cross-modal projection layer employs a heterogeneous feature alignment strategy, upsampling the high-resolution features (112×112×256) from the Conv2 layer to 14×14 resolution via bilinear interpolation and spatially aligning them with the semantic features (14×14×512) from the Conv4 layer. Independent linear projections are then performed on the bimodal features using grouped convolutions (Group=8), generating a query vector Q (14×14×512), a key vector K (14×14×512), and a value vector V (14×14×512). The projection matrix is ​​initialized using Xavier to avoid gradient vanishing. The spatial relationship modeling layer calculates the similarity matrix between Q and K by scaling the dot product attention. The cosine similarity score, after being scaled by a temperature coefficient (τ=√d_k), is normalized to a probability distribution using the Softmax function, generating a spatial attention weight map (14×14×14×14) that dynamically represents the association strength at different spatial locations. The multi-head decomposition layer divides the weight matrix along the channel dimension into eight independent attention heads (Head=8). Each head is mapped to a low-dimensional subspace through a fully connected layer (FC-64), extracting spatial association patterns from the local neighborhood (7×7 window) to generate a multi-head attention feature group (8×14×14×64). The dynamic weight fusion layer uses a gated recurrent unit (GRU) to achieve adaptive aggregation of multi-head features: the GRU's update gate controls the fusion ratio of historical features with the current input, the reset gate filters effective information flow, and the contribution weights of each attention head are iteratively optimized through time steps, outputting a global attention feature (14×14×512). The residual connection layer adds the attention features element-wise to the original Conv4 layer features, using an identity mapping to preserve the initial semantic information and avoid network degradation. The regularized output layer suppresses feature distribution shifts through layer normalization (LayerNorm) and uses depthwise separable convolution (DSeConv) for dimensionality reduction: depthwise convolution (3×3 kernel) extracts local spatial correlations, and pointwise convolution (1×1 kernel) compresses the channel dimension to the target value (14×14×256), ultimately outputting an enhanced multi-scale feature map. This model achieves fine-grained interaction of cross-layer features through multi-head parallelization and dynamic gating mechanisms, significantly improving the characterization ability of complex defects (such as crack-oxidation coexistence), and providing theoretical support for multi-scale detection of resistor sheet defects.

[0081] In the scenario of resistor sheet defect detection, the synergistic effect of bidirectional cross-layer attention network and multi-head self-attention model is manifested in the deep fusion of multi-scale defect features. The high-resolution features of Conv2 layer correspond to the sub-pixel-level gradient information of microcracks and scratches, while the semantic features of Conv4 layer represent macroscopic structural anomalies such as global deformation and warping. Through the spatial attention mechanism, the model can accurately associate local gradient abrupt regions with global deformation trends and dynamically suppress illumination interference in non-defect areas; the channel attention weights quantify the contribution of different feature channels (such as edges, textures, and deformations), strengthening the expression of defect-sensitive features. In the multi-head self-attention model, the cross-modal projection layer achieves spatial alignment of high- and low-resolution features, and the dynamic weight fusion layer establishes a physical association model of cross-scale features through GRU gating, supporting accurate analysis of composite defects (such as cracks accompanied by slurry overflow). This technical system deeply combines local detail preservation with global semantic reasoning, providing a high-precision and robust analytical framework for online detection and classification of resistor sheet defects.

[0082] In hardware implementation, the cross-layer fusion module is based on a heterogeneous computing architecture, employing a Xilinx Alveo U280 accelerator card and an Intel Xeon Gold 6248R processor to work together, achieving high-speed data communication via a PCIe Gen4×16 interface. The FPGA handles high-throughput feature interpolation and initial attention calculations, and is equipped with dual-channel HBM2 memory (8GB capacity, 460GB / s bandwidth) to cache the 112×112 high-resolution feature map of the Conv2 layer and the 14×14 semantic features of the Conv4 layer. The bilinear interpolation module is implemented through FPGA logic units, configured with a second-order polynomial interpolation kernel function, with the interpolation weight matrix preloaded into BlockRAM. The interpolation accuracy error is controlled within ±0.3 pixels, outputting aligned 14×14 low-level detail features. The multi-head self-attention model is accelerated on the CPU using the OpenVINO inference engine. The query vector Q is generated from Conv4 layer features through 1×1 convolutions, with a kernel weight dimension of 512×512, initialized using Xavier uniform distribution. The key vector K and value vector V are extracted from interpolated Conv2 features through grouped convolutions, with 8 groups and 64 convolutional filters per group, resulting in a total of 512 output channels. The spatial relationship modeling layer uses the MKL-DNN library to optimize matrix multiplication operations, and the cosine similarity matrix calculation is parallelized using the SIMD instruction set. The spatial attention weight map has a dimension of 14×14×14×14 and is compressed using CSR sparse format, achieving a compression rate of 50%. The gated recurrent unit (GRU) hidden layer of the dynamic weight fusion layer has a dimension of 256, and the gate weights are initialized orthogonally. During training, the LAMB optimizer is used (learning rate 2e-4, β1=0.9, β2=0.999), and the batch size is set to 64 in mixed precision training mode. The 3×3 depthwise convolutional kernels of depthwise separable convolutions employ a symmetrical padding strategy, padding the edges with 1 pixel to maintain the feature map size. The number of channels in the pointwise convolutional layers is compressed to 256 dimensions, and the number of parameters is reduced to 12% of that of standard convolutions. Residual connections achieve element-wise addition of feature maps through a dedicated data path on the FPGA side. The layer normalization module uses FP16 half-precision to calculate the mean and variance, reducing data transmission latency. In terms of hardware resource allocation, the FPGA logic unit utilization rate is 58% (LUT: 315K / 542K), the HBM2 memory bandwidth utilization is stable at 82%-85%, and the peak memory bandwidth usage on the CPU side reaches 48GB / s. In terms of software parameter configuration, the number of multi-head attention heads is set to 8, the dimension of each head is 64, the compression ratio of the channel attention squeezing-excitation network is set to 16, and the activation function uses GELU instead of ReLU to enhance nonlinear expressiveness. The training dataset contains 3 million images of resistor defects, covering 6 types of defects such as scratches, oxidation, and cracks. The data augmentation strategy uses random rotation (±15°), brightness jitter (±20%), and Gaussian noise injection (σ=0.1).The model training cycle is 80 epochs. The early stopping mechanism is triggered when the validation set loss does not decrease for 5 consecutive times. After the final model accuracy converges, it is deployed to edge computing nodes via ONNX format.

[0083] Traditional methods rely on top-down unidirectional feature transfer. High-resolution detail features gradually become distorted during multiple upsampling processes due to interpolation errors and noise accumulation. In contrast, the bidirectional attention mechanism achieves complementary enhancement of low-level details and high-level semantics through dynamic weight allocation in both spatial and channel dimensions. Spatial attention measures the spatial correlation strength of cross-level features using cosine similarity, enabling convolutional kernels to focus on the microstructure of defect edges (such as crack bifurcation points) while suppressing background texture interference. Channel attention, on the other hand, re-calibrates the importance of feature channels through a squeeze-excitation network, strengthening the saliency of mid-frequency features such as oxide spots. The introduction of depthwise separable convolution decouples spatial filtering from channel transformation, significantly reducing computational complexity while maintaining the integrity of edge structures, making the multi-scale feature fusion process adaptable to the real-time constraints of edge devices. The residual learning mechanism preserves the original feature distribution characteristics through identity mapping, avoiding the gradient annihilation problem in deep network training, while layer normalization improves model convergence stability by standardizing feature distribution. This design not only solves the problem of detail loss caused by fixed-weight fusion in traditional methods, but also improves the characterization ability of composite defects through adaptive enhancement of the channel dimension. It provides a high-precision, low-latency feature fusion solution for industrial vision inspection systems, significantly improving the detection rate and classification robustness of multi-scale defects on the surface of resistor sheets.

[0084] To verify the above results, a comparative experiment was conducted. In the experiment, group A used a cross-layer fusion module (bidirectional cross-layer attention network + depthwise separable convolution); group B used the traditional feature pyramid network (FPN) fusion method. The experiment included a resistor sheet sample library (containing defects such as scratches, oxide spots, and cracks) and a simulated vibration environment on the production line (10-50Hz random disturbance). The experimental data are shown in Table 2.

[0085] Table 2. Data Recording Table for Cross-Layer Fusion Module Application Experiment

[0086] experimental group Number of experiments Detection accuracy (IoU) Fusion time (ms) Minor defect detection rate (%) False positive rate (%) Positioning error (μm) under vibration interference Group A 1 0.92 12.3 98.5 1.2 18.7 Group A 2 0.91 13.1 97.8 1.5 20.1 Group A 3 0.93 11.9 99.0 0.9 17.3 Group A 4 0.94 12.5 98.2 1.1 19.5 Group A 5 0.92 13.0 98.7 1.3 18.9 Group B 1 0.83 38.6 89.2 6.7 52.4 Group B 2 0.85 37.9 90.1 5.9 49.8 Group B 3 0.84 39.2 88.7 7.3 54.1 Group B 4 0.82 40.1 87.5 8.1 57.3 Group B 5 0.86 38.5 89.8 6.5 50.6

[0087] Experiments show that the cross-layer fusion module is significantly better than traditional methods in terms of detection accuracy, real-time performance, and adaptability, meeting the needs of complex working conditions in high-speed production lines.

[0088] In the graph network classification module described above, the prior parameters include geometric feature parameters, physical property parameters, and process parameters. The geometric feature parameters include defect area threshold, shape factor, and edge gradient consistency. The defect area threshold includes micro-defects (0.05~0.2 mm²), medium-sized defects (0.2~1 mm²), and macro-defects (>1 mm²). The shape factor includes ellipticity (>0.6 indicates a regular defect) and fractal dimension (1.2~1.8 characterizes crack complexity). Edge gradient consistency is calculated based on the Sobel operator, using the standard deviation of the edge gradient direction (<15° indicates a continuous crack). The physical property parameters include thermal conductivity deviation and resistivity change rate; thermal conductivity deviation is the difference between the local thermal conductivity extracted by near-infrared imaging and the standard value (>10% is judged as internal delamination); resistivity change rate is combined with the measured data of the four-probe method, and local resistivity fluctuation >5% is marked as an abnormal conductivity defect; the process parameters include slurry diffusion coefficient and layer offset; the slurry diffusion coefficient is calculated based on the printing speed and slurry viscosity to determine the diffusion radius (exceeding ±0.1mm is judged as a process defect); the layer offset is calculated by 3D morphology reconstruction to determine the cumulative error of multi-layer printing offset (>0.05mm triggers graded warning);

[0089] The defect types include surface defects and structural defects; wherein surface defects include:

[0090] Microcracks (Type-S1): width < 0.02 mm, length 0.1~0.5 mm, edge gradient consistency > 0.8;

[0091] Oxidation spots (Type-S2): circular areas (ellipticity > 0.9), with near-infrared reflectance 20% lower than the baseline value;

[0092] Slurry overflow (Type-S3): Diffusion coefficient out of tolerance + edge fractal dimension > 1.5;

[0093] Structural defects include:

[0094] Local delamination (Type-C1): Thermal conductivity deviation >15% and resistivity change rate >8%;

[0095] Edge warping (Type-C2): 3D topography curvature radius < 5mm, stack offset > 0.08mm;

[0096] Internal bubbles (Type-C3): Ultrasonic echo signal attenuation rate >30%, X-ray transmission grayscale variance >25%.

[0097] The defect level is 4, including:

[0098] (1) Grade A (minor defects): only surface micro-defects (Type-S1 / S2), area <0.2mm², resistivity change rate <3%, online repair is allowed; thermal conductivity deviation <5%, process adjustment is not triggered.

[0099] (2) Grade B (acceptable defects): surface defects (Type-S3) or minor structural defects (Type-C1 / C2), area 0.2~1mm²; resistivity change rate 3%~8%, need to be downgraded and process parameter fluctuations recorded.

[0100] (3) Grade C (serious defects): composite defects (such as S3+C1), area > 1 mm², thermal conductivity deviation > 10%; stack offset > 0.1 mm or internal bubble density > 5 bubbles / cm², forced to be taken offline for repair.

[0101] (4) Class D (fatal defects): Structural failure (e.g., Type-C3 bubble diameter > 0.5 mm), resistivity change rate > 15%; thermal conductivity deviation > 20% or edge warping leading to a decrease in mechanical strength > 40%, directly scrapped.

[0102] In implementation, during the node initialization phase, the graph network classification module employs the Node2Vec embedding algorithm to learn the representation of the input multi-scale feature tensors (edge ​​gradients, texture density, deformation curvature). A second-order random walk strategy generates node sequences, with a walk path length of 10 to capture local neighborhood structures. A Skip-gram model with a window size of 5 optimizes the objective function through negative sampling, mapping nodes to a 128-dimensional latent space to generate initial feature vectors. This process preserves the topological relationships of defective features by maximizing node co-occurrence probabilities, providing both geometric and semantic constraints for subsequent graph construction. In the dynamic graph topology construction phase, potential edge connections are filtered based on a cosine similarity threshold (>0.7). Dynamic edge weights are calculated using the multi-head attention mechanism (Head=4) of the Graph Attention Network (GAT): query vectors and key vectors are generated through linear projection of node embeddings. The LeakyReLU activation function (negative slope=0.2) introduces non-linear transformation capabilities. The attention coefficient calculation formula is achieved by multiplying the transpose of the learnable parameter vector α with the matrix of concatenated node features. After weight normalization, an edge weight matrix is ​​generated. This mechanism overcomes the limitations of traditional fixed similarity thresholds, achieving adaptive quantification of the correlation strength of defect features. In the self-supervised message passing phase, a three-layer iterative update strategy based on Gated Recurrent Units (GRUs) is designed: the message function projects neighborhood node features to the latent space through the message transformation matrix W_msg; the aggregation function uses mean pooling to fuse neighborhood weighted information; and the update and reset gates of the GRU dynamically control the fusion ratio of historical states and current inputs, suppressing interference from noisy nodes. Inter-layer residual connections maintain gradient flow stability through identity mapping, preventing deep network degradation. The online hard example mining strategy filters samples with classification uncertainty using a confidence threshold (0.85), triggering the incremental learning module of Node2Vec: fine-tuning the embedding vectors of hard example node sequences based on the Skip-gram model, dynamically updating the graph topology to adapt to production line process fluctuations.

[0103] In implementation, the improved XGBoost multi-label classifier comprises four dimensions: multi-label tree structure expansion, loss function reconstruction, feature fusion optimization, and online adaptive calibration. The multi-label tree structure expansion algorithm upgrades the traditional single-objective decision tree into parallel subtree groups, each subtree corresponding to a specific defect type (e.g., scratches, oxidation, delamination), reducing redundant computation by sharing feature split points. The dynamic feature weighting mechanism concatenates the node weight vectors output by the graph network with the original features, employing sparse matrix compression technology (CSR format) to reduce the curse of dimensionality. The feature grouping embedding strategy strengthens key feature channels through attention weights. In the objective function reconstruction, a Focal Loss function is introduced to address the class imbalance problem: the class balance factor γ=2 reduces the gradient weight of high-frequency defects (e.g., microcracks) while increasing the loss contribution of difficult-to-classify samples (e.g., composite defects). In the formula, the adjustment factor (1-p)^γ dynamically suppresses the gradient update amplitude of easily classified samples. The tree split point selection module employs a proximal gradient descent algorithm to optimize gain calculation. L1 regularization constrains the sparsity of split features; if a feature weight falls below a threshold θ=0.05, a pruning mechanism is triggered, skipping low-contribution features to improve model efficiency. The online incremental learning interface dynamically adjusts model parameters using a Delta update strategy: when the reinforcement learning module reports fluctuations in process parameters, the classifier loads an incremental dataset (containing only new defect samples) for local fine-tuning, avoiding the computational overhead of global retraining. The Shapley value interpreter integrates game theory principles to dynamically analyze feature contribution: it calculates the contribution of each feature to the classification result through marginal effects. When a key feature (such as the resistivity index of local delamination) is detected to deviate from the baseline by 20%, an online calibration module is triggered to compensate and correct the weights of the corresponding tree nodes. This technology system significantly improves the accuracy of classifying complex defect patterns through dynamic weight allocation and multi-objective collaborative optimization, while maintaining the real-time requirements of the production line.

[0104] In the collaborative framework of the graph network classification module and the improved XGBoost, the graph network transforms multi-scale defect features into structured semantic representations through heterogeneous relationship modeling, while XGBoost achieves multi-label hierarchical decision-making based on dynamic weight features. The topological relationships captured by the Node2Vec embedding algorithm provide prior knowledge constraints for XGBoost, reducing the tree model's dependence on redundant features; while the difficult examples fed back by XGBoost drive the incremental optimization of the graph network, forming a bidirectional knowledge transfer mechanism. At the hardware deployment level, the message passing iteration of the graph network and the parallel tree growth of XGBoost are achieved through a heterogeneous computing architecture: graph computing tasks are allocated to GPU acceleration, XGBoost's tree splitting optimization is executed by CPU multi-threading, and data pipeline synchronization is achieved through RDMA high-speed interconnect. This technology system deeply combines the associative reasoning capabilities of the graph model with the efficient decision-making characteristics of the tree model, providing a fully intelligent solution for resistor chip defect detection from feature representation to hierarchical decision-making, significantly improving the system's detection accuracy and adaptability in complex industrial environments.

[0105] During implementation, the graph network classification module performs real-time graph structure construction and message passing calculations on the FPGA. It has built-in dual-channel DDR4 memory (32GB, bandwidth 68.3GB / s) for caching 128-dimensional node embedding vectors and feature weight matrices. The Node2Vec embedding initialization phase is implemented on the FPGA, using a hardware description language to construct a random walk controller with a walk length parameter of 10 and a window size of 5. Dual-port Block RAM is used to store the node adjacency list, and the latency for generating a single walk sequence is controlled within 2.8ms. The Graph Attention Network (GAT) is deployed using the TensorRT acceleration engine. The multi-head attention layer is configured with four independent attention heads, each with a dimension of 32. The LeakyReLU activation function has a negative slope parameter α=0.2. Dynamic edge weight calculation is achieved through matrix concatenation and nonlinear transformation using CUDA kernel functions. Cosine similarity calculation of the 128-dimensional node vectors is processed in parallel, and the inference latency of a single-layer GAT is less than 6.5ms. In the message passing mechanism, the GRU unit implements gating logic on the FPGA using LUT resources. The hidden layer dimension is set to 256, and the message transformation matrix W_m dimension is 256×256. FP16 half-precision quantization is used to compress storage space. The improved XGBoost classifier is deployed on a CPU cluster (Intel Xeon Platinum 8380), receiving dynamic weight feature vectors via RDMA protocol. The multi-label tree structure is configured with 100 parallel subtrees (corresponding to scratches, oxidation, and delamination defects), with a maximum depth of 12 for each tree, a learning rate η=0.15, and an L2 regularization term λ=1.2. The online hard example mining module sets the confidence threshold to 0.85. When incremental training of the Skip-Gram model is triggered, the 128-dimensional embedding vector is updated using an asynchronous gradient descent algorithm, with a batch size of 256 and a momentum coefficient β=0.9. The Shapley value interpreter is integrated at the end of the inference pipeline. When the feature contribution deviates from the baseline by 20%, an online calibration module is activated. This module corrects the tree node split gain weights using Newton's iteration method, with an upper limit of 50 iterations. In terms of hardware resource utilization, the FPGA logic unit utilization rate is 72% (LUT: 398K / 552K), the Jetson platform GPU load is stable at 83%-87%, and the CPU cluster memory bandwidth peaks at 42GB / s.

[0106] Traditional graph convolutional networks rely on fixed adjacency matrices to model feature relationships. This solution, however, uses a Node2Vec embedded random walk strategy combined with the dynamic attention mechanism of a graph attention network to adaptively construct the topological relationships of defect features. Node2Vec's deep random walk strategy captures the relationship between local microstructures (such as crack bifurcation) and macroscopic deformation by adjusting the balance parameters of BFS (breadth-first search) and DFS (depth-first search). Meanwhile, the multi-head attention layer of the graph attention network strengthens the coupling between high-frequency gradient features and low-frequency semantic features through cross-node interactions activated by LeakyReLU. At the message passing level, a three-layer iterative GRU gating mechanism integrates residual connections to alleviate the gradient decay problem in deep networks and suppresses information interference from noisy nodes through a gating screening mechanism, improving the signal-to-noise ratio of feature propagation. The essential innovation of the improved XGBoost lies in the joint optimization of a multi-label tree structure and dynamic feature weighting: the multi-label tree decouples the classification boundary of composite defects through a parallel subtree architecture, and the class balance factor dynamically adjusts the loss weight of low-frequency defects (such as delamination) based on the gradient reconstruction objective function, addressing the sensitivity of traditional methods to sample imbalance. The Shapley value interpreter quantifies the marginal effect of graph node weights on classification results through inverse gradient propagation, establishing an interpretable model to achieve real-time diagnosis of abnormal fluctuations in process parameters (such as resistivity). This graph-tree collaborative heterogeneous model architecture retains the expressive power of graph neural networks for non-Euclidean space features while leveraging the classification robustness of XGBoost in high-dimensional sparse data, solving the challenge of dynamic coupling detection of multi-scale defects in resistive sheets.

[0107] The core optimization of the improved XGBoost multi-label classifier lies in reconstructing the classification boundary modeling method through a multi-label tree structure expansion algorithm. Parallel subtrees are independently constructed for defect types such as scratches, oxidation, and delamination, eliminating feature coupling interference in multi-label association scenarios using traditional single-tree structures. A dynamic feature weighting mechanism is introduced, combining the dynamic weight vector output by the graph neural network to generate a sparse feature compression matrix. The objective function is reconstructed using Focal Loss to dynamically adjust the class balance factor, reducing the gradient weight of high-frequency defects (such as scratches) and enhancing the sensitivity to low-frequency composite defects (such as oxidation accompanied by cracks). An online calibration module driven by Shapley values ​​is integrated to analyze the deviation between feature contribution and process parameters in real time, triggering Newton's iteration method to correct tree node weights and improving the model's adaptability to dynamic process conditions such as coating thickness fluctuations and sintering temperature shifts. These improvements enable the classifier to maintain its advantages in high-dimensional sparse data processing while significantly enhancing its robustness to multi-label imbalanced data. Furthermore, hardware acceleration achieves a synergistic improvement in computational efficiency and interpretability.

[0108] To validate the improved XGBoost multi-label classifier, a comparative experiment was conducted. The experiment used an NVIDIA Jetson AGX Orin and Xilinx Zynq UltraScale+ FPGA hardware platform to build the comparison environment. Group A deployed the improved XGBoost multi-label classifier, while Group B used a standard XGBoost multi-label classification method combined with a MultiOutputRegressor. Both groups were tested on the same resistor sheet defect dataset (containing three labels: scratches, oxidation, and delamination, with a sample ratio of 10:3:1). Group A used a dynamic feature weighting mechanism (FL loss function parameters γ=2, α=0.25), while Group B used the standard cross-entropy loss function. The tree depth was uniformly set to 12 layers, the learning rate was 0.1, and the training iterations were 500. Evaluation metrics included multi-label mean accuracy (mAP), single training time, feature importance accuracy (Pearson correlation based on Shapley values), class balance F1 difference (the absolute difference between the highest and lowest F1 values), and parameter calibration response time. During the experiment, Group A implemented dynamic weight feature selection using the sparse matrix compression module on the FPGA, while Group B relied on the CPU to participate in the calculation of all features. In terms of hardware resource allocation, Group A utilized the FPGA for parallel acceleration of feature compression and weight updates, while Group B used the CPU for serial processing throughout. The experimental records are shown in Table 3.

[0109] Table 3. Comparison Data Record of Improved XGBoost Multi-Label Classifier

[0110] experimental group frequency mAP (%) Training time (ms) Feature Importance Accuracy (Pearson) Class balance F1 difference (Δ) Parameter calibration response (ms) Group A 1 92.3 1450 0.87 0.12 38 2 91.8 1432 0.85 0.15 41 3 93.1 1467 0.89 0.11 37 4 92.5 1429 0.86 0.13 40 5 92.7 1445 0.88 0.14 35 Group B 1 84.6 1689 0.62 0.34 112 2 85.1 1675 0.65 0.32 108 3 83.9 1698 0.59 0.37 115 4 84.2 1703 0.61 0.35 110 5 85.4 1681 0.63 0.33 106

[0111] Experimental results show that the improved XGBoost multi-label classifier achieves a multi-label average accuracy (mAP) of 92.48%, a 7.84% improvement over the ordinary XGBoost's 84.64%, with a particularly significant 23.6% improvement in the F1 score for low-frequency delamination defects, validating the effectiveness of dynamic feature weighting and class balance factors. Regarding training time, the average time for group A was 1444.6 ms, a 14.5% reduction compared to group B's 1689.2 ms, demonstrating the computational advantages of FPGA sparsity compression and hardware acceleration. Feature importance accuracy (Pearson coefficient 0.87 vs. 0.62) improved by 40.3%, proving that the Shapley value interpreter can more accurately quantify the contribution bias of process parameters. Parameter calibration response time decreased from 110.2 ms in group B to 38.2 ms, a 65.3% reduction, indicating that the online calibration mechanism significantly improves the model's real-time adaptability to dynamic process fluctuations. The class imbalance F1 difference decreased from 0.34 to 0.13, indicating that the improved scheme effectively alleviated the classification bias of traditional methods in multi-label imbalance scenarios.

[0112] In the parameter optimization module described above, the Actor network generates the action space (convolution kernel size, classification threshold, and filter intensity) through the policy function π(a|s), and the Critic network evaluates the state value based on the value function V(s). Both networks calculate the advantage function A(s,a) using generalized advantage estimation (GAE) to constrain the policy update amplitude and avoid policy oscillations. The state observation vector is composed of production line process parameters (slurry viscosity, printing speed) and defect statistical features (type distribution, confidence variance) output by XGBoost, and is then normalized before being input into a fully connected network (512 neurons). To address transmission noise interference, the memristor random computation unit encodes the policy parameters into a probability pulse sequence: through the stochastic resonance effect, when the pulse amplitude is lower than the noise floor, the signal features are enhanced using a nonlinear bistable system (Langevin equation), with the dynamic model being dx / dt = -dV(x) / dx + ξ(t), where V(x) is the potential well function and ξ(t) is Gaussian white noise. The optimized parameters are encapsulated using the Probabilistic Pulse Amplitude Modulation (PPM) protocol and transmitted to the front-end module via differential signals through the SPI interface. The online fine-tuning mechanism is based on the Dual-Delay Deep Deterministic Policy Gradient (TD3) algorithm, which suppresses Q-value overestimation through target network delay updates and policy smoothing regularization. The target Q-value is calculated as follows: , where ϵ represents truncated normal distribution noise. Parameter version management uses a circular queue to store historical strategy snapshots. When classifier performance fluctuations exceed limits, the optimal version is retrieved and rolled back using cosine similarity to ensure system stability.

[0113] In terms of hardware deployment, the parameter optimization module integrates an NVIDIA Jetson AGX Xavier edge computing node and a memristor array (Crossbar architecture), connecting to a Basler line scan camera via a CoaXPress 2.0 interface and an SPI interface (50MHz) to the FPGA controller. The PPO model is deployed in a CUDA accelerated environment, the Actor-Critic network is implemented using the PyTorch framework, the batch size is set to 256, the GAE parameter λ=0.95, and the discount factor γ=0.99. The memristor unit is configured as a 128×128 crosspoint array, the pulse code uses a hybrid modulation of 4-bit amplitude and 2-bit timing, and the noise floor is dynamically calibrated through an adaptive threshold detection circuit. Production line process parameters are acquired in real time from the PLC controller via the Modbus TCP protocol at a sampling frequency of 1kHz. XGBoost defect data is serialized in Apache Arrow format and then input into the state vector construction module. At the software level, the TD3 algorithm's policy network update interval is set to 2 training cycles, and the target network smoothing coefficient τ = 0.005. The stochastic resonant coding parameters are determined through offline calibration: potential well depth V_0 = 0.5 eV, noise intensity D = 0.1. The parameter version queue capacity is set to 10, and the rollback trigger condition is three consecutive F1 score fluctuations with a standard deviation > 5%. In actual deployment, the slurry viscosity baseline is calculated using a historical data sliding window (window size 1000 samples), and the fine-tuning trigger threshold is set to a 20% standard deviation offset. After receiving SPI data packets, the deformable convolution controller on the FPGA side updates the convolution kernel offset lookup table (LUT) through register mapping, and the threshold adjuster uses a PID control algorithm to dynamically adjust the classification confidence boundary.

[0114] This module proposes an online optimization framework based on a PPO-TD3 hybrid strategy. It achieves real-time matching of detection parameters with process fluctuations through an Actor-Critic architecture, solving the lag problem of traditional static parameter tuning. Secondly, it designs a memristor stochastic resonance coding scheme to transform strategy parameters into noise-resistant pulse sequences, leveraging the noise enhancement characteristics of nonlinear systems to overcome the bit error rate limitations of traditional binary transmission. Finally, it develops a parameter version management mechanism that supports incremental optimization of production line processes while ensuring detection stability through the rollback of historical strategy snapshots and incremental fine-tuning. These technological innovations enable the system to maintain high-precision detection in noisy and highly volatile industrial environments, while achieving end-to-end adaptive control from parameter optimization to hardware control, providing theoretical methods and engineering practice support for quality control in complex manufacturing scenarios.

[0115] While specific embodiments of the present invention have been described above, those skilled in the art should understand that these specific embodiments are merely illustrative. Those skilled in the art can omit, substitute, and modify the details of the above methods and systems in various ways without departing from the principles and essence of the present invention. For example, combining the above method steps to perform substantially the same function and achieve substantially the same result according to substantially the same method falls within the scope of the present invention. Therefore, the scope of the present invention is defined only by the appended claims.

Claims

1. An online defect detection system for resistor sheets based on machine vision; characterized in that: include: The dynamic compensation module is used to collect RGB and near-infrared dual-modal data from a linear array camera and a multispectral light source. It employs the Retinex multi-scale illumination compensation algorithm to eliminate reflections and suppresses high-frequency noise through bilateral filtering and non-local mean denoising to obtain a standardized image. Within the dynamic compensation module, the Retinex multi-scale illumination compensation algorithm performs multi-channel decomposition on the original RGB image using Gaussian convolution kernels of 15, 80, and 200 scales to extract illumination and reflection components, and eliminates illumination unevenness through logarithmic domain transformation. For reflective areas of the metal foil, the Retinex multi-scale illumination compensation algorithm detects abrupt gradient changes in highlight pixels using Gaussian difference filtering, generates a reflectivity mask matrix, and uses an adaptive weighting function to non-linearly attenuate the reflection components in the highlight areas, generating a reflectivity-corrected image. After eliminating reflections, the dynamic compensation module calculates the weighted mean of the pixel neighborhood using a bilateral filtering kernel and constructs a 15×15 search window using a non-local mean algorithm. It then filters 7×7 similar blocks using block similarity measurement and eliminates salt-and-pepper noise and Gaussian noise using a weighted average iteration method. After denoising, the data is encapsulated into a dual-channel pixel stream using the AXI-Stream protocol, and then aligned and converted into a standardized dual-modal image by a parallel multiply-accumulator array within the FPGA. The multi-scale perception module is used to dynamically adjust the sampling offset of the convolution kernel based on the standardized image through the local gradient entropy driving mechanism of the deformable convolutional network. It employs dilated spatial pyramid pooling to embed multi-dilation rate convolution kernels in Conv1 to Conv4, generating multi-scale feature tensors for high-frequency edges, mid-level textures, and global semantics, which are then output to the cross-layer fusion module. The working method of the multi-scale perception module is as follows: s1. The gradient magnitude of each pixel neighborhood in the input normalized image is statistically analyzed using a local gradient entropy calculation algorithm to generate a gradient entropy distribution map to quantify the region complexity. The calculation formula is as follows: （1） In formula (1), Indicates position The local gradient entropy at a given point is used to quantify the complexity of the 5×5 window pixel neighborhood; Gradient magnitude within the neighborhood The probability distribution; This represents the total number of pixels in the neighborhood. s2, when When the value is ≥1.2, the dynamic adjustment mechanism of the sampling point offset of the deformable convolutional network is triggered. The offset is calculated based on the gradient direction field, and the calculation formula is as follows: （2） In formula (2), , These represent the spatial offset of the sampling points of the convolution kernel; The local gradient direction angle is calculated using the following formula: ,in Representing coordinates and The image gradient field; for high-entropy regions, the 3×3 convolution kernel is driven to shift in the direction of gradient abrupt change, using the dilation rate. Dense convolutional kernels are used to extract high-frequency edge features; s3. Using hollow spatial pyramid pooling, embed a group of two-branch parallel convolutional kernels with dilation rates r=1 and r=6 in Conv2 layer; embed a group of two-branch parallel convolutional kernels with dilation rates r=6 and r=12 in Conv3 layer; embed single convolutional kernels with dilation rates 1 and 12 in Conv1 and Conv4 layers respectively, wherein the receptive field of Conv4 layer is expanded as follows: （3） In formula (3), This is the effective receptive field size of the convolution kernel; The expansion rate is used to control the sampling point interval; The basic convolution kernel size; s4. The outputs of multi-dilation rate convolutions are weighted and fused through a channel attention mechanism to generate cross-scale feature tensors; Furthermore, by using cross-level skip connections, the high-resolution features of Conv2 layer and the semantic features of Conv4 layer are aligned by bilinear interpolation, and a multi-scale fused feature tensor is output. The cross-layer fusion module is used to construct a spatial and channel collaborative attention mechanism between Conv2 and Conv4 layers using a bidirectional cross-layer attention network. It dynamically weights feature associations through a multi-head self-attention model, and generates enhanced multi-scale feature maps through depthwise separable convolutional dimensionality reduction. The working principle of the bidirectional cross-layer attention network is as follows: The 112×112 high-resolution feature map of Conv2 layer is upsampled to 14×14 resolution using a bilinear interpolation algorithm to generate low-level detail features aligned with the semantic feature space of Conv4 layer. A cross-layer relationship matrix is ​​constructed using a multi-head self-attention model. The query vector Q of the cross-layer relationship matrix is ​​generated by 1×1 convolution of Conv4 layer features, and the key vector K and value vector V are extracted by grouped convolution of interpolated Conv2 layer features. In the spatial dimension, the cosine similarity matrix between Q and K is calculated, and a spatial attention weight map is generated by Softmax normalization. The edge details of Conv2 layer and semantic context of Conv4 layer are dynamically weighted and fused. At the same time, in the channel dimension, a squeeze-excitation network is used to calculate channel attention weights, and the fused features are recalibrated by channel after activation by the Sigmoid function. Feature dimensionality reduction is achieved through depthwise separable convolution, where depthwise convolution uses a 3×3 kernel to extract local spatial relationships, and pointwise convolution uses a 1×1 kernel to achieve cross-channel information interaction. The resulting multi-scale enhanced feature map is added to the original Conv4 layer features via skip connections, and a residual learning mechanism is used to avoid gradient vanishing. The graph network classification module maps the enhanced multi-scale feature map into a heterogeneous feature relationship graph. Nodes represent multi-scale defect attributes, including edge gradient, texture density, and deformation curvature. A self-supervised message passing mechanism is constructed based on a graph attention network and the Node2Vec embedding method, outputting dynamic weight features to an improved XGBoost multi-label classifier. Combined with prior parameters, defect types and defect levels are generated. The defect types include surface defects and structural defects; the defect levels are four. The parameter optimization module is used to optimize the online iterative strategy function of the model based on the production line parameters collected in real time from the edge nodes and the defect data output by the improved XGBoost multi-label classifier. It dynamically adjusts the convolution kernel size and classification threshold, and uses a memristor random calculation method to suppress transmission noise with random resonant coding. The optimized parameters are then fed back to the multi-scale perception module and the graph network classification module.

2. The online defect detection system for resistor sheets based on machine vision according to claim 1, characterized in that: The multi-head self-attention model includes a cross-modal projection layer, a spatial relationship modeling layer, a multi-head decomposition layer, a dynamic weight fusion layer, a residual connection layer, and a regularized output layer. The cross-modal projection layer upsamples the Conv2 layer features to a resolution of 14×14 using a bilinear interpolation algorithm, and performs linear projection on the bimodal features using grouped convolution to generate a query vector Q, a key vector K, and a value vector V. The projection weight matrix has a dimension of 512×512. The spatial relationship modeling layer calculates the cosine similarity matrix between Q and K by scaling dot product attention. The similarity score is normalized by Softmax to generate a spatial attention weight map with a weight matrix dimension of 14×14×14×14. The multi-head decomposition layer is used to decompose the weight matrix into 8 independent attention heads along the channel dimension. Each attention head extracts local spatial association features through an independent fully connected layer and outputs a multi-head attention feature group. The dynamic weight fusion layer merges multi-head features through a channel splicing method and uses a gated recurrent unit to dynamically learn the contribution weights of each attention head to generate fused global attention features. The residual connection layer fuses the attention-enhanced features with the original features by adding elements one by one, triggering a residual learning mechanism to preserve the underlying semantic information; the regularized output layer uses layer normalization and depthwise separable convolution to reduce the dimensionality of features and outputs an enhanced multi-scale feature map.

3. The online defect detection system for resistor sheets based on machine vision according to claim 1, characterized in that: The working method of the graph network classification module is as follows: u1. Based on the Node2Vec embedding algorithm, the input multi-scale feature tensor is initialized with nodes. A random walk strategy is used to generate a node sequence, and a 128-dimensional node feature vector is learned using the Skip-gram model. The walk length is set to 10, and the window size is 5 to capture local structural information; the initialized node set The initial vertices that form the graph; u2. Based on cosine similarity, the correlation between node features is calculated. If the similarity threshold exceeds a preset threshold, an initial graph topology is constructed through an improved edge connection generation mechanism. The improved edge connection generation mechanism calculates the attention coefficients between nodes through the multi-head attention layer of the graph attention network, and uses the LeakyReLU activation function to perform a nonlinear transformation on the features of adjacent nodes to generate dynamic edge weights. The calculation formula is as follows: （4） In formula (4), For nodes and The dynamic edge weights are used to characterize the correlation strength of defect features; and This represents the node embedding vector initialized by Node2Vec, with dimension d=128; This represents the LeakyReLU activation function with a negative slope of 0.2; This is a vector concatenation operation; Represents the learnable parameter vector The transpose of the matrix; Represents a node The set of neighboring nodes; For the iteration variable in the summation operation; u3. Utilize a self-supervised message passing mechanism to iteratively update node features, performing three-layer iterative message propagation. Residual connections are used between layers to maintain gradient flow. Each layer updates the state of the central node through a message passing function. The formula expression is: （5） In formula (5), For the first Node at the next iteration eigenvectors; The message transformation matrix; This is a gated loop unit used to filter valid neighborhood information; u4. After three message passes, the node features are mapped into dynamic weight vectors through a fully connected layer and output to the feature input layer of the improved XGBoost classifier. At the same time, samples with a classification confidence score below 0.85 are selected through an online hard example mining strategy, triggering the Node2Vec random walk sequence generation module to incrementally update the node embedding representation based on the Skip-Gram model.

4. The online defect detection system for resistor sheets based on machine vision according to claim 1, characterized in that: The improved XGBoost multi-label classifier receives dynamic weight feature vectors output by a graph neural network through a dynamic feature weighting mechanism. It then constructs parallel decision tree groups using a multi-label tree structure expansion algorithm, where each subtree corresponds to a specific defect type, including scratches, oxidation, and delamination. Next, based on FL reconstruction of the objective function, it reduces the gradient weights of high-frequency defects through a class balancing factor while increasing the loss contribution of composite defects. For the dynamic weight features, a feature grouping embedding strategy is used to concatenate the node weight vectors with the original features, which are then compressed using a sparse matrix and input to the tree split point selection module. During the tree growth stage, the improved XGBoost multi-label classifier uses a proximal gradient descent algorithm to optimize the split gain calculation. If the feature weights are below a preset threshold, a pruning mechanism is triggered to skip irrelevant features. The classification results are transmitted to the reinforcement learning module through an online incremental learning interface, triggering a model parameter update signal. At the same time, the Shapley value interpreter is integrated to dynamically analyze the feature contribution. When the resistivity feature contribution of local delamination is detected to deviate from the baseline by 20%, the tree node weights are corrected through an online calibration method.

5. The online defect detection system for resistor sheets based on machine vision according to claim 1, characterized in that: The parameter optimization module collects production line process parameters and defect statistical features output by the XGBoost classifier in real time through edge computing nodes, and constructs a state observation vector to be input into the near-end policy optimization model. The near-end policy optimization model performs policy evaluation and optimization in parallel through the Actor-Critic architecture. The Actor network calculates the convolution kernel size, classification threshold and morphological filter intensity based on the dominance function, and the Critic network uses the TD error estimate to constrain the policy update magnitude. To address transmission noise interference, the parameter optimization module encodes the strategy parameters into a probability pulse sequence using a memristor random computation unit. A random resonance coding mechanism is employed, and when the pulse amplitude is below the noise floor, a nonlinear random resonance method is used to enhance signal characteristics. The optimized parameters are encapsulated into data packets using a probability pulse amplitude modulation protocol and transmitted back to the deformable convolution controller of the multi-scale sensing module and the threshold adjuster of the graph network classification module via the SPI interface. If the standard deviation of the slurry viscosity is detected to exceed the historical baseline by 20%, an online fine-tuning mechanism for the strategy network is triggered, updating the network weights based on a dual-delay deep deterministic strategy gradient algorithm. Simultaneously, a parameter version management queue is established. When three consecutive strategy updates cause the classifier's F1 score to fluctuate by more than 5%, a version rollback mechanism is triggered to load a snapshot of the historically optimal parameters.

6. The online defect detection system for resistor sheets based on machine vision according to claim 1, characterized in that: The prior parameters include geometric feature parameters, physical property parameters, and process parameters; the geometric feature parameters include defect area threshold, shape factor, and edge gradient consistency; the physical property parameters include thermal conductivity deviation and resistivity change rate; and the process parameters include slurry diffusion coefficient and lamination offset.

7. A method for online classification of defects in resistive sheets based on machine vision, characterized in that: The online defect detection system for resistor sheets based on machine vision, as described in any one of claims 1-5, comprises: Step 1: Simultaneously acquire RGB images and near-infrared reflection data streams of the resistive sheet surface using a linear industrial camera and a multispectral ring light source. Use the Retinex multi-scale illumination compensation algorithm to eliminate metal foil reflection interference. Combine non-local mean and wavelet threshold joint denoising model to suppress high-frequency noise and generate standardized dual-modal image data. Step 2: Based on the local gradient entropy driving mechanism, a deformable convolutional network is used to dynamically adjust the sampling point offset of the convolutional kernel. Through multi-dilation rate convolutional kernels with hollow space pyramid pooling, the features of cracks below 0.2mm, slurry diffusion deformation and overall tilting are captured at Conv1 to Conv4 levels to generate multi-scale feature tensors. Step 3: Construct a spatial and channel collaborative attention mechanism using a bidirectional cross-layer attention network with 112×112 Conv2 layers and 14×14 Conv4 layers. Calculate the association weights between high-resolution details and low-resolution semantics through a multi-head self-attention model. After dynamic weighted fusion, the enhanced multi-scale feature map is generated by depthwise separable convolution dimensionality reduction. Step 4: Map the enhanced multi-scale feature map to a heterogeneous feature relationship graph. Nodes represent multi-scale defect attributes, including edge gradient, texture density and deformation curvature. Initialize edge connection rules based on graph attention network and Node2Vec embedding algorithm. Model the physical mapping between exposed foil spots and overall warping through self-supervised message passing mechanism. Output dynamic weight features to the improved XGBoost classifier. Combine prior parameters to generate defect type and defect level. Step 5: Real-time acquisition of production line process parameters and historical defect database through edge computing nodes; online iteration of strategy function through near-end strategy optimization model; dynamic adjustment of convolution kernel size and classification threshold; and memristor random calculation method with random resonant coding to suppress transmission noise. The optimized parameters are then transmitted back to steps 2 and 4 via probabilistic pulse fault-tolerant transmission.

Images