Resistor disc defect online detection system and grading method based on machine vision

Abstract

The invention discloses a machine vision-based resistor disc defect online detection system and a grading method, relates to the technical field of industrial machine vision detection, and solves the defect problems in the aspects of multi-scale defect dynamic perception, cross-level feature interaction and process adaptive optimization in the prior art. According to the scheme, metal reflection interference is inhibited through Retinex illumination correction and a combined denoising model; adopting a deformable convolution kernel and cavity space pyramid pooling to realize gradient entropy driving dynamic sensing of the multi-scale defect; constructing a bidirectional cross-layer attention network to realize early fusion of high-resolution details and high-level semantics; modeling local-global feature physical association based on a graph attention network and a self-supervised message passing mechanism; integrating reinforcement learning and a memristor random calculation unit to form a closed-loop parameter optimization system; according to the method, the multi-scale defect detection precision, the cross-modal feature fusion efficiency and the system adaptive capacity under complex working conditions are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial machine vision detection, and more particularly to an online detection system and grading method for resistor sheet defects based on machine vision. Background Art

[0002] The machine vision-based online defect detection and grading system for resistor chips uses high-precision industrial cameras to capture surface images, combines multimodal light sources to optimize image quality, and employs deep learning algorithms and feature engineering to segment and classify defects. This system can detect defects such as scratches, oxide layer loss, and microcracks in real time, automatically grading them based on parameters such as defect size and density. It is widely used in electronic component manufacturing quality control, significantly improving inspection efficiency and accuracy.

[0003] Existing technologies primarily achieve defect detection through a combination of image acquisition hardware optimization and algorithmic processing. For example, patent CN105973912A uses a linear array industrial camera combined with a linear light source to capture surface texture. Defect areas are extracted through grayscale difference analysis, and features such as scratches and spots are segmented using morphological filtering. Area and density parameters are then used for classification. Patent CN108537772A proposes a multi-angle continuous imaging strategy for printed defects in positive conductors of chip resistors. This strategy uses image registration to compare standard templates with measured samples, combining contour recognition and the number of pixel changes to determine defect type. Zhu Zhengtao et al. proposed an improved Hough transform algorithm that uses dual-circle detection and low-pass filtering interpolation to locate gaps in circular resistors and extract spot features. Classification is achieved by combining the inscribed circle radius parameter. U.S. Patent US20140198974A1 utilizes high-resolution 3D imaging technology, extracts microcrack features using a convolutional neural network (CNN), and incorporates transfer learning to address the problem of small sample training. In addition, a Springer paper introduces a fusion of the ResNet model and the XGBoost algorithm to optimize multi-label classification performance through data augmentation. However, this approach relies on manually labeled data and is sensitive to noise. While existing technologies improve detection accuracy through hardware optimization and algorithmic improvements, they inevitably expose some shortcomings in multi-scale defect feature extraction and fusion during practical applications: First, traditional feature extraction algorithms (such as the improved Hough transform and SIFT feature matching) rely on fixed-level morphological parameters (such as area and perimeter) and are unable to dynamically adapt to defect features of varying scales. For example, patent CN105973912A uses morphological filtering to segment scratches and spots, but lacks multi-scale representation capabilities for scenarios where submillimeter cracks and oxidation spots coexist. High-frequency gradient information of small defects is easily filtered out by downsampling. While U.S. patent US20140198974A1 utilizes a CNN to extract features, its fixed convolution kernel design makes it difficult to adaptively adjust the receptive field. When the same defect (such as a scratch) appears at different scales due to slurry diffusion, manual adjustment of network depth or parameters is required, limiting real-time performance and generalization. Second, while the ResNet-50 proposed in the Springer paper extracts multi-scale features through hierarchical convolution, its encoder-decoder architecture employs a "delayed fusion" strategy, merging high-level semantic features with underlying details only at the end of the network. This results in the loss of high-resolution information of small defects (such as crack edges) during early downsampling. Although the dynamic threshold segmentation of patent CN108447051B improves adaptability, its K-means clustering relies on manually preset feature weights and cannot achieve dynamic interaction and weight distribution of cross-scale features (such as modeling the correlation between local texture and global deformation). The misjudgment rate increases significantly with the increase in defect scale differences. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the present invention discloses an online detection system and grading method for resistor defects based on machine vision, aiming to solve the problems raised in the background technology.

[0005] In order to achieve the above technical effects, the present invention adopts the following technical solutions: A resistor sheet defect online detection system based on machine vision, comprising: The dynamic compensation module is used to obtain a standardized image based on the RGB and near-infrared dual-modal data collected by the line scan camera and multispectral light source. It uses the Retinex multi-scale illumination compensation algorithm to eliminate reflections and suppress high-frequency noise through bilateral filtering and non-local means denoising. A multi-scale perception module is used to dynamically adjust the convolution kernel sampling offset based on the standardized image through the local gradient entropy driving mechanism of the deformable convolutional network, and use void spatial pyramid pooling to embed multi-expansion rate convolution kernels in Conv1 to Conv4 to generate multi-scale feature tensors of high-frequency edges, mid-order textures and global semantics, and output them to the cross-layer fusion module; The cross-layer fusion module uses a bidirectional cross-layer attention network to build a spatial and channel-coordinated attention mechanism for the Conv2 and Conv4 layers. It dynamically weights feature associations through a multi-head self-attention model and generates enhanced multi-scale feature maps through dimensionality reduction using depthwise separable convolution. A graph network classification module is used to map the enhanced multi-scale feature map into a heterogeneous feature relationship graph, where nodes represent multi-scale defect attributes, including edge gradient, texture density, and deformation curvature. A self-supervised message passing mechanism is constructed based on the graph attention network and the Node2Vec embedding method, which outputs dynamic weight features to an improved XGBoost multi-label classifier and generates defect types and defect levels based on prior parameters. The defect types include surface defects and structural defects, and the defect level is 4. The parameter optimization module is used to dynamically adjust the convolution kernel size and classification threshold based on the production line parameters collected in real time by edge nodes and the defect data output by the improved XGBoost multi-label classifier. It uses the proximal strategy optimization model to iterate the strategy function online, and adopts the memristor random calculation method and random resonance coding to suppress transmission noise. The optimized parameters are then fed back to the multi-scale perception module and the graph network classification module.

[0006] As a further technical solution of the present invention, a method for online grading of resistor sheet defects based on machine vision includes: Step 1: Using a linear array industrial camera and a multispectral ring light source, the RGB image and near-infrared reflectance data stream of the resistor surface are synchronously collected. The Retinex multi-scale illumination compensation algorithm is used to eliminate the interference of metal foil reflections. The non-local mean and wavelet threshold joint denoising model is combined to suppress high-frequency noise and generate standardized dual-modal image data. Step 2: Based on the local gradient entropy driving mechanism, a deformable convolutional network is used to dynamically adjust the offset of the convolution kernel sampling point. Through the multi-expansion rate convolution kernel of the void space pyramid pooling, the cracks below 0.2mm, slurry diffusion deformation and overall tilt features are captured at the Conv1 to Conv4 levels to generate a multi-scale feature tensor. Step 3: A bidirectional cross-layer attention network is used to construct a spatial and channel-wise collaborative attention mechanism for the 112×112 Conv2 layer and the 14×14 Conv4 layer. The multi-head self-attention model is used to calculate the association weights between high-resolution details and low-resolution semantics. After dynamic weighted fusion, the enhanced multi-scale feature map is generated through dimensionality reduction using depthwise separable convolution. Step 4: Map the enhanced multi-scale feature map into a heterogeneous feature relationship graph, where nodes represent multi-scale defect attributes, including edge gradient, texture density, and deformation curvature. Initialize edge connection rules based on the graph attention network and the Node2Vec embedding algorithm, model the physical mapping of exposed foil spots and overall warpage through a self-supervised message passing mechanism, output dynamic weight features to the improved XGBoost classifier, and generate defect types and defect levels based on prior parameters. Step 5: Use edge computing nodes to collect production line process parameters and historical defect databases in real time, iterate the policy function online through the proximal policy optimization model, dynamically adjust the convolution kernel size and classification threshold, and use the memristor random computing method and random resonance coding to suppress transmission noise. The optimized parameters are transmitted back to steps 2 and 4 through probabilistic pulse fault tolerance.

[0007] Based on the above technical solutions, the positive and beneficial effects of the present invention are: 1. The deformable convolution kernel dynamically adjusts the sampling offset direction and density based on local gradient entropy, accurately capturing high-frequency edge details of submillimeter cracks. Dilated spatial pyramid pooling uses convolution kernels with multiple dilation rates (1 / 6 / 12) to concurrently extract features from different receptive fields, covering the full scale range from microscopic cracks to macroscopic deformations. Combined with a reinforcement learning-driven parameter optimization module that adjusts convolution kernel size and classification thresholds in real time, the system can adapt to changes in defect scale caused by process fluctuations such as slurry diffusion and printing offset, completely resolving the issues of missed detection and misjudgment caused by fixed parameters in traditional methods, significantly improving detection robustness under complex working conditions.

[0008] 2. A bidirectional cross-layer attention network introduces a multi-head self-attention mechanism in the shallow encoder layer (Conv2), dynamically fusing high-resolution edge features with high-level semantic features to preserve sub-pixel geometric information of cracks. A graph neural network uses self-supervised message passing to model the physical relationship between local texture and global deformation. Combined with the Node2Vec embedding algorithm, it dynamically generates edge weights, replacing the manually preset rules of traditional K-means clustering. This technology system achieves a coherent representation from pixel-level details to semantic-level context, resolving misclassification issues caused by feature fragmentation or weight fixation, and improving the classification accuracy of complex defects.

[0009] 3. The Retinex multi-scale illumination compensation algorithm, combined with a bilateral filtering-non-local means denoising model, preserves micro-defect edges while suppressing high-frequency noise. The memristor unit utilizes stochastic resonance encoding technology to convert the parameter optimization signal into a noise-resistant pulse sequence, which is transmitted to the front-end module via probabilistic modulation. Combined with a reinforcement learning real-time iteration strategy, the system dynamically adjusts noise suppression strength and feature extraction strategies based on production line process parameters (such as slurry viscosity and printing speed), forming a closed "perception-decision-feedback" loop. This solves the problems of detail loss and limited adaptability associated with traditional methods caused by static filtering or manual parameter adjustment, ensuring consistent high-precision inspection in complex production environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive work, among which: Figure 1 This is a schematic diagram of the principle structure of a resistor sheet defect online detection system based on machine vision according to the present invention; Figure 2 This is a flowchart of the steps of the online classification method for resistor sheet defects based on machine vision of the present invention; Figure 3 This is a working principle diagram of the Retinex multi-scale illumination compensation algorithm of the present invention; Figure 4 This is a diagram showing the working principle of the bidirectional cross-layer attention network of the present invention; Figure 5 This is a principle framework diagram of the improved XGBoost multi-label classifier of the present invention; Figure 6 This is a principle framework diagram of the multi-head self-attention model of the present invention. DETAILED DESCRIPTION

[0011] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0012] In an embodiment, a resistor sheet defect online detection system based on machine vision is provided. Figure 1 Shown, including: The dynamic compensation module is used to obtain a standardized image based on the RGB and near-infrared dual-modal data collected by the line array camera and the multi-spectral light source, using the Retinex multi-scale illumination compensation algorithm to eliminate reflections and suppress high-frequency noise through bilateral filtering and non-local mean denoising; Figure 3As shown in the figure, the working principle of the Retinex multi-scale illumination compensation algorithm is as follows: the Retinex multi-scale illumination compensation algorithm performs multi-channel decomposition on the original RGB image using three-scale Gaussian convolution kernels of 15, 80, and 200, extracts the illumination component and the reflection component, and eliminates the uneven illumination through the logarithmic domain transformation method. For the reflective area of ​​the metal foil, the Retinex multi-scale illumination compensation algorithm detects the gradient mutation of the highlight pixel through Gaussian difference filtering, generates a reflective mask matrix, and uses an adaptive weight function to perform nonlinear attenuation on the reflection component of the highlight area to generate a reflectivity correction image. After eliminating the reflection, the dynamic compensation module uses a bilateral filter kernel to calculate the weighted mean of the pixel neighborhood, and uses the non-local means algorithm to construct a 15×15 search window. The block similarity metric is used to screen 7×7 similar blocks, and the weighted average iterative method is used to eliminate salt and pepper noise and Gaussian noise. The denoised data is encapsulated into a dual-channel pixel stream using the AXI-Stream protocol, aligned and converted into data format by the parallel multiplier-accumulator array in the FPGA, and outputs a standardized dual-modal image. The multi-scale perception module is used to dynamically adjust the convolution kernel sampling offset based on the standardized image through the local gradient entropy driving mechanism of the deformable convolutional network, and use void spatial pyramid pooling to embed multi-expansion rate convolution kernels in Conv1 to Conv4 to generate multi-scale feature tensors of high-frequency edges, mid-order textures and global semantics, and output them to the cross-layer fusion module. The working method of the multi-scale perception module is as follows: s1. Use the local gradient entropy calculation algorithm to perform gradient amplitude statistics on each pixel neighborhood of the input normalized image and generate a gradient entropy distribution map to quantify regional complexity. The calculation formula is: (1) In formula (1), Indicates location The local gradient entropy at , which is used to quantify the complexity of the 5×5 window pixel neighborhood; is the gradient amplitude within the neighborhood The probability distribution of is the total number of neighborhood pixels; s2, when When ≥1.2, the dynamic adjustment mechanism of the sampling point offset of the deformable convolutional network is triggered. The offset is calculated based on the gradient direction field, and the calculation formula is: (2) In formula (2), , Respectively represent the spatial offset of the convolution kernel sampling point; is the local gradient direction angle, and the calculation formula is: ,in Represents coordinates respectively and Image gradient field; for high entropy areas, drive the 3×3 convolution kernel to shift in the direction of gradient mutation, using the expansion rate The dense convolution kernel extracts high-frequency edge features; s3. Through dilated spatial pyramid pooling, a dual-branch parallel convolution kernel group with dilation rates r=1 and r=6 is embedded in the Conv2 layer; a dual-branch parallel convolution kernel group with dilation rates r=6 and r=12 is embedded in the Conv3 layer; and a single convolution kernel with dilation rates of 1 and 12 is embedded in the Conv1 and Conv4 layers respectively. The receptive field of the Conv4 layer is expanded to: (3) In formula (3), is the effective receptive field size of the convolution kernel; is the expansion rate, which is used to control the interval between sampling points; is the basic convolution kernel size; s4. The outputs of the multi-expansion rate convolution are weighted and fused through the channel attention mechanism to generate a cross-scale feature tensor. The high-resolution features of the Conv2 layer are aligned with the semantic features of the Conv4 layer through bilinear interpolation through cross-level skip connections, and a multi-scale fused feature tensor is output. The cross-layer fusion module is used to adopt a bidirectional cross-layer attention network to build a spatial and channel collaborative attention mechanism between the Conv2 layer and the Conv4 layer, dynamically weighted feature association through a multi-head self-attention model, and generate an enhanced multi-scale feature map through dimensionality reduction through deep separable convolution; Figure 4 As shown: The working principle of the bidirectional cross-layer attention network is: The 112×112 high-resolution feature map of the Conv2 layer is upsampled to 14×14 resolution using a bilinear interpolation algorithm to generate low-level detail features that are aligned with the semantic feature space of the Conv4 layer. A cross-layer relationship matrix is ​​constructed through a multi-head self-attention model. The query vector Q of the cross-layer relationship matrix is ​​generated by 1×1 convolution of Conv4 layer features, and the key vector K and value vector V are extracted by grouped convolution of interpolated Conv2 layer features. In the spatial dimension, the cosine similarity matrix of Q and K is calculated, and a spatial attention weight map is generated through Softmax normalization. The edge details of the Conv2 layer and the semantic context of the Conv4 layer are dynamically weighted and fused. At the same time, a squeeze-excitation network is used to calculate the channel attention weight in the channel dimension. After activation with the Sigmoid function, the fused features are recalibrated in the channel direction. Feature dimensionality reduction is performed through depthwise separable convolution, where the depthwise convolution uses a 3×3 kernel to extract local spatial relationships, and the point-by-point convolution uses a 1×1 kernel to achieve cross-channel information interaction; The final generated multi-scale enhanced feature map is added to the original Conv4 layer features via skip connections, and the residual learning mechanism is used to avoid gradient disappearance. Further, such as Figure 6 As shown: the multi-head self-attention model includes a cross-modal projection layer, a spatial relationship modeling layer, a multi-head decomposition layer, a dynamic weight fusion layer, a residual connection layer and a regularized output layer; the cross-modal projection layer upsamples the Conv2 layer features to a resolution of 14×14 through a bilinear interpolation algorithm, and uses grouped convolution to perform linear projection on the bimodal features to generate a query vector Q, a key vector K, and a value vector V. The projection weight matrix dimension is 512×512; the spatial relationship modeling layer calculates the cosine similarity matrix of Q and K by scaling the dot product attention, where the similarity score is normalized by Softmax to generate a spatial attention weight map, and the weight matrix dimension is 14×1 4×14×14; the multi-head decomposition layer is used to decompose the weight matrix into 8 independent attention heads along the channel dimension. Each head extracts local spatial correlation features through an independent fully connected layer and outputs a multi-head attention feature group; the dynamic weight fusion layer merges multi-head features through a channel splicing method, and uses a gated recurrent unit to dynamically learn the contribution weights of each attention head to generate a fused global attention feature; the residual connection layer fuses the attention enhancement features with the original features through element-by-element addition, triggering the residual learning mechanism to retain the underlying semantic information; the regularized output layer uses layer normalization and depth-separable convolution to perform feature dimensionality reduction and output an enhanced multi-scale feature map; The graph network classification module is used to map the enhanced multi-scale feature map into a heterogeneous feature relationship graph, where nodes represent multi-scale defect attributes, including edge gradient, texture density, and deformation curvature. A self-supervised message passing mechanism is constructed based on the graph attention network and the Node2Vec embedding method, and dynamic weight features are output to the improved XGBoost multi-label classifier. The defect type and defect grade are generated in combination with prior parameters. The working method is as follows: u1. Initialize the nodes of the input multi-scale feature tensor based on the Node2Vec embedding algorithm, generate node sequences using a random walk strategy, and obtain a 128-dimensional node feature vector using the Skip-gram model. , where the walk length is set to 10 and the window size is 5 to capture local structural information; the node set after initialization The initial vertices that make up the graph; u2. Calculate the feature correlation between nodes based on cosine similarity. If the similarity threshold exceeds a preset threshold, construct the initial graph topology structure through an improved edge connection generation mechanism. The improved edge connection generation mechanism calculates the attention coefficient between nodes through the multi-head attention layer of the graph attention network, uses the LeakyReLU activation function to perform nonlinear transformation on the features of adjacent nodes, and generates dynamic edge weights. The calculation formula is: (4) In formula (4), For nodes and The dynamic edge weight between them is used to characterize the correlation strength of defect features; and Represents the node embedding vector initialized by Node2Vec, with dimension d=128; Represents the LeakyReLU activation function with a negative slope of 0.2; It is a vector concatenation operation; Represents the learnable parameter vector The transposed matrix of Representation node The set of neighbor nodes of Iteration variable for sum operation; u3. Use the self-supervised message passing mechanism to iteratively update node features and perform three-layer iterative message propagation. Residual connections are used between layers to maintain gradient flow. Each layer updates the central node state through the message passing function. The formula is: (5) In formula (5), For the Nodes at iteration The eigenvector of is the message transformation matrix; It is a gated recurrent unit used to filter effective neighborhood information; u4. After three message transmissions, the node features are mapped into dynamic weight vectors through the fully connected layer and output to the feature input layer of the improved XGBoost classifier. At the same time, the online hard case mining strategy is used to filter samples with classification confidence lower than 0.85, triggering the random walk sequence generation module of Node2Vec, and incrementally updating the node embedding representation based on the Skip-Gram model. Figure 5As shown: the improved XGBoost multi-label classifier receives the dynamic weight feature vector output by the graph neural network through the dynamic feature weighting mechanism, adopts the multi-label tree structure expansion algorithm to construct a parallel decision tree group, where each sub-tree corresponds to a specific defect type, including scratches, oxidation, and delamination; then, based on the FL reconstruction objective function, the gradient weight of high-frequency defects is reduced through the class balance factor, while the loss contribution of complex defects is increased; for dynamic weight features, the node weight vector is spliced with the original features through the feature grouping embedding strategy, and after sparse matrix compression, it is input into the tree split point selection module; in the tree growing stage, the improved XGBoost multi-label classifier optimizes the split gain calculation using the proximal gradient descent algorithm, and if the feature weight is lower than the preset threshold, the pruning mechanism is triggered to skip irrelevant features; the classification result is transmitted to the reinforcement learning module through the online incremental learning interface, triggering the model parameter update signal; at the same time, the Shapley value interpreter dynamically analyzes the feature contribution, and when the resistivity feature contribution of local delamination deviates from the baseline by 20%, the tree node weight is corrected through the online calibration method; The parameter optimization module is used to optimize the model online iteration strategy function based on the production line parameters collected by the edge node in real time and the defect data output by the improved XGBoost multi-label classifier, dynamically adjust the convolution kernel size and classification threshold, and use the stochastic resonance coding to suppress transmission noise through the memristor random calculation method. The parameter optimization module collects the production line process parameters and the defect statistical features output by the XGBoost classifier through the edge computing node in real time, constructs a state observation vector and inputs it into the proximal strategy optimization model; the proximal strategy optimization model executes strategy evaluation and optimization in parallel through the Actor-Critic architecture, wherein the Actor network calculates the convolution kernel size, classification threshold and morphological filtering strength based on the advantage function, and the Critic network uses the TD error estimate value to constrain the strategy update amplitude; for transmission noise interference, the parameter optimization module encodes the strategy parameters into a probability pulse sequence through the memristor random calculation unit, and uses the stochastic resonance coding mechanism to enhance the signal features through the nonlinear stochastic resonance method when the pulse amplitude is lower than the noise baseline; the optimized parameters are packaged into data packets through the probability pulse amplitude modulation protocol, and are returned to the deformable convolution controller of the multi-scale perception module and the threshold adjuster of the graph network classification module through the SPI interface; if the standard deviation of the slurry viscosity exceeds the historical baseline by 20%, the online fine-tuning mechanism of the strategy network is triggered, and the network weight is updated based on the double-delay deep deterministic policy gradient algorithm; at the same time, a parameter version management queue is established, and when the F1 score of the classifier fluctuates more than 5% for three consecutive strategy updates, the version rollback mechanism is triggered to load the historical optimal parameter snapshot.

[0013] In implementation, the machine vision-based resistor sheet defect online grading method, such as Figure 2 As shown, the following steps are included: Step 1: Using a linear array industrial camera and a multispectral ring light source, the RGB image and near-infrared reflectance data stream of the resistor surface are synchronously collected. The Retinex multi-scale illumination compensation algorithm is used to eliminate the interference of metal foil reflections. The non-local mean and wavelet threshold joint denoising model is combined to suppress high-frequency noise and generate standardized dual-modal image data. Step 2: Based on the local gradient entropy driving mechanism, a deformable convolutional network is used to dynamically adjust the offset of the convolution kernel sampling point. Through the multi-expansion rate convolution kernel of the void space pyramid pooling, the cracks below 0.2mm, slurry diffusion deformation and overall tilt features are captured at the Conv1 to Conv4 levels to generate a multi-scale feature tensor. Step 3: A bidirectional cross-layer attention network is used to construct a spatial and channel-wise collaborative attention mechanism for the 112×112 Conv2 layer and the 14×14 Conv4 layer. The multi-head self-attention model is used to calculate the association weights between high-resolution details and low-resolution semantics. After dynamic weighted fusion, the enhanced multi-scale feature map is generated through dimensionality reduction using depthwise separable convolution. Step 4: Map the enhanced multi-scale feature map into a heterogeneous feature relationship graph, where nodes represent multi-scale defect attributes, including edge gradient, texture density, and deformation curvature. Initialize edge connection rules based on the graph attention network and the Node2Vec embedding algorithm, model the physical mapping of exposed foil spots and overall warpage through a self-supervised message passing mechanism, output dynamic weight features to the improved XGBoost classifier, and generate defect types and defect levels based on prior parameters. Step 5: Use edge computing nodes to collect production line process parameters and historical defect databases in real time, iterate the policy function online through the proximal policy optimization model, dynamically adjust the convolution kernel size and classification threshold, and use the memristor random computing method and random resonance coding to suppress transmission noise. The optimized parameters are transmitted back to steps 2 and 4 through probabilistic pulse fault tolerance.

[0014] In the above-mentioned implemented dynamic compensation module, the Retinex multi-scale light compensation algorithm adopts three-scale Gaussian convolution kernel (σ=15 / 80 / 200) to perform frequency domain decomposition on the original RGB image: the large-scale kernel (σ=200) extracts the low-frequency light component to represent the global light distribution, the medium-scale kernel (σ=80) captures the medium-frequency reflection component corresponding to the material surface texture, and the small-scale kernel (σ=15) separates the high-frequency noise and micro-defect features. Linear decoupling of light and reflection components is achieved through multi-channel logarithmic domain transformation, wherein the estimation of the light component adopts constrained least squares optimization, and the edge preservation constraint term is introduced during the reconstruction of the reflection component to suppress the halo artifact. For the high-reflective interference of the metal foil, a reflection area detection model based on Gaussian difference (DoG) filtering is designed: first, a normalized gradient amplitude map is generated, and the high-light edge profile is extracted by a double-threshold hysteresis method, and then a morphological closing operation is used to fill the area to generate a binary mask matrix. In the reflectivity correction stage, an adaptive attenuation function is applied to the mask covered area, and its weight coefficient is dynamically adjusted by the local contrast to achieve nonlinear highlight suppression. In the joint denoising stage, a hybrid domain filtering strategy is adopted: in the spatial domain, the weighted mean value of the pixel neighborhood is calculated based on the bilateral filter kernel (σ_d=2, σ_r=0.1), and the sub-pixel level edge is preserved through color similarity constraint; in the frequency domain, a non-local mean model is constructed, and the SSIM structural similarity of 7×7 similar blocks in a 15×15 search window is calculated, and the weighted average is calculated using an exponential decay function to eliminate salt and pepper noise. Finally, the hardware-level synchronization of the dual-channel pixel stream is realized through the AXI-Stream protocol, and the data format conversion and alignment are completed using the heterogeneous computing architecture in the FPGA, wherein the RGB channel adopts YUV420 to RGB888 color space conversion, and the near-infrared channel realizes dynamic range compression through 16-bit fixed-point number mapping.

[0015] In implementation, this module utilizes a Basler raL2048-48gm line scan camera and a multispectral ring light source (wavelengths of 450nm / 850nm / 1050nm) as the optical acquisition unit. The camera connects to a Xilinx Zynq UltraScale+ MPSoC FPGA via a CoaXPress 2.0 interface, achieving a transmission rate of 6.25Gbps. The light source control module integrates a PWM dimming circuit, dynamically adjusting the flash frequency (50-200Hz) based on production line speed. A parallel pipeline processing architecture is implemented within the FPGA. First, the raw image data (4096×512×12 bits) is cached using a DDR4 controller. The Retinex algorithm is implemented in the programmable logic unit (PLU). Three-scale Gaussian convolution uses a separable filter structure (horizontal / vertical convolution kernels), and a Systolic array is used to accelerate matrix operations. The reflection detection module utilizes a dual-port block RAM to store gradient magnitude maps, and morphological operations utilize a pipeline architecture with 3×3 structuring elements. In the joint denoising module, bilateral filtering pre-calculates spatial-color weights using a lookup table (LUT), and the non-local means algorithm uses a sliding window buffer to reduce repeated data reading. The AXI-Stream interface is configured with a 4-channel, 128-bit bit width, and a crossbar switch matrix is ​​used to achieve time-division multiplexing of RGB and near-infrared data. At the software level, Retinex parameters are optimized using Vivado HLS: Gaussian kernel coefficients use the Q8.8 fixed-point format, and the logarithmic transformation is approximated using the CORDIC algorithm; the reflective attenuation function is set to a piecewise linear model, triggering exponential decay when the local contrast exceeds 30%. During the production line debugging phase, an offline calibration method is used to determine the optimal parameter combination: a standard grayscale plate is used to calibrate illumination uniformity, and a defect sample library (including defects ranging from 0.05-2mm) is used to optimize bilateral filtering parameters. The final configuration is fixed in the FPGA bitstream file.

[0016] In implementation, the traditional metal surface detection system is limited to a single visible light imaging mode, and it is difficult to overcome the double interference of mirror reflection and coating shielding. This module breaks through the single mode data perception bottleneck of internal defects by visible light-near infrared dual-band collaborative imaging combined with Retinex multi-scale decomposition physical reflection model to establish a cross-band correction mechanism of metal foil surface reflection characteristics. At the algorithm level, the existing light compensation methods mostly use fixed threshold highlight suppression strategy, which cannot adapt to the dynamic changes of metal foil reflectivity under different process conditions. This module realizes the nonlinear precise regulation of the reflection component through the gradient-driven adaptive attenuation function, solving the edge blur problem caused by traditional linear compression. In terms of noise suppression, traditional methods often use spatial or frequency domain filtering in isolation. This module innovatively combines the local edge-preserving characteristics of bilateral filtering with the global similarity measurement of non-local mean to construct a hybrid denoising model, which effectively suppresses the composite noise while preserving the sub-pixel level crack edge. At the hardware architecture level, it breaks through the speed bottleneck of traditional CPU serial processing, and realizes the real-time alignment and preprocessing of dual-mode data stream through the deep optimization of FPGA pipeline and parallel computing unit, providing high signal-to-noise ratio input guarantee for the subsequent feature extraction module.

[0017] In the multi-scale perception module implemented above, the local gradient entropy calculation quantizes the gradient magnitude distribution within a pixel neighborhood (5×5 window) based on Shannon's information entropy theory. The gradient magnitude is then calculated using the Sobel operator to generate a gradient magnitude probability density function. When the entropy exceeds a threshold (H ≥ 1.2), the geometric deformation mechanism of the deformable convolution kernel is triggered: a sampling point offset vector is calculated based on the gradient direction field. The gradient direction is obtained using a discrete differential operator (Scharr filter), and the offset magnitude is dynamically adjusted by the normalized entropy value, achieving dense sampling of high-frequency defect areas. In the hierarchical design of the dilation rate, the Conv2 layer utilizes a dual-branch parallel convolution architecture (r = 1 / r = 6). The r = 1 branch extracts submillimeter crack edge features using a regular sampling interval, while the r = 6 branch captures local deformation caused by slurry diffusion by increasing the sampling interval. The dual-branch structure (r = 6 / r = 12) of the Conv3 layer establishes a transition mapping from mesoscopic to macroscopic features, with the r = 12 branch using sparse sampling to cover the global structure of 5mm-scale defects. The receptive field expansion formula shows that when using a k=3 base kernel, the effective receptive range of an r=12 convolution kernel can reach 25×25 pixels, breaking the scale limitation of traditional convolution kernels. The channel attention mechanism dynamically weights feature channels using a squeeze-excitation network (SENet). A global average pooling layer generates channel statistical descriptors, and a two-layer fully connected network learns the nonlinear relationship between channels. The sigmoid activation function outputs normalized weights, enhancing the feature correlation between high-frequency edges and global semantics. Cross-layer skip connections use a bilinear interpolation algorithm to align feature map resolutions. Depthwise separable convolutions (DSCs) reduce computational complexity while preserving the spatial accuracy of the 112×112 high-resolution features in the Conv2 layer.

[0018] In implementation, the gradient entropy calculation unit utilizes a parallel pipeline design: Sobel gradient calculation utilizes a dual-path 5×5 convolution kernel pipeline, the gradient amplitude statistics module integrates a 32-bit accumulator array, and an entropy threshold comparator triggers the deformation control signal in real time. The dilated convolution kernel group is implemented using a parameterized IP core, where the r=1 branch configures a standard convolution unit, and the r=6 / r=12 branches use an interval sampling address generator. Intermediate feature maps are cached in BRAM. The channel attention mechanism is implemented on an ARM Cortex-A53 processor, leveraging the NEON instruction set to accelerate matrix operations. Weight coefficients are written back to the FPGA feature fusion unit via the AXI bus.

[0019] In actual deployment, the gradient entropy threshold is set to 1.2 (empirical optimization value), and the inflation rate configuration follows the progressive expansion strategy of Conv1(r=1)→Conv2(r=1 / 6)→Conv3(r=6 / 12)→Conv4(r=12). During the production line debugging stage, the gradient direction field visualization tool is used to calibrate the accuracy of the offset vector calculation, and the defect sample library (containing 0.1-5mm defects) is used to optimize the initial value of the channel attention weight. During system operation, the Conv2 layer feature map is mapped to the DDR4 memory pool through the HP port, the cross-level jump connection uses the AXI-Stream protocol to realize low-latency data transmission, and the 3x3 depth convolution kernel of the depth separable convolution uses the Winograd algorithm for acceleration.

[0020] The module first proposes a gradient entropy driven convolution kernel deformation algorithm, combines information entropy theory with deformable convolution, realizes adaptive adjustment of sampling density through local complexity quantification, and significantly improves the capture ability of sub-millimeter micro cracks; secondly, a hierarchical inflation rate feature pyramid is constructed, a multi-branch parallel convolution structure is embedded in the shallow network, and a continuous mapping relationship between micro-macro features is established through a progressive receptive field expansion strategy to solve the feature breakage problem caused by the traditional multi-scale method; finally, a cross-level feature reservation architecture is designed, which reduces the dimensionality of high-resolution detail features through depth separable convolution and fuses them with high-level semantic features, reducing the computational complexity while maintaining spatial positioning accuracy.

[0021] When designing the scheme, in order to verify the effectiveness of the multi-scale perception module (Group A), the traditional fixed convolution kernel ResNet-50 (Group B) is selected for comparison. The experimental platform uses the same hardware configuration (Basler camera + FPGA + industrial computer), and the test sample is a standard library of resistors containing 0.05~5mm defects (1000 pictures). Group A deploys deformable convolution kernel and multi-inflation rate hierarchical design, and Group B uses standard ResNet-50 with fixed 3x3 convolution kernel + max pooling. The experiment simulates the process fluctuation (slurry viscosity ±15%) and light fluctuation (500~2000Lux), each group conducts 5 repeated experiments, and the experimental data are recorded in Table 1: Table 1 Multi-scale perception module experimental record table Experimental group parameter Experiment 1 Experiment 2 Experiment 3 Experiment 4 Experiment 5 Group A Detection rate of small cracks (%) 100 100 100 100 100 Large size defect detection rate (%) 98.6 99.2 98.9 99.5 99.1 Feature extraction latency (ms) 3.2 3.1 3.3 3.0 3.2 Light interference false detection rate (%) 1.8 2.1 1.5 1.9 2.0 Cross-batch recognition rate (%) 96.4 95.8 97.2 96.1 96.9 Group B Detection rate of small cracks (%) 72 78 85 80 75 Large size defect detection rate (%) 92.3 93.7 91.5 94.0 93.2 Feature extraction latency (ms) 3.0 3.1 2.9 3.0 3.1 Light interference false detection rate (%) 12.6 15.3 10.9 14.7 13.8 Cross-batch recognition rate (%) 82.1 79.4 84.6 81.3 83.5 As shown in Table 1, the multi-scale perception module (Group A) achieves 100% stable detection in micro crack detection, which is more than 20% higher than the traditional method (Group B), and the false detection rate under light interference is reduced to below 2%, verifying the effectiveness of the gradient entropy driven deformation and inflation rate hierarchical design. The cross-batch recognition rate is improved by about 14%, indicating that the module has strong generalization ability to process fluctuations. Although the feature extraction delay is slightly higher by 0.2ms, the overall performance is significantly better than the fixed convolution kernel scheme, meeting the real-time requirements of high-speed production lines (100m / min).

[0022] In the cross-layer fusion module implemented above, a bidirectional cross-layer attention network is based on spatial-channel collaborative attention and multi-scale feature reconstruction theory. Between the Conv2 layer (112×112 resolution) and the Conv4 layer (14×14 resolution), bilinear interpolation is used to spatially align Conv2 layer features. A bicubic interpolation kernel is then used to weighted interpolate local neighborhood pixels to generate a bottom-level feature map that matches the resolution of the Conv4 layer, preserving subpixel edge details such as microcracks and oxidation spots. This process optimizes the weight distribution of the subpixel convolution kernel to avoid edge jaggedness caused by traditional nearest neighbor interpolation.

[0023] Then, a cross-layer relationship matrix is ​​constructed. The query vector (Q) is projected from the Conv4 layer features to a 512-dimensional semantic space through 1×1 convolution. The key vector (K) and value vector (V) are extracted from the interpolated Conv2 layer features through grouped convolution (Group=8). The grouping strategy reduces the computational complexity by reducing the parameter sharing between channels. In the spatial dimension, the cosine similarity matrix of Q and K is calculated by scaling the dot product attention and introducing the temperature coefficient ( ) adjusts the attention distribution, using a Softmax function for normalization to generate a spatial attention weight map, which dynamically represents the strength of the association between Conv2 layer pixels and Conv4 layer semantic categories. In the channel dimension, a squeeze-excitation network (SENet) is used to generate channel attention weights: global average pooling compresses spatial information to generate channel descriptors. A two-layer fully connected network learns the nonlinear relationship between channels. The sigmoid function outputs normalized weights to enhance the response strength of defect-sensitive channels. The spatial and channel attention outputs are fused via a Hadamard product, and then dimensionality reduction is achieved via depthwise separable convolution. Depthwise convolution (3×3 kernel) extracts local spatial patterns, while pointwise convolution (1×1 kernel) enables cross-channel information interaction, generating an enhanced multi-scale feature map. A residual learning mechanism adds the fused features element-wise to the original Conv4 layer features, preserving the underlying semantic information through identity mapping and mitigating the risk of vanishing gradients.

[0024] The multi-head self-attention model consists of four core stages: cross-modal projection, multi-head decomposition, dynamic aggregation, and regularized output. The cross-modal projection layer employs a heterogeneous feature alignment strategy, upsampling the high-resolution features (112×112×256) from the Conv2 layer to a resolution of 14×14 using bilinear interpolation and spatially aligning them with the semantic features (14×14×512) from the Conv4 layer. Grouped convolution (Group=8) is then used to independently linearly project the bimodal features to generate a query vector Q (14×14×512), a key vector K (14×14×512), and a value vector V (14×14×512). The projection matrix is ​​initialized using Xavier to prevent gradient vanishing. The spatial relationship modeling layer computes the similarity matrix between Q and K using scaled dot-product attention. The cosine similarity scores are scaled by a temperature coefficient (τ = √d_k) and normalized to a probability distribution using the Softmax function. This generates a spatial attention weight map (14×14×14×14), which dynamically represents the strength of associations between different spatial locations. The multi-head decomposition layer partitions the weight matrix into eight independent attention heads (Head=8) along the channel dimension. Each head is mapped to a low-dimensional subspace via a fully connected layer (FC-64). This extracts spatial correlation patterns within a local neighborhood (7×7 window) and generates a multi-head attention feature set (8×14×14×64). The dynamic weight fusion layer uses a gated recurrent unit (GRU) to adaptively aggregate multi-head features. The GRU's update gate controls the ratio of historical features to the current input, while the reset gate filters the valid information flow. The contribution weights of each attention head are iteratively optimized through time steps, outputting a global attention feature (14×14×512). The residual connection layer adds the attention features element-by-element to the original Conv4 layer features, utilizing identity mapping to preserve the initial semantic information and prevent network degradation. The regularized output layer suppresses feature distribution shifts through layer normalization (LayerNorm) and employs depthwise separable convolution (DSEC) for dimensionality reduction: depthwise convolution (3×3 kernel) extracts local spatial correlations, and pointwise convolution (1×1 kernel) compresses the channel dimensions to the target value (14×14×256), ultimately outputting an enhanced multi-scale feature map. This model utilizes multi-head parallelization and a dynamic gating mechanism to achieve fine-grained interaction of features across layers, significantly improving the characterization of complex defects (such as crack-oxidation coexistence), providing theoretical support for multi-scale detection of resistor defects. In the resistor defect detection scenario, the synergistic effect of a bidirectional cross-layer attention network and a multi-head self-attention model is reflected in the deep fusion of multi-scale defect features. The high-resolution features in the Conv2 layer correspond to subpixel gradient information of microcracks and scratches, while the semantic features in the Conv4 layer represent macrostructural anomalies such as global deformation and warpage. Through the spatial attention mechanism, the model accurately correlates regions of local gradient abruptness with global deformation trends, dynamically suppressing illumination interference in non-defective areas. Channel-wise attention weights quantify the contributions of different feature channels (such as edges, textures, and deformations) to enhance the representation of defect-sensitive features. In the multi-head self-attention model, the cross-modal projection layer achieves spatial alignment of high- and low-resolution features, while the dynamic weight fusion layer uses GRU gating to establish a physical correlation model for cross-scale features, enabling accurate analysis of complex defects (such as cracks with slurry overflow). This technical system combines local detail preservation with deep global semantic reasoning, providing a highly accurate and robust analytical framework for online detection and classification of resistor defects.

[0025] In hardware implementation, the cross-layer fusion module is based on heterogeneous computing architecture, with Xilinx Alveo U280 accelerator card working with Intel Xeon Gold 6248R processor through PCIe Gen4x16 interface for high-speed data intercommunication. The FPGA end is responsible for high-throughput feature interpolation and initial attention calculation, with double-channel HBM2 memory (capacity 8GB, bandwidth 460GB / s) used for caching Conv2 layer 112x112 high-resolution feature maps and Conv4 layer 14x14 semantic features. The bilinear interpolation module is implemented through FPGA logic unit, with 2-order polynomial interpolation kernel function configured, and the interpolation weight matrix is preloaded to BlockRAM, with the interpolation accuracy error controlled within ±0.3 pixels, and the output is the aligned 14x14 bottom-level detail features. The multi-head self-attention model is accelerated on the CPU end through the OpenVINO inference engine, the query vector Q is generated from the Conv4 layer features through 1x1 convolution, and the convolution kernel weight dimension is 512x512, which is initialized with Xavier uniform distribution; the key vector K and the value vector V are extracted from the interpolated Conv2 features through grouped convolution, with 8 groups set, 64 convolution filters in each group, and the total number of output channels being 512. The spatial relationship modeling layer optimizes the matrix multiplication operation through the MKL-DNN library, and the cosine similarity matrix calculation is parallelized through the SIMD instruction set, with the spatial attention weight map dimension being 14x14x14x14, which is compressed and stored in CSR sparse format, with the compression rate improved to 50%. The hidden layer dimension of the gate recurrent unit (GRU) of the dynamic weight fusion layer is configured as 256, the gate weight is initialized through orthogonalization, and the LAMB optimizer (learning rate 2e-4, β1=0.9, β2=0.999) is used in the training stage, with the batch size set to 64 in the mixed precision training mode. The 3x3 depth convolution kernel of the depth separable convolution adopts a symmetric padding strategy, with 1 pixel padded on the edge to maintain the feature map size, and the point-by-point convolution layer is compressed to 256 dimensions, with the parameter quantity reduced to 12% of the standard convolution. The residual connection is realized through the FPGA end dedicated data path for element-by-element addition of feature maps, and the layer normalization module uses FP16 half-precision to calculate the mean and variance, reducing the data transmission delay. In terms of hardware resource allocation, the FPGA logic unit occupancy rate is 58% (LUT: 315K / 542K), the HBM2 memory bandwidth utilization rate is stable at 82%-85%, and the CPU end memory bandwidth occupancy peak reaches 48GB / s. In software parameter configuration, the number of multi-head attention heads is set to 8, the dimension of each head is 64, the compression ratio of channel attention squeeze-excitation network is set to 16, and the activation function uses GELU instead of ReLU to enhance the non-linear expression ability. The training data set contains 3 million resistance sheet defect images, covering 6 types of defects such as scratches, oxidation, and cracks, and the data enhancement strategy uses random rotation (±15°), brightness jitter (±20%), and Gaussian noise injection (σ=0.1).The model training cycle is 80 epochs, and the early stopping mechanism is triggered when the validation set loss does not decrease for five consecutive times. After the final model accuracy converges, it is deployed to the edge computing node in ONNX format.

[0026] Traditional methods rely on top-down, unidirectional feature propagation. High-resolution detail features are gradually distorted due to interpolation errors and noise accumulation during multiple upsampling processes. However, the bidirectional attention mechanism achieves a complementary enhancement of low-level details and high-level semantics by dynamically allocating weights across spatial and channel dimensions. Spatial attention quantifies the spatial correlation strength of features across layers using cosine similarity, enabling the convolution kernel to focus on the microstructure of defect edges (such as crack bifurcation points) while suppressing background texture interference. Channel attention, on the other hand, uses a squeeze-excitation network to recalibrate the importance of feature channels, enhancing the saliency of mid-frequency features such as oxidation spots. The introduction of depthwise separable convolution decouples spatial filtering from channel transformation, significantly reducing computational complexity while maintaining the integrity of edge structures. This allows the multi-scale feature fusion process to adapt to the real-time constraints of edge devices. The residual learning mechanism preserves the original feature distribution through identity mapping, avoiding the gradient annihilation problem in deep network training. Layer normalization improves model convergence stability by standardizing the feature distribution. This design not only solves the problem of detail loss caused by fixed weight fusion in traditional methods, but also improves the characterization capability of composite defects through adaptive enhancement of channel dimensions, providing a high-precision, low-latency feature fusion solution for industrial visual inspection systems, and significantly improving the detection rate and classification robustness of multi-scale defects on the surface of resistor sheets.

[0027] To verify the above results, a comparative experiment was conducted. Group A used a cross-layer fusion module (bidirectional cross-layer attention network + depthwise separable convolution), while Group B used a traditional feature pyramid network (FPN) fusion method. The experiment included a resistor sample library (containing defects such as scratches, oxidation spots, and cracks) and a simulated vibration environment (10-50Hz random perturbation) on the production line. The experimental data is shown in Table 2: Table 2 Cross-layer fusion module application experimental data record table Experimental group Number of experiments Detection accuracy (IoU) Fusion time (ms) Minor defect detection rate (%) False positive rate (%) Positioning error under vibration interference (μm) Group A 1 0.92 12.3 98.5 1.2 18.7 Group A 2 0.91 13.1 97.8 1.5 20.1 Group A 3 0.93 11.9 99.0 0.9 17.3 Group A 4 0.94 12.5 98.2 1.1 19.5 Group A 5 0.92 13.0 98.7 1.3 18.9 Group B 1 0.83 38.6 89.2 6.7 52.4 Group B 2 0.85 37.9 90.1 5.9 49.8 Group B 3 0.84 39.2 88.7 7.3 54.1 Group B 4 0.82 40.1 87.5 8.1 57.3 Group B 5 0.86 38.5 89.8 6.5 50.6 Experiments show that the cross-layer fusion module is significantly superior to traditional methods in terms of detection accuracy, real-time performance and adaptability, and can meet the complex working conditions of high-speed production lines.

[0028] In the graph network classification module implemented above, the prior parameters include geometric feature parameters, physical property parameters and process parameters; the geometric feature parameters include defect area threshold, shape factor and edge gradient consistency; the defect area threshold includes micro defects (0.05~0.2mm²), medium defects (0.2~1mm²), and macro defects (>1mm²); the shape factor includes ellipticity (>0.6 is judged as a regular defect) and fractal dimension (1.2~1.8 represents crack complexity); edge gradient consistency is calculated based on the Sobel operator to calculate the standard deviation of the edge gradient direction (<15° is judged as a continuous crack). The physical property parameters include thermal conductivity deviation and resistivity change rate; thermal conductivity deviation is the difference between the local thermal conductivity extracted by near-infrared imaging and the standard value (>10% is determined as internal delamination); the resistivity change rate is combined with the measured data of the four-probe method, and local resistivity fluctuations >5% are marked as conductive abnormal defects; the process parameters include slurry diffusion coefficient and stacking offset. The slurry diffusion coefficient calculates the diffusion radius based on the printing speed and slurry viscosity (exceeding ±0.1mm is determined as a process defect); the stacking offset is calculated by 3D morphology reconstruction to calculate the cumulative error of multi-layer printing offset (>0.05mm triggers a graded warning); The defect types include surface defects and structural defects; surface defects include: Microcracks (Type-S1): width <0.02mm, length 0.1~0.5mm, edge gradient consistency >0.8; Oxidation spots (Type-S2): circular areas (ovality > 0.9), with near-infrared reflectivity 20% lower than the baseline value; Slurry overflow (Type-S3): Diffusion coefficient out of tolerance + edge fractal dimension > 1.5; Structural defects include: Local delamination (Type-C1): thermal conductivity deviation >15% and resistivity change >8%; Edge warping (Type-C2): 3D topography curvature radius <5mm, stacking offset >0.08mm; Internal bubbles (Type-C3): Ultrasonic echo signal attenuation rate >30%, X-ray transmission grayscale variance >25% The defect level is 4, including: (1) Grade A (minor defects): only surface micro-defects (Type-S1 / S2), area <0.2mm², resistivity change rate <3%, online repair is allowed; thermal conductivity deviation <5%, no process adjustment is triggered. (2) Grade B (acceptable defects): surface defects (Type-S3) or minor structural defects (Type-C1 / C2), area 0.2~1mm²; resistivity change rate 3%~8%, need to be downgraded and process parameter fluctuations recorded. (3) Grade C (serious defects): composite defects (such as S3+C1), area >1mm², thermal conductivity deviation >10%; stacking offset >0.1mm or internal bubble density >5 / cm², forced offline repair. (4) Grade D (fatal defect): Structural failure (e.g. Type-C3 bubble diameter > 0.5mm), resistivity change rate > 15%; thermal conductivity deviation > 20% or edge warping resulting in mechanical strength reduction > 40%, directly scrapped.

[0029] In implementation, during the node initialization phase, the graph network classification module uses the Node2Vec embedding algorithm to learn representations of the input multi-scale feature tensor (edge ​​gradient, texture density, and deformation curvature). A second-order random walk strategy is used to generate node sequences, with a walk path length of 10 to capture local neighborhood structure. A Skip-gram model with a window size of 5 uses negative sampling to optimize the objective function, mapping the nodes into a 128-dimensional latent space and generating the initial feature vector. This process preserves the topological relationships of defect features by maximizing node co-occurrence probability, providing both geometric and semantic constraints for subsequent graph construction. During the dynamic graph topology construction phase, potential edge connections are screened based on a cosine similarity threshold (>0.7). Dynamic edge weights are calculated using the multi-head attention mechanism (head=4) of the Graph Attention Network (GAT). The query and key vectors are generated through linear projection of the node embeddings. The LeakyReLU activation function (negative slope=0.2) introduces nonlinear transformation capabilities. The attention coefficient is calculated by matrix multiplication of the transposed matrix of the learnable parameter vector α with the concatenated node features. Weights are normalized to generate the edge weight matrix. This mechanism overcomes the limitations of traditional fixed similarity thresholds and achieves adaptive quantification of the strength of defect feature associations. During the self-supervised message passing phase, a three-layer iterative update strategy based on a gated recurrent unit (GRU) is designed: the message function projects neighborhood node features into the latent space via the message transformation matrix W_msg. The aggregation function uses mean pooling to fuse neighborhood weighted information. The GRU's update gate and reset gate dynamically control the fusion ratio of historical states to current inputs, suppressing interference from noisy nodes. Inter-layer residual connections maintain gradient flow stability through identity mapping, preventing deep network degradation. The online hard example mining strategy uses a confidence threshold (0.85) to filter classification uncertainty samples, triggering Node2Vec's incremental learning module: the embedding vector of the hard example node sequence is fine-tuned based on the Skip-gram model, and the graph topology is dynamically updated to adapt to production line process fluctuations.

[0030] The improved XGBoost multi-label classifier incorporates four key components: multi-label tree structure expansion, loss function reconstruction, feature fusion optimization, and online adaptive calibration. The multi-label tree structure expansion algorithm upgrades the traditional single-objective decision tree into a set of parallel subtrees, each corresponding to a specific defect type (such as scratches, oxidation, and delamination). This algorithm reduces redundant computation by sharing feature splitting points. A dynamic feature weighting mechanism concatenates the node weight vectors output by the graph network with the original features, employing sparse matrix compression (CSR format) to mitigate the curse of dimensionality. A feature grouping embedding strategy uses attention weights to enhance key feature channels. During objective function reconstruction, a Focal Loss function is introduced to address class imbalance: a class balancing factor γ = 2 is used to reduce the gradient weight of high-frequency defects (such as microcracks) while increasing the loss contribution of difficult-to-classify samples (such as complex defects). The adjustment factor (1-p)^γ in the formula dynamically suppresses the gradient update amplitude of easily classified samples. The tree splitting point selection module uses a proximal gradient descent algorithm to optimize gain calculation. L1 regularization is used to constrain the sparsity of split features. If a feature weight falls below a threshold of θ = 0.05, a pruning mechanism is triggered, skipping low-contribution features to improve model efficiency. The online incremental learning interface uses a delta update strategy to dynamically adjust model parameters. When the reinforcement learning module reports process parameter fluctuations, the classifier loads an incremental dataset (containing only new defect samples) for local fine-tuning, avoiding the computational overhead of global retraining. The Shapley value interpreter integrates game theory principles to dynamically analyze feature contributions. The contribution of each feature to the classification result is calculated through marginal effects. When the contribution of a key feature (such as the resistivity indicator of a local delamination) deviates by 20% from the baseline, the online calibration module is triggered to compensate and correct the corresponding tree node weights. This technology system significantly improves the classification accuracy of complex defect patterns through dynamic weight allocation and multi-objective collaborative optimization, while maintaining the real-time performance requirements of the production line.

[0031] In the collaborative framework of the graph network classification module and the improved XGBoost, the graph network transforms multi-scale defect features into structured semantic representations through heterogeneous relationship modeling, while XGBoost implements multi-label hierarchical decision-making based on dynamic weight features. The topological relationships captured by the Node2Vec embedding algorithm provide XGBoost with prior knowledge constraints, reducing the tree model's reliance on redundant features. The difficult examples fed back by XGBoost drive the incremental optimization of the graph network, forming a bidirectional knowledge transfer mechanism. At the hardware deployment level, the message passing iterations of the graph network and the parallel tree growth of XGBoost are implemented through a heterogeneous computing architecture: graph computing tasks are allocated to GPU acceleration, and XGBoost's tree splitting optimization is executed by CPU multithreading, with data pipeline synchronization achieved through RDMA high-speed interconnection. This technical system deeply combines the associative reasoning capabilities of the graph model with the efficient decision-making characteristics of the tree model, providing a full-process intelligent solution for resistor chip defect detection, from feature characterization to hierarchical decision-making, significantly improving the system's detection accuracy and adaptability in complex industrial environments.

[0032] During implementation, the graph network classification module performs real-time graph structure construction and message passing computations on the FPGA. Built-in dual-channel DDR4 memory (32GB, 68.3GB / s bandwidth) is used to cache 128-dimensional node embedding vectors and feature weight matrices. The Node2Vec embedding initialization phase is implemented on the FPGA. A random walk controller is constructed using a hardware description language, with a walk length parameter of 10 and a window size of 5. A dual-port block RAM is used to store the node adjacency list, keeping the latency of a single walk sequence generation to within 2.8ms. The Graph Attention Network (GAT) is deployed using the TensorRT acceleration engine. The multi-head attention layer is configured with four independent attention heads, each with 32 dimensions. The LeakyReLU activation function has a negative slope parameter of α=0.2. Dynamic edge weight calculation uses CUDA kernels for matrix concatenation and nonlinear transformations. Cosine similarity calculations of 128-dimensional node vectors are processed in parallel, resulting in a single-layer GAT inference latency of less than 6.5ms. The GRU units in the message passing mechanism implement gate logic on the FPGA using LUT resources. The hidden layer dimension is set to 256, and the message transformation matrix W_m is 256×256. FP16 half-precision quantization is used for storage compression. An improved XGBoost classifier is deployed on a CPU cluster (Intel Xeon Platinum 8380), receiving dynamic weight feature vectors via the RDMA protocol. The multi-label tree structure is configured with 100 parallel subtrees (corresponding to three defect types: scratches, oxidation, and delamination), with a maximum depth of 12 per tree, a learning rate of η=0.15, and an L2 regularization term of λ=1.2. The online hard example mining module uses a confidence threshold of 0.85. When triggering incremental training of the Skip-Gram model, an asynchronous gradient descent algorithm is used to update the 128-dimensional embedding vector. The batch size is set to 256, and the momentum coefficient β=0.9. A Shapley value interpreter is integrated at the end of the inference pipeline. When feature contributions deviate by 20% from the baseline, an online calibration module is activated. The Newton iteration method is used to correct the tree node splitting gain weights, with a maximum iteration count of 50. In terms of hardware resource utilization, the FPGA logic unit utilization rate is 72% (LUTs: 398K / 552K), the Jetson platform GPU load is stable at 83%-87%, and the CPU cluster memory bandwidth reaches a peak of 42GB / s.

[0033] Traditional graph convolutional networks rely on fixed adjacency matrix to model feature correlation, while the proposed scheme realizes the adaptive construction of defect feature topological relationship through the Node2Vec embedded random walk strategy combined with the dynamic attention mechanism of graph attention network. The deep random walk strategy of Node2Vec captures the local microstructure (such as crack bifurcation) and macro deformation correlation by adjusting the balance parameters of BFS (breadth-first) and DFS (depth-first), while the multi-head attention layer of graph attention network strengthens the coupling strength of high-frequency gradient features and low-frequency semantic features through LeakyReLU activated cross-node interaction. At the message passing level, the three-layer iterative GRU gating mechanism integrates residual connection to alleviate the gradient decay problem of deep network, and suppresses the information interference of noise nodes through the gating filtering mechanism to improve the signal-to-noise ratio of feature propagation. The essence of the improved XGBoost lies in the joint optimization of multi-label tree structure and dynamic feature weighting: the multi-label tree decouples the classification boundary of complex defects through the parallel sub-tree architecture, and the class balance factor dynamically adjusts the loss weight of low-frequency defects (such as delamination) based on the gradient reconstruction objective function, solving the sensitivity of traditional methods to sample imbalance. The Shapley value interpreter quantifies the marginal effect of graph node weight on the classification result through reverse gradient propagation, establishes an explainable model, and realizes real-time diagnosis of abnormal fluctuations of process parameters (such as resistivity). This graph-tree collaborative heterogeneous model architecture not only retains the expression ability of graph neural networks for non-Euclidean space features, but also leverages the classification robustness of XGBoost in high-dimensional sparse data, solving the dynamic coupling detection problem of multi-scale defects in resistive sheets.

[0034] The core optimization of the improved XGBoost multi-label classifier lies in the reconstruction of the classification boundary modeling method through the multi-label tree structure expansion algorithm, which independently constructs parallel sub-trees for defects such as scratches, oxidation, and delamination, eliminating the feature coupling interference of traditional single-tree structure in multi-label correlation scenarios; a dynamic feature weighting mechanism is introduced to generate a sparse feature compression matrix based on the dynamic weight vector output by the graph neural network, and the class balance factor is dynamically adjusted through the Focal Loss reconstruction objective function to reduce the gradient weight of high-frequency defects (such as scratches) and enhance the sensitivity to low-frequency complex defects (such as oxidation accompanied by cracks); an online calibration module driven by Shapley value is integrated to analyze the deviation degree of feature contribution and process parameters in real time, triggering the Newton iteration method to correct the tree node weight, and improving the adaptive ability of the model to dynamic process conditions such as coating thickness fluctuation and sintering temperature shift. These improvements enable the classifier to maintain the advantages of high-dimensional sparse data processing while significantly enhancing the robustness of multi-label unbalanced data, and through hardware acceleration, the computational efficiency and explainability are synergistically improved.

[0035] To validate the improved XGBoost multi-label classifier, comparative experiments were conducted using the NVIDIA Jetson AGX Orin and Xilinx Zynq UltraScale+ FPGA hardware platforms. Group A deployed the improved XGBoost multi-label classifier, while Group B used standard XGBoost combined with the MultiOutputRegressor package for multi-label classification. Both experiments were conducted on the same resistor defect dataset (containing three categories: scratches, oxidation, and delamination, with a sample ratio of 10:3:1). Group A used a dynamic feature weighting mechanism (FL loss function parameters γ=2 and α=0.25), while Group B used a standard cross-entropy loss function. The tree depth was uniformly set to 12 layers, the learning rate was 0.1, and training was repeated 500 times. Evaluation metrics included multi-label mean average precision (mAP), single-shot training time, feature importance accuracy (Pearson correlation based on Shapley values), class-balanced F1 difference (the absolute difference between the highest and lowest F1 values), and parameter calibration response time. During the experiment, Group A implemented dynamic weight feature selection using the FPGA-side sparse matrix compression module, while Group B relied on the CPU to perform full feature calculations. Regarding hardware resource allocation, Group A utilized the FPGA to accelerate feature compression and weight updates in parallel, while Group B relied on the CPU for serial processing throughout the entire process. The experimental records are shown in Table 3: Table 3 Comparison data record table of improved XGBoost multi-label classifier Experimental group frequency mAP (%) Training time (ms) Feature Importance Accuracy (Pearson) Class-balanced F1 difference (Δ) Parameter calibration response (ms) Group A 1 92.3 1450 0.87 0.12 38 2 91.8 1432 0.85 0.15 41 3 93.1 1467 0.89 0.11 37 4 92.5 1429 0.86 0.13 40 5 92.7 1445 0.88 0.14 35 Group B 1 84.6 1689 0.62 0.34 112 2 85.1 1675 0.65 0.32 108 3 83.9 1698 0.59 0.37 115 4 84.2 1703 0.61 0.35 110 5 85.4 1681 0.63 0.33 106 Experimental results show that the improved XGBoost multi-label classifier achieves a multi-label mean average precision (mAP) of 92.48%, a 7.84% improvement over the standard XGBoost's 84.64%. The F1 score for low-frequency delamination defects improves significantly by 23.6%, validating the effectiveness of dynamic feature weighting and class balancing factors. Training time for Group A averaged 1444.6ms, a 14.5% reduction from Group B's 1689.2ms, demonstrating the computational advantages of FPGA sparse compression and hardware acceleration. Feature importance accuracy (Pearson coefficient 0.87 vs. 0.62) increased by 40.3%, demonstrating that the Shapley value interpreter can more accurately quantify process parameter contribution deviations. Parameter calibration response time decreased by 65.3% from 110.2ms in Group B to 38.2ms, demonstrating that the online calibration mechanism significantly improves the model's real-time adaptability to dynamic process fluctuations. The category-balanced F1 difference dropped from 0.34 to 0.13, indicating that the improved scheme effectively alleviated the classification bias of the traditional method in the multi-label imbalance scenario.

[0036] In the above-mentioned implemented parameter optimization module, the Actor network generates the action space (convolution kernel size, classification threshold, filtering intensity) through the policy function π(a|s), the Critic network evaluates the state value based on the value function V(s), and both of them calculate the advantage function A(s, a) through the generalized advantage estimation (GAE), and the policy update amplitude is constrained to avoid policy shock. The state observation vector is spliced by the process parameters of the production line (slurry viscosity, printing speed) and the defect statistical features (type distribution, confidence variance) output by XGBoost, and after normalization, it is input into the fully connected network (512 neurons). In view of the interference of transmission noise, the memristor random computing unit encodes the policy parameters into a probability pulse sequence: through the stochastic resonance effect, when the pulse amplitude is lower than the noise base, the signal characteristics are enhanced by using a nonlinear bistable system (Langevin equation), and the kinetic model is dx / dt = -dV(x) / dx + ξ(t), wherein V(x) is a potential well function, and ξ(t) is a Gaussian white noise. The optimized parameters are packaged through the probability pulse amplitude modulation (PPM) protocol, and are transmitted to the front-end module through the differential signal of the SPI interface. The online fine-tuning mechanism is based on the double-delay deep deterministic policy gradient (TD3) algorithm, and the Q value overestimation is suppressed through target network delay update and policy smoothing regularization, and the target Q value is calculated as wherein ε is a truncated normal distribution noise. The parameter version management adopts a circular queue to store historical policy snapshots, when the performance fluctuation of the classifier exceeds the limit, the optimal version is rolled back through cosine similarity search to ensure system stability.

[0037] In terms of hardware deployment, the parameter optimization module integrates an NVIDIA Jetson AGX Xavier edge computing node and a memristor array (crossbar architecture). This module connects to a Basler line scan camera via a CoaXPress 2.0 interface and an SPI interface (50 MHz) to an FPGA controller. The PPO model is deployed in a CUDA acceleration environment, and the actor-critic network is implemented in PyTorch. The batch size is set to 256, the GAE parameter λ = 0.95, and the discount factor γ = 0.99. The memristor cells are configured as a 128×128 crossbar array, and the pulse encoding uses a hybrid modulation of 4-bit amplitude and 2-bit timing. The noise floor is dynamically calibrated using an adaptive threshold detection circuit. Production line process parameters are acquired in real time from the PLC controller via the Modbus TCP protocol with a sampling frequency of 1 kHz. XGBoost defect data is serialized in the Apache Arrow format and fed into the state vector construction module. At the software level, the TD3 algorithm's policy network update interval is set to 2 training cycles, with a target network smoothing coefficient τ = 0.005. Stochastic resonance encoding parameters are determined through offline calibration: potential well depth V_0 = 0.5 eV, noise intensity D = 0.1. The parameter version queue capacity is set to 10, and a rollback trigger condition is three consecutive F1 score fluctuations with a standard deviation greater than 5%. In actual deployment, the slurry viscosity baseline is calculated using a sliding window of historical data (window size 1000 samples), and the fine-tuning trigger threshold is set to a 20% standard deviation offset. After receiving SPI data packets, the deformable convolution controller on the FPGA updates the convolution kernel offset lookup table (LUT) through a register map. The threshold adjuster uses a PID control algorithm to dynamically adjust the classification confidence boundary.

[0038] This module proposes an online optimization framework based on the PPO-TD3 hybrid strategy. This framework uses an Actor-Critic architecture to achieve real-time matching of detection parameters with process fluctuations, addressing the lag problem inherent in traditional static parameter tuning. Furthermore, a memristor stochastic resonance encoding scheme is designed to convert strategy parameters into noise-resistant pulse sequences, leveraging the noise enhancement characteristics of nonlinear systems to overcome the bit error rate limitations of traditional binary transmission. Finally, a parameter version management mechanism is developed that supports progressive optimization of production line processes while ensuring detection stability through rollback and incremental fine-tuning of historical strategy snapshots. These technological innovations enable the system to maintain high-precision detection in noisy, multi-disturbance industrial environments, while achieving full-link adaptation from parameter optimization to hardware control, providing theoretical methods and engineering practice support for quality control in complex manufacturing scenarios.

[0039] Although specific embodiments of the present invention have been described above, those skilled in the art will appreciate that these specific embodiments are merely illustrative, and that those skilled in the art may omit, substitute, and modify the details of the methods and systems described above without departing from the principles and spirit of the present invention. For example, combining the above method steps to perform substantially the same functions in substantially the same manner to achieve substantially the same results falls within the scope of the present invention. Accordingly, the scope of the present invention is limited solely by the appended claims.

Claims

1. A resistor sheet defect online detection system based on machine vision; characterized by: include: The dynamic compensation module is used to obtain a standardized image based on the RGB and near-infrared dual-modal data collected by the line scan camera and multispectral light source. It uses the Retinex multi-scale illumination compensation algorithm to eliminate reflections and suppress high-frequency noise through bilateral filtering and non-local means denoising. A multi-scale perception module is used to dynamically adjust the convolution kernel sampling offset based on the standardized image through the local gradient entropy driving mechanism of the deformable convolutional network, and use void spatial pyramid pooling to embed multi-expansion rate convolution kernels in Conv1 to Conv4 to generate multi-scale feature tensors of high-frequency edges, mid-order textures and global semantics, and output them to the cross-layer fusion module; The cross-layer fusion module uses a bidirectional cross-layer attention network to build a spatial and channel-coordinated attention mechanism for the Conv2 and Conv4 layers. It dynamically weights feature associations through a multi-head self-attention model and generates enhanced multi-scale feature maps through dimensionality reduction using depthwise separable convolution. A graph network classification module is used to map the enhanced multi-scale feature map into a heterogeneous feature relationship graph, where nodes represent multi-scale defect attributes, including edge gradient, texture density, and deformation curvature. A self-supervised message passing mechanism is constructed based on the graph attention network and the Node2Vec embedding method, which outputs dynamic weight features to an improved XGBoost multi-label classifier and generates defect types and defect levels based on prior parameters. The defect types include surface defects and structural defects, and the defect level is 4. The parameter optimization module is used to dynamically adjust the convolution kernel size and classification threshold based on the production line parameters collected in real time by edge nodes and the defect data output by the improved XGBoost multi-label classifier. It uses the proximal strategy optimization model to iterate the strategy function online, and adopts the memristor random calculation method and random resonance coding to suppress transmission noise. The optimized parameters are then fed back to the multi-scale perception module and the graph network classification module.

2. The machine vision-based online detection system for resistor sheet defects according to claim 1, characterized in that: In the dynamic compensation module, the Retinex multi-scale illumination compensation algorithm performs multi-channel decomposition on the original RGB image using Gaussian convolution kernels at three scales: 15, 80, and 200, extracting illumination and reflection components, and eliminating uneven illumination using a logarithmic domain transformation method. For the reflective area of ​​the metal foil, the Retinex multi-scale illumination compensation algorithm detects sudden changes in the gradient of highlight pixels using a Gaussian difference filter, generates a reflective mask matrix, and uses an adaptive weight function to nonlinearly attenuate the reflection component of the highlight area to generate a reflectivity-corrected image. After eliminating reflections, the dynamic compensation module uses a bilateral filter kernel to calculate the weighted mean of the pixel neighborhood, and uses a non-local means algorithm to construct a 15×15 search window. A block similarity metric is used to screen 7×7 similar blocks, and a weighted average iterative method is used to eliminate salt and pepper noise and Gaussian noise. The denoised data is encapsulated into a dual-channel pixel stream using the AXI-Stream protocol, aligned and converted into data format through the parallel multiplier-accumulator array within the FPGA, and a standardized dual-modal image is output.

3. The online detection system for resistor sheet defects based on machine vision according to claim 1, characterized in that: The working method of the multi-scale perception module is: s1. Use the local gradient entropy calculation algorithm to perform gradient amplitude statistics on each pixel neighborhood of the input normalized image and generate a gradient entropy distribution map to quantify regional complexity. The calculation formula is: （1） In formula (1), Indicates location The local gradient entropy at , which is used to quantify the complexity of the 5×5 window pixel neighborhood; is the gradient amplitude within the neighborhood The probability distribution of is the total number of neighborhood pixels; s2, when When ≥1.2, the dynamic adjustment mechanism of the sampling point offset of the deformable convolutional network is triggered. The offset is calculated based on the gradient direction field, and the calculation formula is: （2） In formula (2), , Respectively represent the spatial offset of the convolution kernel sampling point; is the local gradient direction angle, and the calculation formula is: ,in Represents coordinates respectively and Image gradient field; for high entropy areas, drive the 3×3 convolution kernel to shift in the direction of gradient mutation, using the expansion rate The dense convolution kernel extracts high-frequency edge features; s3. Through dilated spatial pyramid pooling, a dual-branch parallel convolution kernel group with dilation rates r=1 and r=6 is embedded in the Conv2 layer; a dual-branch parallel convolution kernel group with dilation rates r=6 and r=12 is embedded in the Conv3 layer; and a single convolution kernel with dilation rates of 1 and 12 is embedded in the Conv1 and Conv4 layers respectively. The receptive field of the Conv4 layer is expanded to: （3） In formula (3), is the effective receptive field size of the convolution kernel; is the expansion rate, which is used to control the interval between sampling points; is the basic convolution kernel size; s4, the outputs of the multi-expansion rate convolution are weightedly fused through the channel attention mechanism to generate a cross-scale feature tensor; The high-resolution features of the Conv2 layer and the semantic features of the Conv4 layer are bilinearly interpolated and aligned through cross-layer skip connections, and a multi-scale fused feature tensor is output.

4. The machine vision-based online detection system for resistor defects according to claim 1, characterized in that: The working principle of the bidirectional cross-layer attention network is: The 112×112 high-resolution feature map of the Conv2 layer is upsampled to 14×14 resolution using a bilinear interpolation algorithm to generate low-level detail features that are aligned with the semantic feature space of the Conv4 layer. A cross-layer relationship matrix is ​​constructed through a multi-head self-attention model. The query vector Q of the cross-layer relationship matrix is ​​generated by 1×1 convolution of Conv4 layer features, and the key vector K and value vector V are extracted by grouped convolution of interpolated Conv2 layer features. In the spatial dimension, the cosine similarity matrix of Q and K is calculated, and a spatial attention weight map is generated through Softmax normalization. The edge details of the Conv2 layer and the semantic context of the Conv4 layer are dynamically weighted and fused. At the same time, a squeeze-excitation network is used to calculate the channel attention weight in the channel dimension. After activation with the Sigmoid function, the fused features are recalibrated in the channel direction. Feature dimensionality reduction is performed through depthwise separable convolution, where the depthwise convolution uses a 3×3 kernel to extract local spatial relationships, and the point-by-point convolution uses a 1×1 kernel to achieve cross-channel information interaction; The final generated multi-scale enhanced feature map is added to the original Conv4 layer features via skip connections, and the residual learning mechanism is used to avoid gradient disappearance.

5. The online detection system for resistor sheet defects based on machine vision according to claim 1, characterized in that: The multi-head self-attention model includes a cross-modal projection layer, a spatial relationship modeling layer, a multi-head decomposition layer, a dynamic weight fusion layer, a residual connection layer, and a regularized output layer. The cross-modal projection layer upsamples the Conv2 layer features to a 14×14 resolution using a bilinear interpolation algorithm, and uses grouped convolution to linearly project the bimodal features to generate a query vector Q, a key vector K, and a value vector V. The projection weight matrix dimension is 512×512. The spatial relationship modeling layer calculates the cosine similarity matrix of Q and K by scaling the dot product attention, where the similarity score is normalized by Softmax to generate a spatial attention weight map, and the weight matrix dimension is 14×14×14×14; the multi-head decomposition layer is used to decompose the weight matrix into 8 independent attention heads along the channel dimension, and each attention head extracts local spatial correlation features through an independent fully connected layer, and outputs a multi-head attention feature group; The dynamic weight fusion layer merges multi-head features through a channel splicing method, and uses a gated recurrent unit to dynamically learn the contribution weights of each attention head to generate a fused global attention feature. The residual connection layer fuses the attention-enhanced features with the original features by element-by-element addition, triggering the residual learning mechanism to retain the underlying semantic information; the regularized output layer uses layer normalization and depth-wise separable convolution to perform feature dimensionality reduction and output an enhanced multi-scale feature map.

6. The machine vision-based online detection system for resistor sheet defects according to claim 1, characterized in that: The working method of the graph network classification module is: u1. Initialize the nodes of the input multi-scale feature tensor based on the Node2Vec embedding algorithm, generate node sequences using a random walk strategy, and obtain a 128-dimensional node feature vector using the Skip-gram model. , where the walk length is set to 10 and the window size is 5 to capture local structural information; the node set after initialization The initial vertices that make up the graph; u2. Calculate the feature correlation between nodes based on cosine similarity. If the similarity threshold exceeds a preset threshold, construct the initial graph topology structure through an improved edge connection generation mechanism. The improved edge connection generation mechanism calculates the attention coefficient between nodes through the multi-head attention layer of the graph attention network, uses the LeakyReLU activation function to perform nonlinear transformation on the features of adjacent nodes, and generates dynamic edge weights. The calculation formula is: （4） In formula (4), For nodes and The dynamic edge weight between them is used to characterize the correlation strength of defect features; and Represents the node embedding vector initialized by Node2Vec, with dimension d=128; Represents the LeakyReLU activation function with a negative slope of 0.2; It is a vector concatenation operation; Represents the learnable parameter vector The transposed matrix of Representation node The set of neighbor nodes of Iteration variable for sum operation; u3. Use the self-supervised message passing mechanism to iteratively update node features and perform three-layer iterative message propagation. Residual connections are used between layers to maintain gradient flow. Each layer updates the central node state through the message passing function. The formula is: （5） In formula (5), For the Nodes at iteration The eigenvector of is the message transformation matrix; It is a gated recurrent unit used to filter effective neighborhood information; u4. After three message transmissions, the node features are mapped into dynamic weight vectors through the fully connected layer and output to the feature input layer of the improved XGBoost classifier. At the same time, the online hard example mining strategy is used to filter samples with classification confidence lower than 0.85, triggering the random walk sequence generation module of Node2Vec, and incrementally updating the node embedding representation based on the Skip-Gram model.

7. The machine vision-based online detection system for resistor sheet defects according to claim 1, characterized in that: The improved XGBoost multi-label classifier receives the dynamic weight feature vector output by the graph neural network through a dynamic feature weighting mechanism, and uses a multi-label tree structure expansion algorithm to construct a parallel decision tree group, where each subtree corresponds to a specific defect type, including scratches, oxidation, and delamination. Then, based on the FL reconstruction objective function, the gradient weight of high-frequency defects is reduced by the category balance factor, while the loss contribution of complex defects is increased. For the dynamic weight feature, the node weight vector is spliced ​​with the original feature through a feature grouping embedding strategy, and after sparse matrix compression, it is input into the tree splitting point selection module. During the tree growth stage, the improved XGBoost multi-label classifier uses a proximal gradient descent algorithm to optimize the split gain calculation. If the feature weight is lower than the preset threshold, the pruning mechanism is triggered to skip irrelevant features. The classification results are transmitted to the reinforcement learning module through an online incremental learning interface, triggering a model parameter update signal. A Shapley value interpreter is also integrated to dynamically analyze feature contributions. When the resistivity feature contribution of a local delamination is detected to deviate from the baseline by 20%, the tree node weights are corrected through an online calibration method.

8. The machine vision-based online detection system for resistor sheet defects according to claim 1, characterized in that: The parameter optimization module collects production line process parameters and defect statistical features output by the XGBoost classifier in real time through edge computing nodes, constructs a state observation vector and inputs it into the proximal policy optimization model. The proximal policy optimization model performs policy evaluation and optimization in parallel through an actor-critic architecture, where the actor network calculates the convolution kernel size, classification threshold, and morphological filter strength based on the advantage function, and the critic network uses the TD error estimate to constrain the policy update amplitude. To address transmission noise interference, the parameter optimization module encodes the strategy parameters into a probabilistic pulse sequence through a memristor random computing unit, adopts a stochastic resonance encoding mechanism, and enhances the signal characteristics through a nonlinear stochastic resonance method when the pulse amplitude is lower than the noise floor; the optimized parameters are encapsulated into a data packet through a probabilistic pulse amplitude modulation protocol and transmitted back to the deformable convolution controller of the multi-scale perception module and the threshold regulator of the graph network classification module through the SPI interface; if the slurry viscosity standard deviation is detected to exceed 20% of the historical baseline, the online fine-tuning mechanism of the strategy network is triggered, and the network weights are updated based on the double-delay deep deterministic policy gradient algorithm; at the same time, a parameter version management queue is established. When three consecutive policy updates cause the classifier F1 score to fluctuate by more than 5%, the version rollback mechanism is triggered to load the historical optimal parameter snapshot.

9. The method for online grading of resistor sheet defects based on machine vision according to claim 1, characterized in that: The prior parameters include geometric characteristic parameters, physical property parameters and process parameters; the geometric characteristic parameters include defect area threshold, shape factor and edge gradient consistency; the physical property parameters include thermal conductivity deviation and resistivity change rate; the process parameters include slurry diffusion coefficient and stacking offset.

10. A method for online grading of resistor sheet defects based on machine vision, characterized by: A machine vision-based online detection system for resistor sheet defects as described in any one of claims 1 to 8, comprising: Step 1: Using a linear array industrial camera and a multispectral ring light source, the RGB image and near-infrared reflectance data stream of the resistor surface are synchronously collected. The Retinex multi-scale illumination compensation algorithm is used to eliminate the interference of metal foil reflections. The non-local mean and wavelet threshold joint denoising model is combined to suppress high-frequency noise and generate standardized dual-modal image data. Step 2: Based on the local gradient entropy driving mechanism, a deformable convolutional network is used to dynamically adjust the offset of the convolution kernel sampling point. Through the multi-expansion rate convolution kernel of the void space pyramid pooling, the cracks below 0.2mm, slurry diffusion deformation and overall tilt features are captured at the Conv1 to Conv4 levels to generate a multi-scale feature tensor. Step 3: A bidirectional cross-layer attention network is used to construct a spatial and channel-wise collaborative attention mechanism for the 112×112 Conv2 layer and the 14×14 Conv4 layer. The multi-head self-attention model is used to calculate the association weights between high-resolution details and low-resolution semantics. After dynamic weighted fusion, the enhanced multi-scale feature map is generated through dimensionality reduction using depthwise separable convolution. Step 4: Map the enhanced multi-scale feature map into a heterogeneous feature relationship graph, where nodes represent multi-scale defect attributes, including edge gradient, texture density, and deformation curvature. Initialize edge connection rules based on the graph attention network and the Node2Vec embedding algorithm, model the physical mapping of exposed foil spots and overall warpage through a self-supervised message passing mechanism, output dynamic weight features to the improved XGBoost classifier, and generate defect types and defect levels based on prior parameters. Step 5: Use edge computing nodes to collect production line process parameters and historical defect databases in real time, iterate the policy function online through the proximal policy optimization model, dynamically adjust the convolution kernel size and classification threshold, and use the memristor random computing method and random resonance coding to suppress transmission noise. The optimized parameters are transmitted back to steps 2 and 4 through probabilistic pulse fault tolerance.

Images