Industrial vision inspection system

Abstract

The application discloses an industrial visual detection system and belongs to the technical field of industrial image processing and artificial intelligence. The system comprises a multi-source heterogeneous data fusion acquisition module, a multi-scale attention feature extraction module, a contrast distillation defect semantic analysis module and an incremental online model evolution module. Through multi-modal sensor fusion acquisition, spatial pyramid expansion residual network multi-scale feature extraction, teacher-student contrast distillation defect positioning and classification, and the combination of knowledge distillation and elastic weight solidification incremental learning strategy, the application constructs a double closed-loop collaborative mechanism with model evolution closed loop and quality control closed loop, realizes a defect detection rate of more than 99.2%, a micro defect detection rate of 97.8% and a new defect adaptation time of 4 to 8 hours.

Description

Technical Field

[0001] This invention belongs to the field of industrial image processing and artificial intelligence technology, and specifically relates to an industrial vision inspection system. Background Technology

[0002] In modern industrial manufacturing, surface defect detection is a crucial step in ensuring product quality. As electronic components, precision parts, and new energy components continue to evolve towards higher density and miniaturization, defect morphologies are becoming increasingly diverse and their scale range is significantly expanding. This places higher demands on the recognition accuracy and adaptability of visual inspection systems. Traditional manual inspection methods are not only inefficient and susceptible to subjective factors, but also struggle to meet the real-time requirements of high-speed production lines. Statistics show that on typical electronic product manufacturing lines, the defect miss rate for manual inspection is typically between 5% and 15%, and the inspection speed cannot keep up with the pace of modern production lines producing hundreds of products per minute. Therefore, the industry has gradually introduced automated inspection solutions based on machine vision and deep learning technologies to replace or assist manual inspection. In recent years, with the widespread application of technologies such as convolutional neural networks and self-supervised learning in the industrial field, the accuracy and efficiency of automated visual inspection have significantly improved. However, achieving continuous adaptive optimization of the inspection system in complex and ever-changing production environments remains a challenge.

[0003] For example, Chinese patent CN 113192068 A discloses an AI visual inspection system for printed circuit boards (PCBs). This system comprises three main components: a front-end data acquisition and processing module, an edge recognition module, and an artificial intelligence cloud platform. The front-end data acquisition and processing module uses an AOI system to acquire and preprocess image data of the PCB surface. The defect dataset covers defects such as holes, solder plates, wires, fingers, reinforcing plates, characters, protective films, inks, and gold surfaces. The edge recognition module transmits model and data via a 5G network, enabling real-time online monitoring of the product and utilizing edge cloud technology to improve processing speed. The artificial intelligence cloud platform is responsible for AI-powered intelligent re-inspection, including image preprocessing, data cleaning and labeling, defect analysis, model matching, and inference operations. It also supports a complete AI modeling process, encompassing raw data collection, data cleaning, data labeling, model training, and model application optimization. The system distributes the trained model to local or edge servers and iteratively optimizes the model by periodically uploading recognition results.

[0004] However, the aforementioned existing technologies still have the following shortcomings in actual industrial deployment. First, at the data acquisition level, the system relies solely on a single AOI device for two-dimensional image capture. When faced with defects exhibiting three-dimensional deformation characteristics such as dents and warps, as well as spectral characteristic defects such as material variations, the single imaging modality struggles to acquire sufficient discriminative information, leading to a high false negative rate for minute defects under complex operating conditions. Second, at the feature extraction level, the system lacks a dedicated network design for the multi-scale characteristics of industrial defects. Existing solutions struggle to effectively capture both microscopic local textures and macroscopic global structures simultaneously, resulting in a significant decrease in detection performance when the defect scale span is large. Third, at the defect analysis level, the system directly employs a classifier for defect discrimination, failing to establish a continuous representation space for defect semantics. This easily leads to confusion between defects with similar shapes but different categories, and it lacks the ability to infer the relationship between defect causes and process characteristics. Fourth, at the model update level, the system relies on an iterative model of manual annotation accumulation and periodic offline retraining. It typically takes 3 to 5 days from the discovery of a new defect type to the model's adaptation, causing the system to remain undetected for new defect types during this period. Fifth, at the decision-making level, the system only outputs the detection results of a single image, lacking the ability to statistically analyze and predict trends in batch-level quality status, and thus cannot provide forward-looking guidance for timely adjustments to production line process parameters.

[0005] Therefore, how to construct an industrial vision inspection system that integrates multimodal data perception, deep feature adaptive extraction, accurate defect semantic analysis, online continuous model evolution, and quality closed-loop situational awareness, in order to break through the bottlenecks of existing technologies in terms of detection accuracy, model adaptation speed, and intelligent decision-making, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] To address the shortcomings of the existing technologies, this invention provides an industrial vision inspection system that achieves end-to-end intelligent processing from data acquisition to quality decision-making through deep coupling of five core modules and a dual closed-loop collaborative mechanism. This system solves the technical deficiencies of existing systems in multimodal information utilization, micro-defect identification, new defect adaptation, and global quality situation awareness.

[0007] The technical solution of this invention is: an industrial vision inspection system, including a multi-source heterogeneous data fusion acquisition module, a multi-scale attention feature extraction module, a comparative distillation defect semantic analysis module, an incremental online model evolution module, and a quality entropy situational awareness decision-making module.

[0008] The multi-source heterogeneous data fusion acquisition module is equipped with a multi-type sensor array including an area array camera, a line spectrum sensor, and a structured light projector, performing multimodal synchronous acquisition of the industrial product surface. This module uses a sub-pixel spatiotemporal registration algorithm to perform spatial alignment and temporal synchronization processing on the image data acquired by each sensor, with a spatial registration accuracy of no less than 0.1 pixels and a temporal synchronization error of no more than 1 ms. After registration, an adaptive Laplacian pyramid fusion unit adaptively allocates fusion weights based on the local gradient energy of each modality at each spatial location and each pyramid level to generate a high-quality multimodal fused image that fully integrates grayscale texture, spectral material, and 3D topography information. A data quality self-evaluation subunit performs real-time detection of the sharpness and illumination uniformity of the fused image, automatically triggering a re-acquisition command and adjusting the light source compensation and exposure parameters when the quality is substandard.

[0009] The multi-scale attention feature extraction module receives the fused image as input and extracts feature maps at different receptive field scales through four parallel dilated convolutional branches in a spatial pyramid dilated residual network. The dilation rates of each branch are 1, 2, 3, and 5, respectively, corresponding to effective receptive field coverage ranging from 3×3 to 11×11, thereby simultaneously capturing features of both micro-pinhole defects and large-area stain defects in a single forward inference. Subsequently, an adaptive weight adjustment is performed on the feature maps at each scale via a channel-space dual attention mechanism. Channel attention learns the importance of each channel through a bottleneck network with a compression ratio of 16, while spatial attention generates a spatial weight map through 7×7 convolutions to enhance the response of the defect region and suppress background interference. Finally, a 256-channel multi-scale defect feature vector is output.

[0010] The comparative distillation defect semantic analysis module adopts a teacher-student dual-branch comparative distillation network architecture. The teacher encoder is trained on normal product samples to learn the compact representation of normal patterns, while the student decoder reconstructs normal feature patterns under teacher supervision. Pixel-level anomaly scores are calculated through multi-level feature differences to achieve precise defect localization. Simultaneously, a prototype-aware memory stores feature prototypes for each defect category and performs semantic classification of defects through cosine similarity matching. A defect knowledge graph reasoning subunit is integrated to provide auxiliary decision-making information such as defect level, causal process, and remediation measures.

[0011] The incremental online model evolution module selects high-value samples based on a comprehensive score of three dimensions: uncertainty, novelty, and human feedback through an active sample screening engine. It uses an incremental training strategy that combines knowledge distillation and elastic weight solidification to update the network parameters online. While learning new defect patterns, it uses the Fisher information matrix to constrain changes in key parameters to suppress catastrophic forgetting. After the updated model is validated and evaluated, it is deployed without interruption by a hot-switching scheduler.

[0012] Compared with existing technologies, this invention has at least the following beneficial effects: Multimodal fusion using an area array camera, line spectrum sensor, and structured light projector significantly improves the image signal-to-noise ratio by approximately 40%, achieving a defect detection rate of over 99.2% and a false detection rate of less than 1%; spatial pyramid expansion residual network and channel-space dual attention mechanism enhance the ability to identify minute defects, achieving a defect detection rate of 97.8% for areas smaller than 0.01 mm²; teacher-student comparative distillation mechanism establishes a continuous semantic space for defects, improving the classification accuracy of defects with similar morphologies; online incremental evolution mechanism combining knowledge distillation and elastic weight solidification shortens the adaptation time for new defect types to 4 to 8 hours; and defect distribution entropy analysis enables early warning of batch-level anomalies 15 to 30 minutes in advance, reducing the batch defect rate by approximately 60%. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the overall architecture of the industrial vision inspection system provided in this embodiment of the invention. Detailed Implementation

[0014] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0015] Reference Figure 1 As shown, the industrial vision inspection system provided in this embodiment of the invention includes a multi-source heterogeneous data fusion acquisition module 1, a multi-scale attention feature extraction module 2, a comparative distillation defect semantic analysis module 3, an incremental online model evolution module 4, and a quality entropy situation awareness decision module 5. These five modules are connected in series along the data flow direction. The output of the multi-source heterogeneous data fusion acquisition module 1 is preprocessed and then sent to the multi-scale attention feature extraction module 2. The multi-scale feature vectors extracted by the multi-scale attention feature extraction module 2 are sent to the comparative distillation defect semantic analysis module 3 for defect semantic analysis. The detection results output by the comparative distillation defect semantic analysis module 3 are sent to the incremental online model evolution module 4 for sample screening and incremental training, and also to the quality entropy situation awareness decision module 5 for batch quality situation assessment. In one embodiment of the invention, after completing incremental training, the incremental online model evolution module 4 sends the updated network weights back to the multi-scale attention feature extraction module 2 and the comparative distillation defect semantic analysis module 3, forming a model evolution closed loop. The quality entropy situation awareness decision module 5 issues acquisition parameter adjustment instructions to the multi-source heterogeneous data fusion acquisition module 1 based on the quality assessment results, forming a quality control closed loop. The aforementioned dual-closed-loop collaborative mechanism enables the system to continuously optimize its performance during continuous production.

[0016] In terms of system hardware deployment, preferably, the multi-source heterogeneous data fusion acquisition module 1 and the multi-scale attention feature extraction module 2 are deployed on edge computing nodes. These edge computing nodes are equipped with GPU accelerator cards with at least 8GB of video memory, performing INT8 quantization inference on the acquired images to ensure that the inference latency for a single product is controlled within 50ms. The incremental online model evolution module 4 and the quality entropy situational awareness decision-making module 5 are deployed on a cloud server cluster, with each server equipped with at least four GPUs with 24GB of video memory. Data transmission between the production line side where the multi-source heterogeneous data fusion acquisition module 1 is located and the cloud is achieved via industrial Ethernet or a 5G private network, with a network bandwidth of at least 1Gbps and an end-to-end transmission latency of no more than 10ms. This edge-cloud collaborative deployment architecture takes into account both the real-time inference requirements of the edge side and the large-scale training requirements of the cloud.

[0017] The multi-source heterogeneous data fusion acquisition module 1 is equipped with a sensor array, a sub-pixel spatiotemporal registration unit, an adaptive Laplacian pyramid fusion unit, and a data quality self-assessment subunit. The sensor array includes three types of sensing devices: an area scan camera, a line spectrum sensor, and a structured light projector. The area scan camera uses an industrial-grade CMOS camera with a resolution of at least 2448×2048 pixels, equipped with a telecentric optical lens to eliminate perspective distortion, and a pixel size of 3.45µm, achieving a spatial resolution better than 10µm / pixel at the standard working distance. The line spectrum sensor operates in the visible to near-infrared range from 380nm to 1100nm, with a spectral resolution of at least 5nm, used to capture the material reflectance spectrum information of the product surface, thereby identifying material variation defects that are difficult to detect using only grayscale images. The structured light projector uses an coded structured light scheme, with a projection resolution of at least 1280×800 and a projection frequency of at least 30Hz, used to acquire three-dimensional morphological information of the product surface to detect defects with depth variations, such as dents, warping, and scratches.

[0018] The subpixel spatiotemporal registration unit is responsible for performing spatial alignment and temporal synchronization processing on the heterogeneous image data acquired by the three sensors. For spatial registration, this unit first transforms the coordinate system of each sensor to the product surface coordinate system using a pre-calibrated extrinsic parameter matrix. Then, it employs a subpixel registration algorithm based on normalized cross-correlation to perform fine alignment on each image pair, achieving a registration accuracy of no less than 0.1 pixels. For temporal synchronization, this unit uses hardware trigger signals to achieve synchronous acquisition of data from each sensor, with a time synchronization error not exceeding 1ms, ensuring strict correspondence of multimodal data at the same product location. Preferably, the registration process uses a GPU-accelerated parallel computing architecture, ensuring that the total registration time for the three sensor data streams does not exceed 5ms.

[0019] After spatiotemporal registration is completed, the registered data is fed into an adaptive Laplacian pyramid fusion unit. The core idea of ​​this fusion unit is to adaptively allocate fusion weights based on the richness of local information in each modality image, thereby fully preserving the most discriminative feature information in each modality. Specifically, the fusion unit first constructs a 5-layer Laplacian pyramid decomposition for each registered image, obtaining high-frequency detail sub-band images and lowest-frequency approximate sub-band images for each layer. Subsequently, for each layer of the pyramid, the local gradient energy of each modality image at that layer is calculated. In one embodiment of the present invention, the first... Road sensor images in the pyramid Layer position Local gradient energy at Calculated using the following formula:

[0020] ,in: For the first Road sensor images in the pyramid Layer position The local gradient energy at a given point, expressed in units of the squared gray value. The value range is 1 to 3, corresponding to the area array camera, the line spectrum sensor, and the structured light projector, respectively. The value range is 1 to 5, corresponding to the 5 levels of the pyramid; For the first Road image in the first Laplacian decomposition subband image of the layer; Indicated by Centered Pixel neighborhood window and This is the offset within the neighborhood; and The horizontal and vertical image gradients are respectively obtained using the Sobel operator. A larger local gradient energy indicates that the corresponding mode in that region contains richer edge detail information at that scale.

[0021] After obtaining the local gradient energy of each layer in each mode, the fusion weights are calculated through a normalization operation. Road image in the first Layer position Fusion weights at the point Determine using the following formula: ,in: For the first Road image in the first Layer position The fusion weights at each location range from 0 to 1. The sum of the weights of the paths is 1; The total number of sensors, in this embodiment ; Here is the regularization constant, and its value range is... to Preferred selection Its function is to prevent numerical instability caused by a denominator of zero. As can be seen from the formula, the mode with a larger gradient energy at a certain spatial location will obtain a higher fusion weight. This means that in areas with rich texture, the grayscale image of the area array camera contributes more; in areas with depth changes, the 3D information of the structured light contributes more; and in areas with material changes, the spectral image contributes more, thus achieving adaptive complementary fusion of the advantages of each mode.

[0022] Finally, for each layer of the pyramid, the images of each modal subband are weighted and summed according to the above weights, and then the fused images of each layer are reconstructed into a full-resolution multimodal fused image through inverse pyramid transformation. Preferably, the pyramid fusion process is implemented in parallel using CUDA, and the fusion time for processing three 2448×2048 resolution images does not exceed 8ms.

[0023] The data quality self-assessment subunit performs a quality assessment immediately after the fused image is generated. This subunit includes two detection functions: sharpness assessment and illumination uniformity assessment. Sharpness assessment uses the Tenengrad operator to calculate the average global gradient magnitude of the fused image as a sharpness index. When this index is lower than a preset sharpness threshold (set to 70% of the historical average for the product category in this embodiment), the image quality is deemed substandard. Illumination uniformity assessment divides the fused image into... The sub-region grid is used to calculate the coefficient of variation of the grayscale mean of each sub-region. When the coefficient of variation exceeds 15%, the illumination is considered uneven. Once any detection index fails to meet the standard, the data quality self-assessment sub-unit sends a re-acquisition command to the sensor array and triggers light source compensation or exposure adjustment to ensure that the data quality received by subsequent modules meets the detection accuracy requirements.

[0024] The multi-scale attention feature extraction module 2 comprises two core components: a spatial pyramid extended residual network and a channel-space dual attention mechanism. This module receives the multimodal fused image output from the multi-source heterogeneous data fusion acquisition module 1 as input, and outputs a 256-channel multi-scale defect feature vector after multi-scale feature extraction and attention weighting.

[0025] The spatial pyramid dilated residual network was designed to address the problem of large target scale spans in industrial defect detection. Different types of defects exhibit significantly different scale characteristics in images: tiny pinhole defects may occupy only a few pixels, while large stains or discoloration defects may cover a considerable area of ​​the image. To simultaneously capture these different scale defect features in a single forward inference pass, this invention designs a spatial pyramid structure containing four parallel dilated convolutional branches. The dilation rates of the four branches are set to 1, 2, 3, and 5, respectively, with corresponding convolutional kernel sizes of 1, 2, 3, and 5. The resulting effective receptive field sizes are respectively , , and Each branch employs a residual connection structure, containing two dilated convolutional layers, a batch normalization layer, and a ReLU activation function. Each branch outputs a 64-channel feature map. The outputs of the four branches are concatenated along the channel dimension to obtain a 256-channel multi-scale initial feature map.

[0026] Preferably, a step size of 2 is also provided at the input end of the spatial pyramid expanded residual network. Standard convolutional layer and one Max pooling layers are used to perform preliminary downsampling and shallow feature extraction on the input fused image, reducing the spatial resolution to the original size. This reduces subsequent computational load while preserving sufficient spatial detail. Before initial downsampling, the input image is standardized by subtracting the corresponding global mean from each channel and dividing by the standard deviation.

[0027] The channel-spatial dual attention mechanism is applied to the 256-channel multi-scale initial feature map output by the spatial pyramid network. It adaptively adjusts the weights of each channel and spatial location to highlight defect-related feature components while suppressing background noise and irrelevant information. This mechanism consists of a channel attention branch and a spatial attention branch connected in series.

[0028] The channel attention branch first performs global average pooling and global max pooling on the input feature map along the spatial dimension, respectively, to obtain two features of size 1. The two description vectors are fed into a shared two-layer fully connected network for processing. The intermediate layer of the fully connected network has a dimension of [missing information]. (in In this embodiment, the channel compression ratio is... (That is, the intermediate layer dimension is 16), and the output layer dimension is restored to 256. The outputs of the two fully connected networks are summed element-wise and then used to generate channel attention weight vectors through the Sigmoid activation function. Each element takes a value between 0 and 1, representing the importance of the corresponding channel to the current detection task. This weight vector is multiplied channel by channel of the input feature map to obtain the channel-weighted feature map.

[0029] The spatial attention branch receives the channel-weighted feature maps as input and performs mean compression and maximum compression along the channel dimension to obtain two single-channel spatial description maps. These two description maps are then concatenated along the channel dimension and fed into a single... The convolutional layer processes the data, and then a spatial attention weight map is generated using the sigmoid activation function. ,in and These represent the height and width of the feature map, respectively, with the attention value at each spatial location ranging from 0 to 1. In one embodiment of the present invention, to further enhance the attention mechanism's ability to focus on defective regions, an attention activation threshold is set. The value ranges from 0.3 to 0.7, with 0.5 being preferred. When the attention weight value for a certain channel or spatial location is lower than... When this happens, the weight value is forcibly set to zero, thereby achieving hard truncation suppression of low response regions and reducing the interference of background noise on subsequent defect analysis.

[0030] After sequential weighting processing using channel attention and spatial attention, the multi-scale attention feature extraction module 2 outputs a 256-channel multi-scale defect feature vector. ,in and This represents the spatial dimensions of the downsampled feature map. This feature vector fuses data from various sources along the channel dimension. arrive The multi-scale receptive field information adaptively enhances the response of potential defective regions in the spatial dimension through an attention mechanism. Preferably, the entire forward inference process of the multi-scale attention feature extraction module 2 is executed on the GPU of the edge computing node, and the feature extraction time for processing a single 2448×2048 fused image does not exceed 15ms.

[0031] The contrastive distillation defect semantic analysis module 3 receives the multi-scale defect feature vector output by the multi-scale attention feature extraction module 2 as input. It employs a teacher-student dual-branch contrastive distillation network architecture to achieve accurate defect localization and semantic classification. The core design concept of this network architecture is that the teacher encoder is trained only on normal product samples to learn a compact representation of normal patterns, while the student decoder learns to reconstruct normal feature patterns under the supervision of the teacher encoder. When a product image containing defects is input, the feature differences generated between the teacher encoder and the student decoder reflect the location and severity information of the defects.

[0032] The teacher encoder uses a pre-trained deep residual network as its backbone. Its parameters are frozen and fixed after sufficient training on a normal sample set and do not participate in subsequent online updates. The teacher encoder extracts four intermediate layer feature maps of different depths from the multi-scale defect feature vectors. The channel dimensions are 64, 128, 256, and 512, corresponding to a progression from shallow texture edge features to deep semantic abstract features. The student decoder's network structure is symmetrical to the teacher encoder, also producing four intermediate layer feature maps. Its channel dimensions correspond one-to-one with each layer of the teacher encoder.

[0033] A feature calibration unit is set between the feature outputs of each layer of the teacher encoder and the student decoder to strip anomalous pattern features from the student decoder, thereby enhancing its focus on normal patterns. The feature calibration unit consists of a... The system consists of convolutional layers and a batch normalization layer, which precisely aligns the feature dimensions of each layer in the student decoder with the corresponding layers in the teacher encoder. Through this calibration operation, the student decoder is constrained to fit the feature distribution of the teacher encoder on normal samples as closely as possible. When the input contains defects, the output of the student decoder deviates significantly from that of the teacher encoder.

[0034] Pixel-level anomaly scores are calculated based on a measure of the difference between the teacher encoder and the student decoder across multiple feature levels. For the first... Layer feature map, defining the anomaly score map of this layer. Spatial location of teacher and student characteristics Cosine distance at: ,in: and The teacher encoder and student decoder are respectively in the first... Spatial location of layer feature maps The eigenvector at that location; for Norm operations. After upsampling to a uniform spatial resolution using bilinear interpolation, the anomaly score maps at each layer are weighted and summed pixel-wise to obtain the final pixel-level anomaly score map. The weights of each layer are determined through grid search optimization on the validation set. Preferably, the weights of the 3rd and 4th layers are greater than those of the first two layers, so as to rely more on high-level semantic features for anomaly detection. The anomaly detection threshold is adaptively determined based on the principle of maximizing the F1 score on the validation set. When the anomaly score at a certain pixel location exceeds the threshold, it is marked as a defect region.

[0035] After defect localization, the comparative distillation defect semantic analysis module 3 further performs semantic classification of defects through a prototype-aware memory. The prototype-aware memory stores a set of feature prototype vectors for each defect category, with 128 to 512 prototypes per category, preferably 256. Feature prototypes are initialized by performing K-Means clustering on the feature representations of various defects in historical training samples, and are continuously updated using a first-in-first-out queue mechanism during subsequent production. For each detected defect region, the feature descriptor of that region is extracted from the final anomaly score map, and its cosine similarity with the prototypes of each category in the memory is calculated. The category with the highest similarity is taken as the classification result for that defect. Preferably, a classification confidence threshold of 0.75 is set. When the highest similarity is below this threshold, the defect is marked as an unknown type and pushed to the incremental online model evolution module 4 for further processing.

[0036] The comparative distillation defect semantic analysis module 3 also integrates a defect knowledge graph reasoning subunit. This subunit maintains a defect knowledge graph containing over 500 triples, covering more than 30 common industrial defect types and their relationships with process parameters, production line positions, and remedial measures. The triples in the knowledge graph are in the form of (defect entity, relation, attribute / entity), such as (weld bridge defect, common causes, excessively large stencil opening), (weld bridge defect, defect level, level 3), (weld bridge defect, remedial measures, reflow soldering temperature profile adjustment), etc. After obtaining the defect classification results, the knowledge graph reasoning subunit uses graph reasoning to query the defect level, possible causative processes, and recommended remedial measures associated with the defect type, outputting this semantic information along with the defect's location coordinates and confidence level to subsequent modules. Furthermore, the knowledge graph supports indirect association discovery based on path reasoning. For example, when a certain defect is detected to occur frequently, the knowledge graph path can be traced to possible common causative processes, providing root cause analysis clues for quality management personnel. This mechanism enables the system not only to provide defect detection results, but also to provide auxiliary decision-making information to support rapid response on the production line.

[0037] The design goal of Incremental Online Model Evolution Module 4 is to enable the system to continuously learn new defect patterns without interrupting production. This module comprises four core components: an active sample selection engine, a sample buffer, a knowledge distillation incremental training engine, and a hot-swap scheduler.

[0038] The active sample selection engine selects the most valuable samples for model evolution from the defect detection results output by the comparative distillation defect semantic analysis module 3. The evaluation of sample value is based on a comprehensive score across three dimensions, with the comprehensive sample value score being... Calculate using the following formula:

[0039] ,in: The overall value score of the sample is given, with a value ranging from 0 to 1. To compare the highest category probability given by the distillation defect semantic analysis module 3 for this sample, As an uncertainty component, the larger its value, the more uncertain the model's judgment of the sample; The cosine distance between the feature vector of this sample and the nearest prototype in the memory is used as the novelty component. A value exceeding 0.4 indicates that the sample has high novelty. For manual feedback, quality inspectors will review the system's annotation results and assign values ​​accordingly: 0 for correct annotations and 1 for incorrect annotations. , and The weighting coefficients for uncertainty, novelty, and human feedback are respectively, preferably set to 0.35, 0.30, and 0.35. When the overall value score of a sample exceeds the preset selection threshold (preferably 0.5), the sample is selected into the sample buffer.

[0040] The sample buffer has a storage capacity of 1000 to 3000 samples, preferably 2000. When the buffer is full, a priority replacement strategy based on value scores is adopted, that is, the sample with the lowest current value score is removed to make room for new high-value samples. A sample counter for each defect category is maintained in the buffer. When the cumulative number of samples for a certain category reaches the minimum trigger amount for incremental training (preferably 50 new samples) or the total number of samples in the buffer reaches 80% of its capacity, an incremental training process is triggered.

[0041] The knowledge distillation incremental training engine employs a teacher-student architecture to achieve incremental model updates. Before each round of incremental training, the parameters of the currently deployed model are copied to the teacher model. The new student model, under the soft-label supervision of the teacher model, simultaneously learns from new samples in the buffer and replay samples randomly sampled from historical data. The total loss function for training... It consists of three components: ,in: For classification cross-entropy loss, ; For knowledge distillation loss, Specifically, it is defined as the KL divergence between the softened probability distributions of the teacher model and the student model:

[0042] ,in The distillation temperature parameter is used in this embodiment. By increasing the temperature, the probability distribution is made smoother, thus conveying more information about inter-class relationships. For the loss of elastic weight solidification, Its definition is: ,in: The forgetting inhibition coefficient is set to 0.2 in this embodiment; For the Fisher information matrix Diagonal elements, representing parameters The importance of the knowledge already learned; and The first and second models of the student model and teacher model, respectively. The model employs a flexible weight solidification loss to suppress catastrophic forgetting by penalizing significant modifications to important parameters, allowing the model to retain its ability to identify known defect types while learning new defect patterns. Preferably, each incremental training iteration consists of no more than 20 epochs, and the learning rate uses a cosine annealing strategy. decay to .

[0043] The hot-switching scheduler is responsible for safely deploying newly trained models to the inference pipeline. The scheduler first evaluates the new model's performance metrics, including defect detection accuracy, F1 score, and inference latency, on a fixed validation set containing 200 labeled samples. The scheduler only performs a hot-switching operation if the new model's F1 score is not lower than the F1 score of the currently deployed model minus a tolerance threshold (preferably 0.02) and the inference latency does not exceed 110% of the current model's latency. The hot-switching employs a double-buffering mechanism, instantly switching the inference entry pointer after the new model is loaded into the GPU's spare buffer, ensuring that the entire switching process does not interrupt production line detection for more than 100ms. The scheduler also maintains a model version history, retaining snapshots of up to the 10 most recent versions, supporting one-click rollback to any historical version in case of performance degradation in the new model.

[0044] The quality entropy situational awareness decision module 5 receives the defect detection results of each product output by the comparative distillation defect semantic analysis module 3, constructs a global situational profile of batch quality through statistical analysis and information entropy measurement, and performs hierarchical early warning and closed-loop feedback decisions accordingly.

[0045] This module first uses a sliding window to statistically analyze the defect distribution information of products over a recent period. The length of the sliding window is 30 to 80 products, preferably 50 products. For the product group within the window, the frequency distribution of each defect category is statistically analyzed. ,in This represents the total number of defect categories. For the first The frequency of each type of defect occurring within the window. Based on this frequency distribution, the defect distribution entropy is calculated. : , where: when Time definition Defect distribution entropy is used to characterize the degree of disorder in defect occurrence patterns: when the entropy value is low, it means that defects are concentrated in a few types, which usually corresponds to controllable systemic process problems; when the entropy value is high, it means that multiple defect types occur at the same time and there is no obvious dominant type, which usually corresponds to a more serious quality out-of-control state.

[0046] Based on indicators such as defect distribution entropy and defect rate, the quality entropy situational awareness decision module 5 classifies batch quality into four levels. When the defect distribution entropy is below 0.3 and the defect rate is below 0.5%, it is rated as excellent, indicating good production line operation with no intervention required. When the defect distribution entropy is between 0.3 and 0.6 or the defect rate is between 0.5% and 2%, it is rated as good, requiring attention but no immediate action is needed. When the defect distribution entropy is between 0.6 and 0.85 or the defect rate is between 2% and 5%, it is rated as alarm level, triggering a general warning. When the defect distribution entropy exceeds 0.85 or the defect rate exceeds 5%, it is rated as critical level, triggering an emergency warning. Furthermore, when a preset number (preferably 3) of severely defective products appear consecutively, or when a level 4 (most severe) defect appears, an emergency warning is directly triggered regardless of the current entropy and defect rate.

[0047] The Quality Entropy Situation Awareness Decision Module 5 also calculates the batch quality comprehensive index. To provide a more refined quantitative assessment of quality trends:

[0048] ,in: The value ranges from 0 to 1, with higher values ​​indicating better batch quality. The defect rate within the window; The average severity of defects within the window (normalized to the range of 0-1). The temporal clustering degree of defects measures the degree of concentration of defects on the time axis, which is obtained through run statistics of consecutive defective products; , and These are the weighting coefficients for defect rate, average severity, and clustering, respectively.

[0049] Regarding the generation of early warning signals, the general triggering condition for an early warning is: continuous Product (preferred) The defect distribution entropy value exceeds 0.6, or the batch quality comprehensive index... The continuous downward trend lasts for more than 10 window periods. The trigger conditions for emergency warnings are: the occurrence of a level 4 defect, the detection of defects in 3 consecutive products, or the defect distribution entropy value exceeding 0.85. The warning signal contains structured information such as the warning level, triggering reason, current defect distribution statistics, and recommended measures, and is pushed to the production line management system and quality inspection personnel terminals through industrial communication protocols.

[0050] The closed-loop feedback of quality control is one of the core mechanisms that distinguishes this invention from existing technologies. When the quality level assessed by the quality entropy situational awareness decision module 5 changes, it sends an instruction to the multi-source heterogeneous data fusion acquisition module 1 to adjust the acquisition parameters. Specifically, when the quality level drops from excellent or good to alarm level, the quality entropy situational awareness decision module 5 instructs the multi-source heterogeneous data fusion acquisition module 1 to increase the acquisition resolution of the area array camera from the default 2448×2048 to 4096×3072 to capture more refined defect morphological features. At the same time, it increases the projection density of the structured light projector by 50% to enhance the detection sensitivity of small three-dimensional defects, and raises the sharpness threshold of the data quality self-assessment sub-unit to 85% of the historical average to perform more stringent data quality screening. When the quality level further drops to critical level, it also additionally enables the full-band scanning mode of the line spectrum sensor (replacing the default key band sampling mode) to comprehensively acquire material information. Conversely, when the quality level recovers to excellent and remains stable for more than 20 window periods, the quality entropy situational awareness decision module 5 instructs the multi-source heterogeneous data fusion acquisition module 1 to revert all acquisition parameters to their default configurations to reduce system resource consumption. This closed-loop mechanism enables the system to dynamically adjust detection sensitivity based on real-time quality conditions, maintaining high-efficiency detection throughput under normal production conditions and automatically improving detection accuracy to detect potential problems as early as possible when quality anomalies occur.

[0051] Regarding the model evolution closed loop, after the incremental online model evolution module 4 completes one round of incremental training and passes the verification and evaluation by the hot-switching scheduler, it transmits the updated network weight parameters back to the multi-scale attention feature extraction module 2 and the contrastive distillation defect semantic analysis module 3 deployed on the edge computing nodes via industrial Ethernet or 5G private network. The weight back transmission adopts a differential update strategy, transmitting only the changed parameter components to reduce network bandwidth consumption, with the typical single weight update data volume controlled within 50MB. After receiving the updated weights, the edge nodes perform model hot loading, and the entire process ensures uninterrupted inference service through a double buffering mechanism. The collaborative operation mechanism between the model evolution closed loop and the quality control closed loop is as follows: when the quality control closed loop detects a quality decline trend, it notifies the model evolution closed loop to accelerate sample collection and incremental training processes while increasing the sensitivity of collected parameters. Higher-priority incremental training tasks can obtain more computing resources in the cloud, thereby enabling the system to have more precise perception capabilities and faster learning and adaptation capabilities during periods of quality anomalies.

[0052] The technical effectiveness of the industrial vision inspection system provided by this invention is verified and explained below using specific experimental data. The system of this invention was deployed on the printed circuit board production line of an electronics manufacturing company and ran continuously for 30 days, inspecting a total of over 150,000 products. The baseline system in comparison is a traditional scheme using a single AOI camera and an offline trained model.

[0053] In terms of defect detection performance, the overall defect detection rate of the system of this invention reaches 99.2%, an improvement of 4.1 percentage points compared to the baseline system's 95.1%. Specifically, the detection rate of minute defects with an area less than 0.01 mm² reaches 97.8%, an improvement of 9.5 percentage points compared to the baseline system's 88.3%. This is mainly due to the enhanced capture capability of the multi-scale attention feature extraction module for subtle features and the approximately 40% improvement in image signal-to-noise ratio achieved by multi-modal fusion. Regarding the false detection rate, the false detection rate of the system of this invention is less than 1%, a reduction of 2.2 percentage points compared to the baseline system's 3.2%. This is attributed to the more accurate modeling of normal pattern representation by the comparative distillation architecture, which makes the system more robust to normal surface variations.

[0054] In terms of defect classification accuracy, the comparative distillation defect semantic analysis module, combined with the prototype-aware memory bank, achieved a classification accuracy of 96.5% for 30 known defect types. Among them, the accuracy in distinguishing between weld bridge and weld ball defects with similar shapes improved from 82.1% in the baseline system to 94.7%. The defect knowledge graph reasoning subunit achieved a recommendation accuracy of 89.3% in defect cause association, providing effective auxiliary information for quality inspectors to make rapid decisions.

[0055] Regarding model evolution capabilities, the system automatically triggered seven incremental learning updates during the 30-day run, successfully identifying and adapting to three new defect types not present in the initial training set: a novel solder oxidation defect, a solder resist microcrack defect, and a copper foil delamination defect. The adaptation time from the first appearance of a new defect to the model achieving stable recognition capability was 4 to 8 hours, while the baseline system required offline retraining after collecting a large number of samples, with an adaptation period typically of 3 to 5 days. During the seven incremental updates, the model's accuracy in recognizing known defect types consistently remained within 0.5%, validating the effective suppression of catastrophic forgetting by the elastic weight solidification mechanism.

[0056] Regarding quality early warning, the system accurately predicted two batch quality anomaly events within 30 days, with warning lead times of 15 minutes and 22 minutes, respectively. In the first event, the defect distribution entropy value continuously increased from 0.21 to 0.72 within 30 minutes. The quality entropy situational awareness decision module 5 immediately issued a warning after the entropy value exceeded the general warning threshold of 0.6. The production line was immediately shut down for investigation and a faulty reflow oven temperature sensor was found, thus timely preventing the generation of a large batch of defective products. In the second event, the quality entropy situational awareness decision module 5 identified a gradual quality degradation pattern through the continuous downward trend of the batch quality comprehensive index, and issued a warning 22 minutes in advance about solder paste supply batch changes causing soldering quality fluctuations. The timely handling of the two warning events reduced the batch defect rate during the relevant period by approximately 60% compared to the estimated value under no warning conditions.

[0057] In terms of inference efficiency, the end-to-end inference latency for a single product is 38ms, with multimodal fusion acquisition taking approximately 13ms, multi-scale feature extraction taking approximately 15ms, and defect semantic analysis taking approximately 10ms, meeting the production line cycle time requirement of 120 units per minute. During 30 days of continuous operation, the system achieved an availability rate of 99.97%, with only one brief service interruption due to network fluctuations (lasting 42 seconds). During this interruption, edge nodes automatically switched to the local cached model to continue executing detection tasks, preventing any missed product detections.

[0058] In summary, the industrial vision inspection system provided in this embodiment of the invention achieves significant technical improvements in key performance indicators such as defect detection rate, accuracy of minor defect identification, speed of new defect adaptation, and advance quality warning through a deep coupling and dual-closed-loop collaborative mechanism of five modules: multi-source heterogeneous data fusion acquisition, multi-scale attention feature extraction, comparative distillation defect semantic analysis, incremental online model evolution, and quality entropy situational awareness decision-making. This system is not only applicable to the printed circuit board inspection scenario described in this embodiment, but can also be extended to various industrial vision inspection scenarios such as semiconductor wafer inspection, lithium battery electrode inspection, automotive parts surface inspection, and photovoltaic module appearance inspection, demonstrating good versatility and scalability.

[0059] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection defined by the claims of the present invention.

Claims

1. An industrial vision inspection system, characterized in that, It includes a multi-source heterogeneous data fusion and acquisition module, a multi-scale attention feature extraction module, a comparative distillation defect semantic analysis module, and an incremental online model evolution module; The multi-source heterogeneous data fusion acquisition module is equipped with a sensor array including an area array camera, a line spectrum sensor, and a structured light projector. It is used to perform multimodal synchronous acquisition of the surface of industrial products. The image data acquired by each sensor is spatially aligned and temporally synchronized through a sub-pixel spatiotemporal registration algorithm. Then, the adaptive Laplacian pyramid fusion unit allocates fusion weights according to local gradient energy to generate a multimodal fused image. At the same time, the data quality self-evaluation subunit detects the sharpness and illumination uniformity of the multimodal fused image and triggers a re-acquisition command when the quality is not up to standard. The multi-scale attention feature extraction module receives the multimodal fused image as input, extracts feature maps at different receptive field scales through multiple dilated convolution branches set in parallel in the spatial pyramid dilated residual network, and performs adaptive weight adjustment on the feature maps at each scale through a channel-space dual attention mechanism to output a multi-scale defect feature vector. The comparative distillation defect semantic analysis module receives the multi-scale defect feature vector, extracts normal pattern features through the teacher encoder in the teacher-student dual-branch comparative distillation network and reconstructs the feature representation by the student decoder, calculates pixel-level anomaly scores based on the multi-layer feature differences between the teacher encoder and the student decoder to achieve defect localization, and stores feature prototypes of each defect category based on the prototype-aware memory and achieves defect classification through cosine similarity matching. The incremental online model evolution module selects high-value samples from the production process through an active sample screening engine and stores them in a sample buffer. It uses an incremental training strategy that combines knowledge distillation and elastic weight solidification to perform online updates on the network parameters in the multi-scale attention feature extraction module and the contrastive distillation defect semantic analysis module. After the updated model passes the verification and evaluation, it performs hot switching deployment.

2. The industrial vision inspection system according to claim 1, characterized in that, The sub-pixel spatiotemporal registration algorithm has a registration accuracy of no less than 0.1 pixels, and the time synchronization error between sensors is no greater than 1ms; the adaptive Laplacian pyramid fusion unit has 5 pyramid layers, and the regularization constant of the fusion weights ranges from 1×10⁻⁶. -8 Up to 1×10 -6 .

3. The industrial vision inspection system according to claim 1, characterized in that, The channel compression ratio in the channel-space dual attention mechanism is 16, and the attention activation threshold ranges from 0.3 to 0.

7.

4. The industrial vision inspection system according to claim 1, characterized in that, The anomaly determination threshold for the pixel-level anomaly score is adaptively determined based on the principle of maximizing the F1 score of the validation set; the number of feature prototypes stored in each defect category in the prototype-aware memory is between 128 and 512.

5. The industrial vision inspection system according to claim 1, characterized in that, The spatial pyramid dilated residual network contains four parallel dilated convolutional branches with dilation rates of 1, 2, 3 and 5, respectively, and corresponding effective receptive field sizes of 3×3, 5×5, 7×7 and 11×11.

6. The industrial vision inspection system according to claim 1, characterized in that, The total loss function of the incremental training strategy combining knowledge distillation and elastic weight solidification includes three components: classification loss, knowledge distillation loss, and elastic weight solidification loss. The weight coefficients of classification loss, knowledge distillation loss, and elastic weight solidification loss are 0.4 and 0.2 respectively.

7. The industrial vision inspection system according to claim 1, characterized in that, The teacher-student dual-branch contrastive distillation network also includes a feature calibration unit, which is located between the feature outputs of each layer of the teacher encoder and the student decoder. The feature calibration unit is used to strip abnormal pattern features from the student decoder to enhance its focus on normal patterns.

8. The industrial vision inspection system according to claim 1, characterized in that, The active sample screening engine performs screening based on the comprehensive sample value score, which is obtained by weighted summation of uncertainty component, novelty component and human feedback component; the sample buffer has a capacity of 1000 to 3000 samples and adopts a priority replacement strategy based on value score.

9. The industrial vision inspection system according to claim 8, characterized in that, It also includes a quality entropy situational awareness decision module, which receives the defect detection results output by the comparative distillation defect semantic analysis module, statistically analyzes defect distribution information through a sliding window and calculates the defect distribution entropy value to evaluate the batch quality level, generates an early warning signal when the defect distribution entropy value exceeds a preset threshold, and feeds back the batch quality level to the multi-source heterogeneous data fusion acquisition module to dynamically adjust the acquisition parameters. The acquisition parameter adjustment fed back by the quality entropy situational awareness decision module to the multi-source heterogeneous data fusion acquisition module includes: increasing the acquisition resolution of the area array camera and increasing the projection density of the structured light projector when the quality level decreases, and reverting the acquisition parameters to the default configuration after the quality level recovers.

10. The industrial vision inspection system according to claim 9, characterized in that, The sliding window has a length of 30 to 80 products. The preset threshold includes a general warning threshold and an emergency warning threshold. A general warning signal is generated when the defect distribution entropy value exceeds the general warning threshold. An emergency warning signal is generated when the defect distribution entropy value exceeds the emergency warning threshold or when a preset number of serious defects occur consecutively.

Images