Computer vision-based method for detecting surface defects of green printed products

Abstract

The present application relates to the technical field of industrial vision detection, and discloses a green color printing surface defect detection method based on computer vision. The method comprises the following steps: collecting real-time image flow and performing multi-scale decomposition to generate a feature vector sequence; constructing a dynamic defect template set by using a generative adversarial network; encoding features by using a convolutional neural network, calculating a similarity matrix with the template set, and generating a defect matching index; adjusting the detection threshold value adaptively according to the defect matching index, and obtaining a defect probability distribution map through clustering analysis; identifying a defect aggregation area through space-time domain analysis, and starting a local re-inspection mechanism; updating a defect feature prototype library and optimizing network weights by using re-inspection data, and forming a self-learning closed loop; and finally performing full-process quality consistency inspection and outputting a report. The present application realizes dynamic adaptation and continuous optimization of the detection process.

Description

Technical Field

[0001] This invention relates to the field of industrial visual inspection technology, specifically a method for detecting surface defects in green color printed products based on computer vision. Background Technology

[0002] In the production of color printed products, surface defect detection is a crucial step in ensuring product quality. Current mainstream automated detection methods primarily rely on traditional image processing algorithms or deep learning-based classification models. These methods typically require the pre-establishment of a static database containing various known defect samples. Defect classification is then performed by matching real-time captured product images with defect templates in the database or by directly utilizing a trained convolutional neural network model.

[0003] The aforementioned existing technical solutions have several drawbacks: Static defect databases have limited coverage, making it difficult to handle newly emerging or mutated unknown defect types on the production line. This results in insufficient generalization ability of the detection system, making it prone to missing new defects. Detection systems based on fixed models lack self-learning capabilities. Once production line processes change or new defect patterns emerge, it is necessary to manually re-annotate large amounts of data and update the model offline. The entire process is slow and cannot meet the real-time requirements of continuous production. Existing technologies typically analyze each image independently, lacking insight into the spatiotemporal distribution patterns of defects, failing to intelligently allocate computing resources, and exhibiting weak verification capabilities for suspicious areas.

[0004] Industrial production requires a detection solution that can dynamically adapt to changes in defects and possess self-optimization capabilities. The core issues lie in how to overcome the limitations of static template libraries, enabling the system to anticipate and identify potential defect patterns, and how to construct a closed-loop system that can continuously learn from detection practice and become more refined with each inspection, thereby improving the detection rate of complex and ever-changing defects and the system's adaptability. Summary of the Invention

[0005] The purpose of this invention is to provide a computer vision-based method for detecting surface defects in green printed materials, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides a computer vision-based method for detecting surface defects in green printed materials, the method comprising:

[0007] The system collects real-time image streams during the color printing production process, extracts feature layers from the images through multi-scale decomposition, and generates a sequence of feature vectors.

[0008] Construct a prototype library of defect features, train a generative adversarial network based on historical defect data, and output a dynamic defect template set;

[0009] The feature vector sequence of the real-time image stream is encoded using a convolutional neural network, and a similarity matrix with the dynamic defect template set is calculated to generate a defect matching index.

[0010] Based on the defect matching index, the detection parameter thresholds are adaptively adjusted, and a defect probability distribution map is generated through cluster analysis.

[0011] Spatiotemporal domain analysis was performed on the defect probability distribution map to identify defect clustering regions and calculate the defect density within these regions.

[0012] Based on defect density, a local re-inspection mechanism is initiated to perform multiple rounds of scanning on the clustered areas and update the defect feature prototype library.

[0013] Optimize the weight parameters of the convolutional neural network based on the updated defect feature prototype library;

[0014] Perform a full-process quality consistency check, compare the deviation between the real-time image stream and the standard image library, and output a defect detection report.

[0015] Preferably, the real-time image stream acquired during the color printing production process, and the feature layer extracted from the image through multi-scale decomposition, includes:

[0016] A high-resolution vision sensor is configured to capture continuous image frames on the surface of the printed material. Each image frame is processed by wavelet transform and decomposed into a low-frequency baseband layer and a high-frequency detail layer.

[0017] Global texture features are extracted from the low-frequency baseband layer, and edge and blob features are extracted from the high-frequency detail layer. The global texture features and edge and blob features are merged to generate a feature vector sequence.

[0018] Preferably, the construction of the defect feature prototype library and the training of the generative adversarial network based on historical defect data include:

[0019] Collect historical defect image samples, label each sample, and generate defect type labels and location information;

[0020] A generator using a generative adversarial network synthesizes defect feature vectors, a discriminator distinguishes between real defects and synthesized features, and the generator outputs a dynamic defect template set through iterative training.

[0021] The dynamic defect template set is stored in the feature prototype library, and a mapping relationship between the template and the defect type is established.

[0022] Preferably, the step of encoding the feature vector sequence of the real-time image stream using a convolutional neural network and calculating the similarity matrix with the dynamic defect template set includes:

[0023] Design a convolutional neural network structure containing multiple convolutional and pooling layers to reduce the dimensionality and encode the feature vector sequence;

[0024] Calculate the Euclidean distance between the encoded feature vector and each template in the dynamic defect template set to generate a similarity matrix;

[0025] Set a similarity threshold to convert the similarity matrix into a binary matching index.

[0026] Preferably, the adaptive adjustment of the detection parameter threshold based on the defect matching index includes:

[0027] Analyze the time series changes of the matching indicators and calculate the volatility of the indicators;

[0028] Based on the volatility of the indicators, the sensitivity parameters and classification thresholds for defect detection are dynamically adjusted.

[0029] The matching indicators are grouped using a clustering algorithm to generate a defect probability distribution map and identify high-probability areas.

[0030] Preferably, the step of performing spatiotemporal domain analysis on the defect probability distribution map to identify defect clustering regions includes:

[0031] Map the defect probability distribution map onto the coordinate space of the printed surface and calculate the defect probability value at each coordinate point.

[0032] A region growing algorithm is used to start from high-probability points and merge adjacent points to form clustered regions;

[0033] Calculate the defect density of each cluster region and prioritize the regions based on the density values.

[0034] Preferably, the step of initiating a local re-examination mechanism based on defect density includes:

[0035] For clustered areas where the defect density exceeds a preset threshold, additional visual sensors are deployed for focused scanning;

[0036] Perform multiple rounds of image acquisition, compare the results of each round of scanning, and verify the consistency of defects;

[0037] The verified defect data is added to the historical defect database, and the defect feature prototype library is updated.

[0038] Preferably, optimizing the weight parameters of the convolutional neural network based on the updated defect feature prototype library includes:

[0039] Training data is sampled from the updated defect feature prototype library, and the loss function of the convolutional neural network is calculated.

[0040] The network weights are adjusted using the gradient backpropagation algorithm to minimize the loss function;

[0041] Repeat the iterative optimization process until the network converges to a stable state.

[0042] Preferably, the process of performing a full-process quality consistency check includes:

[0043] Retrieve reference images for the corresponding color printed product type from the standard image library and extract reference feature vectors;

[0044] The feature vector sequence of the real-time image stream is compared frame by frame with the reference feature vector to calculate the deviation index;

[0045] Based on the deviation index, a defect detection report is generated, which includes the defect type, location, and severity level.

[0046] Preferably, after outputting the defect detection report, the system further includes:

[0047] The process parameters of the production line are automatically adjusted based on the severity level in the defect detection report.

[0048] Defect data is recorded in the quality log for subsequent model retraining and system optimization.

[0049] Compared with the prior art, the beneficial effects of the present invention are:

[0050] Generative Adversarial Networks (GANs) trained on historical defect data output dynamic defect template sets, replacing traditional static defect sample libraries. GANs learn the deep feature distributions of historical defect data and generate diverse defect feature variants based on this. This expands the "defect knowledge" held by the detection system beyond the limited number of samples already collected, extending it to a dynamic feature space understood and constructed by the model, covering known defects and potential variant forms. When calculating the similarity between real-time image features and the dynamic template set, the system essentially performs matching within a richer and more generalizable feature space, thereby enhancing the sensitivity to identify unknown and variant defects and reducing the risk of missed detections due to minor changes in defect morphology.

[0051] By utilizing defect probability distribution maps to identify defect clustering regions and initiating a local re-inspection mechanism, the data acquired during re-inspection is used to update the defect feature prototype library and further optimize the weight parameters of the convolutional neural network. This mechanism introduces a focus-and-feedback learning loop based on spatiotemporal analysis. The system identifies the clustering effect of defects in the temporal and spatial dimensions through cluster analysis, automatically focusing detection resources on high-probability areas for multiple rounds of refined scanning. This not only improves the confidence level in detecting high-risk areas, but more importantly, it transforms the new data acquired during re-inspection, especially defect samples that cause uncertainty in the system's judgment, into nourishment for model updates. By continuously feeding new situations encountered during online detection back to the feature prototype library and network model, the detection model achieves online adaptation and performance iteration after deployment. This allows the system to autonomously adapt to process drift and newly emerging defect patterns on the production line without interrupting production, gradually improving detection accuracy and stability. Attached Figure Description

[0052] Figure 1 This is a schematic diagram illustrating the working principle of the computer vision-based green color printed product surface defect detection method described in this invention.

[0053] Figure 2 The flowchart for acquiring image streams and extracting features through multi-scale decomposition;

[0054] Figure 3 A flowchart for constructing a prototype library of defect features and training a generative adversarial network;

[0055] Figure 4 This is a stability analysis chart for defect detection;

[0056] Figure 5 This is a diagram verifying the local re-examination mechanism. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Please see Figure 1This invention provides a computer vision-based method for detecting surface defects in green printed materials. The method includes: an image acquisition module continuously capturing a real-time image stream of the printed material surface; each image, after preprocessing, is sent to a multi-scale decomposition unit. This unit uses a wavelet transform algorithm to decompose the image into sub-bands of different frequencies, thereby separating low-frequency information representing the overall texture and high-frequency information containing detailed features. The feature vectors extracted from each sub-band are organized into a sequence form, providing structured data input for subsequent analysis. The system's built-in defect feature prototype library is dynamically maintained through a pre-trained generative adversarial network. This network continuously generates and updates defect templates using historical defect data to adapt to the emergence of new defects. A convolutional neural network encodes the real-time feature sequence and calculates its similarity with a dynamic defect template set to generate a matching index reflecting the probability of defects. Based on this index, the system adaptively adjusts the detection sensitivity threshold and generates a defect probability distribution map through a clustering algorithm. Spatiotemporal analysis of the distribution map identifies defect clustering areas and calculates the defect density in these areas. For high-density areas, the system initiates a local re-inspection mechanism, scheduling auxiliary cameras to perform multi-angle verification and feeding back confirmed defect data to the prototype library for optimizing the convolutional neural network model. Finally, the system performs a full-process quality consistency check, comparing real-time images with a standard image library and outputting an inspection report containing defect type, location, and severity level, adjusting the production process accordingly.

[0059] Example 1: See Figure 2In practical implementation, the high-resolution vision sensor is configured using a linear scan method, with the sensor resolution strictly set to no less than 5 million pixels to ensure the clarity of capturing the microscopic details on the surface of the printed product. The acquisition frame rate is hard-synchronized with the speed of the production line conveyor belt. The encoder provides real-time feedback on the conveyor belt position, dynamically adjusting the exposure time and acquisition interval to ensure that no consecutive image frames are missed. In practical implementation, each acquired image frame is immediately sent to a dedicated wavelet transform processing unit. The wavelet transform processing unit uses the Daubechies wavelet basis function to perform multi-scale decomposition, decomposing each image frame into three scales to obtain a low-frequency approximate sub-band and multiple high-frequency detail sub-bands. The decomposition process is achieved through convolution and downsampling operations. The low-frequency approximate sub-band carries the overall contour and smooth area information of the image, while the high-frequency detail sub-bands contain edge, texture abrupt changes, and noise components. In practical implementation, when extracting global texture features from the low-frequency baseband layer, a gray-level co-occurrence matrix (GLCM) algorithm is applied. The GLCM is constructed based on the spatial relationship of pixel pairs, with calculation directions including 0 degrees, 45 degrees, 90 degrees, and 135 degrees. Statistical parameters such as contrast, energy, homogeneity, and correlation are derived from the matrix. Contrast reflects the image's sharpness and the depth of texture grooves; energy reflects the uniformity and regularity of the image texture; homogeneity measures the consistency of local textures; and correlation represents linear dependencies. In practical implementation, the processing of the high-frequency detail layer focuses on local abrupt changes. The Canny edge detection operator is used to extract edge features. The Canny operator smooths the image using Gaussian filtering, calculates the gradient magnitude and direction, and applies non-maximum suppression and double-threshold hysteresis to connect edges. Simultaneously, a blob detection algorithm is used to identify point-like anomalies on the surface. Blob detection calculates the second derivative of the image based on the Hessian matrix and filters regions whose feature values ​​match blob characteristics. All extracted feature parameters, including low-frequency texture parameters and high-frequency edge speckle parameters, are normalized using a min-max scaling method. This linearly transforms each feature parameter to the [0,1] interval, eliminating the influence of different parameter dimensions. These parameters are then merged into a high-dimensional feature vector according to a predefined order, following the principle of low-frequency before high-frequency and global before local. Each image frame corresponds to one feature vector, and the feature vectors of consecutive frames are arranged chronologically to form a feature vector sequence. In practice, the feature vector sequence is encapsulated into a data packet and transmitted to subsequent processing modules via a high-speed bus such as a PCIe interface. The data packet structure includes the frame number, timestamp, and feature vector data. To ensure real-time performance, DMA (Direct Memory Access) is used to reduce CPU overhead. To ensure processing efficiency, the dimensionality of the feature vectors is compressed using principal component analysis (PCA). PCA calculates the covariance matrix of the feature vectors, solves for eigenvalues ​​and eigenvectors, and selects principal components with a cumulative contribution rate exceeding 95% as a new feature subset, projecting the original high-dimensional features into a low-dimensional space.

[0060] In some embodiments, the deployment of the high-resolution vision sensor takes into account lighting conditions, using a ring-shaped LED light source to provide uniform illumination. The color temperature of the light source is stabilized at 5500K to avoid interference from shadows and reflections. A telecentric lens is used for the sensor to reduce perspective distortion and ensure the geometric accuracy of the image. In some embodiments, the hardware implementation of the wavelet transform processing unit is accelerated using FPGA. Convolution and filtering operations are performed in parallel within the FPGA to improve decomposition speed. The Daubechies wavelet basis function is selected as the db4 wavelet to balance tight support and smoothness. The three-layer decomposition corresponds to the image size being halved layer by layer. The low-frequency subband retains the main structural information, while the high-frequency subband captures detailed changes. Optionally, the construction parameters of the gray-level co-occurrence matrix can be adjusted. For example, the pixel distance d can be set to 1, 2, or 3 pixels, and the angular partitioning can be refined to more directions to adapt to different texture complexities. In addition to the standard set, the extracted statistical parameters can also include derived parameters such as entropy or variance. Optionally, the dual-threshold setting of the Canny edge detection operator is based on the global gradient distribution of the image. The high threshold is selected from the top 20% quantile of the gradient magnitude, and the low threshold is set to half of the high threshold. The scale parameter of the Hessian matrix in blob detection is determined experimentally, and multiple scale fusion results are usually selected. It can be understood that feature vector normalization is a key step to prevent certain large numerical features from dominating distance calculation. The predefined rules for the merging order are stored in the configuration file, supporting dynamic loading and updating. The temporal alignment of the feature vector sequence is handled by interpolation methods to handle frame rate fluctuations. It can also be understood that the dimensionality reduction operation of principal component analysis is completed offline during the training phase. After calculating the projection matrix, matrix multiplication is directly applied in the online phase to achieve fast compression. The threshold for retaining more than 95% of the original information can be fine-tuned according to application requirements.

[0061] In practical implementation, the calibration process for high-resolution vision sensors includes geometric calibration and radiometric calibration. Geometric calibration uses a checkerboard template to calculate intrinsic and extrinsic parameter matrices and correct lens distortion. Radiometric calibration establishes grayscale response curves by photographing a standard whiteboard to ensure image color fidelity. In practical implementation, multi-scale decomposition of wavelet transform is performed in the frequency domain. Fast Fourier Transform accelerates convolution calculations, and the decomposed sub-band data is stored in different memory regions. Low-frequency sub-bands are preserved using lossless compression, while high-frequency sub-bands use lossy compression to balance accuracy and bandwidth. In practical implementation, the calculation optimization of the gray-level co-occurrence matrix uses integral image technology. Image integral values ​​are pre-calculated to accelerate region statistics, and extracted texture parameters are smoothed and filtered to remove isolated noise points. Parameter values ​​are quantized and stored as floating-point numbers. In practical implementation, gradient calculation of the Canny edge detection operator uses Sobel operator convolution. Gradient directions are discretized into eight directions, non-maximum suppression is used to compare neighborhood gradient magnitudes, and double threshold connections are used to traverse weak edge points through depth-first search. In practical implementation, the blob detection algorithm sorts the eigenvalues ​​of the Hessian matrix, selects the region corresponding to the largest eigenvalue as a candidate blob, applies morphological opening operations to remove small noise, and records the blob center coordinates and radius as features. In practical implementation, the minimum and maximum values ​​of feature normalization are obtained statistically from the training set, and the pre-calculated range is directly applied during online processing. The feature merging process is achieved through vector concatenation, and the generated high-dimensional feature vectors may reach hundreds of dimensions. In practical implementation, the encapsulation protocol of the feature vector sequence includes a checksum field to detect transmission errors. Data packets are sent asynchronously through a message queue, and the consumer thread of the processing module parses them in real time. In practical implementation, the projection matrix of principal component analysis is trained based on historical data and updated periodically to adapt to distribution changes. The dimensionality of the reduced features is typically controlled within tens of dimensions to ensure the efficiency of subsequent algorithms.

[0062] In practical implementation, the trigger signal of the high-resolution vision sensor is linked to the production line PLC controller, ensuring precise timing of data acquisition to match the moment the printed product enters the field of view. A double-buffering mechanism is used for image frame buffering to avoid acquisition and processing conflicts. The number of layers in the wavelet transform processing unit is chosen to balance computational overhead and detail preservation; three layers are suitable for most printed product textures, while four layers can be added for particularly fine patterns. The choice of wavelet basis function affects feature representation capability, with the Daubechies wavelet providing a good balance. For global texture feature extraction in the low-frequency baseband layer, the window size of the gray-level co-occurrence matrix is ​​set to 16x16 pixels, with a sliding step of 8 pixels, covering the entire image. Bilinear interpolation is used for parameter calculation to improve accuracy. For edge and blob feature extraction in the high-frequency detail layer, the Gaussian filter standard deviation of the Canny operator is set to 1.0 pixel, and the scale parameters for blob detection are 1, 2, and 3 pixels for multi-resolution fusion. In practical implementation, after feature parameter normalization, the merging order follows feature type priority, prioritizing texture parameters before edge parameters. Within the same type, parameters are ordered by importance, and the timestamps of the feature vector sequence are synchronized with physical time. Data packet transmission employs a high-speed serial protocol, such as CoaXPress, with bandwidth sufficient for real-time streaming. Receiver buffer management prevents data loss. The principal component analysis training set covers various normal and defective samples, ensuring the generalization ability of the projection matrix. During the online compression stage, the dot product of the feature vector and the projection matrix yields a low-dimensional representation.

[0063] In practical implementation, the high-resolution visual sensor layout employs multi-camera stitching to cover the full width of the printed material. Adjacent cameras overlap by 10%, achieving seamless connection through image registration. The acquisition frame rate adaptively adjusts based on the conveyor belt speed, reaching a maximum of 100 frames per second. The wavelet transform processing unit's software implementation is based on the OpenCV library, calling discrete wavelet transform functions to convert the decomposed sub-band data into floating-point types for processing, avoiding precision loss. The gray-level co-occurrence matrix is ​​calculated separately for each color channel, with the final texture parameters taken as the average of each channel. For color images, they are first converted to grayscale or each channel is processed. The dual thresholds of the Canny edge detection operator are automatically determined using the Otsu method, adapting to different image contrasts. The Hessian matrix calculation for blob detection uses Gaussian differential template convolution. The feature normalization range parameters can be learned online and dynamically updated based on recent feature distributions. The predefined rules for the merging order support external configuration. In practice, the feature vector sequence is stored using a circular buffer, with older data automatically overwritten. The sequence length is fixed at the most recent 100 frames, representing the short-term production state. The dimensionality reduction effect of principal component analysis is monitored through reconstruction error assessment, periodic checks of the retained information ratio, and retraining of the projection matrix when necessary.

[0064] In practical implementation, the maintenance of the high-resolution vision sensor includes regular lens cleaning and light source calibration to ensure long-term stability. The acquisition software records quality indicators for each frame of image, such as signal-to-noise ratio. The wavelet transform processing unit utilizes a multi-core CPU for parallel processing, dividing the image into blocks to improve throughput. The decomposition results are displayed in real-time for monitoring. After parameter extraction from the gray-level co-occurrence matrix, outlier detection is performed to remove obvious erroneous data, and parameter sequence smoothing reduces fluctuations. For high-frequency detail subbands, noise suppression is performed first, followed by median filtering preprocessing, and then edge and blob detection to improve feature quality. Before feature merging, correlation analysis is conducted on each parameter to remove redundant features, optimize vector dimensions, and add metadata such as image size to the merged vector. The transmission of feature vector sequences is encrypted to protect intellectual property rights, using the AES algorithm to encrypt data packets, and the decryption key is securely stored. Principal component analysis training data is evenly sampled to represent all defect types, and the projection matrix is ​​serialized and saved, supporting fast loading.

[0065] Example 2: See Figure 3In practice, historical defect data is derived from a library of labeled defect images accumulated over long-term production processes. This library is stored in a distributed file system. Each sample image contains high-resolution RGB data and a corresponding annotation file in JSON format. The annotation file includes defect type labels (e.g., scratches, ink spots, misregistration) and the precise bounding box coordinates of the defect in the image. The bounding box is represented by the coordinates of its top-left vertex and its width and height. The generator structure of the generative adversarial network is designed as a fully connected network. The generator input is a 100-dimensional random noise vector sampled from a standard normal distribution. The generator network contains three hidden layers with 512, 256, and 128 neurons per layer, respectively. The number of neurons in the output layer strictly matches the dimension of the actual defect feature vector. The activation function, except for the Tanh function used in the output layer, is the ReLU function. The discriminator in the generative adversarial network (GAN) employs a convolutional neural network (CNN) structure. The discriminator's input is either a real defect feature vector or a defect feature vector synthesized by the generator. The vector data is first reconstructed into a two-dimensional feature map. The discriminator contains four convolutional layers, each with a 3x3 kernel size and a stride of 2 for downsampling. Each layer is followed by a LeakyReLU activation function, and finally, a fully connected layer outputs a single scalar. This scalar, after activation by the sigmoid function, represents the probability that the input is a real defect feature. During training, the generator strives to generate realistic defect features sufficient to deceive the discriminator, while the discriminator attempts to distinguish between real and synthesized features. Training is iterative; in each iteration, the discriminator parameters are updated first, followed by the generator parameters. Through this adversarial training, the generator can ultimately output a dynamic defect template set covering various defect variants.

[0066] In practical implementation, the dynamic defect template set is stored in the defect feature prototype library in matrix form. Each row of the matrix represents a defect template vector. The defect feature prototype library is managed using a relational database and an index table is established. The index table maps each template vector to a specific defect type and records the template generation time and confidence score. A dedicated convolutional neural network is designed to encode the feature vector sequence of the real-time image stream. This convolutional neural network contains four convolutional layers: the first convolutional layer uses 64 kernels of size 3x3, the second convolutional layer uses 128 kernels of size 3x3, the third convolutional layer uses 256 kernels of size 3x3, and the fourth convolutional layer uses 512 kernels of size 3x3. Each convolutional layer is followed by a ReLU activation function and a max pooling layer with a 2x2 window size and a stride of 2. After the last convolutional layer, a global average pooling layer is connected to compress the feature map into a fixed-length low-dimensional feature vector. The Euclidean distance between the encoded feature vector and each template vector in the defect feature prototype library is calculated. The Euclidean distance is calculated as the square root of the sum of the squares of the differences between corresponding elements of the vectors. All distance values ​​form a similarity matrix, where the rows of the similarity matrix correspond to the real-time feature vectors and the columns correspond to the template vectors in the defect feature prototype library. A learnable similarity threshold is set, with the initial value determined based on the validation set accuracy and dynamically adjusted during operation based on feedback. Each element in the similarity matrix is ​​compared with the similarity threshold; elements below the threshold are set to 1, indicating a successful match, while elements above the threshold are set to 0, generating a binary defect matching index matrix.

[0067] In some embodiments, the construction of the historical defect image database includes a data augmentation step, performing operations such as rotation, scaling, brightness adjustment, and noise addition on the original defect images to expand the diversity of the dataset. The training of the generative adversarial network uses the Adam optimizer with a learning rate set to 0.0002, momentum parameters β1 set to 0.5, β2 set to 0.999, and training batch size set to 64. In some embodiments, the training of the convolutional neural network adopts a phased strategy, first pre-training on a large natural image dataset, and then fine-tuning on the historical defect image database. The network weights are initialized using the He normal initialization method, and the backpropagation error uses the cross-entropy loss function. Optionally, the update strategy of the defect feature prototype database can be set to periodic updates or triggered updates. Periodic updates retrain the generative adversarial network at fixed time intervals, while triggered updates are initiated when a new defect pattern is detected. The similarity threshold can be learned through a reinforcement learning algorithm, and the threshold size is adjusted based on the accuracy feedback of the detection results. Optionally, the calculation of Euclidean distance can be replaced with cosine similarity or Mahalanobis distance to adapt to different feature distribution characteristics. Binarization can employ a soft thresholding method, generating probability values ​​between 0 and 1 instead of hard decisions. It is understood that the training stability of generative adversarial networks is improved through gradient penalty techniques to prevent pattern collapse, and the global average pooling layer of convolutional neural networks helps reduce overfitting and enhance the model's generalization ability. It is also understood that the generation of the defect matching index matrix is ​​performed in real time, and computational efficiency is optimized through matrix operation libraries such as BLAS, supporting GPU-accelerated computation to ensure that the real-time requirements of the production line are met.

[0068] In practical implementation, the management of historical defect data includes version control. After each model update, old versions of data and models are retained for rollback and comparative analysis. In addition to bounding boxes, annotation files can also include pixel-level segmentation masks to provide more accurate defect location information. The choice of dimension for the generator's input noise vector in the generative adversarial network (GAN) affects the diversity of generated templates; 100 dimensions are a common choice balancing complexity and expressive power. The design of the generator network's layers needs to consider training cost and generation quality. The architecture of the discriminator's convolutional neural network needs to match the generator's capabilities to avoid imbalance caused by one side being too strong during training. A gradient penalty term is added to the discriminator's loss function to improve training stability. The dynamic defect template set is stored as a floating-point array, serialized, and stored in a database, while also being cached in memory to accelerate similarity calculation. The input layer of the dedicated convolutional neural network needs to adapt to the dimension of the feature vector sequence, possibly by adding fully connected layers for dimensionality transformation, and gradually increasing the number of convolutional kernels to extract hierarchical features. The calculation of Euclidean distance involves a large number of floating-point operations. Single-precision floating-point numbers are used to reduce computational and storage overhead. The dimension of the similarity matrix is ​​determined by the number of real-time feature vectors and the number of templates. The learnable similarity threshold adjustment algorithm can be dynamically adjusted based on historical accuracy curves using the PID controller principle. The binarized defect matching index matrix is ​​subsequently used for cluster analysis and defect localization.

[0069] In practical implementation, the access interface of the historical defect image library provides query functions based on defect type, severity, production time, etc. The training process of the generative adversarial network (GAN) monitors the quality of generated images and the accuracy of the discriminator. Training convergence is determined when the discriminator accuracy remains below a specific threshold for an extended period. When the generator synthesizes defect feature vectors, it can smoothly transition between different defect features by interpolating noise vectors, observing the continuity in the feature space. The discriminator is trained using batches of a mixture of real defect feature vectors and generator-synthesized defect feature vectors. The index table of the defect feature prototype library supports fuzzy queries. When a new feature vector is similar to multiple templates, it returns the most likely defect types and their probabilities. The feature vector sequence before encoding by the convolutional neural network may need to be padded or truncated to ensure a fixed length. Batch normalization layers can be added after each convolutional layer of the convolutional neural network to accelerate training and improve performance. The fixed-length vector output by the global average pooling layer has a dimension of 512, representing high-level semantic features. Before Euclidean distance calculation, the feature vectors and template vectors undergo L2 normalization to make the distance calculation focus more on direction rather than magnitude. The similarity matrix can be further row-normalized to facilitate horizontal comparison of similarity with different templates. The learnable mechanism of the similarity threshold is implemented through an online learning algorithm, which adjusts the threshold based on feedback signals from recent detection results. Positions with a value of 1 in the binarized defect matching index matrix identify potential defect regions.

[0070] In practical implementation, the historical defect data annotation process employs multi-user annotation and expert arbitration mechanisms to ensure annotation consistency. Training data for the generative adversarial network (GAN) needs balanced sampling to avoid insufficient samples of certain defect types leading to generation bias. The backpropagation error of the generator network originates from the discriminator's judgment results; gradients flow through the discriminator and back to the generator. Initially, the generator's quality is poor, requiring patient adjustment of hyperparameters. The depth and width of the discriminator's convolutional neural network need to match the generator's complexity; an overly simple discriminator cannot provide effective learning signals. The dynamic defect template set is periodically cleaned of low-confidence or outdated templates to maintain the library's timeliness. The training of the dedicated convolutional neural network uses stochastic gradient descent, with a cosine annealing learning rate. A central loss function can be added to the loss function to increase inter-class distance and reduce intra-class distance. Real-time feature vector sequences undergo moving average filtering before being input into the encoding network to smooth instantaneous fluctuations. The distance calculation between the encoded feature vector and the template vector can be parallelized. The similarity matrix is ​​generated at the same frequency as the image acquisition frame rate. Each new frame calculates the similarity with all templates. The adjustment range of the learnable similarity threshold has upper and lower limits to prevent excessive oscillation. The binarized result of the defect matching index matrix is ​​fed into subsequent processes for connected component analysis to preliminarily determine candidate defect regions.

[0071] In practical implementation, a historical defect database is established to trace lineage, recording the production batch and equipment parameters of each defect image. The training environment for the generative adversarial network is configured with a high-performance GPU computing card, and the training cycle typically requires hundreds of iterations. After the generator generates dynamic defect templates, they must be screened by a quality assessment module to remove unreasonable or ambiguous templates. The training data for the discriminator is randomly sampled from the real database and the generated set each time. The access interface for the defect feature prototype library provides a full set of operations for adding, deleting, modifying, and querying, supports template clustering analysis to remove redundancy, and the dedicated convolutional neural network architecture supports variable-length inputs, processing sequence information through recurrent neural networks or self-attention mechanisms. The convolutional layers use the same padding method to maintain the feature map space size. Pooling layer operations reduce the number of parameters and increase translation invariance. A global average pooling layer can be directly connected to a softmax layer for classification tasks. The Euclidean distance calculation module optimizes cache usage to avoid repeatedly reading template data. The similarity matrix is ​​stored in a sparse matrix format to save space because most similarity values ​​are high. The adaptive adjustment algorithm for the similarity threshold considers the overall quality level of the production line; when quality is stable, the threshold is increased to reduce false alarms, and when quality fluctuates, the threshold is decreased to avoid missed detections. The binarized defect matching index matrix, combined with morphological post-processing, eliminates isolated noise points and forms a coherent defect area indication.

[0072] Example 3: In specific implementation, the defect matching index changes over time to form a time series. The time series data is cached using a sliding window of fixed length, with the window size set to the 100 most recent sampling points. Each sampling point corresponds to a defect matching index value. The standard deviation of the defect matching index sequence within the sliding window is calculated as the index volatility, which reflects the short-term stability of defect detection results during production. Based on the calculated index volatility, the sensitivity parameter and classification threshold for defect detection are dynamically adjusted. The adjustment rule is that when the index volatility exceeds a preset upper limit, the sensitivity parameter for defect detection is reduced proportionally, i.e., the similarity threshold is relaxed, to reduce false alarms caused by production fluctuations. When the index volatility is below a preset lower limit, the sensitivity parameter for defect detection is increased proportionally, i.e., the similarity threshold is tightened, to enhance the detection capability of subtle defects. The adjustment direction of the classification threshold is synchronized with that of the sensitivity parameter. Density clustering algorithm is used to perform cluster analysis on all matching index data within the current sliding window. The density clustering algorithm examines the density in the neighborhood of each data point, connects points with connected densities to form clusters, and filters out noise points with densities below the minimum density threshold. Each generated cluster represents a potential defect area. The set of all clusters constitutes a defect probability distribution map, which is a two-dimensional probability field whose coordinates correspond to the coordinates of the printed material surface. The gray value of each pixel in the map represents the probability estimate of the presence of a defect at that location.

[0073] In practical implementation, the defect probability distribution map is mapped to the physical coordinate space of the printed product. The mapping relationship is determined by pre-calibrated camera parameters, including intrinsic and extrinsic matrices. The intrinsic matrix describes the camera's internal geometry and optical characteristics, while the extrinsic matrix describes the transformation relationship between the camera coordinate system and the world coordinate system. A region growing algorithm is used to identify defect clusters. This algorithm uses points in the defect probability distribution map that exceed a preset probability threshold as seed points (typically set to 0.7). The algorithm checks the defect probability values ​​of points within the eight neighborhoods of each seed point. If a neighboring point's defect probability value exceeds a lower growth threshold (typically set to 0.3), it is merged into the current region. This merging process iterates until no new points can be merged, thus forming connected defect clusters. The arithmetic mean of the defect probability values ​​of all points within each defect cluster is calculated as the defect density of that cluster. The defect density calculation formula is:

[0074]

[0075] in: Indicates the first Defect density in a defect cluster region Indicates the first The total number of pixels contained in a defect cluster region. Indicates the first [unit] within the defect cluster region Defect probability value per pixel This indicates a summation operation. Based on the calculated defect density value of each defect cluster region, all defect cluster regions are sorted from high to low according to their defect density values ​​to determine the processing priority of the subsequent local re-inspection mechanism. Defect cluster regions with higher defect densities have higher priority.

[0076] In some embodiments, the calculation of index volatility can use the more robust median absolute deviation instead of standard deviation to reduce the interference of outliers. The size of the sliding window can be dynamically adjusted according to the production rhythm; the window can be reduced to improve response speed during high-speed production and expanded to increase stability during low-speed production. In some embodiments, the generation of the defect probability distribution map can use a Gaussian mixture model to estimate the density of the matching index data. Each Gaussian component corresponds to a potential defect pattern. The seed point selection strategy in the region growing algorithm can be optimized to start growing from local maxima points to avoid blurring of region boundaries. In some embodiments, the calculation of defect density can introduce area weights to prevent small-area, high-probability regions from receiving excessive priority. The sorting strategy for defect cluster regions can be optimized by combining region area and defect density for multi-objective optimization. Optionally, the dynamic adjustment of sensitivity parameters and classification thresholds can be implemented using a fuzzy logic controller. The input is the index volatility and historical false alarm rate, and the output is the parameter adjustment amount, making the adjustment process smoother. It is understood that the mapping accuracy from the defect probability distribution map to physical coordinates directly affects the accuracy of defect localization, requiring high-precision camera calibration and coordinate system transformation. The growth threshold and probability threshold of the region growing algorithm need to be determined experimentally based on specific application scenarios. It is understandable that defect density calculation is a core step in identifying critical defect areas. Accurate density assessment helps to allocate inspection resources rationally and prioritize high-risk areas.

[0077] In practical implementation, the time-series defect matching index data is stored in a circular buffer, with new data overwriting old data. The index volatility is calculated using the Welford online algorithm, which can update the mean and variance point by point without storing all historical data. The sensitivity parameter for defect detection is adjusted proportionally to the degree to which the index volatility deviates from the normal range; the proportionality coefficient is determined experimentally. The adjustment range of the classification threshold has hard upper and lower limits to prevent excessive parameter drift from causing detection failure. The density clustering algorithm uses the DBSCAN algorithm. The DBSCAN algorithm requires setting a neighborhood radius parameter and a minimum number of points parameter. The neighborhood radius parameter affects the granularity of the clusters, and the minimum number of points parameter determines the minimum number of points required to form a cluster. During the generation of the defect probability distribution map, the clustering results are probabilistically calibrated so that the probability value of each point reflects its posterior probability of belonging to a real defect. The coordinate mapping process involves the transformation from image pixel coordinates to camera coordinates, and the transformation from camera coordinates to world coordinates. The world coordinate system uses a fixed corner point of the printed material as its origin. The region growing algorithm uses a queue data structure to manage the neighborhood points to be processed. The growing process records the contour point set of each region, and the region merging condition can consider the gradient information of the probability values. After the defect density is calculated, the density value is normalized to fall between 0 and 1, facilitating cross-region comparison and threshold setting. The sorting results of defect clustered regions are stored in a priority queue, with high-priority regions being scheduled for re-inspection first.

[0078] In practical implementation, the preset upper and lower limits of the indicator volatility are obtained based on statistical data from historical stable production phases, typically using plus or minus two standard deviations of the mean as the boundary. The actual adjustment of the sensitivity parameter is reflected in the numerical modification of the similarity threshold. Before processing by the density clustering algorithm, the matching indicator data is standardized to have a mean of 0 and a variance of 1, avoiding the influence of dimensions on the clustering results. The clusters obtained from clustering are assigned labels to initialize the defect probability distribution map. The probability values ​​of the defect probability distribution map are assigned based on the density of the cluster to which a point belongs and the average similarity of points within the cluster. Clusters with higher density and greater average similarity have higher probability values ​​for their member points. During physical coordinate mapping, image distortion is corrected using a Brownian distortion model to correct radial and tangential distortion, ensuring the geometric accuracy of the coordinate mapping. The seed point selection for the region growing algorithm avoids excessive density. Non-maximum suppression is first applied to the probability map, and growth is only initiated at local maxima. Overlapping regions are allowed during the growth process, followed by subsequent fusion. When calculating defect density, for irregularly shaped regions, pixel counts are used to calculate the area, and the probability average is used as the density measure. The sorting algorithm uses quicksort to arrange regions in descending order of density. The priority queue management supports dynamic updates; when a new region is identified or the density of an old region changes, the queue order is readjusted.

[0079] In practical implementation, the time series analysis module continuously monitors the changing trend of indicator volatility. If the indicator volatility continues to rise, it may indicate a deterioration in production status, requiring the triggering of an early warning. Adjustments to sensitivity parameters are recorded in the log to trace the impact of parameter changes on detection effectiveness. The parameters for the density clustering algorithm are selected through silhouette coefficient evaluation, choosing the parameter combination that maximizes the silhouette coefficient. After the defect probability distribution map is generated, Gaussian filtering is applied for smoothing to eliminate drastic fluctuations in probability values. The accuracy of coordinate mapping is verified by photographing a calibration board with known coordinates, and the reprojection error is calculated to evaluate mapping precision. The growth rules of the region growing algorithm can be extended to consider the continuity of probability values, rather than just threshold comparison. The geometric attributes of defect clustering regions, such as area, perimeter, and density, are simultaneously calculated as auxiliary decision-making information. The defect density ranking results are visualized to help operators intuitively understand the defect distribution. The time performance of the entire processing flow needs to meet real-time requirements; the algorithm complexity has been optimized, and key steps are accelerated using parallel computing.

[0080] In implementation, the sliding window's step size is set to one sampling point for point-by-point updates. The calculation cycle for the index volatility is synchronized with the image acquisition frame rate to ensure real-time response. Adjustment commands for sensitivity parameters and classification thresholds are sent to the core detection module via a message queue, achieving decoupling and asynchronous processing. The distance metric used in the density clustering algorithm is Euclidean distance, suitable for matching the numerical characteristics of the index data. The defect probability distribution map's data structure uses a two-dimensional array, with the array size matching the number of grids on the printed surface, and each grid cell storing a probability value. Camera calibration parameters are periodically checked to prevent parameter drift due to equipment vibration. The implementation of the region growing algorithm avoids excessive recursion depth and uses an iterative approach to prevent stack overflow.

[0081] In practical implementation, the calculation of index volatility considers the autocorrelation of the sequence. For non-stationary sequences, differencing is performed first. A lag is introduced during the sensitivity parameter adjustment process to prevent frequent parameter oscillations. When processing large-scale data, the density clustering algorithm adopts a partitioning strategy, first dividing the data space into grids, and then clustering within the grids. The probability values ​​of the defect probability distribution map are normalized to the range of 0-255 for easy graphical display and storage. Bilinear interpolation is used for physical coordinate mapping to improve the accuracy of coordinate points. The growth direction of the region growing algorithm can be weighted, prioritizing growth in directions with larger probability gradients. After calculating the defect density in defect clusters, regions with densities exceeding the action threshold are marked. The action threshold is dynamically set according to quality requirements. The sorted list of regions is appended with timestamps and region IDs to ensure the traceability of the processing order.

[0082] See Figure 4This chart demonstrates the real-time monitoring capabilities and stability analysis effectiveness of the defect detection system during the production process. The chart uses a dual Y-axis to present the time-series changes of the defect matching index and the corresponding index volatility. The blue curve reflects the dynamic trend of the defect detection signal over production time, while the red curve shows the short-term stability index calculated based on a sliding window. This visualization clearly demonstrates the stability and reliability of defect detection during production. Low volatility indicates stable system operation, while a sudden increase in volatility may indicate changes in production status or potential quality problems. The chart also includes key statistical information, providing intuitive data support for production quality management personnel and helping to promptly identify and address production anomalies.

[0083] Example 4: In specific implementation, for clustered areas where the defect density exceeds a preset threshold, the system automatically initiates a local re-inspection mechanism. The preset threshold is dynamically configured according to product quality level requirements. The local re-inspection mechanism sends instructions to the control center, including the physical coordinates, size, and priority information of the clustered area. The control center dispatches a backup high-resolution vision sensor to move above the clustered area for fixed-point focusing scanning. The backup high-resolution vision sensor is mounted on a high-precision translation stage driven by a servo motor, achieving micron-level positioning accuracy. The re-inspection process involves multiple scans, preset to three to five times. Each scan acquires images from slightly different angles or lighting conditions. Angle changes are achieved by controlling the gimbal deflection, and lighting changes are achieved by adjusting the brightness and color temperature of the ring LED light source to eliminate accidental interference. Comparing the results of multiple scans, only defects that are consistently detected in the vast majority of scans are ultimately confirmed. The confirmation rule is that only defects identified in at least two scans are considered true defects, effectively improving the reliability of the detection results. Once the defect data has been verified and confirmed, including its multi-angle images, extracted feature vectors, defect type, and precise coordinates, it is added to the historical defect database. The historical defect database adopts a time-series database structure, which supports efficient storage and querying by time range.

[0084] In the specific implementation, referring to Table 1, the update of the historical defect database triggers the retraining process of the defect feature prototype library. The generative adversarial network (GAN) uses the expanded historical defect database for incremental training. Incremental training is performed based on the pre-trained model weights, without completely restarting the training, thus incorporating newly confirmed defect features into the dynamic defect template set. Next, the updated defect feature prototype library is used to optimize the convolutional neural network used for encoding. A batch of training data is randomly sampled from the updated defect feature prototype library. The training data includes feature vectors and corresponding defect type labels, with a batch size of 32. Forward propagation obtains the predicted output through the convolutional neural network, and the cross-entropy loss function between the predicted output and the true label is calculated. The cross-entropy loss function measures the difference between the predicted probability distribution and the true distribution. The network weights are adjusted using the gradient backpropagation algorithm. The gradient backpropagation algorithm calculates the partial derivatives of the loss function with respect to the weight parameters of each layer of the convolutional neural network, i.e., the gradient. The adaptive moment estimation algorithm is used to update the weights. The adaptive moment estimation algorithm combines first-order moment estimation and second-order moment estimation, which can adaptively adjust the learning rate of each parameter and reduce the loss value along the gradient direction. The optimization process is carried out iteratively, with different batches of data used in each iteration, until the loss function converges to a stable state on the validation set. The criterion for judging the stable state is that the change in the validation set loss is less than the preset tolerance for ten consecutive iterations, indicating that the network weight parameters have been fully optimized and have a stronger defect feature encoding capability.

[0085] Table 1: Local Re-detection Mechanism Scheduling Parameter Configuration Table

[0086] Defect density range Re-examination level Preset threshold Number of sensors to schedule Scan rounds Range of light variation Angle variation range 0.7≤D<0.8 Level 1 0.7 1 3 5000K±200K ±2° 0.8≤D<0.9 Level 2 0.8 2 4 5000K±300K ±3° D≥0.9 Level 3 0.9 3 5 5000K±500K ±5°

[0087] In some embodiments, the triggering condition for the local re-detection mechanism can be set as a combination of multiple defect density thresholds, such as requiring both the regional defect density and the total regional area to exceed a threshold simultaneously. The scheduling strategy for the backup high-resolution vision sensor can adopt a nearest-first principle, selecting the idle sensor closest to the target aggregation area to perform the task. Optionally, image registration during multi-round scanning uses feature-based registration algorithms, such as SIFT feature matching, to ensure accurate alignment of images acquired under different angles and lighting conditions. Defect consistency verification can employ a voting mechanism, confirming defects detected in more than half of the rounds. Optionally, the update operation of the historical defect database is transactional, ensuring the atomicity and consistency of data addition. The incremental training cycle of the generative adversarial network can be set to be timed, for example, automatically starting training after every 100 new defect samples are collected. It is understood that the optimization process of the convolutional neural network needs to monitor the training loss and validation loss curves to prevent overfitting. When the validation loss starts to rise while the training loss continues to decline, training should be stopped early.

[0088] In practical implementation, the generation of local re-inspection instructions includes safety checks to ensure that the coordinates of the aggregation area are within the effective working range of the sensor. The movement path of the high-precision translation stage is optimized by a motion planning algorithm to avoid collisions with other equipment on the production line. The sequence of multi-round scans is managed by a dedicated scan controller. The controller sets the gimbal angle and light source parameters in a predetermined order and triggers sensor exposure. The images obtained from each round of scanning are immediately preprocessed and processed by the defect detection algorithm. The defect consistency verification module maintains an intermediate result cache, recording the defect detection results at each pixel position in each round of scanning, and finally performs logical AND / OR voting operations to obtain the confirmation result. Before the verified defect data is stored in the historical defect database, it needs to be reviewed by quality inspectors or an expert system. Data that passes the review is marked as high-quality training samples. The retraining process of the defect feature prototype library is executed on a dedicated model training server. The training process consumes a lot of computing resources and is usually scheduled during periods of low system load. The weight optimization of the convolutional neural network uses the stochastic gradient descent algorithm, and the learning rate adopts a step-decreasing strategy. After a certain number of iterations, the learning rate is reduced to half of its original value to facilitate convergence to a deeper local minimum. In addition to observing changes in the loss value, the convergence of the loss function can also be judged by monitoring the classification accuracy on the validation set. When the accuracy no longer improves significantly, the model is considered to have converged.

[0089] In practical implementation, the status of the backup high-resolution vision sensors is monitored in real time by a resource manager. The resource manager maintains a sensor status table, recording the location, working status, and current task of each sensor. When a local re-examination instruction is received, the resource manager allocates appropriate sensor resources based on the re-examination level and the sensor's available status. In the multi-round scanning image acquisition sequence, the changing order of angle and illumination parameters is designed using the Latin hypercube sampling method to ensure effective coverage of the parameter space and reduce the correlation between parameters. The specific logic of defect consistency verification is as follows: for each candidate defect point within the clustered area, the number of times it is marked as a defect in multiple rounds of scanning is counted. If the number is greater than or equal to a preset confirmation threshold (e.g., 60% of the total number of rounds), the point is finally confirmed as a defect point. The historical defect database generates a globally unique identifier for each newly added defect sample and records metadata such as the sample's source, confirmation time, and confirmation round. The incremental training process of the generative adversarial network saves multiple intermediate checkpoints. If the training process is unexpectedly interrupted, training can be resumed from the most recent checkpoint to avoid loss of results. When updating the weights of a convolutional neural network, gradient clipping is used to prevent gradient explosion. An upper limit is set for the gradient norm. If the gradient norm exceeds the upper limit, the magnitude of the gradient vector is reduced proportionally.

[0090] In practice, the execution of local re-inspection tasks has a timeout mechanism. If the sensor fails to reach the designated location or complete the scan within the predetermined time, the task will be marked as a failure and an alarm will be triggered. Simultaneously, other backup sensors will be attempted to be scheduled. Before comparing defects, the image sequences obtained from multiple scans need to undergo image registration and brightness normalization to eliminate image differences caused by changes in viewing angle and illumination, ensuring the fairness of the comparison. The results of defect consistency verification generate a re-inspection report, which details the confirmed defect list, the confirmation round for each defect, location changes, and confidence scores. Detailed logs are kept for updating the historical defect database, including data changes, operator information, and timestamps, meeting audit trail requirements. The retraining process of the defect feature prototype library evaluates the performance of the new model on an independent test set. Only versions that outperform the old model are deployed to the production environment; otherwise, the old version is rolled back. The optimization process of the convolutional neural network can use different loss functions, such as the focus loss function. The focus loss function can alleviate the class imbalance problem by adjusting the modulation factor to make the model focus more on difficult-to-classify samples. The number of iterations in the optimization process is typically set to several thousand to tens of thousands, depending on the size of the training dataset and the complexity of the model. The training process is run on a server equipped with GPUs to accelerate computation.

[0091] In practice, the overall response time of the local re-inspection mechanism is strictly monitored. The response time, from issuing the re-inspection instruction to finally issuing the re-inspection report, must be within the time window allowed by the production cycle; otherwise, it will affect the normal flow rate of the production line. During multi-round scanning, the image quality of each round of scanning is evaluated in real time. Evaluation indicators include image sharpness, contrast, and noise level. If the image quality does not meet the standards, the round of scanning will be automatically retried. For defect areas with blurred boundaries or low signal-to-noise ratios, the defect consistency verification process requires additional rounds of scanning to obtain more evidence until a clear judgment can be made. The historical defect database uses a database sharding and table partitioning strategy to store massive amounts of defect data. Recently frequently accessed hot data is stored on high-speed solid-state drives, while older cold data is archived in an object storage system. Version management of the defect feature prototype library uses semantic version numbers. The version number format is major version number.minor version number.revision number. A major version number update represents a major change in the model structure or interface, a minor version number update represents the addition of new features or defect templates, and a revision number update represents defect fixes. After the convolutional neural network is optimized, the new weight parameters are updated to the online detection system through a blue-green deployment method. First, the new weights are deployed to a few servers for a canary release. After a period of observation to confirm stability, the old weights are then fully replaced to achieve a smooth upgrade.

[0092] See Figure 5This study validated the practical application effect of the local re-inspection mechanism in the defect detection system. A histogram of defect density distribution was used to illustrate the density statistical characteristics of all inspection areas. Different colored threshold lines clearly delineated the three re-inspection levels, providing a clear basis for automated re-inspection decisions. The red fitted curve showed the overall distribution pattern of defect density, helping to understand the distribution pattern of defects on the surface of printed materials. The statistical information area in the chart detailed key data indicators, including the total number of areas, average density value, and number of high-density areas. These data are crucial for optimizing the allocation of inspection resources and improving inspection efficiency. This density distribution-based prioritization mechanism ensures that limited inspection resources are prioritized for the most critical high-risk areas, thereby improving overall inspection efficiency while maintaining inspection quality.

[0093] Example 5: In the specific implementation, during the end-to-end quality consistency check stage, the system retrieves the corresponding reference image from the standard image library based on the model of the currently produced color-printed product, such as "Pharmaceutical Packaging Box - Model A". The standard image library is a relational database, storing fields including color-printed product model, version number, reference image data, image acquisition conditions, and entry timestamp. The reference image is a pre-acquired, rigorously verified sample image of a qualified product. Acquisition conditions such as lighting and camera parameters are standardized. The system obtains the latest reference image data for the corresponding version by querying the product model bound to the current production work order. The feature vector of the reference image is extracted. The feature extraction process is completely consistent with the feature extraction process of the real-time image stream, using the same wavelet transform decomposition, the same texture and edge feature extraction algorithms, and the same feature normalization and merging rules to ensure that the real-time feature vector and the reference feature vector are in the same feature space and are comparable. Each frame of the real-time image stream feature vector is compared with the reference feature vector frame by frame, and the cosine similarity between the two is calculated as the deviation index. The cosine similarity calculates the difference between the two feature vectors in direction, ignoring their magnitude. The calculation formula is the dot product of the two vectors divided by the product of their respective magnitudes. The calculation result is in the range of [-1, 1]. The closer the value is to 1, the more similar the two are. Therefore, the closer the deviation index is to 0, the closer it is to the standard template and the lower the possibility of defects. The larger the deviation index, the greater the deviation and the higher the possibility of defects.

[0094] In practical implementation, based on the calculated deviation index, the system generates a structured defect detection report. The report uses XML or JSON format and details each identified defect, including its type, absolute coordinates on the printed surface, and severity level based on the deviation index. Severity levels are typically divided into three levels: for example, a deviation index in the range [0, 0.1) is defined as "minor," in the range [0.1, 0.3) as "moderate," and in the range [0.3, 1] as "severe." After the defect detection report is output, the system automatically enters the feedback control phase. Based on the overall severity distribution of defects in the report, the control feedback module dynamically adjusts key process parameters of the production line. For example, when the number of "severe" level defects (such as "ink spots") exceeds a threshold, the control feedback module sends a command to the printing press's inking system to appropriately reduce the pressure setting. When "moderate" or higher levels of "misregistration" defects occur frequently, the control feedback module sends a command to the servo driver of the transmission mechanism to fine-tune the positioning accuracy compensation parameters of the transmission mechanism. Meanwhile, all relevant data from this inspection, including raw image data, intermediate processing results, final defect detection reports, and adjusted process parameter records, were fully recorded in the quality log database. The quality log database adopts a time-series database structure, and each record contains a complete timestamp, equipment identifier, and production batch number.

[0095] In practical implementation, the retrieval operation of the standard image library needs to handle version control issues. Multiple versions of reference images may exist for the same printed product model due to design changes. The system automatically selects the correct reference image version based on the effective date of the production task to ensure the accuracy of the comparison benchmark. Feature vector extraction of the reference images is performed offline, calculated and stored upon the reference image being added to the library, avoiding delays introduced by online processing. The cosine similarity calculation between the real-time image stream feature vector and the reference feature vector undergoes vector normalization preprocessing, converting the feature vector into a unit vector, simplifying the dot product calculation to obtain the cosine value. The mapping relationship between the deviation index and severity level is achieved by querying a predefined configuration table. The configuration table supports customization according to the quality requirements of different products, and the definition of severity level can be directly linked to customer acceptance standards. The generation of defect detection reports is an asynchronous process. The report generation service retrieves detection result data from the message queue, assembles it into a complete report, stores it in the file system, and pushes it to the production execution system via an interface. The parameter adjustment strategy of the control feedback module is based on a preset rule base. The rule base contains mapping relationships between various defect types, severity level combinations, and process parameter adjustment amounts. Adjustment instructions are sent to the production line controller via industrial communication protocols such as OPCUA. The recording operations of the quality log database ensure transaction consistency, employ batch writing to improve performance, and the log data is indexed to support multi-dimensional query and analysis by time, equipment, batch, defect type, and other dimensions.

[0096] In some embodiments, the reference images in the standard image library can include images of qualified products from multiple different batches of the same model. The mean vector of the feature vectors of these qualified product images is calculated as the final reference benchmark to encompass normal minor fluctuations in the material. In some embodiments, the calculation of the deviation index can introduce a weighting factor, assigning higher weights to the feature vectors of specific key areas, so that the deviations in key areas contribute more to the overall deviation index. The severity level classification threshold can be adaptively adjusted based on statistical analysis of historical quality data. Optionally, the output format of the defect detection report can support formats that are easy for human readers, such as PDF. Thumbnails of defective areas can be embedded in the report to enhance readability. The parameter adjustment of the control feedback module can adopt a PID control algorithm, making gradual adjustments based on the defect trends of multiple consecutive production batches, rather than making a one-time large-scale adjustment. Optionally, the data in the quality log database can be used to generate statistical process control charts to monitor the capability indices of the production process in real time, such as Cp and Cpk values, providing a basis for preventive maintenance. The update of the standard image library can be subject to an approval process, ensuring that only images authorized by the quality department can be entered as comparison benchmarks. Optionally, cosine similarity calculation can be accelerated using GPUs to meet the stringent real-time requirements of high-speed production lines, and the deviation index calculation cycle can be synchronized with the production cycle to ensure that every product is evaluated. It is understandable that end-to-end quality consistency inspection is the final checkpoint in defect detection. Its comparison with standard images can identify anomalies that differ from known defect templates but deviate from the acceptable standards, effectively supplementing defect template-based detection.

[0097] In practice, the establishment of the standard image library requires strict quality control. The reference images must be selected from qualified samples that have been verified by multiple parties to ensure they represent the acceptable lower limit of quality. The feature vectors of the reference images need to be periodically reviewed to prevent baseline drift due to feature extraction algorithm upgrades. Comparison of the feature vectors of the real-time image stream and the reference image requires consideration of image registration. Before calculating cosine similarity, coarse registration is performed through feature point matching, followed by fine registration within the ROI region to ensure consistency in the comparison area. The deviation index calculation results undergo short-term smoothing filtering, such as using a moving average method, to eliminate occasional fluctuations caused by noise in single-frame images, making the reported defect trend more stable. The position coordinates in the defect detection report need to be compensated based on the relative speed of the camera and conveyor belt, accurately mapping the position in the image coordinate system to the stationary physical coordinate system of the printed product. Before issuing parameter adjustment commands, the control feedback module conducts simulations or small-scale trial runs to assess the potential impact of parameter adjustments on other quality indicators of the production line, avoiding secondary problems. The storage strategy for the quality log database includes regular backups and disaster recovery plans to ensure the long-term security and recoverability of quality traceability data. Access permissions to the database are strictly controlled according to roles.

[0098] In practical implementation, during the new product introduction phase, the standard image library may not yet store corresponding reference images. The system supports temporarily using the image of the first qualified product as a reference benchmark. After accumulating a certain number of qualified product data, a more robust benchmark reference image can be calculated and generated. During the feature vector comparison phase, if the real-time image suffers severe feature extraction failure due to occlusion, reflection, or other reasons, the system will mark that frame as invalid, not perform deviation index calculation, and may trigger a re-inspection mechanism for that workstation. The severity level classification can be dynamic. For example, in the initial stage of production, the severity threshold can be relaxed to quickly stabilize the process, while in the stable production stage, the threshold can be tightened to improve quality. After the defect detection report is pushed to the production execution system, the system will monitor the quality data of subsequent production batches, forming a closed-loop quality control cycle of adjustment-verification-re-adjustment. The data analysis module of the quality log database can automatically generate various quality reports, such as defect Pareto charts and trend charts, providing data support for quality improvement meetings.

[0099] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0100] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for detecting surface defects in green color printed products based on computer vision, characterized in that, The method includes: The system collects real-time image streams during the color printing production process, extracts feature layers from the images through multi-scale decomposition, and generates a sequence of feature vectors. Construct a prototype library of defect features, train a generative adversarial network based on historical defect data, and output a dynamic defect template set; The feature vector sequence of the real-time image stream is encoded using a convolutional neural network, and a similarity matrix with the dynamic defect template set is calculated to generate a defect matching index. Based on the defect matching index, the detection parameter thresholds are adaptively adjusted, and a defect probability distribution map is generated through cluster analysis. Spatiotemporal domain analysis was performed on the defect probability distribution map to identify defect clustering regions and calculate the defect density within these regions. Based on defect density, a local re-inspection mechanism is initiated to perform multiple rounds of scanning on the clustered areas and update the defect feature prototype library. Optimize the weight parameters of the convolutional neural network based on the updated defect feature prototype library; Perform a full-process quality consistency check, compare the deviation between the real-time image stream and the standard image library, and output a defect detection report; The spatiotemporal analysis of the defect probability distribution map to identify defect clustering regions includes: Map the defect probability distribution map onto the coordinate space of the printed surface and calculate the defect probability value at each coordinate point. A region growing algorithm is used to start from high-probability points and merge adjacent points to form clustered regions; Calculate the defect density of each cluster region and prioritize the regions based on the density values.

2. The method for detecting surface defects in green printed products based on computer vision according to claim 1, characterized in that, The process of acquiring real-time image streams during the color printing production process, and extracting image feature layers through multi-scale decomposition, includes: A high-resolution vision sensor is configured to capture continuous image frames on the surface of the printed material. Each image frame is processed by wavelet transform and decomposed into a low-frequency baseband layer and a high-frequency detail layer. Global texture features are extracted from the low-frequency baseband layer, and edge and blob features are extracted from the high-frequency detail layer. The global texture features and edge and blob features are merged to generate a feature vector sequence.

3. The method for detecting surface defects in green printed products based on computer vision according to claim 2, characterized in that, The construction of the defect feature prototype library and the training of the generative adversarial network based on historical defect data include: Collect historical defect image samples, label each sample, and generate defect type labels and location information; A generator using a generative adversarial network synthesizes defect feature vectors, a discriminator distinguishes between real defects and synthesized features, and the generator outputs a dynamic defect template set through iterative training. The dynamic defect template set is stored in the feature prototype library, and a mapping relationship between the template and the defect type is established.

4. The method for detecting surface defects in green color printed products based on computer vision according to claim 3, characterized in that, The process of encoding the feature vector sequence of the real-time image stream using a convolutional neural network and calculating the similarity matrix with the dynamic defect template set includes: Design a convolutional neural network structure containing multiple convolutional and pooling layers to reduce the dimensionality and encode the feature vector sequence; Calculate the Euclidean distance between the encoded feature vector and each template in the dynamic defect template set to generate a similarity matrix; Set a similarity threshold to convert the similarity matrix into a binary matching index.

5. The method for detecting surface defects in green printed products based on computer vision according to claim 4, characterized in that, The adaptive adjustment of detection parameter thresholds based on defect matching indicators includes: Analyze the time series changes of the matching indicators and calculate the volatility of the indicators; Based on the volatility of the indicators, the sensitivity parameters and classification thresholds for defect detection are dynamically adjusted. The matching indicators are grouped using a clustering algorithm to generate a defect probability distribution map and identify high-probability areas.

6. The method for detecting surface defects in green color printed products based on computer vision according to claim 5, characterized in that, The local re-examination mechanism based on defect density includes: For clustered areas where the defect density exceeds a preset threshold, additional visual sensors are deployed for focused scanning; Perform multiple rounds of image acquisition, compare the results of each round of scanning, and verify the consistency of defects; The verified defect data is added to the historical defect database, and the defect feature prototype library is updated.

7. The method for detecting surface defects in green color printed products based on computer vision according to claim 6, characterized in that, The optimization of the weight parameters of the convolutional neural network based on the updated defect feature prototype library includes: Training data is sampled from the updated defect feature prototype library, and the loss function of the convolutional neural network is calculated. The network weights are adjusted using the gradient backpropagation algorithm to minimize the loss function; Repeat the iterative optimization process until the network converges to a stable state.

8. The method for detecting surface defects in green printed products based on computer vision according to claim 7, characterized in that, The full-process quality consistency check includes: Retrieve reference images for the corresponding color printed product type from the standard image library and extract reference feature vectors; The feature vector sequence of the real-time image stream is compared frame by frame with the reference feature vector to calculate the deviation index; Based on the deviation index, a defect detection report is generated, which includes the defect type, location, and severity level.

9. The method for detecting surface defects in green printed products based on computer vision according to claim 8, characterized in that, After outputting the defect detection report, it also includes: The process parameters of the production line are automatically adjusted based on the severity level in the defect detection report. Defect data is recorded in the quality log for subsequent model retraining and system optimization.

Images