A high-precision printing defect detection method and system based on deep learning

Abstract

The application discloses a high-precision printing defect detection method and system based on deep learning, and belongs to the technical field of printing defect detection. The method comprises the following steps: enhancing a printing image by using a multi-objective sea squirt optimization algorithm; constructing a deep learning network model based on physical information constraints, including introducing a Lambertian reflection model and a white pixel percentage as a physical constraint embedded loss function by fusing a double-path parallel structure of a gated recurrent unit (GRU) and a visual ViT (visual transformer); training and optimizing the model by using a transfer learning method; and then processing the enhanced printing image to obtain a printing defect detection result. The application relies on a PINN-GRUViT model and a transfer learning technology to realize efficient defect identification in a small sample and strong noise industrial scene, and provides a reliable technical scheme for printing visual detection. Through image enhancement and a double-path parallel detection strategy, the accuracy and robustness of defect detection are improved, and high-precision printing defect detection is realized.

Description

Technical Field

[0001] This invention relates to the field of printing defect detection technology, and in particular to a high-precision printing defect detection and system based on deep learning. Background Technology

[0002] Traditional printing defect detection mainly relies on manual visual inspection or machine vision methods based on traditional image processing. Manual inspection is inefficient, subjective, and prone to fatigue, making it difficult to meet the real-time requirements of modern high-speed production lines. Traditional machine vision methods are usually based on fixed image processing algorithms (such as threshold segmentation and edge detection), which generally suffer from poor adaptability, high false alarm rates, and insufficient detection accuracy when faced with complex and varied printing patterns, uneven lighting, or subtle defects. In particular, traditional methods struggle to extract effective features for accurate identification of small and diverse defects such as scratches, ink spots, misregistration, and ghosting.

[0003] In recent years, deep learning technology has made breakthroughs in the field of image recognition, especially the application of convolutional neural networks (CNNs) and visual transformers (ViTs) in defect detection. However, existing technologies still have many shortcomings: with small datasets, current systems typically employ a single image processing workflow, which is insufficient to meet high-precision requirements; on-site acquired images suffer from uneven lighting and noise interference, and directly inputting them into the model can lead to a decrease in detection accuracy. Summary of the Invention

[0004] To address the problems of poor generalization ability, single detection path, unstable image quality, and insufficient accuracy of existing printing defect detection models, the present invention aims to provide a high-precision printing defect detection method and system based on deep learning, so as to at least partially solve the above problems.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] In a first aspect, the present invention provides a high-precision printing defect detection method based on deep learning, the method comprising the following steps:

[0007] S1. A multi-objective tunicate optimization algorithm is used to enhance the printed image;

[0008] S2. Construct a deep learning network model based on physical information constraints. By fusing the gated recurrent unit (GRU) and the visual Transformer (ViT) in a dual-path parallel structure, and introducing the Lambertian volume reflection model and the percentage of white pixels as physical constraint embedding loss functions, the PINN-GRUViT model is formed.

[0009] S3. The PINN-GRUViT model is trained and optimized using transfer learning methods;

[0010] S4. The enhanced printed image is processed using the optimized PINN-GRUViT model to obtain the printed defect detection results.

[0011] In some preferred embodiments, in step S1, the multi-objective tunicate optimization algorithm is a multi-objective tunicate algorithm based on chaotic optimization (MOLSSA), which includes:

[0012] An improvement to the Saurus Aquarius (SSA) optimization algorithm is introduced by introducing a Logistic chaotic mapping: when the optimal fitness value changes less than a threshold ε over K consecutive generations, the algorithm is considered trapped in a local optimum; then, the optimal solution is perturbed and updated using a Logistic chaotic sequence to generate d-dimensional chaotic variables, and the algorithm is then updated according to the formula... Perform a perturbation update, where X new X represents a new candidate solution generated through chaotic perturbation; best λ represents the individual with the best solution in the current population; λ represents the perturbation intensity coefficient, used to control the magnitude of the perturbation; c i U represents a chaotic variable generated by the Logistic mapping, with a value range of [0, 1]. Subtracting 0.5 from its value allows the perturbation to be distributed on both sides of the solution; b and L b These represent the upper and lower bounds of the search space, respectively.

[0013] The single-objective tunicate optimization algorithm improved by Logistic chaotic mapping is extended into a multi-objective tunicate optimization algorithm MOLSSA: Pareto dominance and external archive mechanism are introduced, crowding degree sorting is used to ensure population diversity, and multiple leaders are selected from the archive to replace the single optimal leader based on density selection method, and the Pareto front solution set is output.

[0014] In some preferred embodiments, step S1 specifically employs the Bézier surface fitting method of the multi-objective tunicate optimization algorithm to correct for uneven illumination in the acquired printed image, including:

[0015] Step 1: Convert the printed image to the HSV color space, extract the luminance channel V and normalize it;

[0016] Step 2: Create a uniform two-dimensional grid, i.e. Bézier surface control points, according to the set grid size. Divide the printed image into multiple small blocks according to the grid, calculate the average brightness of each small block, and use the average brightness of each small block to initialize the control points at the corresponding positions, so that the control points directly reflect the brightness distribution of the printed image.

[0017] Step 3: Calculate the kernel size of the Gaussian blur based on the smoothness parameter, and apply Gaussian blur to all control points to make the numerical transition between adjacent control points smoother, thus completing the smoothing adjustment process of the adjustment surface.

[0018] Step 4: Input a set of control points and an interpolation parameter δ of 0 to 1, and use the Bessel formula to calculate the interpolation result for the current position;

[0019] Step 5: For each pixel, use the Bezier surface to calculate the corresponding light intensity value, and finally generate a light vector map with the same size as the original image, turning the control points into a complete light surface that covers the entire image;

[0020] Step 6: Calculate the average value of the lighting surface to ensure that the overall brightness remains unchanged after correction. Perform brightness correction according to the formula "corrected brightness = original image brightness - (lighting surface - mean) × correction intensity", and fine-tune it by adding the brightness balance parameter brightness_balance.

[0021] Step 7: Merge the corrected V channel with the original H and S channels to complete image enhancement.

[0022] In some preferred embodiments, information entropy, contrast, and brightness uniformity are used as multi-objective functions to optimize the smoothness, correction strength, and brightness balance parameters in the Bézier surface fitting. The optimization process is as follows:

[0023]

[0024] E(x), C(x), and B(x) represent the information entropy, contrast, and brightness uniformity functions, respectively; g i (x) and h j (x) represent the constraints.

[0025] In some preferred embodiments, in step S2, the PINN-GRUViT model includes:

[0026] The input layer is used to receive image patch sequence tensors, in the format of batch, total number of image patches, and dimension.

[0027] The main network layer includes a GRU branch and a ViT branch. The GRU branch is used to adjust the format of the image patch sequence tensor received by the input layer to the total number of image patches, batch, and dimension, and then input it into a multi-layer GRU to extract temporal features. The ViT branch is used to perform global self-attention feature extraction on the image patch sequence tensor received by the input layer.

[0028] The tail network layer includes a multi-layer linear network used to receive data from the GRU branch and the ViT branch and output the printing defect classification results.

[0029] In some preferred embodiments, step S2, which involves introducing the Lambertian reflection model and the percentage of white pixels as the physical constraint embedding loss function, includes:

[0030] Based on the correlation between image grayscale values ​​and defect probability, and between the percentage of white pixels and defect probability, differential equation constraints are established:

[0031]

[0032] In the formula, G and W represent the percentage of grayscale and white pixels in the image, respectively. Indicates the probability of a defect;

[0033] Construct the loss function:

[0034]

[0035] In the formula, W i and G i represents the percentage of the i-th white pixel and the normalized gray value in a batch during model training, respectively; I is the confidence score output by the model after softmax; B represents the number of samples in a batch during model training; m is a scaling factor used to adjust the weights of the percentage of white pixels and gray value in the loss of physical information.

[0036] In some preferred embodiments, the calculation results of the model's classification of defects in printed images are processed using softmax, and the resulting values ​​are mapped to the value range (0, 1]. The specific formula is as follows:

[0037]

[0038] z i Let P represent the maximum value of the i-th element in a batch of technical results during model training, obtained by the above formula. This value is recorded as the confidence level P of that batch.

[0039] In some preferred embodiments, step S3, the step of training and optimizing the PINN-GRUViT model using transfer learning, includes:

[0040] Pre-train the PINN-GRUViT model on a publicly available classic defect dataset;

[0041] Freeze the tail network layer of the PINN-GRUViT model to retain the feature extraction capabilities of the GRU and ViT branches;

[0042] The PINN-GRUViT model is transferred to adjust weight parameters based on defect samples from printed materials. A domain adaptation strategy is used to reduce the distribution difference between the source domain and publicly available classic defect datasets, thereby enhancing the model's generalization ability and detection accuracy.

[0043] Secondly, the present invention also provides a high-precision printing defect detection system based on deep learning, comprising:

[0044] A conveyor system for transporting printed materials to be inspected;

[0045] There are two image acquisition devices, which are spaced apart along the conveying direction;

[0046] A light source used to illuminate printed materials on the conveyor belt.

[0047] The system includes a workstation for loading the PINN-GRUViT model, receiving printed images acquired by two image acquisition devices, and executing the method described above to output printed defect detection results.

[0048] In some preferred embodiments, the defect detection results include "defective" and "no defect". When the workstation outputs "no defect" as the defect detection result for the printed images acquired by the two image acquisition devices, the corresponding printed product is determined to be "no defect".

[0049] The beneficial effects of this invention using the above technical solution are as follows: The method of this invention, relying on the PINN-GRUViT model and transfer learning technology, achieves efficient defect identification in small-sample, high-noise industrial scenarios, providing a reliable technical solution for visual inspection of printed materials. Through image enhancement preprocessing and a dual-path parallel detection strategy, the accuracy and robustness of defect detection are improved, achieving high-precision printing defect detection. The system of this invention, by setting up two image acquisition devices, can perform two independent inspections, balancing detection accuracy and real-time performance. This enables embedded real-time deployment on high-speed printing production lines, effectively reducing product defect rates and improving the intelligence level and production efficiency of printing enterprises. Attached Figure Description

[0050] Figure 1 This is a flowchart of a high-precision printing defect detection method based on deep learning in this invention;

[0051] Figure 2This is a schematic diagram of the structure of the deep learning network model based on physical information constraints in this invention;

[0052] Figure 3 This is a schematic diagram of a high-precision printing defect detection system based on deep learning according to the present invention;

[0053] Figure 4 This is a flowchart illustrating the workflow of a high-precision printing defect detection system based on deep learning, as described in this invention.

[0054] In the diagram: 1- Assembly line conveyor; 2- Image acquisition device; 3- Workstation; 4- Support; 41- Support leg; 42- Truss; 43- Slide bar; 5- Direction control button; 6- Emergency stop button. Detailed Implementation

[0055] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings. It should be noted that these descriptions are for the purpose of aiding understanding the present invention, but do not constitute a limitation thereof. Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0056] Example 1

[0057] See Figure 1 This is a flowchart illustrating a high-precision printing defect detection method based on deep learning provided in an embodiment of the present invention. Figure 1 The execution entity of the method shown can be a software and / or hardware device. The execution entity of this application can include, but is not limited to, at least one of the following: user equipment, network equipment, etc. User equipment can include, but is not limited to, computers, smartphones, personal digital assistants (PDAs), and the aforementioned electronic devices. Network equipment can include, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of computers or network servers. Cloud computing is a type of distributed computing, consisting of a super virtual computer composed of a group of loosely coupled computers. This embodiment does not impose any limitations on this.

[0058] like Figure 1 As shown in the figure, the high-precision printing defect detection method based on deep learning provided by this invention mainly includes four steps, namely steps S1, S2, S3 and S4, as follows:

[0059] S1. A multi-objective tunicate optimization algorithm is used to enhance the printed image;

[0060] S2. Construct a deep learning network model based on physical information constraints. By fusing the gated recurrent unit (GRU) and the visual Transformer (ViT) in a dual-path parallel structure, and introducing the Lambertian volume reflection model and the percentage of white pixels as physical constraint embedding loss functions, the PINN-GRUViT model is formed.

[0061] S3. The PINN-GRUViT model is trained and optimized using transfer learning methods;

[0062] S4. The enhanced printed image is processed using the optimized PINN-GRUViT model to obtain the printed defect detection results.

[0063] Step S1 of this embodiment describes the enhancement of the printed image, specifically using the multi-objective tunicate optimization algorithm. The following will introduce the tunicate optimization algorithm, the improved tunicate optimization algorithm, the multi-objective tunicate optimization algorithm, the combined Bézier surface fitting algorithm, and the MOLSSA multi-objective optimization algorithm.

[0064] 1. Optimization algorithm for tunicate sea squirt

[0065] The Salp Swarm Algorithm (SSA) is a swarm intelligence optimization algorithm proposed by Mirjalili et al. in 2017, inspired by the swimming characteristics of salp swarms in the ocean. This algorithm simulates the predation behavior of salp swarms in a partially ordered chain-like structure, offering advantages such as fewer parameters and a simpler structure. In the SSA algorithm, the spatial position of individuals is updated through a partially ordered chain-like predation mechanism: the leader individual at the head of the chain guides other followers to optimize the population position through random walks; in the later stages of the algorithm, the range of the leader's random walk is gradually reduced to ensure convergence at the global optimum.

[0066] In the SSA algorithm, salps populations are divided into leaders and followers. The leader is the head of the chain formed during the salps' foraging process, while the rest are followers. The leader's specific task is to guide the entire salps population in its movement and foraging, and the followers follow in turn.

[0067] Similar to other swarm intelligence optimization algorithms, in the SSA algorithm, the position of an individual tunicate is defined in a search space with n variables, and the positions of all tunicates are stored in a matrix x. To find the food source, i.e., the optimal solution F, the mathematical expression for updating the leader's position is as follows:

[0068] Formula 1

[0069] In the formula, F represents the position of the leader in the j-th coordinate system; j This represents the position of the food source in the j-th dimension coordinate system; ub j and lb j c1 and c2 represent the upper and lower bounds of the j-th coordinate system, respectively; c 2 and c3 are random numbers. Equation 1 allows the leader to update the location of food sources, while the coefficient c1, an important parameter in the algorithm's search process, is given by the following equation:

[0070] Formula 2

[0071] In the formula, L represents the maximum number of iterations, and l represents the current iteration number.

[0072] In Equation 1, parameters c2 and c3 are uniformly generated random numbers in the interval [0, 1], which determine the direction and step size of the next position in the j-dimensional space.

[0073] According to Newton's laws of motion, the position update of the follower during SSA optimization uses the following formula:

[0074] Formula 3

[0075] In the formula, x j i This represents the position of the i-th follower from the tunic chain in the j-th coordinate system; t and v0 represent time and initial velocity, respectively; and a and v are given by the formula... and The optimization process iterates in units of time, with the difference between each iteration equal to 1. Considering the initial velocity v0 = 0, the formula is expressed as follows:

[0076] Formula 4

[0077] 2. Improved optimization algorithm for tunicate sea squirt

[0078] The Logistic Map itself is a simple nonlinear dynamic equation, often used to generate chaotic sequences. In optimization algorithms, it's not directly used to "optimize" the objective function, but rather embedded as an enhancement mechanism into existing metaheuristic algorithms to address problems such as getting trapped in local optima, insufficient population diversity, or slow convergence speed in traditional algorithms. The mathematical formula for the Logistic Map is:

[0079] Formula 5

[0080] In the formula, x nLet μ represent the chaotic variable in the nth iteration, and μ represent the control parameter. When μ = 4, the system is in a completely chaotic state, at which point the ergodicity is optimal. In the middle and later stages of the algorithm's iteration process, the obtained optimal solution may not be updated for a long time. Therefore, a chaotic sequence is used to slightly perturb the optimal solution, helping it escape local optima. This invention proposes an improved SSA algorithm (Logistic Salp Swarm Algorithm, LSSA) based on chaotic optimization, using the above-described chaotic optimization process. The specific calculation process is as follows:

[0081] Step 1: During the SSA optimization iteration process, if the change in the optimal fitness value is less than the threshold ϵ for K consecutive generations, it is determined that the system is trapped in a local optimum.

[0082] Step 2: From the current x n Begin by iterating through the Logistic mapping to generate a set of d-dimensional chaotic variables. ;

[0083] Step 3: Perform the perturbation update according to the following formula:

[0084] Formula 6

[0085] Step 4: Calculate X new If its fitness is better than X best If the solution is not found, then it is replaced with the new optimal solution.

[0086] In the formula, X new X represents a new candidate solution generated through chaotic perturbation; best λ represents the individual with the best solution in the current population; λ represents the perturbation intensity coefficient, used to control the magnitude of the perturbation; c i U represents a chaotic variable generated by the Logistic mapping, with a value range of [0, 1]. Subtracting 0.5 from its value allows the perturbation to be distributed on both sides of the solution; b and L b These represent the upper and lower bounds of the search space, respectively.

[0087] 3. Multi-objective tunic algorithm

[0088] The improved single-objective Salp Swarm Optimization Algorithm (LSSA) is extended into a multi-objective optimization algorithm (MOLSSA). The core of this extension is the introduction of Pareto dominance and an external archive (food source pool), along with modifications to the individual update and selection mechanisms, ultimately outputting a complete Pareto front. The objective vector F(x) of the MOLSSA optimization algorithm is defined as:

[0089] Formula 7

[0090] In the formula, f m (x) represents a single objective, where i represents the number of inequality constraints; j represents the number of equality constraints; g i (x) represents the i-th inequality constraint, indicating that a certain quantity cannot be lower than or higher than its set specification, and the solution to x needs to satisfy all g. i A solution is considered feasible only if (x) ≤ 0; h j (x) represents the j-th equality constraint, signifying a fixed relation that a certain quantity follows. The solution to x needs to satisfy all h. i A solution can only be considered feasible after (x) = 0; g i (x)≤0 and h i (x)=0 together constitute the constraint condition of the optimization problem.

[0091] The specific steps of the MOLSSA algorithm based on Pareto dominance transformation are as follows:

[0092] Step 1: Initialize the population And calculate the target vector F(x) for each individual;

[0093] Step 2: Initialize the external archive to store non-dominated solutions;

[0094] Step 3: Find all non-dominated solutions and add them to the archive as candidates;

[0095] Step 4: Calculate the crowding degree of each solution, sort them by crowding degree, and keep the M least crowded solutions to ensure the diversity of the population;

[0096] Step 5: Select multiple solutions from the archive using the density-based selection method as search guides, instead of a single optimal guide;

[0097] Step 6: Repeat Step 3–Step 5 above to obtain the existing non-dominated solution set, which is the Pareto front solution.

[0098] 4. Combining the Bézier surface fitting algorithm and the MOLSSA multi-objective optimization algorithm

[0099] This invention further combines the Bézier surface fitting algorithm and the MOLSSA multi-objective optimization algorithm to perform enhanced predictive processing on printed images to address issues such as uneven illumination. The specific steps are as follows:

[0100] STEP 1: Convert the printed image to the HSV color space, extract the luminance channel V and normalize it;

[0101] STEP 2: Create a uniform two-dimensional grid, i.e. Bézier surface control points, according to the set grid size. Divide the printed image into multiple small blocks according to the grid, calculate the average brightness of each small block, and use the average brightness of each small block to initialize the control points at the corresponding positions, so that the control points directly reflect the brightness distribution of the printed image.

[0102] STEP 3: Calculate the kernel size of the Gaussian blur based on the smoothness parameter, and apply Gaussian blur to all control points to make the numerical transition between adjacent control points smoother, thus completing the smoothing adjustment process of the adjustment surface.

[0103] STEP 4: Input a set of control points and an interpolation parameter δ of 0 to 1, and use the Bessel formula to calculate the interpolation result for the current position;

[0104] STEP 5: For each pixel, use a Bézier surface to calculate the corresponding light intensity value, and finally generate a light vector map with the same size as the original image, turning the control points into a complete light surface that covers the entire image;

[0105] STEP 6: Calculate the average value of the lighting surface to ensure that the overall brightness remains unchanged after correction. Perform brightness correction according to the formula "corrected brightness = original image brightness - (lighting surface - mean) × correction intensity", and fine-tune it by adding the brightness balance parameter brightness_balance.

[0106] STEP 7: Merge the corrected V channel with the original H and S channels to complete image enhancement.

[0107] Then, using information entropy, contrast, and brightness uniformity as multi-objective functions, the smoothness value, correction strength, and brightness balance parameters in the Bézier surface fitting are optimized. Higher information entropy indicates richer image details, higher contrast indicates more distinct light and dark levels, and a brightness uniformity closer to 1 indicates a more uniform brightness distribution. The optimization process is as follows:

[0108] Formula 8

[0109] In the formula, E(x), C(x), and B(x) represent the information entropy, contrast, and brightness uniformity functions, respectively; g i (x) and h j (x) represent the constraints.

[0110] Step S2 of this embodiment describes how to construct a deep learning network model based on physical information constraints, such as... Figure 2The diagram shows the structure of the deep learning network model based on physical information constraints proposed in this embodiment of the invention. This model uses the Lambertian reflection principle and the percentage of white pixels as prior physical laws, integrating them into a deep learning network model combining a gated recurrent network and ViT (Virtual Information Transformation) to serve as constraints for the Physically Informed Neural Network (PINN). This results in the construction of a deep learning network model based on physical information constraints, namely the PINN-GRUViT model. The PINN-GRUViT model will be described in detail below, covering aspects such as the Lambertian reflection model, the percentage of white pixels, the constraint relationship between image features and defects, and the network structure.

[0111] 1. Lambertian reflection model

[0112] Under the Lambertian reflection model, an indirect constraint relationship exists between the image's grayscale and the crack, established through the surface normal vector, reflection coefficient, and illumination direction. However, the necessary conditions for crack occurrence cannot be directly derived. The following is a detailed analysis:

[0113] The core assumption of the Lambertian volumetric reflection model is that the surface of an object is an ideal diffuse reflector, with the reflected light intensity uniformly distributed in all directions and proportional to the cosine of the angle of incidence. Under this model, the gray value I of a point in an image can be expressed as:

[0114] Formula 9

[0115] In the formula, ρ represents the surface reflectance coefficient, which is related to the material color; L is the light source direction vector; and N is the surface unit normal vector.

[0116] Cracks can cause abrupt changes in the local normal vector N of the surface (such as tilting or concave fracture surfaces), thereby altering the value of L·N. If the crack direction is nearly perpendicular to the light source direction L, the reflected light intensity will decrease significantly, leading to a decrease in grayscale value; conversely, if the crack direction is parallel to L, the reflected light intensity may increase, resulting in a higher grayscale value.

[0117] Cracks may expose internal material, causing changes in the local reflection coefficient ρ. Therefore, the constraint between image grayscale and cracks can be expressed as:

[0118] Formula 10

[0119] In the formula, and Let represent the surface reflection coefficient and the unit normal vector of the cracked region, respectively. Therefore, it can be inferred that the probability of a crack appearing is proportional to its image grayscale value. With the light source position constant, The value is the same as N.

[0120] 2. White pixel percentage

[0121] White Pixel Percentage (WPP) is an image analysis technique used to quantify the proportion of white pixels in an image. In paper folding crack detection, the WPP method quantifies the severity of the crack by identifying the cracked area as a white or high-brightness region and calculating the proportion of these areas in the entire image. This method is based on the assumption that cracks caused by folding typically appear as brighter areas than the surrounding paper in an image, thus the severity of the crack can be objectively assessed by calculating the proportion of white pixels.

[0122] The specific steps for implementing the percentage of white pixels are as follows:

[0123] Step 1: Binarize the image using Otsu's automatic thresholding method. The solution method is as follows:

[0124] Formula 11

[0125] In the formula, w1(τ) and w2(τ) represent the probability weights of the two categories, respectively. and These represent the variances of the two categories, and the optimal threshold τ is automatically determined by minimizing the intra-class variance.

[0126] Step 2: Define the pixels with a t value of 255 obtained in the previous step as white pixels, and calculate the number of white pixels;

[0127] Step 3: Calculate the percentage of white pixels using the following formula:

[0128] Formula 12

[0129] In the formula, C w C represents the number of white pixels, and C represents the total number of pixels.

[0130] 3. Constraints between image features and defects

[0131] By using the Lambertian volume reflectance model and the percentage of white pixels, a correlation is established between image features and defects: the higher the image grayscale value, the higher the probability of defects; the larger the percentage of white pixels, the higher the probability of defects. Based on these two proportional relationships, the following differential equation constraints are established:

[0132] Formula 13

[0133] In the formula, G and W represent the percentage of grayscale and white pixels in the image, respectively; This represents the probability of a defect. Since there is a positive correlation between both image features and defects, its derivative is always positive.

[0134] 4. Network Structure

[0135] like Figure 2 As shown, the deep learning network model based on physical information constraints (i.e., the PINN-GRUViT model) proposed in this embodiment of the invention includes an input layer, a main network layer, a feature fusion layer, and a tail network layer.

[0136] The input layer is used to receive image patch sequence tensors, in the format of batch, total number of image patches, and dimension.

[0137] The main network layer comprises GRU and ViT branches. The GRU branch includes a multi-layer gated recurrent unit (GRU). The GRU branch adjusts the format of the image patch sequence tensor received from the input layer to reflect the total number of image patches, batch size, and dimension before inputting it into the multi-layer GRU to extract temporal features. The ViT branch performs global self-attention feature extraction on the image patch sequence tensor received from the input layer. The main network layer aims to integrate the temporal information features of sequence modeling with the advantages of global visual spatial feature extraction.

[0138] The tail network layer consists of multiple linear networks that receive data from the GRU and ViT branches and output the results of print defect classification.

[0139] Vision Transformer (ViT) is a deep learning model that directly applies the Transformer architecture from natural language processing to image recognition tasks, breaking the long-standing dominance of traditional convolutional neural networks in computer vision. It divides an image into small blocks, flattens each block and embeds it as a "visual lexicon," adds positional encoding, and then inputs it into a standard Transformer encoder. It utilizes a self-attention mechanism to capture global contextual information, thereby achieving visual tasks such as image classification. Unlike convolutional neural networks, which rely on local receptive fields to extract features layer by layer, ViT's core advantage lies in its powerful global modeling capability. It can directly capture the dependencies between distant pixels in an image, making it particularly suitable for pre-training on large-scale data and then transferring it to downstream tasks, while requiring relatively fewer computational resources in the process.

[0140] The constructed deep learning network model ultimately outputs image data as classification values. The model's confidence score needs to conform to the image's physical characteristics; that is, the more white pixels and the higher the grayscale value, the higher the model's confidence score. Therefore, the model's image classification results are processed using softmax and then mapped to the value range (0, 1]. The specific formula is as follows:

[0141] Formula 14

[0142] In the formula, z i Let represent the i-th element in the technical results of a batch during model training. Find the largest value obtained from the above formula and denote it as the confidence score P of that batch.

[0143] To express prior physical knowledge, based on the two sets of constraints mentioned above, the physical information loss is constructed in the loss function of deep learning network training as follows:

[0144] Formula 15

[0145] In the formula, W i and G i These represent the percentage of the i-th white pixel and the normalized gray value in a batch during model training, respectively; I is the confidence score output by the model after softmax; B represents the number of samples in a batch during model training; m is a scaling factor used to adjust the weights of the percentage of white pixels and gray value in the physical information loss, with the default setting being m = 0.3.

[0146] Furthermore, in the above process, the grayscale values ​​are normalized using the "max-min" method and projected into the interval (0, 1]. The specific formula for calculating the grayscale normalization value is as follows:

[0147] Formula 16

[0148] In the formula, G max and G min These represent the maximum and minimum grayscale values, respectively. It is a minimum value greater than zero, which ensures that the normalized value is not zero and is used to adapt to the calculation of physical information loss.

[0149] The aforementioned constraints are incorporated into the loss function of the deep learning model, thereby constructing a spatiotemporal characteristic information deep learning network based on physical information constraints (PINN-GRUViT). During model training, its loss function gradually decreases with each iteration. By incorporating these constraints, the model consistently adheres to physical laws throughout the iteration process, seamlessly integrating prior physical knowledge with observed data. This results in a model that simultaneously possesses both physical laws and data characteristics, improving its generalization ability and accuracy.

[0150] Step S3 of this embodiment describes the application of the constructed PINN-GRUViT model to the field of printed material defect detection using transfer learning. Specifically, the step of training and optimizing the PINN-GRUViT model using transfer learning includes:

[0151] Step 1: Pre-training

[0152] Based on publicly available classic defect datasets, defect sample annotation and data cleaning were completed, and training and validation sets containing multiple types of defects were constructed to provide standardized data support for the training of the PINN-GRUViT model, thus obtaining a PINN-GRUViT model pre-trained on publicly available classic defect datasets.

[0153] Step 2: Transfer Learning Optimization

[0154] The tail network layer of the PINN-GRUViT model is frozen, while retaining the feature extraction capabilities of the GRU and ViT branches. Transfer learning is performed on the pre-trained PINN-GRUViT model based on printed defect samples to adjust the weight parameters. The distribution difference between the source domain and the publicly available classic defect dataset is reduced through a domain adaptation strategy, thereby enhancing the model's generalization ability and detection accuracy.

[0155] Once the PINN-GRUViT model has been optimized through transfer learning, it can be deployed. The optimized PINN-GRUViT model is then used to process the enhanced printed images to obtain the defect detection results, which can be categorized as "defective" or "no defect".

[0156] The image enhancement technique employed in this embodiment of the invention can solve the problem of weak image feature information caused by uneven illumination. The processed image improves image feature information indicators, which is beneficial for the model's identification effect on images with uneven illumination. Table 1 shows a comparison of image enhancement indicators before and after processing using the multi-objective optimization algorithm employed in this embodiment of the invention. Two sets of defective images with uneven illumination were processed, and the image feature information indicators before and after processing are shown in Table 1.

[0157] Table 1 - Comparison of image feature information indicators before and after image enhancement

[0158] Table 1 Sample Figure 1 and sample Figure 2 Among the indicators, the enhanced information entropy increased by 6.1% and 11.7% respectively compared to the original image, indicating that the image enhancement method used in this embodiment of the invention can improve the texture complexity of the image, thereby increasing the amount of information in the image features; the enhanced contrast increased by 12.7% and 36.8% respectively compared to the original image, thus obtaining more image detail information; while the enhanced brightness uniformity decreased by 11.7% and 56.5% respectively compared to the original image, indicating that the image enhancement reduced the brightness difference between each patch, making the image brightness more uniform, thereby reducing the unevenness of illumination.

[0159] Table 2 presents the accuracy, precision, recall, and F1-score results of each model in the image classification task, reflecting the differences in classification performance and optimization effects of different models.

[0160] Table 2 - Comparison of Model Performance Indicators

[0161] As shown in Table 2, traditional CNN and GRU, limited by their local feature representation capabilities, achieve accuracies of 0.54 and 0.51 respectively, making it difficult to effectively capture global dependencies in images under limited sample conditions. ViT achieves a performance leap through a global self-attention mechanism, with its accuracy rising to 0.63, validating the advantages of the Transformer architecture in visual tasks. GRU-ViT, which integrates sequence modeling and global features, achieves an accuracy of 0.78, demonstrating the synergistic effect of feature complementarity. PINN-GRUViT, after introducing physical information constraints, achieves an accuracy of 0.79, indicating that incorporating physical information constraints into the model can effectively improve the model's decision consistency and generalization. PINN-GRUViT, optimized through transfer learning, achieves the best results in accuracy, precision, recall, and F1-score (0.89), fully validating the dual optimization effect of physical information integration and transfer learning on image classification performance, providing an effective method for solving small-sample image classification tasks.

[0162] Example 2

[0163] This embodiment provides a high-precision printing defect detection system based on deep learning, such as... Figure 3 As shown, it includes a conveyor belt 1, an image acquisition device 2, a light source, and a workstation 3.

[0164] The conveyor belt 1, for example, is mounted on the support 4. It is used to transport the printed materials to be inspected. The conveyor belt 1 is driven by a motor, and the support 4 is equipped with direction control buttons 5 for controlling the forward and reverse rotation of the motor. The conveying direction can be controlled via these buttons. Emergency stop buttons 6, suitable for manual operation, are fixed at both ends of the conveying direction on the support 4. These two emergency stop buttons 6 provide redundancy and facilitate emergency stopping by the operator at both ends of the support 4.

[0165] The support 4 includes two vertically arranged legs 41, and a truss 42 is connected between the two legs 41. The truss 42 is located on the upper side of the conveyor 1, and a slide rod 43 is slidably connected to the bottom side of the truss 42. The slide rod 43 is long and perpendicular to the conveying direction, and the slide rod 43 can slide on the truss 42 along the conveying direction. The slide rod 43 can be locked in its sliding position by screws.

[0166] Image acquisition device 2 is fixed to slide bar 43, such as an industrial camera, with its field of view facing the conveyor belt 1 to acquire images of the printed materials. A light source is mounted on truss 42 or slide bar 43 to provide illumination to the printed materials on the conveyor belt 1, and then the image acquisition device 2 acquires the images of the printed materials.

[0167] Workstation 3 is electrically connected to image acquisition device 2 and conveyor belt 1. For example, workstation 3 is configured as a display and control device, which is loaded with the PINN-GRUViT model optimized by transfer learning provided in the above embodiment. Workstation 3 receives the printed image acquired by image acquisition device 2, executes the method provided in the above embodiment, and then outputs the printed defect detection result, which includes "defective" and "no defect". Workstation 3 is also used to display the running direction and speed of conveyor belt 1, the printed image acquired by image acquisition device 2, and the defect detection result.

[0168] In a preferred embodiment, two image acquisition devices 2 can be configured. The two image acquisition devices 2 are respectively mounted on two slide bars 43. The two slide bars 43 are arranged at intervals along the conveying direction. The two image acquisition devices 2 can share the same light source, or the two image acquisition devices 2 can each be configured with a light source. Both image acquisition devices 2 are electrically connected to the workstation 3.

[0169] With this setup, workstation 3 can perform two defect checks on the printed material, thereby reducing the false negative rate and improving defect detection efficiency and accuracy. Assuming that when workstation 3 detects the printed material based on an image acquired by one of the image acquisition devices 2, the actual defect detection rate is α; and when workstation 3 detects the printed material based on an image acquired by the other image acquisition device 2, the actual defect detection rate is β, and the two checks are independent and target the same printed material, meaning that if either check identifies a defect, it is considered a defect. The overall detection rate is then:

[0170] Formula 17

[0171] From the above formula, it can be determined that the overall detection rate is always higher than that of any single detection, effectively reducing the probability of missed detection and improving the reliability of defect identification. Let α = 0.89 and β = 0.79, then the detection rates are shown in Table 3:

[0172] Table 3 - Overall detection rate based on two independent tests

[0173] The usage process of the high-precision printing defect detection system based on deep learning provided in this embodiment is as follows:

[0174] Step 1: Model Deployment

[0175] The PINN-GRUViT model optimized through transfer learning in Example 1 is loaded onto workstation 3 to complete the model inference environment configuration, providing computing power and model services for subsequent online detection of dual devices.

[0176] Step 2: Parallel Start-up of Dual Devices

[0177] Two image acquisition devices 2 are started simultaneously and enter the online detection state, respectively executing independent image acquisition and defect identification processes.

[0178] Step 3: First Independent Test

[0179] For the printed image acquired by the first image acquisition device 2 (located upstream in the conveying direction), image enhancement processing is performed on it based on the algorithm provided in Example 1 to optimize the identification of defect features; then the PINN-GRUViT model deployed in workstation 3 is called to perform defect identification and output the judgment result of "defective / no defect".

[0180] Step 4: Second independent test

[0181] For the printed image acquired by the second image acquisition device 2 (located downstream in the conveying direction), image enhancement processing can be performed based on the algorithm provided in Example 1. Alternatively, the PINN-GRUViT model deployed in workstation 3 can be directly called to perform defect identification (image enhancement is omitted to ensure real-time detection and avoid the adverse effects of image enhancement algorithms under uneven lighting conditions, such as...). Figure 4 As shown in the figure, the output "defective / no defect" judgment result is used to form a complementary detection with the first detection.

[0182] Step 5: Results Integration and Output

[0183] Integrate the results of two independent tests: if either independent test result is "defective", the final test result is marked as defective; if both independent test results are "no defect", the result is marked as normal.

[0184] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "front", "rear", etc., indicate the orientation or positional relationship based on the description of the structure of this invention shown in the accompanying drawings. They are only for the convenience of describing this invention and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0185] The terms "first" and "second" in this technical solution are merely designations for corresponding structures that are identical or similar, or that perform similar functions. They do not represent an arrangement of the importance of these structures, nor do they imply any ranking, comparison of size, or other meaning.

[0186] Furthermore, unless otherwise explicitly specified and limited, the terms "installation" and "connection" should be interpreted broadly. For example, a connection can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two structures. Those skilled in the art can understand the specific meaning of the above terms in this invention by considering the overall concept of the invention and the specific context of the solution.

[0187] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A high-precision printing defect detection method based on deep learning, characterized in that, The method includes the following steps: S1. A multi-objective tunicate optimization algorithm is used to enhance the printed image; S2. Construct a deep learning network model based on physical information constraints. By fusing the gated recurrent unit (GRU) and the visual Transformer (ViT) in a dual-path parallel structure, and introducing the Lambertian volume reflection model and the percentage of white pixels as physical constraint embedding loss functions, the PINN-GRUViT model is formed. S3. The PINN-GRUViT model is trained and optimized using transfer learning methods; S4. The enhanced printed image is processed using the optimized PINN-GRUViT model to obtain the printed defect detection results.

2. The method according to claim 1, characterized in that: In step S1, the multi-objective tunicate optimization algorithm is the multi-objective tunicate algorithm MOLSSA based on chaotic optimization, which includes: An improvement to the Saurus Aquarius (SSA) optimization algorithm is introduced by introducing a Logistic chaotic mapping: when the optimal fitness value changes less than a threshold ε over K consecutive generations, the algorithm is considered trapped in a local optimum; then, the optimal solution is perturbed and updated using a Logistic chaotic sequence to generate d-dimensional chaotic variables, and the algorithm is then updated according to the formula... Perform a perturbation update, where X new X represents a new candidate solution generated through chaotic perturbation; best λ represents the individual with the best solution in the current population; λ represents the perturbation intensity coefficient, used to control the magnitude of the perturbation; c i U represents a chaotic variable generated by the Logistic mapping, with a value range of [0, 1]. Subtracting 0.5 from its value allows the perturbation to be distributed on both sides of the solution; b and L b These represent the upper and lower bounds of the search space, respectively. The single-objective tunicate optimization algorithm improved by Logistic chaotic mapping is extended into a multi-objective tunicate optimization algorithm MOLSSA: Pareto dominance and external archive mechanism are introduced, crowding degree sorting is used to ensure population diversity, and multiple leaders are selected from the archive to replace the single optimal leader based on density selection method, and the Pareto front solution set is output.

3. The method according to claim 1, characterized in that: In step S1, the Bézier surface fitting method of the multi-objective tunicate optimization algorithm is specifically used to correct the uneven illumination of the acquired printed image, including: Step 1: Convert the printed image to the HSV color space, extract the luminance channel V and normalize it; Step 2: Create a uniform two-dimensional grid, i.e. Bézier surface control points, according to the set grid size. Divide the printed image into multiple small blocks according to the grid, calculate the average brightness of each small block, and use the average brightness of each small block to initialize the control points at the corresponding positions, so that the control points directly reflect the brightness distribution of the printed image. Step 3: Calculate the kernel size of the Gaussian blur based on the smoothness parameter, and apply Gaussian blur to all control points to make the numerical transition between adjacent control points smoother, thus completing the smoothing adjustment process of the adjustment surface. Step 4: Input a set of control points and an interpolation parameter δ of 0 to 1, and use the Bessel formula to calculate the interpolation result for the current position; Step 5: For each pixel, use the Bezier surface to calculate the corresponding light intensity value, and finally generate a light vector map with the same size as the original image, turning the control points into a complete light surface that covers the entire image; Step 6: Calculate the average value of the lighting surface to ensure that the overall brightness remains unchanged after correction. Perform brightness correction according to the formula "corrected brightness = original image brightness - (lighting surface - mean) × correction intensity", and fine-tune it by adding the brightness balance parameter brightness_balance. Step 7: Merge the corrected V channel with the original H and S channels to complete image enhancement.

4. The method according to claim 3, characterized in that: Using information entropy, contrast, and brightness uniformity as multi-objective functions, the smoothness value, correction strength, and brightness balance parameter in the Bézier surface fitting are optimized. The optimization process is as follows: ； E(x), C(x), and B(x) represent the information entropy, contrast, and brightness uniformity functions, respectively; g i (x) and h j (x) represent the constraints.

5. The method according to claim 1, characterized in that: In step S2, the PINN-GRUViT model includes: The input layer is used to receive image patch sequence tensors, in the format of batch, total number of image patches, and dimension. The main network layer includes a GRU branch and a ViT branch. The GRU branch is used to adjust the format of the image patch sequence tensor received by the input layer to the total number of image patches, batch, and dimension, and then input it into a multi-layer GRU to extract temporal features. The ViT branch is used to perform global self-attention feature extraction on the image patch sequence tensor received by the input layer. The tail network layer comprises multiple linear networks, which are used to receive data from the GRU branch and the ViT branch and output the printing defect classification results.

6. The method according to claim 1, characterized in that: In step S2, the step of introducing the Lambertian volume reflection model and the percentage of white pixels as the physical constraint embedding loss function includes: Based on the correlation between image grayscale values ​​and defect probability, and between the percentage of white pixels and defect probability, differential equation constraints are established: ； In the formula, G and W represent the percentage of grayscale and white pixels in the image, respectively. Indicates the probability of a defect; Construct the loss function: ； In the formula, W i and G i represents the percentage of the i-th white pixel and the normalized gray value in a batch during model training, respectively; I is the confidence score output by the model after softmax; B represents the number of samples in a batch during model training; m is a scaling factor used to adjust the weights of the percentage of white pixels and gray value in the loss of physical information.

7. The method according to claim 6, characterized in that: The model's calculation results for defect classification in printed images are then processed using softmax, and the resulting values ​​are mapped to the value range (0, 1]. The specific formula is as follows: ； z i Let P represent the maximum value of the i-th element in a batch of technical results during model training, obtained by the above formula. This value is recorded as the confidence level P of that batch.

8. The method according to claim 1, characterized in that: Step S3, the step of training and optimizing the PINN-GRUViT model using transfer learning includes: Pre-train the PINN-GRUViT model on a publicly available classic defect dataset; Freeze the tail network layer of the PINN-GRUViT model to retain the feature extraction capabilities of the GRU and ViT branches; The PINN-GRUViT model is transferred to adjust weight parameters based on defect samples from printed materials. A domain adaptation strategy is used to reduce the distribution difference between the source domain and publicly available classic defect datasets, thereby enhancing the model's generalization ability and detection accuracy.

9. A high-precision printing defect detection system based on deep learning, characterized in that, include: A conveyor system for transporting printed materials to be inspected; There are two image acquisition devices, which are spaced apart along the conveying direction; A light source used to illuminate printed materials on the conveyor belt. The workstation is used to load the PINN-GRUViT model, receive printed images acquired by two image acquisition devices, and execute the method described in any one of claims 1-8 to output printed defect detection results.

10. The system according to claim 9, characterized in that: The defect detection results include "defective" and "no defect". When the workstation outputs "no defect" as the defect detection result for the printed images acquired by the two image acquisition devices, the corresponding printed product is determined to be "no defect".

Images