Model training methods, printing defect detection methods and related devices

Abstract

This application proposes a model training method, a printing defect detection method, and related apparatus, relating to the field of model training and inference technology. The model training method includes: acquiring a defect image of a printed product and a first defect mask corresponding to the defect image; performing target processing on the defect image to obtain a defect sample image; wherein the target processing is used to mimic imaging interference factors in the defect detection image of the printed product; and training a target model to be trained based on the defect sample image, the first defect mask, and a standard template image of the printed product, wherein the target model is used to detect printing defects in the printed product, and the standard template image is used as a comparison reference image. The technical solution provided by this application can solve the problem of high false detection rates in traditional defect detection methods in the prior art.

Description

Technical Field

[0001] This application relates to the field of model training and inference technology, specifically to a model training method, a printing defect detection method, and related apparatus. Background Technology

[0002] In the field of printed matter quality inspection (mainly printing defect detection), traditional methods mainly rely on rule-based image processing technology. Although such methods have achieved a certain degree of automated detection, they still have obvious limitations in practical industrial applications. When faced with complex background interference, uneven lighting, slight deformation caused by shooting angle shift, etc., they are often not robust enough and have a high false detection rate. Summary of the Invention

[0003] Based on the defects and shortcomings of the existing technology, this application proposes a model training method, a printing defect detection method and related apparatus, which can solve the problem of high false detection rate in the traditional defect detection methods in the existing technology.

[0004] According to a first aspect of the embodiments of this application, a model training method is provided, the method comprising: Obtain a defect image of the printed material and a first defect mask corresponding to the defect image; The defect image is subjected to target processing to obtain a defect sample image; wherein, the target processing is used to mimic the imaging interference factors of the defect detection image of the printed matter; The target model to be trained is trained based on the defect sample image, the first defect mask, and the standard template image of the printed matter. The target model is used to detect printing defects in the printed matter, and the standard template image is used as a comparison reference image.

[0005] According to a second aspect of the embodiments of this application, a method for detecting printing defects is provided, the method comprising: Acquire defect detection images of the printed material to be inspected; The defect detection image and the standard template image of the printed matter to be inspected are input into the target model, and the defect detection result of the printed matter to be inspected is output by the target model; wherein, the target model is trained by the model training method as described in the first aspect.

[0006] According to a third aspect of the embodiments of this application, a model training apparatus is provided, the apparatus comprising: The first acquisition module is used to acquire a defect image of the printed matter and a first defect mask corresponding to the defect image; An image processing module is used to perform target processing on the defect image to obtain a defect sample image; wherein, the target processing is used to mimic the imaging interference factors of the defect detection image of the printed matter; The model training module is used to train the target model to be trained based on the defect sample image, the first defect mask, and the standard template image of the printed matter. The target model is used to detect printing defects in the printed matter, and the standard template image is used as a comparison reference image.

[0007] According to a fourth aspect of the embodiments of this application, a printing defect detection apparatus is provided, the apparatus comprising: The second acquisition module is used to acquire defect detection images of the printed matter to be inspected; The model inference module is used to input the defect detection image and the standard template image of the printed matter to be detected into the target model, and output the defect detection result of the printed matter to be detected through the target model; wherein, the target model is trained by the model training method as described in the first aspect.

[0008] According to a fifth aspect of the embodiments of this application, an electronic device is provided, including: a memory and a processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the method as described in the first or second aspect by running a program in the memory.

[0009] According to a sixth aspect of the embodiments of this application, a storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the method as described in the first or second aspect.

[0010] According to a seventh aspect of the embodiments of this application, a computer program product or a computer program is provided, the computer program product including the computer program, wherein a processor executes the computer program to implement the steps in the method as described in the first or second aspect.

[0011] In the technical solution provided in this application, by performing target processing on the defect image, the defect image used for model training can be made closer to the shooting effect of the defect detection image in the real detection environment. Furthermore, this application also uses a standard template image of the printed matter as a reference image. Utilizing the precise and noise-free template information of the standard template image, the defect sample image is compared with the standard template image to locate the printing defect. Further, a first defect mask is used as the ground truth to judge the difference between the predicted result and the ground truth. Training the model in this way enables the model to effectively learn the inherent structure and discriminative features of the data, thereby distinguishing between valid differences caused by real defects and interference differences caused by imaging interference factors such as background, lighting, and geometric deformation. When the model trained using this method is used for printing defect detection, it can overcome the feature mismatch problem caused by differences in imaging conditions (such as lighting, angle, resolution, lens distortion, etc.) between the actual acquired defect detection image and the standard template image, enabling effective identification and classification of defect objects in the image even against complex backgrounds, thereby improving model robustness and reducing the false detection rate. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0013] Figure 1 This is a flowchart illustrating a model training method provided in an embodiment of this application.

[0014] Figure 2 This is a schematic diagram of the dual-branch contrastive learning model provided in an embodiment of this application.

[0015] Figure 3 This is a flowchart illustrating a printing defect detection method provided in an embodiment of this application.

[0016] Figure 4 This is a flowchart illustrating the model training and inference process provided in the embodiments of this application.

[0017] Figure 5 This is a block diagram of a model training device provided in an embodiment of this application.

[0018] Figure 6 This is a block diagram of a printing defect detection device provided in an embodiment of this application.

[0019] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] Application Overview Printed materials, as important carriers of information dissemination and product packaging, directly affect brand image, consumer experience, and even product safety. For example, in the modern home appliance manufacturing process, product labels (such as energy efficiency labels, brand trademarks, model labels, safety warnings, etc.) serve as important carriers for conveying product information and showcasing brand image; their printing quality directly impacts consumers' perception of product quality and their trust in the brand.

[0022] In high-speed, continuous printing production, various printing defects are easily generated due to limitations in equipment precision, fluctuations in environmental factors (such as changes in temperature and humidity), unstable ink performance, or operational errors. These defects include, but are not limited to, omissions, ghosting, stains, scratches, character errors, and color deviations. These defects not only affect the aesthetic appearance of the product but may also lead to functional problems. Therefore, printing defect detection is of paramount importance for ensuring product quality, improving production efficiency, and reducing resource waste.

[0023] Traditional printing defect detection methods mainly rely on manual visual inspection or rule-based image processing techniques. Manual inspection is inefficient, costly, and susceptible to subjective factors and fatigue, making it difficult to guarantee consistency and accuracy. While rule-based image processing methods, such as template matching, edge detection, and gray-scale difference, automate defect detection, the acquired images of the printed materials are affected by imaging interference. When the defect detection images have slight rotation, scaling, uneven lighting, background noise, or blurriness, the preset rules are prone to failure, leading to false positives or false negatives.

[0024] For example, template matching methods have high requirements for template selection and alignment. Even slight changes in the size or angle of the printed material can lead to matching failure. Furthermore, some structural defects (such as character adhesion) may be ignored because they do not conform to preset rules. This method is also sensitive to geometric deformations of the image (such as perspective distortion and wrinkles). Even if the printed material itself is defect-free, deformations caused by the shooting angle or unevenness of the attached surface can be identified as "differences," resulting in a high false alarm rate. Edge detection methods are easily affected by noise and can easily misidentify non-defect edges as defects. Gray-scale difference methods, under uneven lighting conditions, may generate a large number of false defect signals due to local brightness differences, leading to a high false detection rate.

[0025] Furthermore, rule-based image processing techniques typically require manually designing complex feature extraction rules for each type of printed material, resulting in poor generalization ability and difficulty in adapting to the diverse production environment of printed materials.

[0026] To address this, this application provides a model training technique that performs target processing on defect images, making the defect images used for model training more closely resemble the actual shooting effect of defect detection images in real detection environments. Furthermore, a standard template image of the printed material is used as a reference image. Utilizing the precise and noise-free template information of the standard template image, the defect sample image is compared with the standard template image to locate the printing defect. Further, a first defect mask is used as the ground truth to determine the difference between the predicted result and the ground truth. This model training allows the model to effectively learn the inherent structure and discriminative features of the data, thereby distinguishing between valid differences caused by real defects and interference differences caused by imaging interference factors such as background, lighting, and geometric deformation. When the model trained using this method is used for printing defect detection, it can overcome the feature mismatch problem caused by differences in imaging conditions (such as lighting, angle, resolution, lens distortion, etc.) between the actual acquired defect detection image and the standard template image, enabling effective identification and classification of defect objects in the image even against complex backgrounds, thereby improving model robustness and reducing the false detection rate.

[0027] Exemplary methods This application also provides a model training method applied to a first electronic device, which may be a terminal device, such as a mobile phone, computer, vehicle computer, or detection device, or a server, such as a cloud server.

[0028] The method is described in detail below through some embodiments. The following embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0029] like Figure 1As shown, the model training method may include steps 101 to 103, as described below.

[0030] Step 101: Obtain the defect image of the printed material and the first defect mask corresponding to the defect image.

[0031] The printed materials mentioned herein may include, but are not limited to: product labels (such as energy efficiency labels, brand trademarks, model labels, safety warnings, etc.), product packaging, and the product itself (such as posters, brochures, etc.) and other items that need to be printed.

[0032] The defect images mentioned here refer to images that include printing defects in printed materials, such as images of missing prints, ghosting, stains, scratches, character errors, color deviations, and other printing defects. These defect images are used for model training.

[0033] A defect mask is a binary or probabilistic image used in image analysis and machine vision to accurately label the location, shape, and category of defective regions in an image. Therefore, the first defect mask corresponding to the defective image described in this embodiment is used to label the location, shape, and category of defective regions in the defective image. Its size corresponds to the defective image and it is used as a ground truth label during model training. Optionally, the first defect mask can be a binary image, where each pixel in the image has a value of 0 or 1, where "1" (generally white) indicates that the location belongs to a defective region, and "0" (generally black) indicates a defect-free region. The first defect mask can also be a probabilistic image, where each pixel in the image has a continuous value between 0 and 1, representing the confidence or probability that the location belongs to a defect. It can be further converted into a binary image by setting a threshold. The type of the first defect mask can be selected according to actual needs.

[0034] Step 102: Perform target processing on the defect image to obtain a defect sample image.

[0035] The target processing described here is used to mimic imaging interference factors (also known as imaging conditions) in defect detection images of printed materials, such as lighting, placement deformation, etc.

[0036] The embodiments of this application, by performing target processing on defect images, can make the defect images used for model training more closely resemble the shooting effect of defect detection images in real detection environments.

[0037] Among them, the defect image after target processing is the defect sample image.

[0038] After obtaining the defect sample image, geometric correction and color space unification with the standard template image can be performed on the defect sample image.

[0039] Step 103: Train the target model to be trained based on the defect sample image, the first defect mask, and the standard template image of the printed material.

[0040] The standard template image mentioned here refers to a precise, noise-free template image for printing.

[0041] In this embodiment, the original design file of the printed matter (usually in PDF format) can be accurately converted into a high-resolution image using a high-fidelity rendering engine and saved as a lossless format (such as PNG or TIFF). Then, by combining Optical Character Recognition (OCR) technology and edge detection technology, key text and image regions (i.e., the printed matter image) are extracted from the aforementioned high-resolution image to obtain a standard template image. This embodiment can obtain corresponding standard template images for different printed matters, thereby establishing a standard template image library. Therefore, operators can upload a batch of original design files of printed matters to be inspected, covering different models, color modes (CMYK or RGB), resolutions, and sizes. The system accurately converts each PDF page into a high-resolution single-page image using a high-fidelity rendering engine and saves it as a lossless format, then extracts the standard template image from it and saves it to the image library.

[0042] The target model described here is used to detect printing defects in printed materials. This target model can be a contrastive learning model.

[0043] In this embodiment, both defect sample images and standard template images can be input into the target model, allowing the target model to learn the differences between the defect sample images and the standard template images, thereby predicting the defect regions in the defect sample images and outputting the prediction results. The prediction results are then compared with a first defect mask, which serves as the ground truth label, to determine the difference between the prediction results and the ground truth. Based on this difference, the model parameters are optimized to improve the model's prediction performance. The training process can be stopped when the difference between the prediction results and the ground truth reaches the convergence condition or when the number of model training iterations reaches a preset number.

[0044] When detecting printing defects in printed materials, it is necessary to acquire images of the printed materials. This is typically achieved using a camera, resulting in a defect detection image. Based on this image, it is then determined whether a printing defect exists. However, image acquisition quality is affected by imaging interference factors, such as ambient lighting, image background, and the placement of the printed materials. These interference factors may be misidentified as printing defects, affecting the accuracy of the detection results.

[0045] This application embodiment, through target processing of defect images, makes the defect images used for model training more closely resemble the actual shooting effect of defect detection images in a real detection environment. Furthermore, this application embodiment uses a standard template image of the printed matter as a reference image. Utilizing the precise and noise-free template information of the standard template image, the defect sample image is compared with the standard template image to locate the printing defect. Further, a first defect mask is used as the ground truth to determine the difference between the predicted result and the ground truth. Training the model in this way allows it to effectively learn the inherent structure and discriminative features of the data, thereby distinguishing between valid differences caused by real defects and interference differences caused by imaging interference factors such as background, lighting, and geometric deformation. When the model trained using this method performs printing defect detection, it can overcome the feature mismatch problem caused by differences in imaging conditions (such as lighting, angle, resolution, and lens distortion) between the actual acquired defect detection image and the standard template image. This allows for effective identification and classification of defect objects in images even against complex backgrounds, thereby improving model robustness, reducing false detection rate, and simultaneously improving defect detection accuracy and reducing false negative rate. Furthermore, the embodiments of this application do not require the manual design of complex feature extraction rules for each type of printed matter, have high generalization ability, and can adapt to the production environment with a wide variety of printed materials.

[0046] Optionally, the prediction results of the target model can be a heatmap or a defect mask.

[0047] A heatmap is a probability distribution map generated by a model corresponding to a spatial location in an input image. The color of each pixel in the heatmap represents the confidence or probability that the location belongs to a specific target (such as a defect, key point, or object category). In this embodiment, the color of each pixel in the heatmap output by the model represents the probability that a printing defect exists at that location. Heatmaps are interpretable intermediate results, facilitating the visualization of areas of interest for the model. By setting a threshold (e.g., probability > 0.5) in the heatmap, it can be converted into a binary defect mask to determine the precise boundaries of the defect.

[0048] If the prediction result is a heatmap, it can be converted into a defect mask using the method described above. If the prediction result is a defect mask, it can be directly compared with the first defect mask.

[0049] In some optional embodiments, the target processing in step 102 includes, but is not limited to, at least one of the following: performing brightness adjustment, contrast adjustment, saturation adjustment, Gaussian blur processing, geometric transformation processing, and sensor noise addition within a preset range. Specifically, brightness adjustment, contrast adjustment, and saturation adjustment are mainly used to simulate illumination as an imaging interference factor; Gaussian blur processing is mainly used to simulate background noise as an imaging interference factor; geometric transformation processing is mainly used to simulate printed material deformation as an imaging interference factor; and sensor noise addition is mainly used to simulate image blurring caused by the acquisition device as an imaging interference factor.

[0050] The preset range can be set based on the imaging interference factors of the defect detection image of the printed matter, and the preset range is different for different processing objects.

[0051] For example, brightness adjustment within a preset range can be a random gain adjustment within a range of ±30%; contrast adjustment within a preset range can be a random gain adjustment within a range of ±20%; saturation adjustment within a preset range can be a random gain adjustment within a range of ±20%; Gaussian blur processing within a preset range can be a random gain adjustment within a kernel size range of 3×3 to 7×7; geometric transformation processing within a preset range can be random rotation ±2°, horizontal or vertical translation ±2%, slight affine distortion, etc.; sensor noise addition within a preset range can be analogous to readout noise and quantization error of image acquisition equipment.

[0052] In this embodiment of the application, by performing target processing operations on the defect image, the defect image used for model training is made closer to the shooting effect of the defect detection image in the real detection environment, thereby improving the adaptability of the target model to the complex detection environment and posture deviation of printed materials.

[0053] In related technologies, model training requires a large amount of real defect image data. However, obtaining real defect images is difficult, and they are hard to generalize to new defects. In some optional embodiments of this application, to solve the problem of scarce real defect samples, the defect images used for model training can be simulated defect images generated based on standard template images of printed materials. The first defect mask is the defect mask corresponding to the simulated defect image.

[0054] Alternatively, a programmable defect simulation system can be used to generate simulated defect images with realistic defect features, as described below.

[0055] Before step 101: obtaining the defect image of the printed matter and the first defect mask corresponding to the defect image, the method may further include steps A1 and A2, as described below: Step A1: Input the standard template image of the printed material into the preset defect simulation system to generate at least one type of first defect mask.

[0056] In this application embodiment, a standard template image of printed matter can be used as input to a preset defect simulation system. The system randomly generates at least one type of first defect mask, which may include, but is not limited to, at least one of the following: linear scratch defects, local dents or wrinkles, dotted stains, ink diffusion defects, and ink loss defects.

[0057] For linear scratch defects, slender, irregular scratch paths can be generated based on Bézier curves or L-system models.

[0058] For localized unevenness or wrinkles, a deformation field can be used to simulate the physical compression effect.

[0059] For point-like stain defects, they can be formed by randomly scattering points using fractal noise or Poisson distribution.

[0060] For ink diffusion or missing defects, morphological expansion and corrosion operations can be combined to simulate printing anomalies.

[0061] Optionally, in the generated simulated defect image, the defect area can be limited to the key content area, so that the defect can cover the main information structure of the printed matter and be accurately simulated.

[0062] Step A2: Based on the first defect mask, generate the corresponding type of defect in the standard template image to obtain the defect image of the printed material.

[0063] After obtaining the first defect mask, the position of the simulated defect in the standard template image can be determined based on the first defect mask, thereby generating the corresponding type of defect in the standard template image and obtaining the defect image of the printed material.

[0064] The above method can generate realistic "pseudo-defect" samples while maintaining the consistency of the overall structure, thus solving the problem of scarce real defect data.

[0065] Optionally, one type of defect can be generated in a standard template image, and the resulting simulated defect image, along with the corresponding standard template image and defect mask, can be saved as a set of data to allow the model to learn each type of printing defect more effectively. Understandably, multiple types of defects can also be generated in a standard template image to simulate the occurrence of multiple types of defects in the same printed material, enabling the model to handle more complex printing defects.

[0066] Optionally, step A2: generating the corresponding defect in the standard template image based on the first defect mask to obtain the defect image of the printed matter may include steps A21 to A24, as described below: Step A21: Determine the defect region in the standard template image that corresponds to the first defect mask.

[0067] Step A22: Determine the minimum bounding rectangle of the defect region.

[0068] Step A23: Determine the average RGB value of all pixels within the bounding box of the smallest outer rectangle.

[0069] Step A24: Fill the defect area with the average RGB value to generate the corresponding type of defect and obtain the defect image.

[0070] For defect regions located in the standard template image based on the first defect mask, a context-aware filling strategy can be used to generate the corresponding defects. Specifically, the minimum bounding rectangle of the defect region is first calculated, then the average RGB value of all pixels within the minimum bounding rectangle is taken, and finally, this average value is used to fill the entire defect region to generate the corresponding type of defect, thus obtaining the simulated defect image. This method is relatively simple and can quickly generate simulated defects, improving the efficiency of simulated defect image generation.

[0071] In some alternative embodiments, the target model can be a two-branch contrastive learning model. This two-branch contrastive learning model includes a first feature extraction branch and a second feature extraction branch, which share the weights of a Vision Transformer (ViT) backbone network. The first and second feature extraction branches can be a dual-input encoder structure, receiving a standard template image and a corresponding defect sample image, respectively, and extracting features from the two frames of images.

[0072] Based on this, step 103: train the target model to be trained according to the defect sample image, the first defect mask, and the standard template image of the printed matter. This may include steps B1 to B4, as described below: Step B1: Input the standard template image into the first feature extraction branch to obtain the first feature, and input the defect sample image into the second feature extraction branch to obtain the second feature.

[0073] Step B2: Determine the second defect mask corresponding to the defect sample image based on the first feature and the second feature.

[0074] In this embodiment, the first feature of the standard template image and the second feature of the defect sample image can be compared and analyzed to determine the second defect mask corresponding to the defect sample image. This second defect mask can be obtained by converting a heatmap predicted by the model.

[0075] In this embodiment, the spatial resolution can be gradually restored through a Transformer decoder or a multi-layer upsampling CNN structure, and the skip connection features from each layer of ViT can be fused to finally output a pixel-level defect probability map.

[0076] Step B3: Calculate the first loss value based on the second defect mask and the first defect mask.

[0077] After obtaining the second defect mask corresponding to the defect sample image, the first loss value between the second defect mask and the first defect mask can be calculated based on a preset loss function (such as the SoftDice loss function).

[0078] Step B4: Adjust the model parameters of the target model based on the first loss value.

[0079] Finally, the network parameters can be optimized through backpropagation based on the first loss value, thereby improving the model's prediction performance. This model training method can achieve pseudo-supervised training using simulated defect sample images even in scenarios without real-world annotations.

[0080] Optionally, after obtaining the first feature and the second feature, the method may further include steps B5 and B6, as described below: Step B5: Calculate the cross-correlation loss and mean squared error loss between the first and second features.

[0081] Step B6: Adjust the model parameters of the target model based on the cross-correlation loss and mean squared error loss.

[0082] In this embodiment, similarity loss can also be calculated between the first feature of the standard template image and the second feature of the defective sample image. The similarity loss described here may include, but is not limited to, registration loss and mean squared error loss (MSE loss). Then, based on the registration loss and MSE loss, the model parameters of the target model are adjusted to optimize the model and improve its prediction performance.

[0083] Among them, cross-correlation loss is used to measure the similarity between two signals (such as images or feature vectors); mean squared error loss is the average of the squares of the differences between the model's predicted value and the true value.

[0084] Optimizing model parameters based on cross-correlation loss allows the model to focus more on measuring the structural similarity and overall distribution consistency between features. This loss is inherently invariant to linear transformations such as changes in illumination intensity and overall contrast shifts, effectively guiding the model to learn robust representations of essential features to fluctuations in imaging conditions. Simultaneously, mean squared error loss, by calculating the point-by-point squared error between feature vectors, forces the model to focus on the accurate restoration of detailed features and precise alignment of spatial locations, ensuring that subtle differences in local defect regions are sensitively captured and preserved.

[0085] Optionally, before calculating the similarity loss between the first feature and the second feature, the first feature can be input into a spatial transformer connected to the first feature extraction branch. This transformer can automatically predict a set of transformation parameters (such as affine transformation) based on the input features and perform corresponding rotation, scaling, and shearing operations on the feature map to achieve automatic alignment between the two branch feature maps, reduce some positional differences, and improve the accuracy of similarity loss calculation.

[0086] Optionally, to enhance the sensitivity to local differences, the target model may also include a self-attention difference enhancement module (SEM) connected to the first feature extraction branch and the second feature extraction branch respectively. This module is used to filter and amplify the effective differences in image features caused by real defects, while suppressing the interference differences caused by non-defect factors.

[0087] Based on this, step B2: Determine the second defect mask corresponding to the defect sample image according to the first feature and the second feature, which may include steps B21 to B22, as described below: Step B21: Input the first feature and the second feature into the self-attention difference enhancement module to determine the effective difference between the first feature and the second feature.

[0088] Step B22: Based on the valid differences, obtain the second defect mask corresponding to the defect sample image.

[0089] In this embodiment, the first and second features can be input into the attention-based differential enhancement module. This module integrates similarity modeling and residual differential activation mechanisms to explicitly amplify subtle local differences, suppress redundant background information, significantly enhance the model's sensitivity to minor defects, effectively capture subtle texture shifts under semantic consistency, and suppress background interference. The differential-aware feature map is output using SEM and combined with a multi-scale decoding head structure to reconstruct spatial details, ultimately generating a high-resolution defect heatmap or binary mask for accurate localization. This is suitable for detecting difficult-to-detect printing defects such as small scratches and edge burrs.

[0090] Optionally, the dual-branch contrastive learning model architecture used in the embodiments of this application can be as follows: Figure 2 As shown, each feature extraction branch can include multiple feature coding layers.

[0091] Standard template images and defect sample images are input into the first and second feature extraction branches, respectively. After processing by the patch merging module, image patches are obtained. These image patches are then passed through linear embedding layers to convert them into initial feature vectors that the model can process. The initial feature vectors undergo deep feature encoding through multiple Swin transform blocks, the core of which is a self-attention mechanism based on windows and shift windows to efficiently capture long-distance dependencies. After each set of Swin transform blocks, the feature map is downsampled (reducing spatial size) and expanded through the patch merging module, thus constructing a multi-level feature pyramid from high resolution and low semantics to low resolution and high semantics. This process is repeated in the left and right branches, generating one-to-one corresponding multi-scale feature maps.

[0092] The Patch Merging module in the diagram is a downsampling operation. Its core purpose is to gradually reduce the spatial resolution (size) of the feature map while increasing the number of channels (feature dimension) during the Transformer's image processing, thereby constructing a hierarchical feature pyramid. In other words, it is responsible for gradually integrating fine-grained local information into coarse-grained semantic information. In summary, Patch Merging is used for downsampling and channel expansion of feature maps.

[0093] The linear embedding layer in the diagram is used to transform the pixel blocks after image segmentation into fixed-dimensional feature vectors (embedding vectors) through a learnable linear projection (fully connected layer).

[0094] The Swin-transformer block in the diagram is the core unit for feature extraction. It employs window-based multi-head self-attention and shift-window multi-head self-attention mechanisms to achieve powerful local and global contextual feature encoding while ensuring computational efficiency. The stacking of multiple blocks forms the network backbone.

[0095] The ASPP (Spatial Pyramid Pooling) in the diagram is a multi-scale contextual information aggregator. By using multiple parallel dilated convolutions with different dilation rates, it simultaneously captures local details of features and broader contextual information, making the model robust to defects of different sizes.

[0096] Upsampling in the figure is used to enlarge the spatial size of low-resolution feature maps.

[0097] This application also provides a printing defect detection method applied to a second electronic device. The second electronic device can be a terminal device, such as a mobile phone, computer, vehicle-mounted computer, or detection equipment, or it can be a server, such as a cloud server. The second electronic device can be the same as or different from the first electronic device.

[0098] The method is described in detail below through some embodiments. The following embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0099] like Figure 3 As shown, the printing defect detection method may include steps 301 to 302, as described below.

[0100] Step 301: Obtain defect detection images of the printed matter to be inspected.

[0101] The defect detection images described here can be acquired by an image acquisition device (such as a camera) and then sent to a second electronic device. For example, a high-definition camera is installed on a printing defect detection production line. This high-definition camera establishes a communication connection with the second electronic device. After the high-definition camera acquires a defect detection image of the printed material, it sends it to the second electronic device through this communication connection. The communication connection between the two can be a wired or wireless communication connection, which can be set according to actual needs. Alternatively, the defect detection image can be acquired by an image acquisition device installed on the second electronic device.

[0102] Step 302: Input the defect detection image and the standard template image of the printed matter to be inspected into the target model, and output the defect detection result of the printed matter to be inspected through the target model.

[0103] The target model described here is obtained by training using the model training method described above.

[0104] In this embodiment, the difference between the standard template image and the defect detection image can be used for self-supervised feature learning, thereby achieving accurate defect localization.

[0105] In some alternative embodiments, this application also provides a printing defect detection system for the printing defect detection method, which can be installed in a second electronic device.

[0106] The system may include: an image acquisition unit, a template management module, a simulation engine module, a deep learning processing unit, and a human-computer interaction interface.

[0107] The image acquisition unit can be an industrial high-definition camera, a light source control system, etc. The template management module can be used for PDF parsing of original printed documents, image standardization, and ROI annotation. The simulation engine module can be used for simulation defect modeling, data augmentation processing, and training data pairing generation, primarily for the model training process. The deep learning processing unit can be used for model training and inference. The human-computer interaction interface can be used to realize defect visualization, alarm prompts, result storage, and traceability.

[0108] Optionally, the system also supports manual feedback during the online detection process. That is, when a new type of defect is confirmed, it can be included in the defect simulation library and the model can be retrained to form a continuous evolution loop of "detection → accumulation → optimization", thereby improving the long-term availability and generalization ability of the system.

[0109] Finally, in the embodiments of this application, through Figure 4 The model training and inference processes provided in the embodiments of this application are summarized below.

[0110] Based on the original design files of the printed material, a standard template image is generated. Then, a simulated defect image is generated based on the standard template image. Next, adaptive model learning is performed based on the standard template image and the simulated defect image. After model training is complete, the model is exported for inference applications. During model inference, defect detection images acquired through image acquisition equipment and the corresponding standard template images are input into the pre-trained model, which performs inference analysis and outputs prediction results. Finally, based on the prediction results, it is determined whether the printed material has printing defects. If printing defects are found, an alarm is triggered; otherwise, the process ends.

[0111] In summary, the technical solution provided in this application, through the technical route of "standard template image generation → simulated generation of diverse defect sample images → defect sample image enhancement processing → construction of an adaptive contrastive learning network → pixel-level defect localization," achieves two advantages: First, it enables the model to effectively learn the inherent structure and discriminative features of the data, thereby distinguishing between valid differences caused by real defects and interference differences caused by imaging interference factors such as background, lighting, and geometric deformation. Second, it overcomes the limitations of traditional methods that rely on a large amount of real defect data and are difficult to generalize to new products. In conclusion, this technical solution has the advantages of high automation, low maintenance cost, and strong robustness, and is particularly suitable for quality monitoring scenarios in high-speed printing production lines.

[0112] Exemplary device Accordingly, this application also provides a model training device applied to a first electronic device, which may be a terminal device, such as a mobile phone, computer, vehicle computer, or detection device, or a server, such as a cloud server.

[0113] like Figure 5 As shown, the device may include: The first acquisition module 501 is used to acquire a defect image of the printed matter and a first defect mask corresponding to the defect image.

[0114] Image processing module 502 is used to perform target processing on the defect image to obtain a defect sample image. The target processing is used to mimic imaging interference factors in the defect detection image of the printed material.

[0115] The model training module 503 is used to train the target model to be trained based on the defect sample image, the first defect mask, and the standard template image of the printed matter.

[0116] The target model is used to detect printing defects in printed materials, and the standard template image is used as a comparison reference image.

[0117] In some optional embodiments, the target processing includes at least one of the following: performing brightness adjustment, contrast adjustment, saturation adjustment, Gaussian blur processing, geometric transformation processing, and sensor noise addition within a preset range.

[0118] The preset range is set based on the imaging interference factors of the defect detection image of the printed matter, and the preset range is different for different processing objects.

[0119] In some alternative embodiments, the apparatus may further include: The defect simulation module is used to input the standard template image of the printed matter into a preset defect simulation system to generate at least one type of the first defect mask.

[0120] The type of the first defect mask includes at least one of the following: linear scratch defect, local bump or wrinkle defect, dot stain defect, ink diffusion defect, and ink loss defect.

[0121] The defect image generation module is used to generate a defect of the corresponding type in the standard template image based on the first defect mask, so as to obtain a defect image of the printed matter.

[0122] In some alternative embodiments, the defect image generation module may include: The first determining unit is used to determine the defect region in the standard template image that corresponds to the first defect mask.

[0123] The second determining unit is used to determine the minimum bounding rectangle of the defect region.

[0124] The third determining unit is used to determine the average RGB value of all pixels within the minimum bounding rectangle.

[0125] The defect image generation unit is used to fill the defect area with the average RGB value to generate a defect of the corresponding type, thereby obtaining the defect image.

[0126] In some alternative embodiments, the target model is a two-branch contrastive learning model, which includes a first feature extraction branch and a second feature extraction branch, wherein the first feature extraction branch and the second feature extraction branch share the weights of the visual self-attention backbone network.

[0127] The model training module 503 may include: The feature extraction unit is used to input the standard template image into the first feature extraction branch to obtain a first feature, and to input the defect sample image into the second feature extraction branch to obtain a second feature.

[0128] The defect reasoning unit is used to determine the second defect mask corresponding to the defect sample image based on the first feature and the second feature.

[0129] The loss calculation unit is used to calculate a first loss value based on the second defect mask and the first defect mask.

[0130] The model optimization unit is used to adjust the model parameters of the target model based on the first loss value.

[0131] In some alternative embodiments, the apparatus may further include: The loss calculation module is used to calculate the cross-correlation loss and mean squared error loss between the first feature and the second feature.

[0132] The model optimization module is used to adjust the model parameters of the target model based on the cross-correlation loss and the mean squared error loss.

[0133] In some optional embodiments, the target model further includes a self-attention difference enhancement module connected to the first feature extraction branch and the second feature extraction branch respectively. The self-attention difference enhancement module is used to filter and amplify the effective differences in image features caused by real defects, while suppressing the interference differences caused by non-defect factors.

[0134] The defect reasoning unit can be specifically used to: input the first feature and the second feature into the self-attention difference enhancement module, determine the effective difference between the first feature and the second feature; and obtain the second defect mask corresponding to the defect sample image based on the effective difference.

[0135] The model training apparatus provided in this embodiment belongs to the same concept as the model training method provided in the above embodiments of this application. It can execute the model training method provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects of the execution method. Technical details not described in detail in this embodiment can be found in the specific processing content of the model training method provided in the above embodiments of this application, and will not be repeated here.

[0136] Accordingly, this application also provides a printing defect detection device, which is applied to a second electronic device. The first electronic device can be a terminal device, such as a mobile phone, computer, vehicle computer, detection equipment, etc., or it can be a server, such as a cloud server.

[0137] like Figure 6 As shown, the device may include: The second acquisition module 601 is used to acquire defect detection images of the printed matter to be inspected.

[0138] The model inference module 602 is used to input the defect detection image and the standard template image of the printed matter to be detected into the target model, and output the defect detection result of the printed matter to be detected through the target model.

[0139] The target model is obtained by training using the model training method described above.

[0140] The printing defect detection device provided in this embodiment belongs to the same concept as the printing defect detection method provided in the above embodiments of this application. It can execute the printing defect detection method provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in this embodiment can be found in the specific processing content of the printing defect detection method provided in the above embodiments of this application, and will not be repeated here.

[0141] It should be understood that the modules in the above map-using device can be implemented by a processor calling software. For example, the device includes a processor connected to a memory containing instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of each unit of the device. The processor can be a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal or external to the device. Alternatively, the units in the device can be implemented as hardware circuits. By designing the hardware circuits, some or all of the unit functions can be implemented. The hardware circuit can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are implemented by designing the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented by a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration files to implement the functions of some or all of the above units. All units of the above device can be implemented entirely by a processor calling software, entirely by hardware circuits, or partially by a processor calling software with the remaining parts implemented by hardware circuits.

[0142] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0143] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0144] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a System-on-Chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0145] Exemplary electronic devices This application also provides an electronic device, such as... Figure 7 As shown, the electronic device includes a memory 700 and a processor 710.

[0146] The memory 700 is connected to the processor 710 and is used to store programs.

[0147] The processor 710 is used to implement the model training method or printing defect detection method in the above embodiments by running the program stored in the memory 700.

[0148] Specifically, the aforementioned electronic device may also include: a communication interface 720, an input device 730, an output device 740, and a bus 750.

[0149] The processor 710, memory 700, communication interface 720, input device 730, and output device 740 are interconnected via a bus. Among them: Bus 750 may include a pathway for transmitting information between various components of a computer system.

[0150] The processor 710 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0151] The processor 710 may include a main processor, as well as a baseband chip, modem, etc.

[0152] The memory 700 stores a program that executes the technical solution of this invention, and may also store an operating system and other key business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 700 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0153] Input device 730 may include a device for receiving data and information input by a user, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0154] Output device 740 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0155] The communication interface 720 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0156] The processor 710 executes the program stored in the memory 700 and calls other devices, which can be used to implement the various steps in the model training method or printing defect detection method provided in the above embodiments of this application.

[0157] Exemplary computer program products and storage media In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the model training method or printing defect detection method described in the embodiments of this application.

[0158] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0159] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0160] Furthermore, embodiments of this application may also be storage media storing computer programs, which are executed by a processor from the steps of the model training method or printing defect detection method described in the embodiments of this application.

[0161] Furthermore, embodiments of this application may also be chips, which include processors and data interfaces. The processor reads instructions stored in the memory through the data interface to execute the steps in the model training method or printing defect detection method described in the embodiments of this application.

[0162] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0163] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0164] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0165] The modules and sub-modules in the devices and terminals in the various embodiments of this application can be merged, divided, and deleted according to actual needs.

[0166] It should be understood that the disclosed terminals, devices, and methods can be implemented in other ways, given the several embodiments provided in this application. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0167] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0168] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0169] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0170] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0171] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A model training method, characterized in that, The method includes: Obtain a defect image of the printed material and a first defect mask corresponding to the defect image; The defect image is subjected to target processing to obtain a defect sample image; wherein, the target processing is used to mimic the imaging interference factors of the defect detection image of the printed matter; The target model to be trained is trained based on the defect sample image, the first defect mask, and the standard template image of the printed matter; wherein the target model is used to detect printing defects in the printed matter, and the standard template image is used as a comparison reference image.

2. The model training method according to claim 1, characterized in that, The target processing includes at least one of the following: performing brightness adjustment, contrast adjustment, saturation adjustment, Gaussian blur processing, geometric transformation processing, and sensor noise addition within a preset range; The preset range is set based on the imaging interference factors of the defect detection image of the printed matter, and the preset range is different for different processing objects.

3. The model training method according to claim 1, characterized in that, Before acquiring the defect image of the printed matter and the first defect mask corresponding to the defect image, the method further includes: The standard template image of the printed matter is input into a preset defect simulation system to generate at least one type of first defect mask; wherein, the type of the first defect mask includes at least one of the following: linear scratch defect, local bump or wrinkle defect, dot stain defect, ink diffusion defect, and ink loss defect. Based on the first defect mask, a defect of the corresponding type is generated in the standard template image to obtain a defect image of the printed matter.

4. The model training method according to claim 3, characterized in that, The step of generating a corresponding defect in the standard template image based on the first defect mask to obtain a defect image of the printed matter includes: Determine the defect region in the standard template image that corresponds to the first defect mask; Determine the minimum bounding rectangle of the defect region; Determine the average RGB value of all pixels within the minimum bounding rectangle; The average RGB value is filled into the defect area to generate a defect of the corresponding type, thus obtaining the defect image.

5. The model training method according to claim 1, characterized in that, The target model is a two-branch contrastive learning model, which includes a first feature extraction branch and a second feature extraction branch. The first feature extraction branch and the second feature extraction branch share the weights of the visual self-attention backbone network. The step of training the target model to be trained based on the defect sample image, the first defect mask, and the standard template image of the printed matter includes: The standard template image is input into the first feature extraction branch to obtain the first feature, and the defect sample image is input into the second feature extraction branch to obtain the second feature; Based on the first feature and the second feature, determine the second defect mask corresponding to the defect sample image; Calculate the first loss value based on the second defect mask and the first defect mask; Based on the first loss value, adjust the model parameters of the target model.

6. The model training method according to claim 5, characterized in that, After obtaining the first feature and the second feature, the method further includes: Calculate the cross-correlation loss and mean squared error loss between the first feature and the second feature; The model parameters of the target model are adjusted based on the cross-correlation loss and the mean squared error loss.

7. The model training method according to claim 5, characterized in that, The target model further includes: a self-attention difference enhancement module connected to the first feature extraction branch and the second feature extraction branch respectively. The self-attention difference enhancement module is used to filter and amplify the effective differences in image features caused by real defects, while suppressing the interference differences caused by non-defect factors. The step of determining the second defect mask corresponding to the defect sample image based on the first feature and the second feature includes: The first feature and the second feature are input into the self-attention difference enhancement module to determine the effective difference between the first feature and the second feature; Based on the effective differences, a second defect mask corresponding to the defect sample image is obtained.

8. A method for detecting printing defects, characterized in that, The method includes: Acquire defect detection images of the printed material to be inspected; The defect detection image and the standard template image of the printed matter to be inspected are input into the target model, and the defect detection result of the printed matter to be inspected is output by the target model; wherein, the target model is trained by the model training method according to any one of claims 1 to 7.

9. An electronic device, characterized in that, include: Memory and processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the method as described in any one of claims 1 to 8 by running a program in the memory.

10. A computer program product, characterized in that, The computer program product stores a computer program, which, when executed by a processor, implements the method as described in any one of claims 1 to 8.

Images