A method, apparatus, and readable medium for detecting defects in a printed product

Abstract

The present application relates to the technical field of printed product quality detection, and in particular to a printed product defect detection method, device and readable medium, which generates a matching grid automatically by grid division, marking and merging of a printed product template to solve the problem of limited adaptation scenarios and high misjudgment rate caused by dependence on a preset defect template; the design of calculating an enhanced difference image between the printed product template and the printed product to be detected to highlight defects and combining a defect detection model to detect defects in the enhanced difference image automatically detects defects in the printed product, solves the misjudgment and missed detection caused by fixed threshold or shape matching in the prior art, and improves the precision and efficiency of printed product defect detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of printed product quality detection, and in particular to a printed product defect detection method, device and readable medium. BACKGROUND

[0002] In the printed product defect detection scenarios such as first piece detection, warehouse inspection and material inspection, manual comparison or traditional machine vision is often used for defect detection.

[0003] However, manual comparison has the problems of low efficiency and large error, which seriously affects the production / storage efficiency, which is specifically embodied in:

[0004] (1) First piece detection scenario: after the design document (PDF / CAD source file) is issued, the first printed product needs to be compared with the electronic file region by region (text, pattern, barcode, etc.). It takes 5-10 minutes for a single A4 paper. The time-consuming manual comparison prolongs the first piece verification period (such as the production line needs to wait for the first piece to be confirmed before starting), which directly affects the subsequent production schedule; and manual judgment depends on experience, which is easy to miss small defects (such as scratches and overprint deviation), resulting in large-scale defective products in batch production.

[0005] (2) Warehouse inspection scenario: when the warehouse receives the goods from the printing factory, it needs to inspect the consistency of the printed product and the electronic file (such as whether the version is correct and whether the content is misprinted). Manual checking needs to check and compare each piece, which takes a long time and cannot meet the rapid warehouse demand (especially during the e-commerce promotion period); subjective judgment is easily affected by fatigue, which may misjudge the defective products as qualified products, resulting in problem batches entering the warehouse.

[0006] (3) Material inspection scenario: when the workshop takes materials, it needs to check the electronic file corresponding to the material number. Workers need to carry paper electronic files or access computer files one by one, which is time-consuming and prone to errors; if the wrong material is taken, it may cause the whole production line to stop and rework due to material errors. In addition, manual inspection is not sensitive to small defects (such as overprint deviation and light scratches), which may use defective materials for production, leading to customer complaints and rework.

[0007] Traditional machine vision has the problems of poor adaptability, high misjudgment rate and poor user experience, and cannot meet the needs of multiple scenarios, which is specifically embodied in:

[0008] (1) Existing automated detection solutions are mostly based on fixed thresholds or shape matching (such as normalized cross correlation NCC). Printed products have global or local differences with electronic files due to paper stretching (±0.05mm deformation caused by humidity changes), uneven scanner lighting, and the like. Fixed thresholds may misjudge normal deformation as defects (such as slight gray changes at the edges of text), and NCC matching may easily mismatch in repeated texture areas (such as packaging box patterns), leading to missed real defects (such as 0.2mm offset of a LOGO position).

[0009] (2) Traditional solutions rely on preset defect templates (such as scratch length ≥1mm, stain area ≥5mm²), but background textures and color depths of different batches of printed products differ. Fixed thresholds may misjudge normal batch differences as defects (such as a slight stain on a light-colored paper being misjudged as a dirty spot), and cannot identify complex defects (such as 0.1mm-level micro scratches), leading to problem batches not being intercepted. SUMMARY

[0010] The present application aims to solve the above-mentioned technical problems.

[0011] In a first aspect, the present application provides a printed product defect detection method, comprising the following steps:

[0012] obtaining a printed product template and dividing the printed product template into a plurality of grids;

[0013] extracting edge features of images in each grid, determining whether the edge features of images in each grid meet a condition, if yes, marking the grid as a fixed grid, otherwise marking the grid as a to-be-merged grid;

[0014] iterating through all to-be-merged grids, performing a first merging on the to-be-merged grids and the closest fixed grids to obtain first merged grids, and performing a second merging on all first merged grids containing the same fixed grid to obtain second merged grids, taking all second merged grids, first merged grids not subjected to the second merging, and fixed grids not subjected to the first merging as matching grids;

[0015] obtaining a printed product to be detected, performing matching and alignment on each matching grid and the printed product to be detected by using a template matching algorithm, and calculating an enhanced difference image between each matching grid of the printed product template and a region subjected to matching and alignment in the printed product to be detected;

[0016] constructing and training a defect detection model to obtain a trained defect detection model, inputting the enhanced difference image into the trained defect detection model for defect detection, and obtaining defects of the printed product to be detected.

[0017] Preferably, the edge features of the image within each grid are extracted, and it is determined whether the edge features of the image within each grid meet the conditions. If they do, the grid is marked as a fixed grid; otherwise, the grid is marked as a grid to be merged. Specifically, this includes:

[0018] The Canny operator is used to extract the edge features of the image within each grid. The edge features include the number, length, width, and density of the edges.

[0019] Determine whether each edge feature within each grid is within the corresponding preset parameter range. If so, mark the grid as a fixed grid; otherwise, mark the grid as a grid to be merged.

[0020] Preferably, the enhanced difference image between each matching grid of the printed product template and the matching region in the printed product to be detected is calculated, specifically including:

[0021] The difference in color channel C between each matching grid in the printed product template and the corresponding pixel in the matching region of the printed product to be inspected is calculated as follows:

[0022] ;

[0023] in, This represents the difference in color channel C between each matching grid in the printed product template and the corresponding pixel within the matching area of ​​the printed product to be inspected. This represents each matching grid in the printed product template. The pixel value of the pixel at position C in color channel C. This indicates the area where the printed product to be inspected matches the alignment. The pixel value of the pixel at the location in color channel C, which includes the R channel, G channel and B channel;

[0024] The color channel C difference between each matching grid in the printed product template and the corresponding pixel in the matching area of ​​the printed product to be inspected is offset to obtain the offset difference result, as shown in the following formula:

[0025] ;

[0026] in, This represents the offset difference result;

[0027] The offset difference result is saturated to obtain the saturated result, as shown in the following formula:

[0028] ;

[0029] in, This indicates the result of the saturation treatment;

[0030] generate an enhanced difference image according to the saturation processing results of all color channels C of each pixel point, the enhanced difference image comprising a neutral gray area and a color difference area, wherein the saturation processing results of all color channels C of each pixel point in the neutral gray area are all 128, and there is no defect in the neutral gray area; and the saturation processing result of at least one color channel of at least one pixel point in the color difference area is not 128, and there is a defect and / or a color difference in the color difference area.

[0031] As preferred, the color difference area comprises a white area, a black area and other color areas, the saturation processing results of all color channels C of each pixel point in the white area are all 255, and there is a multi-printing defect in the white area; the saturation processing results of all color channels C of each pixel point in the black area are all 0, and there is a missing printing defect in the black area; and the saturation processing result of at least one color channel of at least one pixel point in the other color areas is not 0, 128 or 255, and there is a multi-printing defect, a missing printing defect and / or a color difference in the other color areas.

[0032] As preferred, a template matching algorithm is used to match and align each matching grid with the printing product to be detected, specifically comprising:

[0033] create a matching template according to the edge features of the fixed grid in each matching grid, the matching template comprising a template center point, and the template angle of the matching template being 0°;

[0034] match and search the matching template corresponding to each matching grid on the printing product to be detected to obtain a matching result, the matching result comprising a matching center point and a rotation angle;

[0035] construct a transformation matrix M according to the template center point, the matching center point and the rotation angle, as shown in the following formula:

[0036] ;

[0037] wherein θ represents the rotation angle, and clockwise rotation is positive; , ) represents the coordinates of the template center point; , ) represents the coordinates of the matching center point; and respectively represent the translation components of the matching grid relative to the matching result in the x direction and the y direction;

[0038] perform affine transformation on the printing product template according to the transformation matrix M, so that the printing product template coincides with the printing product to be detected to complete the alignment.

[0039] As preferred, the defect detection model is constructed and trained to obtain the trained defect detection model, and specifically includes:

[0040] The defect detection model based on YOLOv12 is constructed.

[0041] The printed product data is collected, and the geometric augmentation method and the photometric augmentation method are used to perform data augmentation on the printed product data to obtain augmented printed product data, wherein the geometric augmentation method includes rotation, horizontal flipping, vertical flipping and scaling, and the photometric augmentation method specifically includes adjusting brightness, adjusting contrast and adding Gaussian noise.

[0042] The enhanced difference image of each printed product in the printed product template and the augmented printed product data is calculated, and the position and type of the defect in the enhanced difference image are labeled to generate a defect label, wherein the type of the defect includes multiple printing, missing printing and offset, the enhanced difference image and the corresponding defect label constitute training data, and the training data is divided into a training set and a validation set.

[0043] The defect detection model is supervised trained using the training set, and verified on the validation set to obtain the trained defect detection model.

[0044] In a second aspect, the present application provides a printed product defect detection device, comprising:

[0045] The grid division module is configured to obtain the printed product template and divide the printed product template into a plurality of grids.

[0046] The grid marking module is configured to extract the edge features of the image in each grid, determine whether the edge features of the image in each grid meet the condition, if yes, mark the grid as a fixed grid, otherwise mark the grid as a to-be-merged grid.

[0047] The grid merging module is configured to traverse all the to-be-merged grids, perform a first merging on the to-be-merged grids and the closest fixed grid to obtain first merged grids, and perform a second merging on all the first merged grids containing the same fixed grid to obtain second merged grids, and all the second merged grids, the first merged grids not subjected to the second merging and the fixed grids not subjected to the first merging are taken as matching grids.

[0048] The difference image calculation module is configured to obtain the printed product to be detected, and calculate the enhanced difference image between each matching grid of the printed product template and the matched and aligned region in the printed product to be detected after matching and aligning each matching grid with the printed product to be detected using the template matching algorithm.

[0049] The defect detection module is configured to construct and train the defect detection model, obtain the trained defect detection model, input the enhanced difference image into the trained defect detection model for defect detection, and obtain the defects of the printed product to be detected.

[0050] In a third aspect, the present application provides an electronic device, comprising one or more processors; a memory for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the implementation manners of the first aspect.

[0051] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, when the computer program is executed by a processor, the method described in any of the implementation manners of the first aspect is implemented.

[0052] In a fifth aspect, the present application provides a computer program product, comprising a computer program, when the computer program is executed by a processor, the method described in any of the implementation manners of the first aspect is implemented.

[0053] Compared with the prior art, the present application has the following beneficial effects:

[0054] (1) The printing product defect detection method provided by the present application automatically detects the defects existing in the printed product by automatically generating matching grids for the printed product template, generating an enhanced difference image, and combining the design of the defect detection model, avoiding the problems of misjudgment and missed detection caused by fixed threshold or shape matching, and the problems of limited adaptation scene and high misjudgment rate caused by relying on a pre-set defect template.

[0055] (2) The printing product defect detection method provided by the present application divides the printed product template into a plurality of matching grids through grid division, grid marking and grid merging, realizes automatic partition creation of a matching template, and creates a matching template according to all the matching grids, and then matches and aligns with the printed product to be detected one by one, realizes full-plate detection, reduces missed detection and missed detection, and improves detection accuracy.

[0056] (3) The printing product defect detection method provided by the present application can identify small scratch defects by calculating the enhanced difference image to highlight defects and combining the trained defect detection model to identify defects, can reduce the misjudgment of small gray scale changes of text edges, background textures of different batches of printed products, and color difference, and is more sensitive to other small defects such as overprint deviation and shallow scratches, greatly reduces misjudgment and missed detection, and improves the accuracy of defect detection. BRIEF DESCRIPTION OF DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings.

[0058] Figure 1 Flowchart of the printing product defect detection method of the embodiments of the present application;

[0059] Figure 2 Schematic diagram of the grid division of the printing product template of the embodiments of the present application;

[0060] Figure 3 Schematic diagram of the grid marking of the printing product template of the embodiments of the present application;

[0061] Figure 4 Schematic diagram of the grid merging of the printing product template of the embodiments of the present application;

[0062] Figure 5 Schematic diagram of the first matching grid of the printing product template of the embodiments of the present application;

[0063] Figure 6 The printing product to be detected of the embodiments of the present application;

[0064] Figure 7 The enhanced difference image between the first matching grid of the printing product template of the embodiments of the present application and the matching area in the printing product to be detected;

[0065] Figure 8 Schematic diagram of the enhanced difference image between the matching grid of the printing product template of another embodiment of the present application and the matching area in the printing product to be detected, where there is a shift defect;

[0066] Figure 9 Schematic diagram of the printing product defect detection device of the embodiments of the present application;

[0067] Figure 10 Schematic diagram of the hardware structure of the electronic device provided by the embodiments of the present application.

[0068] Reference signs: 11, fixed grid; 12, grid to be merged; 13, matching grid; 14, white area; 15, black area; 16, other color area. DETAILED DESCRIPTION

[0069] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0070] Figure 1 A printed product defect detection method provided by an embodiment of the present application is shown, which comprises the following steps:

[0071] S1, a printed product template is acquired, and the printed product template is divided into a plurality of grids.

[0072] Specifically, the printed product template is gridded according to the size of the printed product template and the fine degree of the content. In an embodiment of the present application, the printed product template is divided into 25 red grids with reference to Figure 2 S2, edge features of an image in each grid are extracted, and it is judged whether the edge features of the image in each grid satisfy a condition. If yes, the grid is marked as a fixed grid; otherwise, the grid is marked as a to-be-merged grid.

[0073] S2, edge features of an image in each grid are extracted, and it is judged whether the edge features of the image in each grid satisfy a condition. If yes, the grid is marked as a fixed grid; otherwise, the grid is marked as a to-be-merged grid.

[0074] In a specific embodiment, edge features of an image in each grid are extracted, and it is judged whether the edge features of the image in each grid satisfy a condition. If yes, the grid is marked as a fixed grid; otherwise, the grid is marked as a to-be-merged grid, which specifically comprises:

[0075] The Canny operator is used to extract the edge features of the image in each grid, and the edge features include the number, length, width and tightness of the edges.

[0076] It is judged whether each edge feature in each grid is located in a corresponding preset parameter range. If yes, the grid is marked as a fixed grid; otherwise, the grid is marked as a to-be-merged grid.

[0077] Specifically, the Canny operator is used to extract the edge features of the image in each grid with reference to Figure 3 , the edge features include the number, length, width and tightness of the edges, and edge feature analysis is performed on all the grids in turn. The grid whose edge features satisfy the condition can be used to create a matching template in the following steps, and is marked as a fixed grid 11, as shown by the red grid marked with 1 in Figure 3 The grid whose edge features do not satisfy the condition is insufficient in edge features. If a matching result obtained by matching search according to the edge features of the grid is unreliable, the grid whose edge features do not satisfy the condition cannot be used to create a matching template, and is marked as a to-be-merged grid 12, as shown by the red grid marked with 2 in Figure 3The red grid shown in the figure is the minimum unit marked as 0. The edge features of the image in each grid are extracted by using the Canny operator, which is a prior art, and the specific process details are not described here.

[0078] S3, traversing all the to-be-merged grids, performing the first merging on the to-be-merged grids and the closest fixed grid to obtain first-merged grids, and performing the second merging on all the first-merged grids containing the same fixed grid to obtain second-merged grids, taking all the second-merged grids, the first-merged grids not subjected to the second merging, and the fixed grids not subjected to the first merging as matching grids.

[0079] Specifically, referring to Figure 3 and Figure 4 , the grid merging of the printing product template is performed according to step S3 to generate 13 matching grids 13, as shown in the green grid. Figure 4 Compared with the prior art that needs to manually construct a template area before detection, the matching grids 13 are automatically generated by using the printing product defect detection method proposed in the present application, avoiding the manual labor of dividing the matching grids and solving the problems of no template area detection and the need for manual modeling. Moreover, each matching grid 13 includes a fixed grid 11, so that the image in the matching grid 13 has sufficient edge features to match and align with the printing product to be detected, avoiding the situation that the to-be-merged grid with insufficient features cannot create a matching template and complete the matching alignment, avoiding the situation of insufficient edge features and inaccurate matching, and ensuring the accuracy of the matching alignment.

[0080] S4, obtaining the printing product to be detected, and calculating the enhanced difference image between each matching grid of the printing product template and the area matched and aligned in the printing product to be detected after each matching grid is matched and aligned with the printing product to be detected by using the template matching algorithm.

[0081] In a specific embodiment, each matching grid is matched and aligned with the printing product to be detected by using the template matching algorithm, specifically including:

[0082] creating a matching template according to the edge features of the fixed grid in each matching grid, the matching template including a template center point, and the template angle of the matching template being 0°;

[0083] performing a matching search on the printing product to be detected according to the matching template corresponding to each matching grid to obtain a matching result, the matching result including a matching center point and a rotation angle;

[0084] constructing a transformation matrix M according to the template center point, the matching center point, and the rotation angle, as shown in the following formula:

[0085] ;

[0086] Where θ represents the rotation angle, with counterclockwise being positive; , () represents the coordinates of the center point of the template; , () indicates the coordinates of the matching center point; and These represent the translation components of the matching grid relative to the matching result in the x and y directions, respectively.

[0087] Affine transformation is performed on the printed product template according to the transformation matrix M, so that the printed product template coincides with the printed product to be inspected to complete the alignment.

[0088] In a specific embodiment, calculating the enhanced difference image between each matching grid of the printed product template and the matching region in the printed product to be detected specifically includes:

[0089] The difference in color channel C between each matching grid in the printed product template and the corresponding pixel in the matching region of the printed product to be inspected is calculated as follows:

[0090] ;

[0091] in, This represents the difference in color channel C between each matching grid in the printed product template and the corresponding pixel within the matching area of ​​the printed product to be inspected. This represents each matching grid in the printed product template. The pixel value of the pixel at position C in color channel C. This indicates the area where the printed product to be inspected matches the alignment. The pixel value of the pixel at the location in color channel C, which includes the R channel, G channel and B channel;

[0092] The color channel C difference between each matching grid in the printed product template and the corresponding pixel in the matching area of ​​the printed product to be inspected is offset to obtain the offset difference result, as shown in the following formula:

[0093] ;

[0094] in, This represents the offset difference result;

[0095] The offset difference result is saturated to obtain the saturated result, as shown in the following formula:

[0096] ;

[0097] in, representing the saturation processing result;

[0098] generate an enhanced difference image according to the saturation processing result of all color channels C of each pixel point, the enhanced difference image comprising a neutral gray area and a color difference area, wherein the saturation processing result of all color channels C of each pixel point in the neutral gray area is 128, and no defect exists in the neutral gray area; and the saturation processing result of at least one color channel of at least one pixel point in the color difference area is not 128, and a defect and / or a color difference exists in the color difference area.

[0099] In specific embodiments, the color difference area comprises a white area, a black area and other color areas, the saturation processing result of all color channels C of each pixel point in the white area is 255, and a multi-print defect exists in the white area; the saturation processing result of all color channels C of each pixel point in the black area is 0, and a missing print defect exists in the black area; and the saturation processing result of at least one color channel of at least one pixel point in the other color areas is not 0, 128 or 255, and a multi-print defect, a missing print defect and / or a color difference exists in the other color areas.

[0100] Specifically, all the matching grids 13 are matched with the to-be-detected printed product one by one, and the enhanced difference image between each matching grid 13 of the printed product template and the matching area in the to-be-detected printed product is calculated, so as to finally complete the enhanced difference image between the entire printed product template and the to-be-detected printed product, and realize full-plate detection. In the embodiments of the present application, the to-be-detected printed product is obtained by a scanner. Since the size ratio of the to-be-detected printed product obtained by scanning and the printed product template is different, in the embodiments of the present application, the matching template created includes a scaling parameter, so that the size of each matching grid of the printed product template is the same as that of the matching area in the to-be-detected printed product.

[0101] Reference Figures 5 to 7 In the embodiments of the present application, the enhanced difference image obtained after the offset operation and saturation processing of the difference value of the color channel C of the corresponding pixel point in the matching area in the to-be-detected printed product corresponding to the first matching grid 13 of the printed product template comprises a neutral gray area, a white area 14 and a black area 15, wherein the white area 14 and the black area 15 can highlight a multi-print defect and a missing print defect, respectively.

[0102] Specifically, the printed product of the embodiments of the present application is a color printed product, and the enhanced difference image further comprises an other color area 16. Reference Figure 7The first matching grid 13 of the printed product template and the area within which they match and align with the printed product to be detected include other color areas 16. The process of calculating the enhanced difference image is explained using other color areas 16 as an example. The printed product to be detected is located within other color areas 16 in (…). , The pixel values ​​of the R, G, and B channels of pixel A at position ) are 255, 255, and 255 respectively. The printed product template is located in other color areas 16 within ( , The pixel values ​​of pixel A at position ) in the R, G, and B channels are 114, 123, and 162 respectively. Then:

[0103] ;

[0104] ;

[0105] ;

[0106] Then to , and The offset operation and saturation processing are performed separately to obtain the results. , and Therefore, pixel A in the enhanced difference image appears as a dark blue, different from the color of pixel A on the printed product template. Similarly, the saturation processing results of other pixels are calculated, and other color regions 16 are determined based on the saturation processing results of pixel A and other pixels. In an embodiment of this application, the color of other color regions 16 on the printed product to be detected is white, the same as the background color, thus indicating a printing defect. In a specific embodiment of this application, if the printed product is a black and white page, the enhanced difference image has at most neutral gray areas, black areas, and white areas; if the printed product is a color page, the enhanced difference image includes neutral gray areas, black areas, white areas, and other color areas.

[0107] In another embodiment of this application, if a series of consecutive pixels in the enhanced difference image contain both pixels in white areas and pixels in black areas, then the series of consecutive pixels in the enhanced difference image have an offset defect in the region formed by the enhanced difference image, such as... Figure 8 As shown in the yellow box.

[0108] In actual production, the background texture and color depth of different batches of printed products are different. Compared with using a fixed threshold to judge defects, the printed product defect detection method proposed in the application highlights the regions with defects and regions with color differences by calculating the enhanced difference image, which can facilitate subsequent trained defect detection model for defect recognition, improve detection efficiency, and reduce the misjudgment of normal differences between different batches of printed products as defects, thereby improving defect detection accuracy.

[0109] S5, constructing and training a defect detection model to obtain a trained defect detection model, inputting the enhanced difference image into the trained defect detection model for defect detection to obtain defects of the printed product to be detected.

[0110] In specific embodiments, the defect detection model is constructed and trained to obtain a trained defect detection model, specifically including:

[0111] constructing a defect detection model based on YOLOv12;

[0112] Collecting printed product data and performing data augmentation on the printed product data using a geometric augmentation method and a photometric augmentation method to obtain augmented printed product data, wherein the geometric augmentation method includes rotation, horizontal flipping, vertical flipping and scaling, and the photometric augmentation method specifically includes adjusting brightness, adjusting contrast and adding Gaussian noise;

[0113] Calculate the enhanced difference image of each printed product in the printed product template and the augmented printed product data, and label the position and type of defects in the enhanced difference image to generate a defect label, wherein the type of defects includes multiple printing, missing printing and offset, the enhanced difference image and its corresponding defect label constitute training data, and the training data is divided into a training set and a validation set;

[0114] Supervised training of the defect detection model using the training set and validation on the validation set to obtain a trained defect detection model.

[0115] Specifically, the data set is the basis for model training, and natural defects and artificial simulated defects need to be covered to ensure that the data is real and diverse. Natural defects are collected directly from the production site to collect actual defect samples. For scenes with insufficient samples in natural defects, samples are supplemented through physical simulation and digital generation. Printed products are susceptible to paper deformation, uneven lighting, scanning noise and other interference, and need to simulate real scene changes through augmented data to enhance the model generalization ability. In the embodiments of the present application, the augmented data includes the following two ways: geometric augmentation: rotation, horizontal flipping, vertical flipping, scaling, to avoid excessive deformation leading to loss of defect features; photometric augmentation: adjusting brightness, contrast, adding Gaussian noise, simulating scanner lighting differences, randomly adding blur, simulating out-of-focus scenes.

[0116] In the embodiments of the present application, the enhanced difference image of each printing product in the printing product template and the augmented printing product data is enhanced, the positions and types of defects in the enhanced difference image are labeled, and a YOLO format label is generated to provide a supervision signal for YOLO-v12. Specifically, the types of defects include overprint, missing print, and offset, the white area and the black area in the enhanced difference image are labeled as overprint and missing print respectively, and it is determined whether the other color area only has color difference without defects. If the other color area only has color difference without defects, no labeling is needed, otherwise the type of defect corresponding to the other color area is labeled manually.

[0117] In the training process of the defect detection model, the pre-trained weight is loaded, the labeled training set is input, the loss is calculated through forward propagation, the total loss of the defect detection model = localization loss + confidence loss + classification loss, and the model parameters are updated through back propagation; the mAP (mean average precision) is evaluated using the validation set after each training, and overfitting is monitored. If the training mAP continues to rise but the validation mAP stagnates, it needs to be stopped in advance.

[0118] In the training process of the defect detection model, the defect detection model is also adjusted and optimized, and the defect detection model is adjusted according to the training log and the validation result to improve the detection accuracy and efficiency.

[0119] The printing product defect detection method proposed in the embodiments of the present application performs offset operation and saturation processing on the difference value of the color channel C of each matching grid of the printing product template and the corresponding pixel points in the matching position area of the printing product to be detected, and finally obtains an enhanced difference image to highlight defects, which helps the subsequent defect detection model to identify defects and improves the detection accuracy and efficiency. And all matching grids are matched one by one to perform full-page detection, reducing missed detection, solving the problems of low efficiency and large error of manual regional comparison and the missed detection problem caused by mis-matching of repeated texture areas in traditional machine vision method NCC. Further, since the features of overprint, missing print and offset defects are different in the enhanced difference image, the trained defect detection model is used to identify the defects by inputting the enhanced difference image, and the trained defect detection model is used to identify the defects by calculating the enhanced difference image highlighting defects. It can identify tiny scratch defects, reduce the misjudgment of normal deformation such as tiny gray changes on the edge of text, background texture of different batches of printing products, and color difference, and be more sensitive to other tiny defects such as overprint deviation and light scratches, greatly reducing misjudgment and missed detection, and improving the accuracy of defect detection. Compared with traditional manual comparison and traditional visual solutions, the printing product defect detection method proposed in the present application realizes full-flow automatic detection, improves detection accuracy and efficiency, and the detection time is ≤10 seconds / page.

[0120] The above steps S1-S5 do not necessarily represent the order between the steps, but the step symbols represent that the order between the steps can be adjusted.

[0121] Further referring to Figure 9 , as an implementation of the method shown in the above figures, the present application provides an embodiment of a printed product defect detection device, which corresponds to the method embodiment shown in Figure 1 , and the device can be applied in various electronic devices.

[0122] The embodiment of the present application provides a printed product defect detection device, which comprises:

[0123] A grid division module 1 is configured to obtain a printed product template and divide the printed product template into a plurality of grids;

[0124] A grid marking module 2 is configured to extract edge features of an image in each grid, judge whether the edge features of the image in each grid satisfy a condition, if yes, mark the grid as a fixed grid, otherwise mark the grid as a to-be-merged grid;

[0125] A grid merging module 3 is configured to traverse all the to-be-merged grids, perform a first merging on the to-be-merged grids and the closest fixed grid to obtain a first merged grid, and perform a second merging on all the first merged grids containing the same fixed grid to obtain a second merged grid, and take all the second merged grids, the first merged grid not subjected to the second merging and the fixed grid not subjected to the first merging as matching grids;

[0126] A difference image calculation module 4 is configured to obtain a printed product to be detected, perform matching and alignment on each matching grid and the printed product to be detected by using a template matching algorithm, and calculate an enhanced difference image between each matching grid of the printed product template and a region matched and aligned in the printed product to be detected;

[0127] A defect detection module 5 is configured to construct and train a defect detection model, obtain a trained defect detection model, input the enhanced difference image into the trained defect detection model for defect detection, and obtain defects of the printed product to be detected.

[0128] Figure 10 The hardware structure schematic diagram of the electronic device provided by the embodiment of the present application is shown in FIG. 1. Figure 10 As shown in FIG. 1, the electronic device of the embodiment comprises a processor 1001 and a memory 1002; the memory 1002 is used for storing computer execution instructions; the processor 1001 is used for executing the computer execution instructions stored in the memory to realize each step executed by the electronic device in the above embodiment. For details, please refer to the related description in the foregoing method embodiment.

[0129] Optionally, the memory 1002 can be independent or integrated with the processor 1001.

[0130] When the memory 1002 is independent, the electronic device further includes a bus 1003 for connecting the memory 1002 and the processor 1001.

[0131] The embodiment of the application further provides a computer storage medium, and the computer storage medium stores computer execution instructions. When the processor 1001 executes the computer execution instructions, the method described above is realized.

[0132] The embodiment of the application further provides a computer program product, and the computer program product includes a computer program. When the computer program is executed by the processor 1001, the method described above is realized.

[0133] In the embodiments of the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. In actual implementation, another division mode can be used. For example, a plurality of modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the modules shown or discussed can be indirect coupling or communication connection through some interfaces, devices or modules, and can be electrical, mechanical or other forms.

[0134] The modules illustrated as separate components can or can not be physically separate, and the components illustrated as modules can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to implement the embodiment of the application.

[0135] In addition, the functional modules in each embodiment of the application can be integrated in one processing unit, or each module can be physically present alone, or two or more modules can be integrated in one unit. The unit formed by the above modules can be realized in the form of hardware, or in the form of hardware plus software function unit.

[0136] The integrated modules realized in the form of software function modules can be stored in a computer readable storage medium. The software function modules stored in the storage medium include a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or the processor 1001 to execute some steps of the method of each embodiment of the application.

[0137] It should be appreciated that the processor 1001 described above can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), or the like. The general-purpose processor can be a microprocessor or the processor 1001 can also be any conventional processor 1001, etc. The steps of the methods disclosed in conjunction with the present application can be directly embodied as the execution of the processor 1001 in hardware, or be executed by a combination of hardware and software modules in the processor 1001.

[0138] The memory 1002 can include a high-speed RAM memory, and can also include a non-volatile storage NVM, such as at least one disk memory, and can also be a U disk, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0139] The bus 1003 can be an industry standard architecture (ISA), a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. The bus 1003 can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience, the bus 1003 in the drawings of the present application does not limit to only one bus 1003 or one type of bus 1003.

[0140] The storage medium described above can be realized by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0141] An example storage medium is coupled to the processor 1001 such that the processor 1001 can read information from, and can write information to, the storage medium. Of course, the storage medium can be a part of the processor 1001. Consistent with the teachings provided herein, the processor 1001 can be implemented using a dedicated integrated circuit (ASIC), and / or various other hardware components.

[0142] Those skilled in the art can understand that all or part of the steps of the foregoing method embodiments can be completed by program instruction-related hardware. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, the steps of the foregoing method embodiments are executed; and the foregoing storage medium includes various media capable of storing program codes, such as ROM, RAM, magnetic disks, or optical disks.

[0143] Finally, it should be noted that the above embodiments are merely used to illustrate the technical solutions of the present application, rather than limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some or all of the technical features can be replaced equivalently; and these modifications or replacements do not cause the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting defects in printed products, characterized in that, Includes the following steps: Obtain a printed product template and divide the printed product template into multiple grids; Extract the edge features of the image within each grid, and determine whether the edge features of the image within each grid meet the conditions. If they do, mark the grid as a fixed grid; otherwise, mark the grid as a grid to be merged. Traverse all the meshes to be merged, merge the meshes to be merged with the nearest fixed mesh for the first time to obtain the first merged mesh, and merge all the first merged meshes that contain the same fixed mesh for the second time to obtain the second merged mesh. All the second merged meshes, the first merged meshes that have not been merged for the second time, and the fixed meshes that have not been merged for the first time are used as matching meshes. The printed product to be inspected is obtained, and each matching grid is matched and aligned with the printed product to be inspected using a template matching algorithm. Then, the enhanced difference image between each matching grid of the printed product template and the matched and aligned area in the printed product to be inspected is calculated. A defect detection model is constructed and trained to obtain a trained defect detection model. The enhanced differential image is then input into the trained defect detection model to perform defect detection, thereby obtaining the defects of the printed product to be detected.

2. The method for detecting defects in printed products according to claim 1, characterized in that, The step of extracting edge features from the image within each grid, determining whether the edge features of the image within each grid meet the conditions, and marking the grid as a fixed grid if they do, and otherwise marking the grid as a grid to be merged, specifically includes: The Canny operator is used to extract edge features of the image within each grid, and the edge features include the number, length, width, and density of edges; Determine whether each edge feature within each grid is within the corresponding preset parameter range. If so, mark the grid as a fixed grid; otherwise, mark the grid as a grid to be merged.

3. The method for detecting defects in printed products according to claim 1, characterized in that, The calculation of the enhanced difference image between each matching grid of the printed product template and the matching region in the printed product to be detected specifically includes: The difference in color channel C between each matching grid of the printed product template and the corresponding pixel in the matching region of the printed product to be inspected is calculated as follows: ； in, This represents the difference in color channel C between each matching grid in the printed product template and the corresponding pixel within the matching region of the printed product to be inspected. This indicates that within each of the matching grids of the printed product template The pixel value of the pixel at position C in color channel C. The area representing the matching and alignment of the printed product to be detected The pixel value of the pixel at the location in color channel C, which includes the R channel, G channel and B channel; The difference between the color channel C of each matching grid in the printed product template and the corresponding pixel in the matching area of ​​the printed product to be detected is offset to obtain the offset difference result, as shown in the following formula: ； in, This represents the offset difference result; The offset difference result is then saturated to obtain the saturated result, as shown in the following formula: ； in, This indicates the result of the saturation treatment; An enhanced difference image is generated based on the saturation processing results of all color channels C of each pixel. The enhanced difference image includes a neutral gray region and a heterochromatic region. In the neutral gray region, the saturation processing result of all color channels C of each pixel is 128, and there are no defects in the neutral gray region. In the heterochromatic region, at least one pixel has at least one color channel whose saturation processing result is not 128, and there are defects and / or color differences in the heterochromatic region.

4. The method for detecting defects in printed products according to claim 3, characterized in that, The dissimilar color regions include white regions, black regions, and other color regions. In the white region, the saturation result of all color channels C of each pixel is 255, indicating a multi-print defect in the white region. In the black region, the saturation result of all color channels C of each pixel is 0, indicating a missing print defect in the black region. In the other color regions, at least one pixel has at least one color channel whose saturation result is not 0, 128, or 255, indicating a multi-print defect, a missing print defect, and / or color difference in the other color regions.

5. The method for detecting defects in printed products according to claim 1, characterized in that, The step of using a template matching algorithm to match and align each matching grid with the printed product to be inspected specifically includes: A matching template is created based on the edge features of a fixed grid in each of the matching grids. The matching template includes a template center point and the template angle of the matching template is 0°. The matching template corresponding to each matching grid is used to match and search on the printed product to be inspected to obtain a matching result, which includes the matching center point and rotation angle. The transformation matrix M is constructed based on the template center point, the matching center point, and the rotation angle, as shown in the following equation: ； Where θ represents the rotation angle, with counterclockwise being positive; , () represents the coordinates of the center point of the template; , () indicates the coordinates of the matching center point; and These represent the translation components of the matching grid relative to the matching result in the x and y directions, respectively. The affine transformation of the printed product template is performed according to the transformation matrix M, so that the printed product template coincides with the printed product to be detected to complete the alignment.

6. The method for detecting defects in printed products according to claim 1, characterized in that, The construction and training of the defect detection model to obtain the trained defect detection model specifically includes: Construct a defect detection model based on YOLOv12; Collect printed product data and augment the printed product data using geometric augmentation and photometric augmentation methods to obtain augmented printed product data. The geometric augmentation methods include rotation, horizontal flipping, vertical flipping, and scaling. The photometric augmentation methods specifically include adjusting brightness, adjusting contrast, and adding Gaussian noise. Calculate the enhanced difference image of each printed product in the printed product template and the augmented printed product data, and label the location and type of defects in the enhanced difference image to generate defect labels. The defect types include overprinting, missing printing, and offset. The enhanced difference image and its corresponding defect labels constitute training data, and the training data is divided into a training set and a validation set. The defect detection model is trained in a supervised manner using the training set and validated on the validation set to obtain the trained defect detection model.

7. A defect detection device for printed products, characterized in that, include: The grid division module is configured to acquire a printed product template and divide the printed product template into multiple grids; The grid marking module is configured to extract the edge features of the image within each grid, determine whether the edge features of the image within each grid meet the conditions, and if they do, mark the grid as a fixed grid; otherwise, mark the grid as a grid to be merged. The mesh merging module is configured to traverse all the meshes to be merged, merge the meshes to be merged with the nearest fixed mesh for the first time to obtain a first merged mesh, and merge all the first merged meshes that contain the same fixed mesh for the second time to obtain a second merged mesh, and use all the second merged meshes, the first merged meshes that have not been merged for the second time, and the fixed meshes that have not been merged for the first time as matching meshes; The differential image calculation module is configured to acquire the printed product to be detected, and after matching each of the matching grids with the printed product to be detected using a template matching algorithm, calculate the enhanced differential image between each of the matching grids of the printed product template and the matching area in the printed product to be detected. The defect detection module is configured to build and train a defect detection model to obtain a trained defect detection model, input the enhanced differential image into the trained defect detection model to perform defect detection, and obtain the defects of the printed product to be detected.

8. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images