Print inspection device and method for the optical inspection of a printed image of a printed object

Abstract

The invention relates to a print inspection device (100) for the optical inspection of a printed image (120) of a printed object, wherein a target printed image (121) is assigned to the printed object, comprising: a processor (110) configured to determine a plurality of raster cell images (111) for the printed image (120) and the target printed image (121) based on a subdivision of the printed image (120) and the target printed image (121) into raster cells (130); and to determine a Delta-E raster cell image (112) for each raster cell (130) based on a color difference between a raster cell image of the printed image (120) and a raster cell image of the target printed image (121).to determine for each pixel of the Delta-E raster cell image (112) based on a pixel-specific threshold function (140) whether a pixel defect (113) is present, wherein the pixel-specific threshold function (140) is based on a previous training (150) of a database (160) of Delta-E raster cell images with associated manually specified ground-truth cell images; to assemble the previously determined pixel defects (113) in the respective Delta-E raster cell images (112) into a defect image (114) which provides an overview of the pixel defects (113) of the printed image (120); and to output an inspection result (116) based on a user-specific classification (115) of the defect image (114).

Description

[0001] The present invention relates to the field of optical inspection of a printed image of a printed object.

[0002] After printing, print images are inspected for printing errors. Optical inspection systems are typically used to assess the print image of a printed object, for example, for quality control of printing processes. Existing methods and systems are often based on differential image techniques. Their results depend heavily on manual parameterization and sometimes lead to pseudo-errors.

[0003] The document DE 10 2017 116 882 A1 relates to a print inspection device for optically inspecting a print image of a printed object, wherein a target print object is assigned to the printed object. The print inspection device comprises an image camera for optically capturing the printed object and a processor for further processing the captured print object.

[0004] The published patent application WO 2004 / 056570 A1 discloses a method and a device for real-time control of printed images.

[0005] It is therefore an object of the present invention to provide an efficient concept for the optical inspection of a print image of a printed object.

[0006] In particular, it is an object of the invention to create a concept for the optical inspection of printed images that is less dependent on manual parameterization and leads to fewer printing errors, in particular pseudo-printing errors, than conventional methods.

[0007] This problem is solved by the features of the independent claims. Advantageous further developments are the subject of the dependent patent claims, the description, and the drawings.

[0008] The invention solves this problem by detecting anomalies in the printed image, measuring defect sizes, and classifying the defects. In particular, the invention enables customer-specific defect detection in the printed image using machine learning. This problem is solved, in particular, by a multi-layer architecture that enables customer-specific detection of printing defects. The multiple layers include color calibration, determination of the defect pattern from a calibrated Delta E image, customer-specific classification using machine learning, and detection of different defect degrees.

[0009] This brings with it the technical advantages of reducing parameters by using color distance and machine learning to train customer-specific requirements. The defect classes can be efficiently trained using boundary patterns.

[0010] According to a first aspect, the invention relates to a print inspection device for the optical inspection of a print image of a printed object, wherein a target print image is assigned to the printed object, comprising: a processor that is designed and configured to determine a plurality of raster cell images for the printed image and the target print image based on a subdivision of the printed image and the target print image into raster cells; to determine a Delta-E raster cell image for each raster cell based on a color difference between a raster cell image of the printed image and a raster cell image of the target print image; to determine for each pixel of the Delta-E cell image whether a pixel defect is present based on a pixel-specific threshold function, wherein the pixel-specific threshold function is based on prior training of a database of Delta-E cell images with associated manually specified ground truth cell images;Combine the previously determined pixel defects in the respective Delta-E cell images into a defect image that provides an overview of the pixel defects in the printed image; and output an inspection result based on a user-specific classification of the defect image.

[0011] Such a print inspection device offers the user an efficient optical inspection of a printed image. Due to the prior training, the optical inspection relies less heavily on manual parameterization than is the case with previous devices, resulting in fewer falsely detected print defects; in particular, pseudo-print defects are recognized as such.

[0012] The print inspection device detects anomalies in the printed image and provides the user with a means of measuring defect sizes and classifying them. In particular, the print inspection device enables customer-specific defect detection in the printed image using machine learning.

[0013] The print image represents the actual print image of a printed object to be inspected, which may be printed on a substrate. The target print image may be, for example, a vector font, a vector graphic, or a digital image, and may be known in advance, for example, from a prepress stage.

[0014] The grid can be equidistant, but it can also be composed of any number of polygons. Furthermore, individual cells within the grid can be deactivated. This provides the advantage of allowing very flexible adjustment of regions where testing is not desired.

[0015] The grid cells overlap, allowing local alignment to be performed for each cell. This has the advantage that in cases of film tension that is not optimally adjusted, lamination distortion, or similar situations, local alignment can better align the actual print image with the target print image.

[0016] According to one embodiment, the processor is configured to determine the plurality of raster cell images based on color-calibrated and aligned print images and target print images.

[0017] This has the technical advantage of reducing the number of misidentified printing defects, as the color difference between color-calibrated images can be determined more accurately. Furthermore, the mutual alignment of the images results in fewer streaks or hatching in the defect image, which result from misalignment.

[0018] According to one embodiment, the processor is configured to determine the plurality of raster cell images for a respective color channel of a plurality of color channels of a color space.

[0019] This has the technical advantage of reducing the number of incorrectly detected printing errors, as a printing error that occurs due to a printer color channel defect is easier to detect in the corresponding color channel.

[0020] According to one embodiment, the processor is configured to determine the Delta-E raster cell image based on a Euclidean distance in the color space between the raster cell image of the print image and the corresponding raster cell image of the target print image.

[0021] This provides the technical advantage that the Euclidean distance can be determined more accurately for the respective raster cell images than in the actual-size image. This results in fewer false detections of printing defects. The inspection results of the print inspection device are more accurate.

[0022] According to one embodiment, the prior training of the database of Delta-E cell images with associated ground-truth cell images is carried out based on neural networks (NN), support vector machines (SVM) and / or multi-layer perceptron (MLP) methods.

[0023] This provides the technical advantage that such machine learning methods offer good learning success and are also scalable, i.e. the more complex the network is modeled, the more accurate the inspection result will be.

[0024] According to one embodiment, the pixel-specific threshold function specifies a fragment in the Delta-E raster cell image that is based on an irregularity in an alignment of the print image with respect to the target print image as not being a pixel defect.

[0025] This provides the technical advantage that the print inspection device is error-tolerant with regard to incorrect positioning of the print image relative to the target print image, which may be caused, for example, by the print image not being positioned precisely, e.g., by incorrect positioning of the recording camera or a skewed feed when reading or scanning the print image.

[0026] According to one embodiment, the pixel-specific threshold function specifies a contiguous region of pixels in the Delta-E raster cell image that exceeds a predetermined size as a pixel defect.

[0027] This provides the technical advantage that printing errors based on misprints, such as dots, ink blots or other irregularities, can be accurately detected.

[0028] According to one embodiment, both the training of the database of Delta-E cell images with associated ground-truth cell images and the user-specific classification of the defect image are based on the same training process.

[0029] In such a training process, for example, a user is presented with various print images along with corresponding target print images. The user identifies defects in the print images based on their subjective impression and marks them (labeling process). Based on the same training process, both raster cell images, which display a small section of the image, and the defect image, which represents the entire image area, can be classified. This simplifies the classification process. The print inspection device is less complex.

[0030] According to one embodiment, the training process is based on a classification of a database of print images against associated target value print images based on a subjective error detection of a user.

[0031] This provides the technical advantage that human expert knowledge can be included in the classification, which increases the error detection rate.

[0032] According to one embodiment, the pixel-specific threshold function assigns a binary value of 0 or 1 to each gray value of the Delta-E raster cell image.

[0033] This provides the technical advantage that the threshold function can efficiently distinguish areas with a high confidence level from areas with a low confidence level and can easily represent them in the form of a threshold.

[0034] According to one embodiment, the pixel-specific threshold function comprises a logistic function, e.g., a sigmoid function or a step function with respect to the gray values ​​of the Delta-E raster cell image.

[0035] This provides the technical advantage that a logistic function or a step function is well suited to depicting and displaying color transitions such as those that occur in printing errors.

[0036] According to one embodiment, the defect classification is based on the following features: an area of ​​connected components of the Delta-E cell image, an inner radius of the connected components of the Delta-E cell image, a correlation between the actual print image and the target print image, and a contrast between background and foreground with respect to the Delta-E cell image. To determine streakiness in the horizontal direction, the number of defect pixels per row is additionally considered as a feature. Similarly, columns are considered for vertical streakiness.

[0037] This provides the technical advantage that these features allow for easy and automated detection of related components in the Delta-E cell image that indicate a printing defect. This allows the inspection system to reliably detect printing defects and distinguish them from artifacts caused by mispositioned print images.

[0038] According to one embodiment, the print inspection device comprises a camera configured to optically record the print object in order to obtain the print image.

[0039] This provides the technical advantage that the camera can easily provide the print image. Alternatively, the print image can be provided via a scanner or reader.

[0040] According to one embodiment, the processor is designed to output a printing error when printing the print image with a color printer as an inspection result, wherein the printing error occurs in the form of a color dot or color stripe that is based on a malfunction of a color channel of the color printer.

[0041] This provides the technical advantage of easily detecting a color printer malfunction. After the printer is repaired or replaced, error-free print images are produced again.

[0042] According to a second aspect, the invention relates to a method for optically inspecting a print image of a printed object, wherein a target print image is assigned to the printed object, comprising the following steps: determining a respective plurality of raster cell images for the printed image and the target print image based on a subdivision of the printed image and the target print image into raster cells; determining a Delta-E raster cell image for each raster cell based on a color difference between a raster cell image of the printed image and a raster cell image of the target print image; determining for each pixel of the Delta-E cell image, based on a pixel-specific threshold function, whether a pixel defect is present, wherein the pixel-specific threshold function is based on prior training of a database of Delta-E cell images with associated manually specified ground truth cell images;Combining the previously determined pixel defects in the respective Delta-E cell images into a defect image that provides an overview of the pixel defects of the printed image; and outputting an inspection result based on a user-specific classification of the defect image.

[0043] Such a method offers the user an efficient optical inspection of a printed image. Due to the prior training, the optical inspection relies less heavily on manual parameterization than was previously the case, resulting in fewer falsely detected printing defects; in particular, pseudo-printing defects are recognized as such.

[0044] Disclosed here is a computer program with a program code for carrying out the method according to the second aspect of the

[0045] Invention. This provides the advantage that the method can be carried out automatically.

[0046] The print inspection device can be programmed to execute the program code or parts of the program code.

[0047] The invention can be implemented in hardware and software.

[0048] Further embodiments are explained in more detail with reference to the accompanying drawings. They show: Fig. 1 shows a schematic diagram of a print inspection device 100 for optically inspecting a print image 120 of a printed object according to an embodiment; Fig. 2 shows an architectural diagram of a print inspection device 200 for optically inspecting a print image 250 of a printed object according to an embodiment; Fig. 3 shows print images 301, 302 of a person, each of which has a printing error caused by a printer; Fig. 4 shows three exemplary series of raster cell images, each with input 401, 411, 421, model 402, 412, 422 or target value, and manually specified ground truth 403, 413, 423; Fig. 5 shows a schematic diagram of a pixel-related training 500 of a Delta E raster cell image according to an embodiment; Fig. 6 shows an exemplary image sequence of a target print image 601, an actual print image 602, and an associated defect image 603; Fig.Fig. 7 shows a schematic diagram of a training 700 of the classification layer for generating a defect classifier according to an embodiment; and Fig. 8 shows a schematic diagram of a method 800 for optically inspecting a printed image of a printed object according to an embodiment.

[0049] Fig. 1 shows a schematic diagram of a print inspection device 100 for optically inspecting a print image 120 of a printed object according to one embodiment. A target print image 121, which represents the ideal image of the printed object, is assigned to the printed object.

[0050] The print inspection device 100 comprises a processor 110, which is configured to determine a plurality of raster cell images 111 for the print image 120 and the target print image 121, respectively, based on a subdivision of the print image 120 and the target print image 121 into raster cells 130. The processor 110 is further configured to determine a Delta-E raster cell image 112 for each raster cell 130 based on a color difference between a raster cell image of the print image 120 and a raster cell image of the target print image 121. The processor 110 is further configured to determine, for each pixel of the Delta-E raster cell image 112, whether a pixel defect 113 is present based on a pixel-specific threshold function 140. The pixel-specific threshold function 140 is based on a previous training 150 of a database 160 of Delta-E raster cell images with associated manually specified ground-truth cell images.The processor 110 is further configured to combine the previously determined pixel defects 113 in the respective Delta-E raster cell images 112 into a defect image 114, which provides an overview of the pixel defects 113 of the printed image 120, and to output an inspection result 116 based on a user-specific classification 115 of the defect image 114.

[0051] A processor, as described in this disclosure, is a programmable computing unit, i.e. a machine or an electronic circuit that controls other machines or electrical circuits according to supplied instructions and, in doing so, executes an algorithm or process, which usually involves data processing. The processor can be, for example, a microcontroller or a digital signal processor or a CPU, for example in an embedded system. The term processor here refers to both the component, i.e. the semiconductor chip, and the data-processing logic unit. The term processor also encompasses one or more processor cores, which are now contained in many processor chips, with each core representing a (largely) independent logic unit.

[0052] Ground truth is a term used in statistics and machine learning. It means that machine learning results are checked for accuracy against the real world. The term originates from meteorology, where "ground truth" refers to information obtained on-site. The term implies a kind of reality check for machine learning algorithms.

[0053] Delta E, often written as dE or ΔE, is a measure of the perceived color difference, which is ideally "equally spaced" for all colors present. The delta symbolizes the difference. This allows for quantification of work dealing with colors.

[0054] The L*a*b* color space (also known as CIELAB) describes all perceivable colors. It uses a three-dimensional color space in which the brightness value L* is perpendicular to the color plane (a*, b*). The most important properties of the L*a*b* color model include device independence and perceptual relevance. This means that colors are defined as they are perceived by a standard observer under standard lighting conditions, regardless of the method of their production or reproduction technology. The color model is standardized in EN ISO 11664-4 "Colorimetry - Part 4: CIE 1976 L*a*b* Color space."

[0055] The color difference is usually expressed as Delta E. In EN ISO 11664-4, the term color distance is preferred over color difference. Compared to color difference, it represents the quantified form. Every color that actually occurs, including every color emitted or measured by a device, can be assigned a color location in three-dimensional space. The value of Delta E between the color locations (L*, a*, b*) p and (L*, a*, b*) q is calculated as a Euclidean distance according to EN ISO 11664-4.

[0056] The grid 130 can be a rectangular or square grid, i.e., a grid with rectangular or square grid elements. Alternatively, any other grid shape can be used, e.g., hexagonal, triangular, etc.

[0057] In one embodiment, the processor 110 is configured to determine the plurality of raster cell images 111 based on color-calibrated 255 and aligned 256 print images 120 and target print images 121.

[0058] The goal of color calibration 255 is to measure and / or adjust the color behavior of a device (input or output) in a known state. Calibration refers to establishing a known relationship to a standard color space. Input data can come from device sources such as digital cameras, image scanners, or other measurement devices. These inputs can be specified either in monochrome or in multidimensional color—most commonly in the three-channel RGB (red / green / blue) model.

[0059] Processor 110 can determine the orientation of the print image relative to the target print image and rotate the print image based on the orientation. This provides the advantage of allowing the orientation of the print image relative to the target print image or the masking template to be corrected.

[0060] The processor 110 may be configured in one embodiment to determine the plurality of raster cell images 111 for a respective color channel of a plurality of color channels 246 of a color space 247, such as Figure 2 shown.

[0061] In one embodiment, the processor 110 may be configured to determine the Delta-E raster cell image 112 based on a Euclidean distance in the color space between the raster cell image 111 of the print image 120 and the corresponding raster cell image 111 of the target print image 121.

[0062] Every color that actually occurs, including every color emitted or measured by a device, can be assigned a color location in three-dimensional space. The value of Delta E between the color locations (L*, a*, b*) and (L*, a*, b*) 2 is calculated as a Euclidean distance according to EN ISO 11664-4: Δ E ab ∗ = L 2 ∗ − L 1 ∗ 2 + a 2 ∗ − a 1 ∗ 2 + b 2 ∗ − b 1 ∗ 2 .

[0063] In one embodiment, the prior training 150 of the database 160 of Delta-E cell images with associated ground-truth cell images is performed based on neural networks (NN), support vector machines (SVM) and / or multi-layer perceptron (MLP) methods.

[0064] (Artificial) neural networks (NNs) are usually based on the interconnection of many neurons. The topology of a network, i.e., the assignment of connections to nodes, must be carefully considered depending on its task. After the construction of a network, the training phase follows, in which the network "learns." A neural network learns through the following methods: developing new connections, deleting existing connections, changing the weights (the weights from neuron to neuron), adjusting the thresholds of neurons (if they have thresholds), adding or deleting neurons, and modifying the activation, propagation, or output functions. Furthermore, the learning behavior changes when the activation function of the neurons or the learning rate of the network is changed. In practice, a network "learns" primarily by modifying the weights of the neurons.This enables NNs to learn complex nonlinear functions using a "learning" algorithm that attempts to determine all parameters of the function from existing input and desired output values ​​using an iterative or recursive approach.

[0065] A support vector machine (SVM) serves as a classifier and regressor. A support vector machine divides a set of objects into classes in such a way that the widest possible area around the class boundaries remains free of objects. The starting point for building a support vector machine is a set of training objects, each of which has a known class. Each object is represented by a vector in a vector space. The task of the support vector machine is to fit a hyperplane into this space, which acts as a dividing surface and divides the training objects into two classes. The distance between the vectors closest to the hyperplane is maximized. This wide, empty boundary should later ensure that even objects that do not exactly correspond to the training objects are classified as reliably as possible.

[0066] The perceptron is a simplified artificial neural network. In its basic form (simple perceptron), it consists of a single artificial neuron with adjustable weights and a threshold. Today, this term refers to various combinations of the original model, distinguishing between single-layer and multi-layer Perceptrons (English) multi-layer perceptron, MLP). Perceptron networks convert an input vector into an output vector and thus represent a simple associative memory.

[0067] In one embodiment of the print inspection device 100, the pixel-specific threshold function 140 specifies a fragment 224, 611 (such as in Figure 2 or Figure 6shown) in the Delta-E raster cell image 112, which is based on an irregularity in the alignment of the print image 120 with respect to the target print image 121, as not a pixel defect 113. This can prevent misalignments of the print image with respect to the target print image 121 from being determined as printing errors or defects. Such fragments usually appear in the form of lines or hatching, especially in places where the print image 120 is slightly shifted with respect to the target print image 121.

[0068] In one embodiment of the print inspection apparatus 100, the pixel-specific threshold function 140 specifies a contiguous area 214, 610 of pixels in the Delta-E raster cell image 112 (e.g., as shown in Figure 2 or Figure 6 This allows misprints, which usually cover a certain area, to be recognized as printing errors or defects.

[0069] In one embodiment of the print inspection device 100, both the training 150 of the database 160 of Delta-E cell images with associated ground-truth cell images and the user-specific classification 115 of the defect image 114 are based on the same training process 150. In such a training process, for example, a user is presented with various print images with associated target print images. Based on their subjective impression, the user identifies defects in the print images and marks them (labeling process). The print images marked by the user with associated (actual) print images are learned in the training phase using a machine learning method, such as NN, MLP, or SVM. This enables the machine learning method to also classify new images for print defects based on the learned pattern.

[0070] In one embodiment of the print inspection device 100, the training process 150 is based on a classification of a database of print images (see e.g. 401, 411, 421 in Figure 4 ) compared to the corresponding setpoint print images (see e.g. 402, 412, 422 in Figure 4 ) based on a user's subjective error detection, as described above.

[0071] In one embodiment of the print inspection device 100, the pixel-specific threshold function 140, 505 (see also Figure 5 ) assigns a threshold value in the range from 0 to 1 to each color value of the Delta-E raster cell image 112. The pixel-specific threshold function 140, 505 can comprise, for example, a logistic function, e.g., a sigmoid function or a step function with respect to the gray values ​​of the Delta-E raster cell image 503.

[0072] In particular, the classification 115 may be based on the following features: an area of ​​connected components of the Delta-E raster cell image 503 (see Figure 5 ), an inner radius of the connected components of the Delta-E raster cell image 503, a correlation between the actual print image and the target print image, a contrast of the Delta-E raster cell image 503, in particular a color difference between background and foreground.

[0073] The print inspection device 100 may further comprise a camera configured to optically capture the printed object to obtain the printed image. Alternatively, the print inspection device 100 may comprise a scanner or a reader to scan or read the printed image.

[0074] In one embodiment, the processor 110 is configured to output as inspection result 116 a printing error when printing the print image 120 with a color printer, wherein the printing error occurs in the form of a color dot or color stripe, which is due to a malfunction of a color channel 246 (see Fig. 2 ) of the color printer.

[0075] Fig. 2 shows an architectural diagram of a print inspection device 200 for optically inspecting a print image 250 of a printed object according to an embodiment. The print inspection device 200 is a specific embodiment of the above-mentioned Figure 1 described print inspection device 100.

[0076] An input image 250 corresponds to the print image 120 from Figure 1The input image 250 is assigned to a target print image 241 of a model 240, which corresponds to the ideal image of the print object, i.e., the image without printing defects. In a color calibration step 255, the input image 250 and the target print image 241 are color calibrated.

[0077] As already mentioned above Figure 1 As described, the goal of color calibration 255 is to adapt the color behavior of the input image 250 to the color behavior of the target print image 241. Calibration refers to establishing a known relationship to a standard color space. Inputs are the input image 250 and the target print image 241. These inputs can be specified either monochrome or in multidimensional color - most commonly in the three-channel RGB (red / green / blue) model. After color calibration 255, a color-calibrated input image 251 is generated, e.g., with white balance, as in Fig. 2The target print image 241 can also be displayed as a color-calibrated target print image.

[0078] After color calibration 255, the target print image 241, or the color-calibrated target print image, and the color-calibrated (actual) print image 251 are aligned 256. The images are shifted or rotated relative to each other so that both images are ultimately aligned with each other. This means that the processor 110 converts the position data of both images accordingly so that the position data are aligned with each other. After alignment 256, a target print image 242 and an actual print image 252 are available, which are aligned with each other or with each other.

[0079] Both images (target print image 242 and an actual print image 252) are then split into respective color channels 246 in the existing color space 247, e.g. RGB, so that an actual print image 253 and a corresponding target print image are generated for each color channel 246.

[0080] A grid 244 or raster is placed over the print image 253 and the associated target print image 242 in order to divide the print image 253 and the associated target print image 242 into a plurality of raster cell images 254, each of which represents a partial section of the respective print image. The grid size is predefined or freely selectable. A local position correction 245 can be performed via the grid layer. Each cell or raster cell of the grid 244 is treated in the print inspection device 200 as an independent layer 210, 220, 230, in Figure 2 The three layers 210, 220, 230 are shown as examples.

[0081] For each layer 210, 220, 230, a Delta E raster cell image 201 is determined based on a comparison between the model and the image. For example, the Delta E raster cell image 215 of a first layer 210 can be determined based on a Euclidean distance in the color space between the raster cell image 214 of the print image 253 and the corresponding raster cell image 213 of the target print image 242. The Delta E raster cell image 201 or 215 can be determined, for example, using the following formula: Δ E ab ∗ = L 2 ∗ − L 1 ∗ 2 + a 2 ∗ − a 1 ∗ 2 + b 2 ∗ − b 1 ∗ 2 .

[0082] The value of Delta E is calculated as the Euclidean distance between the color locations (L*, a*, b*), and (L*, a*, b*) 2 .

[0083] Accordingly, the Delta-E raster cell image 225 of a second layer 220 can be determined, for example, based on the Euclidean distance in the color space between the raster cell image 224 of the print image 253 and the corresponding raster cell image 223 of the target print image 242. Accordingly, the Delta-E raster cell image 235 of an n-th layer 230 can be determined, for example, based on the Euclidean distance in the color space between the raster cell image 234 of the print image 253 and the corresponding raster cell image 233 of the target print image 242.

[0084] For each layer 210, 220, 230, a database is trained using an activation function 202 per pixel. The activation function 202 can, for example, be based on the following relationship: P p l = D = 1 1 + e p i − μ σ where pl is a probability quantity, µ its mean, and σ its variance. The probability distribution is therefore characterized by a sigmoid function.

[0085] This is followed by a classification in a classification layer 203 of a training database, e.g. the training database 160 with Delta-E and ground-truth images, as in Figure 1 described to detect an error or no error.

[0086] For each layer 210, 220, 230, connected components in the Delta E raster cell image 201 are detected and classified as faulty or non-faulty. For example, a connected component 216 of the Delta E raster cell image 215 of the first layer 210 is classified as non-faulty, and a connected component 217 of the Delta E raster cell image 215 of the first layer 210 is classified as faulty.

[0087] For the second layer 220, a connected component 226 of the Delta E raster cell image 225 is classified as non-erroneous and a connected component 227 of the Delta E raster cell image 225 is classified as non-erroneous.

[0088] For the n-th layer 230, a connected component 236 of the Delta E raster cell image 235 is classified as non-erroneous and a connected component 237 of the Delta E raster cell image 235 is classified as erroneous.

[0089] As above Figure 1As described above, a pixel-specific threshold function can, for example, classify a fragment 224 in the Delta-E raster cell image 201, which is based on an irregularity in the alignment of the printed image 250 with respect to the target printed image 242, as not being a pixel defect. This can prevent misalignments of the printed image 250 with respect to the target printed image 242 from being determined as printing errors or defects. Such fragments usually appear in the form of lines or hatching, in particular at locations where the printed image 120 is slightly shifted with respect to the target printed image 121. Furthermore, the pixel-specific threshold function can classify a contiguous area 214 of pixels in the Delta-E raster cell image 215, which exceeds a predetermined size or extent, as a pixel defect. This allows misprints, which usually occupy a certain area, to be recognized as printing errors or defects.

[0090] During training, for example, a user is presented with various print images along with corresponding target print images. Based on their subjective impression, the user identifies errors in the print images and marks them (labeling process). The print images marked by the user, along with the corresponding (actual) print images, are learned during the training phase using a machine learning method such as NN, MLP, or SVM. This enables the machine learning method to classify new images for printing errors based on the learned pattern.

[0091] The activation function 202, also referred to as a pixel-specific threshold function, can be based on the following features: an area of ​​connected components of the Delta-E raster cell image 215, 225, 235, an inner radius of the connected components of the Delta-E raster cell image 215, 225, 235, a correlation between color channels 246 of the Delta-E raster cell image 215, 225, 235, a contrast of the Delta-E raster cell image 215, 225, 235, in particular a color difference between background and foreground. To determine streakiness in the horizontal direction, the number of defect pixels per row is additionally considered as a feature. Analogously, columns are considered for vertical streakiness.

[0092] From the defects 212, 232 or non-defects 211, 221, 222, 231 thus determined, a defect image 204 is generated, which gives the viewer an overview of printing defects in the printed image.

[0093] Finally, a customer-specific classification follows, in which an inspection result 206 is derived from the defect pattern 204.

[0094] In one embodiment of the print inspection device 200, both the training of the classification layer 203 with activation function using a database and the customer-specific classification 205 of the defect image 204 are based on the same training process 150, as described above, for example, Figure 1described. In such a training process, for example, a user is presented with various print images with associated target print images. Based on their subjective impression, the user identifies errors in the print images and marks them (labeling process). The print images marked by the user with the associated (actual) print images are learned in the training phase using a machine learning method, such as NN, MLP, or SVM. This enables the machine learning method to classify new images for print errors based on the learned pattern, so that the print inspection device 200 has a higher error detection rate.

[0095] The print inspection device 200 was tested with various data sets of portrait images. A first data set contained 72 portrait images, of which 64 images had color dots within the portrait and 8 images were free of printing defects. A second data set contained 600 portrait images, of which 480 images had color dots within the portrait, 117 images had streaks, and only 3 images were free of printing defects. Each image had only one printing defect, in one color channel. In the test data set, the printing defects were varied across the individual color channels.

[0096] In both data sets, the color dots had areas of 0.4 mm 2< , 0.5 mm 2< , and 0.6 mm 2< . Minor defects, i.e., printing defects with an area of ​​0.4 mm 2< , were detected with a detection rate of 85.53%, and critical defects, i.e., printing defects with an area of ​​0.5 mm 2< and 0.6 mm 2< , were detected with a detection rate of 98.39%.

[0097] Fig. 3shows two printed images 301, 302 of a female person, each of which contains a printing error caused by a printer. In printed image 301, a printing error at location 311, i.e., in the area of ​​the right cheek, was detected by manual inspection. In printed image 302, printing errors at locations 312 and 313, i.e., light horizontal stripes in the facial area, were detected by manual inspection.

[0098] These printing defects are classified as second-class printing defects, which arise due to a printer malfunction and manifest as colored dots or streaks. In this case, either a color channel is missing entirely or a color channel is printing when it shouldn't. This class of defects can be specified more precisely. Second-class printing defects differ from first-class printing defects, which are anomalies such as scratches, dirt, etc. First-class printing defects cannot be specified further.

[0099] First-class printing errors can be detected using statistical anomaly detection, while second-class printing errors can be used to train a classifier, as described above for the Figures 1 and 2 described, which can then detect these defects.

[0100] Fig. 4shows three example series of grid cell images, each with input 401, 411, 421, model 402, 412, 422 or target value and manually specified ground truth 403, 413, 423.

[0101] In the first series with the actual raster cell image 401, which is a grid cell of an actual print image, the model 402, which represents the target value for the raster cell image, and the ground-truth raster cell image 403, the printing error in the actual raster cell image 401, i.e., the dot- or circular area that occupies the entire center of the image 401, is very pronounced and can therefore be easily identified manually by a user. The user can then mark or label the ground-truth raster cell image 403. White areas in the ground-truth raster cell image 403 represent areas with a high confidence level, while black areas represent those with a low confidence level. The confidence level in the ground truth raster cell image 403 is scaled from 0 to 240, where 0 represents the black area with low confidence level and 240 represents the white area with high confidence level.Even if no gray areas are visible in the ground-truth raster cell image 403, the user can also label confidence levels between 0 and 240 (i.e., shades of gray from dark gray to light gray).

[0102] In the second series with the actual raster cell image 411, which is a grid cell of an actual print image, the model 412, which represents the target value for the raster cell image, and the ground-truth raster cell image 413, the printing error in the actual raster cell image 411, i.e. the dot- or circular area which occupies the entire center of the image 411, is only slightly pronounced and can therefore only be detected manually by the user with difficulty after intensive inspection. After careful inspection by the user, they can mark or label the ground-truth raster cell image 413. Due to the user's careful inspection, the ground-truth raster cell image 413 roughly corresponds to the ground-truth raster cell image 403 from the first series, i.e. despite the poorer input data 411, the result is the same.

[0103] In the third series with the actual raster cell image 421, which is a grid cell of an actual print image, the model 422, which represents the target value for the raster cell image, and the ground-truth raster cell image 423, the printing error in the actual raster cell image 421, i.e. the dot- or circular area that occupies the entire center of the image 421, is also only slightly pronounced and can therefore be manually detected by the user only with difficulty after intensive inspection. While the second series shows a light gray error in a light gray image, the third series shows a dark gray error in a dark gray image. Here, too, the user must inspect the image closely in order to then label the ground-truth raster cell image 423. Here too, the ground-truth grid cell image 423 corresponds approximately to the ground-truth grid cell image 403 from the first series due to the user's careful examination, ieDespite the poorer input data 421, the result is the same.

[0104] Fig. 5 shows a schematic diagram of pixel-wise training 500 of a Delta E raster cell image according to one embodiment.

[0105] From the input image, ie the actual raster cell image 502, which is a grid cell of an actual print image, and the model 501, which represents the target value for the raster cell image, the Delta E raster cell image 503 is determined, as described above for the Figures 1 and 2 The ground-truth grid cell image 504, which results from a previous labeling process as described above, Figure 4 is used together with the Delta E grid cell image 503 as input variables for training 507 in order to train a threshold function 505 on the basis of which an optimal threshold 506 can be determined.

[0106] A brightness scale in the Delta E raster cell image 503, which ranges from 0 (black) to 48 (white), is represented in the form of a threshold function 505. Brightness values ​​from 0 to approximately 25 are weighted with the threshold value 1, which corresponds to a high confidence level. Brightness values ​​from approximately 30 to 50 are weighted with the threshold value 0, which corresponds to a low confidence level. In between, i.e. from approximately 25 to approximately 30, the threshold falls monotonically from 1 to 0. Such values ​​correspond to a medium confidence level, which is located in the border area between an area that is recognized as a printing error and an area that is not recognized as a printing error.

[0107] The optimal threshold value 506 is therefore in the range between 25 and 30, e.g. 27 or 28.

[0108] Fig. 6 shows an exemplary image sequence of a target print image 601, a real print image 602 and an associated defect image 603.

[0109] The target print image 601 represents a portrait of a female person. In the actual print image 602, a colored spot or dot can be seen on the person's right cheek, which represents a printing error. In the corresponding defect image 603, which, for example, corresponds to the Figure 1 described defect pattern 114 or the one to Figure 2 In the defect image 204 described above, two different types of printing errors can be identified. Firstly, the aforementioned colored spot or dot 610 is recognized as a printing error. Secondly, lines or hatching 611 in the hairline area are recognized as printing errors, which may have occurred due to an inaccurate alignment of the two images 601, 602.

[0110] Using the above-mentioned Figures 1 and 2According to the customer-specific classification 115, 205 described above, the lines or hatchings 611 can be excluded as printing errors, so that only the colored spot or dot 610 is recognized as a printing error.

[0111] Fig. 7 shows a schematic diagram of a training 700 of the classification layer for generating a defect classifier according to an embodiment.

[0112] On the left side of the figure, real defects 701 are shown, while on the right side, pseudo-defects 702 are illustrated. The real defects 701 are colored spots or dots, while the pseudo-defects 702 are lines or hatchings, which usually arise due to a misalignment of the actual image to the target image. When training the classification layer 703, e.g., with the methods described above. Figure 2The defect classifier is determined using methods described above, such as NN, MLP, SVM, etc., which can distinguish between the two types of defects 701 and 702. With this defect classifier 704, it is then possible to identify only the spots 610 in the defect image 603 of the Figure 6 to be classified as a printing error, but not the stripes 611 or hatching.

[0113] Fig. 8 shows a schematic diagram of a method 800 for optically inspecting a printed image of a printed object according to an embodiment. As described above Figure 1 As described above, a target print image 121 is assigned to the print object. The method carries out the steps of the processor 110, which are described above for Figure 1 and Figure 2 were described in more detail.

[0114] The method 800 comprises determining 801 a respective plurality of raster cell images 111 for the print image 120 and the target print image 121 based on a subdivision of the print image and the target print image into raster cells 130, as described above for the Figures 1 and 2 described.

[0115] The method 800 includes determining 802 a Delta-E raster cell image 112 for each raster cell 130 based on a color distance between a raster cell image 111 of the print image 120 and a raster cell image 111 of the target print image 121, such as described above for the Figures 1 and 2 described

[0116] The method 800 comprises determining 803 for each pixel of the Delta-E raster cell image 112, based on a pixel-specific threshold function 140, whether a pixel defect 113 is present, wherein the pixel-specific threshold function 140 is based on a previous training 150 of a database 160 of Delta-E raster cell images with associated manually specified ground-truth raster cell images, such as described above for the Figures 1 and 2 described.

[0117] The method comprises assembling 804 the previously determined pixel defects 113 in the respective Delta-E raster cell images 112 to form a defect image 114, which provides an overview of the pixel defects 113 of the printed image 120, as described above for example in relation to the Figures 1 and 2 described.

[0118] Furthermore, the method 800 comprises outputting 805 an inspection result 116 based on a user-specific classification 115 of the defect image 114, such as described above for the Figures 1 and 2 described. LIST OF REFERENCE SYMBOLS

[0119] 100Print inspection device 110Processor 111Raster cell images 112Delta-E raster cell images 113Pixel defects 114Defect image 115(Customer-specific) classification 116Inspection result 120Print image 121Target print image 130Grid 140Threshold function 150Training 160Database with Delta-E images and ground-truth images 170User 200Print inspection device 201Delta E raster cell image 202Activation function per pixel 203Classification layer 204Defect image 205Customer-specific classification 206Inspection result 210First layer 211First branch of the first layer with no defect 212Second branch of the first layer with defect 213Raster cell image of the target print image (first layer) 214Raster cell image of the (actual) print image (first layer) 215Delta E raster cell image (first layer) 216Connection component 217Connection component 220Second layer 221First branch of the second layer with no defect 222Second branch of the second layer with no defect 223Raster cell image of the target print image (second layer) 224Raster cell image of the (actual) print image (second layer) 225Delta E raster cell image (second layer) 226Connected component 227Connected component 230n-th layer 231First branch of the n-th layer with no defect 232Second branch of the n-th layer with defect 233Raster cell image of the target print image (n-th layer) 234Raster cell image of the (actual) print image (n-th layer) 235Delta E raster cell image (n-th layer) 236Connected component 237Connected component 240Model 241Target print image 242Target print image after Color calibration 243Division of the target print image into raster cells 244Grid or grid 245Grid layer, local position correction 246Color channels 247Color space, here RGB 250Input image or(Actual) print image 251Input image after white balance 252Input image after alignment with target print image 253Input image after color channel splitting 254Input image after grid cell splitting 255Color calibration 256Alignment (input image with respect to target print image) . 301first image 302second image 311place with printing error in the first image 312Place with printing error in the second image 313Place with printing error in the second image 401 Actual raster cell image of the first series 402 Model of the first series 403 Ground-truth raster cell image of the first series 411 Actual raster cell image of the second series 412 Model of the second series 413 Ground-truth raster cell image of the second series 421 Actual raster cell image of the third series 422 Model of the third series 423 Ground-truth raster cell image of the third series 500Training of a Delta E grid cell image 501Model, ie target value grid cell image 502Input, ie actual value grid cell image 503Delta E grid cell image 504Ground-truth grid cell image 505Threshold function 506Optimal threshold 507Training 601 Target print image 602 (Actual) print image 603 Defect image or error image 610 Print error as colored spot or dot 611 Print error as lines or hatching 700Training the classification layer 701Real defects 702Pseudo-defects 703Classification layer 704Defect classifier 800Method for the optical inspection of a printed image 801first method step 802second method step 803third method step 804fourth method step 805fifth method step

Claims

1. A print inspection device (100, 200) for optically inspecting a print image (120) of a print object, wherein a target print image (121) is assigned to the print object, the print inspection device (100, 200) comprising: a processor (110), which is designed and configured to determine a plurality of raster cell images (111) for the print image (120) and the target print image (121), respectively, based on a subdivision of the print image (120) and the target print image (121) into raster cells (130); to determine for each raster cell (130) a Delta-E raster cell image (112) based on a color distance between a raster cell image of the print image (120) and a raster cell image of the target print image (121); to determine for each pixel of the Delta-E raster cell image (112) whether a pixel defect (113) is present based on a pixel-specific threshold function (140), wherein the pixel-specific threshold function (140) is based on a previous training (150) of a database (160) of Delta-E raster cell images with associated manually specified ground-truth cell images; to combine the previously determined pixel defects (113) in the respective Delta-E raster cell images (112) to a defect image (114), which provides an overview of the pixel defects (113) of the printed image (120); and to output an inspection result (116) based on a user-specific classification (115) of the defect image (114).

2. The print inspection device (100, 200) according to claim 1, wherein the processor (110) is configured to determine the plurality of raster cell images (111) based on color-calibrated (255) print images (120) and target print images (121), which are aligned (256) with each other.

3. The print inspection device (100, 200) according to claim 1 or 2, wherein the processor (110) is configured to determine the plurality of raster cell images (111) for a respective color channel of a plurality of color channels (246) of a color space (247).

4. The print inspection device (100, 200) according to one of the preceding claims, wherein the processor (110) is configured to determine the Delta-E raster cell image (112) based on a Euclidean distance in the color space between the raster cell image (111) of the print image (120) and the corresponding raster cell image (111) of the target print image (121).

5. The print inspection device (100, 200) according to one of the preceding claims, wherein the previous training (150) of the database (160) of Delta-E cell images with associated ground-truth cell images is carried out based on neural networks (NN), support vector machines (SVM) and / or multi-layer perceptron (MLP) methods.

6. The print inspection device (100, 200) according to one of the preceding claims, wherein the pixel-specific threshold function (140) specifies a fragment (224, 611) in the Delta-E raster cell image (112), which is based on an irregularity in an alignment of the print image (120) with respect to the target print image (121) as not being a pixel defect (113).

7. The print inspection device (100, 200) according to one of the preceding claims, wherein the pixel-specific threshold function (140) specifies a contiguous region (214, 610) of pixels in the Delta-E raster cell image (112), that exceeds a predetermined size, as a pixel defect (113).

8. The print inspection device (100, 200) according to one of the preceding claims, wherein both the training (150) of the database (160) of Delta-E cell images with associated ground-truth cell images and the user-specific classification (115) of the defect image (114) are based on the same training process (150).

9. The print inspection device (100, 200) according to claim 8, wherein the training process (150) is based on a classification of a database of print images (401, 411, 421) with respect to associated target value print images (402, 412, 422) based on a subjective error detection of a user.

10. The print inspection device (100, 200) according to one of the preceding claims, wherein the pixel-specific threshold function (140, 505) assigns a binary value 0 or 1 to each gray value of the Delta-E raster cell image (112).

11. The print inspection device (100, 200) according to claim 10, wherein the pixel-specific threshold function (140, 505) comprises a logistic function or a step function with respect to the gray values of the Delta-E raster cell image (503).

12. The print inspection device (100, 200) according to one of the preceding claims, wherein the classification (115) of the defect image (114) is based on the following features: an area of connected components of the Delta-E raster cell image (503), an inner radius of the connected components of the Delta-E raster cell image (503), a correlation between the actual print image (120) and the target print image (121), a contrast of the Delta-E raster cell image (503), in particular a color difference between background and foreground.

13. The print inspection device according to one of the preceding claims, comprising: a camera, which is configured to optically record the print object in order to obtain the print image.

14. The print inspection device (100, 200) according to one of the preceding claims, wherein the processor (110) is configured to output a printing error as an inspection result (116) when printing the print image (120) with a color printer, wherein the printing error occurs in the form of a color dot or color stripe that is based on a malfunction of a color channel (246) of the color printer.

15. A method (800) for the optical inspection of a print image (120) of a print object, wherein a target print image (121) is assigned to the print object, the method (800) comprising the following steps: determining (801) a respective plurality of raster cell images (111) for the print image (120) and the target print image (121) based on a subdivision of the print image and the target print image into raster cells (130); determining (802) for each raster cell (130) a Delta-E raster cell image (112) based on a color distance between a raster cell image (111) of the print image (120) and a raster cell image (111) of the target print image (121); determining (803) for each pixel of the Delta-E raster cell image (112) whether a pixel defect (113) is present based on a pixel-specific threshold function (140), wherein the pixel-specific threshold function (140) is based on a previous training (150) of a database (160) of Delta-E raster cell images with associated manually specified ground-truth raster cell images; combining (804) the previously determined pixel defects (113) in the respective Delta-E raster cell images (112) to a defect image (114), which provides an overview of the pixel defects (113) of the printed image (120); and outputting (805) an inspection result (116) based on a user-specific classification (115) of the defect image (114).

Images