Printed matter inspection device, printed matter inspection method, program and printing system

Abstract

To provide a printed matter inspection device capable of improving accuracy of inspection of a plurality of types of defects of printed matter and improving a processing speed, a printed matter inspection method, a program and a printing system.SOLUTION: A printed matter inspection device for detecting a plurality of types of defects of printed matter by using inspection data acquired based on a captured image obtained by imaging the printed matter and reference data, the digital image of a print object image represented by print data of the printed matter, the printed matter inspection device comprises one or more processors, and one or more storage devices. The one or more processors acquire the reference data, execute, before inspection data is acquired, a plurality of types of preprocessing different depending on defect types of a detection object on the reference data as preprocessing for converting the reference data in order to compare the reference data with the inspection data, generate a plurality of comparison reference data different depending on the defect types of the detection object from the reference data, and cause the one or more storage devices to hold the plurality of comparison reference data.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to a printed matter inspection device, a printed matter inspection method, a program, and a printing system, and in particular to an image processing technique for detecting defects in printed matter. [Background technology]

[0002] Patent Document 1 describes a method for generating a machine learning model for detecting defects in a printed matter by comparing inspection data acquired based on a captured image of a printed matter with reference data acquired based on print digital data using a machine learning model. The method described in Patent Document 1 includes an acquisition step of acquiring learning inspection data based on a captured image of a printed matter to be inspected that is printed based on learning print digital data, learning defect information of the learning inspection data estimated by comparing the learning inspection data with first learning reference data based on a captured image of a reference printed matter that is printed based on the learning print digital data, and second learning reference data based on the learning print digital data, and a generation step of generating a machine learning model by using at least the learning inspection data and the second learning reference data as learning input information and using at least the learning defect information as learning output information.

[0003] Patent Document 2 describes a print inspection device that includes a print data acquisition unit that acquires print data to be input to a printing device when generating a printed matter, an image data acquisition unit that acquires image data of the printed matter, an alignment processing unit that aligns the print data and the image data by applying a specified alignment accuracy, and an inspection processing unit that acquires defect information of the image data based on the print data using the print data and the image data that have been aligned using the alignment processing unit, in which the inspection processing unit applies a learning model that learns the relationship between defect candidates detected from the image data and defects in the printed matter, and the learning model learns using a disturbance-added learning data set that receives as input disturbance-added image data to which a disturbance has been added that shifts the position of the image data within the range of the alignment accuracy in a direction intersecting the relative transport direction between the printing unit and the medium in the printing device, and outputs defect information of the disturbance-added image data. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7110349 [Patent Document 2] International Publication No. 2021 / 054200 Summary of the Invention [Problem to be solved by the invention]

[0005] Patent Document 1 describes a machine learning model that targets various types of defects in printed matter, such as missing ink, dripping ink, dust adhesion, other dot-like defects, unevenness, scratches, or streak-like defects caused by nozzle defects in inkjet nozzles, etc. However, Patent Document 1 assumes that multiple types of defects will be detected using one (single) model, and the same model is used for processing multiple types of defects, which poses a problem in that it is difficult to improve the accuracy of inspection.

[0006] Furthermore, if one attempts to process multiple types of defects using the same model, it is necessary to extract many features from both the inspection data and the reference data, which results in an enormous number of model parameters and increased processing time.

[0007] On the other hand, the invention described in Patent Document 2 aims to solve the problem of suppressing false detection in the detection of streak defects that occur along one direction of a printed matter, and can improve the detection accuracy for a specific type of defect, namely streak defects. However, the invention described in Patent Document 2 has a problem in that it cannot fully handle the detection of multiple types of defects including defects other than streak defects.

[0008] In addition to the above-mentioned issues, when performing real-time inspection on each of multiple printed items that are printed in succession during a print production process, the inspection process for one printed item needs to be carried out within a short period of time before an image of the next printed item can be obtained.

[0009] To improve the accuracy of inspection, complex processing must be performed to extract defects in more detail from the image of the inspection target. However, complex processing takes time, so there is a trade-off between processing time and accuracy. Furthermore, to inspect for multiple types of defects, the processing becomes even more complex, making it even more difficult to improve processing speed (reducing processing time) and improve accuracy at the same time.

[0010] The present disclosure has been made in consideration of the above circumstances, and aims to provide a print inspection device, a print inspection method, a program, and a printing system that can improve the accuracy and processing speed of inspection for multiple types of defects that can occur in printed matter. [Means for solving the problem]

[0011] A print inspection device according to a first aspect of the present disclosure is a print inspection device that detects multiple types of defects on a printed matter using inspection data acquired based on an image of the printed matter and reference data, which is a digital image of the image to be printed indicated by the print data of the printed matter, and is equipped with one or more processors and one or more storage devices, and the one or more processors acquire the reference data and, as a pre-processing step of converting the reference data for comparing the reference data with the inspection data, perform multiple pre-processing steps on the reference data before the inspection data is acquired, which differ depending on the type of defect to be detected, to generate multiple comparison reference data from the reference data that differ depending on the type of defect to be detected, and store the multiple comparison reference data in one or more storage devices.

[0012] According to the first aspect, the reference data to be compared with the inspection data is a digital image obtained based on the print data without going through the printing action by the printing device, so that one or more processors can obtain the reference data prior to obtaining the inspection data. According to the first aspect, before obtaining the inspection data, a plurality of pre-processing steps according to the type of defect to be inspected are performed on the reference data in advance, and a plurality of comparison reference data that differs depending on the type of defect to be detected is stored, so that the number of steps of the process dynamically performed after obtaining the inspection data can be reduced, and the processing speed can be improved.

[0013] In addition, a processing method suitable for detecting defects belonging to each type of defect to be detected can be adopted, and time-consuming and complex pre-processing can be performed on the reference data, thereby making it possible to improve the detection accuracy for each type of defect to be detected.

[0014] A print inspection device according to a second aspect of the present disclosure may be configured such that, in the print inspection device according to the first aspect, the plurality of pre-processing processes include a first feature extraction process that extracts a feature indicative of an image feature of the reference data. The feature extracted by the first feature extraction process can be used as comparison reference data. The first feature extraction process may extract a feature from the reference data, or may extract a feature from an image obtained by image-converting the reference data. Since the feature indicative of the image feature of the reference data has a smaller data volume than the reference data itself, the storage capacity required for storing data can be reduced by storing the feature as comparison reference data instead of storing the reference data.

[0015] A print inspection device according to a third aspect of the present disclosure may be configured in such a way that, in the print inspection device according to the second aspect, the first feature extraction process is performed using a first deep learning model.

[0016] As one of the multiple pre-processing processes, a first feature extraction process using a first deep learning model can be applied.

[0017] A print inspection device according to a fourth aspect of the present disclosure is a print inspection device according to any one of the first to third aspects, wherein the multiple pre-processing steps include an image conversion process that brings the data structure of the reference data closer to the data structure of the inspection data.

[0018] For example, as one of the multiple pre-processing processes, a first feature extraction process using a first deep learning model of the third aspect can be applied, and as the other, an image conversion process of the fourth aspect can be applied.

[0019] A print inspection device according to a fifth aspect of the present disclosure may be configured in such a way that, in the print inspection device according to the fourth aspect, the image conversion process includes a resolution conversion process.

[0020] A print inspection device according to a sixth aspect of the present disclosure may be configured such that, in the print inspection device according to the fourth or fifth aspect, the image conversion processing includes color conversion processing that converts from the color space of the reference data to the color space of the captured image.

[0021] A print inspection device according to a seventh aspect of the present disclosure may be configured in such a way that, in the print inspection device according to the sixth aspect, the reference data is represented as CMYK data including color information for cyan, magenta, yellow, and black, and the captured image is represented as RGB data including color information for red, green, and blue, and the color conversion process includes a process of converting from CMYK data to RGB data.

[0022] A print inspection device according to an eighth aspect of the present disclosure may be configured such that, in the print inspection device according to any one of the fourth to seventh aspects, the image conversion process includes a noise addition process that adds noise contained in the inspection data to the reference data.

[0023] A print inspection device according to a ninth aspect of the present disclosure may be configured in such a way that, in the print inspection device according to the eighth aspect, the noise contained in the inspection data includes at least one of geometric noise and optical noise.

[0024] A print inspection device according to a tenth aspect of the present disclosure may be configured such that, in the print inspection device according to any one of the first to ninth aspects, the multiple pre-processing steps include at least one of an edge extraction process and a blurring process.

[0025] A print inspection device according to an eleventh aspect of the present disclosure may be configured such that, in the print inspection device according to the tenth aspect, the multiple pre-processing steps include an edge extraction process and a blurring process, and the one or more processors are configured to store edge extraction data obtained by performing the edge extraction process and blurred data obtained by performing the blurring process in one or more storage devices.

[0026] A print inspection device according to a twelfth aspect of the present disclosure may be configured in a print inspection device according to any one of the first to eleventh aspects, wherein one or more processors acquire inspection data, generate a plurality of comparison data based on the inspection data, which differ depending on the type of defect to be detected, and perform a comparison process in which each of the plurality of comparison data is compared with corresponding comparison reference data for each type of defect to be detected.

[0027] A print inspection device according to a thirteenth aspect of the present disclosure may be configured in such a way that, in the print inspection device according to the twelfth aspect, the one or more processors perform a plurality of processes on the inspection data that differ depending on the type of defect to be detected in order to obtain a plurality of comparison data, and the plurality of processes performed on the inspection data include a second feature extraction process that extracts features of the inspection data.

[0028] A print inspection device according to a fourteenth aspect of the present disclosure may be configured in such a way that, in the print inspection device according to the thirteenth aspect, the second feature extraction process is performed using a second deep learning model.

[0029] A printed matter inspection device according to a fifteenth aspect of the present disclosure may be configured such that, in the printed matter inspection device according to the twelfth to fifteenth aspects, the comparison process includes processing using a machine learning model that is trained to receive input of comparison data and corresponding comparison reference data and output at least one piece of information regarding the presence or absence of a defect, the intensity of the defect, and the position of the defect.

[0030] A print inspection device according to a sixteenth aspect of the present disclosure is the print inspection device according to the fifteenth aspect, wherein the machine learning model is a third deep learning model.

[0031] A printed matter inspection device according to a seventeenth aspect of the present disclosure may be configured in such a way that, in the printed matter inspection device according to the twelfth to sixteenth aspects, the comparison process includes a difference process that calculates the difference between the comparison data and corresponding comparison reference data.

[0032] A printed matter inspection device according to an 18th aspect of the present disclosure may be configured in such a way that, in the printed matter inspection device according to any one of the 1st to 17th aspects, the multiple types of defects include a first defect which is a streak-like defect and a second defect other than the first defect.

[0033] A printed matter inspection device according to a 19th aspect of the present disclosure may be configured such that, in the printed matter inspection device according to the 18th aspect, one or more processors perform a compression process on each of the reference data and the inspection data to compress image information in the transport direction of the printing medium when the printed matter is printed, in order to perform a process to detect a first defect, and perform a low-resolution process on each of the reference data and the inspection data in order to perform a process to detect a second defect.

[0034] A printed matter inspection device according to a 20th aspect of the present disclosure may be configured in the printed matter inspection device described in the 18th or 19th aspect, so that, as a first pre-processing step in the multiple pre-processing steps, in order to detect a first defect, the one or more processors extract features indicating image features of the reference data using a first deep learning model and store the extracted features in one or more storage devices as first comparison reference data in the multiple comparison reference data, and, as a second pre-processing step in the multiple pre-processing steps, in order to detect a second defect, perform an image conversion process to bring the data structure of the reference data closer to the data structure of the inspection data, and store the converted data obtained by the image conversion process in one or more storage devices as second comparison reference data in the multiple comparison reference data.

[0035] A print inspection device according to a 21st aspect of the present disclosure may be configured in such a way that, in the print inspection device according to the 20th aspect, the image conversion processing includes at least one of resolution conversion processing, color conversion processing, edge extraction processing, blurring processing, and noise addition processing.

[0036] A printing system according to a 22nd aspect of the present disclosure includes a print inspection device according to any one of the 1st to 21st aspects, a printing device that prints based on print data and generates a printed material, and an imaging device that images the printed material.

[0037] A printed matter inspection method according to a 23rd aspect of the present disclosure is a printed matter inspection method that detects multiple types of defects in a printed matter using inspection data acquired based on an image of the printed matter and reference data, which is a digital image of the image to be printed indicated by the print data of the printed matter, in which one or more processors acquire the reference data, and as a pre-processing step of converting the reference data for comparing the reference data with the inspection data, before the inspection data is acquired, perform multiple pre-processing steps on the reference data that differ depending on the type of defect to be detected to generate multiple comparison reference data from the reference data that differ depending on the type of defect to be detected, and store the multiple comparison reference data in one or more storage devices.

[0038] The print inspection method according to the twenty-third aspect may have a configuration including the same specific aspects as the print inspection device according to any one of the second to twenty-first aspects.

[0039] A program according to a 24th aspect of the present disclosure is a program that enables a computer to realize the function of detecting multiple types of defects in a printed matter using inspection data acquired based on an image of the printed matter and reference data, which is a digital image of the image to be printed indicated by the print data of the printed matter, and enables the computer to realize the following functions: a function of acquiring the reference data; a function of performing multiple pre-processing steps on the reference data before the inspection data is acquired, which differ depending on the type of defect to be detected, as a pre-processing step to convert the reference data in order to compare the reference data with the inspection data, to generate multiple comparison reference data from the reference data that differ depending on the type of defect to be detected; and a function of storing the multiple comparison reference data in one or more storage devices.

[0040] The program according to the 24th aspect may be configured to include the same specific aspects as those of the print inspection device according to any one of the 2nd to 21st aspects. In addition, a non-transitory computer-readable recording medium (computer-readable medium) that is a tangible object on which the program according to the 24th aspect is recorded is also included in the scope of the present disclosure. Effect of the Invention

[0041] According to the present disclosure, it is possible to improve both the accuracy of inspection for multiple types of defects that may occur in printed matter and the processing speed. [Brief description of the drawings]

[0042] [Figure 1] FIG. 1 is an overall configuration diagram that shows an outline of an example of the configuration of an inkjet printing machine to which a print inspection device according to an embodiment is applied. [Diagram 2] FIG. 2 is a block diagram showing an example of the hardware configuration of a control device that controls the inkjet printing machine. [Diagram 3] FIG. 3 is a functional block diagram showing the schematic configuration of a control system in a printing system including an inkjet printer and a control device. [Figure 4] FIG. 4 is a flowchart showing an overview of an inspection process for comparing an original image with a scanned image. [Diagram 5] FIG. 5 is a flowchart showing an example of an inspection process corresponding to each of the two types of defects. [Figure 6] FIG. 6 is a flowchart showing an example of an inspection process in which an original image and a scanned image are compared on a pixel-by-pixel basis. [Figure 7] FIG. 7 is an explanatory diagram showing an example of a noise addition process for an original image. [Figure 8] FIG. 8 is an explanatory diagram showing an example of processing for each type of defect to be detected. [Figure 9] FIG. 9 is a block diagram showing the functional configuration of the print inspection device according to the embodiment. [Figure 10] FIG. 10 is an explanatory diagram showing an example of an inspection process using deep learning. [Figure 11] FIG. 11 is a flowchart showing an example of the operation of a printing system including a print inspection device according to an embodiment. [Figure 12]FIG. 12 is a flowchart showing an example of the operation of a printing system including a print inspection device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0043] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification, the same components are given the same reference numerals, and duplicated explanations will be omitted as appropriate.

[0044] Example of an inkjet printer configuration FIG. 1 is an overall configuration diagram that shows a schematic example of the configuration of an inkjet printer 1 to which a print inspection device according to an embodiment is applied. The inkjet printer 1 is a color digital printing device that prints a desired image on a sheet of paper P by a single pass method using four colors of ink: cyan (C), magenta (M), yellow (Y), and black (K). In this embodiment, an example will be described in which aqueous ink is used as the ink for drawing. An aqueous ink refers to an ink in which a coloring material such as a pigment or dye is dissolved or dispersed in water and / or a solvent soluble in water.

[0045] The inkjet printer 1 includes a paper feed section 10, a treatment liquid deposition section 20, a treatment liquid drying section 30, a drawing section 40, an ink drying section 50, and an accumulation section 60.

[0046] The paper feed section 10 includes a paper feed device 12, a feeder board 14, and a paper feed drum 16. Paper P is placed on the paper feed tray 12A in a state of a stack of many sheets. The type of paper P is not particularly limited, but for example, printing paper mainly made of cellulose, such as high-quality paper, coated paper, and art paper, can be used.

[0047] The paper feeder 12 takes out the paper sheets P from a stack set on the paper feed tray 12A one by one from the top, and supplies the paper sheets P to the feeder board 14. The feeder board 14 transports the paper sheets P received from the paper feeder 12 to the paper feed drum 16.

[0048] The paper feed drum 16 receives the paper P fed from the feeder board 14 , and transports the received paper P to the treatment liquid deposition unit 20 .

[0049] The treatment liquid application unit 20 applies treatment liquid to the paper P. The term "treatment liquid" is synonymous with "pretreatment liquid". The treatment liquid may also be called a "precoat", "preconditioner", "undercoat liquid", or "treatment agent". The treatment liquid is a liquid that has the function of aggregating, insolubilizing, or thickening the coloring material components in the ink. The treatment liquid application unit 20 includes a treatment liquid application drum 22 and a treatment liquid application device 24.

[0050] The treatment liquid application drum 22 receives the paper P from the paper feed drum 16, and transports the received paper P to the treatment liquid drying section 30. The treatment liquid application drum 22 is provided with a gripper 23 on the drum circumferential surface, and by rotating while gripping the leading end of the paper P with the gripper 23, the paper P is wrapped around the drum circumferential surface and transported.

[0051] The treatment liquid application device 24 includes an application roller, and applies treatment liquid to the paper P transported by the treatment liquid application drum 22. The application roller is supported by a contact / separation mechanism (not shown) that can move between an application position where the application roller comes into contact with the paper P to apply treatment liquid to the paper P, and a retracted position where the application roller is separated from the paper P and does not apply treatment liquid. The method for applying the treatment liquid is not limited to the roller application method, and may be a blade application method, an inkjet method, a spray method, or the like.

[0052] The treatment liquid drying section 30 dries the paper P on which the treatment liquid has been applied. The treatment liquid drying section 30 includes a treatment liquid drying drum 32 and a hot air blower 34. The treatment liquid drying drum 32 receives the paper P from the treatment liquid application drum 22, and transports the received paper P to the imaging section 40. The treatment liquid drying drum 32 includes a gripper 33 on the drum circumferential surface. The treatment liquid drying drum 32 transports the paper P by rotating while gripping the leading end of the paper P with the gripper 33.

[0053] The hot air blower 34 is installed inside the treatment liquid drying drum 32. The hot air blower 34 blows hot air onto the paper P transported by the treatment liquid drying drum 32, to dry the treatment liquid.

[0054] The imaging section 40 includes an imaging drum 42, a head unit 44, a paper pressing roller 47, and a scanner 48. The imaging drum 42 receives the paper P from the treatment liquid drying drum 32 and transports the received paper P to the ink drying section 50. The imaging drum 42 includes a gripper 43 on the drum circumferential surface, and rotates while gripping the leading end of the paper P with the gripper 43, thereby wrapping the paper P around the drum circumferential surface and transporting it. The imaging drum 42 includes a suction mechanism (not shown), and transports the paper P wrapped around the drum circumferential surface by adsorbing it to the drum circumferential surface. Negative pressure is used for adsorption. The imaging drum 42 includes a number of suction holes on the circumferential surface, and adsorbs the paper P to the circumferential surface of the imaging drum 42 by sucking the paper P from inside the imaging drum 42 through the suction holes.

[0055] The head unit 44 is configured to include inkjet heads 46C, 46M, 46Y, and 46K. The inkjet head 46C is a recording head that ejects droplets of cyan ink. The inkjet head 46M is a recording head that ejects droplets of magenta ink. The inkjet head 46Y is a recording head that ejects droplets of yellow ink. The inkjet head 46K is a recording head that ejects droplets of black ink. Ink is supplied to each of the inkjet heads 46C, 46M, 46Y, and 46K from an ink tank (not shown), which is an ink supply source of the corresponding color, via a piping path (not shown).

[0056] Each of the inkjet heads 46C, 46M, 46Y, and 46K is configured as a line head capable of printing on the paper P transported by the imaging drum 42 in one scan, i.e., by a single pass method. The inkjet heads 46C, 46M, 46Y, and 46K are disposed such that their respective nozzle surfaces face the circumferential surface of the imaging drum 42. The inkjet heads 46C, 46M, 46Y, and 46K are disposed at regular intervals along the transport path of the paper P by the imaging drum 42.

[0057] Although not shown in Fig. 1, a plurality of nozzles, which are ink ejection ports, are two-dimensionally arranged on the nozzle surface of each of the inkjet heads 46C, 46M, 46Y, and 46K. The term "nozzle surface" refers to the ejection surface on which the nozzles are formed, and is synonymous with terms such as "ink ejection surface" or "nozzle formation surface." A nozzle arrangement of a plurality of nozzles arranged two-dimensionally is called a "two-dimensional nozzle array."

[0058] Each of the inkjet heads 46C, 46M, 46Y, and 46K can be configured by connecting a plurality of head modules in the paper width direction. The paper width here refers to the paper width in a direction perpendicular to the transport direction of the paper P. The transport direction of the paper P is called the Y direction. The paper width direction perpendicular to the Y direction is called the X direction. Each of the inkjet heads 46C, 46M, 46Y, and 46K is a line-type recording head having a nozzle row capable of recording an image at a specified printing resolution in one scan over the entire recording area of ​​the paper P in the X direction. Such a recording head is also called a "full line-type recording head" or a "page wide head."

[0059] The specified print resolution may be a print resolution that is predetermined by the inkjet printer 1, or may be a print resolution that is selected by the user or that is automatically selected by a program according to the print mode. The print resolution may be, for example, 1200 dpi (dots per inch) in the X direction and 1200 dpi in the Y direction.

[0060] The paper width direction (X direction) perpendicular to the transport direction of the paper P is sometimes called the nozzle row direction of the line head, and the transport direction of the paper P (Y direction) is sometimes called the nozzle row perpendicular direction.

[0061] In the case of an inkjet head having a two-dimensional nozzle array, a projected nozzle array in which each nozzle in the two-dimensional nozzle array is projected (orthogonally projected) to be aligned along the nozzle array direction can be considered equivalent to a nozzle array in which each nozzle is aligned at approximately equal intervals in the nozzle array direction at a nozzle density that achieves the maximum recording resolution. "Approximately equal intervals" means that the nozzles are substantially equal intervals as droplet ejection points that can be recorded by the inkjet printer 1. For example, the concept of "equal intervals" also includes cases in which the intervals are slightly different in consideration of the movement of droplets on the medium due to manufacturing errors and / or landing interference. The projected nozzle array corresponds to a substantial nozzle array. When the projected nozzle array is taken into consideration, a nozzle number indicating the nozzle position can be associated with each nozzle in the order of the projected nozzles aligned along the nozzle array direction. The nozzle array direction is synonymous with the nozzle alignment direction.

[0062] The nozzle arrangement in each of the inkjet heads 46C, 46M, 46Y, and 46K is not limited, and various nozzle arrangements can be adopted. For example, instead of a two-dimensional matrix arrangement, a linear nozzle arrangement in one row, a V-shaped nozzle arrangement, a broken line nozzle arrangement such as a W-shape in which V-shaped arrangements are repeated, and the like are also possible.

[0063] Ink droplets are ejected from at least one of the inkjet heads 46C, 46M, 46Y, and 46K toward the paper P transported by the drawing drum 42, and an image is formed on the paper P by the ejected droplets adhering to the paper P.

[0064] The imaging drum 42 functions as a relative movement mechanism that moves the inkjet heads 46C, 46M, 46Y, and 46K relative to the paper P. The imaging drum 42 is one form of a mechanism that moves the paper P relatively to the inkjet heads 46C, 46M, 46Y, and 46K. The ejection timing of each of the inkjet heads 46C, 46M, 46Y, and 46K is synchronized with a rotary encoder signal obtained from a rotary encoder (not shown) that is installed on the imaging drum 42. The ejection timing is the timing at which ink droplets are ejected, and is synonymous with droplet ejection timing.

[0065] In this example, a configuration using four colors of ink, CMYK, is illustrated, but the combination of ink colors and the number of colors is not limited to this embodiment, and light ink, dark ink, special color ink, etc. may be added as necessary. For example, a configuration adding an inkjet head that ejects light-colored ink such as light cyan or light magenta, and / or a configuration adding an inkjet head that ejects special color ink such as green, orange, or white, etc. are also possible. Furthermore, the arrangement order of the inkjet heads of each color is not particularly limited.

[0066] Scanner 48 is an image reading device that optically reads an image recorded on paper P by inkjet heads 46C, 46M, 46Y, 46K and generates electronic image data representing the read image. Scanner 48 includes an imaging device that captures an image recorded on paper P and converts it into an electrical signal representing image information. In addition to the imaging device, scanner 48 may include an illumination optical system that illuminates the object to be read and a signal processing circuit that processes a signal obtained from the imaging device to generate digital image data.

[0067] The scanner 48 is preferably configured to be capable of reading color images. In the scanner 48 of this example, for example, a color CCD (Charge-Coupled Device) linear image sensor is used as an imaging device. The color CCD linear image sensor is an image sensor in which light receiving elements having color filters of R (red), G (green), and B (blue) are linearly arranged. Note that a color CMOS (Complementary Metal Oxide Semiconductor) linear image sensor can be used instead of the color CCD linear image sensor. Also, instead of an imaging device having a primary color filter, an imaging device having a complementary color filter can be adopted. The scanner 48 reads an image on the paper P while the paper P is being transported by the drawing drum 42. A scanner installed on the paper transport path in this way may be called an "inline scanner." The scanner 48 may also be a camera. The paper P on which the image is recorded is an example of a "printed material" in this disclosure. The scanner 48 is an example of an "imaging device" in this disclosure. The read image obtained by the scanner 48 is an example of a "captured image" in this disclosure.

[0068] When the paper P, on which an image has been recorded using at least one of the inkjet heads 46C, 46M, 46Y, and 46K, passes through the reading area of ​​the scanner 48, the image on the paper P is read. The image recorded on the paper P may include, in addition to the image to be printed (user image) specified in the print job, a defective nozzle detection pattern for inspecting the ejection state of each nozzle, a test pattern for correcting uneven density, and various other test images.

[0069] The print image is inspected using the data of the read image read by the scanner 48, and the presence or absence of image quality abnormalities is judged. Details of the inspection process will be described later. Based on the data of the read image read by the scanner 48, information such as the density of the image and the ejection state of each nozzle of the inkjet heads 46K, 46C, 46M, and 46Y is obtained. The scanner 48 in this example is disposed on the paper transport path between the head unit 44 and the ink drying section 50, and is configured to read the image before the ink dries, but it is also possible to dispose a scanner in place of or in combination with the scanner 48 so as to read the image after the ink dries.

[0070] The ink drying section 50 dries the paper P on which the image has been formed by the drawing section 40. The ink drying section 50 includes a chain gripper 70, a paper guide 80, and a heat drying processing section 90.

[0071] The chain gripper 70 receives the paper P from the imaging drum 42, and transports the received paper P to the accumulation section 60. The chain gripper 70 includes a pair of endless chains 72 that travel along a specified travel path, and transports the paper P along a specified transport path while gripping the leading edge of the paper P with grippers 74 provided on the pair of chains 72. A plurality of grippers 74 are provided on the chains 72 at regular intervals.

[0072] The chain gripper 70 of this example is configured to include a first sprocket 71A, a second sprocket 71B, a chain 72, and a plurality of grippers 74, and has a structure in which a pair of endless chains 72 are wound around the pair of first sprockets 71A and second sprockets 71B. Only one of the pair of first sprockets 71A and second sprockets 71B and the pair of chains 72 is shown in Fig. 1.

[0073] The chain gripper 70 has a structure in which a plurality of grippers 74 are arranged at a plurality of positions in the feed direction (length direction) of the chain 72. The chain gripper 70 also has a structure in which a plurality of grippers 74 are arranged along the paper width direction between a pair of chains 72. Of the plurality of grippers 74 arranged between the pair of chains 72, only one gripper 74 is shown in Fig. 1.

[0074] The transport path of the paper P by the chain gripper 70 includes a horizontal transport area where the paper P is transported horizontally, and an inclined transport area where the paper P is transported diagonally upward from the end of the horizontal transport area. The horizontal transport area is referred to as the first transport section, and the inclined transport area is referred to as the second transport section.

[0075] The paper guide 80 is a mechanism that guides the transport of the paper P by the chain gripper 70. The paper guide 80 includes a first paper guide 82 and a second paper guide 84. The first paper guide 82 guides the paper P transported through the first transport section of the chain gripper 70. The second paper guide 84 guides the paper transported through the second transport section following the first transport section. Although the detailed structure of the first paper guide 82 is not shown, an adsorption transport device using a suction-type adsorption belt is applied as the first paper guide 82.

[0076] The heating and drying processing unit 90 applies heat to the paper P on which an image has been formed by the drawing unit 40 to evaporate the ink solvent and dry the paper P. The heating and drying processing unit 90 is, for example, a hot air blowing unit, and is disposed opposite the first paper guide 82, blowing hot air onto the paper P transported by the chain gripper 70. When ultraviolet-curable ink is used, an ultraviolet irradiation device is disposed instead of or in combination with the heating and drying processing unit 90.

[0077] The accumulation section 60 includes an accumulation device 62 that receives and accumulates the paper sheets P transported from the ink drying section 50 by the chain gripper 70. The chain gripper 70 releases the paper sheets P at a predetermined accumulation position. The accumulation device 62 includes an accumulation tray 62A, receives the paper sheets P released from the chain gripper 70, and accumulates them in a bundle on the accumulation tray 62A. The accumulation section 60 corresponds to a paper discharge section.

[0078] Although not shown in FIG. 1, the inkjet printer 1 includes a sorting unit 55 that sorts printed matter according to the result of the inspection based on the image read by the scanner 48 in addition to the above configuration (see FIG. 2). The sorting unit 55 is disposed, for example, downstream of the ink drying unit 50 and upstream of the stacking unit 60 in the conveying direction of the paper P. The sorting unit 55 includes a stamper (not shown). The stamper is a device that performs a stamping process that applies ink as a mark indicating that the paper P is a defective printed matter, to the leading edge of the paper P that is a defective printed matter detected by the inspection. Furthermore, the sorting unit 55 may include a mechanism that switches the conveying path so as to lead the defective printed matter to a dedicated stacking tray instead of or in combination with the stamper.

[0079] 1, the inkjet printer 1 is also provided with a maintenance unit that performs maintenance processes such as cleaning of the inkjet heads 46K, 46C, 46M, and 46Y. The maintenance unit is installed alongside the imaging drum 42 in the axial direction of the imaging drum 42.

[0080] Example of hardware configuration of control device 2 is a block diagram showing an example of a hardware configuration of a control device 100 that controls the inkjet printer 1. The control device 100 is configured, for example, by a combination of computer hardware and software. The control device 100 includes a processor 102, a computer-readable medium 104 that is a non-transient tangible entity, a communication interface 106, and an input / output interface 108. The processor 102 includes a CPU (Central Processing Unit). The processor 102 may include a GPU (Graphics Processing Unit). The processor 102 is connected to the computer-readable medium 104, the communication interface 106, and the input / output interface 108 via a bus 110.

[0081] The computer-readable medium 104 includes, for example, a random access memory (RAM) 112, a read-only memory (ROM) 114, and a storage 116. The RAM 112 is a memory that functions as a main storage device. The storage 116 is an auxiliary storage device. The storage 116 may be, for example, a hard disk drive (HDD) device, a solid state drive (SSD) device, or a combination of a plurality of these. A part or the entirety of the storage area of ​​the computer-readable medium 104 may be included in the processor 102.

[0082] The computer-readable medium 104 stores programs and data for implementing the functions of the control device 100. The term "program" includes the concept of a program module. Some of the processing functions of the control device 100 may be implemented using integrated circuits such as a digital signal processor (DSP) or a field programmable gate array (FPGA).

[0083] The control device 100 is connected to an input device 122 and a display device 124. The input device 122 is, for example, a keyboard, a mouse, a multi-touch panel, or other pointing devices, or a voice input device, or an appropriate combination of these. The display device 124 is, for example, a liquid crystal display, an organic electro-luminescence (OEL) display, or a projector, or an appropriate combination of these. The input device 122 and the display device 124 may be integrally configured, such as a touch panel. The input device 122 and the display device 124 may be included in the control device 100, or the control device 100, the input device 122, and the display device 124 may be integrally configured.

[0084] <<Outline of the control system of the inkjet printer 1>> 3 is a functional block diagram showing a schematic configuration of a control system in a printing system 101 including the inkjet printer 1 and the control device 100. The processor 102 functions as a processing unit and / or a control unit that performs various processes by executing instructions of a program stored in a computer-readable medium 104. The processor 102 functions as a system control unit 310, an image processing unit 311, a conveyance control unit 312, a paper feed control unit 313, a treatment liquid application control unit 314, a treatment liquid drying control unit 316, a drawing control unit 318, an inspection processing unit 319, an ink drying control unit 320, a sorting control unit 322, and a paper discharge control unit 324. The processing functions of each of these units may be realized using multiple processors.

[0085] The control device 100 includes a communication unit 304, an image memory 332, a parameter storage unit 334, and a program storage unit 336. The communication unit 304 includes a communication interface 106 (see FIG. 2). The control device 100 is connected to a host computer 350 via the communication unit 304, and can transmit and receive data to and from the host computer 350. Note that the "connection" includes a wired connection, a wireless connection, or a combination of these. The communication unit 304 may be equipped with a buffer memory for accelerating communication processing. The communication unit 304 serves as an image input interface unit for acquiring image data representing an image to be printed. A part of the functions of the communication unit 304 may be realized by the processor 102.

[0086] Image data received from the host computer 350 via the communication unit 304 is stored in the image memory 332. The image memory 332 functions as a temporary storage unit for various data including image data.

[0087] The parameter storage unit 334 stores various parameters used in the inkjet printing machine 1. The various parameters stored in the parameter storage unit 334 are read out via the processor 102 and set in each unit of the device.

[0088] The program storage unit 336 stores programs used by each unit of the inkjet printing machine 1. The various programs stored in the program storage unit 336 are read out via the processor 102 and executed in each unit of the device. The computer-readable medium 104 functions as the image memory 332, the parameter storage unit 334, and the program storage unit 336.

[0089] The system control unit 310 functions as an overall control unit that comprehensively controls each unit of the inkjet printing machine 1. The system control unit 310 also functions as a calculation unit that performs various calculation processes. Furthermore, the system control unit 310 controls reading and writing of data from and to the computer-readable medium 104.

[0090] The image processing unit 311 performs various conversion processes, correction processes, and halftone processes on the image data to be printed. The conversion processes include pixel number conversion, tone conversion, and color conversion. The correction processes include density correction including unevenness correction for suppressing uneven density, and non-ejection correction for suppressing the visibility of image defects caused by defective nozzles. Halftone processing is generally a process of quantizing m-value (m is an integer of 3 or more) multi-tone image data and converting it into n-value (n is an integer of 2 or more and less than m) data. For example, multi-tone image data of 8 bits (256 gradations) or more for each color is essentially understood to be continuous tone image data. The image processing unit 311 converts continuous tone image data of each color of, for example, CMYK, into dot data representing a multi-value dot arrangement of 3 or more values ​​on a pixel-by-pixel basis.

[0091] In this embodiment, the inkjet heads 46C, 46M, 46Y, and 46K can eject three different droplet sizes (dot sizes): small, medium, and large. In this case, the image processor 311 converts the continuous tone color separation image data into a four-level (n=4) signal of "eject large ink droplets," "eject medium ink droplets," "eject small ink droplets," and "no ejection (no droplets)." For example, a dither method or an error diffusion method is used for this type of halftone processing. The image processor 311 also performs correction processing based on the read image obtained from the scanner 48.

[0092] The transport control unit 312 controls the operation of the transport mechanism 11. The transport mechanism 11 includes the elements of the mechanism involved in transporting the paper P from the paper feed unit 10 to the stacking unit 60 described in FIG. 1. The transport mechanism 11 includes the paper feed drum 16, the treatment liquid application drum 22, the treatment liquid drying drum 32, the imaging drum 42, and the chain gripper 70 shown in FIG. 1. The transport mechanism 11 also includes a drive unit such as a motor (not shown) as a power source and a motor drive circuit (not shown). The transport control unit 312 controls the transport speed of the paper P by the transport mechanism 11 in response to a command from the system control unit 310, and also controls the paper P to be transported from the paper feed unit 10 to the stacking unit 60.

[0093] The paper feed control unit 313 operates the paper feed unit 10 in response to a command from the system control unit 310. The paper feed control unit 313 controls the operation of starting and stopping the supply of paper P.

[0094] The treatment liquid application control unit 314 operates the treatment liquid application unit 20 in response to a command from the system control unit 310. The treatment liquid application control unit 314 controls the application operation of the treatment liquid application device 24, such as turning on / off the pre-coat function, the amount of treatment liquid applied, and the application timing.

[0095] The processing liquid drying control unit 316 operates the processing liquid drying unit 30 in response to a command from the system control unit 310. The processing liquid drying control unit 316 controls the drying temperature, the flow rate of the drying gas, and the ejection timing of the drying gas.

[0096] The drawing control unit 318 operates the drawing unit 40 in response to commands from the system control unit 310. The drawing control unit 318 includes a waveform storage unit, a waveform generation unit, and a drive circuit, all of which are not shown. The waveform storage unit stores the waveforms of the drive voltages to be applied to the ejection energy generating elements of the inkjet heads 46C, 46M, 46Y, and 46K. The waveform generation unit generates the waveforms of the drive voltages. The drive circuit generates drive voltages having drive waveforms corresponding to the dot data.

[0097] The drawing control unit 318 controls the ejection operation of each of the inkjet heads 46C, 46M, 46Y, and 46K based on the dot data of each ink color generated through halftone processing by the image processing unit 311 so as to record an image on the paper P transported by the drawing drum 42. That is, based on the dot data generated through processing by the image processing unit 311, the ejection timing and ink ejection amount for each pixel position are determined, a drive voltage according to the ejection timing for each pixel position and the ink ejection amount, and a control signal for determining the ejection timing for each pixel is generated, the drive voltage is supplied to the inkjet heads 46C, 46M, 46Y, and 46K, and dots are recorded on the paper P by the ink ejected from the inkjet heads 46C, 46M, 46Y, and 46K.

[0098] The ink drying control unit 320 operates the ink drying unit 50 in response to a command from the system control unit 310. The ink drying control unit 320 controls the temperature of the drying gas, the flow rate of the drying gas, the ejection timing of the drying gas, and the like.

[0099] The sorting control unit 322 controls the stamping process by the stamper of the sorting unit 55 according to the inspection result of the inspection processing unit 319. The sorting control unit 322 performs the stamping process on the paper P determined to be a defective printed matter, and does not perform the stamping process on the paper P determined to be a good (normal) printed matter.

[0100] The paper discharge control unit 324 operates the stacking unit 60 in response to a command from the system control unit 310. In the case where the stacking device 62 shown in Fig. 1 includes a lifting mechanism, the paper discharge control unit 324 controls the operation of the lifting mechanism in response to an increase or decrease in the number of sheets P. The sheets P, which are printed matter, are stacked in the stacking tray 62A. The sheets P of defective printed matter have ink adhering to their leading edge. This allows the user, a worker (operator), to identify defective printed matter from among the sheets P stacked in the stacking tray 62A.

[0101] Information input via the input device 122 is sent to the system control unit 310. The system control unit 310 executes various processes in accordance with the information input from the input device 122.

[0102] The display device 124 may display various information such as various setting information of the device or abnormality information in response to a command from the system control unit 310. The operator can set various parameters and input and edit various information using the input device 122 while watching the contents displayed on the display device 124. In addition, the inspection result by the inspection processing unit 319 may be displayed on the display device 124. The display device 124 functions as a means for notifying the inspection result. The user can set the utilization form of the inspection result, such as whether or not to display the inspection result on the display device 124, or whether or not to control the stamp processing or the paper discharge destination based on the inspection result, via an appropriate user interface. The control device 100 including the inspection processing unit 319 functions as a print inspection device.

[0103] <Outline of printed matter inspection method> In the present embodiment, the inspection of a printed matter is performed by acquiring inspection data by reading the printed matter to be inspected with the scanner 48, and comparing the inspection data with reference data that serves as the basis for the inspection. The reference data used in this embodiment is digital image data acquired based on the print data without performing a printing operation. In other words, the reference data is the digital image data itself of the image to be printed indicated by the print data input to the printing system 101, or digital image data generated by performing appropriate processing on this digital image data.

[0104] In this specification, digital image data obtained based on print data without performing a printing operation is referred to as an "original image." The original image may be, for example, a raster image (bitmap image) that is separated into each color plate of CMYK by a RIP (Raster Image Processor) from input data in a PDF (Portable Document Format) format. A CMYK raster image may be input as print data, in which case the print data configured with color information of the inks used for printing corresponds to the original image.

[0105] The printed matter to be inspected is a printed matter printed by the inkjet printer 1 based on the print data. In this specification, the image to be inspected obtained by reading the printed matter to be inspected by the scanner 48 is referred to as a "scanned image." A scanned image is synonymous with a read image, and is an example of a "captured image" in this disclosure.

[0106] FIG. 4 is a flowchart showing an overview of an inspection process for comparing an original image BI of a print with a scanned image SI obtained by scanning a printed image. When inspecting a printed matter, the original image BI and the scanned image SI cannot be directly compared as they are. The original image BI for printing is often composed of CMYK color information, whereas general optical sensors such as CCD image sensors and CMOS image sensors mounted on imaging devices such as the scanner 48 are often configured to output RGB color information. In other words, the color information of the image data of the original image BI and the scanned image SI often differ from each other. However, the image sensor that generates the scanned image SI is not limited to a primary color sensor, and a complementary color sensor may be used. CMYK is an example of a color space that represents the data of the original image BI, and RGB is an example of a color space that represents the data of the scanned image SI.

[0107] Also, while the original image BI is configured with a high resolution such as 1200 dpi x 1200 dpi, the scanned image SI often has a different resolution from the original image BI because a high resolution makes the scanner 48 expensive and the resolution in the transport direction must be lowered when the printing speed is high. For example, the scanned image SI is configured with a low resolution such as 600 dpi x 100 dpi. Furthermore, the scanned image SI is subject to geometric noise, optical noise, signal noise, and the like, which are caused by the installation state of the scanner 48.

[0108] Therefore, before the comparison process between the original image BI and the scanned image SI is performed, pre-processing is performed to convert the respective images into data in a format that allows them to be compared with each other.

[0109] That is, the processor 102 performs preprocessing PP1 for original images on the original images BI to generate comparison reference data RD, and also performs preprocessing PP2 for scanned images on the scanned images SI to generate comparison data CD.

[0110] Thereafter, the processor 102 performs a comparison process CP for detecting defects in the printed matter based on the comparison reference data RD and the comparison data CD. The comparison process CP provides defect inspection information including at least one of information on the presence or absence of a defect, the strength of the defect, and the position of the defect on the printed matter.

[0111] The processor 102 outputs the inspection result IR based on the defect inspection information obtained by the comparison process CP. The inspection result IR may include at least one of information indicating the presence or absence of a defect, defect intensity information, and defect position information. Such inspection processing needs to be performed in a very short time from when the scanned image SI is generated until when the scanned image of the next printed matter to be inspected is generated. To improve the accuracy of the inspection, complex processing must be performed so that defects can be extracted in more detail, but performing complex processing takes time. Furthermore, multiple types of defects can occur in printed matter due to various factors.

[0112] The scanned image SI shown in FIG. 4 is an example of an image obtained by capturing an image of a printed matter including a streak 401 and a stain 402 as examples of typical defects that may occur in a printed matter. The streak 401 may be caused by a nozzle discharge defect in an inkjet head. The stain 402 may be caused by ink dripping, adhesion of foreign matter, or the like. When attempting to accurately inspect each of a plurality of types of defects, not limited to the streak 401 and the stain 402, the processing becomes more complicated. In addition, a method using a learning model such as deep learning for the inspection process requires more processing time than an image processing method that does not use a learning model, making it even more difficult to achieve both improved processing speed and improved accuracy.

[0113] Therefore, in the print inspection device according to the present embodiment, in order to detect a plurality of types of defects, the detection method is changed depending on the type of defect to be detected, and a processing method suitable for detecting each defect is applied to perform pre-processing on the original image BI in advance. The processing on the original image BI here is, for example, image conversion processing such as image resolution conversion, CMYK to RGB conversion, and processing for adding noise similar to the noise of the scanned image SI, or feature extraction in machine learning, or a combination of these. By performing processing on the original image BI in advance and storing the processing results before acquiring the scanned image SI, the number of steps required for dynamic processing can be reduced, and the processing speed can be improved.

[0114] That is, in this embodiment, before the printing operation is started, pre-processing PP1 is performed on the original image BI, whose information has already been obtained, before acquiring the scanned image SI, thereby reducing processing time. In addition, pre-processing PP1 also enables information compression, and by storing the information-compressed comparison reference data RD instead of storing the original image BI itself, the memory capacity required for storage is reduced. If the information compressed by pre-processing PP1 (comparison reference data RD) is saved, the print inspection device can discard the data of the original image BI, which also has the effect of reducing memory.

[0115] In this specification, the pre-processing PP1 performed in advance on the original image BI is called "pre-processing." The contents of the pre-processing differ depending on the degree of difference between the original image BI and the scanned image SI, the type of defect to be detected, the inspection method, etc. In addition, the contents of the pre-processing PP2 performed on the scanned image SI also differ depending on the type of defect to be detected, the inspection method, etc.

[0116] The flowchart shown in Fig. 4 is a process flow for detecting defects belonging to a specific type among multiple types of defects that may occur in a printed matter. In the print inspection device according to this embodiment, in order to detect multiple types of defects, the process flow shown in Fig. 4 is carried out by applying different processing contents for each type of defect to be detected (see Fig. 5).

[0117] FIG. 5 shows an example of a flowchart of an inspection process performed for each type of defect when there are two types of defects to be detected. The left diagram F5A in FIG. 5 is a flowchart of an inspection process for detecting a first defect, and the right diagram F5B in FIG. 5 is a flowchart of an inspection process for detecting a second defect. The first defect is, for example, a streak-like defect (hereinafter referred to as a "streak defect"), and the second defect is a defect other than a streak defect (hereinafter referred to as a "non-streak defect"). The second defect includes stain defects such as ink drips. Note that FIG. 5 describes an example of detection of two types of defects, but the same applies when detecting three or more types of defects, and a processing flow for each defect type corresponding to the type of defect to be detected is added. Each of a streak defect and a non-streak defect is an example of a "defect type" in this disclosure.

[0118] As shown in FIG. 5, the processor 102 performs different pre-processing PP11 and pre-processing PP12 on the original image BI depending on the type of target defect, and holds different comparison reference data RD1 and RD2 depending on the type of target defect.

[0119] That is, when the processor 102 acquires the original image BI, the processor 102 performs preprocessing PP11 on the original image BI as a first preprocessing for detecting a first defect, and generates comparison reference data RD1. The generated comparison reference data RD1 is stored in the memory. The processor 102 also performs preprocessing PP12 on the original image BI as a second preprocessing for detecting a second defect, and generates comparison reference data RD2. The generated comparison reference data RD2 is stored in the memory.

[0120] Then, the processor 102 acquires the scan image SI, performs preprocessing PP21 required for inspecting the first defect on the scan image SI, and generates comparison data CD1. The processor 102 performs a comparison process CP1 using the generated comparison data CD1 and comparison reference data RD1 previously stored in memory, and outputs a detection result IR1 of the first defect.

[0121] Similarly, the processor 102 performs pre-processing PP22 required for inspecting the second defect on the scanned image SI to generate comparison data CD2. Note that there may be a configuration in which the scanned image SI is used as the comparison data CD2 without performing the pre-processing PP22.

[0122] The processor 102 performs a comparison process CP2 using the generated comparison data CD2 and comparison reference data RD2 previously stored in memory, and outputs a detection result IR2 of a second defect.

[0123] Regarding pre-processing performed on the original image BI, for example, if the defect to be detected is a thin and minute one such as a streak, a method that uses deep learning to extract fine features is effective. However, if the defect is relatively large, such as a stain caused by an ink drip, it is faster to extract the defect by performing differential processing on corresponding pixel units without using deep learning, but by using image conversion such as image resolution conversion or blurring to bring the data structures of the original image BI and scanned image SI closer together.

[0124] Furthermore, when deep learning is used in the comparison process CP1 as an inspection method, feature extraction is performed as preprocessing PP11 and preprocessing PP21 in order to compare the feature amounts of the original image BI and the scanned image SI. The process of extracting features from an image may be performed using machine learning, or a feature extraction algorithm other than machine learning may be designed and performed. When feature extraction is performed using machine learning, it may be performed using deep learning.

[0125] In the case of deep learning, extracting features takes time, so performing pre-processing 1 on the original image BI can have a significant effect in reducing processing time. In particular, by compressing the image features into a one-dimensional vector, the amount of data can be significantly reduced compared to a two-dimensional x four-channel (CMYK) original image BI, and by storing the features as comparison reference data RD1, the memory reduction effect is significant.

[0126] On the other hand, as another inspection method, when performing a comparison on a pixel-by-pixel basis, such as a differential process between comparison reference data RD2 generated from the original image BI and comparison data CD2 obtained based on the scanned image SI, pre-processing 2 (pre-processing PP12) is performed on the original image BI, for example, image processing such as resolution conversion, CMYK to RGB conversion, and edge processing (see FIG. 6).

[0127] Furthermore, to achieve higher speeds, it is preferable to perform processing to bring the original image BI closer to the scanned image SI, so that the scanned image SI can be compared as is. This can be achieved by performing processing to add noise contained in the scanned image SI to the original image BI (see Figure 7). The noise contained in the scanned image SI includes geometric noise due to installation position and part accuracy, optical noise, and signal noise due to optical sensors, and the noise to be added can be any one of these multiple noises, or all of them.

[0128] Furthermore, when defects are extracted by pixel-by-pixel difference processing between image data as the comparison processing CP2, it is necessary to eliminate image features other than defects. For this reason, it is desirable to hold an edge image that has been subjected to blurring processing to eliminate edge features in the original image BI and edge extraction processing to identify common edge positions between the scanned image SI and the original image BI.

[0129] As for the processing that is performed in advance on the original image BI to bring the original image BI closer to the data structure of the scanned image SI, at least one of various processing such as resolution conversion processing, color conversion processing, edge extraction processing, blurring processing, and noise addition processing may be performed.

[0130] 6 is a flowchart showing an example of an inspection process when comparing an original image BI and a scanned image SI on a pixel-by-pixel basis. The original image BI has a resolution of 1200 dpi×1200 dpi, for example, and color information is CMYK. The original image BI obtained based on the print data is a digital image that is not derived from a captured image of a printed matter, so the edges of the pattern are clear and do not contain noise.

[0131] In contrast, the scanned image SI has a resolution of, for example, 600 dpi × 100 dpi and color information of RGB. The scanned image SI is derived from a captured image of a printed matter, and has less clear edges than the original image BI, and contains noise such as geometric noise and optical noise.

[0132] In this way, the original image BI and the scanned image SI, which have significantly different data structures, cannot be compared pixel by pixel in this state. For this reason, the processor 102 performs preprocessing PP12 on the original image BI, converting it into comparison reference data RD2 that can be compared pixel by pixel with the scanned image SI.

[0133] The pre-processing PP12 shown in FIG. 6 includes the processes of steps S11 to S18 shown below.

[0134] In step S11, the processor 102 performs a color conversion process P11. The color conversion process P11 here may be a CMYK to RGB conversion process that converts CMYK color information into RGB color information.

[0135] In step S12, the processor 102 performs a resolution conversion process. The resolution conversion process may be a process of converting data of 1200 dpi×1200 dpi into data of 600 dpi×100 dpi, which is the same as that of the scanned image SI.

[0136] In step S13, the processor 102 performs edge extraction processing. The edge extraction processing may be processing for extracting edges from a converted image that has been subjected to color conversion processing (step S11) and resolution conversion processing (step S12).

[0137] The processor 102 stores the edge image obtained by the edge extraction process (step S13) in the memory (step S14). This edge image is an example of "edge extraction data" in this disclosure.

[0138] In step S15, the processor 102 performs blurring on the converted image that has been subjected to the color conversion process (step S11) and the resolution conversion process (step S12). The processor 102 stores the blurred image obtained by the blurring process (step S15) in the memory (step S16). This blurred image is an example of the "blurred data" in this disclosure.

[0139] In step S17, the processor 102 performs a noise addition process. The noise addition process may be a process of adding pseudo noise similar to that of the scanned image SI to the converted image that has been subjected to the color conversion process (step S11), the resolution conversion process (step S12), and the blurring process (step S15). The processor 102 stores the noise-added image obtained by the noise addition process (step S17) in the memory (step S18).

[0140] Note that the noise-added image obtained by the noise addition process (step S17) may be held as comparison reference data RD2.

[0141] By performing pre-processing (pre-processing PP12) on the original image BI before acquiring the scanned image SI, the processing time after acquiring the scanned image SI can be reduced.

[0142] When the processor 102 acquires the scanned image SI, the processor 102 performs edge processing on the scanned image SI to erase edges derived from the pattern of the original image BI using an edge image extracted from the original image BI. The processor 102 also performs blurring processing on the scanned image SI. In this manner, the processor 102 generates comparison data from the scanned image SI.

[0143] Then, in step S20, the processor 102 performs a comparison process in which the stored comparison reference data RD2 is compared with the comparison data generated from the scan image SI on a pixel-by-pixel basis. The comparison process may be, for example, a difference process in which the difference between the two is calculated on a pixel-by-pixel basis to generate a difference image.

[0144] When a scanned image SI of a defective printed matter is obtained, the defect is detected by a comparison process in step S20, and defect information is obtained.

[0145] 7 is an explanatory diagram showing an example of noise addition processing for an original image BI. The scanned image SI is slightly tilted due to the installation state of the scanner 48. In addition, the scanned image SI has uneven resolution due to the optical characteristics of the scanner 48. In order to make the data structure of the scanned image SI, which contains such geometric noise and optical noise, closer to the data structure of the original image BI, the processor 102 performs pre-processing on the original image BI, including a color conversion process P11, a resolution conversion process P12, and a noise addition process P13. The color conversion process P11 and the resolution conversion process P12 are the processes described as steps S11 and S12 in FIG. 6.

[0146] The noise addition process P13 includes a rotation process that imparts the same inclination as the inclination of the scanned image SI to the converted image CI obtained by performing the color conversion process P11 and the resolution conversion process P12 on the original image BI, and an image processing process that imparts the same resolution unevenness as the scanned image SI to the converted image CI.

[0147] The data structure of the noise-added image NAI obtained by performing the noise addition process P13 on the converted image CI is close to the data structure of the scanned image SI. The noise-added image NA is an example of the "converted data" in this disclosure.

[0148] Example of processing according to the type of defect to be detected FIG. 8 is an explanatory diagram showing an example of processing for each type of defect to be detected. As described above, in addition to streak defects, multiple types of defects such as dot-shaped defects caused by stains or the like can occur in printed matter. For example, if the defect to be detected by inspecting a printed matter is a streak defect caused by a nozzle discharge failure, the streak continues long in the conveyance direction of the paper P during printing, so that even if the information of the scanned image SI is compressed in the conveyance direction, the streak feature 401A, which is the feature amount of the streak 401, does not disappear in the compressed image CSI. On the other hand, if the information of the scanned image SI is compressed in the conveyance direction, the feature amount of the stain 402 in the compressed image CSI decreases, making it difficult to detect non-streak defects such as stains from the compressed image CSI.

[0149] If the defect to be detected is, for example, a relatively large dot-like stain 402 caused by ink dripping or the like, stain feature 402A, which is the feature amount of stain 402, does not disappear in the low-resolution image LRI even if down-conversion processing (resolution reduction processing) is performed on the scanned image SI. On the other hand, when the resolution of the scanned image SI is reduced, the feature amount of streak 401 disappears in the low-resolution image LRI, making it difficult to detect the streak defect from the low-resolution image LRI.

[0150] Therefore, when detecting streak defects, it is preferable to perform a process to compress information in the transport direction of the scanned image SI, and when detecting non-streak defects such as stains, it is preferable to perform a process to reduce the resolution of the scanned image SI.

[0151] In this way, by applying different processes depending on the type of defect to be detected, it is possible to reduce the data storage capacity while ensuring the detection accuracy. Note that the defect types may be defined from the viewpoint of whether the same processing method or different processing methods are applied as the defect detection processing method.

[0152] <Functional configuration of the print inspection device> 9 is a block diagram showing a functional configuration of a printed matter inspection device 500 according to an embodiment. Here, an example of a process for detecting two types of defects, namely, a streak defect and other defects (hereinafter, referred to as non-streak defects), is shown as an example of multiple types of defects to be detected.

[0153] The print inspection device 500 includes an original image acquisition unit 502, a first pre-processing unit 504, a second pre-processing unit 506, a storage unit 508, a scan image acquisition unit 510, a first defect-specific processing unit 514, a second defect-specific processing unit 516, a defect inspection unit 520, and an inspection result output unit 530. The functions of the print inspection device 500 are realized by a combination of hardware and software of a computer including one or more processors and one or more storage devices. The functions of the print inspection device 500 may be incorporated into the control device 100. In addition, the print inspection device 500 may be configured as an auxiliary device connected to the control device 100.

[0154] The original image acquisition unit 502 acquires the original image BI. The first pre-processing unit 504 performs a first pre-processing on the original image BI to generate first comparison reference data 505 from the original image BI. The first pre-processing is a pre-processing for detecting a streak defect as a first defect to be detected. The first pre-processing corresponds to the pre-processing PP11 described in FIG. 5, and the first comparison reference data 505 corresponds to the comparison reference data RD1 described in FIG. 5. The first pre-processing unit 504 is configured using, for example, a learned (trained) deep learning model trained to extract the feature amount of the original image BI. The deep learning model may be, for example, a convolutional neural network (CNN). In this case, the first comparison reference data 505 may be data of the feature amount output from the deep learning model. The feature amount extraction processing performed by the first pre-processing unit 504 is an example of the "first feature amount extraction processing" in the present disclosure.

[0155] The first pre-processing unit 504 is not limited to a process using deep learning, and may be configured to perform a process of extracting feature amounts using an algorithm other than deep learning. The first pre-processing unit 504 is preferably configured to perform image processing including a compression process of image information suitable for detecting streak defects (a process of compressing an image in the conveying direction of the paper P) as described in FIG. The first comparison reference data 505 generated by the first pre-processing unit 504 is stored in the storage unit 508.

[0156] Similarly, the second pre-processing unit 506 performs a second pre-processing different from the first pre-processing on the original image BI to generate second comparison reference data 507 from the original image BI. The second pre-processing is a pre-processing for detecting a non-streak defect as a second defect to be detected. The second pre-processing corresponds to the pre-processing PP12 described in FIG. 5, and the second comparison reference data 507 corresponds to the comparison reference data RD2 described in FIG. 5. The second comparison reference data 507 generated by the second pre-processing unit 506 is stored in the storage unit 508. The storage unit 508 may be a storage area of ​​the computer-readable medium 104.

[0157] The scanned image acquisition unit 510 acquires the scanned image SI read by the scanner 48. The first defect-specific processing unit 514 performs a first defect-specific processing on the scanned image SI and generates first comparison data 515 from the scanned image SI. The first defect-specific processing corresponds to the preprocessing PP21 described in FIG. 5, and the first comparison data 515 corresponds to the comparison data CD1 described in FIG. 5.

[0158] The first defect-specific processing unit 514 is configured, for example, using a trained deep learning model that has been trained to extract features of the scan image SI. The deep learning model may be, for example, a CNN. In this case, the first comparison data 515 may be feature data output from the deep learning model. The feature extraction process performed by the first defect-specific processing unit 514 is an example of a "second feature extraction process" in the present disclosure.

[0159] The first defect-specific processing unit 514 is not limited to processing using deep learning, and may be configured to perform processing to extract feature amounts using an algorithm other than deep learning. The first defect-specific processing unit 514 is preferably configured to perform image conversion processing including compression processing of image information suitable for detecting streak defects as described in FIG. 8.

[0160] Second defect-specific processing unit 516 performs second defect-specific processing on scanned image SI to generate second comparison data 517 from scanned image SI. The second defect-specific processing corresponds to preprocessing PP22 described in Fig. 5, and second comparison data 517 corresponds to comparison data CD2 described in Fig. 5. Second defect-specific processing unit 516 is preferably configured to perform image conversion processing including resolution reduction processing suitable for detecting defects such as stains described in Fig. 8.

[0161] The defect inspection section 520 includes a first defect inspection section 521 and a second defect inspection section 522 corresponding to the two types of target defects, respectively. The first defect inspection section 521 performs a process for detecting a first defect (here, a streak defect) using the first comparison data 515 and the first comparison reference data 505 held in the storage section 508. That is, the first defect inspection section 521 performs the comparison process CP1 described in FIG. 5. The first defect inspection section 521 may be configured to determine the presence or absence of a defect, estimate the strength or position of a defect, for example, using a learning model using machine learning such as deep learning, or may be configured to calculate the difference between the first comparison data 515 and the first comparison reference data 505. A first inspection result 531 regarding the first defect is obtained by the process in the first defect inspection section 521.

[0162] The second defect inspection unit 522 performs a process for detecting a second defect (here, a defect other than a streak defect) using the second comparison data 517 and the second comparison reference data 507 stored in the storage unit 508. That is, the second defect inspection unit 522 performs the comparison process CP2 described in FIG. 5. The second defect inspection unit 522 may be configured to determine the presence or absence of a defect or estimate the strength or position of a defect using a learning model using machine learning such as deep learning, or may be configured to calculate the difference between the second comparison data 517 and the second comparison reference data 507. A second inspection result 532 regarding the second defect is obtained by the process in the second defect inspection unit 522.

[0163] The inspection result output section 530 performs a process of outputting the inspection results including the first inspection result 531 and the second inspection result 532 obtained by the processing in the defect inspection section 520. For example, when at least one of the first defect and the second defect is detected by the defect inspection section 520, the inspection result output section 530 performs a process of displaying the inspection result on the display device 124. Note that when no defect is detected, the display of the inspection result may be omitted.

[0164] Furthermore, the inspection result output unit 530 may be configured to output a defect detection signal to the sorting control unit 322 when at least one of the first defect and the second defect is detected by the defect inspection unit 520. The sorting control unit 322 controls the stamp processing and the paper transport path in response to the defect detection signal.

[0165] The pre-processing performed by each of the first pre-processing unit 504 and the second pre-processing unit 506 is an example of "plurality of pre-processing" in the present disclosure. The first comparison reference data 505 and the second comparison reference data 507 are an example of "plurality of comparison reference data" in the present disclosure. The first comparison data 515 and the second comparison data 517 are an example of "plurality of comparison data" in the present disclosure.

[0166] Overview of inspection processing using deep learning 10 is an explanatory diagram showing an example of an inspection process using deep learning. The print inspection device 500 may include, for example, a first neural network 610 that extracts features of an original image BI, a second neural network 620 that extracts features of a scanned image SI, and a third neural network 630 that compares the original image features BF output from the first neural network 610 with the scanned image features SF output from the second neural network 620.

[0167] The first neural network 610 may be, for example, a trained model that is composed of a CNN and is trained to receive an input of an original image BI, extract a feature quantity indicating an image feature of the original image BI, and output an original image feature quantity BF. The first neural network 610 may also be a trained model that is trained to receive an input of a converted image obtained by performing an image conversion process including resolution conversion, CMYK to RGB conversion, and conveyance direction compression on the original image BI, extract a feature quantity from the converted image, and output an original image feature quantity BF. The original image feature quantity BF output from the first neural network 610 is preferably in a one-dimensional vector data format.

[0168] The second neural network 620 may be, for example, a trained model that is configured with a CNN and is trained to receive an input of a scanned image SI, extract features from the scanned image SI, and output scanned image features SF. The second neural network 620 may also be a trained model that is trained to receive an input of a converted image obtained by performing an image conversion process including conveyance direction compression on the scanned image SI, extract features from the converted image, and output scanned image features SF.

[0169] The scanned image feature SF output from the second neural network 620 is preferably in a one-dimensional vector data format, similar to the original image feature BF.

[0170] The third neural network 630 may be a learned model trained to receive the original image feature BF output from the first neural network 610 and the scanned image feature SF output from the second neural network 620 as input, and output an inspection result IR for defects. The inspection result IR includes at least one piece of information on the presence or absence of a defect, the intensity of the defect, and the position of the defect. The information indicating the presence or absence of a defect may be, for example, a binary discrete value indicating the presence or absence of a defect. The binary discrete value may be defined as "0" when there is no defect (when no defect exists) and "1" when there is a defect.

[0171] Furthermore, the information indicating the defect intensity may be, for example, a continuous value indicating the defect intensity. The defect intensity indicates the visibility of the defect, and can be defined as a continuous value from 0 to 10, for example, with the absence of a defect being represented as "0" and the greater the defect intensity, i.e., the easier the defect is to be visible, the greater the numerical value. In this case, the third neural network 630 performs regression to predict the numerical value indicating the defect intensity.

[0172] Alternatively, the information indicating the defect intensity may be defined as three or more discrete values ​​that classify the defect intensity. For example, the defect intensity may be defined as three discrete categorical values ​​of "0", "1", and "2", with "0" indicating no defect, "1" indicating a weak defect, and "2" indicating a strong defect. In this case, the third neural network 630 performs three-class classification.

[0173] The information indicating the position of the defect may be defined, for example, as data that quantifies the position coordinates on the image, or may be defined as image information (binary image) in which non-defective positions are "0" and defective positions are "1."

[0174] The third neural network 630 may be configured to output a plurality of pieces of information including two of the following: the presence or absence of a defect, the strength of the defect, and the location of the defect.

[0175] The network structure of the third neural network 630 may take various forms depending on the information output by the third neural network 630.

[0176] The first neural network 610 is applied to the first pre-processing unit 504 in Fig. 9. The original image feature amount BF corresponds to the first comparison reference data 505.

[0177] The second neural network 620 is applied to the first defect-specific processing unit 514 in Fig. 9. The scan image feature value SF corresponds to the first comparison data 515.

[0178] The third neural network 630 is applied to the first defect inspection unit 521 in Fig. 9. The inspection result IR corresponds to the first inspection result 531.

[0179] [Learning method] In the learning phase, the entire network in which the first neural network 610, the second neural network 620, and the third neural network 630 shown in FIG. 10 are combined is treated as a single model, and the parameters of each of the first neural network 610, the second neural network 620, and the third neural network 630 are optimized so that correct inspection results are output.

[0180] The learning (training) data set includes a plurality of data sets in which a plurality of original learning images, a plurality of learning scan images corresponding to the plurality of original learning images, and correct defect information for each of the plurality of learning scan images are linked together. Note that it is preferable that a plurality of learning scan images are prepared for one original learning image.

[0181] During training, original training images of a data set sampled from a training data set are input to a first neural network 610, and feature quantities of the original training images are output from the first neural network 610. A scanned training image is input to a second neural network 620, and feature quantities of the scanned training image are output from the second neural network 620.

[0182] The third neural network 630 receives two feature amounts, the feature amount output from the first neural network 610 and the feature amount output from the second neural network 620, and outputs defect information estimated from a combination of these feature amounts. The estimated defect information output from the third neural network 630 may be an estimated value of defect information including at least one piece of information on the presence or absence of a defect, the strength of the defect, and the position of the defect.

[0183] A machine learning system (not shown) that executes the learning process updates the parameters of the first neural network 610, the second neural network 620, and the third neural network 630 so that the estimated defect information output from the third neural network 630 approaches the correct defect information. That is, the machine learning system uses an error backpropagation method to calculate the amount of update for the parameters of each neural network so that the error between the estimated defect information and the correct defect information is reduced, and updates the parameters.

[0184] The machine learning system may perform mini-batch learning, in which parameters are updated in mini-batch units, or may perform online learning, in which parameters are updated for each piece of learning data.

[0185] After training, the first neural network 610, the second neural network 620, and the third neural network 630 can be used separately as trained models. The trained first neural network 610 can be used as a feature extractor that extracts features of the original image BI. The trained second neural network 620 can be used as a feature extractor that extracts features of the scanned image SI. The trained third neural network 630 can be used as a comparison processor that compares features to detect defects.

[0186] The first neural network 610 is an example of a "first deep learning model" in the present disclosure. The second neural network 620 is an example of a "second deep learning model" in the present disclosure. The third neural network 630 is an example of a "third deep learning model" in the present disclosure. Although deep learning has been described as an example in FIG. 10, a machine learning model other than deep learning, such as a random forest or a support vector machine, may be applied instead of the deep learning model.

[0187] <<Operation of the printing system 101 including the print inspection device 500>> 11 and 12 are flowcharts showing an example of the operation of the printing system 101 including the print inspection device 500.

[0188] In step S110, the processor 102 obtains an original image BI.

[0189] In step S111, the processor 102 performs a first pre-processing on the original image BI to generate first comparison reference data 505. The first pre-processing may include, for example, the resolution conversion processing and CMYK to RGB conversion processing described in Fig. 6, the conveying direction compression processing described in Fig. 8, and a feature extraction processing for extracting features from the compressed image of the original image BI by inputting the compressed image obtained by the compression processing to a deep learning model such as a first neural network 610 (see Fig. 10).

[0190] In step S 112 , the processor 102 stores the first comparison reference data 505 in the storage unit 508 .

[0191] In step S113, the processor 102 performs a second pre-processing on the original image BI to generate second comparison reference data 507. The second pre-processing may be processing including, for example, the resolution conversion processing, CMYK to RGB conversion processing, edge processing, blurring processing, and noise addition processing described in Fig. 6, and the resolution reduction processing described in Fig. 8.

[0192] In step S 114 , the processor 102 stores the second comparison reference data 507 in the storage unit 508 .

[0193] The processes from step S110 to step S114 can be performed before starting the printing operation by acquiring the print data. After step S114, the processor 102 proceeds to step S120 in FIG.

[0194] In step S120, the processor 102 controls the inkjet printing machine 1 to execute printing based on the original image BI.

[0195] In step S121, the processor 102 controls the scanner 48 to capture an image of a printed matter.

[0196] In step S122, the processor 102 obtains a scanned image SI obtained by imaging the printed matter with the scanner 48.

[0197] In step S123, the processor 102 performs a first defect-specific process on the scan image SI to generate first comparison data 515. The first defect-specific process may include, for example, the compression process in the transport direction described in Fig. 9 and a feature extraction process that extracts features of the compressed image by inputting the compressed image CSI obtained by the compression process to a deep learning model such as a second neural network 620 (see Fig. 10).

[0198] In step S124, the processor 102 performs a first defect inspection process using the first comparison data 515 and the first comparison reference data 505 stored in the storage unit 508, and obtains an inspection result (first inspection result 531) regarding the first defect. The first defect inspection process may be configured to obtain defect inspection information using, for example, a machine learning model such as a third neural network 630 (see FIG. 10) or a support vector machine.

[0199] In step S125, the processor 102 performs a second defect-specific process on the scan image SI to generate second comparison data 517. The second defect-specific process may be, for example, an image conversion process including the resolution reduction process described with reference to FIG.

[0200] In step S126, processor 102 performs a second defect inspection process using second comparison data 517 and second comparison reference data 507 stored in storage unit 508, and obtains an inspection result (second inspection result 532) regarding the second defect. The second defect inspection process may be configured to include, for example, a difference process in which second comparison reference data 507 and second comparison data 517 are compared on a pixel-by-pixel basis.

[0201] In step S127, processor 102 performs processing to output the inspection results based on first inspection result 531 obtained by the first defect inspection processing (step S124) and second inspection result 532 obtained by the second defect inspection processing (step S126). The processing to output the inspection results includes, for example, processing to display information on the inspection results on display device 124. The processing to output the inspection results may also include outputting a control signal for stamp processing and / or outputting a control signal for switching the paper transport path to change the accumulation destination (paper discharge destination) of the printed matter.

[0202] In step S128, the processor 102 determines whether or not to end printing of the original image BI. If printing of the number of copies specified in the print job is not complete, the determination result in step S128 is No. On the other hand, if printing of the number of copies specified in the print job is complete, or if, for example, an instruction to stop printing is input from the user interface, or if a sensor provided in the printing system 101 detects an abnormal state that makes it impossible to continue printing and the printing operation is stopped, the determination result in step S128 is Yes.

[0203] If the determination result in step S128 is a No determination, the processor 102 returns to step S120 and continues printing. This causes the operations from step S120 to step S128 to be repeated, and an inspection is performed immediately every time a printed matter is printed.

[0204] If the determination result in step S128 is Yes, the processor 102 ends printing, and ends the flow charts of FIGS.

[0205] <<Hardware configuration of each processing unit and control unit>> The hardware structure of the processing units that execute various processes, such as the system control unit 310, image processing unit 311, transport control unit 312, paper feed control unit 313, treatment liquid application control unit 314, treatment liquid drying control unit 316, drawing control unit 318, inspection processing unit 319, ink drying control unit 320, sorting control unit 322, and paper discharge control unit 324 described in FIG. 3, and the original image acquisition unit 502, first pre-processing unit 504, second pre-processing unit 506, scanned image acquisition unit 510, first defect-specific processing unit 514, second defect-specific processing unit 516, defect inspection unit 520, first defect inspection unit 521, second defect inspection unit 522, and inspection result output unit 530 described in FIG. 9, is various processors as shown below.

[0206] Various types of processors include CPUs, which are general-purpose processors that execute programs and function as various processing units, programmable logic devices (PLDs), which are processors whose circuit configuration can be changed after manufacture, such as GPUs and FPGAs, and dedicated electrical circuits, such as ASICs (Application Specific Integrated Circuits), which are processors with a circuit configuration designed specifically to execute specific processes.

[0207] A processing unit may be composed of one of these various processors, or may be composed of two or more processors of the same type or different types. For example, a processing unit may be composed of multiple FPGAs, or a combination of a CPU and an FPGA, or a combination of a CPU and a GPU. Also, multiple processing units may be composed of one processor. As an example of multiple processing units being composed of one processor, first, as represented by a computer such as a client or a server, there is a form in which one processor is composed of a combination of one or more CPUs and software, and this processor functions as multiple processing units. Second, as represented by a system on chip (SoC), there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units in one IC (Integrated Circuit) chip. In this way, the various processing units are composed of one or more of the above various processors as a hardware structure.

[0208] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.

[0209] About the programs that run computers A program that causes a computer to realize some or all of the processing functions of the printed matter inspection device 500 and the machine learning system described in the above-mentioned embodiments can be recorded on a computer-readable medium such as an optical disk, a magnetic disk, a semiconductor memory, or other tangible, non-transitory information storage medium, and the program can be provided through this information storage medium.

[0210] In addition, instead of providing the program by storing it on such a tangible, non-transitory computer-readable medium, it is also possible to provide the program signal as a download service using a telecommunications line such as the Internet.

[0211] Furthermore, some or all of the processing functions of the above-described print inspection device 500 and the machine learning system may be realized by cloud computing, and may also be provided as SaaS (Software as a Service).

[0212] Effects of the embodiment According to the above-described embodiment, the following effects can be obtained.

[0213] [1] By performing pre-processing on the original image BI before obtaining the scanned image SI, it is possible to reduce the processing time after obtaining the scanned image SI, i.e., to improve the processing speed of the inspection. In addition, since it is possible to perform time-consuming and complex pre-processing such as feature extraction on the original image BI, the accuracy of the inspection is improved.

[0214] [2] Inspection accuracy is improved because appropriate processing can be performed for each type of defect that may occur in printed materials.

[0215] <Inkjet head ejection method> The ejector of the inkjet head includes a nozzle for ejecting liquid, a pressure chamber connected to the nozzle, and an ejection energy generating element for applying ejection energy to the liquid in the pressure chamber. Regarding the ejection method for ejecting droplets from the nozzle of the ejector, the means for generating the ejection energy is not limited to a piezoelectric element, and various ejection energy generating elements such as a heating element and an electrostatic actuator can be used. For example, a method can be adopted in which droplets are ejected by utilizing the pressure of film boiling caused by heating the liquid with a heating element. Depending on the ejection method of the inkjet head, a corresponding ejection energy generating element is provided in the flow path structure.

[0216] About recording media "Paper" is an example of a recording medium used to form an image. The term recording medium is a general term for various terms such as recording paper, printing paper, printing medium, print medium, print medium, image forming medium, image receiving medium, and ejection medium. The material and shape of the recording medium are not particularly limited, and various sheet bodies can be used regardless of material and shape, such as sticker paper, resin sheet, film, cloth, nonwoven fabric, etc. The recording medium is not limited to a sheet medium, and may be a continuous medium such as continuous paper. In addition, the sheet paper is not limited to a cut paper that is pre-arranged to a specified size, and may be obtained by cutting a continuous medium to a specified size at any time.

[0217] <About printing devices> In the above embodiment, a single-pass type inkjet printer 1 has been described as an example of a printing device, but printing devices used to print printed matter can take various forms, whether with or without plates. For example, the printed matter inspection device and printed matter inspection method disclosed herein can be applied to various printing devices, such as a multi-scan type inkjet printer that forms an image by scanning a short inkjet head back and forth, an electrophotographic device, or an offset printer.

[0218] <Terminology> The term "printing device" is synonymous with terms such as printing press, printer, printing device, image recording device, image forming device, image output device, and drawing device.

[0219] "Image" is to be interpreted broadly and includes color images, black and white images, single color images, gradation images, uniform density (solid) images, etc. "Image" is not limited to photographic images, but is used as a comprehensive term including designs, characters, symbols, line drawings, mosaic patterns, color-coded patterns, other various patterns, or appropriate combinations of these. "Printing" includes the concepts of terms such as image formation, image recording, printing, drawing, and printing.

[0220] "others" The present disclosure is not limited to the above-described embodiment, and various modifications are possible without departing from the spirit and scope of the technical idea of ​​the present disclosure. [Explanation of symbols]

[0221] 1. Inkjet printer 10 Paper feed section 11. Transport mechanism 12 Paper feeder 12A paper feed stand 14 Feeder Board 16 Paper feed drum 20 Processing liquid application section 22 Processing liquid application drum 23 Gripper 24 Processing liquid application device 30 Processing liquid drying section 32 Processing liquid drying drum 33 Gripper 34 Hot air blower 40 Drawing section 42 Drawing Drum 43 Gripper 44 Head Unit 46C Inkjet Head 46K Inkjet Head 46M Inkjet Head 46Y Inkjet Head 47 Lola 48 Scanner 50 Ink drying unit 55 Sorting Department 60 Accumulation section 62 Accumulation device 62A Stacking Tray 70 Chain Gripper 71A 1st sprocket 71B 2nd sprocket 72 Chain 74 Gripper 80 Paper Guide 82 First Paper Guide 84 Second Paper Guide 90 Heating and drying processing section 100 Control device 101 Printing System 102 processors 104 Computer-readable medium 106 Communication Interface 108 Input / Output Interface 110 Bus 112 RAM 114 ROM 116 Storage 122 Input Devices 124 Display device 304 Communications Department 310 System Control Unit 311 Image Processing Unit 312 Transport control section 313 Paper feed control unit 314 Processing liquid application control unit 316 Processing liquid drying control unit 318 Drawing control section 319 Inspection Processing Section 320 Ink drying control unit 322 Sorting control section 324 Paper ejection control unit 332 Image Memory 334 Parameter memory section 336 Program Memory Section 350 host computer 401 Line 401A Line Features 402 Dirt 402A Dirt characteristics 500 Print inspection equipment 502 Original image acquisition unit 504 First pre-processing unit 505 First comparison standard data 506 Second pre-processing section 507 Second comparison standard data 508 Storage section 510 Scan image acquisition unit 514 First defect specific processing section 515 First comparison data 516 Second defect specific processing section 517 Second comparative data 520 Defect Inspection Department 521 First Defect Inspection Department 522 Second Defect Inspection Department 530 Inspection result output unit 531 First Test Result 532 Second Test Results 610 The First Neural Network 620 Second Neural Network 630 The Third Neural Network BI Original Image BF Original image feature CD Comparison Data CD1 Comparison Data CD2 Comparison Data CI conversion image CP Comparison Processing CP1 Comparison Processing CP2 Comparison Processing CSI Compressed Images F5A left diagram F5B right image IR Inspection Results IR1 First defect detection result IR2 Second defect detection result LRI low-resolution image NAI Noise added image P paper P11 Color conversion processing P12 Resolution conversion processing P13 Noise addition processing PP1, PP11, PP12 Pretreatment PP2, PP21, PP22 Pretreatment RD Comparison Reference Data RD1 Comparison Reference Data RD2 Comparison Reference Data SI scan images SF scan image features S11~S20 Inspection processing steps S110 to S128 Steps for operating a printing system including a print inspection device

Claims

1. A print inspection device that detects multiple types of defects in a print using inspection data acquired based on an image of the print and reference data that is a digital image of a print target image indicated by print data of the print, comprising: one or more processors; one or more storage devices; The one or more processors: Obtaining the reference data; as a pre-processing for converting the reference data in order to compare the reference data with the inspection data, before the inspection data is acquired, a plurality of pre-processing steps different depending on the type of defect to be detected are performed on the reference data to generate a plurality of comparison reference data different depending on the type of defect to be detected from the reference data; storing the plurality of comparison reference data in the one or more storage devices; Print inspection equipment.

2. The plurality of pre-processing processes include a first feature amount extraction process for extracting a feature amount indicating an image feature of the reference data. The print inspection device according to claim 1 .

3. The first feature extraction process is performed using a first deep learning model. The print inspection device according to claim 2 .

4. the plurality of pre-processing processes include an image conversion process for making a data structure of the reference data closer to a data structure of the inspection data; The print inspection device according to claim 1 .

5. The image conversion process includes a resolution conversion process. The print inspection device according to claim 4.

6. the image conversion process includes a color conversion process of converting a color space of the reference data into a color space of the captured image; The print inspection device according to claim 4.

7. the reference data is represented by CMYK data including color information of cyan, magenta, yellow, and black, and the captured image is represented by RGB data including color information of red, green, and blue, and the color conversion process includes a process of converting the CMYK data into the RGB data. The print inspection device according to claim 6.

8. the image conversion process includes a noise addition process of adding noise contained in the inspection data to the reference data; The print inspection device according to claim 4.

9. The noise in the inspection data includes at least one of geometric noise and optical noise. The print inspection device according to claim 8.

10. The plurality of pre-processing processes include at least one of an edge extraction process and a blurring process. The print inspection device according to claim 1 .

11. The plurality of pre-processing processes include an edge extraction process and a blurring process; The one or more processors: storing edge extraction data obtained by performing the edge extraction process and blurred data obtained by performing the blurring process in the one or more storage devices; The print inspection device according to claim 10.

12. The one or more processors: Obtaining the inspection data; generating a plurality of comparison data that differ depending on the defect type of the detection target based on the inspection data; performing a comparison process for comparing each of the plurality of comparison data with the corresponding comparison reference data for each type of the defect to be detected; The print inspection device according to claim 1 .

13. the one or more processors perform a plurality of processes on the inspection data, the processes differing depending on the defect type of the detection target, to obtain the plurality of comparison data; the plurality of processes performed on the inspection data include a second feature extraction process for extracting features of the inspection data; The print inspection device according to claim 12.

14. The second feature extraction process is performed using a second deep learning model. The print inspection device according to claim 13.

15. the comparison process includes processing using a machine learning model that is trained to receive the comparison data and the corresponding comparison reference data and output at least one piece of information regarding the presence or absence of the defect, the strength of the defect, and the position of the defect. The print inspection device according to claim 12.

16. The machine learning model is a third deep learning model. The print inspection device according to claim 15.

17. The comparison process includes a difference process for calculating a difference between the comparison data and the corresponding comparison reference data. The print inspection device according to claim 12.

18. the plurality of types of defects include a first defect which is a streak-like defect and a second defect other than the first defect; The print inspection device according to claim 1 .

19. The one or more processors: performing a compression process for compressing image information of each of the reference data and the inspection data in a transport direction of a print medium when the printed matter is printed, in order to perform a process for detecting the first defect; performing a resolution reduction process on each of the reference data and the inspection data in order to perform a process for detecting the second defect; The print inspection device according to claim 18.

20. The one or more processors: In order to detect the first defect, as a first pre-processing in the plurality of pre-processings, a feature amount of the reference data is extracted using a first deep learning model, and the extracted feature amount is stored in the one or more storage devices as first comparison reference data in the plurality of comparison reference data; performing an image conversion process to make a data structure of the reference data closer to a data structure of the inspection data as a second pre-processing process in the plurality of pre-processing processes in order to detect the second defect, and storing the converted data obtained by the image conversion process in the one or more storage devices as second comparison reference data in the plurality of comparison reference data; The print inspection device according to claim 18.

21. The image conversion process includes:

21. The print inspection device according to claim 20, further comprising at least one of a resolution conversion process, a color conversion process, an edge extraction process, a blurring process, and a noise addition process.

22. A print inspection device according to any one of claims 1 to 21, a printing device that performs printing based on the print data and generates the printed matter; an imaging device for imaging the printed matter; A printing system comprising:

23. A print inspection method for detecting a plurality of types of defects in a print by using inspection data acquired based on an image of the print and reference data which is a digital image of an image to be printed indicated by print data of the print, comprising: One or more processors obtaining said reference data; as a pre-processing for converting the reference data for comparing the reference data with the inspection data, before the inspection data is acquired, a plurality of pre-processings are performed on the reference data, the pre-processings differing depending on the type of defect to be detected, to generate a plurality of comparison reference data from the reference data, the comparison reference data differing depending on the type of defect to be detected; storing the plurality of comparison reference data in one or more storage devices; A print inspection method that performs

24. A program for causing a computer to realize a function of detecting a plurality of types of defects in a printed matter by using inspection data acquired based on an image of the printed matter and reference data which is a digital image of an image to be printed indicated by print data of the printed matter, the program comprising: The computer includes: A function for acquiring the reference data; a function of performing a plurality of pre-processing steps, which differ depending on the type of defect to be detected, on the reference data before the inspection data is acquired as a pre-processing step of converting the reference data in order to compare the reference data with the inspection data, to generate a plurality of comparison reference data, which differ depending on the type of defect to be detected, from the reference data; A function of storing the plurality of comparison reference data in one or more storage devices; A program to achieve this.

Images