Inspection device, inspection method, inspection program, and printing system

Abstract

This inspection device is provided with a processor. The processor acquires a captured image obtained by capturing an image of a printed material printed by a printing device, and outputs defect information related to a defect of the printed material that has been inspected using only the captured image.

Description

Inspection device, inspection method, inspection program, and printing system

[0001] This disclosure relates to inspection devices, inspection methods, inspection programs, and printing systems.

[0002] Japanese Patent Publication No. 2023-131892 discloses a defect detection device for printed images, comprising: an inspection image acquisition unit that acquires an inspection image by imaging a printed medium on which an image corresponding to a reference image that serves as a reference for the image to be detected for defects is printed; a machine learning model that performs machine learning using training data associated with the type of defect and is capable of outputting a similarity score, which is the degree of similarity of the defect types; and a class discrimination processing unit that uses the machine learning model to determine the type of defect contained in the printed inspection image, calculates the similarity score of the defect types and outputs it.

[0003] Japanese Patent Publication No. 2021-128064 discloses a printed material defect inspection device comprising: a memory for storing instructions to be executed by a processor; a processor for executing the instructions stored in the memory; the processor acquires imaging data based on an image of a printed material; acquires reference data that serves as a standard for inspecting the printed material; takes the imaging data and the reference data as input and outputs first defect information of the printed material using at least one machine learning model capable of detecting defect information; converts the sizes of the imaging data and the reference data by a first ratio, and when M and N are integers of 1 or more, the process of outputting the (M+1)th defect information of the printed material using at least one machine learning model, taking the imaging data and the reference data whose sizes have been converted M times as input, is carried out from 1 to N; and the device determines defects in the printed material based on at least one of the (N+1)th defect information from the first defect information.

[0004] Japanese Patent Application Laid-Open No. 2019-101540 discloses an equipment diagnostic apparatus comprising: a classification means that classifies defects detected from a captured image of an inspection object manufactured by manufacturing equipment into defect types using a discriminant based on machine learning results; and a display means that causes maintenance information of the manufacturing equipment corresponding to the defect type to be displayed on a monitor based on data indicating the relationship between the defect type and maintenance information of the manufacturing equipment.

[0005] Japanese Patent Application Laid-Open No. 2019-117105 discloses a print quality inspection apparatus comprising: a printing condition input unit that inputs printing conditions; an image acquisition unit that acquires an inspected image obtained by capturing the printed surface of a printed material continuously conveyed in a conveyance direction; an inspection reference determination unit that determines an inspection area and an inspection reference based on the inspected image; a comparison inspection unit that compares a reference image with the inspected image to extract a defect candidate image; and a defect type determination unit that determines the defect type of harmful defects and harmless defects using a machine learning model based on the printing conditions and the defect candidate image.

[0006] Inspection apparatuses that perform inspection by comparing a captured image obtained by capturing an image of a printed material to be inspected using a scanner with a correct reference image are widely widespread. Conventional inspection apparatuses have two methods: a case where a captured image obtained by capturing the printed material with an imaging apparatus is used as a reference image, and a case where digital data input to the printing apparatus is used as a reference image. When a captured image is used as a reference image, the accuracy is high, but there is a drawback that it cannot cope with variable printing in which patterns differ for each page. In addition, when digital data is used as a reference image, there is an advantage that it can cope with variable printing, but there is a drawback that transfer of digital data becomes complicated, and color matching and position alignment become difficult. For this reason, it is desired to detect defects in printed materials without using a reference image.

[0007] The present disclosure aims to provide an inspection apparatus, an inspection method, an inspection program, and a printing system that can detect defects in printed materials without using a reference image.

[0008] To achieve the above objective, the inspection apparatus according to the first embodiment includes a processor, the processor acquires an image of a printed material printed by a printing device, and outputs defect information relating to defects in the printed material that have been inspected using only the image.

[0009] In the inspection apparatus according to the second embodiment, the processor is a trained model that has been trained by machine learning, and the defect information is acquired by inputting the captured image into the trained model that outputs the defect information when the captured image is input.

[0010] In the inspection apparatus according to the third embodiment, the processor outputs defect information relating to streak-like defects that occur in the transport direction in which the printing apparatus transports the printed material.

[0011] In the inspection apparatus according to the fourth embodiment, the processor performs filtering on the captured image before inspecting for defects in the printed material.

[0012] The inspection apparatus according to the fifth embodiment is an inspection apparatus according to the first embodiment, wherein the filtering process is a process that smooths the pixel values ​​of each pixel of the captured image along the transport direction in which the printing apparatus transports the printed material.

[0013] In the sixth embodiment of the inspection apparatus, the processor outputs defect information relating to the defect in the flat region extracted from the captured image, as in the inspection apparatus according to the first embodiment.

[0014] In the inspection apparatus according to the seventh embodiment, the processor outputs defect information relating to defects in the printed material that have been inspected using only the captured image until a reference image of the printed material is acquired, and after the reference image of the printed material is acquired, it outputs defect information relating to defects in the printed material that have been inspected based on the captured image and the reference image.

[0015] The inspection method according to the eighth aspect includes a process in which a computer acquires an image of a printed material captured by a printing device, and outputs defect information relating to defects in the printed material that have been inspected using only the image.

[0016] The inspection program according to the ninth embodiment causes a computer to perform a process that includes acquiring an image of a printed material captured by a printing device, and outputting defect information regarding defects in the printed material that have been inspected using only the image.

[0017] The printing system according to the tenth embodiment comprises a printing device and an inspection device according to any of the first to seventh embodiments that outputs defect information relating to defects in printed materials printed by the printing device.

[0018] This disclosure offers the advantage of being able to detect defects in printed materials without using reference images.

[0019] This is a diagram showing the configuration of the printing system. This is a diagram showing the hardware configuration of the printing device. This is a diagram showing the hardware configuration of the inspection device. This is a diagram showing the functional configuration of the inspection device. This is a flowchart of the inspection process.

[0020] Hereinafter, an example of this embodiment will be described in detail with reference to the drawings. In this embodiment, a printing system in which a management device, a printing device, and a client computer are connected via various communication lines such as networks will be described as an example. Figure 1 is a diagram showing the schematic configuration of the printing system 10 according to this embodiment.

[0021] As shown in Figure 1, the printing system 10 according to this embodiment includes a management device 11, a printing device 12, an inspection device 13, and a client computer 14. The management device 11, printing device 12, inspection device 13, and client computer 14 are each connected via a communication line 18 such as a LAN (Local Area Network), WAN (Wide Area Network), the Internet, or an intranet. The management device 11, printing device 12, inspection device 13, and client computer 14 are each capable of sending and receiving various types of data to and from each other via the communication line 18. In this embodiment, the client computer 14 issues a printing instruction to the printing device 12 via the management device 11, and the printing device 12 forms an image on the paper according to the printing instruction.

[0022] In Figure 1, one management device 11, one printing device 12, one inspection device 13, and one client computer 14 are shown, but there may be multiple instances of each, or multiple instances of any one of them.

[0023] The printing apparatus 12 according to this embodiment has multiple functions, such as a printing function for performing printing operations and a post-processing function for performing post-processing on paper on which an image has been formed. The multiple functions may include a reading function for reading a document and obtaining image information representing the document, a copying function for copying the image recorded on the document onto paper, a facsimile function for sending and receiving various data via a telephone line (not shown), a transfer function for transferring document information such as image information read by the reading function, etc., and a storage function for accumulating document information such as image information that has been read.

[0024] In the following explanation, the facsimile function may be referred to as "fax," the reading function as "scan," the printing function as "print," and the copying function as "copy."

[0025] Figure 2 is a block diagram showing the main electrical components of the printing device 12 in the printing system 10 according to this embodiment.

[0026] As shown in Figure 2, the printing apparatus 12 according to this embodiment includes a control unit 20. The control unit 20 may include a CPU (Central Processing Unit), ROM (Read Only Memory), and RAM (Random Access Memory).

[0027] On the other hand, the printing apparatus 12 according to this embodiment is equipped with an HDD (hard disk drive) 26 for storing various data and application programs. The printing apparatus 12 is also equipped with a display control unit 28 connected to a user interface 22, which controls the display of various operation screens and the like on the display of the user interface 22.

[0028] Furthermore, the printing device 12 is connected to the user interface 22 and includes an operation input detection unit 30 that detects operation instructions input via the user interface 22. In the printing device 12, the HDD 26, the display control unit 28, and the operation input detection unit 30 are electrically connected to the system bus 42. In this embodiment, the printing device 12 uses an HDD 26 as the storage unit, but it is not limited to this, and a non-volatile storage unit such as flash memory may be used. Also, in this embodiment, the user interface 22 is a touch panel capable of display and operation input, but it is not limited to this, and a separate user interface with a display and an operation unit may be used.

[0029] Furthermore, the printing apparatus 12 according to this embodiment includes a printing control unit 34 that controls the printing process by the printing unit 24, the transport unit 25 transporting paper to the printing unit 24, the post-processing unit 46, the imaging process (reading process) of the printed material by the inspection unit 47, and the maintenance unit 48.

[0030] The printing unit 24 performs printing using a so-called single-pass method, for example, by using a wide droplet ejection head that has a width greater than the width of the paper. The print control unit 34 converts the print data instructed by the client computer 14 into raster data (bitmap data) and outputs it to the printing unit 24. The printing unit 24 prints on the paper based on the raster data output from the print control unit 34.

[0031] The inspection unit 47 includes a scanner comprising an image sensor such as a CCD (Charge Coupled Device) and an optical system, and outputs an image (scanned image) captured by the image sensor of the printed material printed by the printing unit 24 to the inspection device 13. The maintenance unit 48 performs maintenance on the droplet ejection head.

[0032] The printing device 12 may include a reading control unit that controls the optical image reading operation by the document reading unit and the document feeding operation by the document transport unit. The printing device 12 also includes a communication line interface unit 36 ​​connected to a communication line 18, which transmits and receives communication data with other external devices such as a client computer 14 connected to the communication line 18.

[0033] Furthermore, the printing device 12 may be connected to a telephone line (not shown) and may include a facsimile interface (I / F) unit that transmits and receives facsimile data with a facsimile device connected to the telephone line. The printing device 12 may also include a transmit / receive control unit that controls the transmission and reception of facsimile data via the facsimile interface unit. In the printing device 12, the print control unit 34 and the communication line interface unit 36 ​​are electrically connected to the system bus 42.

[0034] With the above configuration, the printing apparatus 12 according to this embodiment controls the display of information such as operation screens and various messages on the user interface 22 display via the display control unit 28, using the control unit 20. The printing apparatus 12 also controls the operation of the printing unit 24, transport unit 25, post-processing unit 46, inspection unit 47, and maintenance unit 48 via the print control unit 34, and controls the transmission and reception of communication data via the communication line I / F unit 36, using the control unit 20. Furthermore, the printing apparatus 12 grasps the operation content on the user interface 22 based on the operation information detected by the operation input detection unit 30, using the control unit 20, and performs various controls based on this operation content.

[0035] In this embodiment, an example of an application stored on the HDD26 is an application that performs functions such as printing.

[0036] The inspection device 13 acquires an image of the printed material printed by the printing device 12 from the inspection unit 47, and outputs defect information regarding defects in the printed material that were inspected using the image.

[0037] The printing system 10 is managed by a control device 11 which manages a series of manufacturing processes. The series of manufacturing processes include, for example, a production process, a prepress process, a plate-making process, a printing process, a processing process, and a delivery process.

[0038] The management device 11 and client computer 14 are configured to include general-purpose computers, so their description will be omitted.

[0039] Next, the main components of the electrical system of the inspection device 13 according to this embodiment will be described. Figure 3 is a block diagram showing the main components of the electrical system of the inspection device 13 in the printing system 10 according to this embodiment. The inspection device 13 is composed of a general-purpose computer.

[0040] As shown in Figure 3, the inspection device 13 includes a controller 50. The controller 50 is composed of a device including a general-purpose computer.

[0041] The controller 50 includes a CPU (Central Processing Unit) 50A, a ROM (Read Only Memory) 50B, a RAM (Random Access Memory) 50C, and an input / output interface (I / O) 50D. The CPU 50A, ROM 50B, RAM 50C, and I / O 50D are each connected via a bus 50E. The bus 50E includes a control bus, an address bus, and a data bus.

[0042] Furthermore, a communication unit 52 and a storage unit 54 are connected to the I / O 50D.

[0043] The communication unit 52 is an interface for performing data communication with the printing apparatus 12 and the like.

[0044] The storage unit 54 is configured by a non-volatile external storage device such as a hard disk. As shown in FIG. 3, the storage unit 54 stores an inspection program 54A and the like.

[0045] FIG. 4 shows an example of the functional configuration of the inspection apparatus 13.

[0046] Functionally, the inspection apparatus 13 includes an acquisition unit 60 and an output unit 61. These functional configurations are realized when the CPU 50A reads the inspection program 54A from the storage unit 54, develops it into the RAM 50C, and executes it.

[0047] The acquisition unit 60 acquires, from the printing apparatus 12, a captured image obtained by imaging a printed matter printed by the printing apparatus 12, that is, a captured image (scanned image) captured by an inspection unit 47 of the printing apparatus 12.

[0048] The output unit 61 outputs defect information related to defects in the printed matter that is inspected using only the captured image of the printed matter acquired by the acquisition unit 60.

[0049] Specifically, as an example, the output unit 61 is a trained model learned by machine learning, and acquires defect information by inputting a captured image into the trained model that outputs defect information when the captured image is input thereto.

[0050] As mentioned above, the printing unit 24 performs printing using a single-pass method. However, in the single-pass method, streaky defects may occur due to the ink ejected from the nozzle of the droplet ejection head bending or not being ejected at all. For this reason, the output unit 61 may output defect information related to streaky defects that occur in the transport direction as the printing device 12 transports the printed material. Note that the defect information is not limited to streaky defects. For example, the output unit may also output defect information related to other defects such as dot-like defects caused by the adhesion of paper dust or other debris, ink dripping, smudging, and soiling.

[0051] The trained model can be a neural network model constructed by training with training data that includes a large number of images labeled with defects, for example. As an example of a training algorithm, so-called deep learning using convolutional neural networks can be used.

[0052] When an image is input to such a pre-trained model, the model outputs defect information such as the type of defect, the location of the defect in the image, and the severity of the defect. This makes it possible to detect defects in printed materials using only the image, without using a reference image.

[0053] Furthermore, captured images may be inspected using methods other than those employing pre-trained models. For example, in the case of streak-like defects, if the printing resolution is sufficiently high compared to the resolution of the captured image, the defect information in the paper width direction in the captured image will be captured as a single pixel defect. Therefore, the captured image can be divided into two types of pixels, for example, even pixels and odd pixels in the paper width direction, and the difference between the even and odd pixels can be calculated. If the calculated difference is greater than or equal to a threshold, it can be determined that it is a streak-like defect. In this way, by calculating the difference in pixel values ​​between even and odd pixels in the paper width direction, high-frequency defects such as streak-like defects can be detected.

[0054] Furthermore, the output unit 61 may perform filtering on the captured image acquired by the acquisition unit 60 before inspecting for defects in the printed material. Specifically, the filtering process is, for example, a process that smooths the pixel values ​​of each pixel in the captured image along the transport direction in which the printing device 12 transports the printed material. For example, a moving average can be used for the process of smoothing the pixel values, but it is not limited to this.

[0055] Furthermore, the output unit 61 may output defect information relating to defects in flat regions extracted from the captured image acquired by the acquisition unit 60. Here, a flat region is a region that is visible as a region composed of a constant color with little color variation compared to other regions, and is not limited to, but is an example of human skin. Also, known image processing methods can be used as a method for extracting flat regions from the captured image.

[0056] Furthermore, the output unit 61 may perform filtering on the captured image using a filter such as a high-pass filter or a Gaussian filter in the paper width direction of the printed material, thereby removing noise in the flat areas while retaining the peaks of the streaks, and then inspecting for defects in the printed material. This makes it easier to identify edge areas extending in the transport direction of the printed material within the flat areas as streak-like defects. In addition, by removing noise in advance, for example, the parameters required for noise reduction in a trained model become unnecessary, which simplifies training and improves accuracy.

[0057] Alternatively, the output unit 61 may output defect information regarding defects in the printed material inspected using only the captured image until a reference image of the printed material is acquired, and after the reference image of the printed material is acquired, it may output defect information regarding defects in the printed material inspected based on both the captured image and the reference image.

[0058] Here, the reference image is the correct image that should be printed on the printed material, and is an image represented by raster data converted from print data by the print control unit 34. Then, for example, by using a trained model that takes the reference image and an image of the printed material as input and outputs defect information, it is possible to obtain defect information with high accuracy. In this case, the print control unit 34 needs to convert the print data into raster data and transfer it to the inspection device 13 as the reference image, but the inspection device 13 needs to wait to perform the inspection until it receives the reference image transferred from the print control unit 34.

[0059] Therefore, until a reference image of the printed material is acquired, the output unit 61 outputs defect information regarding defects in the printed material that have been inspected using only the captured image, as described above. After acquiring the reference image of the printed material from the print control unit 34, the output unit 61 inputs the captured image and the reference image into the trained model, and then acquires and outputs defect information regarding defects in the printed material from the trained model.

[0060] This makes it possible to perform defect inspection without waiting for the generation of a reference image, thereby shortening the time until printing can begin. Alternatively, a pre-trained model trained using deep learning may be used, or a conversion process may be performed to make the print data closer to the captured image, the difference between the converted reference image and the captured image may be calculated, and defects may be detected based on the calculated difference.

[0061] Next, referring to Figure 5, the processing of the inspection program 54A executed by the CPU 50A of the inspection device 13 will be described.

[0062] Figure 5 is a flowchart showing an example of an inspection process executed by the CPU 50A by the inspection program 54A. The inspection process shown in Figure 5 is executed when the printing process by the printing device 12 is disclosed.

[0063] In step S100, the CPU 50A requests a reference image from the printing device 12.

[0064] In step S101, the CPU 50A acquires an image of the printed material captured from the printing device 12.

[0065] In step S102, the CPU 50A performs a predetermined filtering process on the image acquired in step S101.

[0066] In step S103, the CPU 50A determines whether or not a reference image has been acquired from the printing device 12. If a reference image has not been acquired, the process proceeds to step S104. If a reference image has been acquired, the process proceeds to step S105.

[0067] In step S104, the CPU 50A detects defects in the printed material using only the captured image.

[0068] In step S105, the CPU 50A detects defects in the printed material based on the image acquired in step S101 and the image acquired in step S100.

[0069] In step S106, the CPU 50A outputs defect information related to the defect detected in step S104 or step S105 to, for example, the printing device 12 for display on the user interface 22, or to the management device 11 for storage.

[0070] In this embodiment, an image of the printed material produced by the printing device 12 is acquired, and defect information regarding defects in the printed material is output using only the image until a reference image is acquired. In this way, defects in the printed material can be detected without using a reference image, thus shortening the time until printing can begin.

[0071] In this embodiment, the inspection program 54A was described as being stored in the storage unit 54, but the inspection program 54A may be provided on a storage medium such as a CD-ROM, or it may be downloaded via a network.

[0072] Furthermore, the configuration and operation of the inspection device 13 described in the above embodiment are merely examples and can be modified as needed without departing from the spirit of this disclosure.

[0073] The following additional information is disclosed regarding the embodiments described above.

[0074] (Note) (Note 1) An inspection device comprising a processor, wherein the processor acquires an image of a printed material printed by a printing device, and outputs defect information relating to defects in the printed material that have been inspected using only the image.

[0075] (Note 2) The inspection apparatus according to Note 1, wherein the processor is a trained model trained by machine learning, and the inspection apparatus acquires the defect information by inputting the captured image into the trained model which outputs the defect information when the captured image is input.

[0076] (Note 3) The inspection device according to Note 1 or Note 2, wherein the processor outputs defect information relating to streak-like defects occurring in the transport direction in which the printing device transports the printed material.

[0077] (Note 4) The processor is an inspection apparatus according to any one of Notes 1 to 3, which performs filtering on the captured image before inspecting for defects in the printed material.

[0078] (Note 5) The inspection apparatus according to Note 4, wherein the filtering process is a process that smooths the pixel values ​​of each pixel of the captured image along the transport direction in which the printing apparatus transports the printed material.

[0079] (Note 6) The inspection apparatus according to any one of Notes 1 to 5, wherein the processor outputs defect information relating to the defect in the flat region extracted from the captured image.

[0080] (Note 7) The inspection apparatus according to any one of Notes 1 to 6, wherein the processor outputs defect information relating to defects in the printed material that have been inspected using only the captured image until a reference image of the printed material is acquired, and after a reference image of the printed material is acquired, it outputs defect information relating to defects in the printed material that have been inspected based on the captured image and the reference image.

[0081] (Note 8) An inspection method that includes a computer acquiring an image of a printed material printed by a printing device, and outputting defect information relating to defects in the printed material that have been inspected using only the image.

[0082] (Note 9) An inspection program that causes a computer to perform a process that includes acquiring an image of a printed material printed by a printing device, and outputting defect information regarding defects in the printed material that have been inspected using only the image.

[0083] (Note 10) A printing system comprising: a printing device; and an inspection device described in any one of Notes 1 to 7 that outputs defect information relating to defects in printed materials printed by the printing device.

[0084] Furthermore, the disclosure of Japanese Patent Application No. 2025-056969 is incorporated herein by reference in its entirety. In addition, all documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

Claims

1. An inspection device comprising a processor, wherein the processor acquires an image of a printed material captured by a printing device, and outputs defect information relating to defects in the printed material, which is inspected using only the image.

2. The inspection apparatus according to claim 1, wherein the processor is a trained model trained by machine learning, and the inspection apparatus acquires the defect information by inputting the captured image to the trained model which outputs the defect information when the captured image is input.

3. The inspection apparatus according to claim 1, wherein the processor outputs defect information relating to streak-like defects occurring in the transport direction in which the printing apparatus transports the printed material.

4. The inspection apparatus according to claim 1, wherein the processor performs filtering on the captured image before inspecting for defects in the printed material.

5. The inspection apparatus according to claim 4, wherein the filtering process is a process of smoothing the pixel values ​​of each pixel of the captured image along the transport direction in which the printing apparatus transports the printed material.

6. The inspection apparatus according to claim 1, wherein the processor outputs defect information relating to the defect in the flat region extracted from the captured image.

7. The inspection apparatus according to claim 1, wherein the processor outputs defect information relating to defects in the printed material inspected using only the captured image until a reference image of the printed material is acquired, and after the reference image of the printed material is acquired, the processor outputs defect information relating to defects in the printed material inspected based on the captured image and the reference image.

8. An inspection method that includes a computer acquiring an image of a printed material captured by a printing device, and outputting defect information relating to defects in the printed material, which was inspected using only the image.

9. An inspection program that causes a computer to perform a process that includes acquiring an image of a printed material captured by a printing device, and outputting defect information regarding defects in the printed material, which is inspected using only the image.

10. A printing system comprising: a printing device; and an inspection device according to any one of claims 1 to 7 that outputs defect information relating to defects in printed materials printed by the printing device.

Images