Image printing system and program

Abstract

To bring evaluation of a printed matter close to human sensory evaluation as compared with a case where a feature appearing in a difference image is digitized to inspect a defect.SOLUTION: An image printing system includes: a printing device that prints an image corresponding to print data on a sheet; a scanner that reads the image printed by the printing device; and a functional value determination unit that acquires a functional value representing degree of a defect obtained from visual functional evaluation for a combination of first color information of the print data printed by the printing device and second color information of the image read by the scanner.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an image printing system and a program. [Background technology]

[0002] 2. Description of the Related Art Some printing systems include a process for inspecting printed matter for defects as a post-printing process. In this type of printing system, an image used as a standard for inspection (hereinafter referred to as a "reference image") may be generated from print data corresponding to the printed material. This type of printing system compares an image optically read from the printed material (hereinafter referred to as an "inspection image") with the reference image, and determines that any areas where the pixel values ​​of the different parts exceed a threshold value are defective. Examples of defects include stains and whiteouts. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-4276 Summary of the Invention [Problem to be solved by the invention]

[0004] However, even for the same stain, etc., human evaluation of defects is comprehensive, and even if the features appearing in the difference image generated from the reference image and the inspection image are quantified, the evaluation may not match the human evaluation. For example, even if the magnitude of the numerical value calculated from the features appearing in the difference image is the same, the sensory evaluation may determine that the defect is defective or not.

[0005] The present invention aims to bring the evaluation of printed matter closer to human sensory evaluation, compared to when defects are inspected by quantifying features that appear in a difference image. [Means for solving the problem]

[0006] The invention described in claim 1 is an image printing system having a printing device that prints an image corresponding to printing data on paper, a scanner that reads the image printed by the printing device, and a sensory value determination unit that obtains a sensory value that represents the degree of defect determined from visual sensory evaluation for a combination of first color information of the printing data printed by the printing device and second color information of the image read by the scanner. A second aspect of the present invention is the image printing system according to the first aspect, further comprising a control unit that controls printing based on the sensory value acquired by the sensory value determination unit. The invention described in claim 3 is a program for enabling a computer to realize the function of obtaining a sensory value representing the degree of defect determined from visual sensory evaluation for a combination of first color information of a reference image used as an inspection standard and second color information of an inspection image read from a printed material. The invention described in claim 4 is the program described in claim 3 for further realizing a function of controlling printing based on the acquired information. The invention described in claim 5 is a program described in claim 3 for further realizing the function of rejecting printed matter if the severity of defects in the printed matter is high based on the magnitude of the acquired sensory value. The invention described in claim 6 is a program described in claim 3 for further realizing the function of providing the first color information, the second color information, and the characteristics of the area to be inspected to a learning model and obtaining the corresponding sensory value from the learning model. The invention described in claim 7 is an image printing system having a processor that provides a learning model with first color information of a reference image used as a standard for inspection, second color information of an inspection image read from a printed material, and features of the area to be inspected, and obtains from the learning model a sensory value that represents the degree of defect determined from visual sensory evaluation and corresponds to the combination of the first color information and the second color information, and inspects the printed material for defects. The invention described in claim 8 is a program for causing a computer to realize the function of providing a learning model with first color information of a reference image used as a standard for inspection, second color information of an inspection image read from a printed material, and characteristics of the area to be inspected, and obtaining from the learning model a sensory value representing the degree of defect determined from visual sensory evaluation that corresponds to the combination of the first color information and the second color information, and inspecting the printed material for defects. [Effects of the Invention]

[0007] According to the invention of claim 1, compared to the case where defects are inspected by quantifying features appearing in a differential image, the evaluation of a printed matter can be made closer to human sensory evaluation. According to the invention of claim 2, it is possible to make the evaluation of a printed matter closer to human sensory evaluation. According to the invention of claim 3, the evaluation of the printed matter can be made closer to human sensory evaluation, compared to when defects are inspected by quantifying features that appear in the differential image. According to the invention of claim 4, it is possible to make the evaluation of a printed matter closer to human sensory evaluation. According to the invention described in claim 5, printed matter with severe defects can be rejected. According to the invention of claim 6, the evaluation of the printed matter can be made closer to human sensory evaluation, compared to when defects are inspected by quantifying features that appear in the differential image. According to the seventh aspect of the present invention, the evaluation of a printed matter can be made closer to a human sensory evaluation, compared to when defects are inspected by quantifying features that appear in a differential image. According to the eighth aspect of the invention, the evaluation of a printed matter can be made closer to a human sensory evaluation, compared to when defects are inspected by quantifying features that appear in a differential image. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an image printing system used in the first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of an inspection device. [Figure 3]FIG. 10 is a diagram illustrating an example of data in a threshold table. [Figure 4] 10 is a diagram illustrating an example of the configuration of a sensory value LUT. [Figure 5] FIG. 2 is a diagram illustrating an example of a functional configuration of an inspection device used in the first embodiment. [Figure 6] 5 is a flowchart illustrating a part of an example of an inspection operation for defects by the inspection device used in the first embodiment. [Figure 7] 10 is a flowchart illustrating the remaining part of the example of the defect inspection operation by the inspection device used in the first embodiment. [Figure 8] 10 is a flowchart illustrating an example of an inspection operation for defects by the inspection device used in the second embodiment, where the inspection operation differs from the first embodiment. [Figure 9] 10 is a flowchart illustrating an example of an inspection operation for defects by the inspection device used in the third embodiment, illustrating differences from the first embodiment. [Figure 10] FIG. 10 is a diagram illustrating a machine learning device that learns the relationship between a combination of color information and features of a pixel extracted as a defect candidate and a sensory value obtained from a visual sensory evaluation. [Figure 11] FIG. 13 is a diagram illustrating an example of the functional configuration of an inspection device used in the fourth embodiment. [Figure 12] 1A and 1B are diagrams illustrating an example of a user specifying an area to be inspected for defects. (A) shows an example of a display of the positions of groups made up of pixels extracted as defect candidates, and (B) shows an example of a group being specified by the user. [Figure 13] 10A and 10B are diagrams illustrating an example of inspecting defects in an arbitrary area designated by a user. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. <First Embodiment> <System configuration> FIG. 1 is a diagram illustrating an example of the configuration of an image printing system 1 used in the first embodiment. The image printing system 1 shown in FIG. 1 is made up of a print data generating device 20 and a printing system 30 connected to a network 10. For example, a wired LAN (Local Area Network) or a wireless LAN is used as the network 10. However, the Internet or a mobile communication system such as 5G may also be used as the network 10.

[0010] The print data generating device 20 is a processing device that converts document data into print data. The print data generating device 20 is also called, for example, a DFE (Digital Front End) or a RIP (Raster Image Processor). In this embodiment, print data is output as raster format data (hereinafter referred to as "raster data"). Note that document data is text and images, and is provided from a computer operated by a user.

[0011] The printing system 30 comprises a printing device 310 that prints an image corresponding to the printing data on the surface of paper, an inspection device 320 that receives printed paper (hereinafter also referred to as "printed matter") from the printing device 310 and inspects the printing defects, and a post-processing device 330 that processes the printed matter according to a pre-specified output format. The printing device 310 has a processing unit that processes print data and a printing unit that prints an image on paper. The printing device 310 prints an image on the surface of paper using, for example, an electrophotographic method or an inkjet method. In this embodiment, the paper is assumed to be cut paper. However, the paper may also be roll paper.

[0012] The inspection device 320 optically reads the printed matter discharged from the printing device 310 to obtain an inspection image, and inspects the printed matter for defects in real time. Defects to be detected include, for example, stains and white spots. The inspection device 320 in this embodiment generates a reference image from print data to be used as a reference when inspecting for print defects. The post-processing device 330 is a so-called finisher, and performs, for example, sorting, stapling, folding, and tape binding.

[0013] <Configuration of inspection equipment> FIG. 2 is a diagram illustrating an example of the hardware configuration of the inspection device 320. As shown in FIG. 2 includes a processor 321 that controls the operation of the entire apparatus, a ROM (Read Only Memory) 322 that stores a BIOS (Basic Input Output System) and the like, a RAM (Random Access Memory) 323 that is used as a work area for the processor 321, a display 324 that displays information related to the inspection, an operation reception device 325 that receives user operations, a scanner 326 that reads inspection images from printed materials, a hard disk drive 327 that stores reference images and inspection images, and a communication module 328 that is used for communication with the outside. The processor 321 and each component are connected via a signal line 329 such as a bus.

[0014] The processor 321, ROM 322, and RAM 323 function as a so-called computer. The processor 321 realizes various functions by executing programs. For example, the processor 321 executes a process for detecting defects in printed matter. The display 324 is, for example, a liquid crystal display or an organic EL display. In the present embodiment, the display 324 is integrally provided on the main body of the device, but may be a monitor connected to the inspection device 320.

[0015] The operation reception device 325 is composed of a capacitance type film sensor arranged on the surface of the display 324, and switches, buttons, etc. arranged on the housing. A device that integrates the display 324 and the operation reception device 325 is called a touch panel. The touch panel is used to receive user operations on keys displayed as software (hereinafter also referred to as "soft keys"). The scanner 326 has a light source that irradiates the surface of the printed material being conveyed with linear illumination light, a line sensor that receives the light reflected from the surface of the printed material, and an optical system that forms an image of the reflected light on the line sensor. In this embodiment, an image of the entire printed material to be inspected is called an "inspection image." The inspection image is stored in, for example, RAM 323.

[0016] The hard disk drive 327 stores an inspection threshold table 327A, a sensory value LUT 327B, an inspection log 327C, and the like. The inspection threshold table 327A is a table that stores thresholds according to the characteristics of the image. For example, different thresholds are prepared for bright image parts where dirt is easily noticeable and dark image parts where white areas are easily noticeable. In addition, different thresholds are prepared for edge regions and non-edge regions.

[0017] The threshold value is determined assuming that the print data will be printed on white paper, i.e., that the background color of the test image is white. The white color assumed in the threshold value refers to, for example, a whiteness of 100%. Hereinafter, paper with a whiteness of 100% will also be referred to as white paper. It is desirable to change the threshold value depending on the color of the paper on which the print corresponding to the test image will be printed.

[0018] FIG. 3 is a diagram illustrating an example of data in the threshold table 327A. 3, threshold values ​​are set according to the combination of whether an edge exists in the reference image and the inspection image. In this embodiment, multiple threshold values ​​tables 327A are prepared according to differences in brightness of the reference image.

[0019] For example, "100" is set as the threshold for an area where no edge exists in both the reference image and the inspection image, and "150" is set as the threshold for an area where an edge exists in both the reference image and the inspection image. In addition, the threshold for areas where there are no edges in the reference image but there are edges in the inspection image is set to "50," and the threshold for areas where there are edges in the reference image but there are no edges in the inspection image is set to "150." The former areas are so-called stains, and the latter areas are so-called white areas.

[0020] Returning to the explanation of Figure 2. The sensory value LUT (=Lookup table) 327B is a correspondence table that links a combination of color information of a reference image of a part detected as a defect candidate and color information of an inspection image with a sensory value that represents the degree to which a person visually inspecting the target area perceives it as a defect. The color information of the reference image is an example of "first color information," and the color information of the inspection image is an example of "second color information." The sensory value is a numerical value that represents the result of the sensory evaluation. In this embodiment, the larger the numerical value of the sensory value, the more likely a person is to recognize it as a defect.

[0021] Fig. 4 is a diagram illustrating an example of the configuration of the sensory value LUT 327B. In Fig. 4, the color information of the reference image and the test image is expressed as coordinate values ​​in the Lab color space. For example, brightness L is given as a value between "0" and "100," and complementary color dimensions a and b are given as values ​​between "0" and "128." In the sensory value LUT 327B, the sensory value corresponding to the intersection position of the Lab values ​​of the reference image and the Lab values ​​of the test image is stored. For example, the sensory value corresponding to the combination of (L, a, b) = (90, 96, 50) of the reference image and (L, a, b) = (30, 15, 10) of the test image is "55".

[0022] In this embodiment, the sensory values ​​in the sensory value LUT 327B are the results of a sensory evaluation of whether a difference in color information in an area corresponding to a defect candidate is judged to be a defect when visually observed. In other words, the intersection of the Lab values ​​of the reference image and the Lab values ​​of the inspection image is linked to the result of a sensory evaluation of the difference in the Lab values. In other words, the sensory value is linked to a value that numerically represents the degree to which the human eye would recognize a defect when the Lab values ​​of the reference image are replaced with the Lab values ​​of the inspection image. In this sense, the sensory value in this embodiment differs from the evaluation value calculated from the magnitude of the color difference.

[0023] Returning to the explanation of Figure 2. Inspection log 327C records the results of inspections performed for each printed product. Additionally, the hard disk drive 327 may store a reference image generated from print data. As described above, the reference image in this embodiment is digitally generated from the print data by the processor 321 . In addition, firmware and application programs used for defect inspection are also stored in the hard disk drive 327. Hereinafter, firmware and application programs will be collectively referred to as "programs."

[0024] In this embodiment, the hard disk drive 327 is assumed to be the auxiliary storage device, but a semiconductor memory may also be used. The communication module 328 is configured with a communication module that complies with wired or wireless communication standards, and may be, for example, an Ethernet (registered trademark) module, a USB (Universal Serial Bus) module, or a wireless LAN module.

[0025] Fig. 5 is a diagram illustrating an example of the functional configuration of the inspection device 320 used in the first embodiment. Fig. 5 shows functions related to defect inspection. The functions shown in Fig. 5 are realized through the execution of a program by the processor 321 (see Fig. 2). The inspection device 320 shown in FIG. 5 has, as inspection-related functions, a reference image generation unit 351, an inspection image generation unit 352, an intra-image edge extraction unit 353, a difference image generation unit 354, a threshold setting unit 355, a defect candidate extraction unit 356, a sensory value determination unit 357, and a post-processing control unit 358.

[0026] The reference image generating unit 351 is a functional unit that generates a reference image based on print data provided from the print data generating device 20 (see FIG. 1) to the printing device 310. The reference image in this embodiment is generated assuming that the image is printed on white paper. The reference image is provided to an intra-image edge extraction unit 353 , a difference image generation unit 354 , a threshold setting unit 355 , and a functional value determination unit 357 .

[0027] The inspection image generating unit 352 is a functional unit that generates an inspection image in real time from an image read in a line by the scanner 326 . The inspection image is also provided to an intra-image edge extraction unit 353 , a difference image generation unit 354 , a threshold setting unit 355 , and a functional value determination unit 357 . The printed matter that has passed through the scanner 326 is discharged to a post-processing device 330 (see FIG. 1) at the subsequent stage.

[0028] The intra-image edge extraction unit 353 is a functional unit that extracts edges from both the reference image and the test image. Edges are, for example, the contours or features of a subject, and can be extracted using discontinuities in brightness values. The edge image extracted from the reference image and the edge image extracted from the test image are provided to a difference image generation unit 354 and a threshold setting unit 355. The difference image generating unit 354 has a registration unit 354A as a sub-function, and is a functional unit that generates a difference image between the reference image and the inspection image after registration. The alignment unit 354A aligns the reference image and the inspection image by, for example, comparing an edge image of the reference image with an edge image of the inspection image. The difference image generation unit 354 outputs the generated difference image to the defect candidate extraction unit 356.

[0029] The threshold setting unit 355 is a functional unit that sets a threshold used for defect inspection. In this embodiment, the threshold setting unit 355 reads a threshold corresponding to a combination of the brightness of the reference image, edge information of the reference image, edge information of the inspection image, etc. from the threshold table 327A (see FIG. 3 ), and outputs the threshold to the defect candidate extraction unit 356.

[0030] The defect candidate extraction unit 356 is a functional unit that compares the pixel values ​​of the differential image provided by the differential image generation unit 354 with a threshold value and extracts defect candidates contained in the inspection image. The defect candidate extraction unit 356 determines that pixels with pixel values ​​less than the threshold value are normal, and determines that pixels with pixel values ​​equal to or greater than the threshold value are defect candidates. Even if a pixel exceeds the threshold, it may or may not be evaluated by a person as a defect, and therefore in this embodiment, it is referred to as a "defect candidate." The defect candidate extraction unit 356 outputs the extracted defect candidates to the sensory value determination unit 357.

[0031] The functional value determining unit 357 groups the pixels extracted by the defect candidate extracting unit 356 and determines whether or not each group is a defect. The collection of grouped pixels is an example of a target region. In the present embodiment, the functional value determination unit 357 groups, for example, adjacent pixels among the pixels extracted as candidates. One group may be made up of only one pixel.

[0032] However, even if a pixel extracted as a candidate is not adjacent to another pixel or group of candidates, it may be included in the same group if the distance between the pixels or the shortest distance between the pixel and a pixel constituting the group is equal to or less than a predetermined threshold. Incidentally, if there are multiple groups with pixel distances equal to or less than a threshold for a pixel extracted as a candidate, the pixel will be included in the group with the shorter distance.

[0033] After grouping, the sensory value determination unit 357 determines a representative value of the color information of the pixels belonging to the same group for each of the reference image and the test image. The representative value is calculated, for example, as the average value of the color information of each pixel belonging to the same group. When the representative value for the group to be processed is determined, the sensory value determination unit 357 reads out the sensory value corresponding to the combination of the determined representative values ​​from the sensory value LUT 327 B. The read-out sensory value is output from the sensory value determination unit 357 to the post-processing control unit 358 as the sensory value of the corresponding group.

[0034] When the sensory values ​​have been determined for all groups of defect candidates associated with the same printed matter, the sensory value determination unit 357 notifies the post-processing control unit 358 of this fact. However, the sensory value determination unit 357 may notify the post-processing control unit 358 of the change in the print material to be processed together with the sensory value.

[0035] If the post-processing control unit 358 can distinguish whether the sensory values ​​provided by the sensory value determination unit 357 are linked to the same printed matter or different printed matters, there is no need to notify that the process of determining the sensory values ​​for each printed matter described above has ended. For example, if information identifying the printed matter being inspected is attached to each sensory value, the post-processing control unit 358 can distinguish the sensory values ​​linked to the same printed matter from the information attached to the sensory values.

[0036] The post-processing control unit 358 is a functional unit that determines post-processing for each printed matter that is the subject of inspection. In this embodiment, the post-processing control unit 358 determines the content of post-processing for each printed matter based on the magnitude of one or more sensory values ​​determined for each printed matter that is the subject of inspection. In this embodiment, printed matter is printed in page units. For this reason, the control of post-processing is also said to be performed on a page-by-page basis.

[0037] In this embodiment, the post-processing control unit 358 determines the content of post-processing for the corresponding printed material based on the maximum value of multiple sensory values ​​associated with the same printed material. If there are three defect candidates extracted from the same printed material, and the sensory value of group A is "70," the sensory value of group B is "50," and the sensory value of group C is "30," the largest of these, "70," is determined to be the representative sensory value.

[0038] In this embodiment, the post-processing control unit 358 provides three types of post-processing: "stop printing," "record log," and "ignore." "Stop printing" is a post-processing method intended for printed material with a high degree of defect severity. "Record log" is a post-processing method intended for printed material with a medium degree of defect severity. "Ignore" is a post-processing method intended for printed material with a low degree of defect severity.

[0039] Two thresholds, A and B, are used to determine the post-processing corresponding to these three types. For example, if the sensory value is greater than threshold A, "stop printing" is determined as the post-processing, if the sensory value is equal to or less than threshold A but greater than threshold B, "record log" is determined as the post-processing, and if the sensory value is equal to or less than threshold B, "record log" is determined as the post-processing.

[0040] In addition to these, the contents of post-processing may include, for example, "changing the discharge destination of the printed material" and "instructions for stamping, punching, etc." The "change of print destination" is provided, for example, to distinguish prints with highly severe defects from prints with minor or medium defects detected. "Instructions for stamping, punching, etc." are provided for the purpose of eliminating printed matter with serious defects.

[0041] <Inspection operation> The defect inspection operation will be described below with reference to FIGS. FIG. 6 is a flowchart illustrating a part of an example of an inspection operation for defects by the inspection device 320 (see FIG. 1) used in the first embodiment. FIG. 7 is a flowchart illustrating the remaining steps of the defect inspection operation example performed by the inspection device 320 used in the first embodiment. As described above, the processing operations shown in FIGS. 6 and 7 are realized through the execution of a program by the processor 321 (see FIG. 2).

[0042] When the processor 321 receives a new printed matter from the printing device 310 (see FIG. 1), it starts the processing operation shown in FIG. First, the processor 321 acquires a reference image (step 1), and then acquires an inspection image (step 2). If the same image is to be printed continuously, the reference image may be acquired only once.

[0043] Next, processor 321 extracts edges of the reference image (step 3), and then extracts edges of the inspection image (step 4). Processor 321 then aligns the reference image and the inspection image using the two edge images (step 5), although the alignment of the reference image and the inspection image may be performed before extracting the edge images.

[0044] Next, the processor 321 determines the brightness of the reference image for the pixel to be processed (step 6), and then determines a threshold value using the brightness of the reference image and information on the edge image (step 7). For example, the processor 321 determines a threshold value to be used for determining defects depending on the presence or absence of an edge, etc. Furthermore, the processor 321 calculates the difference in pixel value between the reference image and the inspection image for the pixel to be processed (step 8), and determines whether the calculated difference exceeds a threshold value (step 9). This process corresponds to the process of extracting pixels that are defect candidates.

[0045] After this, processor 321 determines whether or not the determination of all pixels has been completed (step 10). If a negative result is obtained in step 10, processor 321 returns to step 6 and repeats the processes of steps 6 to 9 described above for the remaining pixels that have not been inspected. If a positive result is obtained in step 10, processor 321 groups the defect candidates (step 11), and then obtains sensory values ​​corresponding to each combination of color information of the reference image and the inspection image at positions corresponding to the group (step 12).

[0046] After obtaining the sensory values, processor 321 determines whether or not processing of all groups has been completed (step 13). If there are any unprocessed groups remaining, processor 321 obtains a negative result in step 13. If a negative result is obtained in step 13, processor 321 returns to step 12 and executes the processing of step 12 for one of the unprocessed groups. If there are no more groups left to be processed, processor 321 obtains a positive result in step 13 .

[0047] If a positive result is obtained in step 13, the processor 321 determines the content of post-processing for the entire page (step 14). In other words, the content of post-processing for the printed material to be inspected is determined. After this, processor 321 outputs the result of the inspection (step 15). Processor 321 also executes post-processing of the determined content.

[0048] <Summary> In this embodiment, defects in printed matter are inspected based on the relationship between sensory values ​​that represent the results of how people evaluate color differences as defects for combinations of color information of a reference image and an inspection image corresponding to areas extracted as defect candidates. Therefore, the defect inspection performed by the inspection device 320 (see FIG. 1) can be closer to human sensitivity than when calculation is performed using a general formula defined by color difference.

[0049] <Embodiment 2> In this embodiment, another example of the inspection operation will be described. Therefore, other than the inspection operation, it is the same as in embodiment 1. That is, in this embodiment, the configuration of the image printing system 1 (see FIG. 1) and the hardware configuration and functional configuration of the inspection device 320 (see FIG. 1) are the same as in embodiment 1. 8 is a flowchart illustrating an example of an inspection operation for defects by the inspection device 320 used in the second embodiment, with reference to differences from the first embodiment. In Fig. 8, parts corresponding to those in Fig. 7 are assigned the same reference numerals.

[0050] In this embodiment, once the sensory values ​​for each group are acquired in step 12, processor 321 (see FIG. 2) determines the content of post-processing for the group to be processed (step 21). For example, processor 321 determines one of the following processes for each group: "stop printing," "record a log," or "ignore." After executing step 21, processor 321 determines whether or not processing of all groups has been completed (step 13).

[0051] In this embodiment, if a positive result is obtained in step 13, processor 321 determines the content of post-processing for the entire page of the printed material based on the content of post-processing for each group determined in step 21 (step 14A). For example, if there is one group determined to be "Stop printing," and there are multiple groups determined to be "Record log" and "Ignore," processor 321 determines "Stop printing," which is the post-processing for the most severe defect, as the post-processing content for the entire page. In other words, processor 321 prioritizes the post-processing content of the higher order as the post-processing content for the entire page. In the case of the inspection operation according to this embodiment, it is possible to obtain the same inspection results as in the first embodiment.

[0052] <Third Embodiment> Next, another example of the inspection operation will be described. Therefore, other than the inspection operation, it is the same as in embodiment 1. That is, in the case of this embodiment, the configuration of the image printing system 1 (see FIG. 1) and the hardware configuration and functional configuration of the inspection device 320 (see FIG. 1) are the same as in embodiment 1. 9 is a flowchart illustrating an example of an inspection operation for defects by the inspection device 320 used in the third embodiment, with reference to differences from the first embodiment. In Fig. 9, parts corresponding to those in Fig. 7 are assigned the same reference numerals.

[0053] In this embodiment, when defect candidates are grouped in step 11, processor 321 (see Figure 2) obtains sensory values ​​corresponding to the combination of each color information and each feature of the reference image and inspection image at the position corresponding to the group (step 31). The features here include, for example, the number of defect candidate pixels that make up the group and the degree of dispersion of the defect candidate pixels that make up the group. Incidentally, the features may also include the shape formed by the set of pixels that make up the group. The shape may be, for example, linear or elliptical.

[0054] The number of pixels in the defect candidate that make up the group affects the size of the defect candidate: the larger the number of pixels in the defect candidate that make up the group, the higher the possibility that it will be recognized as a defect. However, even if the number of defect candidate pixels that make up a group is large, if the difference in color information is difficult to perceive, it is not necessarily the case that a person will recognize them as defects.Also, even if the number of defect candidate pixels that make up a group is small, if the difference in color information is easy to perceive, there is a high possibility that a person will recognize them as defects.

[0055] The degree of dispersion of the candidate defect pixels that make up a group is the criterion for determining the range of pixels that are considered to be in one group. If the degree of dispersion is large, the range of candidate pixels included in one group will be wide. On the other hand, if the degree of dispersion is small, the range of candidate pixels included in one group will be narrow. In this embodiment, the sensory value LUT 327B stores a relationship linking the number of pixels of defect candidates that make up a group, the degree of dispersion of the pixels of defect candidates that make up a group, and the sensory values ​​obtained when the defect candidates are visually evaluated based on the combination of color information of the reference image and color information of the inspection image. The degree of dispersion here is what is known as "variance information."

[0056] Therefore, unlike in the first embodiment, the sensory value obtained in step 31 is given as a human sensory evaluation that also includes the characteristics of each group, which is a collection of pixels that are recognized as defect candidates as a whole. Other processing operations are the same as those in embodiment 1. That is, after acquiring the sensory values ​​for each group in step 31, processor 321 determines whether or not the judgment for all groups has been completed (step 13), and when the judgment for all groups has been completed, determines the content of post-processing for the entire page (step 14).

[0057] <Summary> In this embodiment, a sensory value is given to each group, which is a collection of pixels that are defect candidates recognized as a whole, as a result of visual sensory evaluation of defect candidates classified by a combination of color information and features. Human defect evaluation is a comprehensive evaluation that takes into account the size and variation of defect candidates. Moreover, in this embodiment, the sensory values, which are the results of visual sensory evaluation of samples that resemble actual defects, are linked to combinations of color information and features, thereby achieving defect inspection that is closer to human sensibilities.

[0058] <Fourth Embodiment> In this embodiment, an example of performing defect inspection based on a learning model that learns the relationship between the combination of color information and features of pixels extracted as defect candidates and the sensory values ​​obtained from visual sensory evaluation will be described. The configuration of the image printing system 1 (see FIG. 1) in this embodiment and the hardware configuration of the inspection device 320 (see FIG. 1) are the same as those in the first embodiment. FIG. 10 is a diagram illustrating a machine learning device 400 that learns the relationship between a combination of color information and features of pixels extracted as defect candidates and a sensory value obtained from a visual sensory evaluation.

[0059] The machine learning device 400 receives as input a combination of color information of a reference image at a position corresponding to a group of defect candidates, color information of an inspection image at a position corresponding to a group of defect candidates, the number of pixels constituting the group of defect candidates, and the degree of dispersion of the pixels of the defect candidates constituting the group, and is provided with training data that indicates a relationship in which the corresponding sensory value is output. For the machine learning here, for example, a convolutional neural network is used.

[0060] Convolutional neural networks are an example of supervised learning. A convolutional neural network consists of a convolutional layer that extracts local features, a pooling layer that further emphasizes the spatial features of the output of the convolutional layer, and a fully connected layer that combines the results of repeated iterations of a convolutional layer that takes the output of the pooling layer as input and a pooling layer that takes the output of the pooling layer as input.

[0061] By learning using a convolutional neural network, when a combination of color information of a reference image at a position corresponding to a group of defect candidates, color information of an inspection image at a position corresponding to a group of defect candidates, the number of pixels constituting the group of defect candidates, and the degree of dispersion of the pixels of the defect candidates constituting the group is given as input, a learning model 410 is generated that outputs a corresponding sensory value.

[0062] Fig. 11 is a diagram illustrating an example of the functional configuration of the inspection device 320 used in the fourth embodiment. In Fig. 11, parts corresponding to those in Fig. 5 are assigned the same reference numerals. The sensory value determination section 357 in the inspection device 320 shown in FIG. 11 has a learning model 410 instead of the sensory value LUT 327B (see FIG. 5). The sensory value determination unit 357 differs from embodiment 1 in that it provides the learning model 410 with color information of the reference image at a position corresponding to the group of defect candidates, color information of the inspection image at a position corresponding to the group of defect candidates, the number of pixels constituting the group of defect candidates, and the degree of dispersion of the pixels of the defect candidates constituting the group, and obtains the corresponding sensory value from the learning model 410.

[0063] <Summary> In the case of the method according to this embodiment, as in the first embodiment, it is possible to make the defect inspection performed by the inspection device 320 (see FIG. 1) closer to human sensibility than when calculations are made using a general formula defined by color difference.

[0064] <Fifth Embodiment> In the above-described embodiment, defects are inspected for all groups obtained by grouping pixels extracted as defect candidates, but the user may also specify on the screen the area of ​​the inspection image that he or she wishes to inspect. 12A and 12B are diagrams illustrating an example in which a user specifies an area to be inspected for defects. (A) shows an example of displaying the positions of groups G1 to G4, each consisting of pixels extracted as defect candidates, and (B) shows an example of group G specified by the user.

[0065] An operation screen 500A shown in FIG. 12(A) and an operation screen 500B shown in FIG. 12(B) are displayed on, for example, the display 324 (see FIG. 2). The operation screen 500A shown in FIG. 12(A) displays statements 510 prompting the user to perform an operation, such as "A defect candidate has been found" and "Do you want to specify the candidate you want to inspect?", an inspection image 520, a button 530 that the user operates if he or she wants to specify a candidate, and a button 540 that the user operates if he or she does not want to specify a candidate. The positions of the four groups G1 to G4 are indicated by dashed lines in the examination image 520. In the case of the operation screen 500A, the mouse cursor is positioned over the button 530.

[0066] Operation screen 500B shown in FIG. 12(B) is displayed when button 530 is clicked on operation screen 500A. The operation screen 500B displays a statement 550 saying "Please specify the candidate you wish to test," a button 560 to be operated when instructing the execution of the test once the candidate has been specified, a button 570 to be operated when canceling the execution of the test, and a statement 580 prompting the user to operate "Do you wish to test?" 12(B) shows a state in which group G1 is selected with the mouse cursor, and when button 560 is clicked in this state, defect inspection is executed.

[0067] <Sixth Embodiment> In the case of embodiment 5, the user specifies the group to be inspected from among the areas corresponding to the groups extracted as defect candidates by the inspection device 320 (see Figure 1), but in this embodiment, we will explain the case where the user specifies an arbitrary area. FIG. 13 is a diagram for explaining an example of inspecting defects in an arbitrary area portion designated by the user. The operation screen 600 shown in FIG. 13 is also displayed on, for example, the display 324 (see FIG. 2).

[0068] On the operation screen 600, a message 610 saying "Please specify the area you wish to inspect," a reference image 620, and an area portion 630 designated by the mouse cursor are displayed. When the user specifies an arbitrary area portion to be used for defect inspection, information specifying the specified area portion is directly provided to the sensory value determination unit 357 . In this case, the sensory value determination unit 357 outputs the sensory value for the region portion designated by the user using the method explained in the first embodiment and the like.

[0069] <Other embodiments> (1) Although the embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the scope of the above-described embodiments. It is clear from the claims that various modifications and improvements to the above-described embodiments are also included in the technical scope of the present invention.

[0070] (2) In the above-described embodiment, an image generated from print data is used as a reference image, but an optically read image may also be used as a reference image.

[0071] (3) In the above-described embodiment, the relationship between the color information of the reference image, the color information of the test image, and the sensory values ​​obtained from the visual sensory evaluation is described as the relationship between the combination of coordinate values ​​in the Lab color space and the sensory values, but other color spaces such as the L*a*b* color space, the HSV color space, or the HSL color space may also be used.

[0072] (4) In the above-described embodiment, the sensory value LUT 327B has been described as defining the relationship between a combination of color information of an inspection target area and a sensory value, or storing the relationship between a combination of color information and features of an inspection target area and a sensory value. However, the relationship used for inspection may be selected depending on the content of the print job. For example, white spaces are easily noticeable in areas filled with black, and in printed forms, there is a high need to detect stains that may be mistaken for decimal points. For this reason, the sensory value LUT 327B used for defect inspection may be selected depending on the content of the print job. Furthermore, when training the training model 410 (see FIG. 10), multiple training models 410 may be trained according to the content of the print job.

[0073] (5) The processor in each of the above-mentioned embodiments refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPUs, etc.) as well as dedicated processors (e.g., GPUs (Graphical Processing Units), ASICs (Application Specific Integrated Circuits), FPGAs (Field Programmable Gate Arrays), programmable logic devices, etc.). Furthermore, the operations of the processor in each of the above-described embodiments may be performed by a single processor alone, or may be performed by multiple processors located in physically separate locations in cooperation with each other. Furthermore, the order in which the operations of the processors are performed is not limited to the order described in each of the above-described embodiments, and may be changed individually. [Explanation of symbols]

[0074] 1...image printing system, 10...network, 20...printing data generation device, 30...printing system, 310...printing device, 320...inspection device, 327A...threshold value table, 327B...sensory value LUT, 327C...inspection log, 330...post-processing device, 351...reference image generation unit, 352...inspection image generation unit, 353...intra-image edge extraction unit, 354...difference image generation unit, 354A...alignment unit, 355...threshold value setting unit, 356...defect candidate extraction unit, 357...sensory value determination unit, 358...post-processing control unit, 400...machine learning device, 410...learning model

Claims

1. a printing device that prints an image corresponding to the print data on paper; a scanner for reading an image printed by the printing device; a sensory value determination unit that acquires a sensory value that indicates the degree of defect obtained from a visual sensory evaluation for a combination of first color information of the print data printed by the printing device and second color information of the image read by the scanner; An image printing system having:

2. a control means for controlling printing based on the sensory value acquired by the sensory value determination unit; The image printing system of claim 1 further comprising:

3. On the computer, a function of acquiring a sensory value representing the degree of defect obtained from a visual sensory evaluation for a combination of first color information of a reference image used as an inspection standard and second color information of an inspection image read from a printed material; A program to achieve this.

4. A function to control printing based on the acquired information, The program according to claim 3, further realizing the above.

5. A function of rejecting a printed matter when the severity of defects in the printed matter is high based on the magnitude of the acquired sensory value; The program according to claim 3, further realizing the above.

6. a function of providing the first color information, the second color information, and features of the inspection target area to a learning model and obtaining the corresponding sensory value from the learning model; The program according to claim 3, further realizing the above.

7. a processor; The processor: providing a learning model with first color information of a reference image used as a standard for inspection, second color information of an inspection image read from a printed material, and features of a target area for inspection; obtaining, from a learning model, a sensory value that indicates the degree of defect obtained from a visual sensory evaluation and corresponds to a combination of the first color information and the second color information, and inspecting the print for defects; Image printing system.

8. On the computer, providing a learning model with first color information of a reference image used as a standard for inspection, second color information of an inspection image read from a printed material, and features of a target area for inspection; a function of inspecting the printed matter for defects by obtaining, from a learning model, a sensory value that indicates the degree of defect obtained from a visual sensory evaluation corresponding to the combination of the first color information and the second color information; A program to achieve this.

Images