Image processing apparatus, printing system, image processing method, and program

Abstract

To provide an image processing apparatus that improves accuracy of inspecting a printed material even when print density has changed over time.SOLUTION: An image processing apparatus is provided which inspects a printed material printed by a printer. The image processing apparatus corrects a first difference between density of a partial area in a print image obtained by scanning a printed material to be inspected and density of an area corresponding to the partial area in a reference image obtained by scanning a printed material being an inspection standard, by using a second difference between density of an area including the partial area in the print image and density of an area corresponding to the including area in the reference image. The image processing apparatus has determination means that determines whether a defective area is included in the print image on the basis of the corrected first difference.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing device, a printing system, an image processing method, and a program. [Background technology]

[0002] Conventionally, inspections using image processing have been performed to check the quality of printed matter. There is a technology that automatically inspects printed matter by comparing a reference image of a print that is good (meets inspection standards) with the print image data of the print to be inspected. Electrophotographic printing devices may produce prints with different densities or colors depending on the state of the printing device (e.g., the amount of remaining toner), even when printing from the same input data.

[0003] In light of the above problems, Patent Document 1 proposes a technology for inspecting printed materials by comparing a master image generated to inspect defects in the printed image when the printing device is calibrated with the printed image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-178970 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in Patent Document 1, a master image is generated only when the printing device is calibrated, and therefore defects in printed matter that occur due to changes in print density over time cannot be detected.

[0006] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide an image processing apparatus that improves the inspection accuracy of printed matter even when the print density changes over time. [Means for solving the problem]

[0007] In order to achieve the object of the present invention, an image processing device according to one embodiment of the present invention has the following configuration: That is, the image processing device inspects printed materials printed by a printing device, and includes: a correction unit that corrects a first difference between the density of a partial area in a printed image obtained by scanning a printed material to be inspected and the density of an area corresponding to the partial area in a reference image obtained by scanning a printed material that serves as an inspection standard, using a second difference between the density of an area in the printed image that includes the partial area and the density of an area in the reference image that corresponds to the included area; and a determination unit that determines whether the printed image includes a defective area based on the corrected first difference. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide an image processing apparatus that improves the inspection accuracy of printed matter even when the print density changes over time. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of the hardware configuration of a printing system according to a first embodiment. [Figure 2] FIG. 1 is a block diagram showing an example of the functional configuration of an image processing apparatus according to a first embodiment. [Figure 3] 5 is a flowchart illustrating an example of image processing according to the first embodiment. [Figure 4] 5 is a flowchart illustrating a process for generating differential data according to the first embodiment. [Figure 5] 3A and 3B are diagrams for explaining the relationship between a pixel of interest and a surrounding area according to the first embodiment. [Figure 6] 5A and 5B are diagrams showing an example of changes in print density over time that occur in a printed image according to the first embodiment. [Figure 7] 3A and 3B are views showing a reference image and a printed image according to the first embodiment. [Figure 8] FIG. 10 is a block diagram showing an example of the functional configuration of an image processing apparatus according to a second embodiment. [Figure 9]10 is a flowchart showing an example of image processing according to the second embodiment. [Figure 10] 10 is a flowchart illustrating a process for determining a global density difference according to the second embodiment. [Figure 11] A diagram explaining the user interface (UI). DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the claimed invention. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0011] (First embodiment) The image processing device calculates a first difference between the density of a partial area in a printed image obtained by scanning a printed material to be inspected and the density of an area corresponding to the partial area in a reference image obtained by scanning a printed material that serves as an inspection standard. The image processing device corrects the first difference using a second difference between the density of an area in the printed image that contains the partial area and the density of an area in the reference image that corresponds to the contained area. The image processing device determines whether the printed image contains a defective area based on the corrected first difference.

[0012] The image processing device evaluates the quality of the print to be inspected by comparing pixel values ​​of a reference image that indicates the target print output with the printed image. To do this, the image processing device corrects the pixel values ​​of the reference image based on the global density difference between the reference image and the printed image. The global density difference refers to the density difference of a given toner color relative to the reference image that occurs uniformly over time across the entire printed image.

[0013] The image processing device corrects pixel values ​​of the reference image, for example, using average pixel values ​​of local regions of the reference image and the printed image. The image processing device determines whether the printed image contains a defective region based on difference data between the pixel values ​​of the corrected reference image and the pixel values ​​of the printed image. This allows the image processing device to inspect printed materials with high accuracy while reducing the effects of global density differences caused by changes in print density over time.

[0014] However, if the global density difference becomes too large, color shift occurs in the printed image, and the image processing device determines that the printed matter does not meet the inspection standard. Therefore, the image processing device does not correct pixel values ​​of the reference image when the global density difference exceeds a predetermined threshold.

[0015] 6 is a diagram showing an example of a change in print density over time that occurs in a printed image according to the first embodiment, where the horizontal axis represents time and the vertical axis represents print density in the printed image.

[0016] Global density difference refers to the density difference of a given toner color relative to a reference image that occurs uniformly over time across the entire printed image, as shown in Figure 6. An electrophotographic printing device 190 may print prints with different densities or colors even with the same input data, depending on the status of the printing device 190 (for example, the amount of remaining toner). In Figure 6, the print density decreases between time t0 and t1. On the other hand, toner is replenished to the printing device at time t2, causing the print density to increase.

[0017] FIG. 1 is a diagram illustrating an example of the hardware configuration of a printing system according to the first embodiment.

[0018] The printing system 10 includes an image processing device 100, a printing server 180, and a printing device 190.

[0019] The printing server 180 generates a print job and submits the print job to the printing device 190 .

[0020] The printing device 190 is a device that forms an image on a recording medium, such as an electrophotographic copier or printer. The printing device 190 forms an image on a recording medium supplied from a paper supply unit 191 based on a print job. The recording medium is paper (for example, printing paper), but is not limited to this. A user sets printing paper in advance in the paper supply unit 191 of the printing device 190. When a print job is input, the printing device 190 forms an image on one or both sides of the printing paper, and transports the printing paper along a transport path 192 to the image processing device 100.

[0021] The image processing device 100 includes a CPU 101, a RAM 102, a ROM 103, a main memory device 104, a reading device 105, a printing device I / F 106, a general-purpose I / F 107, a UI panel 108, and a main bus 109. The image processing device 100 also includes a conveying path 110 for printed materials connected to a conveying path 192, an output tray 111 for printed materials that have passed inspection, and an output tray 112 for printed materials that have failed inspection.

[0022] Based on the reference image, the image processing device 100 inspects the paper (i.e., the printed matter) transported via the transport path 192 and the transport path 110. The image processing device 100 transports the printed matter to an output tray according to the inspection result of the printed matter.

[0023] The CPU 101 is a processor that controls all the components of the image processing device 100 .

[0024] The RAM 102 functions as the main memory and work area for the CPU 101 .

[0025] The ROM 103 stores the programs executed by the CPU 101 .

[0026] The main memory device 104 stores applications executed by the CPU 101 and data used for image processing.

[0027] The reading device 105 is a device that reads printed matter, such as a scanner. The reading device 105 reads one or both sides of the printed matter sent from the printing device on the conveying path 110, thereby obtaining a reference image or a printed image.

[0028] The printing device I / F 106 is connected to the printing device 190. The printing device I / F 106 acquires operation information of the printing device 190 by synchronizing the processing timing of the printing device 190 with that of the printing device 190.

[0029] The general-purpose I / F 107 is a serial bus interface such as USB or IEEE1394, and is used, for example, by the user to retrieve log data.

[0030] The UI panel 108 is a user interface screen, such as a liquid crystal display, that displays various data. The UI panel 108 displays the status and settings of the image processing device 100. The UI panel 108 also includes a touch panel and input buttons for receiving instructions from the user.

[0031] The main bus 109 connects each unit of the image processing device 100. This allows the CPU 101 to control each unit of the image processing device 100 or the printing system 10. For example, the CPU 101 controls the synchronization of each transport path, and transports the printed matter to the output tray 111 or the output tray 112 depending on the inspection result of the printed matter.

[0032] FIG. 2 is a diagram illustrating an example of the functional configuration of the image processing apparatus according to the first embodiment.

[0033] The image processing device 100 includes an input terminal 201 , a reading unit 202 , a reference data holding unit 203 , a print data holding unit 204 , a generating unit 205 , an inspecting unit 206 , a managing unit 207 , and an output terminal 208 .

[0034] The input terminal 201 receives a control signal that is input in synchronization with the output of printed matter from the printing device 190 or as needed.

[0035] When a control signal is input to the input terminal 201, the reading unit 202 acquires an image of the printed material on the conveying path 110. The image read by the reading device 105 is classified as a reference image or a printed image.

[0036] The reference data storage unit 203 stores the reference image.

[0037] The print data storage unit 204 stores print images.

[0038] The generating unit 205 generates difference data based on the density (pixel value) of an area (pixel) in the print image and the density (pixel value) of an area (pixel) in the reference image that corresponds to the area of ​​the print image. This difference data corresponds to first difference data.

[0039] The inspection unit 206 determines whether the printed image contains a defective area based on the difference data. Next, the inspection unit 206 outputs a control signal to an internal driving unit of the printing system 10 via the output terminal 208 based on the determination result of the printed image.

[0040] The management unit 207 exchanges information with each unit of the image processing device 100. The management unit 207 collects information on the number of images currently being processed and whether or not an error has occurred. The management unit 207 outputs the collected information as a log as necessary and outputs a control signal to the printing system 10.

[0041] The output terminal 208 outputs a control signal for operating the printing system 10 based on the results of the inspection of the printed matter by the inspection unit 206 .

[0042] 3 is a flowchart illustrating an example of image processing according to the first embodiment. The CPU 101 reads out from the ROM 103 a program in accordance with the flowchart shown in FIG.

[0043] In S301, the image processing device 100 acquires a reference image from the reference data storage unit 203. The reference image will be described below. First, the user prints a predetermined number of printed materials using the printing device 190. The user selects printed materials that are free of defects (meet the inspection criteria) from the printed materials. Next, the reading unit 202 acquires a reference image by reading the selected printed materials. The reference data storage unit 203 stores the reference image. The reference image is expressed in RGB 8-bit format (expressed as a three-dimensional vector array having each RGB channel as an element). In order to acquire the reference image, the user needs to visually inspect the printed materials. Note that in the processes from S301 onwards, the image processing device 100 inspects the printed materials, so the user does not need to visually inspect the printed materials.

[0044] In S302, the management unit 207 sets a count (count of print products) indicating the number of print products that have passed the inspection to 1. The count is used in the processing loop from S303 onwards, which will be described later, until the number of print products that have passed the inspection reaches a predetermined number.

[0045] In the processes of S303 to S311, the image processing device 100 classifies the printed matter printed by the printing device 190 into printed matter that passed the inspection and printed matter that failed. Here, the image processing device 100 repeats the processes of S303 to S311 for the printed matter printed by the printing device 190 until the number of printed matters that passed the inspection reaches a predetermined number.

[0046] In S304, the reading unit 202 acquires a print image by reading a printout printed by the printing device 190. The print image is stored in the print data storage unit 204. The print image is represented in RGB 8-bit format, similar to the reference image.

[0047] In S305, the generation unit 205 generates difference data D based on the difference between the density of a partial region of the reference image and the density of a partial region of the print image corresponding to the partial region of the reference image. The difference data D will be described below. First, the generation unit 205 corrects pixel values ​​of a region of the reference image based on the global density difference between the reference image and the print image. Next, the generation unit 205 obtains difference data D based on the difference between pixel values ​​of the corrected region of the reference image and the region of the print image. Here, the difference data D is an image having the same size as the reference image and the print image.

[0048] In S306, the inspection unit 206 performs processing to detect defective areas in the printed image based on the difference data D. The difference data D stores, for each pixel, a value indicating the difference between the density of the area in the corrected reference image and the density of the area in the printed image. Therefore, if the difference data D contains a pixel (area) that satisfies a predetermined condition, the inspection unit 206 detects that area as an area indicating a defect in the printed matter (defective area).

[0049] A defective area on a print is, for example, an area where the area including pixels whose difference values ​​exceed a threshold exceeds a predetermined area, or an area that forms a shape such as a streak. The inspection unit 206 detects defective areas on the print image by filtering the difference data D or by calculating the sum of pixel values ​​in rows or columns corresponding to the streak direction.

[0050] The inspection unit 206 determines that the printed matter is unacceptable if it detects a defective area in the printed image based on the difference data D. On the other hand, the inspection unit 206 determines that the printed matter is acceptable if the pixel value in the difference data D is 0 or a sufficiently small value (i.e., there is no difference in pixel value between the area of ​​the reference image and the area of ​​the printed image).

[0051] In S307, the inspection unit 206 determines the inspection result for the printed matter based on whether or not a defective area in the printed image has been detected based on the difference data D. If the inspection unit 206 has not detected a defective area in the printed image (No in S307), it determines that the printed matter is acceptable, and the process proceeds to S308. If the inspection unit 206 has detected a defective area in the printed image (Yes in S307), it determines that the printed matter is unacceptable, and the process proceeds to S310.

[0052] In S308, the inspection unit 206 outputs a control signal to the printing system 10 to transport the printed matter to the output tray 111.

[0053] In S309, the management unit 207 increments the count of the print products by 1, and the process proceeds to S311. A print product is a print product that has passed the inspection.

[0054] In S310, the inspection unit 206 outputs a control signal to the printing system 10 to transport the printed matter to the output tray 112, and the process proceeds to S311. The image processing device 100 is equipped with a plurality of output trays 112 corresponding to the degree of defective areas in the printed image. The image processing device 100 transports printed matter that has been determined to be unacceptable based on the degree of defective areas in the printed image to the respective output trays 112. The degree of defective areas in the printed image is, for example, the size and number of defective areas.

[0055] In S311, the inspection unit 206 determines whether the count of the print products has reached a predetermined number. If the inspection unit 206 determines that the count of the print products has reached the predetermined number, the processing ends. If the inspection unit 206 determines that the count of the print products has not reached the predetermined number, the processing returns to S303.

[0056] FIG. 4 is a flowchart illustrating a differential data generation process according to the first embodiment.

[0057] The print image P and the reference image R are images of the same size and have no positional or rotational misalignment during scanning. The position of pixel P(x,y) in the print image P corresponds to the position of pixel R(x,y) in the reference image R.

[0058] In S401, the generation unit 205 sets pixel P(x, y) of the print image P as a pixel of interest and repeats the processes of S401 to S409.

[0059] In S402, the generation unit 205 calculates the average pixel value mp for each RGB channel in the surrounding area (neighborhood area) of the pixel of interest P(x, y). Here, mp is a three-dimensional vector. Each pixel corresponds to one of the RGB channels.

[0060] Here, FIG. 5 is a diagram for explaining the relationship between the pixel of interest and the surrounding area.

[0061] In FIG. 5, pixel 501 is a pixel of interest P(x, y). Region 502 is a 7×7 region centered on pixel 501, and is a region for acquiring an average pixel value mp (average value acquisition region). Note that the average value acquisition region has a size of 7×7, but it may be a region of a predetermined size. Furthermore, although the average pixel value is used in this embodiment, this is not limiting, and a median pixel value, a most frequent pixel value, etc. may also be used.

[0062] In S403, similar to the processing in S402, the generation unit 205 calculates the average pixel value mr for each RGB channel in the surrounding area of ​​the pixel of interest R(x, y) of the reference image R. Like mp, mr is a three-dimensional vector corresponding to each RGB channel.

[0063] In S404, the generation unit 205 calculates d0, which is a candidate pixel value for pixel (x, y) in the difference data D, according to the following (Equation 1). d0 is the difference between pixel values ​​at the same position in the print image and the reference image. d0 is also a three-dimensional vector corresponding to each RGB channel. d0=P(x,y)-R(x,y) (Equation 1)

[0064] In S405, the generation unit 205 calculates d1, which is a candidate for a pixel value different from d0 at the pixel (x, y) of the difference data D, according to the following formula (2). d1 is the difference between the pixel value of the region of the reference image corrected by the difference (mp - mr) between the average pixel value mp of the printed image P and the average pixel value mr of the reference image R, and the pixel value of the region of the printed image. That is, the average pixel value mp of the peripheral region of the target pixel P(x, y) and the average pixel value represented by R(x, y)+(mp - mr) match. Therefore, it becomes possible to take the difference while canceling out the global density difference between the printed image P and the reference image R. d1 is a three-dimensional vector corresponding to each channel of RGB, similar to d0. Also, d1 includes at least one statistical value such as the least square error of RGB values, the difference in luminance values, and the color difference in Lab values.

[0065] TIFF2023169761000002.tif15140 (Formula 2)

[0066] Here, th is a predefined threshold value. When the absolute value of the difference (mp - mr) of the average pixel value (average density) is smaller than the threshold value (mp - mr < th), the variation by the average density is corrected. When the absolute value of the difference (mp - mr) of the average density is larger than the threshold value (mp - mr > th), since it is corrected by the threshold value instead of the difference of the average density, the value of d1 becomes larger.

[0067] In S406, the generation unit 205 calculates the squared norms of d0 and d1 respectively, and determines which of the squared norms of d0 and d1 is smaller. When the generation unit 205 determines that the squared norm of d0 is smaller than the squared norm of d1 (Yes in S206), the process proceeds to S407. When the generation unit 205 determines that the squared norm of d0 is not smaller than the squared norm of d1 (No in S206), the process proceeds to S408.

[0068] In S407, the generation unit 205 determines the squared norm of d0 as the value of the pixel (x, y) of the difference data D, and the process proceeds to S409.

[0069] In S408, the generation unit 205 determines the square norm of d1 as the value of the pixel (x, y) of the difference data D, and the process proceeds to S409.

[0070] In S409, the generation unit 205 determines whether the processes of S401 to S408 have been performed for all pixel positions of the print image P and the reference image R. If the generation unit 205 determines that the processes have been performed for all pixel positions, the process ends. If the generation unit 205 determines that the processes have not been performed for all pixel positions, the process returns to S401.

[0071] By performing the processes of S401 to S409 for all pixel positions of the print image P and the reference image R, all pixel values ​​of the difference data D are determined, and therefore the difference data D (difference image) is obtained. The difference data D has the same size and number of channels as the print image P and the reference image R. The pixel (x, y) of the difference data D corresponds to the position of the pixel (x, y) of each of the print image P and the reference image R.

[0072] The generation unit 205 corrects the image values ​​of the reference image R based on the global density difference between the reference image R and the print image P. Then, the generation unit 205 generates difference data D based on the difference in image values ​​between the corrected reference image R and the print image P. The reason for correcting the density of the region of the reference image R will be explained below.

[0073] 6 again, the print density decreases between time t0 and time t2, when the printing device 190 starts printing the printed material. On the other hand, the print density increases after time t2 because the printing device 190 is replenished with toner. Generally, fluctuations in print density occur across the entire printed image.

[0074] If the print density variation in the printed image caused by the output characteristics of the printing device 190 is excessive, the user will visually perceive the print density variation in the printed matter. In other words, such a print matter will have areas of poor printing (defects). However, unlike when print density differences exist only in localized areas of the printed image, if the density difference between the reference image and the printed image occurs uniformly across the entire printed image, the visual density difference is less noticeable. In this case, the density difference occurring in the printed image may be acceptable.

[0075] Printed matter is inspected to ensure that the printed matter has a certain level of quality. However, if the inspection standards for printed matter are made stricter, the number of rejected printed matter increases. This may have a negative impact on productivity, delivery time, and costs of printed matter. Therefore, the inspection standards for printed matter are set appropriately according to the quality required by the user.

[0076] Below, we will explain density differences that are not visually noticeable in printed images. Even if the printing device 190 prints a printed product at time t1 in Figure 6 based on the same input data as at time t0, the print density will be shifted by width 601. The difference in print density (width 601) appears as a difference in pixel values ​​in the difference data D generated by the generation unit 205.

[0077] When detecting defective areas in the printed image based on the difference data D in S306, using d0 (by performing inspection without considering acceptable density differences) can reduce the inspection accuracy of the printed material and result in erroneous detection of defective areas. Therefore, to generate difference data D taking into account uniform density differences that occur throughout the printed image, the inspection unit 206 corrects the density of the reference image region based on the distribution of pixel values ​​in local regions of the printed image. That is, when mp≧mr and th≧mp-mr, the inspection unit 206 corrects d1=P(x,y)-R(x,y)-(mp-mr). d1 is the difference between the density of the reference image region and the density of the printed image region minus the difference in average pixel values ​​(mp-mr). Alternatively, it can be interpreted that the difference is calculated after matching the average pixel values ​​of the printed image P and the reference image R based on d1=(P(x,y)-mp)-(R(x,y)-mr). By calculating d1, the inspection unit 206 can remove the allowable density difference from the difference data D, thereby reducing the influence of global density differences and inspecting the printed matter.

[0078] However, if the global density difference is too large, printed matter that is significantly different in color from printed matter that meets the inspection criteria will pass the inspection, and so the global density difference may not be acceptable. Therefore, by setting a threshold for the amount of correction for density fluctuation using the term min{th,(mp-mr)}, color fluctuations beyond a certain level are not tolerated.

[0079] FIG. 7 is a diagram showing a reference image and a print image according to the first embodiment.

[0080] A reference image 701 includes a pixel of interest 702 and an average value acquisition area 703. A print image 704 includes a pixel of interest 705 and an average value acquisition area 706.

[0081] The printed image 704 has a defective area 707 (shaded area) whose pixel values ​​are significantly different from those of the reference image 701. Here, the pixel value of the pixel of interest in the area excluding the defective area 707 of the printed image 704 is the same as the pixel value of the pixel of interest in the reference image 701. In other words, the pixel values ​​of the pixel of interest 702 and the pixel of interest 705 are the same, so d0 is 0 for the pixel of interest (x, y) in the difference data D. However, mr and mp used to calculate d1 have different values ​​due to the influence of the defective area 707 present in the average value acquisition area 706.

[0082] In other words, the calculation formula for d1 adjusts pixel values ​​to cancel the difference between mr and mp. Therefore, even though there is no difference in pixel values ​​between the area of ​​the reference image 701 and the area other than the defective area 707 of the printed image 704, the difference between the target pixel 702 and the target pixel 705 includes the difference of the defective area 707.

[0083] When d1 is used in the periphery of the area where the defective area 707 and the average value acquisition area 706 overlap, the image value is recorded in the target pixel D(x, y) in the difference data D so that the defective area 707 appears blurred and expanded.

[0084] Therefore, when generating the differential data D, the generation unit 205 uses, for each pixel, the smaller difference between the squared norm of d1 and the squared norm of d0 as the pixel value of the area indicated by the differential data D. This reduces the influence of global density differences that are visually less noticeable by adjusting the average pixel value, and prevents the pixel values ​​of the defective area 707 from being included in the density of the area indicated by the differential data D. This improves the accuracy of generating the differential data D, thereby improving the inspection accuracy and productivity of printed materials.

[0085] Furthermore, although the generation unit 205 generates difference data based on the density of an area of ​​the reference image and the density of an area of ​​the print image, this is not limiting. The generation unit 205 may generate partial difference data based on the density of a portion of an area of ​​the reference image and the density of a portion of an area of ​​the print image. The inspection unit 206 may inspect the print image based on the partial difference data.

[0086] Furthermore, in the image processing device 100, the generation unit 205 may calculate mr in S403 each time inspection processing is performed on a new print image, or may reuse mr used in previous processing. For example, if the reference image is not changed for a certain period after its creation, the reference data holding unit 203 stores mr calculated from the reference image used in previous processing. The generation unit 205 reads out the stored mr when processing each print image.

[0087] Furthermore, the print image P and the reference image R are images of the same size and do not have positional or rotational misalignment during scanning. However, in reality, positional or rotational misalignment may occur in the print image P when a printed material is scanned. Therefore, the image processing device 100 performs correction on the print image taking into account the positional or rotational misalignment during scanning.

[0088] For example, the image processing device 100 performs feature point extraction processing on the reference image and the print image when the print image is acquired (between S304 and S305) to acquire corresponding points (coordinates) in both images. Next, the image processing device 100 corrects for positional deviations during scanning using a transformation formula (e.g., affine transformation) that matches the feature points of both images. Examples of feature point extraction processing include SIFT, SURF, ORB, and AKAZE.

[0089] Furthermore, in S404, the generation unit 205 may calculate, for the pixel of interest P(x,y), not only the difference from the pixel of interest R(x,y) of the reference image, but also the difference from pixels in a region surrounding the pixel of interest R(x,y). In this case, the generation unit 205 may select a pixel from the region surrounding the pixel of interest R(x,y) that has the smallest difference from the pixel of interest P(x,y), and obtain the difference from the selected pixel as d0. In other words, the generation unit 205 may calculate, as corresponding points (coordinates) between the images, a feature point in the region surrounding the pixel of interest R(x,y) that has the smallest density difference from the pixel of interest P(x,y).

[0090] Furthermore, because there is a large change in pixel values ​​at the edge portions of the reference image R and the print image P, even a slight deviation (for example, about one pixel) that has little visual impact is recorded as a large difference in the area indicated by the difference data D. Therefore, in the process of S407 or S408 that determines the pixel (x, y) of the difference data D, the generation unit 205 may correct the density (pixel value) of the area indicated by the difference data by multiplying d0 or d1 by a weighting coefficient that takes into account the edge portions of both images.

[0091] Specifically, the edge components of the reference image R and the print image P are obtained in advance using an edge extraction filter. Then, the value of the pixel (x, y) of the difference data D may be determined by multiplying d0 or d1 by a weighting coefficient that reduces the difference in the edge region. The weighting coefficient may be a coefficient that corresponds to the edge component. Furthermore, the frequency characteristics of each region of the reference image R and the print image P may be obtained, and the weighting coefficient may be a weighting coefficient that reduces the pixel values ​​of the difference data D in the high frequency region. This makes it possible to prevent large difference values ​​from being recorded in the regions (pixels) of the difference data D.

[0092] (Second embodiment) In the second embodiment, a global density difference between a reference image and a printed image is detected, and the reference image is updated (replaced) with the printed image based on the detected density difference. The second embodiment will be described with respect to the differences from the first embodiment.

[0093] FIG. 8 is a diagram illustrating an example of the functional configuration of an image processing apparatus according to the second embodiment.

[0094] The image processing device 100 includes a generating unit 801 , a determining unit 802 , and an updating unit 803 .

[0095] The generating unit 801 uses d0 instead of d1 to generate difference data D based on the difference between the density of the region of the reference image and the density of the region of the print image. The generating unit 801 acquires information for determining whether there is a global density difference between the reference image and the print image.

[0096] The determining unit 802 determines whether or not there is a global density difference between the reference image and the print image based on the information acquired by the generating unit 801.

[0097] If the determining unit 802 determines that there is a global density difference, the updating unit 803 corrects the reference image.

[0098] Fig. 9 is a flowchart showing an example of image processing according to the second embodiment. In Fig. 9, S901 is performed instead of S305, and the processes of S902 and S903 are executed after S309. Note that a description of the same processes as in Fig. 3 will be omitted.

[0099] In S901, the generation unit 801 generates difference data D using d0 based on the difference between the reference image acquired in S301 and the print image acquired in S304. The generation unit 801 also acquires information for determining whether a global density difference occurs between the reference image and the print image.

[0100] The determining unit 802 determines whether or not a global density difference occurs between the reference image and the print image based on the information acquired by the generating unit 801.

[0101] 10 is a flowchart illustrating a process for determining a global density difference according to the second embodiment. The print image P and the reference image R are images of the same size, and do not have a positional or rotational shift during scanning. The positions of pixel (x, y) of the print image P, pixel (x, y) of the reference image R, and pixel (x, y) of the difference data D correspond to each other.

[0102] In S1001, the generation unit 801 initializes a variable Dev to 0. The variable Dev represents the number of pixels in the difference data whose pixel value difference between the print image and the reference image satisfies a predetermined condition. The variable Dev represents the number of pixels (areas) whose pixel value difference between the print image and the reference image falls outside a predetermined range when converted to grayscale, but is not limited to this.

[0103] In S1002, the generation unit 801 sets pixel P(x, y) of the print image as a pixel of interest and performs the processes of S1002 to S1008 for all pixels of the print image.

[0104] In S1003, the generation unit 801 calculates d0 as the difference in pixel value between a pixel P(x, y) of the print image P and a pixel R(x, y) of the reference image R.

[0105] In S1004, the generation unit 801 determines the pixel value of the pixel (x, y) of the difference data D as the value of d0 calculated in S1003.

[0106] In S1005 to S1008, the generation unit 801 acquires information necessary to determine whether a global density difference occurs between the print image P and the reference image R.

[0107] In S1005, the generation unit 801 grayscales (converts to one dimension) the pixel values ​​of pixel P(x,y) and pixel R(x,y). Next, the generation unit 801 calculates the difference in pixel value between the grayscaled pixel P(x,y) and pixel R(x,y) as a variable dL. In FIG. 10, the grayscale calculation is shown as gray(P(x,y)) and gray(R(x,y)). The grayscale processing (tone conversion processing) of pixel values ​​can be performed using any method.

[0108] For example, the generation unit 801 calculates gray(x,y) using the linear sum with weighting coefficients shown in (Equation 3) with the values ​​of the RGB channels of pixel (x,y) as r(x,y), g(x,y), and b(x,y), respectively. Gray(x,y) represents the pixel value after grayscaling processing. gray(x,y)=0.21×r(x,y)+0.72×g(x,y)+0.072×b(x,y) (Equation 3)

[0109] The generating unit 801 also calculates gray(x, y) using a conversion function that converts luminance, brightness, or density into grayscale from the color space of the image acquired by the reading unit 202. Furthermore, the generating unit 801 may acquire an LUT (lookup table) that defines the characteristics of the conversion function, and acquire gray(x, y) by LUT processing.

[0110] In S1006, the generation unit 801 uses the variable dL, Th1 as a predefined first threshold, and Th2 as a predefined second threshold. The generation unit 801 determines whether dL is equal to or greater than Th1 and equal to or less than Th2 (equal to or greater than the first threshold and equal to or less than the second threshold (0≦Th1≦Th2)). If the generation unit 801 determines that dL is equal to or greater than Th1 and equal to or less than Th2 (Yes in S1006), the process proceeds to S1008. If the generation unit 801 determines that dL is not equal to or greater than Th1 and equal to or less than Th2 (No in S1006), the process proceeds to S1007.

[0111] In S1007, the generation unit 801 increments the value of the variable Dev by 1, and the process proceeds to S1008.

[0112] In S1008, the generation unit 801 determines whether the processes of S1002 to S1007 have been performed for all pixel positions of the print image P and the reference image R. If the generation unit 801 determines that the processes of S1002 to S1007 have been performed for all pixel positions of the print image P and the reference image R (Yes in S1008), the process proceeds to S1009. If the generation unit 801 determines that the processes of S1002 to S1007 have not been performed for all pixel positions of the print image P and the reference image R (No in S1008), the process returns to S1002.

[0113] In S1009, the determination unit 802 determines whether the value of the variable Dev is greater than or equal to 1. If the determination unit 802 determines that the value of the variable Dev is greater than or equal to 1 (Yes in S1009), the process proceeds to S1010. If the determination unit 802 determines that the value of the variable Dev is not greater than or equal to 1 (No in S1009), the process proceeds to S1011.

[0114] In S1010, the determination unit 802 determines that there is no global density difference between the print image P and the reference image R, and the process proceeds to S306.

[0115] In S1011, the determination unit 802 determines that there is a global density difference between the print image P and the reference image R, and the process proceeds to S306.

[0116] In S306, the inspection unit 206 detects defective areas in the print image based on the differential data D. The differential data D is an image acquired using only d0. However, in the process of S306, differential image data acquired using d1 and d0 may be used, similar to the differential data D used in the first embodiment.

[0117] In S902, the update unit 803 refers to the density difference determination results performed in S1009 to S1011 and determines whether or not a global density difference has been detected between the print image P and the reference image R. If the update unit 803 determines that a global density difference has been detected between the print image P and the reference image R (Yes in S902), the process proceeds to S310. If the update unit 803 determines that a global density difference has not been detected between the print image P and the reference image R (No in S902), the process proceeds to S308.

[0118] In S903, the update unit 803 updates (replaces) the reference data in the reference data storage unit 203 with the print image acquired in S304, and the process proceeds to S311.

[0119] In S311, the image processing apparatus 100 returns the process to S303 and repeats the process for the next printed matter. In particular, if the reference image is updated in S903, the updated reference image is used as the reference image to be used in the next and subsequent processes.

[0120] According to the second embodiment, it is determined whether a global density difference exists between the printed image and the reference image based on the difference in pixel values ​​between the printed image and the reference image and a predetermined threshold value, and the reference image is updated based on the determination result.

[0121] (Example of user notification) It has been explained that prints are inspected while adaptively adjusting density to accommodate acceptable density variations that occur in the printed image due to changes in print density over time. On the other hand, if the density variations in the printed image are large, the print will fail the inspection. In this case, there is a possibility that some abnormality has occurred in the output of the printing device 190. Furthermore, there is a possibility that the printing device 190 will continuously or frequently output defective prints. If a printing device 190 with some abnormality in the output continues to print, it will result in a decrease in productivity, so the user needs to investigate the cause of the density variations in the prints.

[0122] Therefore, when the detected density variation is greater than a predetermined threshold, the image processing device 100 notifies (displays) the status of the printing device 190 as an external device, for example, on the UI panel 108. This allows the user to determine whether to calibrate the printing device 190 or whether to continue printing printed materials using the printing device 190. Furthermore, the image processing device 100 may pause the inspection process that is being performed when notifying the UI panel 108 of the status of the printing device 190.

[0123] As described above, when a large density difference is detected from a printed image, the occurrence of printed matter containing printing defects can be reduced by promptly notifying the user of the state of the printing device.

[0124] (User Interface) FIG. 11 is a diagram illustrating a user interface (UI).

[0125] As shown in FIG. 11(a), the user operates a slide bar 1101 to set a tolerance (threshold) for the global density difference (density fluctuation) between the print image and the reference image.

[0126] As shown in FIG. 11(b), three thresholds (threshold 1102, threshold 1103, and threshold 1104) are set in advance. Threshold 1104 (shown as "small") is the smallest of the three thresholds. Threshold 1103 (shown as "medium") is the middle of the three thresholds. Threshold 1102 (shown as "large") is the largest of the three thresholds. The user can set any threshold by selecting one of thresholds 1102 to 1104, which are represented by radio buttons. In FIG. 11(b), threshold 1103 is selected as the density fluctuation tolerance. However, the user interface is not limited to the above and may have a format that allows the user to set a desired threshold.

[0127] (Other Examples) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0128] The disclosure of this specification includes the following image processing device, printing system, image processing method, and program. (Item 1) An image processing device that inspects a printed matter printed by a printing device, a correction means for correcting a first difference between the density of a partial area in a printed image obtained by scanning a printed matter to be inspected and the density of an area corresponding to the partial area in a reference image obtained by scanning a printed matter that is an inspection standard, using a second difference between the density of an area in the printed image that includes the partial area and the density of an area in the reference image that corresponds to the included area; and a determination unit that determines whether or not the print image includes a defective area based on the corrected first difference. Image processing device. (Item 2) the determining means determines whether the print image includes a defective area based on whether the difference in density of the partial area indicated by the corrected first difference exceeds a threshold value. Item 1. An image processing device according to item 1. (Item 3) the correction means selects, based on a comparison between the first difference and a correction result of the first difference based on the second difference, the first difference or the correction result of the first difference based on the second difference as the density difference of the partial region indicated by the corrected first difference. Item 3. The image processing device according to item 1 or 2. (Item 4) the determining means determines whether the printed matter to be inspected satisfies the inspection standard based on whether the second difference exceeds a threshold value. 4. The image processing device according to any one of items 1 to 3. (Item 5) the correction means replaces the reference image with the print image when a third difference between the density of the partial region in the gradation-converted print image and the density of a region in the gradation-converted reference image corresponding to the partial region of the print image is equal to or greater than a first threshold value and the number of partial regions that do not satisfy a second threshold value or less is equal to or greater than a threshold value; 5. The image processing device according to any one of items 1 to 4. (Item 6) the first difference is the smallest difference among the differences between the density of the partial region in the print image and the density of each partial region in the reference image that is located near a region corresponding to the partial region in the print image; 6. The image processing device according to any one of items 1 to 5. (Item 7) the correction means corrects the density difference of the partial region indicated by the first difference with a weight that adjusts the difference. Item 3. The image processing device according to item 3. (Item 8) a user interface for setting the threshold value for the second difference; Item 5. The image processing device according to item 4. (Item 9) a notification unit that notifies an external device of the status of the printing device based on whether the second difference exceeds a threshold value; Item 5. The image processing device according to item 4. (Item 10) an acquisition means for acquiring the reference image and the print image; a calculation unit for calculating the density of an area in the print image that includes the partial area and the density of an area in the reference image that corresponds to the area in the print image that includes the partial area; 10. The image processing device according to any one of items 1 to 9. (Item 11) The second difference is a statistical value of at least one of a least square error of RGB values, a difference in luminance values, and a color difference of Lab values ​​within the region. 11. The image processing device according to any one of items 1 to 10. (Item 12) a server that generates a print job for a printing device; a printing device that prints the printed material based on the print job; a reading device for scanning the printed matter; and an image processing device according to any one of claims 1 to 11. Printing system. (Item 13) An image processing method executed by an image processing device that inspects a printed matter printed by a printing device, comprising: a correction step in which a generation means of the image processing device corrects a first difference between the density of a partial area in a printed image obtained by scanning a printed matter to be inspected and the density of an area corresponding to the partial area in a reference image obtained by scanning a printed matter that is an inspection standard, using a second difference between the density of an area in the printed image that includes the partial area and the density of an area in the reference image that corresponds to the included area; The determination means of the image processing device has a determination means that determines whether or not the printed image includes a defective area based on the corrected first difference. Image processing methods. (Item 14) 12. A program for causing a computer to function as each means of the image processing device according to any one of items 1 to 11.

[0129] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0130] 10: Printing system 100: Image processing device 101:CPU 102:RAM 103:ROM 104: Main memory 105: Reading device 106: Printing device I / F 107: General-purpose I / F 108: UI Panel 109: Main bus 110: Transport path 111: Output tray 112: Output tray

Claims

1. a correction means for correcting a first difference between a density of a partial region in a printed image obtained by scanning a printed matter to be inspected and a density of a region in a reference image serving as an inspection standard that corresponds to the partial region, using a second difference between a density of a region in the printed image that includes the partial region and a density of a region in the reference image that corresponds to the region that includes the partial region; and an inspection means for inspecting the print image based on the corrected first difference. Image processing device.

2. the inspection means determines whether or not the print image includes a defective area based on whether or not the corrected first difference exceeds a threshold value. The image processing device according to claim 1 .

3. the correction means selects, as the first difference after correction, the first difference or the correction result of the first difference based on the second difference, based on a comparison between the first difference and a correction result of the first difference based on the second difference. The image processing device according to claim 1 .

4. the inspection means determines whether the printed matter to be inspected satisfies the inspection criterion based on whether the second difference exceeds a threshold value. The image processing device according to claim 1 .

5. the correction means replaces the reference image with the print image when a third difference between a density of the partial region in the tone-converted print image and a density of a region in the tone-converted reference image corresponding to the partial region of the print image is equal to or greater than a first threshold value and the number of partial regions that do not satisfy a second threshold value or less is equal to or greater than a threshold value; The image processing device according to claim 1 .

6. the first difference is the smallest difference among differences between the density of the partial region in the print image and the density of each partial region in the reference image that is adjacent to a region corresponding to the partial region in the print image; The image processing device according to claim 1 .

7. The correction means corrects the first difference by a weight. The image processing device according to claim 3 .

8. a display control means for displaying a user interface for setting the threshold value for the second difference; The image processing device according to claim 4.

9. a notification unit that notifies an external device of a state of a printing device that prints the printed matter based on whether the second difference exceeds a threshold value. The image processing device according to claim 4.

10. an acquisition means for acquiring the reference image and the print image; a calculation unit that calculates a density of a region in the print image that includes the partial region and a density of a region in the reference image that corresponds to the region in the print image that includes the partial region, The image processing device according to claim 1 .

11. The second difference is at least one statistical value of a least square error of RGB values, a difference in luminance values, and a color difference of Lab values ​​within the region. The image processing device according to claim 1 .

12. a server for generating print jobs for a printing device; a printing device that prints the printed matter based on the print job; A reading device for scanning the printed matter; The image processing device according to any one of claims 1 to 11, Printing system.

13. a correction process for correcting a first difference between a density of a partial region in a printed image obtained by scanning a printed matter to be inspected and a density of a region in a reference image serving as an inspection standard, which corresponds to the partial region, by using a second difference between a density of a region in the printed image that includes the partial region and a density of a region in the reference image that corresponds to the region that includes the partial region; and an inspection step of inspecting the print image based on the corrected first difference. Image processing methods.

14. A program for causing a computer to function as each of the means of the image processing device according to any one of claims 1 to 11.

Images