Image processing apparatus, method for controlling image processing apparatus, program, and inspection system

Abstract

To achieve both detection accuracy of a print defect and calculation cost in inspection of a printed matter.SOLUTION: An image processing apparatus that inspects a printed matter formed by a printing apparatus includes an image acquisition unit that acquires a reference image serving as an inspection reference of the printed matter and a read image obtained by reading the printed matter by a reading device, an alignment unit that performs alignment between the reference image and the read image, an extraction unit that extracts a defect candidate area having a possibility of a printing defect in a correspondence relationship between the reference image and the aligned read image, and a detection unit that detects a defect based on a difference value of pixel values in a search range larger than a search range of the defect candidate area for pixels included in the defect candidate area extracted by the extraction unit.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a technique for inspecting printed matter. [Background technology]

[0002] Printed materials output from a printing device may suffer from stains caused by ink, toner, or other colorants adhering to unintended locations, or from insufficient colorants adhering to areas where an image should be formed, resulting in color loss. Systems for inspecting print defects include, for example, print inspection systems that scan printed materials output from a printing device using a camera or line sensor and automatically inspect the scanned image to determine whether printing is being performed properly. Such print inspection systems detect print defects in the prints being inspected based on the difference between a reference image serving as the inspection standard and an inspection image representing the image data of the prints being inspected. When performing full inspections in real time in a print inspection system that matches the printing speed of the printing device, extremely high-speed computing performance is required for real-time inspection processing, which contributes to the increased cost of print inspection systems. Patent Document 1, for example, discloses a method for reducing the amount of computation required to reduce costs by switching between inspection processing with high computational costs for areas of the inspection image with many edges and inspection processing with low computational costs for areas of the inspection image with few edges. There is also a need for technology that can detect print defects across the entire inspection image with high accuracy. As a method for detecting print defects with high accuracy, Patent Document 2 discloses a method in which defect candidate areas that may be print defects are extracted based on the difference between a reference image and an image to be inspected, and additional defect determination processing is performed on the defect candidate areas based on edge information of the image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-124635 [Patent Document 2] Japanese Patent Publication No. 2020-030054 Summary of the Invention [Problem to be solved by the invention]

[0004] The method of Patent Document 1 performs inspection processing by combining a computationally expensive inspection process using high-precision alignment and advanced resolution conversion processing with a computationally less expensive inspection process using simple alignment and simple resolution conversion processing. The high-precision alignment in Patent Document 1 uses affine transformation. Affine transformation, a linear alignment process, cannot correct local distortions that occur when scanning printed materials. Therefore, when attempting to detect minute defects, overdetection may occur due to insufficient alignment accuracy. Furthermore, the simple alignment in Patent Document 1 uses only parallel translation or no alignment at all. While the technology in Patent Document 1 can reduce the amount of calculation, for example, when detecting minute defects near a pattern, there is a possibility that misalignment of the pattern may be overdetected as a defect in a location where there is no defect, as with affine transformation. Therefore, the method of Patent Document 1 does not provide high accuracy in detecting defects.

[0005] In the method of Patent Document 2, defect candidate areas are inspected by comparing the edge directions of a reference image and an inspection image. If the edge directions are different, it is deemed a defect, and if they are the same, it is deemed an overdetection caused by insufficient pattern alignment accuracy. This makes it possible to accurately detect minute defects near patterns while suppressing overdetection. The method of Patent Document 2 has the advantage of being able to accurately detect defects. However, because edge information is processed in addition to difference information, the amount of data to be processed increases, resulting in high computational costs. As described above, with conventional technology, it has been difficult to achieve both high accuracy in detecting print defects and low computational costs. [Means for solving the problem]

[0006] The image processing device according to the present disclosure is an image processing device that inspects printed matter formed by a printing device, and is characterized by comprising: an image acquisition means that acquires a reference image that serves as an inspection standard for the printed matter and a read image obtained by reading the printed matter using a reading device; an alignment means that aligns the reference image with the read image; an extraction means that extracts a defect candidate area that may be a printing defect and that corresponds to the reference image and the aligned read image; and a detection means that detects the printing defect based on the difference value of pixel values ​​in a search range that is larger than the search range of the defect candidate area for pixels included in the defect candidate area extracted by the extraction means. [Effects of the Invention]

[0007] According to the technology of the present disclosure, it is possible to achieve both high accuracy in detecting print defects and low computational costs. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing the configuration of a print inspection system. [Figure 2] FIG. 1 is a diagram showing the configuration of a print inspection system including a cloud. [Figure 3] FIG. 1 is a block diagram showing the functional configuration of an image processing apparatus. [Figure 4] 1 is a flowchart showing the flow of an image processing method. [Figure 5] 3A and 3B are diagrams showing examples of UIs in a UI panel 108. [Figure 6] 10A and 10B are diagrams showing specific examples of alignment between a reference image and a read image. [Figure 7] 10A and 10B are diagrams showing specific examples of simple inspection and detailed inspection. [Figure 8] 10 is a flowchart showing the flow of processing by an extraction unit 305. [Figure 9] 10 is a flowchart showing the flow of processing by a detection unit 306. [Figure 10] 10 is a flowchart showing the flow of processing by an extraction unit 305 in the second embodiment. [Figure 11]11 is a flowchart showing the flow of processing by an extraction unit 305 in the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that the following embodiments do not limit the present disclosure, and not all combinations of features described in the embodiments are necessarily essential to the solutions of the present disclosure. Note that identical components will be described with the same reference numerals. Furthermore, each process (step) in a flowchart will be indicated with a reference numeral beginning with "S."

[0010] [First embodiment] The image processing device according to this embodiment performs a simple inspection to extract defect candidate areas that may be print defects based on the difference between a reference image, which serves as an inspection standard for printed matter, and an inspection image.The image processing device then performs a detailed inspection to calculate the difference for the extracted defect candidate areas under conditions with a larger search range than that of the simple inspection.

[0011] <<Print inspection system configuration>> 1 is a diagram showing the overall configuration of a print inspection system that outputs and inspects printed matter, including an image processing device 100 according to this embodiment. The print inspection system according to this embodiment includes the image processing device 100 and a printing device 190. The print inspection system according to this embodiment may further include a printing server 180.

[0012] The printing server 180 generates a print job including a document to be printed, and submits the print job to the printing device 190. The printing device 190 forms an image on a sheet based on the print job submitted from the printing server 180. The printing device 190 has a paper feed unit 191, and a user can supply sheets to the paper feed unit 191 in advance. When a print job is submitted, the printing device 190 forms an image on one or both sides of the sheet while transporting the sheet supplied to the paper feed unit 191 along a transport path 192, and transports the printed sheet to the image processing device 100.

[0013] The image processing device 100 inspects printed sheets (inspection target sheets) for defects. The inspection target sheets are obtained by forming an image on a sheet using a printing device 190, and are transported through a transport path 192 inside the printing device 190. The image processing device 100 internally includes a CPU 101, RAM 102, and ROM 103. The image processing device 100 also includes an image reading device 104, a network interface (I / F) 105, a printing device I / F 106, a general-purpose I / F 107, a user interface (UI) panel 108, and a main bus 109. The image processing device 100 also includes a print medium transport path 110 connected to the transport path 192 of the printing device 190. The image processing device 100 also includes an output tray 111 to which inspection target media that have been determined to pass the inspection are output, and an output tray 112 to which inspection target media that have been determined to fail the inspection are output. 1, the output tray 111 and the output tray 112 are connected to the CPU 101 via the main bus 109. Depending on the inspection result of the inspection target sheet, the destination of the inspection target sheet is set to the output tray 111 or the output tray 112.

[0014] Each function of the image processing device according to each embodiment can be realized by a computer including a processor and a memory. For example, a processor such as CPU 101 executes a program stored in a memory such as RAM 102 or ROM 103 to realize the function of each unit. A processor such as CPU 101 can also control each module in the image processing device 100 as needed. Note that the image processing device according to each embodiment of the present disclosure may be configured by a plurality of processing devices connected via a network, for example.

[0015] The CPU 101 is a processor that controls each unit within the image processing device 100. The RAM 102 temporarily stores applications executed by the CPU 101, data used in image processing, etc. The ROM 103 stores a group of programs executed by the CPU 101.

[0016] The image reading device 104 scans and reads one or both sides of a printed sheet conveyed from the printing device 190 on the conveying path 110 and acquires the image data. The conveying path 110 serves as the background when the image reading device 104 reads the image of the printed sheet, and therefore may have a color (e.g., black) that makes it easy to distinguish it from the print medium in the image. The printing device I / F 106 is connected to the printing device 190, and the image processing device 100 can communicate with the printing device 190 via the printing device I / F 106. For example, the printing device 190 and the image processing device 100 can be synchronized via the printing device I / F 106, and the printing device 190 and the image processing device 100 can notify each other of their operating statuses.

[0017] The UI panel 108 outputs information to the user. The UI panel 108 may be a display device such as a liquid crystal display, and functions as a user interface for the image processing device 100. The UI panel 108 can, for example, inform the user of the current status or settings of the image processing device 100. The UI panel 108 may also be equipped with an input device such as a touch panel or buttons. The input device can accept instructions from the user. The UI panel 108 may be composed of a single touch panel. The main bus 109 is a transmission path connecting each module of the image processing device 100.

[0018] While the conveying path 110 conveys the printed sheet output from the printing device 190, the image processing device 100 performs an inspection process to check for defects on the printed sheet based on image data of the printed sheet acquired by the image reading device 104. If the inspection process results in a pass, the printed sheet is conveyed to the output tray 111. If the inspection process results in a fail, the printed sheet is conveyed to the output tray 112. By this operation, only printed sheets determined to have no defects are output onto the output tray 111. Note that the print inspection system according to this embodiment may realize each function of this embodiment using another system configuration that performs processing via a cloud server.

[0019] FIG. 2 is a diagram showing a system configuration that performs processing via a cloud server in addition to the configuration of the print inspection system in FIG. 1. The hardware configuration of the cloud server, which is an additional configuration, will be described. The cloud server 200 of this embodiment is one or more server devices that provide cloud services over the Internet. The cloud server 200 receives image data of printed sheets acquired by the image reading device 104 of the image processing device 100, performs inspection processing on the image data of the printed sheets, and transmits the inspection results to the image processing device 100 as necessary. The cloud server 200 includes a CPU 201, a RAM 202, a ROM 203, a storage device 204, and a network I / F 205. The components are connected to each other by a system bus 207.

[0020] The CPU 201 uses the RAM 202 as a work memory, executes programs stored in the ROM 203, and performs overall control of each component of the cloud server 200 via the system bus 207. The storage device 204 is, for example, a hard disk drive (HDD), a solid state drive (SSD), or flash memory, and stores various data handled by the cloud server 200. The CPU 201 writes data to the storage device 204 and reads data stored in the storage device 204 via the system bus 207.

[0021] The network I / F 205 is an interface for connecting to the Internet. The cloud server 200 receives requests from, for example, a web browser of the image processing device 100 or transmits and receives image data via the network I / F 205. The cloud server 200 may be configured from a single server device or multiple server devices. Furthermore, the functions of multiple server devices may be realized by a single server device using virtualization software. Furthermore, the cloud server 200 may be interconnected not only with the image processing device 100 but also with the printing device 190 to manage print jobs or inspection results.

[0022] <<Functional configuration of image processing device>> FIG. 3 is a block diagram showing the functional configuration of the image processing apparatus 100 according to this embodiment. A reference image acquisition unit 301 acquires manuscript data, which is the original data of a printed matter, by reading it into the RAM 102. Note that, although manuscript data is acquired as the reference image in this embodiment, the reference image may also be acquired from a scanned image of a printed sheet on which printing has been performed by the printing device 190. A scanned image acquisition unit 302 acquires a scanned image of a printed sheet on which printing has been performed by the printing device 190. Image data of the acquired scanned image is stored in the RAM 102. In this embodiment, the scanned image acquisition unit 302 acquires image data of a scanned image obtained by the image reading device 104 reading a printed sheet on the conveying path 110. An inspection information acquisition unit 303 acquires information related to the inspection job and inspection level related to the inspection information setting based on user operations acquired via the UI panel 108, etc.

[0023] The alignment unit 304 aligns the reference image and the scanned image so that their patterns match. The extraction unit 305 compares pixel values ​​between the reference image and the scanned image to be inspected, and extracts defect candidate areas based on the compared data. The detection unit 306 performs a detailed inspection of the pixels calculated as defect candidate areas to detect print defects. The determination unit 307 determines whether or not there is a print defect in the inspection target image based on the detection results of the detection unit 306. The notification unit 308 notifies the user of various information and controls the display of a UI on the UI panel 108 to allow the user to input information necessary for processing.

[0024] <<Processing Executed by the Image Processing Device>> The processing performed by the image processing device 100 according to this embodiment, which has the above-described configuration, will be described below. Fig. 4 is a flowchart showing the flow of processing performed by the image processing device 100. In S401, the examination information acquisition unit 303 displays a UI screen on the UI panel 108 for receiving instructions from the user so that the user can input information necessary for the examination, and acquires examination information based on user operations, etc.

[0025] FIG. 5 illustrates an example of a UI screen displayed in S401. In FIG. 5, an inspection job setting button 501 is a button for setting information about an inspection job, which associates a print job that manages image data and sheet information used for inspection with information about inspection level settings. The user presses the inspection job setting button 501 to specify an inspection job that has been created and registered in advance. The specified inspection job is stored in RAM 102. An inspection level setting area 502 is a screen for setting the inspection level to be used for inspection. The user selects the inspection level for each inspection area from a pull-down menu, and the selected value is stored in RAM 102. A display window 503 displays the image to be used for inspection, and the user sets the inspection level for each area while viewing the displayed image. In this embodiment, three inspection areas can be specified as inspection areas: a priority inspection area, a standard inspection area, and a simple inspection area. The user can specify each area using a mouse or touch panel. If no area is specified, inspection is performed at the inspection level specified for the standard inspection area. The inspection level is associated with processing parameters required to detect defects at each inspection level, depending on the size and contrast of the defects to be detected. During inspection, defects are detected according to the inspection level selected by the user. In this embodiment, the inspection level is set from five levels. The higher the inspection level, the more defects with low contrast and small size are detected. The lower the inspection level, the more defects with high contrast and large size are detected. In this embodiment, three types of inspection areas and five inspection levels are maintained, but the inspection areas or inspection levels may be set to other numbers. The Inspection Execution button 504 is a button for executing the inspection process. When the Inspection Execution button 504 is pressed, the inspection process is executed based on the inspection job settings, inspection level settings, and information set in the display window 503. Information such as the type and position of defects determined by the inspection process is stored in RAM 102. The Inspection Result window 505 is a window that displays a defect map showing defect information in the inspection image and the determination result of the number of failed images. Information about the type and position of defects detected by the inspection process is displayed in the Inspection Result window 505 along with the corresponding image area.

[0026] In S402, the reference image acquisition unit 301 acquires original document data or a scanned image, which is the original data of the printed matter, and stores the acquired image in RAM 102 as a reference image. When a scanned image is used as the reference image, the image reading device 104 acquires the scanned image obtained by reading the inspection target medium on the conveyance path 110 and stores the image in RAM 102. In this embodiment, the scanned image is an image expressed in 8 bits for each of RGB, but an image expressed in 16 bits may also be used. Note that the image reading device 104 generates a scanned image by reading the inspection target sheet, but this is also applicable to inspecting an image acquired using a different device. For example, a scanned image acquired by a device other than the image reading device 104 may be stored in an auxiliary storage device (not shown). In this case, the scanned image acquisition unit 302 can acquire the scanned image from the auxiliary storage device. Furthermore, when original document data is used as the reference image, overdetection may occur due to differences in image characteristics between the original document data and the scanned image. Therefore, in this embodiment, correction processing is performed on the original document data for image characteristics such as color reproduction characteristics and thin line reproduction characteristics. In the color reproduction correction process, a lookup table (LUT) is created in advance that describes the correspondence for converting four-channel CMYK data into three-channel RGB data, which is the scanned image data, according to the characteristics of the printing device and output conditions such as the sheet used for printing. Correction is performed by referencing the LUT.

[0027] Furthermore, in the thin line correction process, a thin line chart is output and read in advance to calculate correction data relating to the reproduction of thin lines between the document data and the read image. In this embodiment, a filter coefficient for matching the line width with that of the read image based on the line profile of the thin line, and a filter coefficient for correcting edge blurring during scanning are calculated. A reference image is generated by applying a correction process to the document data using these two types of filter coefficients. In addition, a process is performed to convert the resolution of the document data to match the image size and resolution of the read image used during inspection. Note that, although the resolution of the reference image and the read image in this embodiment is 600 dpi, images with other resolutions, such as 300 dpi, may also be used.

[0028] In S403, the read image acquisition unit 302 acquires a read image obtained by the image reading device 104 reading the sheet to be inspected on the conveying path 110, and stores the image in the RAM 102. In S404, the inspection information acquisition unit 303 acquires the inspection job set by the inspection job setting button 501 and the inspection level set in the inspection level setting area 502 as inspection information.

[0029] In S405, the registration unit 304 aligns the reference image with the scanned image to generate an inspection image from the scanned image. In this embodiment, affine transformation processing is used for the alignment. The registration unit 304 extracts registration reference points representing the characteristics of the reference image and the scanned image using a known image feature point extraction method such as SIFT or SURF. Here, SIFT stands for Scale Invariant Feature Transform. SURF stands for Speeded Up Robust Features. Geometric transformation parameters for affine transformation are calculated based on the extracted feature points. Note that although affine transformation is used in this embodiment, alignment between images may also be performed using other geometric transformation parameters such as projective transformation. Non-rigid registration processing such as FFD (Free-form deformation) may also be used.

[0030] A specific example of alignment between a reference image and a read image is shown in Figure 6. Figure 6(a) shows the reference image. Figure 6(b) shows an example of a read image before alignment. In Figure 6(b), the read image before alignment is shifted downward compared to the reference image. By performing the above-described alignment on the read image before alignment, a read image after alignment (image to be inspected) aligned with the reference image can be obtained, as shown in Figure 6(c).

[0031] In S406, the extraction unit 305 compares the reference image with the image to be inspected, and extracts a defect candidate area based on the compared data. Details of the processing by the extraction unit 305 will be described later. In S407, the CPU 101 (control unit) of the image processing device 100 determines whether the number of pixels in the defect candidate area calculated in S406 is greater than a threshold value. If the number of pixels in the defect candidate area is greater than the threshold value, the processing proceeds to S409. If the number of pixels in the defect candidate area is equal to or less than the threshold value, the processing proceeds to S408.

[0032] The threshold value used in this embodiment is determined based on the allowable time for print inspection and the estimated inspection time. The allowable time for print inspection is the inspection processing time allowed per image when inspecting in real time, determined by the print speed (pages per minute) specified for each paper setting of the print job. The allowable time for print inspection is a value determined on the assumption that inspection is performed in real time in accordance with the print speed, without accumulating data related to the inspection image and inspection processing within the image processing device 100. The estimated inspection time is an estimate of the time required for inspection, including simple inspection and detailed inspection. The simple inspection time increases as the paper size increases, and the correspondence between paper size and inspection time is stored in a table in advance. The detailed inspection time increases as the area of ​​the defect candidate area increases, and the correspondence between the area and detailed inspection time is stored in a table in advance. The CPU 101 (controller) of the image processing device 100 references these two tables, calculates the maximum area of ​​the defect candidate area that fits within the allowable time for print inspection, and determines whether to perform a detailed inspection using this area as a threshold.

[0033] In S408, the detection unit 306 performs a detailed inspection of the pixels in the defect candidate area calculated in S406. Details of the processing by the detection unit 306 will be described later. In S409, if a detailed inspection is performed, the notification unit 308 notifies the user that the detailed inspection will not be performed because the inspection process will not be completed within a predetermined time. In this case, a map of the defect candidate area calculated in S406 is displayed in the inspection result window 505.

[0034] Specific examples of the simple inspection in S406 and the detailed inspection in S408 are shown in FIG. 7. FIG. 7(a) shows a pixel of interest in the image to be inspected. FIG. 7(b) shows an example in which the search range on the reference image in the simple inspection is a 3x3 search window. Specifically, in the area enclosed by the thick line frame, inspection begins from the top left pixel and continues to the bottom right pixel, and the pixel with the smallest difference from the pixel of interest is extracted. This search method is merely an example, and the search may be performed in a different order. Because the search range for the detailed inspection is larger than that for the simple inspection, FIG. 7(c) shows an example in which the search range on the reference image in the detailed inspection is a 5x5 search window. The search range for the simple inspection or the detailed inspection is not necessarily limited to a square. As shown in FIG. 7(d), the search range for the simple inspection or the detailed inspection may be a rectangular search window, such as a 3x7.

[0035] In S410, the notification unit 308 displays the results of processing in S408 in the inspection result window 505. If a print defect is detected, an image of the defect map and an enlarged image of the defect map are displayed in the inspection result window 505. The defect map is a map that indicates the coordinate positions of the defect if a print defect is present. The defect map is an image file of the same vertical and horizontal dimensions as the reference image and the image to be inspected. If a print defect is present, for example, a pixel value of RGB=(255,255,255) (white) is stored at the corresponding coordinate position. If there is no print defect, a pixel value of RGB=(0,0,0) (black) is stored at the coordinate position. In this embodiment, if there is no print defect, all pixel values ​​are stored as 0. If there is a point-like print defect, a pixel value of RGB=(255,0,0) is stored at the corresponding coordinate position. If there is a line-like print defect, a pixel value of RGB=(0,255,0) is stored at the corresponding coordinate position. Other values ​​may be set for the above pixel values. If it is determined in S407 that the number of pixels in the defect candidate area is greater than the threshold value and the process proceeds to S409, the map of the defect candidate area calculated in S406 is displayed in the inspection result window 505.

[0036] <<Operation of the Extraction Unit 305 in S406>> 8 is a flowchart showing the processing in the extraction unit 305. In S801, the extraction unit 305 acquires the reference image acquired in S402. In S802, the extraction unit 305 acquires the inspection target image that has been aligned in S405. In S803, the extraction unit 305 calculates the difference value of pixel values ​​at the same coordinates between the reference image acquired in S801 and the inspection target image acquired in S802.

[0037] In S804, the extraction unit 305 determines whether all differences within the predetermined search range have been calculated for the images acquired in S801 and S802. If all differences within the predetermined search range have been calculated, the process proceeds to S805. If all differences within the predetermined search range have not been calculated, the process returns to S803. In this embodiment, the predetermined search range is calculated by searching for pixels within a range obtained by shifting the reference image by ±1 pixel vertically and horizontally relative to the scanned image, and calculating the differences. Note that it is also possible to calculate only the difference values ​​between the same coordinates without searching. If no search is required, the processes of S804 and S805 are skipped, and after executing the process of S803, the process proceeds to S806. In S805, the extraction unit 305 calculates the smallest difference value among the difference values ​​calculated within all search ranges as the difference value.

[0038] In S806, the extraction unit 305 compares the difference value calculated in S805 with a predetermined threshold. Pixels whose difference values ​​calculated in S805 are greater than the predetermined threshold are binarized as RGB = (255, 255, 255), and pixels whose difference values ​​calculated in S805 are less than the threshold are binarized as RGB = (0, 0, 0). The inspection level acquired in S404 holds a contrast threshold for the difference value for each inspection level, as well as the size of dots and lines. The predetermined threshold is determined by referring to the contrast threshold corresponding to the inspection level, and the pixels to be inspected are binarized. The binarized pixel values ​​are saved as a defect candidate map. The defect candidate map is a map that indicates target pixels as defect candidate areas that may be causing print defects. The defect candidate map is an image file with the same vertical and horizontal dimensions as the reference image and the scanned image. As a result of the binarization, if the pixel is a defect candidate, the pixel value of RGB=(255,255,255) is stored at the corresponding coordinate position, and if the pixel is not a defect candidate, the pixel value of RGB=(0,0,0) is stored at the corresponding coordinate position.

[0039] In S807, the extraction unit 305 performs an expansion process on the values ​​binarized in S806. The expansion process is a process for preventing omission of pixels that are the subject of detailed inspection, and in this embodiment, the defect candidate map is expanded by three pixels. Note that the expansion process may be performed using a different size, or may not be performed at all. Furthermore, the defect candidate map may be shaped, such as by removing noise or filling in defect candidate areas, by combining an opening process and a closing process. In S808, the extraction unit 305 saves the defect candidate map that is the processing result of S807, and ends the processing flow of the flowchart shown in FIG. 6.

[0040] <<Operation of the detection unit 306 in S408>> 9 is a flowchart showing the processing in the detection unit 306. In S901, the detection unit 306 acquires the reference image acquired in S402. In S902, the detection unit 306 acquires the inspection target image that has been aligned in S405. In S903, the detection unit 306 acquires the defect candidate map calculated in S406.

[0041] In S904, the detection unit 306 refers to the defect candidate map acquired in S903 and determines whether the pixel is a target pixel for detailed inspection. If the pixel value stored in the defect candidate map is RGB=(255,255,255), the process proceeds to S905 to perform detailed inspection. If the pixel value stored in the defect candidate map is RGB=(0,0,0), the process proceeds to S909 without performing detailed inspection.

[0042] In S905, the detection unit 306 calculates the difference in pixel values ​​between the reference image acquired in S901 and the inspection target image acquired in S902. In S906, the detection unit 306 determines whether the differences in all predetermined search ranges have been calculated for the images acquired in S901 and S902. If the differences in all predetermined search ranges have been calculated, the process proceeds to S907. If the differences in all predetermined search ranges have not been calculated, the process returns to S905. In this embodiment, the predetermined search range is calculated by shifting the reference image by a predetermined number of pixels vertically and horizontally relative to the scanned image acquired in S902, and searching for pixels within the range to calculate the difference. Note that in this embodiment, the predetermined search range is shifted by ±3 pixels, but this is not limited to this. For example, another numerical value such as ±5 pixels may be used. If there is a difference in the amount of misalignment between the main scanning direction and the sub-scanning direction, separate search ranges may be set vertically and horizontally in the image. By calculating the difference in a predetermined search range in the detailed inspection, it becomes possible to suppress overdetection due to insufficient alignment precision near the picture portion, etc.

[0043] In S907, the detection unit 306 calculates the smallest value among the difference values ​​calculated within all search ranges. In this embodiment, a simple difference with a fixed size of one pixel is used as the search window, but this is not limiting. A 3x3 or 5x5 search window may also be used to calculate the difference. Increasing the search window increases the computational cost, but improves defect detection performance.

[0044] In S908, the detection unit 306 performs defect detection processing on the difference value calculated in S907. Pixels for which the calculated difference is greater than a predetermined threshold are determined to be defective pixels. In this embodiment, a contrast threshold for the difference, and the sizes of dots and lines, are stored for each inspection level, and the presence or absence of dot-like and line-like print defects is determined based on the contrast of these differences and the shape sizes. In this embodiment, if there is no print defect, a pixel value of 0 (RGB=(0,0,0)) is stored in the target pixel. If there is a dot-like print defect, a pixel value of RGB=(255,0,0) is stored at the coordinate position. If there is a line-like defect, a pixel value of RGB=(0,255,0) is stored at the coordinate position. The pixel values ​​used when a defect is present are not limited to these. Other values ​​that can identify each defect may also be applied.

[0045] In S909, the detection unit 306 stores a pixel value of RGB=(0,0,0) in the target pixel of the defect map. In S910, the detection unit 306 determines whether detailed inspection has been performed on all target pixels for which detailed inspection is to be performed. If detailed inspection has been performed on all target pixels for which detailed inspection is to be performed, the process proceeds to S911. If detailed inspection has not been performed on all target pixels for which detailed inspection is to be performed, the process returns to S904. In S911, the detection unit 306 saves the defect map and ends the processing flow of the flowchart shown in FIG. 7.

[0046] As described above, by using the method of this embodiment, it is possible to achieve both high accuracy in detecting print defects and low computational costs by changing the search range when calculating the difference between a simple inspection (small search range or no search) and a detailed inspection (large search range).

[0047] [Second embodiment] In the first embodiment, a process was described in which defect candidate areas were extracted under conditions of a small search range in a simple inspection, and the extracted defect candidate areas were then inspected in detail under conditions of a large search range. In this embodiment, a method of inspection will be described in which inspection resolution is added to the inspection conditions in addition to the search range used in calculating the difference. Adding inspection resolution to the conditions makes it possible to inspect faster than in the first embodiment. Note that the operation of the extraction unit 305, which is the difference between the first embodiment and this embodiment, will be described, and other processes will not be described as they are the same as in the first embodiment.

[0048] Operation of the extraction unit 305 in the second embodiment 10 is a flowchart showing the flow of processing performed by the extraction unit 305. In S1001, the extraction unit 305 acquires the reference image acquired in S402. In S1002, the extraction unit 305 acquires the inspection object image that has been aligned in S405.

[0049] In S1003, the extraction unit 305 performs resolution conversion processing on the reference image acquired in S1001 and the inspection target image acquired in S1002. In this embodiment, the resolution of the acquired reference image and inspection target image is 600 dpi, with the simple inspection performed using images with a resolution of 300 dpi and the detailed inspection performed using images with a resolution of 600 dpi, but this is not limited to this. The images may be resized to other resolutions, such as 150 dpi for the simple inspection and 300 dpi for the detailed inspection. The resolution used for the simple inspection is set to a lower resolution than the resolution used for the detailed inspection to increase speed. In this embodiment, interpolation processing using a bilinear method is used for the resizing processing, but this is not limited to this. Interpolation processing using a nearest neighbor method or a bicubic method may also be used.

[0050] In S1004, the extraction unit 305 calculates the difference between pixel values ​​at the same coordinates between the reference image acquired in S1001 and the inspection target image acquired in S1002. In S1005, the extraction unit 305 determines whether or not differences have been calculated for all predetermined search ranges for the images acquired in S1001 and S1002. If differences have been calculated for all predetermined search ranges, the process proceeds to S1006. If differences have not been calculated for all predetermined search ranges, the process returns to S1004. Note that it is also possible to calculate only difference values ​​between the same coordinates without performing a search. If a search is not performed, the processes of S1005 and S1006 are skipped, and after performing S1004, the process proceeds to S1007. In S1006, the extraction unit 305 calculates the smallest difference value among the difference values ​​calculated within all search ranges as the difference value.

[0051] In S1007, the extraction unit 305 compares the difference value calculated in S1006 with a predetermined threshold. Pixels whose difference values ​​calculated in S1006 are greater than the predetermined threshold are binarized as RGB=(255,255,255), and pixels whose difference values ​​calculated in S1006 are equal to or less than the threshold are binarized as RGB=(0,0,0). In S1008, the extraction unit 305 performs expansion processing on the values ​​binarized in S1007. In S1009, the extraction unit 305 saves the defect candidate map, which is the processing result of S1008, and ends the processing flow of the flowchart shown in FIG. 10.

[0052] As described above, by using the method of this embodiment, it is possible to achieve both high accuracy in detecting print defects and low computational costs by changing the search range when calculating the difference and by changing the inspection resolution between simple inspection (low resolution) and detailed inspection (high resolution).

[0053] [Third embodiment] In this embodiment, a method for changing the inspection sensitivity used for inspection will be described as a difference between the inspection conditions for simple inspection and detailed inspection. A problem with simple inspection is that the search range for difference calculation is smaller than that for detailed inspection, which may result in overlooking of defect candidates. In this embodiment, detailed inspection is performed at an inspection sensitivity specified by the user, while simple inspection applies an inspection sensitivity that is stricter than the inspection sensitivity specified by the user, thereby reducing overlooking of defect candidate areas. The following describes the processing of the extraction unit 305, which is a difference between this embodiment and the first embodiment. Since the other processing is the same as that of the first embodiment, a description thereof will be omitted.

[0054] Operation of the extraction unit 305 in the third embodiment 11 is a flowchart showing the flow of processing performed by the extraction unit 305. In S1101, the extraction unit 305 acquires the reference image acquired in S402. In S1102, the extraction unit 305 acquires the read image after alignment performed in S405.

[0055] In S1103, the extraction unit 305 sets inspection parameters with higher inspection sensitivity than the user specification. The extraction unit 305 acquires the inspection level set in the inspection level setting area 502 in S404 and performs binarization processing in S1107 using a contrast threshold of an inspection level stricter than the inspection level. In this embodiment, the contrast threshold of an inspection level one level higher than the user specification is set, but a contrast threshold of two levels higher or the strictest inspection level may also be used. Furthermore, if the user specifies the strictest inspection level, a contrast threshold corresponding to an inspection level exceeding the upper limit of the inspection level that the user can specify may be stored inside the image processing device 100 and used as the contrast threshold.

[0056] In S1104, the extraction unit 305 calculates the difference value of pixel values ​​at the same coordinates between the reference image acquired in S1101 and the inspection target image acquired in S1102. In S1105, the extraction unit 305 determines whether or not differences have been calculated for all predetermined search ranges for the images acquired in S1101 and S1102. If differences have been calculated for all search ranges, the process proceeds to S1106. If differences have not been calculated for all search ranges, the process returns to S1104. Note that it is also possible to calculate only difference values ​​between the same coordinates without searching. If no search is to be performed, the processes of S1105 and S1106 are skipped, and after performing the process of S1104, the process proceeds to S1107. In S1106, the extraction unit 305 calculates the smallest value of the difference values ​​calculated for all search ranges as the difference value.

[0057] In S1107, the extraction unit 305 compares the difference value calculated in S1106 with a predetermined threshold. Pixels whose difference values ​​calculated in S1106 are greater than the predetermined threshold are binarized as RGB=(255,255,255), and pixels whose difference values ​​calculated in S1106 are less than or equal to the threshold are binarized as RGB=(0,0,0). In S1108, the extraction unit 305 performs an expansion process on the values ​​binarized in S1107. In S1109, the extraction unit 305 saves the defect candidate map, which is the processing result of S1108, and ends the processing flow of the flowchart shown in FIG. 11.

[0058] As described above, by using the method of this embodiment, the inspection sensitivity during simplified inspection can be set to be stricter than the value specified by the user, thereby reducing the number of defect candidate areas that are overlooked. This makes it possible to achieve both high print defect detection accuracy and low computational costs.

[0059] [Other embodiments] The present disclosure can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0060] The disclosure of the above-described embodiment includes the following configurations.

[0061] (Configuration 1) An image processing device for inspecting printed matter formed by a printing device, comprising: an image acquisition means for acquiring a reference image that serves as an inspection standard for the printed matter and a read image obtained by reading the printed matter using a reading device; an alignment means for aligning the reference image with the read image; an extraction means for extracting a defect candidate area that may be a printing defect and that corresponds to the reference image and the aligned read image; and a detection means for detecting the printing defect based on a difference value between pixel values ​​of a search range that is larger than the search range of the defect candidate area for pixels included in the defect candidate area extracted by the extraction means.

[0062] (Configuration 2) The image processing device according to Configuration 1, further comprising a notification means for notifying various information, wherein the notification means notifies that the detection process performed by the detection means will not be completed within a predetermined time if the number of pixels representing the defect candidate area is greater than a threshold value.

[0063] (Configuration 3) The image processing device according to Configuration 2, wherein the threshold value is the maximum area of ​​the defect candidate region for which the detection process performed by the detection means is completed within an allowable time for the inspection.

[0064] (Configuration 4) The image processing device according to configuration 2 or 3, wherein the notification means displays the notification on a display device.

[0065] (Configuration 5) The image processing device according to Configuration 2, wherein the notification means displays the defect candidate area and / or the area in which the print defect is detected on a display device so as to distinguish it from the area without the print defect.

[0066] (Configuration 6) The image processing device according to any one of configurations 1 to 5, wherein the search range is a square or rectangular range.

[0067] (Configuration 7) The image processing device according to Configuration 1, wherein the process of extracting the defect candidate area performed by the extraction means is performed at a lower resolution than the process of detecting the print defect performed by the detection means.

[0068] (Configuration 8) The image processing device according to Configuration 1, characterized in that the process of extracting the defect candidate area performed by the extraction means is performed using a contrast threshold value having a higher value than a contrast threshold value specified by a user operation.

[0069] (Configuration 9) A control method for an image processing device that inspects printed matter formed by a printing device, comprising the steps of: acquiring a reference image that serves as an inspection standard for the printed matter and a read image obtained by reading the printed matter using a reading device; aligning the reference image with the read image; extracting a defect candidate area that may be a printing defect and that corresponds to the reference image and the aligned read image; and detecting the printing defect for pixels included in the defect candidate area extracted in the extraction step based on the difference value of pixel values ​​in a search range that is larger than the search range of the defect candidate area.

[0070] (Configuration 10) A program for causing a computer to function as the image processing device according to any one of configurations 1 to 8.

[0071] (Configuration 11) An inspection system comprising a printing device that prints an image on a sheet and an image processing device that inspects printed matter formed by the printing device, wherein the image processing device comprises: an image acquisition means that acquires a reference image that serves as an inspection standard for the printed matter and a read image read by a reading device of the printed matter; an alignment means that aligns the reference image with the read image; an extraction means that extracts a defect candidate area that may be a printing defect and that corresponds to the reference image and the aligned read image; and a detection means that detects the printing defect for pixels included in the defect candidate area extracted by the extraction means based on the difference value of pixel values ​​in a search range that is larger than the search range of the defect candidate area.

Claims

1. An image processing device that inspects a printed matter formed by a printing device, an image acquisition means for acquiring a reference image serving as an inspection standard for the printed matter and a read image obtained by reading the printed matter with a reading device; a positioning unit for performing positioning between the reference image and the read image; an extraction unit for extracting defect candidate areas that correspond to the reference image and the aligned read image and that may be print defects; a detection means for detecting the print defect based on a difference value of pixel values ​​of pixels included in the defect candidate area extracted by the extraction means within a search range that is larger than the search range of the defect candidate area; An image processing device comprising:

2. and a notification means for notifying various information.

2. The image processing device according to claim 1, wherein the notification means notifies the user that the detection process performed by the detection means will not be completed within a predetermined time if the number of pixels representing the defect candidate area is greater than a threshold value.

3. 3. The image processing apparatus according to claim 2, wherein the threshold value is a maximum area of ​​the defect candidate region for which the detection process performed by the detection means can be completed within an allowable time for the inspection.

4. 3. The image processing apparatus according to claim 2, wherein the notification means displays the notification on a display device.

5. 3. The image processing apparatus according to claim 2, wherein the notification means displays the defect candidate area or the area in which the print defect is detected on a display device so as to be distinguished from the area without the print defect.

6. 2. The image processing device according to claim 1, wherein the search range is a square or rectangular range.

7. 2. The image processing apparatus according to claim 1, wherein the process of extracting the defect candidate area performed by the extracting means is performed at a lower resolution than the process of detecting the print defect performed by the detecting means.

8. 2. The image processing apparatus according to claim 1, wherein the process of extracting the defect candidate area performed by the extracting means is executed using a contrast threshold value having a value higher than a contrast threshold value designated by a user operation.

9. A control method for an image processing device that inspects a printed matter formed by a printing device, comprising: a step of acquiring a reference image serving as an inspection standard for the printed matter and a read image obtained by reading the printed matter with a reading device; a step of aligning the reference image with the read image; extracting defect candidate areas that are in a corresponding relationship between the reference image and the aligned read image and that may be print defects; detecting the print defect based on a difference value of pixel values ​​in a search range that is larger than the search range of the defect candidate area for pixels included in the defect candidate area extracted in the extraction step; having 2. A method for controlling an image processing apparatus comprising:

10. A program for causing a computer to function as the image processing device according to any one of claims 1 to 8.

11. An inspection system including a printing device that prints an image on a sheet, and an image processing device that inspects a printed matter formed by the printing device, The image processing device includes: an image acquisition means for acquiring a reference image serving as an inspection standard for the printed matter and a read image obtained by reading the printed matter with a reading device; a positioning unit for performing positioning between the reference image and the read image; an extraction unit for extracting defect candidate areas that correspond to the reference image and the aligned read image and that may be print defects; a detection means for detecting the print defect based on a difference value of pixel values ​​of pixels included in the defect candidate area extracted by the extraction means within a search range that is larger than the search range of the defect candidate area; Equipped with An inspection system characterized by:

Images