Inspection device and method for controlling the same, and program

Abstract

To solve the problem in which: executing processing of creating all reference images from image data input as a job always before executing the job takes time to generate the reference images, which extends the time required for processing of the job.SOLUTION: An inspection device is to inspect an image printed on a sheet, and the inspection device executes first image processing on image data used for the printing of the image, and stores the image data on which the first image processing is executed as a first reference image. The inspection device determines whether the first reference image corresponding to a reference image usable in a job to be inspected is stored in storage means, and when determining that the corresponding first reference image is stored, executes, on the first reference image, second image processing of converting the first reference image to a second reference image corresponding to the job. The inspection device reads the image on the sheet printed according to the job to acquire a scan image, and performs processing of collating the acquired scan image with the second reference image.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to an inspection device, a control method thereof, and a program. [Background technology]

[0002] In printed matter printed by a printing device, coloring materials such as ink and toner may adhere to unintended locations, causing stains. Alternatively, a sufficient amount of coloring material may not adhere to a location where an image should be formed by adhering coloring material, causing color loss, where the color of that location becomes lighter than the original color. Such stains and color loss, or other printing abnormalities, reduce the quality of printed matter. Therefore, it is necessary to inspect printed matter for such abnormalities and guarantee the quality of printed matter.

[0003] Visual inspection by an inspector to check for printing abnormalities requires a lot of time and cost, so in recent years, inspection systems have been proposed that perform inspections automatically without relying on human visual inspection. Specifically, the system aligns the digital image (reference image) used for printing with the scanned image obtained by scanning the printed matter, and performs image matching and judgment processing to detect the presence or absence of printing abnormalities and judge the image quality of the printed matter.

[0004] Patent Document 1 proposes a method for performing high-speed processing in an image matching technique that converts a digital image into a reference image and compares the reference image with a scanned image. According to this method, high-speed processing is achieved by performing a rasterization (RIP) process on image data input as a job to generate a reference image before executing the job. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-43032 Summary of the Invention [Problem to be solved by the invention]

[0006] However, if the process of generating all reference images from image data input as a job is always performed before executing the job, there is a problem that it takes time to generate the reference images, which increases the time required to process the job.

[0007] An object of the present invention is to solve at least one of the problems of the above-mentioned conventional techniques.

[0008] An object of the present invention is to provide a technique that can efficiently generate a reference image while maintaining high speed image matching through comparison with a reference image. [Means for solving the problem]

[0009] In order to achieve the above object, an inspection device according to one aspect of the present invention has the following configuration: An inspection device for inspecting an image printed on a sheet of paper, a first image processing means for performing a first image processing on image data used for printing the image; a storage means for storing the image data that has been subjected to the first image processing by the first image processing means as a first reference image; a determining means for determining whether the first reference image corresponding to a reference image that can be used in the job to be inspected is stored in the storage means; a second image processing means for performing a second image processing on the first reference image to convert the first reference image into a second reference image corresponding to the job when the determination means determines that the corresponding first reference image is stored; a reading unit for reading an image on a sheet of paper printed in accordance with the job to obtain a scanned image; The image reading device further comprises a comparison unit that performs a comparison process between the scanned image obtained by the reading unit and the second reference image. Effect of the Invention

[0010] According to the present invention, it is possible to efficiently generate a reference image while maintaining high speed image matching through comparison with the reference image.

[0011] Other features and advantages of the present invention will become apparent from the following description taken in conjunction with the accompanying drawings, in which the same reference numerals are used to designate the same or similar components throughout the drawings. [Brief description of the drawings]

[0012] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention. [Figure 1] FIG. 1 is a diagram showing an example of a system configuration including an inspection device according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram illustrating the hardware configuration of the image forming apparatus according to the first embodiment. [Diagram 3] FIG. 2 is a diagram illustrating a mechanism of the image forming apparatus according to the first embodiment. [Figure 4] FIG. 2A is a schematic diagram illustrating the internal configuration of the inspection device according to the first embodiment, and FIG. 2B is a top view of a conveyor belt as viewed from the inspection sensor side. [Diagram 5] FIG. 2 is a block diagram illustrating the configuration of an inspection device control unit of the inspection device according to the first embodiment. [Figure 6] 4 is a flowchart for explaining an inspection process by the inspection device according to the first embodiment. [Figure 7] 4A to 4C are diagrams for explaining the direction in which paper is stored in a paper storage container in the first embodiment. [Figure 8] 4 is a view showing an example of an inspection setting screen displayed on an operation unit / display unit of the inspection device according to the first embodiment. [Figure 9] 4A to 4C are diagrams showing a color conversion table and an affine transformation formula according to the first embodiment. [Figure 10] 10 is a flowchart illustrating an inspection process by an inspection device according to a modified example of the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, the embodiments of the present invention will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0014] [Embodiment 1] FIG. 1 is a diagram showing an example of a system configuration including an inspection device 200 according to the first embodiment of the present invention.

[0015] The image forming apparatus 100 processes various input data and prints out. The inspection device 200 receives the printed matter printed and discharged by the image forming apparatus 100 and inspects the contents of the printed matter. The finisher 300 receives the output paper (printed matter) inspected by the inspection device 200 and performs post-processing such as bookbinding. The image forming apparatus 100 is connected to an external print server and a client PC via a network. The inspection device 200 is also connected to the image forming apparatus 100 on a one-to-one basis via a communication cable. The finisher 300 is also connected to the image forming apparatus 100 on a one-to-one basis via a communication cable different from the above. The inspection device 200 and the finisher 300 are also connected to each other via a different communication cable. This allows the image forming apparatus 100, the inspection device 200, and the finisher 300 to communicate with each other. In the first embodiment, an in-line inspection system that consistently performs image formation, image inspection, and finishing is shown.

[0016] FIG. 2 is a block diagram illustrating a hardware configuration of the image forming apparatus 100 according to the first embodiment.

[0017] This image forming apparatus 100 is an example of the image forming apparatus of the present invention, and includes a controller 21, a printer unit 206, and a UI (user interface) unit (operation unit) 23. The UI unit 23 includes various switches and displays for operation.

[0018] Image data or document data created by a software application such as a printer driver (not shown) on a client PC on the network or a print server is transmitted as PDL data to image forming apparatus 100 via a network (e.g., a local area network). In image forming apparatus 100, controller 21 receives the transmitted PDL data. Controller 21 is connected to printer unit 206, and upon receiving PDL data from a client PC or a print server, controller 21 converts the PDL data into print data that can be processed by printer unit 206 and outputs the print data to printer unit 206.

[0019] The printer unit 206 prints an image based on the print data output from the controller 21. The printer unit 206 according to the first embodiment is an electrophotographic printer engine. However, the printing method is not limited to this, and may be, for example, an inkjet (IJ) method.

[0020] The UI unit 23 is operated by a user and is used by the user to select various functions and give operation instructions. The UI unit 23 includes a display unit with a touch panel on its surface, and a keyboard on which various keys such as a start key, a stop key, and a numeric keypad are arranged.

[0021] Next, the controller 21 will be described in detail. The controller 21 has a network I / F (interface) unit 101, a CPU 102, a RAM 103, a ROM 104, an image processing unit 105, an engine I / F (interface) unit 106, and an internal bus 107. The network I / F unit 101 is an interface for receiving PDL data transmitted from a client PC or a print server. The CPU 102 controls the entire image forming apparatus 100 using programs and data stored in the RAM 103 and ROM 104, and executes the processes performed by the controller 21, which will be described later. The RAM 103 provides a work area used when the CPU 102 executes various processes. The ROM 104 stores programs and data for causing the CPU 102 to execute various processes, which will be described later, and setting data for the controller 21.

[0022] The image processing unit 105 performs image processing for printing on the PDL data received by the network I / F unit 101 according to the setting from the CPU 102, and generates print data that can be processed by the printer unit 206. The image processing unit 105 performs rasterization on the received PDL data in particular, thereby generating image data (RIP data) having multiple color components per pixel. Multiple color components are independent color components in a color space such as RGB (red, green, blue). The image data has, for example, 8-bit (256 gradations) values ​​for each color component for each pixel. That is, the image data is multi-value bitmap data including multi-value pixel data. In addition to the image data, the rasterization also generates attribute data indicating the attributes of the pixels of the image data for each pixel. This attribute data indicates which type of object the pixel belongs to, and is a value indicating the type of object, such as a character, line, graphic, image, or background. The image processing unit 105 uses the generated image data and attribute data to generate print data by performing image processing such as color conversion from RGB color space to CMYK (cyan, magenta, yellow, black) color space and screen processing.

[0023] The engine I / F unit 106 is an interface that transmits print data generated by the image processing unit 105 to a printer unit 206. The internal bus 107 is a system bus that connects the above-mentioned units and transmits control signals and the like.

[0024] FIG. 3 is a diagram illustrating the mechanism of the image forming apparatus 100 according to the first embodiment.

[0025] The image forming apparatus 100 includes a scanner unit 301, a laser exposure unit 302, a photosensitive drum 303, an image creation unit 304, a fixing unit 305, a paper feed / transport unit 306, and a printer control unit 308 that controls these. The scanner unit 301 illuminates a document placed on a document table to optically read the document image, and converts the image into an electrical signal to create image data. The laser exposure unit 302 makes a light beam, such as a laser beam modulated according to the image data, incident on a rotating polygon mirror 307 that rotates at a constant angular velocity, and irradiates the photosensitive drum 303 as a reflected scanning light. The image creation unit 304 rotates the photosensitive drum 303, charges it with a charger, and develops the latent image formed on the photosensitive drum by the laser exposure unit 302 with toner. The toner image is then transferred to paper, and minute toner particles that remain on the photosensitive drum without being transferred are collected at that time. The image formation is realized by having four consecutive development units (development stations) for a series of electrophotographic processes.

[0026] The four developing units, arranged in the order of cyan (C), magenta (M), yellow (Y), and black (K), sequentially perform image formation operations for magenta, yellow, and black after a predetermined time has elapsed since the start of image formation at the cyan station.

[0027] Fixing unit 305 has rollers, belts, etc., and further has a built-in heat source such as a halogen heater, and uses heat and pressure to melt and fix the toner on the paper onto which the toner image has been transferred by imaging unit 304. When printing on cardboard paper, the paper is thick and has poor thermal conductivity, so the speed at which the paper passes through fixing unit 305 must be, for example, half the normal speed. As a result, when printing on cardboard paper, the paper transport speed of each unit other than fixing unit 305 is also halved, and the printing speed of image forming apparatus 100 itself is halved.

[0028] The paper feed / transport unit 306 has one or more paper storage units such as a paper cassette or a paper deck, and in response to instructions from the printer control unit 308, separates one sheet from multiple sheets stored in the paper storage unit and transports it to the image creation unit 304. The toner images of each color are transferred onto the paper transported in this manner by the developing station described above, and finally a full-color toner image is formed on the paper. When images are formed on both sides of the paper, the paper that has passed through the fixing unit 305 is controlled to pass through a transport path that transports it again to the image creation unit 304.

[0029] Printer control unit 308 communicates with controller 21, which controls the entire image forming apparatus 100, and executes control according to instructions from the controller. Printer control unit 308 also issues instructions to ensure smooth operation in harmony with the entire apparatus, while managing the states of the above-mentioned scanner, laser exposure, image creation, fixing, and paper feed / transport sections.

[0030] FIG. 4A is a diagram illustrating an outline of the internal configuration of the inspection device 200 according to the first embodiment.

[0031] A sheet (printed matter) output from the image forming apparatus 100 is drawn into the inspection device 200 by a paper feed roller 401. Thereafter, the printed matter is read by an inspection sensor 403 on the conveyor belt 402 while being conveyed by the conveyor belt 402. An inspection device control unit 405 performs an inspection process using image data (scanned image) obtained by the inspection sensor 403 reading the printed matter. The inspection device control unit 405 also controls the entire inspection device 200. The inspection results are sent to the finisher 300. The printed matter after inspection is discharged by a paper discharge roller 404. Although not shown here, the inspection sensor 403 may be configured to read from below the conveyor belt 402 so that it can also handle double-sided printed matter.

[0032] FIG. 4B is a top view of the conveyor belt 402 as seen from the inspection sensor 403 side.

[0033] Here, the inspection sensor 403 is a line sensor that reads an image of the entire surface of the conveyed printed matter 410 line by line as shown in the figure. The irradiation device 411 irradiates the printed matter 410 when the inspection sensor 403 reads it. The skew detection irradiation device 412 is a device for detecting whether the printed matter 410 is skewed with respect to the conveying direction when the printed matter 410 is conveyed on the conveyor belt 402. The skew detection irradiation device 412 irradiates the conveyed printed matter 410 with light from an oblique direction, so that the inspection sensor 403 reads a shadow image of the edge of the printed matter 410 and detects skew of the printed matter 410. In the first embodiment, the inspection sensor 403 is configured to read the shadow image of the edge of the printed matter 410, but a reading sensor other than the inspection sensor 403 may be used.

[0034] FIG. 5 is a block diagram illustrating the configuration of the inspection device control unit 405 of the inspection device 200 according to the first embodiment.

[0035] The inspection device control unit 405 is entirely controlled by the control unit 503. The control unit 503 has a CPU 515, which loads a program stored in the memory unit 504 into the memory (not shown) of the control unit 503 and executes the program to execute various processes described below. The image input unit 501 receives a scanned image obtained by reading a printed material with the inspection sensor 403. The CPU 515 stores the received scanned image in the memory unit 504. The communication unit 502 also communicates with the controller 21 of the image forming apparatus 100. This communication involves receiving image data (reference image) used for printing that corresponds to the scanned image, and transmitting and receiving inspection control information. The CPU 515 also stores the received reference image and inspection control information in the memory unit 504.

[0036] One of the pieces of inspection control information exchanged between the image forming apparatus 100 is synchronization information for matching the scanned image (inspection image) and the reference image, such as print job information, number of copies, and page order information. The other is inspection result information and control information for controlling the operation of the image forming apparatus 100 accordingly. The synchronization information is required to synchronize the reference image and the scanned image because the order of the scanned image received by the inspection apparatus 200 and the reference image used to print the scanned image differs due to double-sided printing or printing of multiple copies. The synchronization information is also required to synchronize the reference image and the scanned image because one reference image may correspond to multiple scanned images. The inspection control information exchanged between the inspection apparatus 200 and the finisher 300 is inspection result information and control information for controlling the operation of the finisher 300 accordingly.

[0037] The operation of the inspection processing unit 513 is controlled by the CPU 515 of the control unit 503. The inspection processing unit 513 performs an inspection process to sequentially inspect pairs of corresponding scanned images and reference images based on synchronization information, which is one of the pieces of inspection control information exchanged between the image forming apparatus 100 and the inspection processing unit 513. Details of the inspection processing unit 513 will be described later. When the inspection process is completed, the judgment result is sent to the control unit 503 and displayed on the operation unit / display unit 505. If the result of this judgment is that there is an image abnormality, the control of the image forming apparatus 100 and the finisher 300 are switched through the communication unit 502 in a method designated in advance by the user via the operation unit / display unit 505. For example, the image forming process by the image forming apparatus 100 is stopped, and the paper output tray of the finisher 300 is switched to an escape tray.

[0038] Next, the configuration of the inspection processing unit 513 will be described.

[0039] The skew detection unit 506 is a module that detects the skew angle of the scanned image. As described above with reference to FIG. 4B, the scanned image is scanned so that a shadow is cast on the edge of the printed matter. This is because the inspection sensor 403 scans the shadow of the edge of the printed matter that is cast when the printed matter, which is pulled into the inspection device 200 and conveyed on the conveyor belt 402, is irradiated by the skew detection irradiation device 412. The shadow is used to detect the skew angle of the printed matter. Based on the skew angle thus detected, the image transformation unit 509, which will be described later, performs correction processing.

[0040] The color conversion unit 507 is a module that performs intermediate color conversion of the reference image. The reference image is rasterized in the CMYK color space by the image processing unit 105. On the other hand, the scanned image of the inspection target is drawn in the RGB color space read by the inspection sensor 403. Therefore, the reference image needs to be color-converted from the CMYK color space to the RGB color space, but the RGB values ​​of the scanned image vary greatly depending on the paper type on which the image of the inspection target is printed. Here, the reference image is converted to RGB values ​​common to each paper type, and then converted to RGB values ​​for each paper type by the paper type correspondence unit 511 described later. The RGB values ​​common to each paper type are, for example, three-layer data that expresses each RGB signal in 256 gradations corresponding to sRGB, which is an RGB color space independent of the printer unit 206. Note that sRGB in the first embodiment refers to the standard specification of the RGB color space established by the IEC (International Electrotechnical Commission). In this way, the color conversion unit 507 converts the reference image into an RGB image. Here, for example, the conversion may be performed using a CMYK to RGB conversion table (lookup table) as shown in FIG. 9(A). In this case, pixels on grid points are color-converted to RGB by referring to this conversion table, but pixels not on grid points have their RGB values ​​calculated by interpolation from adjacent grid points.

[0041] The resolution conversion unit 508 is a module that converts the resolution of the scanned image and the reference image. The scanned image and the reference image may have different resolutions when they are input to the inspection device control unit 405. Also, the resolution of the image used in each module of the inspection processing unit 513 may differ from the resolution of the input image. In such a case, the resolution conversion unit 508 performs the resolution conversion. For example, assume that the scanned image has 600 dpi in the main scan and 300 dpi in the sub scan, and the reference image has 1200 dpi in the main scan and 1200 dpi in the sub scan. Here, if the resolution required by the inspection processing unit 513 is 300 dpi in both the main scan and the sub scan, each image is reduced and resized to 300 dpi in both the main scan and the sub scan. The resizing method here may be a known method that takes into account the calculation load and the required accuracy. For example, if resizing is performed using the SINC function, the calculation load is heavy, but a highly accurate resizing result can be obtained. Furthermore, if scaling is performed using the nearest neighbor method, the calculation load is light, but the resulting scaling results will be low in accuracy.

[0042] The image transformation unit 509 is a module that performs image transformation of the scanned image and the reference image. There are geometric differences between the scanned image and the reference image due to the expansion and contraction of paper during printing, the skew of the paper on which the image is printed, the skew of the printed matter during scanning, and the like. The image transformation unit 509 corrects the geometric differences by performing image transformation based on information obtained by the skew detection unit 506 and the registration unit 510 described later. For example, the geometric differences are linear transformation (rotation, enlargement / reduction, shear) and parallel translation. The geometric differences can be expressed as affine transformation, and the geometric differences can be corrected by obtaining affine transformation parameters from the skew detection unit 506 and the registration unit 510. Note that the information obtained from the skew detection unit 506 is only parameters related to the rotation of the image (skew angle information).

[0043] The registration unit 510 is a module that performs registration between a scan image and a reference image. It is assumed that the scan image and the reference image input to the registration unit 510 are images with the same resolution. The higher the image resolution, the higher the registration accuracy, but the greater the calculation load. By correcting the image in the image deformation unit 509 based on the parameters obtained by the registration, it is possible to obtain the scan image and the reference image used in the matching unit 512 described later. As the registration method, various registration methods are possible, but in the first embodiment, in order to reduce the calculation load, a method of performing registration of the entire image using information on a partial area of ​​the image, rather than the entire image, is used. The registration according to the first embodiment consists of three steps: selection of a patch for registration, registration for each patch, and estimation of affine transformation parameters. Each step will be described below.

[0044] First, the selection of patches for alignment will be explained. Here, a "patch" refers to a rectangular area in an image. In selecting patches for alignment, multiple patches suitable for alignment are selected from the reference image. Patches suitable for alignment are considered to be those with large corner features. A corner feature is a feature in which there are two prominent edges with different directions in a certain local vicinity (the intersection of two edges). The corner feature is a feature that represents the strength of this edge feature. Various methods have been devised based on differences in the modeling of "edge features."

[0045] One method for calculating corner features is a well-known method called the Harris corner detection method. The Harris corner detection method calculates a corner feature image from a horizontal differential image (horizontal edge feature image) and a vertical differential image (vertical edge feature image). This corner feature image is an image that expresses the edge amount of the weaker of the two edges that make up the corner feature. Since both of the two edges of a corner feature should be strong edges, the magnitude of the corner feature is expressed by whether or not the relatively weaker edge has a strong edge amount. A corner feature image is calculated from a reference image, and a part with a large corner feature is selected as a patch suitable for alignment. If areas with large corner features are simply selected in order as patches, patches may be selected only from biased areas. In such a case, the number of areas without patches in the surrounding area increases, and the information of the area cannot be used for image deformation, so it is not suitable for aligning the entire image.

[0046] Therefore, when selecting a patch, consideration is given not only to the size of the corner feature amount but also to how the patch is distributed within the image. Specifically, even if the corner feature amount of a certain patch candidate region is not large in the entire image, if the value in a local region of the image is large, the patch is selected. This makes it possible to distribute the patches within the reference image. Parameters for patch selection include the size of the patch and the number of patches (or density). As the patch becomes larger and the number of patches increases, the accuracy of alignment improves, but the calculation load increases.

[0047] Next, we will explain the alignment for each patch. In the alignment for each patch, alignment is performed between the alignment patch in the reference image selected in the previous step and the corresponding patch in the scanned image.

[0048] As a result of the alignment, two types of information can be obtained. The first is the center coordinates (refpX_i, refpY_i) of the alignment patch in the i-th (i=1 to N, N is the number of patches) reference image. The second is the position (scanpX_i, scanpY_i) of the center coordinates in the scanned image. The alignment method may be any method that estimates the amount of shift so long as it can obtain the relationship between (refpX_i, refpY_i) and (scanpX_i, scanpY_i). For example, a method may be used in which the alignment patch and the corresponding patch are brought into frequency space using FFT (Fast Fourier Transform) and the correlation there is taken to estimate the amount of shift.

[0049] Finally, we will explain how to estimate the affine transformation parameters. The affine transformation is a coordinate transformation method expressed by the equation in FIG.

[0050] In this formula, there are six types of affine transformation parameters: a, b, c, d, e, and f. Here, (x, y) corresponds to (refpX_i, refpY_i), and (x', y') corresponds to (scanpX_i, scanpY_i). The affine transformation parameters are estimated using this correspondence obtained from N patches. For example, it is possible to obtain the affine transformation parameters using the least squares method. Based on the obtained affine transformation parameters, the image deformation unit 509 deforms the reference image or the scanned image. In this way, an image after alignment correction is created, and it becomes possible to make a set of the reference image and the scanned image used by the matching unit 512.

[0051] The paper type correspondence unit 511 is a module that performs processing on the reference image according to the paper type. Specifically, it performs color conversion according to the paper type and image rotation according to the paper orientation. This module performs image processing on the reference image according to the paper type and orientation based on the inspection control information stored in advance in the memory unit 504 and the paper information used for printing transmitted from the controller 21 of the image forming apparatus 100. Regarding color conversion, color conversion according to the paper type is performed based on the reference image converted to an RGB color space common to the paper type by the color conversion unit 507. The color conversion is performed by targeting the RGB values ​​read by the inspection sensor 403 for each paper type. The color conversion according to the paper type may be performed using, for example, an RGB to RGB color conversion table (lookup table) as shown in FIG. 9B. Note that a plurality of lookup tables may be provided and switched according to the paper type.

[0052] Then, the paper type correspondence unit 511 rotates the reference image. Here, the image is rotated 90 degrees to match the orientation of the notified printout (paper). For example, as shown in Fig. 7, even with the same data, a scanned image is acquired in which the image is rotated depending on the orientation of the paper stored in the paper storage. For this reason, it is necessary to rotate the reference image to match the orientation of that scanned image.

[0053] The collation unit 512 is a module that compares the scanned image with a reference image processed by the paper type correspondence unit 511. The scanned image and reference image input to the collation unit 512 are images with the same resolution. In addition, it is a prerequisite that the reference image or scanned image has been corrected by the image transformation unit 509 based on the information obtained by the position adjustment unit 510 so that the images can be compared. The collation unit 512 executes the collation process using the reference image and the scanned image. At this time, the collation process is performed based on parameters notified from the operation unit / display unit 505.

[0054] The operation unit / display unit 505 is a touch screen user interface, and accepts from the user settings for processing in the inspection processing unit 513. For example, the operation unit / display unit 505 displays a setting screen as shown in Fig. 8, and accepts settings for image processing by the inspection processing unit 513 from the user.

[0055] FIG. 8 is a diagram showing an example of an inspection setting screen displayed on the operation unit / display unit 505 of the inspection device 200 according to the first embodiment.

[0056] Here, buttons 801 to 805 are provided as test settings that can be adjusted by the user, and these buttons 801 to 805 correspond to Settings 1 to 5. The user can select any of these buttons and press a "Yes" button 806 to set the test setting corresponding to the selected button. A "No" button 807 is a button for canceling the settings on this screen.

[0057] Here, when the button 801 is selected to set "Setting 1", the collation unit 512 detects an abnormality when the color difference of stains, scratches, etc. determined by the inspection of the scanned image is "5" or more. On the other hand, when the button 805 is selected to set "Setting 5", the collation unit 512 detects an abnormality when the color difference of stains, scratches, etc. determined by the inspection of the scanned image is "50" or more. In this way, the smaller the setting number of the inspection setting, the more the collation unit 512 detects an abnormality even if the color difference of a slight stain, scratch, etc. is. In this way, the user can set the inspection in the collation unit 512 by selecting a button corresponding to one of Settings 1 to 5. Note that color difference parameters are previously associated with each setting value. Therefore, the color difference parameters corresponding to the setting value selected by the user on the inspection setting screen are notified to the collation unit 512.

[0058] In the first embodiment, the color difference of the stains and scratches to be detected is adjusted by the inspection setting value, but the present invention is not limited to this, and may be, for example, the size of the stains and scratches to be detected. For example, the size may be adjustable within a range of 0.1 mm to 3 mm.

[0059] The inspection process will be described below with reference to FIG.

[0060] 6 is a flowchart for explaining the inspection process by the inspection device 200 according to the embodiment 1. Note that the process shown in this flowchart is realized by the CPU 515 of the control unit 503 executing a program stored in the memory unit 504.

[0061] First, in S601, the CPU 515 determines from the inspection control information whether the data to be printed in the current print job is data that has been printed before. To determine whether the data is data that has been printed before, for example, a reprint flag may be prepared in the inspection control information. If it is determined that the data is data that has been printed before, the process proceeds to S612, and if it is determined that the data is not data that has been printed before, the process proceeds to S602. In S602, the CPU 515 receives the print job via the communication unit 502, acquires the data of the print job, and stores it in the memory unit 504.

[0062] Next, the CPU 515 performs pre-processing of S603 to S605. The CPU 515 processes the reference image received from the image forming apparatus 100 and held in the memory unit 504 via the communication unit 502. The processing here is paper-independent, and the resulting reference image is stored again in the memory unit 504. First, in S603, the CPU 515 performs color conversion on the reference image. Here, the color conversion unit 507 converts the reference image into RGB values ​​common to each paper type. Next, the process proceeds to S604, where the CPU 515 performs resolution conversion on the reference image. Here, the resolution conversion unit 508 converts the image into a resolution required by the inspection processing unit 513. Then, the process proceeds to S605, where the CPU 515 stores the converted reference image in the memory unit 504 together with an ID (identification information). This ID is included in the inspection control information, and it is sufficient that the reference image is uniquely identified, and may be, for example, a string of numbers such as "202208010001" that indicates the printing date and time. Also for example, the identification information of the reference image may be specified in the print job that uses the reference image.

[0063] It is assumed that the reference image input here is a CMYK image with 1200 dpi in the main scan direction and 1200 dpi in the sub-scan direction. Through the above-mentioned pre-processing, the image is converted into an RGB image with 300 dpi in both the main scan and sub-scan directions, which is common to all paper types, and the image size becomes very small. Specifically, the resolution is reduced to 1 / 16 and the number of channels is reduced by one. As a result, the capacity of the memory unit 504 can be kept small.

[0064] Next, in S606 to S609, CPU 515 performs pre-processing so that the reference image becomes an image that can be compared with the scanned image of the image printed in the input job. In S606, CPU 515 selects an image pair of the scanned image to be inspected and the reference image, using the inspection control information received from image forming apparatus 100 held in memory unit 504 via communication unit 502. Here, the corresponding reference image is acquired from memory unit 504 based on the ID described in the inspection control information. Then, based on the paper information used for printing transmitted from controller 21, color conversion is performed according to the paper type so that the scanned image and the reference image can be compared. Here, the paper information used for printing is notified after the print job is executed, because the paper type and orientation are not determined until the print job is executed.

[0065] Next, in S607, the CPU 515 performs color conversion processing to match the paper type of the printed matter using the paper type correspondence unit 511. Here, the reference image with RGB values ​​common to all paper types is converted into a color space that matches the paper type, enabling accurate matching with the scanned image.

[0066] Next, the process proceeds to S608, where CPU 515 rotates the reference image to match the orientation of the paper in paper type correspondence unit 511. The orientation of the paper is obtained from the inspection control information, and the reference image is rotated so that the orientations of the scanned image and the reference image match. Next, the process proceeds to S609, where CPU 515 aligns the scanned image and the reference image. Here, CPU 515 first processes the scanned image and the reference image in alignment unit 510 to obtain affine transformation parameters. Then, CPU 515 uses the affine transformation parameters obtained from alignment unit 510 to perform correction processing of the reference image in image transformation unit 509 to make the coordinate system of the reference image the same as that of the scanned image, thereby making it a reference image that can be used for matching.

[0067] Then, the process proceeds to S610, and the CPU 515 performs a collation and judgment process using the scan image and the reference image obtained in S609. At this time, the CPU 515 processes the scan image and the reference image in the collation unit 512. At this time, the reference value for judging whether or not there is an image abnormality is based on the setting value set in the inspection setting screen of FIG. 8 described above. Next, the process proceeds to S611, and the CPU 515 displays the result of the inspection process in S610 on the operation unit / display unit 505. At this time, if the image of the final judgment result is simply displayed, it is difficult for the user to understand what kind of image abnormality occurred. Therefore, the final judgment result image is composited with the scan image and displayed on the operation unit / display unit 505. This composition may be any composition method as long as it is a composition method that makes it easy to understand the location of the image abnormality. For example, in the image of the final judgment result, the part of the image abnormality may be displayed in red on the scan image.

[0068] On the other hand, if it is determined in S601 that the data to be printed in the current print job is previously printed data, the process proceeds to S612 where the CPU 515 determines whether a reference image of the previously printed data is stored in the memory unit 504. This determination can be made by checking whether the ID of the reference image described in the inspection control information is stored in the memory unit 504. If there is a matching ID, the process proceeds to S606 using the reference image corresponding to that ID, and if not, the process proceeds to S602 to generate a reference image.

[0069] Here, the case of previously printed data will be described. When previously printed data is used, the process proceeds to S606 using the data after pre-processing stored in memory unit 504 in S605 to generate a reference image. Whether the data is previously printed or not can be identified by determining in S612 whether the ID stored together with the reference image matches the ID described in the inspection control information of the input print job. Even if the paper type or paper orientation has been changed from the previous printing in this input print job, a reference image common to the paper type generated in the previous printing is generated and saved. Therefore, by using this saved reference image, a reference image corresponding to the paper of the print to be inspected can be generated with a minimum of processing.

[0070] Incidentally, a storage limit may be set for the stored reference image common to all paper types, taking into consideration the capacity of the memory 504 and changes in the profile in accordance with the state of the image forming apparatus. In this way, the available storage area of ​​the storage means may be secured by deleting from the storage means reference images generated before a certain period of time.

[0071] As described above, according to the embodiment, a reference image common to paper types generated in previous printing is saved and can be used in a subsequent corresponding job. As a result, if a subsequent print job corresponds to the saved reference image, even if the paper type or paper orientation is changed in the subsequent print job, image matching can be performed on the corresponding reference image by simply performing processing for the change in the paper type or paper orientation. This reduces the time required to generate a reference image, and allows for quick matching of the scanned image with the reference image.

[0072] (Modification of the first embodiment) Image processing according to a modified example of the first embodiment of the present invention will be described below.

[0073] In the above-mentioned first embodiment, a specific example of when reprinting is executed is described, but the paper type and orientation may change not only depending on the job. In the image forming apparatus 100, if the paper runs out during printing, a different paper storage may be automatically selected. Even if the paper type is different, a similar type of paper may be automatically fed, or a paper with a different orientation may be fed.

[0074] Therefore, in this modified example, not only when a job is submitted, but also when the paper storage unit used for printing in the image forming device is automatically or manually switched, a determination is made as to whether a corresponding reference image is stored, and if so, processing similar to that of the above-described embodiment 1 is performed.

[0075] Fig. 10 is a flowchart for explaining the inspection process by the inspection device 200 according to this modified example. The process shown in this flowchart is realized by the CPU 515 of the control unit 503 executing a program stored in the memory unit 504. In Fig. 10, the same processes as those in Fig. 6 described above are denoted by the same reference numerals, and the description thereof will be omitted.

[0076] First, in S1001, the CPU 515 determines whether or not a paper out state has occurred. If a paper out state has occurred, the process proceeds to S1002, where the process waits for paper to be replenished or for the paper storage unit to be used to be switched to resolve the paper out state. If paper is replenished or the paper storage unit to be used is switched in this way, the process proceeds to S1003, where the CPU 515 determines whether the newly fed paper is of the same type. If the paper type is different, the process proceeds to S1005, but if the paper type is the same, the process proceeds to S1004, where the process determines whether the paper orientation is the same as the paper orientation that was printed up until then. If the paper orientation is determined to be different, the process proceeds to S1005, and if not, the process proceeds to S601. In S1005, as in S612 in FIG. 6, the process determines whether or not a reference image corresponding to the print data is stored, and if a reference image is stored, the process proceeds to S606 in FIG. 6 to execute color conversion to match the new paper type. Although S601 is provided here, S601 may be omitted and the process may proceed to S602. Also, in S1005, if a corresponding reference image exists and only the paper type is different, the process may proceed to S606, and if only the paper orientation is different, the process may proceed from S1005 to S607.

[0077] As a result, even if the paper type or paper orientation is changed during printing, image matching can be performed by simply performing processing for the change in paper type or paper orientation on the corresponding reference image. This allows efficient generation processing of the reference image while maintaining high speed when performing image matching.

[0078] In addition, a reference image common to paper types generated in previous printing may not only be stored in the device itself, but may also be sent to other inspection devices so that the reference image can be used by multiple inspection devices or inspection systems.

[0079] Also, by allowing the user to select whether or not to create and save a common reference image, if the reference image is not saved, it is possible to immediately perform resolution conversion (S604), color conversion according to the paper type (S606), and color conversion according to the paper orientation (S607).

[0080] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0081] This specification and drawings disclose the following inspection apparatus, its control method, and program.

[0082] [Item 1] An inspection device for inspecting an image printed on a sheet of paper, a first image processing means for performing a first image processing on image data used for printing the image; a storage means for storing the image data that has been subjected to the first image processing by the first image processing means as a first reference image; a determining means for determining whether the first reference image corresponding to a reference image that can be used in the job to be inspected is stored in the storage means; a second image processing means for performing a second image processing on the first reference image to convert the first reference image into a second reference image corresponding to the job when the determination means determines that the corresponding first reference image is stored; a reading unit for reading an image on a sheet of paper printed in accordance with the job to obtain a scanned image; A comparison unit that performs a comparison process between the scanned image acquired by the reading unit and the second reference image; An inspection device comprising:

[0083] [Item 2] 2. The inspection device according to item 1, wherein the storage means stores image data that has been subjected to the first image processing, including identification information of the first reference image.

[0084] [Item 3] The inspection device described in item 2, characterized in that the determination means determines whether the corresponding first reference image is stored based on inspection control information and the identification information included in the job.

[0085] [Item 4] The inspection device described in any one of items 1 to 3, characterized in that the determination means determines that the corresponding first reference image is stored when inspection control information included in the job indicates reprinting.

[0086] [Item 5] 5. The inspection device according to any one of items 1 to 4, wherein the first image processing includes a process of converting a color space of the image data into image data of a color space common to a plurality of paper types.

[0087] [Item 6] 6. The inspection device according to item 5, wherein the image data in a color space common to the plurality of paper types is image data in an RGB color space that is independent of the printer engine used to print the paper.

[0088] [Item 7] 7. The inspection device according to any one of items 1 to 6, wherein the first image processing includes a process of converting a resolution of the image data.

[0089] [Item 8] The inspection device described in any one of items 1 to 7, characterized in that the second image processing includes a process of converting the first reference image into the second reference image in a color space corresponding to the type of paper printed according to the job.

[0090] [Item 9] An inspection device as described in any one of items 1 to 8, characterized in that the second image processing includes a process of converting the first reference image into the second reference image that is aligned with the orientation of the paper printed according to the job.

[0091] [Item 10] 10. The inspection device according to any one of items 1 to 9, wherein the second image processing includes a registration process for aligning the scanned image with the second reference image.

[0092] [Item 11] The printing method further includes a detection unit for detecting skew of a sheet printed according to the job, Item 11. The inspection device according to item 10, wherein the alignment process aligns the scanned image with the second reference image based on the skew of the paper detected by the detection means.

[0093] [Item 12] 12. The inspection device according to any one of items 1 to 11, further comprising: a means for, when the determination means determines that the corresponding first reference image is not stored, applying a second image processing to the first reference image generated by the first image processing means to perform the first image processing, thereby generating the second reference image.

[0094] [Item 13] The inspection device described in any one of items 1 to 12, characterized in that the determination means determines whether the corresponding first reference image is stored in response to switching of a paper storage cabinet that supplies paper to be printed according to the job.

[0095] [Item 14] The inspection device described in any one of items 1 to 12, characterized in that the determination means determines whether the corresponding first reference image is stored in response to a change in the type of paper to be printed according to the job.

[0096] [Item 15] The inspection device described in any one of items 1 to 12, characterized in that the determination means determines whether the corresponding first reference image is stored in response to a change in orientation of the paper to be printed according to the job.

[0097] [Item 16] 16. The inspection device according to any one of items 1 to 15, wherein the first reference image has a data volume smaller than that of image data used to print the image.

[0098] [Item 17] 17. The inspection device according to any one of items 1 to 16, further comprising a setting means for setting a reference value for determining the presence or absence of an image abnormality based on a result of the comparison by the comparison means.

[0099] [Item 18] 18. The inspection device according to item 17, further comprising a display means for displaying a result of the determination of the presence or absence of the image abnormality.

[0100] [Item 19] A control method for controlling an inspection device that inspects an image printed on a sheet, comprising: a first image processing step of performing a first image processing on image data used for printing the image; a storage step of storing the image data subjected to the first image processing in the first image processing step as a first reference image; a determining step of determining whether the first reference image corresponding to a reference image that can be used in the job to be inspected is stored in the storage step; a second image processing step of performing a second image processing on the first reference image to convert the first reference image into a second reference image corresponding to the job when the determining step determines that the corresponding first reference image is stored; a reading step of reading an image on a sheet of paper printed in accordance with the job to obtain a scanned image; a comparison step of comparing the scanned image acquired by the reading step with the second reference image; A control method comprising the steps of:

[0101] [Item 20] A program that causes a computer to function as each of the means of the inspection device described in any one of items 1 to 18.

[0102] The present invention is not limited to the above-described embodiments, and various modifications and variations can be made without departing from the spirit and scope of the present invention. Therefore, the following claims are appended to apprise the public of the scope of the present invention. [Explanation of symbols]

[0103] 100... image forming apparatus, 200... inspection device, 300... finisher, 403... inspection sensor, 411... irradiation device, 412... irradiation device for detecting skew, 503... control unit, 504... memory unit, 506... skew detection unit, 507... color conversion unit, 510... alignment unit, 511... paper type correspondence unit, 512... collation unit, 513... inspection processing unit, 515... CPU

Claims

1. An inspection device that inspects an image printed on paper, a first image processing means for performing a first image process on image data used to print the image; a storage means for storing the image data that has been subjected to the first image processing by the first image processing means as a first reference image; a determining means for determining whether the first reference image corresponding to a reference image that can be used in the job to be inspected is stored in the storage means; a second image processing means for performing second image processing on the first reference image to convert the first reference image into a second reference image corresponding to the job when the determination means determines that the corresponding first reference image is stored; a reading unit for reading an image on a sheet of paper printed in accordance with the job to obtain a scanned image; a comparison unit that performs a comparison process between the scanned image acquired by the reading unit and the second reference image; An inspection device comprising:

2. 2. The inspection device according to claim 1, wherein the storage means stores the image data that has been subjected to the first image processing, including identification information of the first reference image.

3. 3. The inspection device according to claim 2, wherein the determining means determines whether the corresponding first reference image is stored based on the inspection control information and the identification information included in the job.

4. 2. The inspection device according to claim 1, wherein the determining unit determines that the corresponding first reference image is stored when inspection control information included in the job indicates reprinting.

5. 2. The inspection device according to claim 1, wherein the first image processing includes a process of converting a color space of the image data into image data in a color space common to a plurality of paper types.

6. 6. The inspection device according to claim 5, wherein the image data in a color space common to the plurality of paper types is image data in an RGB color space that is independent of the printer engine used to print the paper.

7. The inspection device according to claim 1 , wherein the first image processing includes a process of converting the resolution of the image data.

8. 2. The inspection device according to claim 1, wherein the second image processing includes converting the first reference image into the second reference image in a color space corresponding to the type of paper printed according to the job.

9. The inspection device according to claim 1 , wherein the second image processing includes converting the first reference image into the second reference image that matches the orientation of the paper printed in accordance with the job.

10. The inspection device according to claim 1 , wherein the second image processing includes a registration process for aligning the scanned image with the second reference image.

11. The printer further includes a detection unit for detecting skew of paper printed according to the job, 11. The inspection device according to claim 10, wherein the alignment process aligns the scanned image with the second reference image based on the skew of the paper detected by the detection unit.

12. The inspection device described in claim 1, further comprising a means for, when the determination means determines that the corresponding first reference image is not stored, performing second image processing on the first reference image generated by the first image processing means to generate the second reference image.

13. The inspection device according to claim 1, characterized in that the determination means determines whether the corresponding first reference image is stored in response to switching of a paper storage cabinet that supplies paper to be printed according to the job.

14. 2. The inspection device according to claim 1, wherein the determining unit determines whether the corresponding first reference image is stored in response to a change in the type of paper to be printed in accordance with the job.

15. 2. The inspection device according to claim 1, wherein the determining means determines whether the corresponding first reference image is stored in response to a change in the orientation of the paper to be printed in accordance with the job.

16. 2. The inspection device according to claim 1, wherein the first reference image has a data volume smaller than that of the image data used to print the image.

17. 2. The inspection device according to claim 1, further comprising a setting means for setting a reference value for determining the presence or absence of an image abnormality based on the result of the comparison by the comparison means.

18. 18. The inspection device according to claim 17, further comprising a display means for displaying the result of the determination of the presence or absence of the image abnormality.

19. A control method for controlling an inspection device that inspects an image printed on paper, comprising: a first image processing step of performing a first image process on the image data used to print the image; a storage step of storing the image data subjected to the first image processing in the first image processing step as a first reference image; a determining step of determining whether the first reference image corresponding to a reference image that can be used in the job to be inspected is stored in the storing step; a second image processing step of, when the determining step determines that the corresponding first reference image is stored, performing second image processing on the first reference image to convert the first reference image into a second reference image corresponding to the job; a reading step of reading an image on a sheet of paper printed in accordance with the job to obtain a scanned image; a comparison step of comparing the scanned image acquired by the reading step with the second reference image; A control method comprising:

20. A program that causes a computer to execute a control method for controlling an inspection device that inspects an image printed on paper, The control method includes: a first image processing step of performing a first image process on the image data used to print the image; a storage step of storing the image data subjected to the first image processing in the first image processing step as a first reference image; a determining step of determining whether the first reference image corresponding to a reference image that can be used in the job to be inspected is stored in the storing step; a second image processing step of, when the determining step determines that the corresponding first reference image is stored, performing second image processing on the first reference image to convert the first reference image into a second reference image corresponding to the job; a reading step of reading an image on a sheet of paper printed in accordance with the job to obtain a scanned image; a comparison step of comparing the scanned image acquired by the reading step with the second reference image; A program comprising:

Images