Image processing device, image processing method, and program

Abstract

To suppress the occurrence of an erroneous inspection by preventing a failure of positioning by nonlinear transformation.SOLUTION: In positioning an inspection image acquired by reading a print and a reference image, only when a fixed condition is satisfied after positioning by linear transformation, the inspection image acquired by positioning by linear transformation is further subjected to positioning by nonlinear transformation.SELECTED DRAWING: Figure 14

Description

Technical Field

[0001] The present disclosure relates to printed matter inspection technology. Background Art

[0002] In printed matter output from a printing apparatus, stains such as adhesion of color materials such as ink and toner to unintended locations may occur. Alternatively, insufficient color material may adhere to a location where an image is to be formed, resulting in color omission where the color becomes lighter than it should originally be. These so-called printing defects, such as stains and color omissions, degrade the quality of printed matter. In order to guarantee the quality of printed matter, inspection for printing defects is performed.

[0003] Visual inspection, in which an inspector visually checks for the presence or absence of printing defects, requires a lot of time and cost. Therefore, inspection systems that perform automatic inspection without relying on visual inspection have been proposed. In an inspection system, there is a method in which a scanned image (inspection image) obtained by scanning a printed matter to be inspected is compared with an image (reference image) registered in advance as an inspection standard, and the presence or absence of printing defects is inspected based on the difference between the two images. In the case of such an inspection method by comparing images with each other, alignment of the two images greatly affects inspection accuracy, so it is important to perform alignment with high precision. As a common alignment technique, alignment by linear transformation using affine transformation or the like is known. However, alignment by linear transformation has the problem that it cannot cope with local distortion (partial magnification variation) caused by conveyance unevenness or paper stretching, resulting in decreased alignment accuracy. In this regard, as a technique for aligning two images with different local distortions, there is an alignment technique using non-linear transformation represented by, for example, FFD (Free-Form Deformations). Prior Art Documents Patent Documents

[0004] Patent Document 1 Japanese Unexamined Patent Publication No. 2014-117841 [Overview of the project] [Problems that the invention aims to solve]

[0005] In FFD (Front-Field Display), multiple control points are placed in an image, and corresponding control points are searched for in both the reference image and the inspection image. The image is then deformed to minimize the discrepancy between these corresponding control points. However, if the objects in the inspection image do not match between the reference image, for example, if the inspection image contains characters or shapes that do not exist in the reference image, the system may misidentify the corresponding control points. This can lead to the image being deformed to minimize the discrepancy between control points that do not actually correspond. As a result, the inspection is performed with the two images misaligned, leading to the problem of misinspections where printing defects are incorrectly detected.

[0006] This disclosure is made in view of the above points, and aims to prevent alignment failures due to nonlinear transformations and suppress the occurrence of erroneous inspections. [Means for solving the problem]

[0007] The inspection device relating to this disclosure is an inspection device for inspecting printed matter output from a printing device, and reads the printed matter And I got The system includes an alignment means for aligning an inspection image with a reference image used as a standard in the inspection, wherein the alignment means performs alignment on the inspection image using a linear transformation. The aforementioned reference image is set to image data used for the printing process of the printed material. In this case, the inspection image obtained by the linear transformation is further subjected to a nonlinear transformation. [Effects of the Invention]

[0008] According to this disclosure, it is possible to prevent alignment failures due to nonlinear transformations and suppress the occurrence of erroneous inspections. [Brief explanation of the drawing]

[0009] [Figure 1]FIG. 1 is a diagram illustrating an example configuration of an inspection system. [Figure 2] FIG. 2 is a functional block diagram illustrating an internal configuration of an image processing unit. [Figure 3] FIG. 3 is a diagram illustrating an example of a UI screen for registering paper information. [Figure 4] FIG. 4 is a schematic diagram illustrating a printing process when a printed object already exists on a paper sheet. [Figure 5] FIG. 5 is a diagram illustrating an example of a case where association of control points fails. [Figure 6] FIG. 6 is a schematic diagram illustrating a printing process when no printed object exists on a paper sheet. [Figure 7] FIG. 7 is a flowchart illustrating a procedure of inspection processing. [Figure 8] FIG. 8 is a flowchart illustrating details of distortion information generation processing. [Figure 9] FIG. 9 is a diagram illustrating an example of a distortion correction chart. [Figure 10] FIG. 10 is a diagram illustrating an example of distortion information. [Figure 11] FIG. 11 is a flowchart illustrating details of defect detection processing. [Figure 12] (a) and (b) are diagrams illustrating filters. [Figure 13] FIG. 12 is a diagram illustrating an example of a UI screen displaying inspection results. [Figure 14] FIG. 13 is a flowchart illustrating details of alignment processing according to the first embodiment. [Figure 15] (a) and (b) are diagrams illustrating feature points in alignment. [Figure 16] FIG. 14 is a diagram illustrating an example of a case where control points are arranged in a grid. [Figure 17] (a) and (b) are diagrams illustrating the difference between a case where a reference image for variable printing is scan data and a case where the reference image is RIP data. [Figure 18] FIG. 15 is a diagram illustrating an example of a UI screen for registering reference image information. [Figure 19] FIG. 16 is a flowchart illustrating details of alignment processing according to the second embodiment. [Figure 20]A flowchart showing details of alignment processing according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the present invention according to the claims, and not all combinations of features described in the present embodiment are essential to the solution of the present invention. The same constituent elements are denoted by the same reference numerals, and descriptions thereof are omitted. Embodiment 1

[0011] <Overall System Configuration> FIG. 1 is a diagram showing an example configuration of an inspection system according to the present embodiment. The inspection system 100 shown in FIG. 1 includes a server 101, a printing apparatus 102, and an inspection apparatus 105. In the inspection system 100, the printing apparatus 102 performs printing processing based on a print job generated by the server 101 to output a printed matter, and the inspection apparatus 105 inspects the printed matter for the presence or absence of defects.

[0012] The server 101 generates a print job and transmits the generated print job to the printing apparatus 102. A client terminal (not shown) is communicably connected to the server 101 via a network. The server 101 receives a print job generation request or the like from the client terminal, generates a print job, and inputs the print job to the printing apparatus 102.

[0013] The printing device 102 performs printing operations to form an image on paper based on a print job received from the server 101. In this embodiment, an electrophotographic printing method is assumed, but other printing methods such as offset printing or inkjet printing may also be used. The printing device 102 includes a paper feed unit 103. The paper feed unit 103 is pre-loaded with paper according to the user's purpose. In this specification, "paper" is a concept that includes plastic sheets and is not limited to paper in the narrow sense. Based on the print job received from the server 101, the printing device 102 transports the paper loaded in the paper feed unit 103 along the transport path 104, forms an image on one or both sides of the paper, and outputs the paper with the image formed on it (i.e., the printed material) to the inspection device 105.

[0014] The inspection device 105 includes a CPU 106, RAM 107, ROM 108, main memory unit 109, image reading unit 110, printer interface 111, general-purpose interface 112, UI panel 113, and image processing unit 118, which are connected to each other via a main bus 114. The inspection device 105 also includes a transport path 115 connected to the transport path 104 of the printer 102, an output tray 116, and an output tray 117.

[0015] The CPU 106 is a processor that controls the entire inspection device 105. The RAM 107 functions as the main memory and work area of ​​the CPU 106. The ROM 108 stores multiple programs executed by the CPU 106. The main memory unit 109 stores applications executed by the CPU 106 and data used for image processing. The image reading unit 110 performs a scanning process to optically read one or both sides of the printed material to be inspected, output from the printing device 102, and generates a scanned image of the printed material. For example, one or more reading sensors (not shown) provided near the transport path 115 are used to read one or both sides of the transported printed material. The reading sensors may be provided only on one side, or they may be provided on both the front and back sides of the transported printed material in order to read both sides simultaneously. In a configuration where the reading sensor is provided on only one side of the printed material, the printed material, after reading one side, is transported to a double-sided transport path (not shown) in the transport path 115, where the front and back sides of the printed material are reversed and the reading sensor reads the other side.

[0016] The image processing unit 118 compares the image obtained by scanning the printed material to be inspected, generated by the image reading unit 110 (hereinafter referred to as the "inspection image"), with a pre-registered image that serves as a reference for inspection (hereinafter referred to as the "reference image") to check for printing defects. Prior to inspection, it also aligns the inspection image with the reference image. The reference image is sometimes referred to as the "correct image." Details of the image processing unit 118 will be described later.

[0017] The printer interface 111 adjusts the timing (synchronization) for processing printed materials output from the printer 102 and notifies each other of their operating status. The general-purpose interface 112 is a serial bus interface such as USB or IEEE1394. For example, by connecting a USB memory stick to the general-purpose interface 112, data such as logs stored in the main memory unit 109 can be written to the USB memory stick and taken out, or data stored in the USB memory stick can be imported into the inspection device 105. The UI panel 113 is, for example, a liquid crystal display (display unit) with touch panel functionality. The UI panel 113 functions as the user interface for the inspection device 105, displaying the current status and settings to the user. The user can also input various instructions by directly operating buttons displayed on the liquid crystal display.

[0018] <Overview of the inspection device> The inspection device 105, having received the printed material output from the transport path 104 of the printing device 102, reads the printed material with the image reading unit 110 and generates a scanned image of the printed material. If the scanned image generated by the image reading unit 110 is a scanned image of a distortion correction chart, the image processing unit 118 compares the scanned image of the chart (hereinafter referred to as the "chart image") with a reference image that serves as the basis for the chart and generates distortion information used for alignment by nonlinear transformation. On the other hand, if the scanned image generated by the image reading unit 110 is a scanned image of the printed material to be inspected (inspection image), the image processing unit 118 checks for the presence or absence of printing defects based on the difference obtained by comparing the inspection image with the reference image. Here, there are two methods of inspection: a RIP inspection method that uses RIP data used in the printing process as the reference image, and a scan inspection method that uses scanned data obtained by scanning a product sample, etc. However, in this embodiment, the explanation will be based on the RIP inspection method. Printing defects include smudges where ink or toner adheres to unintended areas, and color fading where insufficient colorant adheres to areas where an image should be formed, resulting in lighter colors than intended. All of these degrade the quality of the printed material. The inspection device 105 outputs printed materials that pass the inspection to the output tray 116 and printed materials that fail the inspection to the output tray 117. In this way, only printed materials that meet a certain quality standard can be collected in the output tray 116 as deliverables.

[0019] <Details of Image Processing Unit 118> Figure 2 is a functional block diagram showing the internal configuration (software configuration) of the image processing unit 118. The image processing unit 118 includes software modules 201 to 208, each responsible for image acquisition, selection of inspection items, generation of distortion information, image alignment, determination of the operation mode during alignment, setting of inspection parameters, execution of inspection, and output of inspection results. The functions of each of these modules are realized by the CPU 106 reading the program stored in the ROM 108 into the RAM 107 and executing it.

[0020] The image acquisition module 201 acquires inspection images or chart images from the image reading unit 110. The image acquisition module 201 also acquires pre-registered reference images from the RAM 107 or main memory unit 109.

[0021] The inspection item setting module 202 sets the inspection items, specifying which types of printing defects to inspect, based on user selections via an inspection setting UI screen (not shown) displayed on the UI panel 113. The inspection setting UI screen allows the user to select the type of paper to be used for printing, the paper size, whether to print on one side or both sides, and the type of printing defect to be inspected. The inspection item setting module 202 sets the target of execution a defect detection process that can detect the selected type of printing defect from among the multiple types of defect detection processes that the inspection module 207 can execute. Examples of detectable printing defect types include dot-shaped defects (spots), line-shaped defects (streaks), surface-shaped defects, and image irregularities.

[0022] The mode determination module 204 determines the operation mode of the alignment process based on paper information indicating whether any characters or shapes (objects) are already printed on the paper to be used. Paper information is registered according to user selection via a UI screen as shown in Figure 3, which is displayed on the UI panel 113. The UI screen 300 shown in Figure 3 has two types of radio buttons: button 301 to select when characters or shapes such as a company logo are already printed on the paper, and button 302 to select when no such printed objects exist (blank paper). Here, the cases where printed objects exist and where they do not will be explained using specific examples.

[0023] Figure 4 is a schematic diagram illustrating the printing process when there are pre-printed objects on the paper. In Figure 4, image 401 is raster image data obtained by interpreting the PDL (Page Description Language) included in the print job with a RIP (Raster Image Processor). Hereafter, this image data will be referred to as "RIP data". Currently, a postal code box 402a is pre-printed on paper 402, and as a result of printing using image 401 on this paper 402, a printed document 403 is obtained. Image 404 is a scanned image obtained by reading the printed document 403. In this embodiment of the RIP inspection method, prior to inspection, alignment is performed between image 401 as a reference image and the scanned image 404 as an inspection image. Currently, the reference image 401 contains three objects 401a to 401c: a sun, a human face, and clouds, but there is no object corresponding to the postal code box 402a. Therefore, in free-form deformation (FFD) operations targeting reference image 401 and inspection image 404, it is possible that control points cannot be properly associated between the faces 401b and 404b of the same object. Figure 5 shows the result of FFD when control points are incorrectly associated between the face 401b of the person in reference image 401 and the postal code box 404d in inspection image 404, and it can be seen that the face of the person is distorted. In this way, associating control points between different objects will result in a failure of alignment by FFD.

[0024] Figure 6 is a schematic diagram illustrating the printing process when there are no printed objects on the paper. In Figure 6, image 601 is a raster image obtained by interpreting the PDL included in the print job, and is the same as image 401 in Figure 4 above. Now, nothing is printed on the paper 602; it is blank. When the printing process is performed on this paper 602 using image 601, a printed object 603 is obtained. Image 604 is a scanned image obtained by reading the printed object 603. In the RIP inspection method, as described above, image 601 is used as a reference image, and alignment is performed between it and the scanned image 604, which is the inspection image. Now, since the paper 602 is blank, the reference image 601 contains three objects 601a to 601c: the sun, a person's face, and clouds, and similarly, the inspection image 604 contains three objects 604a to 604c: the sun, a person's face, and clouds. In other words, the objects in both images are perfectly identical. Therefore, it is unlikely that control points will be incorrectly mapped between different objects, and the possibility of failure in FFD alignment is reduced.

[0025] Based on the above facts, in this embodiment, the decision to perform FFD (Format Field Diode) during the alignment process is made based on the paper information registered by the user, which indicates whether or not there are printed objects on the paper. That is, if the paper information indicates that there are no printed objects on the paper, the operation mode is set to perform FFD, and if it indicates that there are printed objects on the paper, the operation mode is set to not perform FFD. Note that the UI screen shown in Figure 3 is merely an example and is not limited to it. For example, any UI screen that allows the user to select and register information indicating whether or not there are printed objects on the paper, such as by selecting from a pull-down menu, would suffice.

[0026] The alignment module 205 performs alignment processing between the inspection image and the reference image according to the operation mode of the alignment processing determined by the mode determination module 204. Details of the alignment processing will be described later.

[0027] The parameter setting module 206 sets parameters according to the inspection items set in the inspection item setting module 202. In this case, the parameters include filters to highlight the type of printing defect selected by the user, thresholds for identifying printing defects, and so on.

[0028] The inspection module 207 executes defect detection processing for the inspection items set in the inspection item setting module 202. The inspection result output module 208 displays the results of the defect detection processing performed by the inspection module 207 on the UI panel 113.

[0029] <Inspection Process Flow> Figure 7 is a flowchart showing the procedure for the inspection process performed by the image processing unit 118. The series of processes shown in the flowchart of Figure 7 are realized by the CPU 106 reading the program stored in the ROM 108 into the RAM 107 and executing it. In the following explanation, the symbol "S" represents a step.

[0030] In S701, preparation processing is performed. Specifically, the setting of inspection items based on user selection, the setting of parameters corresponding to the set inspection items, and the determination of the operation mode during alignment based on registered paper information are performed by the respective software modules 202, 206, and 204.

[0031] In S702, the strain information generation module 203 performs the strain information generation process. Figure 8 is a flowchart detailing the strain information generation process. The following is a detailed explanation following the flow in Figure 8.

[0032] <<Distortion Information Generation Process>> S801 acquires print settings related to the printed material to be inspected, specifically setting information for the printing process, such as the type of paper to be used, the paper size, and whether it is double-sided or single-sided printing.

[0033] In the next step, S802, print data for the distortion correction chart is generated based on the paper size information included in the acquired print settings. Figure 9 shows an example of a distortion correction chart. Marks 902 are arranged in a grid pattern across the entire surface of the chart 901 shown in Figure 9. By reducing the spacing between the marks 902, more accurate distortion information can be obtained. The number of marks 902 may be changed according to the paper size. Also, although the chart 901 shown in Figure 9 uses crosses as the shape of the marks, squares or other shapes may also be used. Then, in S803, a reference image of the chart is generated based on the PDL included in the print data of the chart generated in S802.

[0034] In the next step, S804, the reference image of the chart generated in S803 is transmitted to the printing device 102 via the printing device I / F 111, and a printing process based on the reference image is performed to obtain a printed copy of the chart. Then, in S805, the image reading unit 110 reads the printed copy of the chart obtained in S804, and a scanned image of the chart is generated.

[0035] Next, in S806, the process of detecting the mark positions from the chart reference image generated in S803 is performed. The method of detecting the mark positions is not particularly limited, but for example, one method is to extract the pixel area of ​​the mark using template matching and calculate the centroid of that pixel area to determine the mark position. At this time, in order to make each mark identifiable, an index is simultaneously obtained based on the mark's position, such as the jth row and ith column from the top left of the paper.

[0036] Next, in S807, the scanned chart image generated in S805 is aligned to the reference image using a linear transformation. One example of a linear transformation for alignment is to calculate an affine matrix that minimizes the sum of the Euclidean distances of the mark positions and then perform the affine transformation. Since the affine transformation is a deformation that rotates, translates, scales, and shears the entire image, it is possible to align the scanned chart image to the reference image while preserving local distortions within the image.

[0037] Next, in S808, the mark positions are detected from the scanned image of the chart that was aligned in S807, using the same method as in S806. Then, in S809, distortion information is generated, linking the mark positions in the reference image detected in S806 and the mark positions in the scanned image detected in S808 to the index of each mark. Figure 10 shows an example of distortion information in table format, and the coordinate information (x,y) indicating the position of each mark in the reference image and the scanned image are listed. In the example in Figure 10, the mark in the 3rd column of the 1st row is shifted in the x direction (there is distortion). This concludes the explanation of the distortion information generation process. Returning to the explanation of the flowchart in Figure 7.

[0038] In S703, the image acquisition module 201 acquires a reference image of the RIP data that has been registered in advance from RAM 107 or main memory 109. This reference image is registered prior to the start of the inspection process and is associated with the print job. In the RIP inspection method, the reference image is registered, for example, by interpreting the PDL included in the print job sent from server 101, acquiring the raster-format image data obtained through the printing device I / F 111, and storing it in RAM 107 or main memory 109.

[0039] In S704, the image acquisition module 201 acquires inspection images generated by reading printed materials output from the printing device 102 with the image reading unit 110. The inspection images may be acquired sequentially by synchronizing the reading operation of the image reading unit 110 with the transported printed materials, or inspection images that have been read in advance and stored in the main memory unit 109 may be acquired.

[0040] In S705, the alignment module 205 performs alignment processing between the reference image acquired in S703 and the inspection image acquired in S704. Details of this alignment processing will be described later.

[0041] In S706, the inspection items to be executed are determined from the inspection items set in the preparation process of S701. If multiple inspection items are set, the inspection items to be executed are determined in order, for example, those that are pre-registered to be executed preferentially, or those corresponding to the type of printing defect initially selected by the user.

[0042] In S707, the inspection module 207 executes defect detection processing for the inspection items determined to be executed in S706. Figure 11 is a flowchart detailing the defect detection process. The following is a detailed explanation following the flow in Figure 11.

[0043] ≪Defect Detection Process≫ In S1101, a difference image is generated that shows the difference between the aligned inspection image and the reference image. The difference image can be obtained, for example, by comparing corresponding pixels between the aligned inspection image and the reference image and obtaining the difference value of the pixel value (e.g., the intensity value for each RGB component) for each pixel. The method for calculating the difference may be changed depending on the inspection item being performed.

[0044] Next, in S1102, filtering is performed on the generated difference image to emphasize specific shapes. For example, Figure 12(a) shows a filter to emphasize point-like defects, and Figure 12(b) shows a filter to emphasize linear defects. These filters are changed according to the inspection item being performed.

[0045] Next, in S1103, a binarization process is performed on the filtered difference image. As a result, if the pixel value (difference value) of each pixel constituting the difference image exceeds a predetermined threshold, a "1" is assigned to that pixel, and if it is below the threshold, a "0" is assigned to that pixel, resulting in the generation of an image (hereinafter referred to as the "difference binary image").

[0046] Next, in S1104, it is determined whether the number of pixels assigned the value "1" among the pixels that make up the differential binarized image exceeds a predetermined number. If it exceeds the predetermined number, the process proceeds to S1105. On the other hand, if it does not exceed the predetermined number, it is determined that there are no significant defects in the inspected item, and this process is exited.

[0047] In S1105, information regarding the detected defect is stored. Specifically, the inspection item (type of printing defect) and the location of the detected defect (location coordinates in the image) are associated and stored in RAM 107 or main memory 109. After that, the defect detection process ends. This concludes the description of the defect detection process. Returning to the explanation of the flowchart in Figure 7,

[0048] In S708, it is determined whether defect detection processing has been performed for all inspection items set in the preparation process in S701. If defect detection processing has been completed for all inspection items, the process proceeds to S709. On the other hand, if there are any unprocessed inspection items, the process returns to S706 to determine the next inspection item to be executed and the process continues.

[0049] In S709, the inspection result output module 208 displays the inspection results for all inspection items set in the preparation process of S701 on the UI panel 113. Figure 13 is an example of a UI screen that displays the inspection results. The result display screen 1301 displays the inspection image 1302 that was the target of the defect detection process. For example, the words "Point-like defect" are displayed near the defect 1303 which has been determined to be a point-like defect. Similarly, the words "Linear defect" are displayed near the defect 1304 which has been determined to be a linear defect. The coordinates 1305 and 1306 of each defect in the inspection image 1302 are also displayed. Note that the inspection result display method shown in Figure 13 is just one example; any display method that allows the user to recognize the results for each set inspection item is acceptable, such as displaying different colors for each type of defect. Once the display of the inspection results is complete, the inspection process ends.

[0050] The above describes the inspection process performed by the image processing unit 118.

[0051] <Details of alignment process> Figure 14 is a flowchart detailing the alignment process of S705 described above according to this embodiment. The purpose of this alignment process is to suppress misalignment between the reference image and the inspection image and to improve the accuracy of detecting the difference between the two images. In this embodiment, after applying alignment by linear transformation, whether or not to further apply alignment by nonlinear transformation is switched depending on whether or not there are printed objects on the paper.

[0052] In S1401, the inspection image is aligned using a linear transformation. For alignment using a linear transformation, an affine transformation is used, for example, similar to S807 in the distortion information generation process described above. Figure 15(a) is a schematic diagram when the contour of the paper is used as a feature point. When the contour of the paper is used as a feature point, the contour is obtained based on a known edge detection method (e.g., the Canny method), extracting the outermost edge of the inspection target area 1502 of image 1501, and the intersection of the edges 1503 is used as a feature point. Figure 15(b) is a schematic diagram when a marker for alignment is used as a feature point. In image 1511, there is no contour in the inspection target area 1512, but there is a contour in the non-inspection target area 1513. Therefore, since the intersection of the edges cannot be detected, an alignment marker 1514 is placed in a location where no object exists, and the position of this marker is used as a feature point. Note that the feature points described here are just examples, and for example, edge detection may be performed within the image, and feature points may be set from the detected edges. Additionally, the paper outline and image edge information may be combined as distinctive features.

[0053] Once the alignment process using linear transformation is complete, in S1402, the mode determination module 204 decides whether to continue the alignment process using nonlinear transformation based on the paper information. Specifically, if the paper information registered by the user indicates that there are no printed objects on the paper, the system decides to operate in a mode that does not perform alignment using nonlinear transformation and exits this process. On the other hand, if the paper information registered by the user indicates that there are printed objects on the paper, the system decides to operate in a mode that performs alignment using nonlinear transformation and proceeds to S1403. Here, let's review alignment using nonlinear transformation. Alignment using nonlinear transformation can correct local distortions, and in addition to FFD, there are other methods such as thin plate spline (TPS) and landmark LDDMM method. Here, using the case of FFD as an example, we will describe a method in which the adjustment of control points is performed in multiple stages.

[0054] In S1403, the inspection image, which has been aligned by linear transformation in S1401, is further aligned using FFD. The specific flow of FFD is as follows: First, the distortion information generated and saved in S702 is read, and control points are placed based on this distortion information. Figure 16 shows an example in which L x M control points are arranged in a grid on the inspection image I. In Figure 16, the coordinates of the control point in the l row and m column are p m,l This is represented by (l=1,..,L, m=1,..,M). In this case, the accuracy of distortion correction can be further improved by increasing the number of control points to be placed compared to the number of marks in the aforementioned chart (see Figure 9). The position of each control point is determined analytically, for example, by the least squares method. Specifically, the correspondence between the mark positions in the chart image and its reference image is treated as the correspondence between the feature points in the inspection image and its reference image, and control point coordinates are found and placed such that the shift in mark positions is minimized when the chart is transformed according to equation (1) described later. Next, the inspection image I after affine transformation is transformed using the following equation (1) to generate the first distortion-corrected image I'.

[0055]

number

[0056] In equation (1) above, w(x,y) is expressed by equation (2) below, which is an equation for calculating the distortion-corrected coordinates of the coordinates (x,y) in the inspection image I after affine transformation.

[0057]

number

[0058] In equation (2) above, the bases B0(t), B1(t), B2(t), and B3(t) are expressed by equations (3) to (6) below, respectively. Also, u and v are expressed by equations (7) and (8) below, respectively. Furthermore, δ x , δ y These are expressed by equations (9) and (10) below, respectively. Here, H and W are the vertical and horizontal dimensions of the image, respectively.

[0059]

number

[0060]

number

[0061]

number

[0062]

number

[0063]

number

[0064]

number

[0065] [Number]

[0066] [Number]

[0067] Subsequently, control points are arranged on the first distortion-corrected image I' to generate a second distortion-corrected image I''. At this time, L'×M' control points in vertical and horizontal directions are arranged in a grid pattern. Since the purpose of the second distortion correction is to correct distortion that could not be completely corrected in the first distortion correction, it is desirable to arrange the control points more finely (that is, L×M < L'×M'). Then, while updating the positions of the arranged control points, pixel update is performed using the above-mentioned formula (1). In the pixel update, a distance d between the second distortion-corrected image I'' represented by the following formula (11) and a reference image T is obtained, and when the distance d becomes equal to or less than a preset threshold, the pixel update is completed.

[0068] [Number]

[0069] When the second distortion-corrected image I'' is thus completed, the FFD process is terminated.

[0070] The above is the content of the alignment process according to the present embodiment. Note that in the flow of FIG. 7, distortion information is generated each time an inspection process is executed, but the generated distortion information may be stored in a main storage unit 109 or the like and read out in subsequent processes. When the print settings are the same as before, the cost of printing the chart can be reduced by reusing the stored distortion information.

[0071] According to this embodiment, in the alignment of images prior to inspection, if there is a printed object on the paper, only alignment by linear transformation is performed, and alignment by nonlinear transformation is not performed. If there is no printed object on the paper, alignment by linear transformation is performed followed by alignment by nonlinear transformation. This makes it possible to prevent alignment failures by nonlinear transformation that occur under certain conditions and suppress the occurrence of misinspections. Embodiment 2

[0072] In Embodiment 1, the decision of whether to perform alignment using a nonlinear transformation after alignment using a linear transformation was made based on whether or not there was a printed object on the paper. Incidentally, as mentioned above, in addition to the RIP inspection method, there is also a scan inspection method that uses scanned data of product samples or printed materials that have passed visual inspection as reference images. Next, Embodiment 2 will be described in which the decision of whether to perform alignment using a nonlinear transformation after alignment using a linear transformation is made based on whether the data type of the reference image is RIP data or scanned data. Note that the explanation of contents common to Embodiment 1, such as the system configuration, will be omitted, and the following explanation will focus on the differences.

[0073] <Concept of this embodiment> For example, there is a printing method called "variable printing" in which different data (generally called "variable data") for each part of a direct mail recipient's address and other information is sequentially inserted into the image data to be printed. Figures 17(a) and (b) illustrate the difference when the reference image is scanned data and when it is RIP data in this variable printing method. However, the paper is assumed to be blank.

[0074] When the reference image is scanned data, as shown in Figure 17(a), reference image 1701 is treated as a common reference image for all copies. Therefore, there will always be an inspection image 1702 in which the string representing the variable data "address" does not match that in reference image 1701. Thus, when inspecting variable-data printed materials using a scan inspection method, a situation occurs where the characters and numbers corresponding to the variable data do not match between the reference image and the inspection image. On the other hand, when the reference image is RIP data, as shown in Figure 17(b), a number of reference images 1711 corresponding to the number of copies is prepared according to the variable data. In other words, since the reference image 1711 and inspection image 1712, which are always in a corresponding relationship, are compared, the string representing the variable data "address" will always match between the two images. Thus, when inspecting variable-data printed materials using a RIP inspection method, a number of reference images corresponding to the number of copies is prepared, so there is no mismatch between the objects contained in the reference image and the objects contained in the inspection image.

[0075] As described above, in the inspection of printed materials using variable data printing, object mismatches do not occur between the reference image and the inspection image in the RIP inspection method, but object mismatches do occur in the scan inspection method. Therefore, in this embodiment, the alignment operation mode is determined depending on whether the data type of the reference image is scan data or RIP data. Specifically, the mode determination module 204 determines the operation mode in the alignment process based on reference image information that indicates whether the image data registered as the reference image is scan data or RIP data. The reference image information is registered according to user selection via a UI screen as shown in Figure 18, which is displayed on the UI panel 113. The UI screen 1800 shown in Figure 18 has two types of radio buttons: button 1801 to select when scan data is used as the reference image and button 1802 to select when RIP data is used as the reference image. The mode determination module 204 in this embodiment controls the system so that alignment by nonlinear transformation is not performed when the reference image information indicates scan data, and alignment by nonlinear transformation is performed when it indicates RIP data. Note that the UI screen shown in Figure 18 is merely an example and is not limited thereto. For example, a UI screen that allows the user to select and register whether the reference image is scanned data or RIP data, perhaps by selecting from a pull-down menu, would suffice.

[0076] <Details of alignment process> Figure 19 is a flowchart detailing the alignment process according to this embodiment.

[0077] In S1901, the inspection image is aligned using a linear transformation. The details of the alignment using the linear transformation are the same as in S1401 in the flowchart in Figure 14, so the explanation is omitted.

[0078] Once the alignment process using linear transformation is complete, in S1902, the mode determination module 204 determines, based on the reference image information, whether or not to continue the alignment process using nonlinear transformation. If the reference image information indicates that it is scan data, the operation mode is set to one in which alignment using nonlinear transformation is not performed, and the process is exited. On the other hand, if the reference image information indicates that it is RIP data, the operation mode is set to one in which alignment using nonlinear transformation is performed, and the process proceeds to S1903.

[0079] In S1903, the inspection image, which has been aligned using linear transformation in S1901, is further aligned using nonlinear transformation. The details of the alignment using nonlinear transformation are the same as in S1403 in the flowchart of Figure 14, so the explanation is omitted.

[0080] The above describes the alignment process according to this embodiment.

[0081] According to this embodiment, in the alignment of images prior to inspection, if the reference image is scan data, only alignment by linear transformation is performed, and alignment by nonlinear transformation is not performed. This prevents alignment failures due to nonlinear transformation that occur under certain conditions during variable printing, and suppresses the occurrence of misinspections. Embodiment 3

[0082] The aforementioned embodiments 1 and 2 can also be combined. Note that the system configuration and other aspects common to embodiments 1 and 2 will not be explained, and the following will focus on the alignment process, which is a key difference. Figure 20 is a flowchart detailing the alignment process according to this embodiment.

[0083] In S2001, the inspection image is aligned using a linear transformation. The details of the alignment using the linear transformation are the same as in S1401 in the flowchart in Figure 14, so the explanation is omitted.

[0084] Once the alignment by linear transformation is complete, in S2002, similar to S1902 in the flow chart of Figure 19, the mode determination module 204 determines the next processing based on the reference image information. If the reference image information indicates that it is scan data, the operation mode is determined to not perform alignment by nonlinear transformation, and this process is exited. On the other hand, if the reference image information indicates that it is RIP data, the process proceeds to S2003.

[0085] In S2003, similar to S1402 in the flow chart of Figure 14, the mode determination module 204 determines whether to continue the alignment process using nonlinear transformation based on the paper information. Specifically, if the paper information indicates that there are no printed objects on the paper, the operation mode is determined to not perform alignment using nonlinear transformation, and the process is exited. On the other hand, if the paper information indicates that there are printed objects on the paper, the operation mode is determined to not perform alignment using nonlinear transformation, and the process is exited.

[0086] In S2004, the inspection image, which has been aligned using linear transformation in S2001, is further aligned using nonlinear transformation. The details of the nonlinear transformation alignment are the same as in S1403 in the flow chart of Figure 14, so the explanation is omitted.

[0087] The above describes the alignment process according to this embodiment. As an example of the application of this embodiment, consider the case where, in variable printing using RIP data as a reference image, the recipient's name and address are inserted as variable data. In this case, when alignment is first performed on the inspection image to be processed using linear transformation (S2001), the process proceeds to S2003 because the reference image is RIP data. If the paper is blank, then alignment is performed using nonlinear transformation (determined as "none" in S2003). On the other hand, if, for example, the recipient's name and address of the issuer are already printed on the paper, the process is exited without performing alignment using nonlinear transformation (determined as "yes" in S2003).

[0088] As described above, it is also possible to control alignment by combining Embodiments 1 and 2.

[0089] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of the system or device read and execute the program. Furthermore, the present invention can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0090] Furthermore, the disclosure of this embodiment includes the following configurations and methods.

[0091] (Composition 1) An inspection device for inspecting printed materials output from a printing device, Alignment means for aligning the inspection image generated by reading the printed material with a reference image used as a standard in the inspection, Equipped with, The aforementioned alignment means is The aforementioned inspection image is subjected to alignment using linear transformation. If certain conditions are met, the inspection image obtained by the linear transformation is subjected to nonlinear transformation. An inspection device characterized by the following features.

[0092] (Configuration 2) The inspection apparatus according to configuration 1, characterized in that the aforementioned certain condition is that image data used for the printing process of the printed material is set as the reference image.

[0093] (Composition 3) The inspection apparatus according to configuration 1, characterized in that the aforementioned certain condition is that there are no printed objects on the paper used in the printing process of the printed material.

[0094] (Composition 4) The inspection apparatus according to configuration 1, characterized in that the aforementioned certain conditions are that image data for the printing process of the printed material is set as the reference image, and that there are no printed objects on the paper used for the printing process.

[0095] (Composition 5) The inspection apparatus according to configuration 1, characterized in that if the aforementioned certain conditions are not met, the alignment by the nonlinear transformation is not performed.

[0096] (Composition 6) Furthermore, the inspection apparatus according to the configuration described above, further comprising an inspection means for inspecting an inspection image obtained by performing nonlinear transformation alignment using the alignment means based on the reference image.

[0097] (Composition 7) The system further includes a registration means for registering information indicating whether the image data set as the reference image is image data for the printing process of the printed material, or image data obtained by reading a printed material without printing defects. The inspection apparatus according to configuration 2 or 4, characterized in that the alignment means determines, based on the registered information, whether or not image data for the printing process of the printed material is set as the reference image.

[0098] (Composition 8) The system further includes a registration means for registering information indicating whether or not there are printed objects on the paper used in the printing process, The inspection apparatus according to configuration 3 or 4, characterized in that the alignment means determines, based on the registered information, whether or not there are any printed objects on the paper used for the printing process.

[0099] (Composition 9) The inspection apparatus according to any one of configurations 1 to 8, characterized in that the alignment by the linear transformation is alignment using an affine transformation.

[0100] (Composition 10) The inspection apparatus according to any one of configurations 1 to 8, characterized in that the alignment by the nonlinear transformation is alignment using free-form deformation.

[0101] (Method 1) An inspection method for inspecting printed materials output from a printing device, A positioning step involves reading the printed material and aligning the inspection image generated with a reference image used as a standard in the inspection. Includes, In the aforementioned alignment step, The aforementioned inspection image is subjected to alignment using linear transformation. If certain conditions are met, the inspection image obtained by the linear transformation is subjected to nonlinear transformation. A testing method characterized by the following features.

[0102] (Composition 11) A program for causing a computer to function as a testing device as described in any of configurations 1 to 10.

Claims

1. An inspection device for inspecting printed materials output from a printing device, Alignment means for aligning the inspection image obtained by reading the aforementioned printed material with a reference image used as a standard in the inspection, Equipped with, The aforementioned alignment means is The aforementioned inspection image is subjected to alignment using linear transformation. If the reference image is set to image data used for printing the printed material, then the inspection image obtained by the linear transformation alignment is subjected to a nonlinear transformation alignment. An inspection device characterized by the following features.

2. An inspection device for inspecting printed materials output from a printing device, Alignment means for aligning the inspection image obtained by reading the aforementioned printed material with a reference image used as a standard in the inspection, Equipped with, The aforementioned alignment means is The aforementioned inspection image is subjected to alignment using linear transformation. If there are no printed objects on the paper used for printing the aforementioned printed material, then the inspection image obtained by the linear transformation alignment is subjected to a non-linear transformation alignment. An inspection device characterized by the following features.

3. An inspection device for inspecting printed materials output from a printing device, Alignment means for aligning the inspection image obtained by reading the aforementioned printed material with a reference image used as a standard in the inspection, Equipped with, The aforementioned alignment means is The aforementioned inspection image is subjected to alignment using linear transformation. If the reference image is set to image data used for printing the printed material, and there are no printed objects on the paper used for printing, then the inspection image obtained by the linear transformation alignment is subjected to a nonlinear transformation alignment. An inspection device characterized by the following features.

4. The inspection apparatus according to claim 1 or 3, characterized in that the alignment means does not perform alignment by nonlinear transformation when no image data used for printing the printed material is set as the reference image.

5. Furthermore, the inspection apparatus according to any one of claims 1 to 3 is characterized by comprising an inspection means for inspecting an inspection image that has been aligned by the alignment means based on the reference image.

6. The system further includes a registration means for registering information indicating whether the image data set as the reference image is image data used for the printing process of the printed material, or image data obtained by reading a printed material without printing defects. The inspection apparatus according to claim 1 or 3, characterized in that the alignment means determines, based on the registered information, whether or not image data to be used for printing the printed material is set as the reference image.

7. The system further includes a registration means for registering information indicating whether or not there are printed objects on the paper used in the printing process, The inspection apparatus according to claim 2 or 3, characterized in that the alignment means determines, based on the registered information, whether or not there are any printed objects on the paper used for the printing process.

8. The inspection apparatus according to any one of claims 1 to 3, characterized in that the alignment by the linear transformation is alignment using an affine transformation.

9. The inspection apparatus according to any one of claims 1 to 3, characterized in that the alignment by the nonlinear transformation is alignment using FFD.

10. An inspection method for inspecting printed materials output from a printing device, A positioning step involves aligning the inspection image obtained by reading the printed material with a reference image used as a standard in the inspection. Includes, In the aforementioned alignment step, The aforementioned inspection image is subjected to alignment using linear transformation. If the reference image is set to image data used for printing the printed material, then the inspection image obtained by the linear transformation alignment is subjected to a nonlinear transformation alignment. A testing method characterized by the following features.

11. A program for causing a computer to perform the inspection method described in claim 10.

Images