Image processing apparatus, printing system, image processing method, and storage medium

Abstract

To improve alignment accuracy in inspection of a printed matter.SOLUTION: An image processing apparatus that performs inspection of a printed material includes an acquisition unit (203) that acquires a print position displacement amount between a reference image that is a reference of inspection and an inspection image that is a target of inspection, a determination unit (204) that determines, according to the print position displacement amount, a positioning region that is a range on two dimensional coordinates for performing positioning of the inspection image and the reference image with respect to the inspection image and includes a picture existing in the inspection image, and a positioning unit (205) that performs positioning of the inspection image and the reference image on the basis of a range of the positioning region.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE The present disclosure relates to the inspection of printed matter. [Background technology]

[0002] Conventionally, the presence or absence of printing defects in an inspection image has been inspected by comparing the reference image with the inspection image. However, because the inspection image is generated by scanning the printed material while it is being transported, misalignment of the printed material during transport can cause misalignment of the printing position of the inspection image relative to the reference image. Therefore, Patent Document 1 proposes a system that aligns the inspection image with the reference image before inspecting the inspection image. The inspection image includes an area of ​​the paper on which a pattern exists and an area outside the paper. The reference image includes an area of ​​the paper on which the reference pattern exists and an area outside the paper. The system of Patent Document 1 aligns the inspection image and the reference image, including their respective areas outside the paper. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-175441 Summary of the Invention [Problem to be solved by the invention]

[0004] As described above, in Patent Document 1, alignment is performed including areas outside the paper. For example, FIG. 23(a) shows an example of alignment including areas outside the paper using free-form deformations (FFD), known as non-rigid alignment. In FIG. 23(a), as a result of alignment including areas outside the paper, the paper edge of the inspection image is deformed, and the image around the paper edge is distorted. If an inspection is performed with the image distorted, the inspection will be judged as NG. In other words, even if the image was not originally distorted, the image will be distorted, resulting in an overdetection. Such overdetection occurs every time the printing position is misaligned. Therefore, as shown in FIG. 23(b), the paper edge of the inspection image is deformed to absorb the misalignment of the paper edge from the reference image, distorting the surrounding image. This means that overdetection occurs every time the printing position is misaligned. As a result, even non-defective products whose printing position misalignment is within the user's tolerance are judged as NG, resulting in an increase in paper waste. In other words, local distortion occurs in the inspection image after alignment, which can be affected by misalignment of the print. Therefore, an object of the present disclosure is to improve the alignment accuracy in the inspection of printed matter. [Means for solving the problem]

[0005] An image processing device according to one aspect of the present disclosure is an image processing device for inspecting printed matter, and is characterized by comprising: an acquisition means for acquiring the amount of printing misalignment between a reference image that serves as the basis for the inspection and an inspection image that is the subject of the inspection; a determination means for determining an alignment area for the inspection image, which is a range on a two-dimensional coordinate system for aligning the inspection image with the reference image and which includes a pattern present in the inspection image, based on the amount of printing misalignment; and an alignment means for aligning the inspection image with the reference image based on the range of the alignment area. [Effects of the Invention]

[0006] According to the present disclosure, it is possible to improve the alignment accuracy in the inspection of printed matter. [Brief explanation of the drawings]

[0007] [Figure 1] 1 shows a configuration diagram of the entire printing system. [Figure 2] FIG. 2 is a block diagram showing the functional configuration of the image processing device in FIG. 1. [Figure 3] 2 is a flowchart illustrating image processing executed by the image processing device of FIG. 1. [Figure 4] 4A and 4B are diagrams showing an example of a reference image and an inspection image used in the image processing of FIG. 3. [Figure 5] 4 is a flowchart illustrating the print position deviation amount acquisition process in step S303 of FIG. 3. [Figure 6] 4 is a flowchart illustrating the alignment area determination process in S304 of FIG. 3. [Figure 7] 4 is a flowchart illustrating the alignment process in S305 of FIG. 3. [Figure 8] 4A and 4B are diagrams showing examples of alignment regions of the reference image and the inspection image used in the process of S306 in FIG. 3. [Figure 9] 10A and 10B are diagrams illustrating examples of the positions of the four corners of a sheet of a reference image and an inspection image. [Figure 10] 4A and 4B are diagrams showing examples of alignment regions of the reference image and the inspection image used in the process of S304 in FIG. 3. [Figure 11] FIG. 8 is a diagram showing an example of control points arranged in a grid pattern on the inspection image in the process of S701 in FIG. 7. [Figure 12] 10A and 10B are diagrams showing an example in which register marks are further set on each of the reference image and the inspection image in the second embodiment. [Figure 13] 10 is a flowchart illustrating a registration area determination process according to the second embodiment. [Figure 14] 14 is a flowchart illustrating the exclusion process in S1301 of FIG. 13. [Figure 15] FIG. 14 is a diagram showing an example of an area to be excluded in S1301 of FIG. [Figure 16] 10A and 10B are diagrams showing an example in which additional printing is performed on each of a reference image and an inspection image in the third embodiment. [Figure 17] 13 is a flowchart illustrating a registration area determination process according to the third embodiment. [Figure 18] FIG. 18 is a diagram showing an example of an area to be excluded in S1601 of FIG. 17. [Figure 19] FIG. 13 is a diagram showing another example of an excluded region in the third embodiment. [Figure 20] 10 is a flowchart illustrating image processing executed by the image processing device of FIG. 1 according to the fourth embodiment. [Figure 21] 13A to 13C are diagrams showing examples of various regions set in each of a reference image and an inspection image in the fourth embodiment. [Figure 22] 13A and 13B are diagrams illustrating an example of a user warning notification and an examination setting change notification in the fourth embodiment. [Figure 23] FIG. 10 is a diagram showing a conventional example in which a pattern around the edge of a paper is distorted due to alignment to eliminate misalignment of a print position. DETAILED DESCRIPTION OF THE INVENTION

[0008] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the present disclosure, and not all combinations of features described in the following embodiments are necessarily essential to the solutions of the present disclosure. Note that the same reference numerals are used to designate the same components.

[0009] (overview) Printed materials output from a printing device are coated with coloring materials such as ink and toner in intended locations. However, if such coloring materials are coated in unintended locations on the printed material, they may cause stains on the printed material. Alternatively, even if such coloring materials are coated in intended locations on the printed material, insufficient coating may result in a lighter color than intended, resulting in color loss on the printed material. The stains and color loss described above constitute print defects and can reduce the quality of printed materials. Therefore, the quality of printed materials has traditionally been assured by inspecting for print defects. However, visual inspection for print defects requires a lot of time and cost, so systems exist that automatically inspect for print defects. Such systems inspect for print defects by, for example, aligning a reference image with an inspection image and then calculating the presence or absence of print defects based on the difference between the reference image and the inspection image. Here, the reference image is registered in advance as an inspection standard for printed materials. Meanwhile, the inspection image is generated by scanning a conveyed printed material. Inspection images can suffer from localized deformation due to flexing of the printed material during transport and scanning. To address this localized deformation, a registration process that absorbs the localized deformation is required. However, when printing an inspection image on paper to create a printout, misalignment of the print position on the paper can occur due to misalignment of the paper during transport. Furthermore, misalignment of the paper during printing can cause the edge of the paper to be transported more than expected, resulting in a misalignment between the edge of the paper on the reference image and the edge of the paper on the inspection image. As a result, the reference image and the inspection image are aligned with each other in a state where the edge of the paper on the reference image and the edge of the paper on the inspection image are not aligned. Therefore, the edge of the paper on the inspection image is deformed to align with the edge of the paper on the reference image. The deformation of the edge of the paper on the inspection image is propagated to the image around the paper, distorting the image around the paper. This can result in localized distortion of a portion of the inspection image after alignment, potentially reducing alignment accuracy. Therefore, a technology has been proposed that reduces the impact of misalignment of the inspection image relative to the paper, even when misalignment of the inspection image occurs.For example, a technology has been proposed for switching from nonlinear alignment processing to linear alignment processing using affine transformation or the like in a fixed region at the edge of an image. However, because the alignment processing is switched in a fixed region at the edge of an image, if the amount of print misalignment of the test image is greater than the fixed region, the test image, including an area outside the paper, may be aligned with a reference image that does not include the area outside the paper. In this case, local distortion occurs at a portion of the image edge, reducing alignment accuracy. Therefore, in the present disclosure, an alignment region for aligning the test image with the reference image is determined based on the amount of print misalignment between the reference image and the test image. Furthermore, the test image and the reference image are aligned within the alignment region. These processes can reduce the impact of print misalignment due to relative misalignment between the edge of the paper and the test image. This can improve alignment accuracy in the inspection of printed materials. Details of this disclosure are described below.

[0010] (First embodiment) In the first embodiment, an alignment area for aligning an inspection image with a reference image is determined based on the amount of misalignment between the inspection image and the reference image, which is the inspection target. This process reduces the effect of misalignment even in an inspection image with a large misalignment.

[0011] (Printing System) FIG. 1 shows the overall configuration of a printing system. The printing system includes an image processing device 100, a printing server 180, and a printing device 190. The printing system outputs printed materials and inspects the printed materials for printing defects. The image processing device 100, the printing server 180, and the printing device 190 all have information processing functions for processing various types of information. When the image processing device 100, the printing server 180, and the printing device 190 are not particularly distinguished from each other in terms of their information processing functions, the image processing device 100, the printing server 180, and the printing device 190 are also referred to as information processing devices. The information processing functions executable by the information processing device may be implemented in a terminal such as a smartphone or tablet. In this case, the terminal such as a smartphone or tablet may have the information processing functions. In the example of FIG. 1, the image processing device 100 is disposed downstream of the printing device 190.

[0012] (Printing server 180; Printing device 190) The printing server 180 has a function of generating a print job for a document to be printed and submitting the print job to the printing device 190. The printing device 190 has a function of forming an image on a recording medium based on the print job submitted from the printing server 180. Hereinafter, the recording medium will also be referred to as printing paper, as appropriate. The printing server 180 may be configured as a cloud service. The printing device 190 can use offset printing, electrophotography, and other methods. In this embodiment, it is assumed that the printing device 190 prints on printing paper using electrophotography, but the process units for charging, exposing, developing, transferring, and fixing are not shown. The printing device 190 includes a paper feed unit 191 and a conveying path 192. The user loads printing paper into the paper feed unit 191 in advance. In the printing device 190, the printing paper is fed from the paper feed unit 191 via the conveying path 192 to each process unit (not shown). Alternatively, printing paper may be fed to printing device 190 from an ADF (Auto Document Feeder) (not shown). Alternatively, a paper feed deck may be disposed in the front section of printing device 190, and printing paper may be supplied from the paper feed deck. When a print job is input from printing server 180, printing device 190 transports printing paper set in paper feed unit 191 along transport path 192, forms images on one or both sides of the paper, and sends it to image processing device 100.

[0013] (Image processing device 100) The image processing device 100 performs an inspection process to check for defects in printed matter. As shown in FIG. 1, for example, a printing device 190 is disposed upstream of the image processing device 100. An image is formed on the printed matter by the printing device 190, and the printed matter is supplied to the image processing device 100 via a conveying path 192. Alternatively, the printed matter may be placed on an ADF (not shown) and supplied from the ADF to the image processing device 100. Alternatively, the printed matter may be supplied to the image processing device 100 from an inserter (not shown). In other words, the image processing device 100 functions as an inspection processing device. Here, the printed matter refers to a sheet of printing paper on which an image has been formed. The image processing device 100 includes a CPU 101, a RAM 102, a ROM 103, and a main memory device 104 as units provided within the image processing device 100. The image processing device 100 also includes, as internal units, an image reading device 105, a printing device interface (I / F) 106, a general-purpose I / F 107, and a user interface (UI) panel 108. The image processing device 100 also includes a main bus 109, a conveying path 110, a first output tray 111, and a second output tray 112. The conveying path 110 is connected to a conveying path 192 and conveys print paper. For example, printed matter that has passed inspection is supplied to the first output tray 111. On the other hand, for example, printed matter that has failed inspection due to the discovery of defects is supplied to the second output tray 112. Note that the inspection results of printed matter may be classified into more detailed categories than just pass or fail. While FIG. 1 illustrates an example in which one image reading device 105 is provided inside the image processing device 100, this is not a limitation. For example, the image reading device 105 may be disposed along the conveying path 110 at a position facing the front side of the printed material and at a position facing the back side of the printed material. Also, a reversing path for reversing the front and back sides of the printed material may be provided on the conveying path 110. The image formed on the back side of the printed material that has been reversed by the reversing path may be read by the image reading device 105.

[0014] The CPU 101 is a processor that controls all components within the image processing device 100. The RAM 102 functions as the CPU 101's main memory, work area, etc. The ROM 103 stores programs executed by the CPU 101. The main storage device 104 stores applications executed by the CPU 101, data used in image processing, etc. The image reading device 105 is composed of, for example, a line sensor. The image reading device 105 can read images formed on one or both sides of a printed material sent from the printing device on the conveying path 110 and acquire the images as image data. Hereinafter, the image reading device 105 will also be referred to as the scanner 105 as appropriate. The printing device I / F 106 is connected to the printing device 190, and can synchronize the processing timing of the printing device 190 and the printed material and communicate each other's operating status. The general-purpose I / F 107 is a serial bus interface such as USB or IEEE 1394, allowing a user to export data such as logs or import other data into the image processing device 100. The UI panel 108 includes, for example, an LCD display and a touch panel laminated on the LCD display, and is configured as an LCD display with a touch panel. The UI panel 108 functions as a user interface for the image processing device 100, displaying and informing the user of the current status and settings. The UI panel 108 also accepts instructions from the user via the touch panel. Note that the UI panel 108 does not necessarily have to include a touch panel. In this case, the UI panel 108 may be provided with, for example, buttons for accepting user operations. The main bus 109 is connected to each unit of the image processing device 100. That is, each unit of the image processing device 100 can transmit and receive various data via the main bus 109. For example, data of an inspection image read by the image reading device 105 can be supplied to the CPU 101 via the main bus 109. In addition, although not shown, each unit of the image processing device 100 and a printing system including the image processing device 100 may operate in response to instructions from the CPU 101.For example, the image processing device 100 may synchronize the movement of the conveying path 192 and the conveying path 110, or switch whether to send the printed material to the first output tray 111 or the second output tray 112 depending on the inspection results. The image processing device 100 may also include a GPU in addition to the CPU 101. The image processing device 100 may also cause an external server (not shown) to execute part of the processing of the CPU 101.

[0015] In short, the image processing device 100 carries out the inspection process described below based on the image data of the printed matter read by the scanner 105 while transporting the printed matter supplied from the printing device 190 along the transport path 110. If the result of the inspection process shows that the printed matter passes the inspection, it is transported to the first output tray 111, and if not, it is transported to the second output tray 112. This operation makes it possible to collect only the printed matter whose quality has been confirmed in the first output tray 111 as deliverables for delivery.

[0016] (Functional configuration of image processing device 100) FIG. 2 is a block diagram showing the functional configuration of the image processing device 100 of FIG. 1. The functional configuration shown in FIG. 2 is realized when the functional configuration and a program for realizing the functional configuration are supplied to the image processing device 100 shown in FIG. 1 and the image processing device 100 executes the program. The image processing device 100 executes image processing using the functional configuration shown in FIG. 2. An example of image processing executed by the image processing device 100 will be described later using FIG. 3. In the example of FIG. 2, the functional configuration of the image processing device 100 of FIG. 1 includes a reference image input unit 201, an inspection image input unit 202, a print position deviation amount acquisition unit 203, an alignment area determination unit 204, an alignment unit 205, and an inspection unit 206. The reference image input unit 201 has a function of capturing a reference image including a reference pattern that serves as the inspection standard. The reference image including the reference pattern that serves as the inspection standard is stored, for example, in RAM 102 or main memory device 104. The reference image input unit 201 is configured to output such a reference image to the print misalignment amount acquisition unit 203, the alignment area determination unit 204, the alignment unit 205, and the inspection unit 206, and input it to each output destination. The inspection image input unit 202 is configured to import an inspection image including a pattern to be inspected. The inspection image including the pattern to be inspected is an image obtained by scanning a printed material with the scanner 105. The inspection image input unit 202 is configured to output the inspection image to the print misalignment amount acquisition unit 203, the alignment area determination unit 204, and the alignment unit 205, and input it to each output destination. The print misalignment amount acquisition unit 203 is configured to acquire the amount of print misalignment by comparing the printing positions of the reference image and the inspection image. The print misalignment amount acquisition unit 203 is configured to output the acquired amount of print misalignment to the alignment area determination unit 204. Alignment area determination unit 204 has a function to determine an alignment area for aligning the test image with the position of the reference image according to the amount of misalignment between the print positions of the reference image and the test image. Alignment area determination unit 204 has a function to output the determined alignment area to alignment unit 205. Alignment unit 205 has a function to adjust the range of the test object by aligning the image area on the alignment area of ​​the test image with the reference image.The alignment unit 205 has a function of outputting the inspection image with the alignment region aligned to the inspection unit 206. The inspection unit 206 has a function of inspecting the presence or absence of printing defects in the inspection image by comparing the inspection image with the alignment region aligned to a reference image. The inspection unit 206 has a function of outputting the inspection results to the printing device 190 or the UI panel 108.

[0017] (Image processing by image processing device 100) The image processing performed by the image processing device 100 will be described with reference to Figs. 3 and 4. Fig. 3 is a flowchart illustrating the image processing performed by the image processing device 100 of Fig. 1. Fig. 4 is a diagram showing an example of a reference image 401 and an inspection image 405 used in the image processing of Fig. 3. Fig. 4(a) is a diagram showing the reference image 401. Fig. 4(b) is a diagram showing the inspection image 405. A program for executing the contents of the flowchart shown in Fig. 3 is stored in ROM 103, and is executed when, for example, a user issues an instruction to start inspection of the inspection image via the UI panel 108.

[0018] That is, the process shown in Fig. 3 is realized by the CPU 101 reading out a program stored in the ROM 103 into the RAM 102 and executing it. Specifically, the process shown in Fig. 3 is executed when inspection of the inspection image starts. Note that some or all of the functions of the steps in Fig. 3 may be realized by hardware such as an ASIC or electronic circuit. The symbol "S" in the explanation of each process indicates a step in the flowchart.

[0019] In S301, the CPU 101 acquires a reference image 401 for inspection. Specifically, the reference image input unit 201 acquires the reference image 401 for inspection from the RAM 102 or the main storage device 104. The reference image input unit 201 inputs the acquired reference image 401 for inspection to the print position deviation amount acquisition unit 203, the alignment unit 205, and the inspection unit 206. In this embodiment, as shown in FIG. 4(a), a pattern 403 is printed on a printing paper 402 of the reference image 401. Here, the pattern 403 included in the reference image 401 corresponds to the reference pattern that serves as the inspection standard. In FIG. 4(a), the area outside the printing paper 402 and filled with a single color indicates a state in which reflected light from the printing paper 402 is not obtained, assuming that the area outside the printing paper 402 is also scanned by the scanner 105. Note that in this embodiment, an alignment area 404 is set. The size and position of the alignment area 404 are set so as to include the entire picture 403. Furthermore, the reference image 401 is expressed, for example, as 8-bit grayscale image data. A grayscale image is an image that expresses shades of black and white, as opposed to a color image, with each pixel expressed in 8 bits and containing only brightness information and no color information. With 8-bit grayscale image data, it is possible to express shades up to 2 to the power of 8 = 256 levels. Therefore, a pixel value of 0 indicates a black pixel, and a pixel value of 255 indicates a white pixel.

[0020] In S302, the CPU 101 acquires an inspection image 405. Specifically, the inspection image input unit 202 acquires the inspection image 405 obtained by the scanner 105 reading the printed material to be inspected. The inspection image input unit 202 inputs the acquired inspection image 405 to the print position deviation amount acquisition unit 203, the alignment area determination unit 204, and the alignment unit 205. As shown in FIG. 4(b), a pattern 407 is printed on the printing paper 406 of the inspection image 405. In FIG. 4(b), the area outside the printing paper 406 that is filled with a single color indicates that the area outside the printing paper 406 was also read by the scanner 105, and therefore reflected light from the printing paper 406 was not obtained. Note that in this embodiment, as shown in FIG. 4(b), a pattern 407 is printed on the printing paper 406 of the inspection image 405. Due to a misalignment in the conveyance position of the printing paper 406 during printing, the image 407 is printed at a position shifted downward and to the left from the image 403 of the reference image 401 .

[0021] In S303, the CPU 101 executes a print position deviation amount acquisition process. Specifically, the print position deviation amount acquisition unit 203 compares the print position of the reference image 401 on the printing paper 402 with the print position of the inspection image 405 on the printing paper 406. An example in which the four corners of the printing paper 402 are used as the print position of the reference image 401 on the printing paper 402 and the four corners of the printing paper 406 are used as the print position of the inspection image 405 will be described later. Through the process of S303, the print position deviation amount acquisition unit 203 acquires the print position deviation amount between the reference image 401 and the inspection image 405. The print position deviation amount acquisition process will be described in detail later using FIGS. 5 and 9.

[0022] In S304, the CPU 101 executes alignment area determination processing. Specifically, the alignment area determination unit 204 determines an alignment area for aligning the reference image 401 and the inspection image 405 based on the amount of print position deviation between the reference image 401 and the inspection image 405. Details of the alignment area determination processing will be described later using FIGS. 6 and 10.

[0023] In S305, CPU 101 executes alignment processing. Specifically, alignment unit 205 aligns the alignment region of inspection image 405 with the alignment region of the reference image. Details of the alignment processing will be described later with reference to FIGS. 7 and 11. In S306, CPU 101 inspects the inspection image for print defects. Specifically, inspection unit 206 inspects the inspection image for defects by comparing the inspection image after alignment processing with the reference image. FIG. 8 shows an example of the alignment regions of the reference image and inspection image used in the processing of S306 in FIG. 3. FIG. 8(a) shows an alignment region 404 including a pattern 403 of reference image 401. FIG. 8(b) shows an alignment region 801 including a pattern 407 of inspection image 405. The alignment area 801 in FIG. 8(b) has been aligned by geometric transformation from the alignment area 404 in FIG. 8(a) in accordance with the amount of misalignment between the reference image 401 and the inspection image 405. According to inspection settings preset by the user, the alignment area 404 of the reference image and the aligned alignment area 801 of the inspection image are compared and inspected. In this embodiment, since no printing defects were detected in the alignment area 801 of the inspection image 405, the CPU 101 outputs a pass inspection result to the RAM 102 or main storage device 104 and the printing device 190, and ends the process.

[0024] (Image processing by the print position deviation amount acquisition unit 203) The print position deviation amount acquisition process of S303 in FIG. 3 will be described in detail below with reference to FIGS. 5 and 9. FIG. 5 is a flowchart illustrating the print position deviation amount acquisition process of S303 in FIG. 3. FIG. 9 is a diagram illustrating an example of the four corner positions of the paper for the reference image 401 and the inspection images 405 and 915. Here, the four corner positions of the paper for the reference image 401 refer to the four corner positions of the printing paper 402. Furthermore, the four corner positions of the paper for the inspection image 405 refer to the four corner positions of the printing paper 406. Furthermore, the four corner positions of the paper for the inspection image 915 refer to the four corner positions of the printing paper 906. FIG. 9(a) is a diagram illustrating an example of the four corner positions of the paper for the reference image 401. FIG. 9(b) is a diagram illustrating an example of feature points of the picture 403 in the reference image 401. FIG. 9(c) is a diagram illustrating an example of feature points of the picture 407 in the inspection image 405. 9(d) is a diagram showing an example of an inspection image 915 generated by aligning the pixel positions of the feature points of the pattern 907 in the inspection image 405 with the pixel positions of the feature points of the pattern 403 in the reference image 401. A program for executing the contents of the flowchart shown in FIG.

[0025] That is, the process shown in Fig. 5 is realized by CPU 101 reading a program stored in ROM 103 into RAM 102 and executing it. Specifically, the process shown in Fig. 5 is executed when the process of S303 is called. Note that some or all of the functions of the steps in Fig. 5 may be realized by hardware such as an ASIC or electronic circuit. The symbol "S" in the explanation of each process indicates a step in the flowchart.

[0026] In S501, the CPU 101 acquires the positions of the four paper corners of the reference image 401. Specifically, the print position deviation amount acquisition unit 203 binarizes the reference image 401, which is a grayscale image, using a known modal method, Otsu's method, or the like. The modal method converts image data into two values—black and white—using the valley value as a threshold when the image data has bimodal characteristics. Otsu's binarization method assumes that an image composed of image data can be divided into two classes, bright and dark image areas, and converts the data into two values—black and white—using the threshold value that maximizes the degree of separation between the classes. The degree of separation is expressed as inter-class variance / intra-class variance = inter-class variance / (total variance - inter-class variance). Therefore, in Otsu's binarization method, it is sufficient to increase the inter-class variance. Furthermore, whether the modal method or Otsu's method is selected, the pixel values ​​are converted into either a pixel value of 0, which indicates a black pixel, or a pixel value of 255, which indicates a white pixel. Furthermore, each pixel value is stored as a set with its pixel position. Since the edges of the printing paper 402, the printing paper 406, and the printing paper 906 are converted to white pixels, the positions of the paper edges can be found from the pixel positions stored together with the white pixels. Specifically, the print position deviation amount acquisition unit 203 acquires, from among the paper edge pixels that have become white pixels with a pixel value of 255, the pixels closest to the upper left, upper right, lower right, and lower left edges of the printing paper 402 of the reference image 401 as the paper corner positions of the reference image 401. In this embodiment, point C in FIG. 9(a) 901 ~C 904 Pixel position (x 901 ,y 901 )~(x 904 ,y 904 ) are acquired as the positions of the four corners of the paper of the reference image 401.

[0027] In S502, the CPU 101 calculates parameters for matching the pixel positions of feature points in the reference image 401 with those in the inspection image 405. Here, the feature points of the reference image 401 refer to feature points of the pattern 403 on the printing paper 402 of the reference image 401. Furthermore, the feature points of the inspection image 405 refer to feature points of the pattern 403 recorded on the printing paper 402 of the inspection image 405. Specifically, the print position deviation amount acquisition unit 203 detects the feature points of the reference image 401 and the inspection image 405 using known Sobel filter processing, Harris corner detection processing, or the like. The Sobel filter processing is a filtering process that emphasizes edges, which are boundaries between bright and dark areas in an image, by taking the difference between adjacent pixels, weighting the difference in the same direction as the pixel of interest, and averaging it. The Harris corner detection processing is a process that detects points where pixel values ​​change significantly by multiplying the square of the difference between the pixel value after movement and the pixel value at the original position by the value of a window function, and calculating the sum of the movement in all directions. Whether the Sobel filter process or the Harris corner detection process is selected, the boundary between the bright and dark areas in the image is obtained. In this embodiment, the point F in FIG. 905 ~F 909 is detected as a feature point of the reference image 401. Also, point F in FIG. 910 ~F 914are detected as feature points in the inspection image 405. The print misalignment amount acquisition unit 203 calculates geometric transformation parameters that match the detected feature points in the reference image 401 with those in the inspection image 405. Matching may be performed using feature values ​​extracted by the well-known AKAZE (Accelerated KAZE) algorithm, or by the K-nearest neighbor method. AKAZE is a faster version of KAZE that detects feature points and describes feature values. These results may be used for matching, such as brute-force matching. On the other hand, the K-nearest neighbor method is a matching technique that finds the K feature points closest to a given feature point and predicts the class of the given feature point based on a majority vote. If alignment is possible using the above-mentioned various types of matching, the print misalignment amount acquisition unit 203 calculates the geometric transformation parameters required for alignment as feature point alignment parameters. The geometric transformation parameters may be calculated as parameters required for well-known affine transformation, projective transformation matrices, etc.

[0028] In S503, CPU 101 acquires the positions of the four corners of the paper of inspection image 405. Specifically, print position deviation amount acquisition unit 203 calculates the positions of the four corners of the paper of inspection image 405 when the feature points of inspection image 405 and the feature points of reference image 401 are aligned, based on the feature point alignment parameters calculated in the process of S502. In this embodiment, as shown in FIG. 9D , inspection image 915 is generated in a state in which the feature points of inspection image 405 are aligned with the pixel positions of the feature points of reference image 401, based on the feature point alignment parameters calculated in the process of S502. Here, the state in which the feature points of inspection image 405 are aligned with the pixel positions of the feature points of reference image 401 means a state in which inspection image 405 is geometrically transformed into inspection image 915 based on the feature point alignment parameters. Therefore, inspection image 915 is a transformed inspection image of inspection image 405. Furthermore, in the same manner as in the process of S501, the inspection image 915 is subjected to the process of determining the four corner points C 916 ~C 919 Pixel position (x 916 ,y 916 )~(x 919,y 919 In the process of S504, the print position deviation amount acquisition unit 203 calculates the print position deviation amount (Δx, Δy) from the pixel positions of the four corners of the paper of the reference image 401 calculated in the processes of S501 and S503 and the pixel positions of the four corners of the paper of the inspection image 915 according to the following formula (1).

[0029] TIFF2026025429000002.tif6170

[0030] That is, CPU 101 performs the following calculation based on the four corners of the transformed paper included in inspection image 915, which is a coordinate-transformed inspection image obtained by geometrically transforming inspection image 405 based on the geometric transformation parameters, and the four corners of the reference paper included in reference image 401. That is, CPU 101 calculates the amount of print position deviation in two-dimensional coordinates. As described above, inspection image 405 is geometrically transformed to inspection image 915 based on the geometric transformation parameters. This process translates the coordinate position of image 407 in inspection image 405 to image 907 in inspection image 915 by the amount of deviation from the coordinate position of image 403, which is the reference image in reference image 401. Therefore, the amount of print position deviation can be calculated by calculating the difference between the coordinate positions of the four corners of the reference paper included in reference image 401 and the coordinate positions of the four corners of the paper included in inspection image 915.

[0031] In this embodiment, due to a misalignment of the transport position of the printing paper 406 during printing, the image 407 in Figure 8 is printed at a position shifted to the lower left from the image 403 of the reference image 401, so the amount of print position shift in the lower left direction indicated by the arrow 920 in Figure 9(d) is calculated.

[0032] (Image processing by the alignment area determination unit 204) Details of the alignment region determination process of S304 in FIG. 3 will be described below with reference to FIGS. 6 and 10. FIG. 6 is a flowchart illustrating the alignment region determination process of S304 in FIG. 3. FIG. 10 is a diagram showing examples of the alignment regions of the reference image 401 and the inspection image 405 used in the process of S304 in FIG. 3. FIG. 10(a) is a diagram showing an example of the alignment region 404 of the reference image 401 used in the process of S304 in FIG. 3. FIG. 10(b) is a diagram showing an example of the alignment region 801 of the inspection image 405 used in the process of S304 in FIG. 3. A program for executing the contents of the flowchart shown in FIG. 6 is stored in ROM 103.

[0033] That is, the process shown in Fig. 6 is realized by CPU 101 reading a program stored in ROM 103 into RAM 102 and executing it. Specifically, the process shown in Fig. 6 is executed when the process of S304 is called. Note that some or all of the functions of the steps in Fig. 6 may be realized by hardware such as an ASIC or electronic circuit. The symbol "S" in the explanation of each process indicates a step in the flowchart.

[0034] In S601, the CPU 101 acquires the alignment area 404 of the reference image 401. Specifically, the reference image input unit 201 acquires the alignment area 404 of the reference image 401 from the RAM 102 or the main storage device 104. The reference image input unit 201 inputs the acquired alignment area 404 to the alignment area determination unit 204. In this embodiment, as shown in FIG. 10(a), the alignment area 404 is set within the range of the printing paper 402 as an area that includes all of the image 403 printed on the printing paper 402 of the reference image 401. The position of the upper left corner of the alignment area 404 is determined by selecting the upper left point C 901is set as the origin, and the position is (w0, h0) away in the lower right direction. The shape of the alignment area 404 is a rectangle specified by width w and height h. The width w is set to include the entire horizontal shape of the picture 403 and is the shortest length. The height h is set to include the entire vertical shape of the picture 403 and is the shortest length. Furthermore, w0 is the distance from point C 901 and point C 904 h0 is determined by the shortest distance between the line segment connecting point C and the left edge of the alignment area 404. 901 and point C 902 The distance is determined by the shortest distance between the line segment connecting the two and the top edge of the alignment area 404.

[0035] In S602, the CPU 101 acquires the amount of print misalignment. Specifically, the print misalignment amount acquisition unit 203 inputs the amount of print misalignment (Δx, Δy) to the alignment area determination unit 204. That is, the alignment area determination unit 204 acquires the amount of print misalignment (Δx, Δy) acquired in the processing of S504.

[0036] In S603, the CPU 101 translates the rectangular region 1001 of the inspection image 405 to the alignment region 801 by the amount of print positional deviation. Specifically, the alignment region determination unit 204 performs the following process based on the alignment region 404 of the reference image 401 input in the process of S601 and the amount of print positional deviation (Δx, Δy) input in the process of S602. That is, the alignment region determination unit 204 translates the rectangular region 1001 of the inspection image 405 by the amount of print positional deviation (Δx, Δy). Specifically, in this embodiment, as shown in FIG. 10(b), the CPU 101 calculates the rectangular region 1001 based on the alignment region 404 of the reference image 401. The position of the upper left corner of the rectangular region 1001 is determined by determining the upper left point C 916 The origin is set to be a position (w0, h0) away in the lower right direction. The shape of the rectangular area 1001 is specified by a width w and a height h, similar to the alignment area 404 of the reference image 401.

[0037] In S604, the CPU 101 determines the alignment area 801 of the inspection image 405. Specifically, the alignment area determination unit 204 determines the rectangular area 1001 translated in the process of S603 as the alignment area 801 of the inspection image. In this embodiment, the CPU 101 determines the upper left point C 916 is set as the origin, and a position (w0+Δx, h0+Δy) away in the lower right direction is determined as the upper left corner of the alignment area 801. The shape of the alignment area 801 is specified by a width w and a height h, as shown in FIG.

[0038] That is, CPU 101 maps alignment area 404 on reference image 401 onto inspection image 405. This process results in rectangular area 1001 on inspection image 405, which has the same coordinate position and size as alignment area 404. Then, an area obtained by translating rectangular area 1001 by the amount of print misalignment is determined as alignment area 801 of inspection image 405. After this process, if alignment processing is performed using non-rigid alignment between alignment area 801 and alignment area 404, only the area within alignment area 801 will be affected by the alignment. Therefore, the impact of print misalignment on the paper edge of inspection image 405 is reduced.

[0039] (Image processing of alignment unit 205) The registration process of S305 in FIG. 3 will be described in detail below with reference to FIGS. 7 and 11. FIG. 7 is a flowchart illustrating the registration process of S305 in FIG. 3. FIG. 11 is a diagram showing an example of control points arranged in a grid pattern on the inspection image 405 in the process of S701 in FIG. 7. The registration process of S305 is free-form deformations (FFD), also known as non-rigid registration. Non-rigid registration enables registration including not only image shift and rotation but also local scaling and positional deviation. In non-rigid registration, multiple control points that control the shape of the image are arranged in a grid pattern on the image, and the image is deformed by moving each control point individually. FIG. 11(a) is a diagram showing an example of multiple control points arranged on the registration region 801. FIG. 11(b) shows an example of control point P l,m The image region D in the vicinity of l,m 11(c) is a diagram showing an example of a control point coordinate space obtained by coordinate transformation using a control point 1101 to be updated as the origin. A program for executing the contents of the flowchart shown in FIG. 7 is stored in ROM 103.

[0040] That is, the process shown in Fig. 7 is realized by CPU 101 reading a program stored in ROM 103 into RAM 102 and executing it. Specifically, the process shown in Fig. 7 is executed when the process of S305 is called. Note that some or all of the functions of the steps in Fig. 7 may be realized by hardware such as an ASIC or electronic circuit. The symbol "S" in the explanation of each process indicates a step in the flowchart.

[0041] In S701, the CPU 101 arranges control points. Specifically, the alignment unit 205 arranges a plurality of control points on an alignment region 801 of the inspection image 405. In this embodiment, as shown in FIG. 11(a), L×M control points are arranged in a grid pattern on the inspection image 405. At this time, the distance δ between the control points is calculated from L, M and the size of the alignment region 801 of the inspection image 405. In this embodiment, the coordinates of the point 1101 shown in FIG. 11(a) are calculated as the coordinates P of the control point. l,m (l=1,···,L,m=1,···,M). Coordinates P l,m The control point (l=1,···,L, m=1,···,M) represents the control point in the lth row and mth column arranged on the alignment region 801 of the inspection image 405 .

[0042] In S702, the CPU 101 updates the control point. Specifically, the positioning unit 205 updates the position of the control point in the l-th row and m-th column according to the following equation (2).

[0043] TIFF2026025429000003.tif8170

[0044] Here, μ represents a weighting coefficient, which may be changed according to the update speed of the control point, or a fixed value may be set. In this embodiment, μ is set to a fixed value of 0.1. ∇c is the weighting coefficient of the control point P as shown in the following equation (3) and FIG. 11(b). l,m The image region D in the vicinity of l,m is the differential value of the sum of squares of the difference in pixel values ​​between the inspection image and the reference image after alignment.

[0045] TIFF2026025429000004.tif9150

[0046] I'(x,y) is the pixel value of the inspection image after alignment at pixel position (x,y), I ref (x,y) is the pixel value of the reference image at pixel position (x,y). Σ ∈Dl,m is the image region D l,m In this embodiment, the sum of the image area D l,m is the control point P l,mA rectangular area 1102 having a width of ±2δ in the x and y directions is defined as having a center at .

[0047] In S703, CPU 101 updates the pixels. Specifically, alignment unit 205 updates the inspection image in accordance with the following equation (4).

[0048] TIFF2026025429000005.tif6150

[0049] I(x,y) is the pixel value at pixel position (x,y) of the inspection image before alignment. w(x,y) is the pixel position after alignment, as shown in the following equation (5), and is expressed using (u,v). Since w(x,y) is expressed using (u,v), it corresponds to the pixel position (x,y) of the inspection image before alignment.

[0050] TIFF2026025429000006.tif6150

[0051] Specifically, (u, v) is expressed by the following equations (6) and (7). Also, as shown by point 1103 in FIG. 11(c), the pixel position (u, v) is calculated by normalizing the pixel position (x, y) by the distance δ between the control points at the time of placement and calculating the pixel position (u, v) from the control point 1101 (=P l,m ) is the coordinate position converted onto the control point coordinate space with the origin. Therefore, w(x, y) corresponds to the pixel position (x, y) of the inspection image before alignment.

[0052] TIFF2026025429000007.tif11150

[0053] B(t) is a cubic B-spline function as shown in the following equation (8). Since the B-spline function itself has locality, updating a control point only affects nearby control points.

[0054] TIFF2026025429000008.tif21150

[0055] In S704, the CPU 101 determines whether the update is complete. If the update is complete, the CPU 101 ends the process. If the update is not complete, the CPU 101 returns the process of S704 to the process of S702. Specifically, the alignment unit 205 determines whether the update of the control points is complete. In this embodiment, the pixel value I' of the inspection image and the pixel value I of the reference image after alignment are calculated. ref If the difference between the pixel value I' of the inspection image and the pixel value I of the reference image is equal to or less than a predetermined threshold, the registration unit 205 determines that the update of the control points is complete and ends the process. ref If the difference is greater than the predetermined threshold, the positioning unit 205 determines that the update of the control points is not complete, and returns the process of S704 to the process of S702.

[0056] In this embodiment, the alignment region 404 of the reference image 401 shown in FIG. 8(a) is aligned with the alignment region 801 of the inspection image 405 shown in FIG. 8(b). Here, the position of the alignment region 801 is determined by geometric transformation according to the amount of print misalignment. Furthermore, the alignment region 404 of the reference image 401 and the alignment region 801 of the inspection image 405 are compared and inspected according to inspection settings preset by the user. In this embodiment, since no printing defects were detected in the image area on the alignment region 801 of the inspection image 405, the inspection unit 206 outputs a pass inspection result to the RAM 102, main storage device 104, or printing device 190, and ends the process.

[0057] By performing the image processing described above, robust alignment is possible even for inspection images with large print position deviations, and over-detection and paper waste can be reduced.

[0058] Furthermore, in this embodiment, the four corners of the white pixel area of ​​the binary image are acquired as the four corner positions of the printing paper, but they may also be acquired using edge detection processing such as a Sobel filter, and there are no particular limitations on the method for acquiring the paper edge positions.

[0059] Furthermore, in this embodiment, a method of parallel translation of the alignment area has been described using an example in which the paper in the inspection image 405 is not tilted. However, even if the paper is tilted, the alignment area can be geometrically transformed using a known affine transformation, and the geometric transformation method is not limited to parallel translation.

[0060] Furthermore, in this embodiment, a method of geometrically transforming rectangular region 1001 of inspection image 405 into alignment region 801 has been described, but the present invention is not limited to this. For example, for inspection image 915, which has been aligned with reference image 401 in terms of feature points, an image region identical to alignment region 404 of reference image 401 may be determined as the alignment region of inspection image 405. Therefore, inspection image 915 may be geometrically transformed depending on the amount of printing misalignment, and the target of geometric transformation is not particularly limited to the alignment region.

[0061] In this embodiment, in order to calculate the pixel value I'(x, y) of the inspection image after alignment at the pixel position (x, y), P l-1,m-1 ~P l+2,m+2 However, the number of control points is not limited to 16. For example, four neighboring control points may be used, and the number of control points is not particularly limited.

[0062] In this embodiment, the completion of updating of the control points is determined based on the difference in pixel values ​​between the reference image 401 and the inspection image 405 after alignment, but this is not particularly limited. For example, as shown in equation (3), the determination may be made based on whether the absolute value of ∇c is equal to or less than a predetermined threshold, and the determination method is not particularly limited to the difference in pixel values.

[0063] (Second embodiment) In the first embodiment, an alignment area for aligning the reference image and the test image is determined based on the amount of misalignment of the print position. This process enables robust alignment even when the print position misalignment, which is the relative position misalignment between the print paper and the image in the test image, is large. However, if the margins of the print image are narrow, part of the alignment area determined based on the amount of misalignment of the print position, extends off the paper, which can reduce the accuracy of alignment between the reference image, which does not include the area outside the paper, and the test image, which does include the area outside the paper. Therefore, in the second embodiment, the area that extends off the paper from the alignment area geometrically transformed based on the amount of misalignment of the print position, is excluded. This process enables robust alignment even when the print image has narrow margins and the test image has large misalignment. This process will be described with reference to Figures 12 to 15.

[0064] FIG. 12 shows an example in which register marks are further set in each of a reference image 1201 and an inspection image 1206 in the second embodiment. FIG. 12(a) shows an example in which register marks 1204 are further set in the reference image 1201. FIG. 12(b) shows an example in which register marks 1209 are further set in the inspection image 1206. FIG. 13 is a flowchart illustrating the alignment area determination process in the second embodiment. FIG. 14 is a flowchart illustrating the process of S1301 in FIG. 13. FIG. 15 is a diagram illustrating an example of an area to be excluded in S1301 in FIG. 13. FIG. 15(a) shows an area 1403 of an alignment area 1401 in the inspection image 1206 that overlaps with an excluded area 1402 outside the paper. FIG. 15(b) shows an area 1404 excluded from the alignment area 1205 in the reference image 1201.

[0065] In this embodiment, a reference image 1201 is used, as shown in FIG. 12(a). A picture 1203 is printed on a printing sheet 1202 of the reference image 1201. In this embodiment, the picture 1203 is configured such that the picture 403 is laid out six times on the printing sheet 1202. Also, register marks 1204 are printed on the printing sheet 1202. The register marks 1204 indicate the cutting position of each surface. The picture 1203 and the register marks 1204 are to be inspected. Also, an alignment area 1205 is set in the reference image 1201. The alignment area 1205 is set to a position and size that includes all of the pictures 1203 and all of the register marks 1204. Also, the reference image 1201 is assumed to be, for example, 8-bit grayscale image data.

[0066] In this embodiment, an inspection image 1206 is used, as shown in FIG. 12(b). A picture 1208 is printed on a printing paper 1207 of the inspection image 1206. In this embodiment, the picture 1208 is configured such that the picture 403 is placed six times on the printing paper 1207. Print marks 1209 are also printed on the printing paper 1207. The print marks 1209 indicate the cutting position of each face. The picture 1208 and the print marks 1209 are the objects of inspection. In the example of FIG. 12(b), due to a misalignment of the conveyance position of the printing paper 1207 during printing, the picture 1208 and the print marks 1209 are printed at positions shifted to the lower left from the picture 1203 and the print marks 1204 of the reference image 1201.

[0067] (Image processing by the alignment area determination unit 204) The alignment region determination process in Figure 13 is the same as the alignment region determination process in Figure 6 except for the processing in S1301. Therefore, the processing in S1301 will be mainly described below. In S1301, CPU 101 executes exclusion processing. Details of the exclusion processing will be described later using Figures 14 and 15, but this processing excludes from the alignment region any area of ​​rectangular region 1001 translated in the processing in S603 that overlaps with an exclusion region that was previously set to be excluded. Next, details of the exclusion processing will be described using Figure 14.

[0068] In S13011, the CPU 101 sets the area outside the paper in the inspection image 1206 as the exclusion area 1402. Here, the area outside the paper refers to the area of ​​the inspection image 1206 in FIG. 12B excluding the printing paper 1207, and corresponds to the exclusion area 1402 in FIG. 15A. In S13012, the CPU 101 determines whether the coordinates of the rectangular area 1001 translated by the amount of print positional deviation (Δx, Δy) are included in the exclusion area 1402. If the coordinates of the rectangular area 1001 translated by the amount of print positional deviation (Δx, Δy) are included in the exclusion area 1402, the CPU 101 advances the process of S13012 to the process of S13013. In S13013, the CPU 101 excludes the portion of the rectangular area 1001 that is included in the exclusion area 1402 from the alignment area 1401. On the other hand, if the coordinates of the rectangular area 1001 translated by the amount of print position deviation (Δx, Δy) are not included in the exclusion area 1402, the CPU 101 ends the processing of S13012. Note that, as for the alignment area 1205 of the reference image 1201, as will be described later, an area 1404 translated in the opposite direction of the amount of print position deviation (Δx, Δy) is excluded from the alignment area 1205 of the reference image 1201. Here, the area 1404 is indicated by a diagonally shaded area.

[0069] For example, as shown in FIG. 15( a), of the alignment area 1401 obtained by translating the rectangular area 1001 by the amount of print misregistration (Δx, Δy), an area 1403 that overlaps with an exclusion area 1402 outside the paper is excluded from the alignment area 1401 of the inspection image 1206. Here, the exclusion area 1402 is shown as a gray area. The area 1403 is shown as a diagonally shaded area. Similarly, as shown in FIG. 15( b), an area 1404 obtained by translating the excluded area 1403 in the inspection image 1206 in the opposite direction by the amount of print misregistration (Δx, Δy) is excluded from the alignment area 1205 of the reference image 1201. Here, the area 1404 is shown as a diagonally shaded area.

[0070] Therefore, as shown in Figure 15(a), the area of ​​the alignment area 1401 of the inspection image 1206 that overlaps with the exclusion area 1402 outside the paper is excluded. Also, as shown in Figure 15(b), the area of ​​the alignment area 1205 of the reference image 1201 that overlaps with the exclusion area outside the paper is excluded. This image processing makes it possible to align the area on the paper in the alignment area 1401 with the area on the paper in the alignment area 1205. In other words, it becomes possible to align the alignment areas on the paper.

[0071] As described above, by excluding the area that extends outside the paper from the alignment area 1401, which is obtained by geometrically transforming the rectangular area 1001 according to the amount of print position deviation, robust alignment is possible even for inspection images with narrow margins of the printed image and large print position deviation.

[0072] (Third embodiment) In the second embodiment, we described image processing that robustly aligns even test images with narrow margins and large print misalignment by excluding areas of the alignment area geometrically transformed according to the amount of print misalignment that extend off the paper. However, when overprinting onto a previously printed material, the alignment accuracy between the test image, which has large print misalignment between the preprinted and overprinted images, and the reference image, which has small print misalignment between the preprinted and overprinted images, can be reduced. Therefore, in this embodiment, we exclude areas of the alignment area geometrically transformed according to the amount of print misalignment that extend into the preprinted area. This process enables robust alignment even when overprinting onto a previously printed material, even when printing onto a previously printed material. This process will be described using Figures 16 to 19.

[0073] FIG. 16 is a diagram showing an example in which overprinting has been performed on each of a reference image 1501 and an inspection image 1506 in the third embodiment. FIG. 16(a) is a diagram showing an example in which overprinting has been performed on the reference image 1501. FIG. 16(b) is a diagram showing an example in which overprinting has been performed on the inspection image 1506. FIG. 17 is a flowchart illustrating the alignment area determination process in the third embodiment. FIG. 18 is a diagram showing an example of an area to be excluded in S1601 of FIG. 16. FIG. 18(a) is a diagram showing an example in which an area 1703 that overlaps with a pre-print area 1702 is excluded from an alignment area 1701 of the inspection image 1506. FIG. 18(b) is a diagram showing an example in which an area 1704 obtained by translating the area 1703 excluded in the inspection image 1506 is excluded from an alignment area 1505 of the reference image 1501. FIG. 19 is a diagram showing another example of an excluded region in the third embodiment.

[0074] In this embodiment, as shown in Fig. 16(a), a reference image 1501 is used. In the reference image 1501, a picture 1504 is printed on a printing paper 1503 on which a picture 1502 has been printed in advance. The picture 1504 serves as an inspection standard. An alignment area 1505 is also set in the reference image 1501. The alignment area 1505 is set to a position and size that includes the entire picture 1504. The reference image 1501 is, for example, 8-bit grayscale image data.

[0075] 16(b), an inspection image 1506 is used. In the inspection image 1506, a design 1509 is printed on a printing paper 1508 on which a design 1507 has been printed in advance. The design 1509 is the object of inspection. Furthermore, the inspection image 1506 is printed at a position shifted to the left of the design 1504 that has been printed on the reference image 1501 due to a misalignment in the transport position of the printing paper 1508 during the overprinting.

[0076] (Image processing by the alignment area determination unit 204) The alignment area determination process in FIG. 17 is the same as the alignment area determination process in FIG. 6 except for the processing in S1601. Therefore, the following mainly describes the processing in S1601. In S1601, the CPU 101 excludes the pre-print area from the alignment area. Specifically, the alignment area determination unit 204 excludes from the alignment area the area of ​​the rectangular area 1001 translated in the processing in S603 that overlaps with the pre-print area of ​​the inspection image 1506. In this embodiment, as shown in FIG. 18(a), the area obtained by translating the rectangular area 1001 of the inspection image 1506 by the print position deviation amount (Δx, Δy) is set as the alignment area 1701. Of the alignment area 1701, an area 1703 (shaded area) that overlaps with the pre-print area 1702 (gray area) in which the image 1507 is printed is excluded from the alignment area 1701 of the inspection image 1506. Similarly, as shown in Figure 18(b), an area 1704 (shaded area) obtained by translating the area 1703 excluded from the inspection image 1506 in the opposite direction of the printing position shift amount (Δx, Δy) is excluded from the alignment area 1505 of the reference image 1501.

[0077] As described above, by excluding areas that extend into the pre-printing area from the alignment area that has been geometrically transformed according to the amount of printing position shift, it is possible to perform robust alignment against printing position shift even when overprinting a previously printed material.

[0078] In this embodiment, the area overlapping with the pre-printing area is excluded from the alignment area, but when printing on paper that has been pre-cut, the processing position and the printing position may be misaligned. Even in this case, the area overlapping with the pre-processing area may be excluded from the alignment area.

[0079] 19, when a previously printed and scanned image 1901 is used as the reference image, the user may exclude the variable print area from the inspection area when inspecting a variable print material in which the print image differs in each inspection image, such as a barcode 1902. Even in this case, the area that overlaps with the inspection exclusion area 1903 specified by the user may be excluded from the alignment area 1904.

[0080] (Fourth embodiment) In the first to third embodiments, image processing was described in which an alignment area was determined based on the amount of print misalignment. However, if the determined alignment area did not include the user-specified inspection setting area, alignment accuracy in that area could decrease, making it impossible to perform the inspection process using the user-specified inspection settings, or resulting in overdetection. Therefore, in the third embodiment, processing for issuing a warning to the user when the alignment area determined based on the amount of print misalignment does not match the user-specified inspection settings will be described with reference to FIGS. 20 to 22.

[0081] FIG. 20 is a flowchart illustrating image processing executed by the image processing apparatus of FIG. 1 in the fourth embodiment. FIG. 21 is a diagram illustrating an example of various areas set in each of a reference image and an inspection image in the fourth embodiment. FIG. 21(a) is a diagram illustrating an example of a pre-print area 2102 and a priority inspection area 2104 set in a reference image 2101. FIG. 21(b) is a diagram illustrating an example of a pre-print area 2102 and an alignment area 2106 set in an inspection image 2105. FIG. 21(c) is a diagram illustrating an example in which a portion of the priority inspection area 2104 included in the reference image 2101 has been removed. FIG. 22 is a diagram illustrating an example of a user warning notification and an inspection setting change notification in the fourth embodiment. FIG. 22(a) is a diagram illustrating an example of a notification that high-precision inspection cannot be performed. FIG. 22(b) is a diagram illustrating an example of a notification that a change to a simplified inspection will be made.

[0082] (Processing of inspection unit 206) In the image processing in Fig. 20, the processes other than the processes in S2001 and S2002 are the same as those in Fig. 3. The image processing in this embodiment will be described below with reference to Fig. 20.

[0083] In S2001, the CPU 101 acquires inspection setting information. Specifically, the inspection unit 206 acquires the inspection setting information. The inspection setting information is information including at least one of inspection sensitivity and inspection area. The inspection setting information is specified, for example, by the user. For example, as shown in FIG. 21(a), a pre-print area 2102 and a priority inspection area 2104 are set in a reference image 2101. A simple inspection with low inspection sensitivity is performed on the pre-print area 2102. A priority inspection with high inspection sensitivity is performed on the priority inspection area 2104. The priority inspection area 2104 is an area where a pattern 2103 is overprinted. High-precision alignment is performed in the priority inspection area 2104. Therefore, the same area as the priority inspection area 2104 is set as the alignment area.

[0084] In S2002, the CPU 101 issues a notification. Specifically, if there is a mismatch between the inspection setting information acquired in the process of S2001 and the alignment area determined in the process of S304, the inspection unit 206 notifies the user of a warning. For example, FIG. 21(b) shows an example in which the image 2003 is recorded in a position shifted to the left due to a conveyance position shift during overprinting. Also, in the example of FIG. 21(b), a portion of the alignment area 2106 overlaps a portion of the pre-print area 2102 in the inspection image 2105. The alignment area 2106 is an area obtained by translating a rectangular area of ​​the inspection image 2105 using geometric transformation in accordance with the amount of print position shift. Note that an area 2117 overlapping the alignment area 2106 and the pre-print area 2102 is represented by diagonal lines and is excluded from the alignment area 2006. On the other hand, in the example of FIG. 21(c), in the reference image 2101, region 2114, which is obtained by translating region 2117 of inspection image 2105 in the direction opposite to the amount of print misregistration (Δx, Δy), is represented by diagonal lines and is excluded from the key inspection region 2104 of the reference image 2101. Therefore, part of the key inspection region 2104 is excluded from the alignment region 2106. This makes it impossible to align with high precision. Therefore, the alignment region 2106 determined based on the amount of print misregistration does not match the key inspection region 2104. Therefore, in this embodiment, as shown in FIG. 22(a), the result of the mismatch determination with the inspection settings is output to the printing device 190 or the UI panel 108, and a warning is notified to the user.

[0085] As described above, if the alignment area determined based on the amount of print misalignment does not match the user-specified inspection settings, a warning is sent to the user, thereby avoiding overdetection due to reduced alignment accuracy in the mismatched area.

[0086] In addition, in this embodiment, if a mismatch occurs between the inspection settings and the alignment area, a warning is notified to the user. However, the inspection settings may be changed to match the alignment area and the user may be notified of the changes to the inspection settings; the notification method is not limited to a warning. For example, as shown in FIG. 22(b), if the alignment area determined based on the amount of print misalignment does not match the priority inspection area setting, the mismatch area excluded from the alignment area may be changed to a simple inspection area that does not require high-precision alignment. The changes to the inspection settings may be output to the printing device 190 or the UI panel 108 and notified to the user.

[0087] <Other embodiments> Although various examples and embodiments of the present disclosure have been shown and described above, the spirit and scope of the present disclosure are not limited to the specific descriptions in this specification. The present disclosure is not limited to the above-described embodiments, and various modifications may be made. Furthermore, the present disclosure may be realized by appropriately combining parts of the above-described embodiments.

[0088] (Variation 1) For example, although an example of an affine transformation that performs a parallel translation as a geometric transformation has been described, the geometric transformation is not limited to this. A projective transformation may also be used as the geometric transformation.

[0089] (Variation 2) Although the above description is of an example in which the processes of S303, S304, and S305 are executed by the image processing device 100, the present invention is not limited to this. For example, a server (not shown) capable of transmitting and receiving various signals to and from the image processing device 100 via the Internet may execute at least one of the processes of S303, S304, and S305. Alternatively, the print server 180 may execute at least one of the processes of S303, S304, and S305. Alternatively, a cloud service (not shown) capable of providing various services to the image processing device 100 via the Internet may execute at least one of the processes of S303, S304, and S305. In this way, by executing at least one of the processes of S303, S304, and S305 outside the image processing device 100, the load on the CPU 101 can be reduced while allocating the CPU 101's resources to other processes.

[0090] (Variation 3) Furthermore, while an example has been described in which the inspection image 915 is generated in a state in which the feature points of the inspection image 405 are aligned with the pixel positions of the feature points of the reference image 401, the present invention is not limited to this. For example, an inspection image may be generated in a state in which the feature points of the reference image 401 are aligned with the pixel positions of the feature points of the inspection image 405. In short, it is sufficient to be able to acquire the amount of relative printing positional deviation between the inspection image 405 and the reference image 401.

[0091] (Variation 4) Furthermore, for example, although an example in which the shape of the alignment region 801 is rectangular has been described, the shape is not particularly limited to this. The alignment region 801 is determined by the hardware performance of the image processing device 100. For example, if the resolution for determining the alignment region 801 in the image processing device 100 is 3 mm, the shape of the alignment region 801 can be determined at 3 mm intervals.

[0092] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. The program may also be provided by recording it on a computer-readable storage medium.

[0093] The disclosure of the present embodiment includes configurations typified by the following image processing device, printing system, image processing method, and program.

[0094] <Configuration 1> An image processing device that inspects printed matter, an acquisition unit for acquiring a print position deviation amount between a reference image serving as a reference for the inspection and an inspection image that is the object of the inspection; a determining means for determining, in accordance with the amount of print misalignment, a positioning area for the inspection image, the area being a range on a two-dimensional coordinate system for aligning the inspection image with the reference image and including a pattern present in the inspection image; and a registration means for performing registration between the inspection image and the reference image based on the range of the registration area; An image processing device comprising:

[0095] <Configuration 2> 2. The image processing device according to configuration 1, wherein the acquisition means obtains geometric transformation parameters for performing a geometric transformation between feature points of a reference pattern present in the reference image and feature points of the pattern.

[0096] <Configuration 3> The image processing device described in configuration 2 is characterized in that the acquisition means calculates the amount of print position deviation on the two-dimensional coordinate system based on four corners of a transformed paper sheet included in a coordinate-transformed inspection image geometrically transformed from the inspection image based on the geometric transformation parameters, and four corners of a reference paper sheet included in the reference image.

[0097] <Configuration 4> 4. The image processing apparatus according to configuration 3, wherein the acquisition means starts acquiring the amount of print position deviation when the reference image and the test image are supplied.

[0098] <Configuration 5> a rectangular area including the feature points of the pattern is set within the range of the inspection image; 4. The image processing device according to configuration 3, wherein the determining unit determines the rectangular area translated based on the amount of print position deviation as the alignment area.

[0099] <Configuration 6> The image processing device according to configuration 2, wherein the alignment means aligns the position of the reference image with the position of the inspection image by non-rigid alignment between a range of a reference alignment area including the reference pattern and a range of the alignment area.

[0100] <Configuration 7> 2. The image processing device according to configuration 1, wherein the determining means excludes from the alignment area any area that overlaps with a preset exclusion area.

[0101] <Configuration 8> 8. The image processing device according to configuration 7, wherein the determining means includes, within the range of the inspection image, an area outside the paper as the exclusion area.

[0102] <Configuration 9> 8. The image processing device according to claim 7, wherein the determining means includes a pre-print area in the exclusion area within the range of the inspection image.

[0103] <Configuration 10> 8. The image processing device according to claim 7, wherein the determining means includes a pre-processing area within the range of the inspection image in the exclusion area.

[0104] <Configuration 11> 8. The image processing apparatus according to claim 7, wherein the determining means includes a user-specified region within the range of the inspection image in the exclusion region.

[0105] <Configuration 12> 2. The image processing apparatus according to configuration 1, further comprising an inspection means for inspecting the inspection image for print defects when the alignment is performed.

[0106] <Configuration 13> The image processing device according to configuration 1 further comprises a notification means for issuing a warning to a user when there is a mismatch between the priority inspection area within the range of the inspection image and the alignment area.

[0107] <Configuration 14> The image processing device described in configuration 1 is characterized in that, if there is a portion that does not match between the priority inspection area within the inspection image and the alignment area, it sets the mismatched portion as a simple inspection area and notifies the user of the setting information of the simple inspection area.

[0108] <Configuration 15> an image processing device according to any one of configurations 1 to 14; a print server for generating a print job for the inspection image; a printing device that generates the inspection image on a recording medium based on the print job and transports the recording medium on which the inspection image has been generated to the image processing device; A printing system comprising:

[0109] <Configuration 16> An image processing method for inspecting printed matter, comprising: an acquisition step of acquiring a print position deviation amount between a reference image serving as a reference for the inspection and an inspection image that is the object of the inspection; a determining step of determining, in accordance with the amount of print misalignment, a positioning area for the inspection image, the area being a range on a two-dimensional coordinate system for aligning the inspection image with the reference image and including a pattern present in the inspection image; a registration step of aligning the inspection image with the reference image based on the range of the registration area; An image processing method comprising:

[0110] <Configuration 17> 17. A program for causing a computer to execute each step of the image processing method according to claim 16. [Explanation of symbols]

[0111] 201 Reference image input unit 202 Inspection image input unit 203 Print position deviation amount acquisition unit 204 Alignment area determination unit 205 Alignment section 206 Inspection Department

Claims

1. An image processing device that inspects printed matter, an acquisition unit for acquiring a print position deviation amount between a reference image serving as a reference for the inspection and an inspection image that is the object of the inspection; a determining means for determining, in accordance with the amount of print misalignment, a positioning area for the inspection image, the area being a range on a two-dimensional coordinate system for aligning the inspection image with the reference image and including a pattern present in the inspection image; and a registration means for performing registration between the inspection image and the reference image based on the range of the registration area; An image processing device comprising:

2. 2. The image processing apparatus according to claim 1, wherein said obtaining means obtains geometric transformation parameters for performing a geometric transformation between feature points of a reference pattern present in said reference image and feature points of said pattern.

3. The image processing device according to claim 2, characterized in that the acquisition means calculates the amount of printing position deviation on the two-dimensional coordinate system based on the four corners of the transformed paper contained in the coordinate-transformed inspection image geometrically transformed from the inspection image based on the geometric transformation parameters and the four corners of the reference paper contained in the reference image.

4. 4. The image processing apparatus according to claim 3, wherein the acquisition unit starts acquiring the amount of print position deviation when the reference image and the inspection image are supplied.

5. a rectangular area including the feature points of the pattern is set within the range of the inspection image; 4. The image processing apparatus according to claim 3, wherein the determining unit determines the rectangular area translated based on the amount of print position deviation as the alignment area.

6. 3. The image processing device according to claim 2, wherein the alignment means aligns the position of the reference image with the position of the inspection image by non-rigid alignment between a range of a reference alignment area including the reference pattern and a range of the alignment area.

7. 2. The image processing apparatus according to claim 1, wherein the determining unit excludes from the alignment area any area of ​​the alignment area that overlaps with a preset exclusion area.

8. 8. The image processing apparatus according to claim 7, wherein the determining unit includes, in the exclusion area, an area outside the paper within the range of the inspection image.

9. The image processing apparatus according to claim 7 , wherein the determining unit includes a pre-print area in the exclusion area within the range of the inspection image.

10. 8. The image processing apparatus according to claim 7, wherein the determining means includes a pre-processing area within the range of the inspection image in the exclusion area.

11. 8. The image processing apparatus according to claim 7, wherein the determining means includes a user-specified region within the range of the inspection image in the exclusion region.

12. 2. The image processing apparatus according to claim 1, further comprising an inspection unit that inspects the inspection image for print defects when the alignment is completed.

13. 2. The image processing apparatus according to claim 1, further comprising a notification unit that issues a warning to a user when there is a mismatch between the priority inspection area within the range of the inspection image and the alignment area.

14. The image processing device described in claim 1 further comprises a notification means for, when there is a mismatch between the priority inspection area within the inspection image and the alignment area, setting the mismatched area as a simple inspection area and notifying the user of the setting information of the simple inspection area.

15. An image processing device according to any one of claims 1 to 14; a print server for generating a print job for the inspection image; a printing device that generates the inspection image on a recording medium based on the print job and transports the recording medium on which the inspection image has been generated to the image processing device; A printing system comprising:

16. An image processing method for inspecting printed matter, comprising: an acquisition step of acquiring a print position deviation amount between a reference image serving as a reference for the inspection and an inspection image that is the object of the inspection; a determining step of determining, in accordance with the amount of print misalignment, a positioning area for the inspection image, the area being a range on a two-dimensional coordinate system for aligning the inspection image with the reference image and including a pattern present in the inspection image; a registration step of aligning the inspection image with the reference image based on the range of the registration area; An image processing method comprising:

17. A program for causing a computer to execute each step of the image processing method according to claim 16.

Images