Image processing device, image processing method, and program

Abstract

To perform positioning of an image obtained by reading a print material at an image as a reference of an inspection with high accuracy even in the case where a marker is printed for one inspection unit only one by one.SOLUTION: An image processing device executes image processing on the basis of image data obtained by reading a print image continuously printed on a sheet. One marker is printed on the print image for one inspection unit. The image processing device acquires inspection object image as image data obtained by reading the print image in the one inspection unit as an inspection object, and an adjacent image as image data obtained by reading a region that is adjacent in a conveyance direction of the sheet of the print image of the one inspection unit as an inspection object and contains the marker. The image processing device generates a positioning image made by positioning the inspection object image on a reference image on the basis of positions of the marker on the inspection object image and the marker on the adjacent image.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to image registration processes for inspecting printed matter. [Background technology]

[0002] In a printing process for forming an image on a recording medium, stains may occur when color materials such as ink or toner adhere to unintended locations. Also, color loss may occur, where a sufficient amount of color material does not adhere to a location where an image is to be formed, resulting in a lighter color than intended. Print defects such as stains and color loss reduce the quality of printed matter. Therefore, it is necessary to guarantee the quality of printed matter by inspecting for the presence or absence of print defects. Visual inspection to check for the presence or absence of print defects requires a lot of time and cost, so an inspection system that performs the inspection automatically has been proposed.

[0003] Such an inspection system, for example, aligns a reference image registered in advance with a scanned image of the printed matter to be inspected, and detects the presence or absence of defects based on the difference. As a method of alignment, as shown in Patent Document 1, a method is known in which position detection markers printed on the four corners of the paper are used, the position detection markers are read to obtain the amount of misalignment of the printed matter, and geometric correction is performed. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-092777 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, to reduce ink consumption, there are cases where only one marker for position detection is printed per unit of the inspection target. In this case, there is an issue that paper misalignment accumulates at positions far from the marker, reducing the registration accuracy. In particular, when a continuous sheet such as roll paper is used as the recording medium, the entire scanned image may expand or contract due to a missynchronization between the paper transport speed and the scanner scanning speed. This results in a large amount of misalignment at positions far from the marker, and there is a risk of reducing the registration accuracy.

[0006] The present disclosure aims to align, with high accuracy, an image obtained by scanning a printed material with an image serving as a reference for inspection, even when only one marker is printed per inspection unit. [Means for solving the problem]

[0007] The image processing device disclosed herein is an image processing device that performs image processing based on image data obtained by reading a printed image that is continuously printed on a sheet, and the printed image is affixed with one marker per inspection unit, and is characterized in that the image processing device is equipped with an acquisition means that acquires an inspection target image, which is image data obtained by reading the printed image of the inspection unit to be inspected, and an adjacent image, which is image data obtained by reading an area adjacent to the printed image of the inspection unit to be inspected in the conveying direction of the sheet and including the marker, and a generation means that generates an aligned image in which the inspection target image is aligned with a reference image based on the positions of the marker on the inspection target image and the adjacent image. Effect of the Invention

[0008] According to the present disclosure, even if only one marker is printed per inspection unit, it is possible to align an image obtained by scanning the printed material with high accuracy to an image serving as the inspection reference. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating the configuration of an entire printing system including an image processing apparatus. [Diagram 2] FIG. 2 is a block diagram showing a functional configuration of the image processing device. [Diagram 3] 4 is a flowchart showing the overall flow of image processing executed by the image processing device. [Figure 4] FIG. 13 is a diagram illustrating scanning of a print image. [Diagram 5] FIG. 13 is a diagram showing an example of a reference image. [Figure 6] 13 is a flowchart showing the flow of an aligned image generating process. [Figure 7] FIG. 11 is a diagram showing an example of an inspection result. [Figure 8] 11A and 11B are diagrams illustrating an example of a binary image detected by a paper left and right edge detection process. [Figure 9] FIG. 13 is a diagram showing an example of four corners of an inspection target page. [Figure 10] FIG. 13 is a diagram showing an example of alignment to a reference position. [Figure 11] FIG. 13 is a diagram showing an example of alignment processing using only one reference position marker. [Figure 12] FIG. 13 is a diagram showing an example of scanned image data according to the first modified example. [Figure 13] FIG. 11 is a diagram showing an example of scanned image data in the second embodiment. [Figure 14] 10 is a flowchart showing the flow of aligned image generation processing according to the second embodiment. [Figure 15] 10 is a flowchart showing the flow of a marker position detection process according to the second embodiment. [Figure 16] 13A and 13B are diagrams illustrating examples of reference position markers according to the second embodiment. [Figure 17] FIG. 13 is a diagram illustrating an example of scanned image data according to the third embodiment. [Figure 18] 13 is a flowchart showing the flow of aligned image generation processing according to the third embodiment. [Figure 19] FIG. 13 is a diagram showing another example of the arrangement of the reference position markers. [Figure 20] FIG. 13 is a block diagram showing a functional configuration of a fourth embodiment. [Figure 21] 13 is a flowchart showing the flow of image processing according to the fourth embodiment. [Figure 22] FIG. 11 is a diagram showing an example of display image data. [Figure 23] 13 is a flowchart showing the flow of a display process according to the second modified example. [Figure 24] FIG. 13 is a diagram showing an example of display image data of Modification Example 2. [Diagram 25] 13 is a flowchart showing the flow of image processing according to the fifth embodiment. [Figure 26] 13A to 13C are diagrams illustrating examples of a print document and scanned image data according to a fifth embodiment. [Figure 27] 13 is a flowchart showing the flow of aligned image generation processing according to the fifth embodiment. [Figure 28] 11A and 11B are diagrams illustrating the alignment accuracy using a reference position marker; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that the following embodiments do not limit the present disclosure, and all of the combinations of features described in the embodiments are not necessarily essential to the solution of the present disclosure.

[0011] <First embodiment> In the first embodiment, a case will be described in which one reference position marker is printed at the beginning of each page. In this specification, a page refers to a print image for one inspection unit, and multiple pages of print images are printed continuously on a sheet such as roll paper.

[0012] (Printing system 1 configuration) 1 is a diagram showing an example of the overall configuration of a printing system 1 including an image processing device 100 of the present disclosure. As shown in the figure, the printing system 1 has the image processing device 100, a printing server 180, and a printing device 190. The printing server 180 has a function of generating a print job for a document to be printed and submitting the print job to the printing device 190. The printing device 190 has a function of forming an image on a recording medium based on the print job submitted from the printing server 180.

[0013] In this embodiment, the recording medium is a continuous sheet such as rolled paper. The recording medium is not limited to paper, but broadly includes vinyl, cloth, plastic film, metal plate, glass, ceramics, wood, leather, and other materials capable of receiving ink.

[0014] The printing device 190 may be an inkjet printing device, an electrophotographic printing device, or the like. In this embodiment, an inkjet printing device will be described as an example. The printing device 190 has a paper feed unit 191. The paper feed unit 191 holds a pre-set roll of paper so that it can rotate around a horizontal axis, and unwinds and supplies the roll of paper to the printing device 190. When a print job is input, the printing device 190 transports the roll of paper set in the paper feed unit 191 along a transport path 192, forms an image on one or both sides of the roll of paper, and sends it to the image processing device 100.

[0015] The image processing device 100 performs an inspection process to check for defects on the roll paper, i.e., the printed matter, on which the printing device 190 has formed an image and which has been transported via the transport path 192. That is, the image processing device 100 functions as an inspection processing device. As pre-processing for the inspection, the image processing device 100 executes an alignment image generation process to align the page to be inspected with an image that serves as the reference for the inspection (hereinafter, the reference image). The alignment image generation process and the inspection process will be described in detail later.

[0016] The image processing device 100 internally includes a CPU 101, a RAM 102, a ROM 103, a main memory device 104, an image reading device 105, an interface (I / F) 106, a general-purpose I / F 107, a user interface (UI) panel 108, and a main bus 109. The interface (I / F) 106 is an interface with a printing device 190.

[0017] Furthermore, the image processing device 100 has a transport path 110 and a paper discharge unit 111. The transport path 110 is connected to a transport path 192 of the printing device 190, and is a path for transporting printed matter printed by the printing device 190 to the paper discharge unit 111. The paper discharge unit 111 winds up the roll paper printed by the printing device 190 around a horizontal axis.

[0018] The CPU 101 is a processor that comprehensively controls each unit in the image processing device 100. The CPU 101 calls up programs stored in the ROM 103 or the main storage device 104, etc., and executes various processes using the RAM 102 as a work area. The RAM 102 has a volatile storage area and functions as the main memory, work area, etc. of the CPU 101. The ROM 103 has a non-volatile storage area and stores a group of programs executed by the CPU 101.

[0019] The main storage device 104 stores a group of programs executed by the CPU 101, data used in image processing, etc. The group of programs includes programs related to inspection processing and alignment image generation processing, which will be described later.

[0020] Image reading device (scanner) 105 reads one or both sides of a printed material sent from printing device 190 on conveyance path 110, and inputs the read image data to CPU 101. In this embodiment, scanner 105 reads the printed material page by page in order starting from the first page of the printed material, and stores the read image data in RAM 102 or main storage device 104. The reading of printed material by scanner 105 will be described later.

[0021] The printing device I / F 106 is connected to the printing device 190, and synchronizes processing timing with the printing device 190 and transmits and receives information such as the operating status of each other. The general-purpose I / F 107 is a serial bus interface such as USB or IEEE1394, and can connect to an external device, a storage medium, etc. via the general-purpose I / F 107. A user can write data such as a log to a storage medium, transmit the data to an external device, or import some data into the image processing device 100.

[0022] The UI panel 108 includes a display device such as a liquid crystal display, and functions as a user interface for the image processing device 100. The UI panel 108 displays on the display information related to the current status, settings, and inspection results input from the CPU 101. For example, the UI panel 108 displays, as inspection results, image data of prints that have passed the inspection, image data of prints that have failed the inspection due to detected defects, image data of detected defects, and the like. Note that the inspection results may be classified not only into two types, pass and fail, but also into more detailed classifications.

[0023] A UI panel 108 includes input devices such as a touch panel and various operation buttons, receives instructions from a user, and inputs the instructions to the CPU 101. A main bus 109 connects the various components of the image processing apparatus 100 together.

[0024] Additionally, the CPU 101 controls the operations of the image processing device 100 and each part of the printing system 1. For example, the CPU 101 can synchronize the conveying speeds on the conveying paths 110 and 192, change the conveying speed according to the inspection result of the printed matter, or stop conveying. Furthermore, in addition to the CPU 101, the image processing device 100 may include a GPU, which is a processor for image processing.

[0025] Overall, the image processing apparatus 100 conveys a printed material sent from a printing device 190 along a conveying path 110, reads the printed material with a scanner 105, and performs image processing and inspection processing, which will be described below, on the read image data.

[0026] The results of the inspection process are stored in the RAM 102 or the main memory device 104. The roll paper then goes through post-processing, such as cutting and removing unnecessary parts of the label paper from the release paper. In the post-processing, the printed matter is sorted based on the results of the inspection process. In this way, only those products whose quality has been confirmed can be collected as deliverables.

[0027] (Functional configuration of image processing device 100) Fig. 2 is a block diagram showing an example of a functional configuration of the image processing device 100. Each unit shown in Fig. 2 is realized, for example, by supplying a program for realizing the function to the image processing device 100 and having the CPU 101 of the image processing device 100 call and execute the program. As shown in Fig. 2, the image processing device 100 has a scan image acquisition unit 201, a reference image acquisition unit 202, an alignment image generation unit 203, and an inspection unit 204.

[0028] The scanned image acquisition unit 201 acquires an inspection target image, which is image data obtained by scanning a printed image of an inspection target page, and an adjacent image, which is image data obtained by scanning an area before or after the inspection target page and including a reference position marker. The inspection target image and the adjacent image may be directly acquired as image data read by the scanner 105, or may be stored in the RAM 102 or the main storage device 104. The acquired inspection target image and adjacent image are output to the registration image generation unit 203.

[0029] Note that the front and back of a page refer to the front and back in the transport direction of the roll paper. In the following explanation, an adjacent image scanned before the page to be inspected is called a front image, and an adjacent image scanned after the page to be inspected is called a back image. The page to be inspected is the page to be inspected, and all pages are processed in order starting from the first page.

[0030] In this embodiment, a reference position marker, which is a marker for position alignment, is printed at the beginning of each page, and the scan image acquisition unit 201 acquires the rear image as the adjacent image. If the reference position marker is printed at the end of each page, the scan image acquisition unit 201 acquires the front image as the adjacent image. It is not necessary to read image data for one page for the adjacent image, and at least the scan image data of the area where the reference position marker of the rear page (or the front page) is printed is acquired.

[0031] The scanned image acquisition unit 201 acquires the inspection target image and the adjacent image so that the inspection target image of one page (one inspection unit) is present between two reference position markers before and after it. That is, the adjacent image is read so that the reference position marker on the inspection target image and the reference position marker on the adjacent image are adjacent to each other via the printed image of one inspection unit.

[0032] The reference image acquisition unit 202 acquires image data (hereinafter referred to as reference image) that is to be used as a reference for inspection and that is stored in advance in the RAM 102 or the main storage device 104, and outputs the acquired reference image to the inspection unit 204. The reference image will be described later.

[0033] The aligned image generating unit 203 generates an aligned image by aligning the inspection target image with the reference image based on the positions of the reference position marker on the inspection target image and the reference position marker on the adjacent image (the rear image in the first embodiment) acquired by the scan image acquiring unit 201. The generated aligned image is output to the inspection unit 204. The generation of the aligned image will be described in detail later.

[0034] The inspection unit 204 compares the aligned image generated by the aligned image generation unit 203 with the reference image to inspect the presence or absence of defects on the page to be inspected. The inspection result, for example, result information such as "defect present" or "no defect present" and defect image data indicating the position of the detected defect are output to the printing device 190 or the UI panel 108. The output of the inspection result will be described later.

[0035] In this embodiment, the image processing device 100 is configured to have the scan image acquisition unit 201, the reference image acquisition unit 202, the alignment image generation unit 203, and the inspection unit 204 in one device, but these functional units may be configured by combining a plurality of devices. For example, a computer that performs processing up to generating the alignment image and a computer that performs the inspection processing may be configured as separate computers.

[0036] (Operation of image processing device 100) Next, the operation of the image processing device 100 will be described. Fig. 3 is a flowchart showing the overall flow of image processing executed by the image processing device 100. The CPU 101 reads out a program capable of implementing the flowchart shown in Fig. 3 from the RAM 102 or the main storage device 104, loads it in the RAM 102, and executes it to implement each function. In the following description, "S" refers to each step.

[0037] Prior to explaining FIG. 3, the positional relationship between the print head 402 and the scanner 105 of the printing device 190 in this embodiment and the transport direction of the roll paper will be explained with reference to FIG.

[0038] As shown in Figure 4(a), the print head 402 of the printing device 190 is installed perpendicular to the transport direction 400 of the roll paper 401. The scanner 105 is installed forward of the print head 402 in the transport direction 400, perpendicular to the transport direction 400 of the roll paper 401. The direction indicated by the arrow 400 in the figure is the transport direction of the roll paper 401. A printed image 404 is formed on the roll paper 401 by ejecting ink from multiple ejection ports provided in the print head 402 onto the roll paper 401 as it is transported.

[0039] The scanner 105 scans, for each page, the image printed on the roll paper 401. The scanner 105 is, for example, a full-line sensor in which multiple detection elements are arranged across a width equal to or greater than the width of the roll paper.

[0040] The scanning speed of the scanner 105 is synchronized to match the transport speed of the roll paper. That is, the reading speed per line of an image (scanning speed [line / msec]) is adjusted to match the speed at which one line of the image is transported (transport speed [line / msec]). However, in actual operation, fluctuations in the rotation speed of the transport rollers can cause a misalignment with the scanning speed. When a misalignment occurs, the entire scanned image expands or contracts in the transport direction. Also, deflection can occur due to the material of the roll paper 401, causing local distortion in the scanned image.

[0041] 4(b) is a diagram showing an example of a scanned image 406 obtained by reading a printed image 404 on roll paper 401 with the scanner 105. Due to the aforementioned missynchronization between the scanning speed and the transport speed, a scanned image 406 that is stretched vertically is obtained. This shows a case where the roll paper transport speed is relatively slow compared to the scanning speed. Conversely, if the roll paper transport speed is relatively fast compared to the scanning speed, a scanned image that is shortened vertically is obtained.

[0042] Furthermore, in the scanned image 406 shown in Fig. 4(b), due to misalignment in the roll paper width direction when the roll paper is transported, the roll paper 401 is scanned in a state in which its position has been shifted to the right overall. In the scanned image 406 in Fig. 4(b), the black areas on both ends are pixels that indicate the outer areas of the roll paper 401. When the roll paper 401 shifts to the right, the black area on the right is scanned as being narrower than the black area on the left.

[0043] In this embodiment, as shown in Fig. 4(a), reference position markers 404a, 405a, ... are printed at the upper left corner of each of the printed images 404, 405, ... of each page. A marker image 407 indicating the reference position marker 404a of the printed image 404, which is the page to be inspected, is scanned in the scanned image 406 in Fig. 4(b). Furthermore, the reference position marker 405a of the printed image 405, which is the following page, is scanned as in the marker image 410 of the scanned image 409 of the following page shown in Fig. 4(c).

[0044] The process shown in the flowchart of FIG. 3 under such conditions will be described. In S301, the CPU 101 (scanned image acquisition unit 201) acquires a scanned image (image to be inspected) 406 of a page to be inspected stored in the RAM 102 or the main storage device 104.

[0045] As described above, the inspection target image 406 is assumed to be elongated in the vertical direction due to a mismatch between the scan speed and the transport speed, and the roll paper is shifted to the right when scanned due to a positional misalignment in the roll paper width direction. Also, a marker image 407 corresponding to the reference position marker 404a of the inspection target page 404 is shown on the inspection target image 406. Also, it is assumed that a dot-like defect 408 is shown on the inspection target image 406. This defect 408 is, for example, an ink droplet that has dripped from the print head 402 and unintentionally adhered to the printed material.

[0046] In S302, the CPU 101 (scanned image acquisition unit 201) acquires scanned image data (back image 409) of a page behind the page to be inspected, which is stored in the RAM 102 or the main storage device 104.

[0047] 4(c) shows an example of the rear image 409. In this embodiment, one reference position marker is printed at the beginning (upper left) of each page, so the CPU 101 acquires an image area near the beginning of the print image 405 of the rear page, including at least the reference position marker 410, as the rear image 409. The CPU 101 acquires, for example, a scanned image of a predetermined length with a margin taking into account the amount of deviation, as the rear image 409.

[0048] As with the inspection target image 406, the rear image 409 is scanned as an image stretched in the vertical direction due to a mismatch between the scan speed and the roll paper transport speed. Also, the roll paper on the rear image 409 is shifted to the right due to a positional deviation in the roll paper width direction.

[0049] In S303, the CPU 101 (reference image acquisition unit 202) acquires reference image data (hereinafter, referred to as reference image 501) stored in the RAM 102 or the main storage device 104. The reference image 501 is image data that serves as an inspection reference.

[0050] 5(a) is a diagram showing an example of a reference image 501. The reference image 501 is an image that does not include any defects, and in which a reference position marker 502 on the image and the left and right edges of the roll paper are aligned to their respective reference positions. The reference position is determined from the design values ​​of the pixel position of the reference position marker on the scanned image, and the design values ​​of the pixel positions of the left and right edges of the roll paper. These design values ​​are found from the original data of the printed matter, print job information, and scan timing.

[0051] For example, if the manuscript data is A3 size (297 x 420 mm), the print job information has margin areas of 10 mm on each of the top, bottom, left and right, the print position of the reference position marker to be printed 5 x 5 mm in the center of the 10 x 10 mm margin area at the top left of the manuscript data, and the roll paper transport speed is set to 16 [line / msec] (= 16 / 600 dpi * 25.4 [mm / msec]), and scanning is performed with a scan resolution of 600 x 600 dpi and a scan speed of 16 [line / msec], the scanned data on the 59th line (= 2.5 mm / 25.4 * 600 dpi) becomes the reference position.

[0052] In this embodiment, reference image 501 is aligned so that the upper left edge of reference position marker 502 is at point 503, the left edge of the roll paper is on line 504, and the right edge is on line 505. Also, one reference position marker is printed per page, but as shown in Figure 5(b), when short printed materials are continuously printed in the roll paper transport direction, a reference image 506 consisting of multiple printed materials grouped together as one inspection unit may be used.

[0053] In this case, the reference image 506 may be obtained by aligning a scanned image of a printed image on which one reference position marker 507 is printed for each inspection unit to the reference position. Therefore, in the following description, "page" can be read as "inspection unit".

[0054] In this specification, the coordinates in the reference image 501 shown in Fig. 5(a) or the reference image 506 shown in Fig. 5(b) are referred to as the reference coordinates. For example, the vertical expansion or contraction of a scan image (image to be inspected, rear image, etc.) acquired by the scan image acquisition unit 201 is an expansion or contraction relative to the reference images 501, 506. Similarly, a shift in the width direction in a scan image is a shift in the width direction relative to the reference images 501, 506. The Y direction of the reference coordinates is the transport direction of the roll paper, and the X direction is a direction perpendicular to the transport direction.

[0055] The Y coordinate of the reference coordinates is a design value determined by the vertical size of the manuscript data of the print product, the paper size of the print job information, and the scan timing by the scanner 105. The X coordinate is a design value determined by the horizontal size of the manuscript data of the print product, the paper size of the print job information, and the resolution of the scanner 105.

[0056] In S304, the CPU 101 (aligned image generating unit 203) executes aligned image generating processing. In the aligned image generating processing, the CPU 101 detects one reference position marker each from the inspection target image and the rear image acquired in S301 and S302. The CPU 101 generates an image (hereinafter referred to as an aligned image) in which the inspection target image is aligned to the above-mentioned reference positions 503, 504, and 505 based on the positions of the detected reference position markers.

[0057] Details of the registration image generation process will be described later with reference to Fig. 6. The generated registration image and reference images 501, 506 are in a state where the left and right edges of the roll paper and the positions of the reference position markers are aligned.

[0058] In S305, the CPU 101 (inspection unit 204) inspects the presence or absence of defects on the print image of the inspection target page by comparing pixel values ​​between the aligned image generated in S304 and the reference image acquired in S303. The CPU 101 (inspection unit 204) generates a defect image 701 indicating the pixel area of ​​the detected defect 408, for example, as shown in Fig. 7, and outputs it to the printing device 190 or the UI panel 108 as an inspection result, and ends the processing of this flowchart.

[0059] (S304: Alignment image generation process) Next, details of the aligned image generation process of S304 will be described with reference to the flowchart of Fig. 6. The process of each step of the flowchart of Fig. 6 is executed by the CPU 101 of the image processing device 100 (the aligned image generation unit 203 in Fig. 2).

[0060] In S601, the CPU 101 detects the marker position of the inspection target page. For example, when the inspection target image 406 shown in Fig. 4B is acquired, the CPU 101 performs edge detection processing on the inspection target image 406 to detect pixels of the edge portion of the reference position marker 407 on the inspection target image 406. The edge detection processing may be a Sobel filter processing, a Harris corner detection processing, or any other known processing.

[0061] 4(d) is an enlarged view of the vicinity of the reference position marker 407 on the inspection target image 406. As shown in FIG. 4(d), pixels 411-414 on the upper left, upper right, lower right, and lower left edge portions of the reference position marker 407 are detected by edge detection processing. In this embodiment, the CPU 101 determines the position of pixel 411 on the upper left edge of the detected pixels 411-414 on the edge portions as the representative position P of the reference position marker 407. 407 (x 407 ,y 407 The CPU 101 detects the representative position P of the reference position marker 407 detected from the inspection target image 406. 407 (x 407 ,y 407 ) is stored in a predetermined storage area (for example, RAM 102).

[0062] In S602, the CPU 101 determines whether the inspection target image 406 is the start page or the end page of printing. In this embodiment, a reference position marker is printed at the beginning of each page, so in S602 it is determined whether it is the end page. If the inspection target image 406 is the end page, the reference position marker is not printed after it, so the process proceeds to S603. If the inspection target page is not the end page, it is determined that the reference position marker will be printed on the following page, and the process proceeds to S604.

[0063] When the reference position marker is printed at the end of each page, the CPU 101 determines in S602 whether or not it is the start page, and if it is the start page, the process proceeds to S603. If it is not the start page, the process proceeds to S604.

[0064] In S603, the CPU 101 estimates the marker position of a page before the start page or after the end page. In this embodiment, the reference position marker is not printed on a page after the end page. Therefore, the representative position of the reference position marker of the subsequent page is estimated based on the representative position of the reference position marker 407 of the inspection target image 406 detected in S601.

[0065] Specifically, CPU 101 calculates the relative distance between the reference position markers on the pages from the document data of the printed matter and the print job information. For example, if the document data is A3 (297 mm x 420 mm) size and the print job information specifies a margin area of ​​10 mm on each of the top, bottom, left, and right, the relative distance is 420 mm + 10 mm = 430 mm.

[0066] The CPU 101 calculates the relative distance by dividing the calculated relative distance by the representative position P 407 By adding this, the position P of the reference position marker 410 on the back page is 410 (x 410 ,y 410 The CPU 101 estimates the position P of the reference position marker 410 on the next page. 410 (x 410 ,y 410 ) is stored in a predetermined storage area (for example, RAM 102).

[0067] When the inspection target image is a scanned image of the start page, the CPU 101 calculates the relative distance between the reference position markers between the pages as the representative position P of the reference position marker 407 of the inspection target image 406. 407 This estimates the pixel position of the reference position marker on the previous page.

[0068] Furthermore, the method of estimating the marker position on the previous or following page is not limited to adding or subtracting the relative distance between the reference position markers between the pages described above. For example, estimation may be performed by taking into consideration information on the expansion and contraction of the paper or the marker detection position on the previous or following page detected on the previous page.

[0069] In S604, the CPU 101 detects the marker position of the previous or next page. In this embodiment, edge detection processing is performed on the scanned image (rear image) 409 of the rear page shown in FIG. 4C to detect pixels of the edge portion of the reference position marker 410 on the rear image 409. As in S601, the position of the pixel of the upper left edge portion among the pixels of the detected edge portion is set as the representative position P of the reference position marker 410. 410 (x 410 ,y 410 The CPU 101 detects the representative position P of the reference position marker 410 on the back page. 410 (x 410 ,y 410 ) is stored in a predetermined storage area (for example, RAM 102).

[0070] In S605, CPU 101 generates a binary image of the inspection target page. Specifically, CPU 101 performs a binarization process on inspection target image 406 using a method such as the mode method or Otsu's method to generate a binary image. The process of S605 generates, for example, a binary image 800 as shown in Fig. 8. In the binary image 800, the paper white areas at the left and right ends of the roll paper are converted to white pixels with a pixel value of "255".

[0071] In S606, the CPU 101 detects the left and right edges of the paper from the binarized image 800 generated in S605. For example, the CPU 101 determines the pixel position with the smallest X coordinate for the pixel position of y=y0 among the pixels with a pixel value of "255" in the binarized image 800 as the pixel position P L The CPU 101 detects the pixel positions of the left edge of the paper detected for each y pixel position as a point group P L From this, an approximate straight line L indicating the left edge of the paper is obtained by approximation processing such as linear regression processing. L Calculate.

[0072] Similarly, for the right edge of the paper, the CPU 101 determines the pixel position with the maximum x coordinate for the pixel position of y=y0 among the pixels with a pixel value of "255" as the pixel position P RThe CPU 101 detects the pixel positions of the right edge of the paper detected for each y pixel position as a point group P R From the right edge of the paper, the approximate straight line L R Calculate.

[0073] In S607, the CPU 101 determines the four corner positions of the inspection target page. The CPU 101 then calculates the representative positions P of the reference position markers 407 and 410 detected or estimated from the inspection target image 406 and the rear image 409. 407 (x 407 ,y 407 ), P 410 (x 410 ,y 410 ) and the approximate straight line L L , L R From this, the positions of the four corners of the page to be inspected are calculated.

[0074] In this embodiment, the point C1 shown in FIG. 9A is the upper left corner of the page to be inspected, and the point C2 is the upper right corner. The point C1 corresponds to the representative position P of the reference position marker 407 on the image to be inspected 406. 407 (x 407 ,y 407 ) and the approximate straight line L L A straight line L perpendicular to U Point C2 is the intersection point of the line L U and the approximate straight line L at the right edge of the paper R It is the intersection point with.

[0075] 9B, the CPU 101 calculates the point C4 as the lower left corner of the page to be inspected, and the point C3 as the lower right corner. The point C4 corresponds to the representative position P 410 (x 410 ,y 410 ) and the approximate straight line L L A straight line L perpendicular to B Point C3 is the intersection point of the line L B and the approximate straight line L at the right edge of the paper R In this way, the intersection points C1 to C4 are calculated as the four corner positions of the inspection target page. That is, the reference position marker 410 of the rear page is used to determine the four corner positions of the inspection target page.

[0076] In S608, the CPU 101 aligns the inspection target image 406 obtained by scanning with a reference position based on the four corner positions C1 to C4 of the inspection target page determined in S607. In this embodiment, the pixel positions of points C3 and C4 determined from the rear image 409 are converted to pixel positions on the inspection target image 406 based on the manuscript data and print job information of the printed matter and the scan timing.

[0077] Specifically, the document data is A3 size (297 x 420 mm), the print job information specifies margin areas of 10 mm on each of the top, bottom, left and right, the printing position of the reference position marker to be printed at the center of the 10 x 10 mm margin area at the top left of the document data, 5 x 5 mm, the transport position to transport the roll paper centered to the left and right, and a transport speed of 16 [line / msec] (= 16 / 600 dpi * 25.4 [mm / msec]). The scan speed of the line scanner is set to 16 [line / msec] (= 16 / 600 dpi * 25.4 [mm / msec]). When scanning with a resolution of 600 x 600 dpi, a horizontal pixel count of 7620 pixels, and a scan speed of 16 [line / msec], the pixel position of intersection C3 is (7313 pixels ≒ 7620-297 / 254*600) / 2-1,10216 pixels ≒ 59-430 / 254*600), and the pixel position of intersection C4 is (303 pixels ≒ (7620-297 / 254*600) / 2+1,10216 pixels).

[0078] CPU 101 further aligns the upper left and upper right corner positions C1 and C2 of inspection target image 406 and the lower right and lower left corner positions C3 and C4 converted into pixel positions on the inspection target image to the positions of points 1001 to 1004 shown in Fig. 10(a). The alignment can be performed using known geometric transformation processing such as affine transformation and projective transformation.

[0079] Points 1001 to 1004 are design values ​​for the four corner positions of the page to be inspected on the scanned image, which are determined from the original data and print job information of the printed matter, and the scan timing. Points 1001 and 1004 are points on line 504 of the left edge of the roll paper on reference image 501 shown in FIG. 5(a). Points 1002 and 1003 are points on line 505 of the right edge of the roll paper on reference image 501. The y coordinates of points 1001 and 1002 are the same as point 503, which is the design value for the upper left corner position of the reference position marker printed at the beginning of the page. The y coordinates of points 1003 and 1004 are coordinates shifted downward from the y coordinate of point 503 by the page height.

[0080] By carrying out the above-mentioned geometric transformation process, the upper left edge P 407 Point 503, left edge of roll paper L L Line 504, right edge of roll paper L R are aligned on a straight line 505. As a result, as shown in Fig. 10(b), the image to be inspected (aligned image 1000) is aligned to the same reference position as the reference image 501 shown in Fig. 5(a).

[0081] In S609, the CPU 101 detects feature points on the aligned image 1000 generated in S608 and the reference image 501, and aligns the detected feature points with each other. For the detection of feature points, a known feature point detection process such as a Sobel filter process or a Harris corner detection process may be used. For matching of feature points with each other, a known feature amount matching process such as the AKAZE method or the K-nearest neighbor method may be used. For aligning of feature points with each other, a known geometric transformation process such as an affine transformation or a projective transformation is performed.

[0082] Through the above processing, the CPU 101 generates an aligned image 1000 by aligning the inspection object image 406 with the reference image 501 .

[0083] In the example shown in this embodiment, an image stretched vertically as shown in Figures 4(b) and (c) is scanned due to a mismatch between the scanning speed when the printed material is scanned by the scanner 105 and the roll paper transport speed of the printing device 190. In this case, if the scanned image is aligned to the reference position using only the detection position of a single reference position marker 407 printed at the top of each page, an image as shown in Figure 11 is obtained.

[0084] That is, it is possible to align the intersections C1 and C2 with the design points 1001 and 1002 at the upper left and upper right corners of the page to be inspected from the detected position of the reference position marker 407, but alignment is not performed taking into account the vertical expansion and contraction. Therefore, a positional deviation occurs between the bottom of the scanned image and the reference image.

[0085] On the other hand, by aligning the image to be inspected to the reference position based on the detected positions of the reference position marker 407 on the page to be inspected and the reference position marker 410 on the following page, it becomes possible to perform highly accurate alignment taking into account vertical expansion and contraction, as shown in Figure 10(b).

[0086] The inspection process of S305 is performed using the aligned image 1000 generated in S609. In the inspection process, the image processing device 100 compares pixel values ​​of corresponding pixels between the aligned image 1000 generated in S609 and the reference image 501 shown in Fig. 5(a) to inspect the presence or absence of defects on the print image of the page to be inspected. As a result, for example, the defect 408 shown in Fig. 4(b) is detected. The CPU 101 generates a defect image 701 (Fig. 7) in which the position of the defect 408 is clearly indicated by a mark, color, or the like, and outputs the image to the printing device 190 or the UI panel 108 together with information on the presence or absence of the defect.

[0087] By carrying out the image processing as described above, even if one reference position marker is printed on each page, the scanned image of the print of the inspection target page (inspection target image) can be aligned with high accuracy to the reference image using the reference position markers printed on the inspection target page and the page before or after it. As a result, defects such as stains on the inspection target page can be detected with high accuracy.

[0088] In the above example, since a reference position marker is printed at the beginning of each page, the reference position marker of the following page, which is the page adjacent in the roll paper transport direction through the page to be inspected, is used for alignment. Similarly, if one reference position marker is printed at the end of each page, the reference position marker of the previous page, which is the page adjacent in the roll paper transport direction through the page to be inspected, can be used to align the image to be inspected with high precision to the reference image.

[0089] Also, when the page to be inspected is the end page, the marker position of the page after the end page is estimated from the reference position marker relative distance between pages, which is obtained from the manuscript data of the printed matter and the print job information. Similarly, when the page to be inspected is the start page, the marker position of the page before the start page can be estimated from the reference position marker relative distance between pages, which is obtained from the manuscript data of the printed matter and the print job information, and alignment can be performed.

[0090] If the length of the end page in the roll paper transport direction is longer than a predetermined threshold, a synchronization error accumulates between the scan speed when the printed material is scanned by scanner 105 and the roll paper transport speed of printing device 190, reducing the accuracy of estimating the marker position on the following page. In that case, CPU 101 may notify the user of the reduced estimation accuracy.

[0091] Furthermore, the CPU 101 may store history data of the detection positions of the reference position markers detected on a page prior to the page to be inspected in the RAM 102. If the pixel position of the currently detected reference position marker is an outlier compared with the history data group, the CPU 101 may use a pixel position estimated from the history data group instead of the pixel position of the outlier reference position marker.

[0092] <Modification of the first embodiment> Next, a modified example of the first embodiment will be described with reference to Fig. 12. In this modified example, an example of alignment will be described in which markers for other purposes such as correcting ejection position deviations for each color are printed at the top of the first page. Also, each page other than the first page will have one reference position marker printed at the end.

[0093] Fig. 12(a) shows a scanned image of an inspection target page other than the start page (inspection target image 1201), Fig. 12(b) shows a scanned image of a page previous to the inspection target page (previous image 1204), and Fig. 12(c) shows a scanned image of the start page of a print job.

[0094] 12(a) and 12(b), reference position markers 1202 and 1205 are printed at the end of each page to be inspected other than the start page. On the start page, markers A0, A1, and A2 for correcting the ejection position deviation of each color ejected from the print head 402 are printed at the top of the page, and a reference position marker 1214 similar to the other pages is printed at the end, as shown in FIG. 12(c).

[0095] (Operation of image processing device 100) Except for S301, S302, and S304, the flow of image processing in this modified example is the same as that described in the first embodiment with reference to Fig. 3 and Fig. 6. Image processing in this modified example 1 will be described below with reference to Fig. 12.

[0096] In S301, the CPU 101 (scanned image acquisition unit 201) acquires a scanned image (image to be inspected) of a page to be inspected stored in the RAM 102 or the main storage device 104. In this modified example, it is assumed that an image to be inspected 1201 shown in Fig. 12(a) is acquired in S301. Since one reference position marker 1202 is printed at the end of each page, the image to be inspected 1201, which is a scanned image, also has the reference position marker 1202 at the end of the page.

[0097] As an example, due to a mismatch between the scanning speed when scanning a printed material with scanner 105 and the roll paper transport speed of printing device 190, inspection target image 1201 shown in Fig. 12(a) is an image that is stretched vertically. Also, due to a misalignment in the roll paper width direction when the roll paper is transported, the position of the roll paper in inspection target image 1201 is shifted to the right. These image stretches and shifts are based on reference image 501.

[0098] Also, it is assumed that a point-like defect 1203 appears on the inspection target image 1201. This defect 1203 is, for example, an ink droplet that has dripped from the print head 402 and unintentionally adhered to the printed matter.

[0099] In S302, the CPU 101 (scanned image acquisition unit 201) acquires a scanned image of the page previous to the page to be inspected, which is stored in the RAM 102 or the main storage device 104. In this modified example, because one reference position marker is printed at the end of each page, the CPU 101 acquires a previous image 1204 as shown in FIG. 12(b). As with the inspection target image 1201, the previous image 1204 is a scanned image that has expanded or contracted vertically due to a misalignment between the scan speed and the roll paper transport speed. In addition, due to a misalignment in the roll paper width direction when the roll paper is transported, the position of the roll paper on the previous image has shifted to the right.

[0100] The reference image input process in S303 is the same as in the first embodiment.

[0101] In S304, the CPU 101 generates an aligned image by aligning the scanned image of the page to be inspected (image to be inspected 1201) to the reference position from the detected positions of the reference position markers 1202, 1205 on the scanned images of the page to be inspected and the previous page.

[0102] The procedure for the registration image generation process is the same as the procedure shown in Fig. 6. In this modification, one reference position marker is printed at the end of each page, so that for pages to be inspected except the starting page, an image is generated that is registered with the reference position using the reference position markers 1202 and 1205 on the scanned image data of the page to be inspected and the previous page. For the starting page, the image to be inspected is registered with the reference position using, for example, the marker A01 at the top end of the markers A01 to A03 for different purposes printed at the beginning and the reference position marker 1214 printed at the end.

[0103] The start page will now be described. CPU 101 performs edge detection processing on scanned image 1210 of the start page (image to be inspected), and detects pixels, for example, at the upper left edge of marker A01 for correcting ejection position deviation at the beginning and at the lower left edge of reference position marker 1214 at the end, as representative positions. CPU 101 binarizes scanned image 1210 of the start page and detects the left and right edges of the paper. CPU 101 then determines the positions of the four corners of scanned image 1210 of the start page from the detected representative positions of markers A01 and 1214 and the left and right edges of the paper.

[0104] Based on the determined positions of the four corners of the page to be inspected, CPU 101 aligns scanned image 1210 of the start page to the reference position. The reference position is calculated from the manuscript data of the printed matter, print job information, and scan timing. CPU 101 then aligns feature points on scanned image 1210 after alignment and the reference image to generate an aligned image.

[0105] By carrying out the image processing described above, when a reference position marker for alignment is printed at the end of each page, the scanned image can be aligned to the reference position with high accuracy. Also, when a marker for a different purpose from the reference position marker of the print image is printed at the beginning of the first page of a print job, the marker for the different purpose and the reference position marker at the end of the page are used for alignment, so that all pages including the start page can be aligned with high accuracy.

[0106] <Second embodiment> In the first embodiment, a method for aligning a scanned image of a page to be inspected to a reference position using a single reference position marker printed at the beginning or end of each page has been described. However, due to a missynchronization between the scanning speed of the scanner 105 and the roll paper transport speed of the printing device 190, or a misalignment in scanning timing (a deviation from the design value of the scan start time), a part of the reference position marker may extend beyond the scanned image. As a result, the detection accuracy of the reference position marker may decrease.

[0107] Therefore, in the second embodiment, a reference position marker having a characteristic end shape is used. When the reference position marker protrudes from the scanned image of the page to be inspected, the image processing device 100 determines the representative position of the reference position marker by using a part of the reference position marker, i.e., the part of the reference position marker that does not protrude.

[0108] 13(a) is a diagram showing a scanned image (image to be inspected) 1301 of a page to be inspected acquired in the second embodiment. In this embodiment, when a printed material is scanned by the scanner 105, an image stretched in the vertical direction is scanned due to a mismatch between the scanning speed of the scanner 105 and the roll paper transport speed of the printing device 190. Also, due to a misalignment in the width direction of the roll paper when it is transported, the roll paper on the image to be inspected 1301 is shifted to the right.

[0109] In the second embodiment, it is assumed that one reference position marker is printed at the top of each page, and the entire reference position marker 1302 of the page to be inspected is captured in the inspection target image 1301.

[0110] 13B shows a scanned image (rear image) 1303 of the rear page acquired in the second embodiment. As an example, when a printed material is scanned by the scanner 105, a mismatch between the scanning speed of the scanner 105 and the roll paper transport speed of the printing device 190 results in an image that is stretched vertically. Also, the roll paper on the rear image is shifted to the right due to a misalignment in the roll paper width direction when the roll paper is transported. Also, a misalignment in the scan timing when the printed material of the rear page is scanned causes part of the reference position marker 1304 of the rear page to extend beyond the rear image 1303.

[0111] (Operation of the aligned image generating unit 203) The flow of the aligned image generation process in the second embodiment will be described. Fig. 14 is a flowchart showing the flow of aligned image generation processing in the second embodiment. The processing in Fig. 14 is similar to that in the first embodiment (Fig. 6) except for S1401 and S1404. That is, S1402 and S1403 are similar to S602 and S603 in Fig. 6, and S1405 to S1409 are similar to S605 to S609 in Fig. 6.

[0112] In S1401, the CPU 101 detects the marker position of the inspection target page. In S1401, the CPU 101 determines whether the reference position marker 1302 on the inspection target image 1301 protrudes from the inspection target image 1301. If it protrudes, the CPU 101 estimates the representative position of the reference position marker from a partial area of ​​the reference position marker 1302. If it does not protrude, the CPU 101 determines the representative position of the reference position marker from the entire reference position marker 1302. Details of the marker position detection process will be described later with reference to FIG. 15.

[0113] In S1402, the CPU 101 determines whether the inspection target image 407 is the start page or the end page of printing, similarly to S601. If a reference position marker is printed at the top of each page and it is determined in S1402 that it is the end page, it determines that the reference position marker will not be printed on the following page, and proceeds to S1403. If the inspection target page is not the end page, it determines that the reference position marker will be printed on the following page, and proceeds to S1404.

[0114] When the reference position marker is printed at the end of each page, the CPU 101 determines in step S1402 whether or not it is the start page, and if it is the start page, the process proceeds to step S1403.

[0115] In S1403, CPU 101 estimates the marker position of a page before the start page or after the end page, similar to S603. In this embodiment, a reference position marker is not printed on pages after the end page. Therefore, the pixel position of the reference position marker of the subsequent page is estimated based on the pixel position of reference position marker 407 of inspection target image 406 detected in S1401. CPU 101 stores the estimated position of the reference position marker of the subsequent page in a predetermined storage area (for example, RAM 102).

[0116] In S1404, CPU 101 detects the marker position of the previous or next page. In S1404, CPU 101 determines whether the reference position marker 1304 of the scanned image of the previous or next page protrudes from the scanned image. In the second embodiment, since the reference position marker is printed at the beginning of each page, it determines whether the reference position marker 1304 of the scanned image (rear image) 1303 of the rear page protrudes from the rear image 1303.

[0117] If it protrudes, the position of the reference position marker is estimated from a partial region of the reference position marker 1304. If it does not protrude, the representative position of the reference position marker is determined from the entire reference position marker 1304. Details of the marker position detection process will be described later with reference to FIG.

[0118] The representative positions of the reference position markers detected in S1401 and S1404 are stored in a predetermined storage area.

[0119] (Marker position detection processing operation of the aligned image generating unit 203) Here, the marker position detection process in S1401 and S1404 will be described in detail. FIG. 15 is a flowchart showing the flow of the marker position detection process executed in S1401 and S1404.

[0120] In S1501, the CPU 101 (alignment image generating unit 203) detects the characteristic parts (marker characteristic parts) of the reference position marker from the scan images (inspection target image 1301, rear image 1303). The marker characteristic parts are, for example, the end points of the reference position marker or the intersection points of the lines that make up the reference position marker. The CPU 101 performs edge detection processing such as Sobel filter processing or Harris corner detection processing on the target image 1301 to detect pixels of the edge part from the reference position marker 1302, and the detected pixels of the edge part are regarded as the marker characteristic parts.

[0121] FIG. 16(a) is a diagram showing an enlarged view of the vicinity of the reference position marker 1302. The reference position marker 1302 shown in this example is formed of two parallel horizontal lines and one vertical line connecting the midpoints of the horizontal lines. Due to this shape, the endpoints of each line segment and the intersections of the vertical and horizontal lines are detected as edge portions. Also, in order to detect protrusion in the roll paper transport direction, the shape of the reference position marker 1302 is symmetrical overall from top to bottom. Points 1601 to 1606 in FIG. 16(a) indicate pixels of the edge portion detected as a marker characteristic portion.

[0122] In this embodiment, the number of detected edge parts is compared with a predetermined threshold to determine whether all marker features have been detected. For example, in the case of the reference position marker 1302 having the shape shown in Fig. 16(a), the number of marker features (edge ​​parts) is six in total, so the predetermined threshold is set to "6". In other words, if all six marker features are detected, it is determined that there is no protrusion, and if the number of detections is less than six, it is determined that there is protrusion.

[0123] Similarly, the CPU 101 detects marker features from a rear image 1303 shown in FIG. 16(b) is an enlarged view of the vicinity of the reference position marker 1304 on the rear image 1303. End points 1607 and 1609 of the horizontal lines and an intersection point 1608 of the vertical and horizontal lines of the reference position marker 1304 are detected as edges. That is, three marker features are detected from the rear image 1303.

[0124] In S1502, CPU 101 compares the marker features detected from the reference position marker on the scanned image with a preset threshold value, and determines whether all of the marker features have been detected. As described above, in this embodiment, the threshold value is set to six. That is, if the number of edges of the reference position marker detected from the scanned image is six, CPU 101 determines that the reference position marker does not protrude, and proceeds to S1503. If the number of edges of the reference position marker detected from the scanned image is less than six, CPU 101 determines that the reference position marker protrudes, and proceeds to S1504.

[0125] In S1503, the CPU 101 determines the position of the reference position marker by using all the marker features detected in S1501. In the example shown in Fig. 16(a), the reference position marker 1302 is entirely captured on the target image 1301, and edge portions 1601 to 1606 are detected as the marker features. In this embodiment, the pixel position of the upper left point 1601 among the detected marker features is determined as the representative position P of the reference position marker 1302. 1302 (x 1302 ,y 1302) is determined.

[0126] In S1504, the CPU 101 estimates the position of the reference position marker by using some of the detected marker characteristic parts. In the example shown in FIG. 16B, the reference position marker 1304 protrudes from the rear image 1303, and the only detected marker characteristic parts are the three edge parts 1607, 1608, and 1609. In this case, the CPU 101 estimates the representative position P of the reference position marker 1304 on the rear page by using some of the marker characteristic parts detected in S1501. 1304 (x 1304 ,y 1304 ) to determine

[0127] Specifically, the CPU 101 performs template matching processing on the rear image 1303 using a template image 1611 of the lower portion of the reference position marker shown in Fig. 16(c). The CPU 101 calculates the left end point position 1610 of the upper horizontal line of the reference position marker 1304 from the matching position with the template image 1611 and the print size of the reference position marker obtained from the manuscript data of the printout. The calculated end point 1610 is set as the representative position P of the reference position marker 1304 of the rear page. 1304 (x 1304 ,y 1304 ) is determined.

[0128] When the representative position of the reference position marker is determined by the processes of S1503 and S1504, the process of this flowchart ends.

[0129] By using a reference position marker having a characteristic portion at its end and performing the processing of the flowchart in Figure 15, even if the reference position marker extends beyond the scanned image data, the reference position of the reference position marker can be determined based on the positions of some of the reference position markers that do not extend beyond the scanned image data.

[0130] In this embodiment, the reference position marker is printed at the top of each page, and an example has been described in which the upper part of the reference position marker protrudes, but the present invention can also be applied to a case in which the lower part of the reference position marker protrudes.

[0131] That is, CPU 101 performs template matching processing using a template image corresponding to a portion of the marker characteristic portion that does not protrude, and detects the position where the template image matches. Then, the representative position of the reference position marker is determined from the matching position with the template image and the print size of the reference position marker obtained from the manuscript data of the printed matter. If one reference position marker is printed at the end of each page, the pixel position of the lower left edge of the reference position marker on the scanned image data of the page to be inspected and the previous page is determined as the representative position of the reference position marker.

[0132] The subsequent processes (S1405 to S1409) are the same as those in the first embodiment (S605 to S609).

[0133] This allows the image processing device 100 to align the inspection target image to the reference position with high accuracy. In particular, even if a reference position marker printed only once at the top or bottom of a page extends beyond the scanned image due to a missynchronization between the scanning speed of the scanner 105 and the roll paper transport speed, the representative position of the marker is estimated from a partial area of ​​the reference position marker. This allows the inspection target image to be aligned to the reference position with high accuracy.

[0134] The shape of the reference position marker may be any shape that allows determination of whether or not it protrudes in the vertical direction (paper transport direction). Therefore, the reference position marker may be any shape that has features such as edges on the top and bottom, and is not limited to the shape of the reference position marker 1302 in FIG. 16(a). In the above example, the protrusion of the reference position marker is determined by the number of detected edge portions (marker feature portions), but the determination method is not limited to the number of edges. For example, the protrusion may be determined by template matching using a template image that includes the overall shape of the reference position marker.

[0135] <Third embodiment> In the first and second embodiments, a method for detecting the position of each reference position marker printed at the beginning or end of each page and aligning the scanned image of the page to be inspected to the reference position has been described. However, for example, in an inkjet printing device 190, a sudden ejection failure of the print head may cause image quality degradation such as bleeding on the reference position marker of a specific page, resulting in failure to detect the reference position marker.

[0136] Therefore, in the third embodiment, a method is described for aligning a scanned image to a reference position with high precision based on the positions of the reference position markers on the page before and after the abnormal page where detection of the reference position marker has failed, even if detection of the reference position marker has failed.

[0137] 17 shows scanned images 1701, 1702, and 1703 of the (i-1)th page, the ith page, and the (i+1)th page. In this embodiment, the image processing (FIG. 3) described in the first or second embodiment is executed for the scanned image of each page in order from the starting page, and the aligned image generation processing is executed for the scanned image of the ith page.

[0138] 17, in this embodiment, when a printed material is scanned by scanner 105, a mismatch between the scanning speed and the roll paper transport speed of printing device 190 results in the scanned image being elongated in the vertical direction compared to reference image 501 shown in Fig. 5(a), due to a misalignment in the roll paper width direction when the roll paper is transported. Also, the roll paper in the scanned image is shifted to the right compared to reference image 501, due to a misalignment in the roll paper width direction when the roll paper is transported.

[0139] One reference position marker is printed at the top of each page, and reference position markers 1704 to 1706 appear on the scanned images 1701 to 1703 of the (i-1)th to (i+1)th pages, respectively. Also, it is assumed that the reference position marker 1705 on the i-th page has smudged due to a sudden ejection failure of the print head 402 when printing the page to be inspected.

[0140] (Operation of the aligned image generating unit 203) Fig. 18 is a flowchart showing the flow of the aligned image generation process in the third embodiment. In the flowchart in Fig. 18, steps other than S1801 to S1807 are the same as the aligned image generation process shown in Fig. 6 in the first embodiment. The following description will focus on the points that differ from the flowchart in Fig. 6.

[0141] In S1801, the CPU 101 detects the pixel position (representative position) of the reference position marker on the inspection target image, similarly to S601. In S1802, the CPU 101 detects the pixel position (representative position) of the reference position marker of the previous or subsequent page, similarly to S604.

[0142] In this embodiment, among the pixels of the edge portion of the detected reference position marker, the pixel position of the upper left edge portion is set as the representative position of the reference position marker. The representative positions of the reference position markers of each page detected in S1801 and S1802 are stored in a predetermined storage area (for example, RAM 102).

[0143] Next, in S1803, CPU 101 determines whether the reference position marker of the inspection target page has been detected normally. If the detection position of the reference position marker on the target image detected in S1801 is stored in a predetermined storage area (for example, RAM 102), CPU 101 determines that the reference position marker has been detected normally (S1803; YES) and proceeds to S1804. If the detection position of the reference position marker is not stored in a predetermined storage area (for example, RAM 102), CPU 101 determines that the inspection target page is an abnormal page in which the detection of the reference position marker has failed (S1803; NO) and proceeds to S1808.

[0144] 17 is the page to be inspected, the CPU 101 determines that the reference position marker 1704 on the inspection target image 1701 has been detected normally, and proceeds to S1804. Also, when the i-th page is the page to be inspected, the CPU 101 fails to detect the position of the blurred reference position marker 1705 on the target image 1702, determines that it is an abnormal page, and proceeds to S1808.

[0145] In step S1804, the CPU 101 determines whether the reference position marker of the previous or next page has been detected correctly. In this embodiment, since the reference position marker is printed at the top of the page, the CPU 101 performs the determination for the next page.

[0146] CPU 101 determines whether the reference position marker has been detected normally based on whether the detected position of the reference position marker on the scanned image of the subsequent page detected in S1802 is stored in a predetermined storage area (e.g., RAM 102). If the detected position of the reference position marker is stored in a predetermined storage area, it is determined that the reference position marker of the subsequent page has been detected normally, and the process proceeds to S1811. If the detected position of the reference position marker is not stored in a predetermined storage area, it is determined that the subsequent page is an abnormal page in which detection of the reference position marker has failed, and the process proceeds to S1805.

[0147] For example, when page (i-1) is the page to be inspected, the position detection of the smudged reference position marker 1705 on the scanned image data 1702 of the subsequent page (page i) has failed. Therefore, the subsequent page is determined to be an abnormal page in which the position detection of the reference position marker has failed, and the process proceeds to S1805.

[0148] In S1805, CPU 101 determines whether there is a page two pages after (or two pages before) the page to be inspected. If a reference position marker is printed at the top of the page, the page to be inspected and the reference position markers of the page to be inspected and the page to be inspected after are used to align the page to the reference image. If it is determined in S1804 that the reference position marker of the page to be inspected is abnormal, it determines whether there is another page after that, that is, a page two pages after the page to be inspected.

[0149] Similarly, if a reference position marker is printed at the end of the page, the reference position markers of the inspection target page and the previous page are used to align the inspection target page with the reference image. If it is determined in S1804 that the reference position marker of the previous page is abnormal, it is determined whether there is a page before that, that is, a page two pages before the inspection target page.

[0150] If there is a page two pages after (or two pages before) the page to be inspected, proceed to S1806, and if there is no page two pages after (or two pages before) the page to be inspected, proceed to S1807. Note that there is no page two pages after (or two pages before) the page to be inspected, meaning that it is the page one before the end page (or the page one after the start page).

[0151] In S1806, CPU 101 detects the position of the reference position marker two pages after (or two pages before) the page to be inspected. CPU 101 stores the pixel position of the detected reference position marker in a predetermined storage area (e.g., RAM 102). For example, if the page to be inspected is the (i-1)th page, CPU 101 detects the pixel position of reference position marker 1706 on scanned image 1703 of page (i+1), which is two pages after.

[0152] The detection of the pixel position of the reference position marker is the same as the detection process described above. That is, the CPU 101 determines the position of the pixel at the upper left edge of the reference position marker 1706 detected from the scanned image 1703 of the (i+1)th page as the representative position of the reference position marker.

[0153] In S1807, CPU 101 estimates the marker position on the page following (or preceding) the page being inspected. The estimation of the marker position may be performed in the same manner as in S603. Specifically, CPU 101 calculates the relative distance of the reference position marker between pages, which is determined from the manuscript data of the printed matter and the print job information. By adding the calculated relative distance to the pixel position of the reference position marker on the page being inspected detected in S1801, it is possible to estimate the pixel position of the reference position marker 410 on the following page. Note that this estimation method is one example, and is not limited to this method.

[0154] In S1808, CPU 101 determines whether the reference position marker of the previous or next page has been detected normally. In this embodiment, since the reference position marker is printed at the top of the page, the determination is made for the next page. CPU 101 determines whether the reference position marker has been detected normally based on whether the detection position of the reference position marker on the scanned image of the next page detected in S1802 is stored in a predetermined storage area (for example, RAM 102).

[0155] If the detected position of the reference position marker is stored in a predetermined storage area, it is determined that the reference position marker of the subsequent page has been detected normally, and the process proceeds to S1809. If the detected position of the reference position marker is not stored in a predetermined storage area, it is determined that the subsequent page is an abnormal page in which detection of the reference position marker has failed, and the process proceeds to S1810.

[0156] In S1809, the CPU 101 acquires the representative position of the reference position marker on the scan image of the previous page, which was stored in a predetermined storage area when the registration image of the previous page was generated from the page to be inspected. In this embodiment, when the i-th page is the page to be inspected, the CPU 101 inputs the representative position of the reference position marker 1704, which was stored in a predetermined storage area (e.g., the RAM 102) when the registration image of the (i-1)th page was generated on the scan image 1701.

[0157] If the reference position marker is printed at the end of the page, a determination is made about the previous page in S1808. If the detected position of the reference position marker on the scanned image of the previous page detected in S1802 is stored in a predetermined storage area (for example, RAM 102), CPU 101 determines that the reference position marker of the previous page has been correctly detected. In this case, CPU 101 acquires the reference position marker of the previous page and proceeds to S1809. In S1809, CPU 101 acquires a representative position of the reference position marker on the scanned image of the page after the page to be inspected.

[0158] In S1810, the CPU 101 fails to detect the reference position marker on the page to be inspected and the page before or after it, and determines that there are two consecutive abnormal pages, and issues an error notification. The CPU 101 notifies the printing device 190 or the UI panel 108 of the error, and then ends this flowchart.

[0159] If the answer is YES in S1804, or after the processes of S1806, S1807, and S1809, CPU 101 proceeds to the process of S1811 and subsequent steps. The processes of S1811 to S1815 are similar to the processes of S605 to S609, but the calculation of the four corner positions differs depending on which page the reference position marker is detected (estimated).

[0160] The CPU 101 binarizes the inspection target image in S1811, and detects the left and right edges of the paper of the binarized target image in S1812.

[0161] In S1813, the CPU 101 determines the positions of the four corners of the page to be inspected. (1) If the reference position markers of the page to be inspected and the page following it are both detected correctly, the CPU 101 determines the four corner positions using the reference position markers of the page to be inspected and the page following it, as in the first embodiment.

[0162] (2) If the reference position marker of the page to be inspected is detected normally, but the reference position marker of the following page is abnormal, CPU 101 uses the reference position marker of the following page (two pages behind) to determine the four corner positions of the page to be inspected. In other words, the four corner positions are determined using the reference position markers of the page to be inspected and the page two pages behind.

[0163] That is, when the reference position marker of the page to be inspected is detected normally but the reference position marker cannot be detected from the adjacent image, the alignment image generation unit 203 (CPU 101) generates an alignment image based on the reference position marker detected from the image to be inspected and the reference position marker detected from the image adjacent to the adjacent image.

[0164] (3) If the reference position marker of the inspection target page is detected normally but the reference position marker of the following page is abnormal, and there is no page further back (two pages back), CPU 101 estimates the position of the reference position marker of the page one page back from the inspection target page and determines the four corner positions of the inspection target page. In other words, the four corner positions are determined using the reference position marker of the inspection target page and the estimated reference position marker position of the following page.

[0165] In other words, if the reference position marker in the image to be inspected is detected normally but the reference position marker cannot be detected from the adjacent image, and an image adjacent to the adjacent image cannot be obtained, the alignment image generation unit 203 (CPU 101) estimates the position of the reference position marker in the adjacent image based on the position of the reference position marker detected from the image to be inspected and the relative distance of the reference position marker between inspection units (pages) determined based on the manuscript data and print job information, and generates an alignment image based on the estimated position of the reference position marker and the position of the reference position marker detected from the image to be inspected.

[0166] (4) When the reference position marker of the inspection target page is abnormal, if the reference position markers of the previous page and the following page are detected normally, the reference position markers of the previous page and the following page are used to determine the four corner positions of the inspection target page. In other words, if the markers cannot be detected from the inspection target image, the registration image generating unit 203 (CPU 101) generates a registration image based on the markers detected from the adjacent image in front of the inspection target image in the transport direction and the markers detected from the adjacent image in the rear.

[0167] (5) If the reference position marker of the inspection target page is abnormal, and further if the reference position marker of the following page is also abnormal (if the reference position marker cannot be detected from the inspection target image or the adjacent image), the CPU 101 notifies an error.

[0168] The above-mentioned aspects (1) to (5) are examples in which the reference position marker is printed at the top of the page. In the case where the reference position marker is printed at the bottom of the page, the above-mentioned aspects can be applied by reversing the front and back of the page.

[0169] The CPU 101 generates an aligned image in which the four corner positions are aligned with the reference positions of the reference image in S1814, and further aligns the feature points of the aligned image and the reference image in S1815, and then executes the inspection process.

[0170] As described above, when the position detection of the reference position marker fails on a specific page, the CPU 101 performs alignment using the reference position markers on the pages before and after the abnormal page where detection failed. This enables alignment using two markers, and enables highly accurate alignment of the scanned image to the reference position.

[0171] The marker position of the abnormal page may be estimated by interpolation or the like from the detected positions of the reference position markers of pages where the reference position markers are normally detected.

[0172] In addition, an example has been given in which the scanned image of the page to be inspected is aligned using two reference position markers that have been correctly detected, but the scanned image of the page to be inspected may also be aligned using three or more reference position markers.

[0173] <Modification of the third embodiment> In the third embodiment, one reference position marker is printed at the same position on each page (inspection unit), and even if there is an abnormal reference position marker, the scanned image of the inspection target page is aligned to the reference position based on the marker detection positions before and after the abnormal reference position marker. As a variation of this, an example will be described in which alignment is performed using reference position markers printed at different positions on each page depending on the page size.

[0174] FIG. 19 is a diagram illustrating a modified example of the third embodiment. For example, when two types of manuscript data with different manuscript sizes are printed alternately, if the reference position marker is printed at the same position on each page (such as the beginning), there will be some parts where the distance between the reference position markers is long and some parts where it is short between pages. In such a case, the printing position of the reference position marker may be set to a different position on each page, as shown in FIG.

[0175] For example, when printing pages that are short and pages that are long in the roll paper transport direction alternately, reference position markers 1901 and 1903 are added to the top of the short pages 1905 and 1907, and reference position marker 1902 is added to the center of the long page 1906.

[0176] When aligning the scanned image of the second page as the page to be inspected, CPU 101 detects the positions of a total of three reference position markers: reference position marker 1902 at the center of the page, and reference position markers 1901 and 1903 on pages 1 and 3. Then, using these three reference position markers, the scanned image of the second page (image to be inspected) is aligned to the reference position.

[0177] That is, the four corner positions of the scanned image of the second page are determined using the reference position markers 1901, 1903 of the first and third pages, and when aligning the four corner positions to the reference position, the reference position marker 1902 of the second page is also used for alignment. This makes it possible to align the positions with high accuracy without any bias between pages.

[0178] <Fourth embodiment> If a defect is detected in the inspection process, an image 701 showing the inspection result is displayed on the UI panel 108 (FIG. 7). In this case, in a normal display form in which only the page to be inspected is displayed, it is difficult for the user to confirm whether the detected defect is due to a position shift caused by a detection position error of the reference position marker of the previous or following page.

[0179] For example, originally, point P(x p ,y p ), point Q(x q ,y q The positions of the reference position markers of the inspection target page and the following page detected at the position P'(x p ,y p +Δy), point Q'(x q ,y q If the image is detected at a position of -Δy (+Δy), an image that is shifted by -Δy in the transport direction is generated when the image is aligned to the reference position.

[0180] Therefore, in the fourth embodiment, the detected position of the reference position marker on the previous or subsequent page is displayed separately from the display of the inspection result, so that the user can easily visually recognize the detected position error of the marker.

[0181] (Functional configuration of image processing device 100) Fig. 20 is a block diagram showing an example of a functional configuration of the image processing device 100 in the fourth embodiment. Each unit shown in Fig. 20 is realized by supplying a program for realizing the function to the image processing device 100 shown in Fig. 1 and causing the image processing device 100 to execute the program.

[0182] 20, in the fourth embodiment, the image processing device 100 has a scan image acquisition unit 201, a reference image acquisition unit 202, an alignment image generation unit 203, an inspection unit 204, and a display control unit 2001. The units other than the display control unit 2001 are similar to the scan image acquisition unit 201, the reference image acquisition unit 202, the alignment image generation unit 203, and the inspection unit 204 shown in the first embodiment, and therefore are denoted by the same reference symbols and will not be described repeatedly.

[0183] The display control unit 2001 generates a display image of the area near the reference position marker of the previous or next page from the representative position of the reference position marker of the previous or next page detected when generating the aligned image and the input scanned image of the previous or next page. The display control unit 2001 outputs the generated display image to the printing device 190 or displays it on the UI panel 108.

[0184] (Operation of image processing device 100) Fig. 21 is a flowchart showing the flow of image processing executed by CPU 101 in the fourth embodiment. In Fig. 21, each step other than S2105 has the same content as the image processing shown in Fig. 3 in the first embodiment. That is, S2101 to S2104 are the same as S301 to S304 in Fig. 3, and S2106 is the same as S305 in Fig. 3. The process of S2105, which is different from the first embodiment, will be described below.

[0185] In S2105, the CPU 101 (display control unit 2001) generates a display image including an area near the reference position marker from the scanned image of the previous or next page, and outputs it to the printing device 190 or displays it on the UI panel .

[0186] In this embodiment, one reference position marker is printed at the top of each page. In addition, in the registration image generation process in S2104, as shown in FIGS. 9A and 9B, the pixel position P of the reference position marker 407 of the scan image 406 of the page to be inspected and the pixel position P of the reference position marker 410 of the scan image 409 of the following page are determined. 407 (x 407 ,y 407 ), P 410 (x 410 ,y 410 ) is detected.

[0187] In this case, the CPU 101 determines the representative position P of the reference position marker 410 on the back page. 410 (x 410 ,y 410 ) (marker vicinity image 2201) is cut out and displayed together with the inspection result image.

[0188] FIG. 22(a) shows the detection position P of the reference position marker 410 on the back page. 410 (x 410 ,y 410 The marker vicinity image 2201 is an example of an image (marker vicinity image 2201) in the vicinity of the representative position P of the reference position marker 410 from the scanned image of the rear page. 410 (x 410 ,y 410) is an image cut out from the area near the representative position P 410 (x 410 ,y 410 ) and includes all or part of the reference position marker 410. It is preferable that the edge 2202 of the paper or part of the printed image 405 on the rear page is also included so that the positional relationship of the reference position marker 410 with the surrounding images can be understood.

[0189] Fig. 22(b) is a diagram showing a display example of the marker vicinity image 2201 on the UI panel 108. As shown in Fig. 22(b), the display image of the marker vicinity image 2201 is output to the UI panel 108 and displayed alongside the inspection result image 701. Note that the display example of the marker vicinity image 2201 and the arrangement of the inspection result image 701 and the marker vicinity image 2201 are merely examples and are not limited to these examples.

[0190] By performing the image processing described above, if a defect is detected during inspection, the user can easily visually confirm the detection position of the reference position marker on the previous or subsequent page, thereby making it possible to check whether or not there is a position shift due to an error in the detection position of the marker.

[0191] In order to allow the user to more easily visually recognize the position of the reference position marker, the CPU 101 may highlight the representative position of the reference position marker in the marker vicinity image 2201. For example, in the example of FIG. 22(b), the detected position P 410 In addition, the CPU 101 may enlarge or reduce the marker vicinity image 2201 so that the user can easily visually recognize the position of the reference position marker on the previous or next page.

[0192] 22(c), the detection setting area of ​​the reference position marker 410 may be highlighted with a rectangle 2203 or the like. The detection setting area is an image area to which detection processing is applied when the CPU 101 detects the reference position marker from the scanned image. It is assumed that an area in the scanned image in which the reference position marker appears and which takes into account deviations due to paper transport, etc., is set in advance as the detection setting area.

[0193] 22(d), a composite image may be displayed by combining a marker vicinity image 2204 of the following page (or the previous page) with the defect image data 701 of the inspection result. 410 An X mark may be displayed on the

[0194] Furthermore, as shown in FIG. 22(e), a display setting operation section 2205 may be further displayed on the screen displaying the test result image 701 and the image near the marker 2201, so that the display / non-display of the image near the marker 2201 can be switched by the user's operation.

[0195] The display setting operation unit 2205 is an operation unit for inputting a setting for displaying / hiding the marker vicinity image 2201. For example, the display setting operation unit 2205 accepts an operation for switching between "always ON" for always displaying the marker vicinity image 2201, "ON when inspection is NG" for displaying the marker vicinity image 2201 only when a defect is found in the inspection process, and "always OFF" for not always displaying the marker vicinity image 2201. Note that the display setting operation unit 2205 is not limited to the example of FIG. 22(e), and may also be displayed when displaying the detection setting region shown in FIG. 22(c) or when performing the linked display shown in FIG. 22(d).

[0196] As described above, according to the fourth embodiment, when a defect is detected in the inspection process, an image showing the inspection result and an image of the vicinity of the reference position marker on the previous or next page are displayed. This enables the user to visually check whether the defect detected in the inspection process was detected due to a position shift caused by a detection position error of the marker, improving usability.

[0197] <Modification of the fourth embodiment> A modification of the fourth embodiment will be described with reference to FIGS.

[0198] In the fourth embodiment, a method was described in which, when a defect is detected in the inspection process, the detected position of the reference position marker on the previous or next page is displayed so that the user can easily visually confirm it. However, when the reference position marker on a page before the start page or after the end page is estimated in the above-mentioned S603 (first embodiment), even if an image near the estimated position is displayed as the marker vicinity image 2201, only a mark of the estimated position is drawn on the white of the roll paper. Therefore, it is difficult for the user to confirm the estimation accuracy of the estimated position.

[0199] In variant example 2, we explain a method that allows a user to easily visually check the estimation accuracy of the estimated position of the reference position marker on a page before the start page or after the end page by superimposing and displaying image data of the area near the reference position marker on the start page or end page.

[0200] (Operation of the display control unit 2001) 23 is a flowchart showing the flow of display processing executed by the display control unit 2001 of the image processing device 100. This flowchart illustrates the processing executed in S2105 of the flowchart in FIG.

[0201] In S2301, CPU 101 determines whether the page to be inspected is the start or end page. If the page to be inspected is the start or end page (S2301; YES), the process proceeds to S2302. If the page to be inspected is not the start or end page (S2301; NO), the process proceeds to S2305. For example, if a reference position marker is printed at the beginning of each page, it is determined in S2301 whether the page to be inspected is the end page. If a reference position marker is printed at the end of each page, it is determined in S2301 whether the page to be inspected is the start page.

[0202] In S2302, CPU 101 generates a display image of the detection position of the reference position marker on the start or end page. When one reference position marker is printed at the top of each page, a marker vicinity image 2401 including an area near the detection position of reference position marker 2402 on the end page (Nth page) is generated as shown in Fig. 24(a). The marker vicinity image 2401 includes the reference position marker 2402 included in the scanned image of the end page (Nth page), a part 2404a (upper left corner of the printed image) of the end page (Nth page), a paper white area 2405, an outer area 2406 of the paper, etc.

[0203] In addition, to make it easier for the user to visually confirm the estimation accuracy of the position of the reference position marker estimated after the end page, the marker vicinity image 2401 is cropped to include a portion 2403 of the printed image contained in the scanned image of the page (page N-1) before the end page (page N).

[0204] In S2303, CPU 101 generates a display image of the estimated position of the reference position marker on a page before the start page or after the end page. In this embodiment, one reference position marker is printed at the beginning of each page, so as shown in Fig. 24(b), an image 2411 of the vicinity of the estimated position 2413 of the reference position marker on the page (N+1 page) after the end page (Nth page) is generated as a display image. Image 2411 shows a part 2404b (the lower left corner of the printed image) of the printed image of the end page (Nth page), a paper white area 2415, an outer area 2412 of the paper, etc.

[0205] A paper white area 2415 and an outer area 2412 of the paper are the scanned image of the following page (page N+1) on which nothing is printed. An estimated position 2413 of the reference position marker on the following page (page N+1) is not actually scanned, so nothing equivalent to the marker appears in the image 2411. To make it easier for the user to visually confirm the accuracy of the estimated position of the reference position marker, the generated image 2411 is cropped to include a portion 2404b (the lower left corner of the printed image) of the end page (page N).

[0206] In S2304, the CPU 101 superimposes an image 2401 near the detected position of the reference position marker on the start or end page generated in S2302 and an image 2411 near the estimated position of the reference position marker on the previous or next page generated in S2303. In this embodiment, as shown in Fig. 24(c), a superimposed image 2421 is generated by superimposing a marker vicinity image 2401 near the detected position of the reference position marker 2402 on the end page and an image 2411 near the estimated position of the reference position marker on the page after the end page.

[0207] By visually checking the positional deviation between the printed image 2403 of the page before the end page and the printed image 2404b of the end page included in the superimposed image 2421, the user can infer the accuracy of the estimated position of the reference position marker. In other words, if there is a positional deviation between the printed image 2403 of the page before the end page and the printed image 2404b of the end page, it can be inferred that there is a corresponding deviation in the estimated position of the reference position marker. It can be inferred that the smaller the positional deviation, the more accurately the estimated position of the reference position marker of the subsequent page is estimated.

[0208] In S2305, CPU 101 generates an image (marker vicinity image) of the vicinity of the detection position of the reference position marker on the previous or next page. For example, when one reference position marker is printed at the top of each page, a marker vicinity image 2431 of the vicinity of the detection position of the reference position marker on the next page is generated as shown in FIG. 24(d).

[0209] In S2306, the CPU 101 performs highlighting processing on the marker vicinity image 2431 generated in S2305 or the superimposed image 2421 generated in S2304. For example, the CPU 101 performs highlighting processing such as drawing a mark at the detected position (representative position) or estimated position of the reference position marker.

[0210] When highlighting is performed on the superimposed image 2421, the CPU 101 generates a highlight image 2441, for example, as shown in FIG. 24(e). A mark 2442 indicating the estimated position of the reference position marker is drawn on the highlight image 2441. Similarly, when highlighting is performed on the generated marker vicinity image 2431 in S2305, the CPU 101 generates a highlight image 2451, for example, as shown in FIG. 24(f). A mark 2452 indicating the detected position of the reference position marker 2402 is drawn on the highlight image 2451. Note that the shape of the mark used for highlighting is merely an example, and is not limited to this example.

[0211] As described above, in the second modification, when the CPU 101 displays the estimated position of the reference position marker on a page before the start page or after the end page, the CPU 101 displays an image of the vicinity of the marker detection position on the previous or next page in a superimposed manner, which allows the user to estimate the estimation accuracy.

[0212] <Fifth embodiment> In the above embodiment, the position of the reference position marker of a page before the start page or a page after the end page is estimated from the manuscript data and print job information of the printed matter, and the scan timing. However, if the length of the end page in the roll paper transport direction is longer than a predetermined threshold, a synchronization error accumulates between the scan speed when the printed matter is scanned by the scanner 105 and the roll paper transport speed of the printing device 190, and the estimation accuracy may decrease.

[0213] Therefore, in the fifth embodiment, for the starting page or ending page, a reference position marker is printed at both the beginning and end of the page, and these two reference position markers are used to align the scanned image of the starting page or ending page to the reference position.

[0214] (Operation of image processing device 100) Fig. 25 is a flowchart showing the flow of image processing executed by CPU 101 in the fifth embodiment. In Fig. 25, steps other than S2501 and S2502 are similar to the image processing in the first embodiment (Fig. 3). That is, steps S2503 to S2507 are similar to steps S301 to S305 in Fig. 3. The image processing in this embodiment will be described below, focusing on the points that differ from the image processing in the first embodiment.

[0215] In this embodiment, it is assumed that one reference position marker is printed at the beginning of each page.

[0216] In S2501, CPU 101 determines whether the page to be inspected is the start or end page. In this embodiment, since one reference position marker is printed at the beginning of each page, in S2501, it is determined whether the page to be inspected is the end page. If the page to be inspected is the end page (S2501; YES), the process proceeds to S2502. If the page to be inspected is not the end page (S2501; NO), S2502 is skipped and the process proceeds to S2503.

[0217] When a reference position marker is printed at the end of each page, in S2501, it is determined whether the page to be inspected is the start page, and if it is the start page, the process proceeds to S2502. If not, the process proceeds to S2503.

[0218] In S2502, CPU 101 generates print manuscript data in which reference position markers are added to the beginning and end of the end page, which is the page to be inspected. FIG. 26A is a diagram showing print manuscript data 2601 of the end page. As shown in FIG. 26A, CPU 101 generates print manuscript data for print manuscript data 2601 of the end page in which reference position marker 2603 is added to the end of the page in addition to reference position marker 2602 at the beginning that has already been added to each page. CPU 101 outputs the generated print manuscript data 2601 to printing device 190.

[0219] Printing is performed based on the print document data 2601 output to the printer 190. The printed roll paper is then transported to the scanner 105, which reads the printed image on the roll paper. The read scanned image data is stored in the RAM 102 or the main storage device 104.

[0220] In S2503, the CPU 101 acquires a scanned image (image to be inspected) of the page to be inspected stored in the RAM 102 or the main storage device 104, similarly to S301. In S2504, similarly to S302, the CPU 101 acquires a scanned image of the page before or after the page to be inspected stored in the RAM 102 or the main storage device 104. In this embodiment, since a reference position marker is printed at the top of the page, a scanned image of the after page (after image) is acquired.

[0221] In S2505, the CPU 101 acquires the reference image stored in the RAM 102 or the main storage device 104, similarly to S303.

[0222] In S2506, similarly to S304, CPU 101 detects reference position markers from the inspection target image and the rear image, and generates an image (aligned image) in which the inspection target image is aligned with each reference position of the above-mentioned reference image based on the detected positions. The alignment image generation process for the end page in S2506 will be described later.

[0223] In S2507, CPU 101 inspects the presence or absence of defects on the print image of the inspection target page by comparing pixel values ​​of the aligned image of the inspection target page with the reference image, as in S305. CPU 101 (inspection unit 204) outputs defect image data 701 indicating the pixel area of ​​the detected defect 408 as shown in FIG. 7, for example, to printing device 190 or UI panel 108 as an inspection result, and ends the processing of this flowchart.

[0224] (Operation of the aligned image generating unit 203) Fig. 27 is a flowchart showing the flow of the aligned image generation process executed in S2506 described above. In Fig. 27, the steps other than S2703 are similar to the aligned image generation process (Fig. 6) of the first embodiment. That is, S2701 to S2702 are similar to S601 to S602 in Fig. 6, and S2704 to S2709 are similar to S604 to S609 in Fig. 6. The image processing in this embodiment will be described below, focusing on the differences from the image processing in the first embodiment.

[0225] In S2701, CPU 101 detects the representative position of the reference position marker from the scanned image of the page to be inspected (image to be inspected), similarly to S601. For example, if the reference position marker is printed at the top of the page, CPU 101 detects the pixel position of the upper left edge detected by edge detection processing as the representative position of the reference position marker. If the reference position marker is printed at the bottom of the page, CPU 101 detects the pixel position of the lower left edge as the representative position of the reference position marker.

[0226] In S2702, CPU 101 determines whether the page to be inspected is the start or end page of printing. In this embodiment, since a reference position marker is printed at the beginning of each page, it is determined whether the page to be inspected is the end page. If the page to be inspected is the end page, the process proceeds to S2703. If the page to be inspected is not the end page, the process proceeds to S2704.

[0227] In S2703, the CPU 101 detects the representative position of a reference position marker (hereinafter referred to as an added marker) added to the end of the last page. The detected representative position of the added marker is stored in a predetermined storage area. In this embodiment, the CPU 101 performs edge detection processing such as Sobel filter processing and Harris corner detection processing on the scanned image 2604 of the last page shown in FIG. 26(b). The CPU 101 detects the pixels of the upper left edge portion of the reference position marker 2602 and the lower left edge portion of the added marker 2603 on the image data.

[0228] By the process of S2703, the pixel position of the pixel 2605 at the upper left edge portion of the detected reference position marker 2602 is set to the representative position P 2602 (x 2602 ,y 2602 ) is detected. Also, the pixel position of pixel 2606 at the lower left edge of the additional marker 2603 is detected as the position P 2603 (x 2603 ,y 2603 ) is detected.

[0229] In S2704, the CPU 101 detects the representative position of the reference position marker of the previous or next page (the next page in this embodiment) in the same manner as in S604. The detected representative position of the reference position marker is stored in a predetermined storage area (for example, the RAM 102). The pixel position of the upper left edge of the reference position marker of the next page is detected as the representative position of the reference position marker 410.

[0230] The subsequent processing of S2705 to S2709 is the same as the processing of S605 to S609 in Fig. 6. That is, for a normal page, the four corner positions C1 to C4 of the inspection target page are determined based on the positions of the reference position marker of the inspection target page and the reference position marker of the following page. Also, when the inspection target page is the end page, the four corner positions C1 to C4 of the inspection target page are determined based on the positions of the reference position marker printed at the beginning of the inspection target page and the additional marker added to the end.

[0231] 28(a) is a diagram for explaining a method for determining the positions of the four corners of the end page. As shown in FIG. 28(a), the representative position of the reference position marker 2602 at the top of the page on the scanned image 2604 of the end page is P 2602 (x 2602 ,y 2602 ), and the representative position of the additional marker 2603 at the end of the page is P 2603 (x 2603 ,y 2603 The design value of the pixel position of the upper left corner 1001 of the page to be inspected on the scanned image is (x 1001 ,y 1001 ), and the design value of the pixel position of the lower left corner 1004 is (x 1004 ,y 1004 These design values ​​are calculated from the print document data, print job information, and scan timing.

[0232] 28(b) is a graph showing the change in the magnitude of positional deviation Δy in the roll paper transport direction (Y direction) relative to the design value depending on the Y position. The horizontal axis of the graph represents the Y direction position (position in the roll paper transport direction), and the vertical axis represents the magnitude of positional deviation Δy.

[0233] As shown in FIG. 28B, when the scanned image data 2604 is aligned to the reference position (design value) using only the reference position marker at the top of the page, the y-coordinate y′ of the reference position marker 2602 after alignment is 2602 The y-coordinate error Δy(y´ 2602 However, since it is not possible to perform positioning that takes into account the expansion and contraction in the Y direction, the y coordinate y' 2603 The y-coordinate error Δy(y´ 2603 )=y 2603 -y 2602 -y 1004 +y 1001 and becomes larger.

[0234] On the other hand, when the scanned image data 2604 is aligned to the reference position using the reference position markers 2602 and 2603 at the beginning and end of the last page, alignment can be performed taking into account the expansion and contraction in the Y direction. 2602 )=Δy(y´ 2603 )=0.

[0235] By performing the image processing described above, it is possible to detect the reference position markers printed at both the beginning and end of the starting or ending page, and to align the scanned image of the starting or ending page to the reference position with high precision.

[0236] In this embodiment, the reference position markers printed at both the beginning and end of the page are detected only for the start page or the end page, but a job start / end reference position marker printed at the beginning of the print job start page or the end of the end page may be used for alignment. Therefore, a reference position marker with a different design may be used for alignment on each page, and the design of the reference position marker is not limited to being the same on each page.

[0237] Although the embodiments of the present disclosure have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. In addition, the image processing and display processing described in each embodiment may be appropriately combined. In addition, it is clear that a person skilled in the art can come up with various modified or revised examples within the scope of the technical ideas disclosed in this application, and it is understood that these also naturally belong to the technical scope of the present disclosure.

[0238] <Other embodiments> The present disclosure can also be realized by a process in which a program for implementing one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more functions.

[0239] The disclosure of the above-described embodiment includes the following configurations. (Configuration 1) An image processing device that performs image processing based on image data obtained by reading print images that are continuously printed on a sheet, The printed image is provided with one marker for each inspection unit; an acquisition means for acquiring an inspection target image, which is image data obtained by reading the printed image of the inspection unit to be inspected, and an adjacent image, which is image data obtained by reading an area of ​​the printed image of the inspection unit to be inspected that is adjacent in a conveying direction of the sheet and includes the marker; a generating means for generating a registration image by registering the inspection target image with a reference image based on the positions of the markers on the inspection target image and the markers on the adjacent image; An image processing device comprising:

[0240] (Configuration 2) The image processing device according to configuration 1, wherein the acquisition means acquires the image to be inspected and the adjacent image such that the image to be inspected for one inspection unit is present between the marker on the image to be inspected and the marker on the adjacent image.

[0241] (Configuration 3) if a part of the marker is not detected from the inspection target image or the adjacent image, The image processing device according to configuration 1 or 2, wherein the generating means estimates a position of the marker based on the detected portion of the marker, and generates an aligned image by aligning the image to be inspected with a preset reference image based on the estimated position of the marker.

[0242] (Configuration 4) 4. The image processing device according to configuration 3, wherein the marker has a feature for determining whether a part or an entirety of the marker has been read.

[0243] (Configuration 5) 5. The image processing device according to configuration 4, wherein the characteristic portion appears at an end of the marker in a conveying direction of the sheet.

[0244] (Configuration 6) If the marker cannot be detected from the inspection target image, The image processing device according to configuration 1 or 2, wherein the generation means generates the alignment image based on a marker detected from an adjacent image in front of the inspection target image in the transport direction and a marker detected from an adjacent image in the rear of the inspection target image.

[0245] (Configuration 7) If the marker is normally detected in the inspection target image and the marker cannot be detected in the adjacent image, The image processing device according to configuration 1 or 2, wherein the generation means generates the aligned image based on a marker detected from the image to be inspected and a marker detected from an image adjacent to the adjacent image.

[0246] (Configuration 8) When the marker is normally detected in the image to be inspected, and the marker cannot be detected from the adjacent image, and an image adjacent to the adjacent image cannot be obtained, The image processing device according to configuration 1 or 2, wherein the generation means estimates a position of the marker in the adjacent image based on a position of the marker detected from the image to be inspected and a relative distance of the marker between inspection units determined based on manuscript data and print job information, and generates the aligned image based on the estimated position of the marker and the position of the marker detected from the image to be inspected.

[0247] (Configuration 9) If the marker cannot be detected from the inspection target image and the adjacent image, 3. The image processing device according to configuration 1 or 2, further comprising a notification unit for notifying an error.

[0248] (Configuration 10) The image processing device according to configuration 1 or 2, characterized in that when the print image to be inspected is an inspection unit related to the start or end of printing and there is no adjacent image, the generation means estimates the position of the marker in the adjacent image based on the position of the marker detected from the image to be inspected and the relative distance of the marker between inspection units determined based on manuscript data and print job information, and generates the aligned image based on the estimated position of the marker and the position of the marker detected from the image to be inspected.

[0249] (Configuration 11) 11. The image processing device according to any one of configurations 1 to 10, further comprising a display control means for displaying an image of a region in the vicinity of the marker detected from the adjacent image.

[0250] (Configuration 12) an inspection unit that inspects the print image to be inspected for defects by comparing the alignment image with the reference image; 12. The image processing apparatus according to claim 11, wherein the display control means displays an image of a region adjacent to the marker detected from the adjacent image together with an image showing a result of the inspection by the inspection means.

[0251] (Configuration 13) 12. The image processing device according to claim 11, wherein the display control means displays the position of the marker detected from the adjacent image in an emphasized manner.

[0252] (Configuration 14) 12. The image processing device according to claim 11, wherein the display control means generates a composite image by combining an image of a region in the vicinity of the marker detected from the adjacent image with the registration image, and displays the composite image.

[0253] (Configuration 15) The image processing device according to configuration 11, characterized in that when the print image to be inspected is an inspection unit related to the start or end of printing and there is no adjacent image, the display control means displays the estimated position of the marker in the adjacent image by superimposing it on an image of the area near the marker in the image to be inspected.

[0254] (Configuration 16) 15. The image processing device according to any one of configurations 11 to 14, further comprising an operation means for switching between displaying and not displaying an image of a region in the vicinity of the marker detected from the adjacent image.

[0255] (Configuration 17) The image processing device according to configuration 10, further comprising a warning means for warning that the estimation accuracy of the position of the marker in the adjacent image will decrease if the length of the printed image of the one inspection unit in the transport direction is equal to or longer than a predetermined threshold.

[0256] (Configuration 18) Regarding the inspection unit related to the start or end of printing, a marker adding means for adding a marker to the unit of inspection in addition to the markers added one per unit of inspection, The image processing device according to any one of configurations 1 to 17, characterized in that, for an inspection unit related to the start or end of printing, the generation means generates the alignment image based on the position of the marker that is added one per inspection unit and the position of the marker added by the marker adding means.

[0257] (Configuration 19) An image processing method for performing image processing based on image data obtained by reading print images continuously printed on a sheet, comprising: The printed image is provided with one marker for each inspection unit; acquiring an inspection target image, which is image data obtained by reading the printed image of the inspection unit to be inspected, and an adjacent image, which is image data obtained by reading an area of ​​the printed image of the inspection unit to be inspected that is adjacent to the sheet conveyance direction and includes the marker; generating a registration image by registering the inspection target image with a reference image based on the positions of the markers on the inspection target image and the markers on the adjacent image; 13. An image processing method comprising:

[0258] (Configuration 20) A program for causing a computer to function as an image processing device that reads images printed continuously on a sheet and executes image processing, The printed image is provided with one marker for each inspection unit; an acquisition means for acquiring an inspection target image, which is image data obtained by reading the printed image of the inspection unit to be inspected, and an adjacent image, which is image data obtained by reading an area of ​​the printed image of the inspection unit to be inspected that is adjacent in a conveying direction of the sheet and includes the marker; a generating means for generating a registration image by registering the inspection target image with a preset reference image based on the positions of the markers on the inspection target image and the markers on the adjacent image; A program to function as a

Claims

1. An image processing device that performs image processing based on image data obtained by reading print images that are continuously printed on a sheet, A marker is attached to the printed image for each inspection unit, an acquisition means for acquiring an inspection target image, which is image data obtained by reading the printed image of the inspection unit to be inspected, and an adjacent image, which is image data obtained by reading an area of ​​the printed image of the inspection unit to be inspected that is adjacent in a sheet conveyance direction and includes the marker; a generating means for generating a registration image by registering the inspection target image with a reference image based on the positions of the markers on the inspection target image and the markers on the adjacent image; An image processing device comprising:

2. 2. The image processing device according to claim 1, wherein the acquisition means acquires the inspection target image and the adjacent image such that the inspection target image for one inspection unit is present between the marker on the inspection target image and the marker on the adjacent image.

3. if a part of the marker is not detected from the inspection target image or the adjacent image, 2. The image processing device according to claim 1, wherein the generating means estimates a position of the marker based on the detected portion of the marker, and generates an aligned image by aligning the image to be inspected with a preset reference image based on the estimated position of the marker.

4. 4. The image processing apparatus according to claim 3, wherein the marker has a feature for determining whether the marker has been read in part or in whole.

5. The image processing apparatus according to claim 4 , wherein the characteristic portion appears at an end portion of the marker in a conveying direction of the sheet.

6. If the marker cannot be detected from the inspection target image, The image processing device according to claim 1 , wherein the generating means generates the alignment image based on a marker detected from an adjacent image in front of the inspection target image in the transport direction and a marker detected from an adjacent image in the rear of the inspection target image.

7. If the marker is normally detected in the image to be inspected and the marker cannot be detected in the adjacent image, 2 . The image processing apparatus according to claim 1 , wherein the generating means generates the aligned image based on a marker detected from the image to be inspected and a marker detected from an image adjacent to the adjacent image.

8. When the marker is normally detected in the image to be inspected, and the marker cannot be detected from the adjacent image, and an image adjacent to the adjacent image cannot be obtained, 2. The image processing device according to claim 1, wherein the generating means estimates the position of the marker in the adjacent image based on the position of the marker detected from the image to be inspected and the relative distance of the marker between inspection units determined based on manuscript data and print job information, and generates the aligned image based on the estimated position of the marker and the position of the marker detected from the image to be inspected.

9. If the marker cannot be detected from the inspection target image and the adjacent image, 2. The image processing apparatus according to claim 1, further comprising a notification unit for notifying an error.

10. 2. The image processing device according to claim 1, wherein, when the print image to be inspected is an inspection unit related to the start or end of printing and there is no adjacent image, the generation means estimates the position of the marker in the adjacent image based on the position of the marker detected from the image to be inspected and the relative distance of the marker between inspection units determined based on manuscript data and print job information, and generates the aligned image based on the estimated position of the marker and the position of the marker detected from the image to be inspected.

11. 2. The image processing apparatus according to claim 1, further comprising a display control means for displaying an image of a region adjacent to the marker detected from the adjacent image.

12. an inspection unit that inspects the print image to be inspected for defects by comparing the alignment image with the reference image; 12. The image processing apparatus according to claim 11, wherein the display control means displays an image of a region adjacent to the marker detected from the adjacent image together with an image showing a result of the inspection by the inspection means.

13. 12. The image processing apparatus according to claim 11, wherein the display control means displays the position of the marker detected from the adjacent image in an emphasized manner.

14. 12. The image processing apparatus according to claim 11, wherein the display control means generates a composite image by combining an image of a region in the vicinity of the marker detected from the adjacent image with the registration image, and displays the composite image.

15. The image processing device according to claim 11, characterized in that when the print image to be inspected is an inspection unit related to the start or end of printing and there is no adjacent image, the display control means displays the estimated position of the marker in the adjacent image by superimposing it on an image of the area near the marker in the image to be inspected.

16. 15. The image processing apparatus according to claim 11, further comprising an operation unit for switching between displaying and not displaying an image of a region in the vicinity of the marker detected from the adjacent image.

17. 11. The image processing device according to claim 10, further comprising a warning means for warning that the estimation accuracy of the position of the marker in the adjacent image will decrease if the length of the printed image of the one inspection unit in the transport direction is equal to or greater than a predetermined threshold value.

18. Regarding the inspection unit related to the start or end of printing, a marker adding means for adding a marker to the unit of inspection in addition to the markers added one per unit of inspection, The image processing device according to claim 1, characterized in that, for an inspection unit related to the start or end of printing, the generation means generates the alignment image based on the position of the marker added by the marker adding means, the marker being added one per inspection unit.

19. An image processing method for performing image processing based on image data obtained by reading print images continuously printed on a sheet, comprising: A marker is attached to the printed image for each inspection unit, acquiring an inspection target image, which is image data obtained by reading the printed image of the inspection unit to be inspected, and an adjacent image, which is image data obtained by reading an area of ​​the printed image of the inspection unit to be inspected that is adjacent to the sheet conveyance direction and includes the marker; generating a registration image by registering the inspection target image with a reference image based on the positions of the markers on the inspection target image and the markers on the adjacent image; 13. An image processing method comprising:

20. A program for causing a computer to function as an image processing device that executes image processing based on image data obtained by reading print images that are continuously printed on a sheet, A marker is attached to the printed image for each inspection unit, an acquisition means for acquiring an inspection target image, which is image data obtained by reading the printed image of the inspection unit to be inspected, and an adjacent image, which is image data obtained by reading an area of ​​the printed image of the inspection unit to be inspected that is adjacent in a sheet conveyance direction and includes the marker; a generating means for generating a registration image by registering the inspection target image with a reference image based on the positions of the markers on the inspection target image and the markers on the adjacent image; A program to function as a

Images