Image processing apparatus, image processing method, and program

Abstract

To improve a situation in which an appropriate error cannot be returned when a sensor streak is included in an image to be inspected.SOLUTION: An image processing device according to the present invention includes: acquisition means for acquiring a read image obtained by reading a printed matter; and determination means for tracing a contour of the printed matter from each of a plurality of points on the read image to determine the presence or absence of streaks included in the read image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an image processing technology for inspecting printed matter.

Background Art

[0002] Conventionally, an inspection device for automatically inspecting the quality of printed matter is known. The printed matter is inspected by comparing a reference image for inspecting the printed matter with an inspection target image obtained by scanning the printed matter with a sensor.

[0003] As a pre - preparation for inspection, when aligning the reference image and the inspection target image, contour point groups representing the contour and vertices of the paper may be used as reference points. As a technique for extracting the contour point group of the paper, Patent Document 1 discloses a technique for detecting the vertex coordinates of the paper by performing contour tracking of the paper starting from sampling points in a scanned image.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, when the inspection target image read by the sensor contains sensor streaks caused by various noise factors such as sensor failures, adhesion of dust such as paper powder, and dirt, there is a problem that the direct cause of the error cannot be correctly returned. Specifically, when starting contour tracking of the paper area from a single starting point, the extraction of the paper contour fails, but an error unrelated to the sensor streak is output.

[0006] For example, if the sensor marks are black streaks, the black streaks will appear in a way that divides the paper area. As a result of contour tracking, the paper area will be extracted as smaller than it actually is, leading to a paper size error message.

[0007] On the other hand, when the sensor streaks were white streaks, the white streaks appeared as if the paper area had been breached, causing the tracking points to extend beyond the edges of the scanned image during contour tracking, resulting in an error message indicating an overflow error.

[0008] As a result of these challenges, even if a reading error was caused by a sensor, it was not possible to notify the user, inspection equipment, or printing equipment. This prevented appropriate measures such as sensor maintenance from being taken, potentially leading to frequent inspection failures. Alternatively, because it was not possible to determine that sensor streaks were the cause, it was not possible to continue inspections while excluding the area where the sensor streaks occurred. [Means for solving the problem]

[0009] The image processing apparatus according to the present invention is characterized by comprising an acquisition means for acquiring a read image obtained by reading a printed material, and a determination means for determining the presence or absence of streaks in the read image by tracking the contour of the printed material from each of a plurality of points on the read image. [Effects of the Invention]

[0010] According to the present invention, if the image to be inspected contains sensor streaks, an appropriate error can be returned. [Brief explanation of the drawing]

[0011] [Figure 1] A block diagram showing the configuration of a printing inspection device. [Figure 2] A flowchart illustrating the process of a print quality control device. [Figure 3] A flowchart illustrating the process of extracting point clouds from the paper contour. [Figure 4]Specific examples when the position coordinates of the first contour point group and the second contour point group match. [Figure 5] Specific examples when the position coordinates of the first contour point group and the second contour point group do not match. [Figure 6] Another specific example when the position coordinates of the first contour point group and the second contour point group do not match. [Figure 7] Another specific example when the position coordinates of the first contour point group and the second contour point group do not match. [Figure 8] A flowchart showing the processing when the sensor streak generation part is excluded from the inspection target area and the inspection process is continued. [Figure 9] A flowchart showing the processing of extracting the paper contour point group in Example 2. [Figure 10] An example of an inspection target image in Example 2. [Figure 11] A flowchart showing the processing of extracting the paper contour point group in a modified example of Example 2. [Figure 12] An example of an inspection target image in a modified example of Example 2. [Figure 13] A flowchart showing the processing of extracting the paper contour point group in Example 3. [Figure 14] An example of an inspection target image in Example 3. [Figure 15] A flowchart showing the detailed processing procedure of extracting the paper contour point group in Example 4.

Best Mode for Carrying Out the Invention

[0012] Hereinafter, this embodiment will be described with reference to the drawings. Note that the following embodiments do not limit the present invention, and not all combinations of the features described in each embodiment are essential for the solution means of the present invention. In addition, various forms within the scope not departing from the gist of the present invention are also included in the present invention, and parts of the following embodiments can be appropriately combined.

Examples

[0013] In Example 1, when performing contour tracking of the paper from the inspection target image, two starting points for contour extraction are set so as to sandwich from both outer sides in the direction in which sensor streaks may occur. Then, an example of comparing the position coordinates of each vertex calculated from each contour tracked from each starting point will be described. If the position coordinates of the vertices calculated from the contours tracked from each starting point do not match, it is considered that the sensor streaks have divided or damaged the paper area, and an error indicating the presence of sensor streaks is notified.

[0014] (Configuration of the printed product inspection device) FIG. 1 is a block diagram showing the configuration of the printed product inspection device.

[0015] The printed product inspection device 1 is an image processing device that inputs an inspection target image and outputs an inspection result and an error message. The printed product inspection device 1 includes an inspection target image acquisition unit 11, a paper contour point group extraction unit 12, a reference image creation unit 13, an image alignment unit 14, an image inspection unit 15, and an error notification unit 16. The functions of each unit will be described below.

[0016] The inspection target image acquisition unit 11 acquires an inspection target image, which is a read image obtained by reading a printed matter printed by a printing device with a sensor. At this time, for example, a 600 dpi in-line sensor is used to read the inspection target image in a state including the paper area and its peripheral area. The peripheral area corresponds to, for example, the surface portion of a conveyance belt on which the inspection target image is set, and a color of a member with high contrast to the paper is desirable. Note that the read resolution of the inspection target image may be different in the vertical and horizontal directions. Also, in a case where high-speed inspection such as real-time inspection at any time is required, the inspection target image acquisition unit 11 may be configured to perform the printed product inspection process immediately after the printing unit of the printing device.

[0017] The paper contour point cloud extraction unit 12 extracts the contour point cloud of the paper from the image to be inspected. Here, the contour point cloud represents the vertices of the four corners of the paper when the contour of the paper is correctly extracted. At this time, the paper contour point cloud extraction unit 12 has the following: an image preprocessing unit 121, a scan direction acquisition unit 122, a first contour tracking start point determination unit 123, a second contour tracking start point determination unit 124, a first contour point cloud calculation unit 125, a second contour point cloud calculation unit 126, a contour point cloud comparison unit 127, and a sensor streak determination unit 128.

[0018] The image preprocessing unit 121 performs preprocessing on the image to be inspected. Here, in order to perform contour tracing in the paper contour point cloud extraction unit 12, the image to be inspected is converted into a monochrome binary image. Contour tracing may also be performed on an image other than a monochrome binary image, for example, a monochrome multi-level image, in which case the image to be inspected should be converted into an image format that can be used for contour tracing. Note that if contour tracing is possible without preprocessing, preprocessing is not particularly necessary and may be skipped. Also, if the resolution of the reference image and the image to be inspected are different, a resolution conversion is performed to match the resolution.

[0019] The scan direction acquisition unit 122 acquires the scan direction when reading the image to be inspected with the sensor. The scan direction represents the direction in which the sensor scans the image to be inspected relative to the image to be inspected, and can be either the up-and-down direction (vertical direction) or the left-and-right direction (horizontal direction) with respect to the image to be inspected. In the following explanation and drawings, the scan direction will always be described as the up-and-down direction. The scan direction here can be either from top to bottom or bottom to top. Note that the scan direction is usually constant, so it is not necessary to acquire it every time an image to be inspected is acquired. If the scan direction is known in advance, the information can be stored and the scan direction acquisition unit can be skipped.

[0020] The first contour tracking start point determination unit 123 determines the first starting point for contour tracking in the image to be inspected. The first starting point is determined so as to sandwich one side of the scan direction in which sensor streaks may occur.

[0021] The second contour tracking start point determination unit 124 determines a second start point for contour tracking in the image under inspection. The second start point is determined so as to sandwich the first start point from the opposite side with respect to the scan direction in which sensor streaks may occur.

[0022] The first contour point cloud calculation unit 125 calculates a first contour point cloud obtained by performing contour tracing from a first starting point.

[0023] The second contour point cloud calculation unit 126 calculates a second contour point cloud obtained by performing contour tracing from a second starting point.

[0024] The contour point cloud comparison unit 127 compares the position coordinates of the first contour point cloud and the second contour point cloud.

[0025] The sensor streak detection unit 128 determines the presence or absence of sensor streaks based on the comparison results of the contour point cloud comparison unit 127.

[0026] The reference image creation unit 13 creates a reference image. The reference image is a reference image equivalent to a sample used as a standard during inspection. It is either RIP data created during printing, or data obtained by processing the RIP data using various image processing filters and color conversion processes so that the RIP data closely resembles the printed image. Alternatively, it is image data created by scanning and combining multiple printed materials. When inspecting the same image consecutively, it is not necessary to create a reference image each time. The creation process in the reference image creation unit 13 can be skipped, and a reference image reserved separately in memory or elsewhere can be referenced.

[0027] The image alignment unit 14 aligns the reference image with the image to be inspected based on the contour point cloud of the paper.

[0028] The image inspection unit 15 compares the aligned reference image with the image to be inspected to check whether or not the image to be inspected contains defects. This comparison is performed by calculating a difference image.

[0029] The error notification unit 16 provides error notification when sensor streaks occur.

[0030] The print inspection device 1 may be a standalone device or integrated as a unit within the printing device. The print inspection device 1, and the processing flow and algorithms described below, operate on a PC (personal computer), a dedicated computer, or embedded circuits within the printer. The means, methods, and algorithms of each part are implemented as software or hardware. Calculations in each part are performed by computing devices such as a CPU or GPU, and the data used is read and written using memory such as RAM or ROM, or a large-capacity storage HDD connected to the PC. Furthermore, the reading (scanning) of the printed material to be inspected is performed using reading devices such as line sensors, scanners, or cameras provided by the print inspection device 1 or the printing device. Data input and output are performed using input devices such as a keyboard or mouse, and display devices such as a monitor.

[0031] (Print inspection process flow) Figure 2 is a flowchart showing the processing procedure of the print inspection machine. The following explanation will follow this flowchart. Each step (process) is represented by the letter S before it.

[0032] After the print inspection process begins, in S1, the printout printed by the printing device is read by a sensor to acquire an image of the item to be inspected. The image of the item to be inspected is acquired to include the paper area and its surrounding area.

[0033] Next, in S2, the paper contour point cloud is extracted. The detailed processing procedure will be described later.

[0034] Next, in S3, the reference images created by the reference image creation unit 13 are acquired. Reference images are created for each image to be inspected before inspection and are used for image alignment with the images to be inspected and during image inspection. Alternatively, only one reference image may be prepared, and alignment with each image to be inspected may be performed in the next step. When acquiring the reference image, the paper contour point cloud in the reference image is also acquired and used during alignment.

[0035] Next, in S4, image alignment is performed between the image to be inspected and the reference image based on the paper contour point cloud of the image to be inspected obtained in S2 and the paper contour point cloud of the reference image obtained in S3. Here, the paper vertices are used as the paper contour point cloud. Various known alignment techniques such as affine transformation are used for image alignment.

[0036] Next, in S5, image inspection is performed. Image inspection checks for defects by calculating the difference between the aligned reference image and the image to be inspected.

[0037] Next, in S6, it is determined whether or not the printed material has defects. Pixels whose difference calculated in S5 is greater than or equal to a predetermined threshold are determined to have defects, and pixels whose difference is less than the predetermined threshold are determined to be free of defects. If it is determined to be free of defects (No), the process moves to S7, where the printed material is ejected into the tray for approved items, and then to S9. On the other hand, the results of the image inspection process in S5 are referred to, and if it is determined to have defects (Yes), the process moves to S8, where the printed material is ejected into the tray for rejected items, and then to S9.

[0038] Finally, in S9, it is determined whether the inspection of all printed materials has been completed. If the inspection of all printed materials has not been completed (No), the process returns to S2 and continues inspecting the next printed material. If the inspection of all printed materials has been completed (Yes), the print inspection process is terminated.

[0039] This concludes the explanation of this flowchart.

[0040] (Processing flow for extracting point clouds from paper contours) Figure 3 is a flowchart showing the detailed processing procedure for extracting the paper contour point cloud as shown in S2. The following explanation will follow this flowchart.

[0041] First, in S101, the image to be inspected is preprocessed. Here, the image to be inspected is converted to a monochrome binary image in order to perform contour tracking in the following steps. If the resolution differs from that of the reference image, a resolution conversion is performed to match the resolution.

[0042] Next, in S102, the first and second starting points are determined based on the scan direction acquired by the scan direction acquisition unit 122.

[0043] Next, in S103, contour tracing is performed from the first starting point. Contour tracing is performed using methods such as the octa-connection method or the quad-connection method.

[0044] Next, in S104, the first vertex group is calculated from the first contour point group. Here, the vertices of the contour are calculated as the contour point group.

[0045] Next, in S105, contour tracing is performed from the second starting point. As in S103, methods such as the octa-connection method or the quad-connection method are used.

[0046] Next, in S106, the second set of vertices is calculated from the second set of contour points. Similar to S104, the vertices of the contour are calculated as the contour point set.

[0047] Next, in S107, we compare the first vertex group with the second vertex group.

[0048] Next, in S108, based on the comparison results in S107, it is determined whether the coordinates (four corners) of the vertex group match. If the coordinates of the vertex group match, in S109, they are determined to be the paper vertices to be used in subsequent processing, and the paper contour point cloud extraction process is terminated. On the other hand, if the coordinates of the vertex group do not match, in S110, a sensor streak generation error is notified, and the print inspection process is forcibly terminated.

[0049] This concludes the explanation of this flowchart.

[0050] (Specific example of paper contour point cloud extraction in images under inspection) Here, a specific example of the paper contour point cloud extraction process in the image under inspection will be explained using Figures 4 to 7 below.

[0051] Figure 4 is an explanatory diagram illustrating a specific example where the positional coordinates of the first contour point cloud and the second contour point cloud coincide. This example assumes that the image being inspected does not contain sensor streaks.

[0052] Figure 4(a) shows the first starting point 102 and the second starting point 103 in the image 101 to be inspected. The first starting point 102 and the second starting point 103 are set at the left and right edges of the image to be inspected, respectively, as starting points for contour extraction, so as to sandwich the scan direction (up and down direction in the figure) where sensor streaks may occur from both outside (left and right directions). In other words, they are the endpoints of the read image, and it is even better if they are the endpoints in a direction perpendicular to the scan direction.

[0053] Figure 4(b) shows the state where the paper contour is searched in the inward direction (rightward direction) of the image to be inspected, perpendicular to the scanning direction, starting from the first starting point 102, and point 104 is reached.

[0054] Figure 4(c) shows the state where the paper contour is searched in the inward direction (leftward) of the image to be inspected, perpendicular to the scanning direction, starting from the second starting point 103, and point 105 is reached.

[0055] Figure 4(d) shows the state after contour tracking has started from point 104 and contour point clusters 106, 107, 108, and 109 have been extracted.

[0056] Figure 4(e) shows the state after contour tracing is started from point 105 and contour point clusters 110, 111, 112, and 113 have been extracted.

[0057] Comparing Figure 4(d) and Figure 4(e), it can be seen that the corresponding position coordinates of the first contour point cloud and the second contour point cloud are the same. These contour point clouds correspond to the vertices of the four corners of the paper. Thus, when the image to be inspected does not contain sensor streaks, the position coordinates of the first contour point cloud and the second contour point cloud are the same.

[0058] Figure 5 is an explanatory diagram illustrating a specific example where the positional coordinates of the first contour point cloud and the second contour point cloud do not coincide. This example shows the case where sensor black streaks 117 appear in the image being inspected.

[0059] Figure 5(a) shows the first starting point 115 and the second starting point 116 in the image 114 to be inspected. The first starting point 115 and the second starting point 116 are set at the left and right edges of the image to be inspected, respectively, as starting points for contour extraction, so as to sandwich the scan direction (up and down direction in the figure) where sensor streaks may occur from both the outside (left and right direction).

[0060] Figure 5(b) shows the state where the paper contour is searched in the inward direction (rightward direction) of the image to be inspected, perpendicular to the scanning direction, starting from the first starting point 115, and point 118 is reached.

[0061] Figure 5(c) shows the state where the paper contour is searched in the inward direction (leftward) of the image to be inspected, perpendicular to the scanning direction, starting from the second starting point 116, and point 119 is reached.

[0062] Figure 5(d) shows the state after contour tracing is started from point 118 and contour point clusters 120, 121, 122, and 123 have been extracted.

[0063] Figure 5(e) shows the state after contour tracing is started from point 105 and contour point clusters 124, 125, 126, and 127 have been extracted.

[0064] Comparing Figure 5(d) and Figure 5(e), it can be seen that the corresponding corner coordinates of the first contour point cloud and the second contour point cloud do not match. Thus, when the image under inspection contains sensor black streaks, the position coordinates of the first contour point cloud and the second contour point cloud do not match.

[0065] Figure 6 is an explanatory diagram showing a different specific example from Figure 5, where the positional coordinates of the first contour point cloud and the second contour point cloud do not coincide. This example shows the case where sensor white streaks 131 appear in the image being inspected.

[0066] Figure 6(a) shows the first starting point 129 and the second starting point 130 in the image 128 to be inspected. The first starting point 129 and the second starting point 130 are set at the left and right edges of the image to be inspected, respectively, as starting points for contour extraction, so as to sandwich the scan direction (up and down direction in the figure) where sensor streaks may occur from both the outside (left and right direction).

[0067] Figure 6(b) shows the state where the paper contour is searched in the inward direction (rightward direction) of the image to be inspected, perpendicular to the scanning direction, starting from the first starting point 129, and reaching point 132, which corresponds to the edge of the image to be inspected.

[0068] Figure 6(c) shows the state where, starting from the second starting point 130, the paper contour is searched in the inward direction (leftward) of the image to be inspected, perpendicular to the scanning direction, and point 133, which corresponds to the edge of the image to be inspected, is reached.

[0069] Figure 6(d) shows the state after starting contour tracking from point 132 and extracting contour point clusters 134, 135, and 136. In this example, contour tracking is terminated when point 136, which is the edge of the image being inspected, is reached.

[0070] Figure 6(e) shows the state after starting contour tracking from point 133 and extracting contour point clusters 137, 138, and 139. In this example as well, contour tracking is terminated when point 139, which is the edge of the image being inspected, is reached.

[0071] Comparing Figure 6(d) and Figure 6(e), it can be seen that the corresponding position coordinates of the first contour point cloud and the second contour point cloud do not match. Thus, when contour tracking is terminated when the edge of the image being inspected is reached, and the image being inspected contains sensor white streaks, the position coordinates of the first contour point cloud and the second contour point cloud will not match.

[0072] Figure 7 is an explanatory diagram showing a specific example different from Figures 5 and 6, where the position coordinates of the first contour point cloud and the second contour point cloud do not coincide. This example, like Figure 6, is when sensor white streaks 131 appear in the image under inspection. The difference from Figure 6 is that, even if the contour tracking reaches a point corresponding to the edge of the image under inspection, the contour tracking is not interrupted, but continues until the starting point of the contour tracking is reached.

[0073] Figure 7(a) shows the first starting point 129 and the second starting point 130 in the image 128 to be inspected. The first starting point 129 and the second starting point 130 are set at the left and right edges of the image to be inspected, respectively, as starting points for contour extraction, so as to sandwich the scan direction (up and down direction in the figure) where sensor streaks may occur from both the outside (left and right direction).

[0074] Figure 7(b) shows the state where the paper contour is searched in the inward direction (rightward direction) of the image to be inspected, perpendicular to the scanning direction, starting from the first starting point 129, and point 132 is reached.

[0075] Figure 7(c) shows the state where the paper contour is searched in the inward direction (leftward) of the image to be inspected, perpendicular to the scanning direction, starting from the second starting point 130, and point 133 is reached.

[0076] Figure 7(d) shows the state after contour tracking has started from point 132 and eight points from the contour point cluster 134 to 141 have been extracted. In this way, contour tracking continues even when the edge of the image being inspected is reached.

[0077] Figure 7(e) shows the state after starting contour tracking from point 133 and extracting eight points from the contour point cluster 142 to 149. In this example as well, contour tracking continues even when reaching the edge of the image being inspected.

[0078] Comparing Figure 7(d) and Figure 7(e), it can be seen that the corresponding position coordinates of the first contour point cloud and the second contour point cloud do not match. Thus, when contour tracking continues even after reaching the edge of the image being inspected, if the image being inspected contains sensor white streaks, the position coordinates of the first contour point cloud and the second contour point cloud will not match.

[0079] As explained above using several specific examples, the method of extracting contour point clouds in the inspected image differs depending on whether or not sensor streaks occur, and the presence or absence of sensor streaks can be determined by whether or not the positional coordinates of the first contour point cloud and the second contour point cloud match.

[0080] (modified version) Here, we will explain various variations.

[0081] The above explanation used the case with two starting points as an example, but it is also possible to set more than two starting points, perform contour tracking using each as a starting point, and compare the extracted contour point sets. If all corresponding points in the multiple contour point sets do not match, a sensor streak generation error can be notified.

[0082] Furthermore, while examples were shown using the vertices of the contour point groups extracted in each case as the first and second contour point groups, it is also possible to use not only the vertices of the contour point groups, but all or part of the contour point groups that constitute the contour. More specifically, it is also possible to compare whether the first and second contour point groups match or not. Comparing all contour point groups is equivalent to comparing the entire process during contour tracking, and corresponds to comparing whether the tracking processes of the first and second contour point groups follow the same contour and meet along the way.

[0083] Furthermore, during contour tracking in steps S103 and S105 in Figure 3, it is determined each time whether the coordinates of the contour tracking target have reached the edge of the image being inspected. If the contour point cloud can be detected to the end without reaching the edge of the image being inspected, the possibility that the sensor streak is a white streak decreases. Therefore, in step S110, it may be possible to identify it as a black sensor streak rather than a sensor streak and then notify an error.

[0084] Furthermore, starting points 1 and 2 should ideally begin the search from both outer edges of the area in the image being inspected where printed materials are likely to exist. If the paper size of the image being inspected is known in advance, the starting positions may be adjusted accordingly.

[0085] Alternatively, instead of notifying of a sensor streak error in S110 and forcibly terminating the inspection process, the area where the sensor streak occurred may be excluded from inspection, and the inspection process may continue in the other areas where no sensor streaks occurred. In this case, instead of forcibly terminating due to the sensor streak error notification, only a notification that a sensor streak has occurred is issued, and the inspection process continues to the end.

[0086] Figure 8 is a flowchart showing the processing procedure when the areas where sensor streaks occur are set as areas excluded from inspection, and the inspection process continues in other areas where no sensor streaks occur. In Figure 8, instead of forcibly terminating the print inspection process after S110, steps S111 and S112 are added to the processing flow in Figure 3. In S111, contour tracking continues after avoiding or repairing the areas where sensor streaks occur, and the paper vertices are determined. Next, in S112, the areas where sensor streaks occur are set as areas excluded from inspection.

[0087] As explained above, in Example 1, when tracing the contour of paper from the image to be inspected, multiple starting points for contour extraction are set so as to surround the direction in which sensor streaks may occur from both outer sides, and the position coordinates of each vertex calculated from each contour traced from each starting point are compared. If the position coordinates of the vertices calculated from the contours traced from each starting point do not match, it is assumed that the sensor streaks have divided or breached the paper area, and an error indicating the presence of sensor streaks is notified. According to Example 1, if the image to be inspected contains sensor streaks, the cause of the error when the sensor streaks occur can be correctly returned. This allows for appropriate measures such as sensor maintenance, or the continuation of inspection while avoiding the area where the sensor streaks occur. [Examples]

[0088] In Example 1, when sensor streaks were found in the image under inspection, the system could correctly return the cause of the error at the time of sensor streaking. However, it could not distinguish whether the sensor streaks were black or white. Therefore, in Example 2, in addition to the features of Example 1, after a mismatch in the contour point cloud occurs, the system determines whether the contour tracking target extends beyond the edge of the image under inspection. If there is no extension, it notifies the system of an error indicating the presence of black sensor streaks. If there is an extension, and the extension width is small, it notifies the system of an error indicating the presence of white sensor streaks. This allows the system to distinguish between black and white sensor streaks and correctly return the cause of the error at the time of sensor streaking. From here on, the explanation of content common to Example 1 will be omitted, and only the content characteristic of Example 2 will be explained.

[0089] (Processing flow for extracting point clouds from paper contours) Figure 9 is a flowchart detailing the paper contour point cloud extraction process in Example 2. The following explanation will follow this flowchart. Steps S101 to S109 are the same as in Example 1, so their explanation will be omitted. The steps added in Example 2 will be explained below.

[0090] S201 is performed if the coordinates of the vertex group do not match in S108. In S201, it is determined whether or not there is any overhang at the edge of the image to be inspected. Overhang at the edge of the image is determined by whether or not the tracking point of the contour tracing reaches the edge of the image.

[0091] In S201, if there is no overflow at the edges of the image, the process moves to S202, notifies a sensor black streak error, and forcibly terminates the print inspection process. At this time, the image to be inspected, for example as shown in Figure 10(a), has a sensor black streak in the center of the image to be inspected, and there is no overflow at the edges of the image as shown in 201 and 202.

[0092] On the other hand, if there is overhang at the edge of the image in S201, the process moves to S203. In S203, the overhang width is calculated. The overhang width is calculated by determining the number of consecutive white pixels at the edge of the image. For example, it may be calculated as 3 pixels.

[0093] Next, in S204, if the overhang width is smaller than a predetermined threshold, in S205, a sensor white streak error is notified, and the print inspection process is forcibly terminated. The predetermined threshold is, for example, 10 pixels. In this case, the image to be inspected, for example as shown in Figure 10(b), has a sensor white streak in the center of the image to be inspected, and overhangs have occurred at the edges of the image shown in 203 and 204 with a width smaller than the threshold.

[0094] On the other hand, if the overhang width is not smaller than the threshold, S206 notifies a paper overhang error and forcibly terminates the print inspection process. In this case, the image being inspected, for example as shown in Figure 10(c), has paper tilt, and the overhang is larger than the threshold width at the edges of the image shown in 205 and 206.

[0095] This concludes the explanation of this flowchart.

[0096] (modified version) The determination of whether or not there is an overhang at the edge of the image in S201 may be made at any time during the execution of contour tracing in S103 and S105. In that case, the processing flow may be such that when the pixel being contour-traced reaches the edge of the image being inspected, the processing in S103 and S105 is interrupted and the process moves to S203, which is the step of calculating the overhang width.

[0097] Furthermore, even if the overhang width is small, it is possible that one corner of the paper may slightly overhang due to a significant tilt of the paper, rather than a sensor streak. To distinguish whether or not this is the case, S207 may be added to the processing flow in Figure 9, as shown in Figure 11. In S207, if the overhang positions are approximately the same in the direction orthogonal to the scan direction, for example as in Figure 12(a), the process moves to S205 and a sensor white streak error is notified. Otherwise, for example as in Figure 12(b), the process moves to S206 and a paper overhang error is notified. The determination of whether or not the overhang positions are approximately the same is made by comparing a predetermined number of pixels with a threshold, for example, 20 pixels. In this way, the characteristic that sensor streaks occur in an approximately straight line with respect to the scan direction is used to distinguish between sensor streaks and paper tilt.

[0098] Furthermore, exceptionally, depending on the relative size of the paper and the area outside the paper (black frame), and the amount of tilt and misalignment of the paper, it may be possible for only the top or bottom of the paper to slightly protrude. In such cases, it is determined that a partial sensor streak has occurred at either the top or bottom edge. However, if the possibility of a partial sensor streak occurring is extremely low and the accuracy of paper transport is a more significant factor, the system may determine that the paper has protruded even if the protrusion width is short.

[0099] As explained above, in Example 2, when performing contour tracking of paper from the image to be inspected, after a mismatch in the contour point cloud occurs, it is determined whether the contour tracking target extends beyond the edge of the scanned image. If there is no extension, an error indicating a black sensor streak is notified, and if there is an extension and the extension width is small, an error indicating a white sensor streak is notified. This allows the system to distinguish between black and white sensor streaks and correctly return the cause of the error when the sensor streak occurs. [Examples]

[0100] Examples 1 and 2 basically assumed that sensor streaks occurred across the entire surface of the image to be inspected in the scanning direction, and in particular, that the sensor streaks occurred in two places at the edges of the image. Therefore, in the case of partial sensor streaks that occurred only on a part of the paper contour, the position coordinates of the corresponding points in the first contour point cloud and the second contour point cloud coincided, and thus the sensor streaks were not detected.

[0101] In Example 3, in addition to Example 1, the number of first contour point clouds (vertices) and the number of second contour point clouds (vertices) are calculated, and if the number of contour point clouds is other than 4, an error is notified indicating that there are partial sensor streaks in a part of the paper area. This makes it possible to correctly return the cause of the error when sensor streaks occur, even if the inspection target image contains partial sensor streaks that only affect a part of the paper contour. From here on, the explanation of content common to Examples 1 and 2 will be omitted, and only the content characteristic of Example 3 will be explained.

[0102] (Processing flow for extracting point clouds from paper contours) Figure 13 is a flowchart detailing the paper contour point cloud extraction process in Example 3. The following explanation will follow this flowchart. Steps S101 to S110 are the same as in Example 1, so their explanation will be omitted. The steps added in Example 3 will be explained below.

[0103] From S301 onwards, processing is performed if the position coordinates of the contour point cloud match in S108.

[0104] First, in step 301, the number of points in the first contour point cloud is calculated. Here, these correspond to the vertices of the extracted contour.

[0105] Next, in step 302, the number of points in the second contour point cloud is calculated. Here again, these correspond to the vertices of the extracted contour.

[0106] Next, in step 303, it is determined whether the number of contour point clouds is 4 in both cases. If the number of contour point clouds is 4 in both cases, the process moves to S109, where the paper vertices are determined and the paper contour point cloud extraction process is terminated.

[0107] If the number of contour point clouds is not 4, the process moves to S304, a partial sensor streak error is reported, and the print inspection process is forcibly terminated. At this time, the image being inspected has, for example, sensor black streaks 216 or sensor white streaks 217 on a part of the paper contour, as shown in Figures 14(a) and (b). In this case, as indicated by multiple white circles in Figures 14(c) and 14(d), which correspond to Figures 14(a) and (b), there are 8 first contour point clouds and 8 second contour point clouds.

[0108] Thus, when sensor black streaks or sensor white streaks appear on a portion of the paper contour, the positional coordinates of the first contour point cloud and the second contour point cloud coincide because the paper area is a closed plane. However, the number of contour point clouds will be 8 each, instead of the 4 that should be extracted during paper contour extraction. This characteristic is used to distinguish between partial sensor black streaks and partial sensor white streaks.

[0109] This concludes the explanation of this flowchart.

[0110] As explained above, Example 3 describes an example in which, when performing contour tracking of paper from an image to be inspected, the number of first contour point clouds and the number of second contour point clouds are calculated, and if the number of contour points is other than 4, an error is notified indicating that there are partial sensor streaks in a part of the paper area. This makes it possible to correctly return the cause of the error when the sensor streaks occur, even if the image to be inspected contains partial sensor streaks that only affect a part of the paper contour. [Examples]

[0111] In Example 3, while partial sensor streaks could be identified, it was not possible to distinguish whether they were white or black streaks.

[0112] In Example 4, in addition to Example 1, the shape of the first contour point cloud and the shape of the second contour point cloud are determined, and if the shape includes a concave part, it is determined to be a sensor black streak, and if the shape includes a convex part, it is determined to be a sensor white streak, and an error indicating the presence of the respective streak is notified. As a result, even if the image to be inspected contains partial sensor streaks that only cover a part of the paper contour, it is possible to distinguish whether the sensor streak is a black streak or a white streak and correctly return the cause of the error when the sensor streak occurred. From here on, the explanation of content common to Examples 1 to 3 will be omitted, and only the content characteristic of Example 4 will be explained.

[0113] (Processing flow for extracting point clouds from paper contours) Figure 15 is a flowchart detailing the processing procedure for extracting paper contour points in Example 4. The following explanation will follow this flowchart. Steps S101 to S110 are the same as in Example 1, so their explanation will be omitted. The steps added in Example 4 will be explained below.

[0114] From S401 onwards, the process is performed if the position coordinates of the contour point cloud match in S108.

[0115] First, in step 401, the shape of the first contour point cloud is calculated. The shape is determined by analyzing geometric information such as the number of vertices in the closed space region formed by the contour point cloud, the angle of each vertex, the length of each edge, and its direction.

[0116] Next, at 402, the shape of the second contour point cloud is calculated. This calculation is performed in the same manner as the calculation of the shape of the first contour point cloud.

[0117] Next, in step 403, it is determined whether both shapes are quadrilaterals. If the number of contour point clouds is quadrilaterals, the process moves to S109, where the paper vertices are determined and the paper contour point cloud extraction process is terminated. On the other hand, if the number of contour point clouds is not quadrilaterals, the process moves to S404.

[0118] In S404, it is determined whether or not the shape includes a recess. The shape is analyzed, and if it includes a recess, the process moves to S406. On the other hand, if the shape is analyzed and does not include a recess, the process moves to S405, and a partial black sensor streak error is notified.

[0119] In S406, it is determined whether or not the shape includes protrusions. If the shape includes protrusions, the process moves to S407 and notifies of a partial white sensor streak error. If the shape does not include protrusions, the process moves to S408 and notifies of a paper shape error.

[0120] In each of steps S405, S407, and S408, the print inspection process is forcibly terminated as soon as the processing is complete.

[0121] In this case, the image being inspected, for example, using Figure 14 as an example, shows that sensor black streaks 216 or sensor white streaks 217 appear on a part of the paper contour, as shown in Figures 14(a) and (b). In the case of Figure 14(a), since the sensor black streaks 216 are present, the contour point cloud has a shape that includes concave parts. In the case of Figure 14(b), since the sensor white streaks 217 are present, the contour point cloud has a shape that includes convex parts.

[0122] Thus, when sensor black streaks or sensor white streaks appear on a portion of the paper contour, the paper area becomes a closed plane. Therefore, although the positional coordinates of the first contour point cloud and the second contour point cloud coincide, the shape of the contour point cloud, which should ideally be a rectangle during paper contour extraction, becomes a shape that includes concave or convex parts, deviating from a rectangle. This characteristic is used to distinguish between partial sensor black streaks and partial sensor white streaks.

[0123] This concludes the explanation of this flowchart.

[0124] As explained above, according to Example 4, when performing contour tracking of paper from an image to be inspected, the shape of the first contour point cloud and the shape of the second contour point cloud are determined, and if the shape includes a concave part, it is determined to be a sensor black streak, and if the shape includes a convex part, it is determined to be a sensor white streak. Then, an error indicating the presence of each type of streak is notified. As a result, even if the image to be inspected contains partial sensor streaks that only cover a part of the paper contour, it is possible to distinguish whether the sensor streak is a black streak or a white streak and correctly return the cause of the error when the sensor streak occurred.

[0125] [Other examples] In the above embodiments, the case where one sensor streak occurs was described as an example, but the present invention can also be applied when multiple sensor streaks occur. The positional relationship between the first group of vertices and the second group of vertices is compared, and if the separation distance is large, it is determined that there is an unextracted paper area between them, and an error is notified indicating that there may be two or more sensor streaks. Alternatively, the starting point of contour extraction can be traced from outside the edge perpendicular to the scan direction of the unextracted area, and the presence or absence of sensor streaks can be analyzed in more detail by tracing the paper contour of the unextracted area. The contour tracing process can be repeated recursively until there are no more unextracted areas.

[0126] Furthermore, information on the pixel values ​​within the contour point cloud can also be used to determine whether the contour point cloud is within the paper area (pixel value of 1 or 255, etc. (white)) or outside the paper area (pixel value of 0 (black)), and the presence or absence of sensor streaks may be analyzed in more detail accordingly.

[0127] Furthermore, the above embodiments were described on the premise that the image to be inspected includes the white area of ​​the paper around the edges. On the other hand, in the case of an image to be inspected that is printed without borders across the entire surface of the paper, the position of the paper may be obtained separately from the image or set position, and a white pixel area may be added to the edges of the paper so that contour tracking can be performed.

[0128] Furthermore, depending on the relative size of the paper and the area outside the paper (black frame), and the amount of tilt and misalignment of the paper, it is possible that only the top or bottom of the paper may slightly protrude. In such cases, it is determined that a partial sensor streak has occurred at either the top or bottom edge. However, if the possibility of a partial sensor streak occurring is extremely low and the accuracy of paper transport is a more significant factor, the system may determine that the paper has protruded even if the protrusion is short.

[0129] Furthermore, the present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. [Explanation of symbols]

[0130] 1. Printing inspection device 11. Image acquisition unit for the subject of inspection 12. Paper contour point cloud (vertex) extraction unit 13. Reference Image Creation Section 14 Image alignment section 15. Imaging Inspection Department 16 Error Notification Section

Claims

1. an acquisition means for acquiring a scanned image obtained by scanning a printed material; An image processing apparatus comprising: a determining unit that determines whether or not a streak is present in the read image by tracing the outline of the print from each of a plurality of points on the read image.

2. 2. The image processing apparatus according to claim 1, wherein the scanned image includes a paper area of ​​the printed matter and its surrounding area.

3. 3. The image processing apparatus according to claim 1, wherein the plurality of points are both end points of the read image.

4. 4. The image processing apparatus according to claim 1, wherein the plurality of points are both end points in a direction perpendicular to a direction in which the scanned image is read.

5. the determining means compares the position coordinates of the plurality of contour points of the printed matter obtained as a result of the tracking to determine whether they match among the plurality of read images; The determining means determines that the read image contains streaks when the position coordinates do not match.

5. The image processing device according to claim 1, wherein the image processing device further comprises: a processor for processing the image;

6. a width calculation means for calculating a width of the portion of the read image that extends beyond the printed matter; Furthermore, 6. The image processing apparatus according to claim 5, wherein the determining means determines whether or not the streak is present based on the result of the comparison and the size of the width.

7. A calculation means for calculating the number of the plurality of points; a number comparison means for comparing the number of the plurality of points in the plurality of read images; Furthermore, 7. The image processing apparatus according to claim 1, wherein the determining means determines that the read image contains partial streaks based on the result of the comparison by the number comparing means.

8. a shape calculation means for calculating a shape of the printed matter indicated by the plurality of points; a shape comparison means for comparing the shapes of the plurality of read images; Furthermore, 7. The image processing apparatus according to claim 1, wherein the determining means determines that the read image contains partial streaks based on the result of the comparison by the shape comparing means.

9. 9. The image processing device according to claim 1, wherein the plurality of points include vertices of the printed matter.

10. The device further includes a notification unit that notifies an error based on the result of the determination by the determination unit.

10. The image processing device according to claim 1, wherein the image processing device is a computer.

11. The determining means further includes means for continuing to determine other portions of the scanned image where streaks occur based on the result of the notification means.

11. The image processing device according to claim 10.

12. A program for causing a computer to function as each of the means of the image processing device according to any one of claims 1 to 11.

13. an acquisition step of acquiring a scanned image obtained by scanning a printed material; a determining step of determining whether or not a streak is present in the read image by tracing the outline of the print from each of a plurality of points on the read image; An image processing method comprising:

Images