Image processing device, control method thereof, and program thereof

Abstract

The present invention provides, for example, a novel mechanism. [Solution] The image processing device comprises an acquisition unit that acquires an image of one side of a sheet, an image processing unit that detects features of the sheet image, a determination unit that determines whether or not there is an abnormality in the sheet image based on the features, a display setting unit that sets the content to be displayed on the screen if there is an abnormality in the sheet image, and an output unit that outputs the content. The display setting unit sets a reference point in the sheet image, sets whether or not to display relative position information that indicates the relative position of the abnormality with respect to the reference point, and sets the relative position information to be displayed on the screen if the display of relative position information is set to be enabled.

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, a control method thereof, and a program thereof.

Background Art

[0002] Conventionally, in the printing industry, in order to ensure that there are no abnormalities and no problems with the quality of the printed products delivered to the orderer, inspection (quality inspection) work is performed after printing. There is a technology that automatically performs this inspection work as post-processing of a printing machine. In an example of such an inspection technology, first, reference image data representing a printed product without abnormalities is acquired and registered. Next, the input image data is printed on a recording medium (printing paper) by an image forming apparatus and read as image data by a sensor (reading device). The presence or absence of abnormalities in the printed matter is inspected by comparing the read data read by the sensor with the registered reference image data. The inspection result is displayed to the user through a screen or output as an inspection report by summarizing the inspection results.

[0003] By the way, the user refers to the result of this inspection and actually recognizes the abnormalities on the printed product. As reasons for recognizing abnormalities, for example, even if a sample is determined to have a very slight abnormality, it may actually be a quality level that can be tolerated by a human when viewed, or it may only be determined to be abnormal because an attachment (dust) temporarily enters during reading. If such a sample can be recognized, by treating it as a qualified product, waste such as printing paper, coloring materials, and time required for reprinting can be suppressed. On the other hand, when it is recognized that the abnormality is severe, calibration can be performed on the image forming apparatus or a maintenance service can be requested. Furthermore, when an abnormality is recognized, it is possible to reflect it in subsequent inspection settings so that a more appropriate inspection can be performed. Therefore, it is important to be able to smoothly confirm the inspection result (for example, the position of the abnormality) through the display of the inspection result and the inspection report. Patent Document 1 describes an example of displaying the position of an abnormality in such a case.

Prior Art Documents

Patent Documents

[0004] [Patent Document 1] Japanese Patent Publication No. 2021-037736 [Overview of the project] [Problems that the invention aims to solve]

[0005] In displaying the location of abnormalities in inspection results, one approach is to illustrate the printed document, highlighting the location of the abnormality, or showing its coordinates on the page to present the information to the user. However, this method of display results in a uniform display of the abnormality location regardless of the printed content. Therefore, depending on the content of the printed document, the user may not be able to easily recognize the abnormality location, potentially reducing user usability.

[0006] The present invention has been made in view of at least one of the above-mentioned problems, and provides a novel mechanism that makes it possible to easily recognize the location of printing defects according to the content of the printed material. [Means for solving the problem]

[0007] According to one aspect of the present invention, An acquisition unit that acquires an image of one side of the sheet, The image processing unit processes the sheet image acquired by the acquisition unit and detects the features of the sheet image, A determination unit determines whether or not there is an abnormality in the sheet image acquired by the acquisition unit, based on the features detected by the image processing unit. If the determination unit determines that there is an abnormality in the sheet image, the display setting unit sets the content to be displayed on the screen, The system includes an output unit that outputs the content set by the display setting unit, The display setting unit sets a reference point in the sheet image and sets whether or not to display relative position information indicating the relative position of the abnormality with respect to the reference point. The provided image processing apparatus is characterized in that, when the display setting unit is set to enable the display of the relative position information, it sets the relative position information to be displayed on the screen. [Effects of the Invention]

[0008] According to the present invention, the location of printing defects can be easily recognized regardless of the type of printed material. [Brief explanation of the drawing]

[0009] [Figure 1] Schematic diagram of a print image inspection system according to one embodiment. [Figure 2] A diagram showing the hardware configuration of an image processing device according to one embodiment. [Figure 3] A diagram showing the hardware configuration of each device according to one embodiment. [Figure 4] A diagram showing the functional blocks of each device according to one embodiment. [Figure 5] A flowchart showing the processing procedure according to one embodiment. [Figure 6] Figure showing a display screen according to one embodiment. [Figure 7] A flowchart showing the processing procedure according to one embodiment. [Figure 8] A diagram illustrating the Hough transform according to one embodiment. [Figure 9] Figure showing a display screen according to one embodiment. [Figure 10] Figure showing a display screen according to one embodiment. [Figure 11] Figure showing a display screen according to one embodiment. [Figure 12] Figure showing a display screen according to one embodiment. [Figure 13] Figure showing a display screen according to one embodiment. [Figure 14] Figure showing a display screen according to one embodiment. [Figure 15] Figure showing a display screen according to one embodiment. [Figure 16] Figure showing a display screen according to one embodiment. [Figure 17]Figure showing a display screen according to an embodiment [Figure 18] Figure showing a display screen according to an embodiment [Figure 19] Figure showing functional blocks of each device according to an embodiment

Mode for Carrying Out the Invention

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential for the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0011] In this specification, the term "image forming apparatus" broadly includes apparatuses that form (record) an image on a recording material (recording medium), such as a single-function printer, a copier, a multifunction machine, a commercial printer, and the like.

[0012] (System Configuration) Using FIG. 1, the overall configuration of the print image inspection system 100 according to this embodiment will be described. The print image inspection system 100 includes an image processing apparatus 101, an image forming apparatus 102, an inspection processing apparatus 103, an output control apparatus 104, and a communication cable 105 that connects each apparatus.

[0013] The image processing apparatus 101 has a role as a printer server. That is, the image processing apparatus 101 performs RIP (Raster Image Processor) processing on the input print data and document data to convert it into a bitmap, and performs a process of generating image data for image formation (printing) (hereinafter also referred to as print data). Further, the image processing apparatus 101 also functions as a server that manages control related to printing and print jobs performed by the image forming apparatus 102. Hereinafter, the image processing apparatus 101 is also referred to as a DFE (Digital Front End).

[0014] The image forming apparatus 102 is a device that forms, or prints, an image on a recording medium (printing paper) based on the print data generated by the image processing apparatus 101, and is a so-called printer. Possible image forming methods include offset printing, electrophotography, or inkjet printing, but the present invention can be applied to any method as long as an image can be formed on the recording medium. In this embodiment, the image forming apparatus 102 is assumed to be an electrophotographic image forming apparatus.

[0015] The inspection processing device 103 determines, for each printed item, that there are no abnormalities in the printed material produced by the image forming apparatus 102 and that there are no problems with its quality. Based on the determination result of the inspection processing device 103, the output control device 104 switches the paper output destination and performs post-processing (such as binding) as necessary. The output control device 104 is also called a finisher. These devices work together to realize the function of acquiring printed output materials that have been confirmed to be free of abnormalities through inspection.

[0016] (Hardware configuration of the image processing device 101) The hardware configuration of the image processing device 101 will be explained using Figure 2. The image processing device 101 includes a CPU 201, RAM 202, ROM 203, storage device 204, system interface (I / F) 205, network I / F 206, output I / F 207, general-purpose I / F 208, and main bus 209. In addition, the image processing device 101 is connected to an output device 210 via the output I / F 207, and to an input device 211 and an external storage device 212 via the general-purpose I / F 208.

[0017] The CPU (Central Processing Unit) 201 is a processor that comprehensively controls all parts of the image processing device 101. While this explanation describes the case where the CPU 201 controls the entire image processing device 101, the entire image processing device 101 may also be controlled by multiple hardware components sharing the processing. Furthermore, some of the control processing of the CPU 201 may be performed by hardware such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array). The RAM (Random Access Memory) 202 functions as the main memory and work area of ​​the CPU 201. The ROM (Read Only Memory) 203 stores the programs executed by the CPU 201. The storage device 204 here refers to a large-capacity auxiliary storage device that stores, for example, applications executed by the CPU 201 and data used for image processing.

[0018] The system interface 205 includes connection modules such as connectors and is connected to each device within the print image inspection system 100, namely the image forming apparatus 102, the inspection processing apparatus 103, and the output control device 104. The image processing apparatus 101 also communicates its operating status with each other via the system interface 205, synchronizes timing, and transmits and receives data with each device. The network interface 206 includes a communication module (e.g., a Network Interface Card) and connects to a network outside the print image inspection system 100 to transmit and receive data. The output interface 207 includes connection modules such as connectors and is an image output interface such as HDMI (High-Definition Multimedia Interface, registered trademark). The image processing apparatus 101 and the output device 210, such as a liquid crystal display, are connected via this output interface 207. The output device 210 functions as a user interface that displays the status of the image processing apparatus 101 to the user.

[0019] The general-purpose I / F 208 is a bus interface that includes connectors conforming to standards such as USB or IEEE 1394. The image processing device 101 is connected to an input device 211, such as a keyboard or mouse, via this general-purpose I / F 208, and receives information on user operations (instructions). The image processing device 101 is also connected to an external storage device 212 via the general-purpose I / F 208, and can save data such as logs to the external storage device 212 and retrieve data from the external storage device 212. The main bus 209 connects each piece of hardware of the image processing device 101 in a communicative manner.

[0020] The hardware configuration of the image processing device 101 is not limited to the configuration described above. For example, the external devices described above, such as the output device 210, may be located inside the image processing device 101 via the main bus 209. Also, the output device 210 and the input device 211 may be integrated as a touch panel display or the like. The image processing device 101 may have a GPU (Graphics Processing Unit), which is a processor specialized for high-speed parallel computing, and may be configured to perform part of the control processing of the CPU 201 on the GPU.

[0021] (Hardware configuration of other devices) The hardware configurations of the image forming apparatus 102, inspection processing apparatus 103, and output control device 104 will be explained using Figure 3. However, the hardware configurations in each apparatus that are similar to those of the image processing apparatus 101 shown in Figure 2 will not be explained.

[0022] The image forming apparatus 102 is connected to other devices within the print image inspection system 100 via a system interface 301. The image forming apparatus 102 comprises a paper feeding unit 302, a paper transport unit 303, an image forming unit 304, and a touch panel display (referred to as the image forming UI panel in this paper) 305 which serves as a user interface (UI). The paper feeding unit 302 supplies printing paper into the image forming apparatus 102 through the paper transport unit 303. The paper transport unit 303 transports the printing paper supplied from the paper feeding unit 302 using rotating rollers (not shown) or the like. The paper transport unit 303 is provided so as to traverse the image forming apparatus 102, the inspection processing unit 103, and the output control device 104, and can transport printing paper to these devices.

[0023] The image forming unit 304 forms an image on the printing paper transported along the paper transport unit 303 based on the printing data sent from the image processing device 101. For example, in the case of an electrophotographic image forming apparatus, an image is formed by transferring and fixing colorants (toners) held in an image shape on a photosensitive drum by voltage control onto the printing paper. The image forming UI panel 305 is a touch panel type display that serves as the UI of the image forming apparatus 102 and corresponds to the output device 210 and input device 211 of the image processing device 101. The image forming apparatus 102 is also configured to include a CPU 313, RAM 314, ROM 315, storage device 316, and main bus 317, similar to the image processing device 101 described above.

[0024] The inspection processing device 103 is connected to other devices within the print image inspection system 100 via a system I / F 306. The inspection processing device 103 includes a reading device 307 and a touch panel display (also called an inspection UI panel) 308 which serves as a user interface (UI). The reading device 307 is a scanner that can acquire image data (hereinafter referred to as read data) from printed materials transported through the paper transport unit 303 using an internal light source and a sensor (light receiving unit) such as a CCD (Charge Coupled Device). In this embodiment, it is assumed that the printed materials can be acquired in an image format in which each pixel is, for example, RGB (Red, Green, Blue) 3 channels and 8 bits. The inspection UI panel 308 is a touch panel display which serves as the UI of the inspection processing device 103 and corresponds to the image forming UI panel 305 of the image forming apparatus 102. Furthermore, the inspection processing device 103 is configured to include a CPU 318, RAM 319, ROM 320, storage device 321, and main bus 322, similar to the image processing device 101 described above.

[0025] The output control device 104 is connected to other devices in the print image inspection system 100 via a system I / F 3088. The output control device 104 consists of a control drive unit 309, a transport branch section 310 which is a branch of the paper transport unit 303, a printed material discharge unit 311, and a printed material storage unit 312. The control drive unit 309 switches the transport path of the printed material transported by the paper transport unit 303 to the transport branch section 310 according to the inspection result of the inspection processing device 103. The printed material storage unit 312 is provided at the end of the transport branch section 310. Meanwhile, the printed material discharge unit 311 discharges printed materials that have failed inspection. The printed material storage unit 312 stores printed materials that have passed inspection. Through this operation of the control drive unit 309, the products that have passed inspection and those that have failed inspection are stored separately, so that the printed products that have been confirmed to be free of defects are loaded into a single tray. Furthermore, the output control device 104, like the image processing device 101 described above, is configured to include a CPU 323, RAM 324, ROM 325, and a main bus 326.

[0026] (Functional configuration of the print image inspection system) Figure 4 illustrates the functional blocks of the print image inspection system 100. Figure 4 shows examples of functional blocks for each of the devices described above. The CPU in each device within the print image inspection system 100 executes processing by operating the parts described below in each device.

[0027] The image processing device 101 has a functional configuration comprising a print job acquisition unit 4102, a print data generation unit 4103, and a print data transmission unit 4104. The terminals (4101, 4105) are input / output data terminals for the functions of the image processing device 101. The print job acquisition unit 4102 acquires print jobs, including submitted image data, that are to be printed and inspected. Print jobs are transmitted, for example, from a personal computer (not shown) that can communicate with the image processing device 101 via a network, along with a print instruction.

[0028] Input terminal 4101 corresponds to, for example, network I / F 206. Alternatively, a print job with predetermined print settings applied to image data stored in external storage device 212 may be input. In such a case, input terminal 4101 corresponds to general-purpose I / F 208. Alternatively, the image processing device 101 may generate a print job according to the print settings instructed by the user via input device 211 for the input image data supplied as described above. In such a case, input terminal 4101 corresponds to RAM 202 which provides instruction signals and storage device 204 which supplies image data.

[0029] The print data generation unit 4103 receives a print job, refers to the submitted image information and print setting information included in the print job, performs RIP processing, and generates print data to be passed to the image forming apparatus 102. The print data transmission unit 4104 transmits the print data to the image forming apparatus 102 via the output terminal 4105. In this case, the output terminal 4105 corresponds to, for example, the system I / F 205.

[0030] The image forming apparatus 102 includes a print data acquisition unit 4202, a print data processing unit 4203, an image forming drive unit 4204, and a print data transmission unit 4205. The terminals (4201, 4206) are input / output data terminals for the functions of the image forming apparatus 102. The print data acquisition unit 4202 acquires print data transmitted by the print data transmission unit 4104 via the input terminal 4201. In this case, the input terminal 4201 corresponds to, for example, a system I / F 301.

[0031] The print data processing unit 4203 performs color conversion processing, halftone (gradation quantization) processing, and correction processing on the print data according to the image forming characteristics of the image forming apparatus 102. Hereinafter, the print data processed by the print data processing unit 4203 according to the image forming characteristics will be referred to as image forming data. The image forming drive unit 4204 drives the image forming unit 304 to print an image on the printing paper based on the image forming data. The print data transmission unit 4205 transmits the print data or image forming data and a signal indicating that printing has been performed to the inspection processing apparatus 103 via the output terminal 4206. In this case, the output terminal 4206 corresponds to the system I / F 301.

[0032] The inspection processing device 103 includes a read data acquisition unit 4302, a read data analysis unit 4303 (an example of an "image processing unit"), a print data acquisition unit 4304, and a print data conversion unit 4305. The inspection processing device 103 also includes an inspection processing unit 4306 (an example of a "judgment unit"), an inspection result transmission unit 4307, an inspection result storage unit 4308, an inspection result display unit 4309 (an example of an "output unit"), and a report creation unit 4310 (an example of an "output unit"). The terminals (4301, 4311) are input / output data terminals for the functions of the inspection processing device 103. The read data acquisition unit 4302 receives data and signals transmitted by the print data transmission unit 4205 from the input terminal 4301, drives the reading device 307, and acquires read data of the printed material. In this case, the input terminal 4301 corresponds to the system I / F 306.

[0033] The read data analysis unit 4303 analyzes the characteristics of the read data. The print data acquisition unit 4304 acquires the print data transmitted by the print data transmission unit 4205. The print data conversion unit 4305 performs resolution conversion, color conversion, etc., on the print data acquired by the print data acquisition unit 4304 to generate data for generating reference image data for the inspection by the inspection processing unit 4306. The inspection processing unit 4306 generates reference image data and performs an inspection process to determine whether or not there is an abnormality in the printed material by comparing it with the read data to be inspected. The inspection result transmission unit 4307 receives the inspection results performed by the inspection processing unit 4306 and transmits the inspection results to the output control device 104 via the output terminal 4311. In this case, the output terminal 4311 corresponds to the system I / F 306. The inspection result storage unit 4308 stores and accumulates the inspection results. The inspection result display unit 4309 shows the inspection results stored in the inspection result storage unit 4308 to the user. The report creation unit 4310 compiles the test results and creates a test report.

[0034] The output control device 104 includes an inspection result acquisition unit 4402 and an output control unit 4403. The terminals (4401, 4404) correspond to input / output data terminals for the functions of the output control device 104. The inspection result acquisition unit 4402 acquires the inspection result transmitted by the inspection result transmission unit 4307 via the input terminal 4401. The output control unit 4403 drives the control drive unit 309 based on the inspection result and guides the printed material to either the printed material ejection unit 311 or the printed material storage unit 312, for example. Furthermore, the output control unit 4403 outputs a signal from the output terminal 4404 indicating that processing has been completed. In this case, the output terminal 4404 corresponds to the system I / F 3088.

[0035] (Processing Example 1) Figure 5 illustrates the flowchart of the sequence processing performed by the print image inspection system 100. In this embodiment, the abnormalities detected in the print image inspection are indicated on the image at their relative positions from a reference point, according to the characteristics of the image read as the inspection target, thereby improving user operability. In this process, for example, it is assumed that some printing has been done on the printing paper in advance. Hereafter, this printing paper with some printing done in advance will also be referred to as preprint paper. The method by which the image forming apparatus 102 prints on the preprint paper will be referred to as the preprint method. Printing on preprint paper may also be referred to as overprinting. However, the preprint method does not necessarily have to be used.

[0036] In pre-printing using the pre-print method, image elements common to many types of printing paper are printed. For example, printing a company logo in a fixed position on all types of printing paper is one such example. The reason for pre-printing some images even though the image forming apparatus 102 will print the images is, for example, cost-effectiveness. More specifically, the image forming apparatus 102 uses, for example, electrophotography, but when printing a large quantity of the same image (high-volume printing), offset printing is usually more cost-effective than electrophotography. Therefore, image elements common to many types of printing paper are pre-printed using offset printing to create pre-printed paper. Then, the image forming apparatus 102 prints a smaller quantity of the same image than the pre-printed paper (low-volume printing). For relatively small quantities of printing, electrophotography is usually more cost-effective than offset printing. In this way, pre-printing is performed to improve cost-effectiveness by utilizing the advantages of different printing methods.

[0037] The sequence processing shown in Figure 5 involves the print image inspection system 100 performing printing and print image inspection processing using a preprint method. This sequence processing is achieved by the CPU in each device within the print image inspection system 100 reading programs stored in the storage medium of each device and controlling the aforementioned parts.

[0038] S501 to S504 are processes performed in the inspection processing device 103. First, in S501, the print image inspection system 100 operates in preprint paper reading mode, and the reading data acquisition unit 4302 acquires reading data from the preprint paper. Preprint paper reading mode is one of the operating modes of the print image inspection system 100 that operates based on user instructions. In this mode, the print image inspection system 100 does not print on the preprint paper set in the paper feed unit 302 by the user, but reads the content pre-printed on the preprint paper (an example of "an image of one side of a sheet") using the reading data acquisition unit 4302. That is, when the user gives an instruction to start operation, the preprint paper in the paper feed unit 302 is transported by the paper transport unit 303 and passes through the image forming unit 304 without image formation. Then it goes straight to the reading device 307, where the reading data of the preprint paper (an example of "sheet image") is acquired. The preprinted paper passes through the reader 307 and is then discharged to the print output unit 311. In this way, the content preprinted on the preprinted paper is acquired as preprinted paper reading data. It is desirable that the preprinted paper used in this case is of good quality and free from defects.

[0039] In this embodiment, it is assumed that the pre-printed paper has pre-printing as shown in Figure 6. That is, a framework composed of horizontal and vertical line segments is printed as a form (group of cells) that serves as a format for entries. Reference numeral 601 represents the entire pre-printed paper, and the frame 602 represents the pre-printed form. This type of pre-printing is carried out by printing a large quantity of forms common to forms and documents in advance, and then printing small quantities of those forms while changing the specific content to be written within them.

[0040] In S502, the read data analysis unit 4303 analyzes the preprinted paper read data and estimates and acquires the image features. Figure 7 illustrates a detailed flowchart of the preprinted paper read data analysis and feature acquisition process performed in S502. The process in Figure 7 is also a process in the inspection processing unit 103, and is realized by the CPU 318 of the inspection processing unit 103 reading a program stored in a storage medium such as RAM 319 and controlling the various parts described below.

[0041] In S701, the read data analysis unit 4303 binarizes the preprint paper read data depending on whether it is the image drawing area or the paper area, and obtains binarized data. As described above, the read data by the reading device 307 is obtained in image format where each pixel is RGB 8 bits, so it can be binarized depending on whether each pixel value is a numerical value representing the paper area or not. For example, binarized data is obtained in which the image drawing area (i.e., the preprinted area, the frame 602) has a pixel value of "1" and the paper area has a pixel value of "0".

[0042] In S702, the data analysis unit 4303 calculates the features of the binarized data acquired in S701. This feature calculation uses known methods for image data, or especially for processing binary image data. Specifically, it performs contour tracking (labeling) and line extraction (e.g., Hough transform) on pixels with a pixel value of "1". It also uses the bounding box, area (number of pixels), and centroid as features. The bounding box is defined as a rectangle that is bound to at least one object written on the paper and can contain all objects within it.

[0043] In S703, the read data analysis unit 4303 estimates and acquires the features of the preprinted paper read data based on the feature quantities obtained in S702. In this embodiment, we will explain the case in which the features of the preprinted paper read data are estimated as a form type as shown in Figure 6. That is, in the case of a typical form type, the frame lines that make up the form are connected and are considered to form a single form. Therefore, when contour tracking is performed, which is a process that scans the entire binarized data and acquires the number and position of objects consisting of connected black pixel values ​​(an example of a "first value"), the number of objects detected is likely to be one or a small number. Note that objects with a small area (number of pixels) are considered to have no meaning as image elements and are merely noise, so such objects may be ignored.

[0044] Furthermore, typical forms are thought to spread out over a large portion of the paper. This is because if they are too small, they are unsuitable for writing on or viewing. In the example in Figure 6, the frame 602 is formed over an area of ​​more than half of the overall outline 601 of the paper. In other words, among the objects obtained by contour tracing, those with a large bounding rectangle surrounding the object can be considered candidates for representing a form. And if that candidate form is indeed a form, the area of ​​the region enclosed by that bounding rectangle is likely to occupy a certain or greater ratio to the entire paper. Alternatively, the vertical and horizontal dimensions of the bounding rectangle are likely to be a certain or greater ratio to the vertical and horizontal dimensions of the paper, respectively.

[0045] Furthermore, a typical form is thought to consist of a border, with the interior left blank as a field for writing. In other words, the area (number of black pixels) of a candidate form object is likely to be small compared to the area of ​​its circumscribing rectangle, and is likely to be below a certain ratio. In addition, the line segments of a typical form are thought to consist of horizontal (left and right) and vertical (up and down) line segments, and are not angled (not diagonal). It can be assumed that multiple such lines exist to form a form. Therefore, if we scan the entire binarized data and apply the Hough transform, which is a process that detects the equations of straight lines in the image, it is highly likely that multiple horizontal and vertical lines will be detected.

[0046] In the Hough transform, the equation of a straight line is expressed as xcosθ+ysinθ=ρ, where x and y (where x is the horizontal and y is the vertical) are given by parameters θ and ρ. In the Hough transform, each pixel, which represents black in the binarized data, is voted for in the (θ,ρ) space as shown in Figure 8, based on the possible parameter (θ,ρ) values ​​it could take if it were assumed to lie on a straight line. Since the parameters of an actual straight line receive votes from many pixels, the equation of a line segment in the image can be identified. In Figure 8, the detection points that received the most votes are indicated by ×. A horizontal line (i.e., y=ρ) is represented as θ=90° and a vertical line (i.e., x=ρ) is represented as θ=0°, so in a typical form, detection points should be concentrated around θ=0° and 90°. If the form consists of multiple cells (2x2), three line segments will be detected: the frames at both ends of the region and the dividing line in the middle. Note that the Hough transform may be applied only to the region of the bounding rectangle of the form candidate, rather than to the entire binarized data. To summarize the above, the set of conditions for form type feature determination in this embodiment in S702-S703 is, for example, as follows. Starting from 1, if the condition is met, the process moves to the next condition determination.

[0047] 1. (Number of objects) The result of contour tracing is that the number of objects is less than or equal to a predetermined threshold Th1. For example, let Th1 = 4.

[0048] 2. (Form Spread) There exists an object whose area enclosed by a bounding rectangle occupies a ratio greater than or equal to a predetermined threshold Th2 relative to the entire paper. This object is designated as a candidate for the form. For example, let Th2 = 0.5.

[0049] 3. (Blank) The total area of ​​the form candidate objects (number of black pixels) is less than or equal to a predetermined threshold Th3 when expressed as a ratio to the area of ​​the region enclosed by its bounding rectangle. For example, Th3 = 0.15.

[0050] 4. Applying the Hough transform to the circumscribing rectangle of the (horizontal and vertical) form candidates reveals that at least three linear parameters are detected in the vicinity of θ=0° and θ=90°.

[0051] If an object exists that satisfies all of the above conditions 1 to 4, the read data analysis unit 4303 estimates that the preprinted paper read data has the characteristics of a form. Note that other conditions may be added or omitted. For example, the condition that a line segment exists in the end region of the circumscribing rectangle may be added.

[0052] After the execution of S703, the process proceeds to S503 in Figure 5. In S503, the read data analysis unit 4303 identifies information for display according to the characteristics of the preprint paper read data acquired in S502 and stores it in RAM 319 or the like. In the following, it is assumed that the characteristics of the preprint paper read data are classified into the above form type. The information stored in this embodiment includes, for example, the type of preprint paper read data, the coordinates of the upper left corner of the form, information about the structure of the form, and display settings that indicate that the abnormal position will be displayed in relative position.

[0053] The type of preprinted paper data refers to a characteristic of this data, such as being of the form type. The coordinates of the top-left corner of the form are, for example, the coordinates of the pixel value "1" at the top-left corner of the object estimated to be a form. Alternatively, they may be the coordinates of the top-left vertex of the bounding rectangle. In this embodiment, these coordinates are used as a reference point in this form type.

[0054] Furthermore, the read data analysis unit 4303 analyzes the structure information of the form and stores it in the RAM 319 or the like. That is, since the equation of each line is known by the Hough transform in S702, the coordinates of the intersection can be determined by calculation. Once the intersection coordinates are known, the structure of the form can be analyzed, such as which cell on the form corresponds to each coordinate within this object. In this embodiment, the read data analysis unit 4303 stores the analysis results as form structure information in the RAM 319 or the like.

[0055] Furthermore, in relatively complex structures where not all line segments penetrate the entire object vertically or horizontally, but disappear midway (a state known as "cell merging"), the intersection points calculated simply from the equation of a straight line may differ from the actual structure. Therefore, it is advisable to perform a process to improve accuracy by verifying whether the calculated intersection coordinates actually define the cells, such as by template matching of the line intersection images.

[0056] Furthermore, if the cells in the preprinted paper reading data contain item names or other information, the cells and item names are linked and stored as structural information of the form. Item names are either recognized and recorded by OCR (Optical Character Recognition) within the cells, or the necessary areas are stored as human-readable images. In addition to the items within the cell, if a neighboring cell that serves as a key to be linked can be identified, the item names of those cells may also be stored. Then, in S504, the reading data analysis unit 4303 stores the acquired preprinted paper reading data itself.

[0057] In S505, the output control unit 4403 of the output control device 104 discharges the preprinted paper to the printed material discharge unit 311 via the paper transport unit 303. Note that no inspection process is performed in preprint reading mode. Then, the preprint reading mode ends.

[0058] From S506 onwards, printing to preprint paper and inspection processing are performed in preprint paper printing mode. For this inspection, the inspection processing device 103 acquires reading data from the printed material to be inspected and reference image data which serves as the inspection standard.

[0059] In other words, in S506, the print job acquisition unit 4102 acquires the print job input to the print image inspection system 100. As mentioned above, the print job is transmitted, for example, from a personal computer (not shown) that can communicate with the image processing device 101 via a network, along with a print instruction. Alternatively, the image processing device 101 may add print settings to image data stored in the external storage device 212 to create a print job.

[0060] In S507, the print data generation unit 4103 refers to the image information and print setting information included in the input print job and performs RIP processing. Through this RIP processing, the input print job is converted into a bitmap reflecting the print settings, becoming print data representing the image to be formed by the image forming apparatus 102. Then, the print data transmission unit 4104 transmits this print data to the print data acquisition unit 4202 of the image forming apparatus 102.

[0061] In S508, the print data acquisition unit 4202 of the image forming apparatus 102 receives the print data. The print data processing unit 4203 then processes the print data according to the image forming characteristics of the image forming apparatus 102 to generate image forming data. This processing includes, for example, color conversion processing, halftone processing, and correction processing. Generally, image forming apparatuses have various image forming characteristics depending on the model and machine, due to the characteristics of the colorants and devices used. The image forming data created in S508 is data that the image forming unit 304 uses to form an image based on the differences between the print data and the image forming characteristics.

[0062] More specifically, the color conversion process converts the colors represented by the image data for printing into the amount of colorant used for image formation. In this embodiment, the colorant used by the image forming unit 304 is toner, and four colors are used: C (cyan), M (magenta), Y (yellow), and K (key plate, black). The colorant is determined by a 3D, 4D, or 1D LUT (Look-Up Table) conversion that shows the conversion relationship between color and colorant amount, which has been acquired in advance as image formation characteristics. Halftone processing is performed to quantize the image data for printing, which is represented by multi-level pixel values, into decimal values ​​that the image forming apparatus 102 can directly represent (for example, binary values ​​representing the presence or absence of toner at a certain pixel position). For example, dithering or error diffusion is used for this. Correction processing is a process that performs edge enhancement to address image characteristics that the image forming apparatus 102 cannot fully reproduce, such as sharp edges. For example, an edge enhancement filter is used for this process. After performing these various processes, image formation data is generated from the print data.

[0063] In S509, the image forming drive unit 4204 drives the image forming unit 304 to print an image on the preprint paper being transported along the paper transport unit 303, based on the image forming data. In the electrophotographic method, the amount of toner specified in the image forming data is held in an image shape on the photosensitive drum by voltage control. After transferring this toner to the printing paper, the toner on the printing paper is fixed to create the printed material. The print data transmission unit 4205 then transmits the print data or image forming data to the inspection processing unit 103. The print data transmission unit 4205 also transmits a signal to the inspection processing unit 103 indicating that printing has been performed. Note that the following flow describes the data transmitted to the inspection processing unit 103 as print data that has not undergone image processing by the print data processing unit 4203. However, this flow is also applicable when the data is image forming data after image processing, or when it is both print data and image forming data.

[0064] In S510, the print data acquisition unit 4304 of the inspection processing device 103 receives print data and a signal indicating that printing has been performed. The read data acquisition unit 4302 drives the reader 307 to read the printed material (an example of "image of one side of a sheet") being transported along the paper transport unit 303 and acquires read data (an example of "sheet image"). As previously described, the read data is an 8-bit image with RGB 3 channels per pixel. In S511, the print data conversion unit 4305 performs conversion processing on the print data to generate a part of the reference image data used in the inspection performed by the inspection processing unit 4306. This conversion includes, for example, resolution conversion processing and color conversion processing. The reference image data is image data representing a printed product without defects. "One side of a sheet" refers to one side of the printed material being transported along the paper transport unit 303, and "sheet image" refers to the image obtained by reading this "one side of a sheet". Furthermore, in the case of a preprint method, one side of the preprinted paper is an example of "one side of the sheet," and the data read from it is also an example of a "sheet image."

[0065] Incidentally, during the inspection process, this reference image data is compared with the read data obtained from reading the printed material being inspected. Since the print data does not contain any abnormalities, in this respect, the print data is suitable as reference image data. However, there are cases where the read data and the print data are not suitable for comparison. One reason for this is that the read data includes images that have been pre-printed on preprint paper, but the print data does not include information about the pre-printed images. In other words, integration of the pre-printed data and the print data is necessary, which will be discussed in S512. Therefore, the data generated in S511 is part of the reference image data.

[0066] The second reason is the difference in image format. For example, while the scanned data is in RGB 3-channel format, the image forming apparatus 102 uses CMYK 4-color toner, so the print data is not necessarily in RGB 3-channel format, but is often in CMYK 4-channel format. Also, the image resolution of the scanned data is determined by the transport speed of the paper transport unit 303 and the reading frequency of the reader 307, but this image resolution is unrelated to the printing processing system. Therefore, the image resolution of the scanned data and the print data do not necessarily match. Thus, step S511 is a process to eliminate this difference and generate (part of) reference image data suitable for comparison with the scanned data.

[0067] The resolution conversion process performed in S511 converts the resolution of the print data to match the resolution of the read data. Alternatively, instead of matching the resolution of the read data, it may be set to an arbitrary inspection resolution that adequately captures abnormalities and does not result in an excessively large image size. Furthermore, the color conversion process performed in S511 is carried out by LUT conversion, which describes the relationship between the pixel values ​​of the print data and the read data. This LUT is created in advance by measuring the correspondence between each pixel value of the print data and which pixel value is recorded as in the read data after printing and reading. In addition, the read data has the image formation characteristics of the image forming unit 304 and the reading characteristics of the reading device 307 added to it. Therefore, a correction process may be performed to simulate and add these characteristics to the print data. In this way, a portion of the reference image data is obtained from the print data. Note that when the print data transmission unit 4205 transmits the image formation data, the above conversion may be applied to the image formation data instead of the print data to generate a portion of the reference image data.

[0068] In S512, the inspection processing unit 4306 reads the preprint paper reading data stored in S504, combines this data with a portion of the reference image data generated in S511, and generates reference image data to be used for inspection. For example, both images may be aligned using the four corners of the paper, and then one image may be used as the base and the other image may be superimposed and combined. For the alignment process, for example, a projection transformation matrix can be used. Alternatively, for example, alpha blending of both images may be used to generate the reference image. This method is effective when there is overlap in the printing.

[0069] In S513, the inspection processing unit 4306 reads the display information stored in S503 and updates the display information if there is any update information related to it. In other words, in this embodiment, if there are no items in the cells of the preprint paper reading data, it is assumed that the items are in the print data. Therefore, as in S503, if the cells of the print data contain item names, the inspection processing unit 4306 updates the display information by associating the cells with the item names and storing them in the RAM 319 or the like. In S514, the inspection processing unit 4306 performs an inspection process to check whether there is an abnormality in the printing section by comparing the read data and the reference image data. Note that S514 includes pre-processing prior to the inspection process. Specifically, pre-processing includes a resolution conversion process to match the image resolution of the read data and the reference image data, and a positioning process to align the positions of both images.

[0070] The resolution conversion process matches the resolution of the reference image data to the resolution of the scanned data if the reference image data was generated at a different image resolution. The alignment process matches the position and orientation of the reference image and the scanned image. That is, the reference image data created from print data is upright and located at the reference position. On the other hand, in actual printing where many copies are printed, the printing paper is transported at high speed. Therefore, when printing an image on such paper, the image is not always printed at exactly the same position and orientation due to the precision of image formation and paper transport. Thus, the scanned data obtained from such printed materials may contain positional and orientation errors. Therefore, since the presence of positional and orientation errors makes it unsuitable for comparison with the reference image data, an alignment process is performed to eliminate positional and orientation errors.

[0071] For the alignment process, a projection transformation matrix using the four corners of the paper may be used, as described in S512 above. Alternatively, since the read data and the reference image data contain the same pattern, feature points from the image may be used for the alignment process. Specifically, the inspection processing unit 4306 performs a feature point extraction process (for example, a method such as AKAZE) from both images to obtain corresponding points in both image data. Furthermore, the inspection processing unit 4306 finds a transformation formula (projection transformation matrix) that matches the corresponding points and performs the transformation to correct for the misalignment of the two images.

[0072] Once this preprocessing is complete, the inspection processing unit 4306 compares the reference image data and the read data pixel by pixel. If the read data is free of abnormalities and closely matches the reference image data, the difference will remain in the range of small values ​​close to 0. On the other hand, if an abnormality is recorded in the read data, a large absolute difference will occur in that part. Typical examples of abnormalities include round abnormalities (referred to as "dots," etc.) caused by the adhesion of colorant to unintended areas during printing, abnormalities caused by insufficient adhesion of colorant to the intended area ("color gaps"), or linear abnormalities ("streaks"). If such abnormalities exist, pixels with large absolute differences will occur clustered to some extent for each type of abnormality. The inspection processing unit 4306 extracts the abnormal areas by applying a spatial filter of a predetermined shape to these differences, and compares the reaction value of the spatial filter with a predetermined threshold to determine whether or not to consider the difference as an abnormality. In this way, the inspection processing unit 4306 determines whether or not there are abnormalities in the read data.

[0073] In S515, the inspection result storage unit 4308 stores the inspection results acquired in S514. Specifically, it records the page where the anomaly occurred, the type of anomaly, and the location of the anomaly. The inspection result storage unit 4308 also performs analysis to display the anomaly according to the characteristics of the image and records the results. Details of the process will be described later in conjunction with the explanation of the anomaly display (S518). Then, the inspection result transmission unit 4307 transmits the inspection results to the output control device 104.

[0074] In S516, the inspection result acquisition unit 4402 of the output control device 104 receives the inspection result. The output control unit 4403 then drives the control drive unit 309 according to the inspection result to guide the printed materials being transported in the paper transport unit 303 to their destination (printed material discharge unit 311 or printed material storage unit 312). In other words, the output control unit 4403 controls the paper transport so that if the inspection is passed, the printed materials are guided to the printed material storage unit 312, and if the inspection is failed, the printed material discharge unit 311.

[0075] In S517, the CPU 313 of the image forming apparatus 102 determines whether or not it has completed printing the specified number of copies in the print job. This information on the number of copies to print is included in the print data received from the image processing apparatus 101 in S507. If it has not completed printing the specified number of copies, the process returns to S509 and repeats the processes from S509 to S516 until it has completed printing the specified number of copies. On the other hand, if the CPU 313 determines that it has completed printing the specified number of copies in the print job, the process proceeds to S518.

[0076] In S518, the inspection result display unit 4309 of the inspection processing device 103 receives inspection result information from the inspection result storage unit 4308 and displays the inspection results, particularly abnormal information, to the user via the inspection UI panel 308. The report creation unit 4310 also receives inspection result information from the inspection result storage unit 4308 and creates an inspection report regarding the inspection results. The displayed abnormal information and inspection report include information on the overall inspection results of the job, such as the total number of prints, the number of accepted items, and the number of rejected items. For rejected items, the location, type, and actual details of the abnormality are shown as evidence, referencing the read data. Reference image data may also be referenced in addition to the read data for comparison.

[0077] Using Figure 9, the inspection results displayed on the inspection UI panel 308 by the inspection result display unit 4309 in S518 will be explained. In Figure 9, frame 901 represents the entire paper of the read data. This frame 901 corresponds to the paper outline 601 in the preprinted paper in Figure 6. Frame 902 is a preprinted form and corresponds to frame 602 in Figure 6. Point 903 is located at the upper left vertex of the form, which is one of the display information acquired in S503, and is the reference point in this embodiment. The read data actually includes not only the form that was preprinted on the preprinted paper, but also an image printed in S509 (overprinted image). In the display example in this embodiment, an overprinted image is actually included, but it is omitted in Figure 9 for illustrative purposes. Now, let's assume that an abnormal "dot" is detected at the × mark 904 in Figure 9. The coordinates of the detection position from the upper left of the paper are (X1, Y1) (in millimeters). In the following explanation, the X direction will be defined as the horizontal direction on the display screen, and the Y direction will be defined as the vertical direction.

[0078] In this embodiment, in S515, the inspection result storage unit 4308 analyzes and stores abnormality information for displaying the abnormal location according to the characteristics of the image. Specifically, the inspection result storage unit 4308 calculates and obtains the relative coordinates of the abnormality with respect to the reference point 903 on the image. In this case, the relative coordinates are (X2, Y2) (units are the same as above). The inspection result display unit 4309 identifies and obtains which cell corresponds to the location of the abnormality from the display information obtained in S503. If a positional misalignment is detected between the read data and the reference image data during the inspection process in S514, and the position of both images is aligned, the inspection result display unit 4309 also corrects the position of the corresponding cell. As shown in Figure 9, the corresponding cell for the location of the abnormality is cell (2, 2) indicated by region 905 with the upper left of the image as the reference point. The inspection result display unit 4309 also identifies and obtains the item name associated with the corresponding cell. This acquired information is stored in S515 along with the test results.

[0079] Furthermore, the inspection result display unit 4309 displays a horizontal arrow 906 and a vertical arrow 907 indicating the location of the anomaly from the reference point 903. Arrows 906 and 907 are examples of "relative position information indicating relative position" and "display elements." Arrow 906 is an example of a "line segment parallel in the left-right direction," and arrow 907 is an example of a "line segment parallel in the up-down direction." The inspection result display unit 4309 also displays these arrows along with their position information (X2mm, Y2mm) (an example of a "display element"). The inspection result display unit 4309 also displays the corresponding cell (2, 2) as area 905, color-coded to distinguish it from other cells (an example of a "display element"). In addition, the inspection result display unit 4309 displays an anomaly information display area 908 (an example of a "display element"). The abnormal information display area 908 displays the inspection results stored in the RAM 319, etc., by the inspection result storage unit 4308 in S515, i.e., information about the abnormality, such as the page on which the abnormality occurred, the type of abnormality, and the location of the abnormality.

[0080] More specifically, n, N, m, M, a, and A in the anomaly information display section 908 are illustrative symbols and are actually represented by numerical values. Also, "front side" in the display section indicates the distinction between the front and back sides of the printed material being inspected. The location of the anomaly is displayed along with the absolute coordinates (X1, Y1) with the upper left vertex of frame 901 as the origin, as well as the relative coordinates (X2, Y2) from the reference point 903. In addition, the position information of the corresponding cell, (2, 2), is displayed. Furthermore, the item name associated with the cell, which is the information for display, is shown as "cell content". Note that C is an illustrative symbol and is actually displayed as a string of characters representing the item. Note that the inspection result display unit 4309 may display the information shown in the anomaly information display section 908 superimposed on frame 901, rather than displaying it separately as a section. Note that the absolute coordinates (X1, Y1) are an example of "absolute position information indicating an absolute position".

[0081] The report generation unit 4310 creates an inspection report using the same method for displaying abnormal locations as the inspection result display unit 4309. The report generation unit 4310 also saves the created report to a storage medium such as the RAM 319 of the inspection processing device 103 so that the user can refer to the report.

[0082] (One aspect of action / effect) According to the print image inspection system 100 described above, it is possible to automatically determine whether or not there are defects in the printed material and collect only the printed output without defects. In addition, the color of the cell where the defect occurs is displayed differently from the color of other cells. Therefore, the user can roughly grasp the location of the defect on the entire sheet of paper. Furthermore, the location information of the defect is concisely displayed as relative position information with respect to a reference point. Therefore, the user can pinpoint the location of the defect in detail from the roughly grasped area. In other words, according to the print image inspection system 100 described above, the absolute position and relative position of the defect location are displayed complementaryly. Therefore, it helps the user to smoothly confirm the location of the defect and has a significant effect in improving user usability.

[0083] More specifically, users identify the type of anomaly by actually observing the anomaly information display or by referring to the inspection report, and by checking the quality of the actual printed output. For example, even if an anomaly is judged as very minor, slightly exceeding the judgment threshold, it is assumed that the quality level is acceptable to a human eye. It is also assumed that an anomaly may be judged when temporary contaminants (dust) are present during data acquisition and are unrelated to the printed output, and that the contaminants can be easily removed. In such cases, a printed output that would otherwise be acceptable is removed as an anomaly, resulting in waste of printing paper, colorants, and time for reprinting. Therefore, if an item judged as an anomaly is inspected and found to be of normal quality, it may be classified as an acceptable item.

[0084] Furthermore, the user can check the status of the image forming apparatus 102 by referring to the display of abnormality information or the inspection report. If severe abnormalities are repeatedly observed, there is a high possibility that a malfunction has occurred in the image forming apparatus 102, and continuing printing will result in the output of many defective products. Therefore, in order to identify the cause of the abnormality, the user may check the status of the image forming apparatus 102 by observing the abnormality and make a decision to perform calibration or request maintenance services depending on the status. In addition, the user can observe the status of the sample that has been determined to be abnormal and adjust the inspection level of subsequent inspections based on that status. Thus, the display of abnormality information or the inspection report described above allows for smooth confirmation of the location of the abnormality, thereby improving the operability of the user when checking the quality of the actual printed output and checking the status of the image forming apparatus 102.

[0085] In the case of a form, most of the content is assumed to reside within the form's boundaries. Therefore, by placing a reference point at the endpoint of the form (top-left coordinate) and indicating the location of an anomaly based on its relative position from the reference point, the location is shown using coordinate values ​​within a meaningful image, without including margins. Consequently, the location of the anomaly can be indicated with numerical values ​​within a necessary and sufficient range. In other words, in the above coordinate system, X1 > X2 and Y1 > Y2, allowing the location of the anomaly to be indicated with more concise numerical values. Furthermore, coloring the corresponding cells provides a clue to intuitively understanding the location of the anomaly. Similarly, displaying the cell's item name also provides a clue to intuitively understanding the location of the anomaly for users familiar with the form's structure. The cell's item name also serves as a clue to whether the content printed within the form is correct or not. Displaying these clues makes it easier for users to pinpoint the approximate location of an anomaly even when illustrating the read data and indicating the location of the anomaly, without necessarily showing the entire image of the read data. Note that the method of indicating the location of an anomaly based on its relative position from the reference point may be used interchangeably with the method of indicating the location of an anomaly based on absolute coordinates on the paper.

[0086] <First variation> In the above embodiment, the object to be inspected was a sheet on which a form-shaped image was printed, but in the first modified example, the object to be inspected is a sheet on which an image with other features is printed. Components similar to those in the above embodiment are denoted by the same reference numerals, and their descriptions are omitted. Furthermore, processes with the same step numbers as in the above embodiment are executed by the same components as in the above embodiment. The sequence processing in the first modified example is also realized by the CPU in each device within the print image inspection system 100 reading the program stored in the storage medium of each device and controlling the aforementioned components.

[0087] Using Figure 10, we will explain the preprint paper according to the first modified example. In the preprint paper according to the first modified example, a pre-printed area 1002 extending vertically is pre-printed on the left side of the frame 1001 representing the entire paper. Such pre-printing is performed when the right side of the pre-printed area 1002 is left blank for the main image, and the pre-printed area 1002 is printed as a fixed secondary image. The main image is printed in small quantities with varying content, but the common secondary image is pre-printed in large quantities. Note that the pre-printed area 1002 does not necessarily have to be drawn entirely within the area shown in the figure, and may include the paper area inside. In this type, since the common pre-printed area 1002 is located on one side of the paper (an example of a "determined edge") and is drawn widely vertically, it can provide a positional guide to the main image, much like a ruler. This type will also be referred to as the ruler type below.

[0088] In the first modified example, similar to the embodiment described above, the read data analysis unit 4303 acquires binarized data in S701, calculates feature quantities of the binarized data in S702, and in S703 estimates and acquires the features of the preprinted paper read data based on the feature quantities calculated in S702. Below, we will explain an estimation method that estimates whether or not the features of the preprinted paper read data are of the ruler type as shown in Figure 10.

[0089] In a typical ruler-type layout, it is likely that a large portion of the page, including the center, is left blank for the main image. In other words, the circumscribing rectangle that encloses all objects is likely to be biased towards a portion of the page. Figure 10 shows an example where it is biased towards the left side of the page. Therefore, the criteria for determining whether it is a ruler-type layout include the fact that the coordinates of the rightmost vertex of the circumscribing rectangle are less than or equal to a certain value relative to the page width. Next, in order to appropriately provide the user with clues about the location of the anomaly as a ruler-type layout, it is desirable that the pre-printed section 1002 extends in a direction perpendicular to the direction in which it exists. In the example in Figure 10, the pre-printed section 1002 extends in a vertical direction perpendicular to the left direction. Therefore, the criteria for determining whether it is a ruler-type layout include the fact that the height of the circumscribing rectangle is greater than or equal to a certain ratio relative to the page height.

[0090] Furthermore, the determination criterion for determining whether something is a ruler type includes the fact that objects are drawn in a certain proportion or more within the area of ​​the pre-printed section 1002. In other words, the above two determination criterion conditions include the possibility that different objects exist separately at both ends (top and bottom, left and right, diagonal) of the bounding rectangle, and that much of the interior of the bounding rectangle is the paper area. Therefore, an additional condition is set that the area of ​​the objects contained within the bounding rectangle is in a certain proportion or more to the area of ​​the bounding rectangle. To summarize the above, the set of conditions for determining the ruler type in S702 to S703 in the first modified example is, for example, as follows. However, thresholds and the like are omitted.

[0091] 1. (Unidirectional bias) The contour tracing results show that the bounding rectangle containing all objects is biased in one direction. This can be determined from the width, height, and vertex coordinates of the bounding rectangle.

[0092] 2. (Expansion in another direction) The bounding rectangle containing all the objects extends in a direction perpendicular to the object's orientation relative to the paper. This can be determined from the width or height of the bounding rectangle.

[0093] 3. (Drawing within the area) The area of ​​the objects contained within the bounding rectangle is at least a certain ratio to the area of ​​the bounding rectangle that contains all the objects.

[0094] If all of the above conditions are met, the characteristics of the preprinted paper read data are presumed to be ruler-type. Note that additional conditions may be added or omitted. For example, to use the preprinted section 1002 as a positional guide like a ruler, it is preferable that the preprinted section 1002 is non-uniform and contains a pattern. To detect such a preprinted section 1002, the read data analysis unit 4303 may refer to the pixel values ​​of the preprinted paper read data corresponding to the area of ​​the preprinted section 1002 and calculate feature quantities. Here, feature quantities for confirming a non-uniform image include, for example, the variance of pixel values ​​or frequency analysis values.

[0095] In S503, the read data analysis unit 4303 identifies and stores information for display. The information stored in the first modified example is the type of preprinted paper read data and information about the ruler structure. The type of preprinted paper read data is that the characteristic of this preprinted paper read data is ruler type. The information about the ruler structure is the coordinates (for 4 points) of the pixels with a pixel value of "1" in the preprinted unit 1002 that are closest to each vertex of the circumscribed rectangle.

[0096] Using Figure 11, the abnormal information of the inspection result displayed on the inspection UI panel 308 by the inspection result display unit 4309 in S518 according to the first modified example will be explained. In Figure 11, frame 1101 represents the entire paper of the read data. This corresponds to frame 1001, which represents the entire paper of the preprinted paper shown in Figure 10. The pre-printing unit 1102 corresponds to the pre-printing unit 1002 in Figure 10. The illustration of the image printed in S509 is omitted, as in Figure 9. Point 1103 is the upper right point of the ruler structure information, which is one of the display information acquired in S503, and is set as the reference point on the image. In this way, of the four pixel vertices of the ruler structure information, the upper vertex that is closest to the center of the image is determined as the reference point.

[0097] Here, let's assume that an abnormal "dot" is detected at the position indicated by the "x" mark in Figure 11. The inspection result display unit 4309 indicates the relative coordinates (X2, Y2) from the reference point 1103 to the abnormal position 1104 using a horizontal arrow 1105 and a vertical arrow 1106. The inspection result display unit 4309 also displays the position information (X2 mm and Y2 mm) along with these arrows. Point 1107 is a position on the ruler that includes the same Y coordinate component as the abnormal position 1104. The inspection result display unit 4309 further displays the abnormality preview area 1108. The abnormality preview area 1108 displays an enlarged image 1109 of the vicinity of the abnormal position 1104 and an enlarged image 1109 of the vicinity of point 1107 (an example of "enlarged display"). In addition, in S518 according to the first modified example, the report creation unit 4310 creates an inspection report using the same abnormal position display method as the inspection result display unit 4309.

[0098] (One aspect of action / effect) As described above, the print image inspection system 100 according to the first modified example analyzes preprint paper data, acquires information for ruler-type errors, and uses it to display the location of errors. In the case of ruler-type errors, the main image is thought to be printed in the space left empty by the preprint section 1102. Therefore, a reference point is placed at an endpoint of the preprint section 1102 that acts as the ruler, close to the error, and the location of the error is indicated by its relative position from the reference point. Thus, the location of the error can be indicated with a more concise numerical value. Furthermore, since the preprint section 1002 is a common section printed in large quantities, displaying the position on the ruler (preprint section 1102) corresponding to the location of the error in the error preview section 1108 can provide a clue for the user to intuitively grasp the location of the error rather than using coordinate values. Thus, even with the first modified example, it is possible to support the user in easily confirming the location of errors from the error information display or inspection report.

[0099] In the first modification, an example was described in which the pre-printed section 1002 is biased to the left, but it can be biased in any direction: up, down, left, or right. Also, in the first modification, the reference point is selected from the four endpoints of the pre-printed section 1102, which is the information for display, and is the uppermost vertex closest to the center of the image. The reason for selecting the uppermost point is that the coordinate direction is familiar because the upper left is often considered the origin in images. The reason for selecting a point close to the center of the image is to indicate the abnormal position within a more necessary and sufficient range. However, the method of selecting the reference point is not limited to this. For example, the one closest to the abnormal position among the four endpoints of the pre-printed section 1102, which is the information for display, may be selected.

[0100] <Second variation> The second modification illustrates a case where the object to be inspected has other characteristics, similar to the first modification. Components identical to those described above are denoted by the same reference numerals, and their descriptions are omitted. Furthermore, processes with the same step numbers as in the above embodiment are executed by the same components as in the above embodiment. The sequence processing in the second modification is also realized by the CPU in each device within the print image inspection system 100 reading the program stored in the storage medium of each device and controlling the aforementioned components.

[0101] The preprint paper relating to the second modification will be explained using Figure 12. The preprint paper read in S501 relating to the first modification is assumed to have pre-printing as shown in Figure 12. That is, multiple circular pre-printed sections 1202 are pre-printed on the frame 1201 representing the entire paper. The pre-printed sections 1202 represent a repeating pattern (an example of an "arrangement"), and it is assumed that the overprint image will be printed over the pre-printed sections 1202. The pre-printed sections 1202 could be, for example, a company or organization's logo. This type will be referred to as the pattern type below. The arrangement of the pre-printed sections 1202 is assumed to be fixed.

[0102] In the second modified example, similar to the above embodiment, the read data analysis unit 4303 acquires binarized data in S701, calculates the feature quantities of the binarized data in S702, and estimates and acquires the features of the preprinted paper read data based on the calculated feature quantities in S703. Below, we will explain an estimation method that estimates whether or not the features of the preprinted paper read data are of a pattern type as shown in Figure 12.

[0103] In the case of a typical pattern, the same pattern is repeated. That is, when contour tracking is performed on binarized data, there is a high probability that multiple objects with almost the same area and circumscribing rectangle will be obtained. Next, it is desirable that the obtained objects be arranged regularly for a pattern to be considered a pattern. The reason for this is that it is possible to determine whether or not it is a pattern by calculating the coordinate values ​​of the centroid of the black pixel among the binarized pixels of each object and comparing these coordinate values. That is, if the vectors between the centroids of the closest objects are approximately the same vector, it can be determined that the objects are likely to be arranged regularly. Therefore, the set of conditions for determining the pattern type in S702 to S703 in the second modified example is as follows.

[0104] 1. (Multiple identical objects) As a result of contour tracing, multiple objects with equivalent area and bounding rectangles are obtained.

[0105] 2. (Regular arrangement) The difference in coordinates of the centroids of the closest objects can be represented by a roughly constant vector.

[0106] If all of the above conditions are met, the read data analysis unit 4303 estimates that the preprinted paper read data has pattern-type features. Note that additional conditions may be added or omitted. For example, area conditions may be imposed such that each object is neither too small nor too large, or the distance of the circumscribing rectangle may be compared in addition to the centroid. Furthermore, similarity between objects may be guaranteed by template matching.

[0107] Furthermore, in S503 related to the second modified example, the read data analysis unit 4303 identifies information for display and stores information on the type of preprint paper read data and the pattern structure. The type of preprint paper read data is that the characteristics of this preprint paper read data are of the pattern type. The information on the pattern structure is the centroid position for each object, the vertex positions of the circumscribing rectangle, and the position of the reference point. Multiple reference points are determined for each object. That is, the coordinates of the leftmost pixel (the pixel with the smallest x-coordinate) in the area of ​​pixel value "1" corresponding to each object are determined as the reference point.

[0108] Using Figure 13, the abnormal information of the inspection result displayed on the inspection UI panel 308 by the inspection result display unit 4309 in S518 of the second modified example will be explained. In Figure 13, frame 1301 represents the entire paper of the read data. This corresponds to frame 1201, which represents the entire paper in the preprinted paper shown in Figure 12. The pre-printing unit 1302 corresponds to the pre-printing unit 1202 in Figure 12. Note that the illustration of the printed image in S509 is omitted, as in Figure 9. Here, let's assume that an abnormality "dot" is detected at the position indicated by the X mark in Figure 13. And let's assume that this detection position is within the bounding rectangle of pattern 1304. Note that point 1305 is the reference point in pattern 1304 that contains the abnormality within the bounding rectangle.

[0109] The inspection result display unit 4309 indicates the relative coordinates (X2, Y2) from the reference point 1305 to the abnormal position 1303 using a horizontal arrow 1306 and a vertical arrow 1307. The inspection result display unit 4309 also displays the pattern 1304 containing the abnormal position in a different color from the other patterns. The inspection result display unit 4309 also displays an abnormal information display area 1308. The abnormal information display area 1308 indicates which pattern 1304 containing the abnormality within its bounding rectangle is in the read data frame 1301, using (2, 1) as the reference point (1, 1) for the top left of the entire paper, meaning it is the second in the X direction and the first in the Y direction. Such a display can be derived from the fact that the centroid coordinates of each regularly arranged pattern are known.

[0110] Furthermore, Figure 14 is used to explain the display format of the abnormal location when the detected abnormality is not within the bounding rectangle of any pattern. Note that the symbols shown in Figure 14 are almost the same as those in Figure 13, so the differences will be explained. The abnormal location 1403 is not included within the bounding rectangle of any pattern 1402. In such a case, the coordinates of the leftmost pixel (the one with the smallest X coordinate) in the pattern closest to the centroid are selected as the reference point. Note that the dashed lines in Figure 14 are shown to clarify the pattern containing the reference point. Note that in Figure 14, since there is no abnormality within the pattern, pattern 1404 is not displayed with a different color from the other patterns. The report creation unit 4310 creates an inspection report using the same abnormal location display method as the inspection result display unit 4309. This second modification also produces the same effects as the embodiment described above.

[0111] <Third variation> In the third modification, similar to the first modification, an example is given where the object to be inspected has other characteristics. Note that components identical to those described above are denoted by the same reference numerals, and their descriptions are omitted. Furthermore, processes with the same step numbers as in the above embodiment are executed by the same components as in the above embodiment. The sequence processing in the third modification is also realized by the CPU in each device within the print image inspection system 100 reading the program stored in the storage medium of each device and controlling the aforementioned components.

[0112] Using Figure 15, we will explain the preprinted paper according to the third modified example. The preprinted paper read by S501 according to the third modified example is assumed to have pre-printing as shown in Figure 15. That is, multiple pre-printed sections 1502 are pre-printed on a frame 1501 that represents the entire paper. The pre-printed sections 1502 are located along the four sides of the periphery of the printing paper (examples of "frame-shaped" and "rectangular" shapes), and it is assumed that printing on this preprinted paper will be done inside the empty spaces of the pre-printed sections 1502. The pre-printed sections 1502 could be, for example, for decorating the document. This type will be referred to as the frame type below.

[0113] In the third modified example, similar to the above embodiment, the read data analysis unit 4303 acquires binarized data in S701, calculates the feature quantities of the binarized data in S702, and estimates and acquires the features of the preprinted paper read data based on the calculated feature quantities in S703. Below, we will explain the estimation method for estimating whether or not the features of the preprinted paper read data are of the frame type as shown in Figure 15.

[0114] In the case of a typical frame type, similar to the form type, the bounding rectangle of the pre-printed section 1502 is considered to extend across the entire paper. Also, the number of detected objects is likely to be one or a small number. Furthermore, since the interior is left empty for printing, the ratio of the area of ​​the object's pixel value "1" to the area of ​​the bounding rectangle is small. The difference from the form type is that there is a large, consolidated area of ​​only the paper inside. This difference can be determined by referring to the pixel values ​​inside this bounding rectangle. Therefore, the set of conditions for determining the frame type in S702-S703 related to the third modified example is, for example, as follows.

[0115] 1. (Number of objects) The result of contour tracing shows that the number of objects is less than or equal to a predetermined number.

[0116] 2. (Spread) There exists an object whose bounding rectangle occupies a certain ratio or more of the entire paper. This object will be designated as a frame candidate.

[0117] 3. (Internal blank) The area of ​​the frame candidate object (number of pixels of "1") is less than or equal to a predetermined ratio to the area of ​​its circumscribed rectangle. Furthermore, an inscribed rectangle consisting only of the paper area can be placed within the circumscribed rectangle at a ratio greater than or equal to a certain ratio of the circumscribed rectangle.

[0118] If all of the above conditions are met, this preprinted paper scan data is presumed to be of frame type. In addition, other conditions may be added, such as the absence of objects in the area outside the frame and the presence of a preprinted area 1502 in the edge region of the bounding rectangle.

[0119] In S503, the read data analysis unit 4303 identifies and stores information for display. In the third modified example, the information stored is the type of preprinted paper read data and information about the frame structure. The type of preprinted paper read data is that the characteristic of this preprinted paper read data is frame type. The information about the pattern structure is the coordinates of the pixel value "1" closest to the vertex position of the circumscribing rectangle of the object representing the frame. These become the positions of the reference points.

[0120] Using Figure 16, the abnormal information of the inspection result displayed on the inspection UI panel 308 by the inspection result display unit 4309 in S518 of the third modified example will be explained. In Figure 16, frame 1601 represents the entire paper of the read data. Frame 1601 corresponds to frame 1501 in Figure 15, which represents the entire paper in the pre-printed paper. The pre-printing unit 1602 corresponds to the pre-printing unit 1502 in Figure 15. Note that the illustration of the printed image in S509 is omitted, as in Figure 9. Now, suppose an abnormal "dot" is detected at the position indicated by the "x" mark. In such a case, the point closest to the "x" mark among the coordinates of the four corners, which are information of the structure of the stored frame, is selected as the reference point 1604 (an example of an "outer vertex of the frame").

[0121] The inspection result display unit 4309 indicates the relative coordinates (X2, Y2) from the reference point 1604 to the abnormal position 1603 using a horizontal arrow 1605 and a vertical arrow 1606. The inspection result display unit 4309 also displays the position information (X2 mm and Y2 mm) along with these arrows. Although not shown in Figure 16, the abnormal position may be displayed by referring to the portion of the pre-printed frame 1602 that has the same X or Y coordinate as the abnormal position 1603, similar to Figure 11 in the first modified example. Furthermore, in S518 in the third modified example, the report creation unit 4310 creates an inspection report using the same method of displaying the abnormal position as the inspection result display unit 4309. This third modified example also produces the same effects as the embodiment described above.

[0122] <Fourth variation> In the fourth modification, it is assumed that the preprinted paper read by S501 in the first modification is pre-printed as shown in Figure 17. That is, a pre-printed area 1702 is pre-printed in a relatively small area in the lower right of the frame 1701 that shows the entire paper. Such pre-printing can occur when the pre-printed area 1702 is a company logo or various marks. In the first modification, because the condition is imposed that the pre-printed area 1702 has an extent, the preprinted paper shown in Figure 17 is not judged to have the characteristics of a ruler type. However, let's assume that a reference point is set in the pre-printed area 1702 and the position of the anomaly is displayed relative to it.

[0123] Figure 18 will be used to explain the display format of the abnormal location in such cases. In Figure 18, frame 1801 represents the entire paper of the read data and corresponds to frame 1701 in Figure 17. The pre-printed section 1802 corresponds to the pre-printed section 1702 in Figure 17. Note that the illustration of the image printed in S509 is omitted, as in Figure 9. The reference point is assumed to be set at the upper left vertex of the pre-printed section 1802. Now, let's assume that an abnormal "dot" is detected at the position indicated by the X mark in Figure 18.

[0124] In this case, the relative coordinates (X2, Y2) from the reference point 1803 to the abnormal position 1804 will have a larger absolute value than the absolute coordinates (X1, Y1) with the top-left vertex of the paper as the origin. More generally, because the pre-printed area 1802 is small, the magnitude of the possible values ​​for absolute and relative coordinates does not change much. However, because the pre-printed area 1802 is printed in the lower right, the position of the reference point relative to the origin of the absolute coordinates changes significantly, and the abnormal display is greatly affected by this. Furthermore, even if a part of the pre-printed area 1802 is referenced and displayed relatively, it contributes little to the ease of understanding the abnormal position or the overall image. As a result, in this situation, displaying the pre-printed area 1802 relative to the reference point 1803 is considered to have little display benefit to the user and may even confuse the user.

[0125] Therefore, in S503, the read data analysis unit 4303 (an example of a "display setting unit") may set it so that relative display based on the image is not performed if the image does not meet predetermined conditions as a result of the analysis. In other words, it is not always convenient for the user to perform relative display based on the image, and it is considered to be in the user's best interest to determine when relative display should be performed and then implement it. In such cases, the read data analysis unit 4303 may store the setting to not perform relative display in RAM 319 or the like as display information in S503, and this information may be transmitted to subsequent flows.

[0126] <Other variations> When displaying relative positions in the horizontal (x) direction from right to left and the vertical (y) direction from bottom to top, as shown in Figure 18, X2 and Y2 may be displayed as negative numbers. However, since the direction is indicated by arrows 1805 and 1806, the absolute value of the distance may also be displayed. Alternatively, negative numbers may be included in the coordinate values, while absolute values ​​may be used to indicate distances in the figure.

[0127] Furthermore, the embodiments and modifications described above illustrate examples of the preprint method. In the preprint method, a large number of preprinted sheets with similar objects pre-printed are generated. Therefore, users become accustomed to seeing the objects, and relative display, which uses a part of the object as a positional reference, makes it easier for users to identify anomalies. Also, in the preprint method, preprinting and subsequent overprinting are performed at different times. Therefore, the images printed in preprinting and subsequent overprinting can be structurally separated to some extent. Therefore, estimation of features by categorizing them is easy. In addition, having a structure in which images can be separated and easily categorized leads to ease of user understanding. Furthermore, the jobs for preprinting and overprinting are different, and the print data does not contain the image information of preprinting. For this reason, information for preprinting is acquired separately in advance in order to integrate the information of both images (S501~S505). And, since image feature analysis is performed during the execution of these processes, delays in subsequent anomaly detection processing are suppressed. Thus, the compatibility of the preprint method with anomaly image inspection is also an advantage.

[0128] However, in the print image inspection system 100 according to this embodiment, printing on preprinted paper is not required, as described above. In the case of printing that is not done using the preprint method, the analysis processing corresponding to S501 to S505 in the embodiment can be performed on the read data acquired in S510. Alternatively, the analysis processing may be performed on the print data to be processed in S511. This is because if the preprint method is not used, the print data will include objects such as form types. Figure 19 will be used to explain the functional blocks of each device in the print image inspection system when analysis processing is performed on print data. The difference between the functional blocks in Figure 4 and the functional blocks in Figure 19 is that the inspection processing device 103 includes a print data analysis unit 1901. The print data analysis unit 1901 performs analysis processing on the print data to analyze the image features. The print data analysis unit 1901 then acquires the features of the object and the object to be inspected. Such a print image inspection system is applicable when preprinted paper is not used.

[0129] Furthermore, in the embodiments and modifications described above, the flow for determining whether or not an image possesses each characteristic (type) has been described separately. However, these determinations may be integrated and performed as a single flow. Also, the display method for indicating the location of the anomaly described above may be partially implemented or partially modified. In addition, as an alternative to analyzing image data such as preprint paper scan data and estimating and acquiring its characteristics, the user may specify information about the characteristics in advance. In such a case, the image characteristics can be acquired without estimation, and some determination processing can be omitted. Alternatively, since the criteria for the characteristics are known, the thresholds and flow of the determination processing may be changed to handle more complex cases of those characteristics.

[0130] Furthermore, the methods used for estimating and acquiring image features are not limited to those described above, and alternative methods that achieve similar effects may be employed. For example, in the embodiment, an edge detection filter may be used instead of the Hough transform, or a machine learning model may be used to determine whether or not the image data has form features. In addition, the display method showing relative positions in the image may be switched on or off at the user's discretion depending on the display mode accepted by the inspection processing device 103.

[0131] Furthermore, the configuration within the print image inspection system 100, or the classification of devices and functional parts within each device constituting the print image inspection system 100, is not limited to the above. In the above embodiment, what was described as a separate configuration may be an integrated device, and conversely, what was an integrated device may be classified as a separate group of devices. For example, it may be a configuration such as an inline inspection machine that performs image processing, image formation, inspection, and paper discharge in an integrated manner. Also, although paper is given as an example of a recording material above, the recording material may be other sheet materials.

[0132] <Other Embodiments> The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0133] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention.

[0134] The disclosures herein include the following image processing apparatus, control methods thereof, and programs. [Item 1] An acquisition unit that acquires an image of one side of the sheet, The image processing unit processes the sheet image acquired by the acquisition unit and detects the features of the sheet image, A determination unit determines whether or not there is an abnormality in the sheet image acquired by the acquisition unit, based on the features detected by the image processing unit. If the determination unit determines that there is an abnormality in the sheet image, the display setting unit sets the content to be displayed on the screen, The system includes an output unit that outputs the content set by the display setting unit, The display setting unit sets a reference point in the sheet image and sets whether or not to display relative position information indicating the relative position of the abnormality with respect to the reference point. The image processing apparatus is characterized in that, when the display setting unit is set to enable the display of the relative position information, it sets the relative position information to be displayed on the screen. [Item 2] The aforementioned sheet image includes objects that are not subject to the determination of whether or not there is an abnormality, The image processing unit detects the object, The image processing apparatus according to item 1, characterized in that the display setting unit sets the reference point at the position of the object detected by the image processing unit. [Item 3] The image processing apparatus according to item 2, characterized in that the display setting unit sets the display of the relative position information to "on" when the object satisfies the conditions of a predetermined type, and sets the content to be displayed on the screen according to the conditions of the predetermined type. [Item 4] The image processing apparatus according to item 3, characterized in that the predetermined type includes at least one of a form with a writing area provided inside by a border, a ruler provided along a predetermined side of a sheet, a pattern in which multiple images are arranged, and a frame-shaped type. [Item 5] The image processing apparatus according to item 4, characterized in that, when the object is of form type, the display setting unit sets a display that can identify the cell containing the abnormality among the cells included in the form. [Item 6] The image processing apparatus according to item 5, characterized in that the display setting unit sets the display of the contents described in the cell containing the abnormality. [Item 7] The image processing apparatus according to item 4, characterized in that, when the object is of a pattern type, the display setting unit sets the reference point in the pattern closest to the position where the anomaly was detected. [Item 8] The image processing apparatus according to item 4 or 7, characterized in that the display setting unit sets a display that can distinguish the pattern from other patterns. [Item 9] The image processing apparatus according to item 4, characterized in that, when the object is of the rectangular frame type, the display setting unit sets one of the outer vertices of the frame as the reference point. [Item 10] The image processing apparatus according to item 4, characterized in that, when the object is a rectangular ruler, the display setting unit sets the point closest to the center of the sheet image among the vertices of the ruler as the reference point. [Item 11] The image processing apparatus according to item 4 or 10, characterized in that the display setting unit sets an enlarged display of the vicinity of the abnormality location and the vicinity of the reference point when the object is of the ruler type. [Item 12] The image processing apparatus according to any one of items 3 to 11, characterized in that the display setting unit sets a display element that includes line segments parallel in the left-right direction and line segments parallel in the up-down direction in the sheet image in order to identify the relative position of the abnormality with respect to the reference point on the screen. [Item 13] The image processing device according to any one of items 3 to 12, characterized in that the image processing device binarizes each pixel value of the object, traces the contour of the pixel that has become a first value, and estimates the predetermined type by performing at least one of the following processes: detection of a bounding rectangle that circumscribes the detected contour, detection of a straight line in the bounding rectangle, detection of the area of ​​the region enclosed by the bounding rectangle, detection of the area obtained by summing the pixels, and detection of the centroid of the pixels. [Item 14] The acquisition unit includes a reading unit that reads a sheet on which the object to be determined is not formed and on which the object is formed. The image processing apparatus according to any one of items 2 to 13, characterized in that the image processing unit detects the object from the sheet image read by the reading unit. [Item 15] The image processing apparatus according to item 2, characterized in that the display setting unit sets the display of the relative position information to none when the object is located in the corner of the sheet image. [Item 16] The image processing apparatus according to any one of items 1 to 15, further characterized in that the display setting unit sets absolute position information indicating the absolute position of the abnormality in the sheet image for display on the screen. [Item 17] A method for controlling an image processing device, The acquisition unit performs an acquisition process to acquire an image of one side of the sheet, The image processing unit processes the sheet image acquired by the acquisition unit and performs an image processing step to detect the features of the sheet image, The determination unit performs a determination step of determining whether or not there is an abnormality in the sheet image acquired by the acquisition unit, based on the features detected by the image processing unit. The display setting unit performs a display setting step in which it sets the content to be displayed on the screen when the determination unit determines that there is an abnormality in the sheet image, The output unit includes an output step of outputting the content set by the display setting unit, The display setting unit sets a reference point in the sheet image and sets whether or not to display relative position information indicating the relative position of the abnormality with respect to the reference point. The control method for an image processing apparatus is characterized in that, when the display setting unit is set to enable the display of the relative position information, it sets the relative position information to be displayed on the screen. [Item 18] A program for causing a computer to execute each step in a control method for an image processing device, wherein the control method is: The acquisition unit performs an acquisition process to acquire an image of one side of the sheet, The image processing unit processes the sheet image acquired by the acquisition unit and performs an image processing step to detect the features of the sheet image, The determination unit performs a determination step of determining whether or not there is an abnormality in the sheet image acquired by the acquisition unit, based on the features detected by the image processing unit. The display setting unit performs a display setting step in which it sets the content to be displayed on the screen when the determination unit determines that there is an abnormality in the sheet image, The output unit includes an output step of outputting the content set by the display setting unit, The display setting unit sets a reference point in the sheet image and sets whether or not to display relative position information indicating the relative position of the abnormality with respect to the reference point. The program is characterized in that, when the display setting unit is set to enable the display of the relative position information, it sets the relative position information to be displayed on the screen. [Explanation of symbols]

[0135] 100: Print image inspection system, 101: Image processing device, 102: Image forming device, 103: Inspection processing device, 104: Output control device, 105: Communication cable, 903: Reference point, 906, 907: Arrow, 908: Anomaly information display area

Claims

1. An acquisition unit that acquires an image of one side of the sheet, The image processing unit processes the sheet image acquired by the acquisition unit and detects the features of the sheet image, A determination unit determines whether or not there is an abnormality in the sheet image acquired by the acquisition unit, based on the features detected by the image processing unit. If the determination unit determines that there is an abnormality in the sheet image, the display setting unit sets the content to be displayed on the screen, The system includes an output unit that outputs the content set by the display setting unit, The display setting unit sets a reference point in the sheet image and sets whether or not to display relative position information indicating the relative position of the abnormality with respect to the reference point. The image processing apparatus is characterized in that, when the display setting unit is set to enable the display of the relative position information, it sets the relative position information to be displayed on the screen.

2. The aforementioned sheet image includes objects that are not subject to the determination of whether or not there is an abnormality, The image processing unit detects the object, The image processing apparatus according to claim 1, characterized in that the display setting unit sets the reference point at the position of the object detected by the image processing unit.

3. The image processing apparatus according to claim 2, characterized in that the display setting unit sets the display of the relative position information to "on" when the object satisfies the conditions of a predetermined type, and sets the content to be displayed on the screen according to the conditions of the predetermined type.

4. The image processing apparatus according to claim 3, characterized in that the predetermined type includes at least one of a form with a writing area provided inside by a border, a ruler provided along a predetermined side of a sheet, a pattern in which multiple images are arranged, and a frame-shaped type.

5. The image processing apparatus according to claim 4, wherein the display setting unit sets a display that can identify the cell containing the abnormality among the cells included in the form when the object is of form type.

6. The image processing apparatus according to claim 5, characterized in that the display setting unit sets the display of the contents described in the cell containing the abnormality.

7. The image processing apparatus according to claim 4, characterized in that, when the object is of a pattern type, the display setting unit sets the reference point in the pattern closest to the position where the anomaly was detected.

8. The image processing apparatus according to claim 4, characterized in that the display setting unit sets a display that makes the pattern distinguishable from other patterns.

9. The image processing apparatus according to claim 4, characterized in that, when the object is of the rectangular frame type, the display setting unit sets one of the outer vertices of the frame as the reference point.

10. The image processing apparatus according to claim 4, characterized in that, when the object is a rectangular ruler, the display setting unit sets the point closest to the center of the sheet image among the vertices of the ruler as the reference point.

11. The image processing apparatus according to claim 4, wherein the display setting unit sets an enlarged display of the vicinity of the abnormality location and the vicinity of the reference point when the object is of the ruler type.

12. The image processing apparatus according to claim 3, wherein the display setting unit sets a display element that includes line segments parallel in the left-right direction and line segments parallel in the up-down direction in the sheet image in order to identify the relative position of the abnormality with respect to the reference point on the screen.

13. The image processing apparatus according to claim 3, characterized in that the image processing unit binarizes each pixel value of the object, and estimates the predetermined type by performing at least one of the following processes: tracking the contour of the pixel that has become a first value; detecting a bounding rectangle that circumscribes the detected contour; detecting a straight line in the bounding rectangle; detecting the area of ​​the region enclosed by the bounding rectangle; detecting the area obtained by summing the pixels; and detecting the centroid of the pixels.

14. The acquisition unit includes a reading unit that reads a sheet on which the object to be determined is not formed and on which the object is formed. The image processing apparatus according to claim 2, characterized in that the image processing unit detects the object from the sheet image read by the reading unit.

15. The image processing apparatus according to claim 2, characterized in that the display setting unit sets the display of the relative position information to none when the object is located in the corner of the sheet image.

16. The image processing apparatus according to any one of claims 1 to 15, wherein the display setting unit further sets absolute position information indicating the absolute position of the abnormality in the sheet image for display on the screen.

17. A method for controlling an image processing device, The acquisition unit performs an acquisition process to acquire an image of one side of the sheet, The image processing unit processes the sheet image acquired by the acquisition unit and performs an image processing step to detect the features of the sheet image, The determination unit performs a determination step of determining whether or not there is an abnormality in the sheet image acquired by the acquisition unit, based on the features detected by the image processing unit. The display setting unit performs a display setting step in which it sets the content to be displayed on the screen when the determination unit determines that there is an abnormality in the sheet image, The output unit includes an output step of outputting the content set by the display setting unit, The display setting unit sets a reference point in the sheet image and sets whether or not to display relative position information indicating the relative position of the abnormality with respect to the reference point. The control method for an image processing apparatus is characterized in that, when the display setting unit is set to enable the display of the relative position information, it sets the relative position information to be displayed on the screen.

18. A program for causing a computer to execute each step in a control method for an image processing device, wherein the control method is: The acquisition unit performs an acquisition process to acquire an image of one side of the sheet, The image processing unit processes the sheet image acquired by the acquisition unit and performs an image processing step to detect the features of the sheet image, The determination unit performs a determination step of determining whether or not there is an abnormality in the sheet image acquired by the acquisition unit, based on the features detected by the image processing unit. The display setting unit performs a display setting step in which it sets the content to be displayed on the screen when the determination unit determines that there is an abnormality in the sheet image, The output unit includes an output step of outputting the content set by the display setting unit, The display setting unit sets a reference point in the sheet image and sets whether or not to display relative position information indicating the relative position of the abnormality with respect to the reference point. The program is characterized in that, when the display setting unit is set to enable the display of the relative position information, it sets the relative position information to be displayed on the screen.

Images