Inspection device, control method for inspection device, and program

Abstract

The present invention provides a novel mechanism that enables, for example, inspection of printed sheets to conform to user intent, print job, or post-processing. [Solution] The inspection device comprises an acquisition unit that acquires an image including the sheet to be inspected, a setting unit that sets a reference point that serves as the basis for corner folding of the sheet to be inspected, and a determination unit that determines whether or not the sheet to be inspected has corner folding based on the reference point and the contour of the sheet to be inspected in the image.

Description

Technical Field

[0001] The present invention relates to an inspection device, a control method for an inspection device, and a program.

Background Art

[0002] In recent years, a printing system for inspecting a printed sheet printed by a printing device in the printing device or an inspection device is known. The inspection device reads an image of the conveyed printed sheet and analyzes the read image to determine whether the printed sheet is normal. That is, the inspection device can detect, for example, the deviation of the printing position of a pattern from the paper (printing position deviation), printing abnormalities, corner folding of the printed sheet, etc. Regarding the corner folding of the printed sheet, Patent Document 1 describes a technique for determining corner folding based on the angle of the paper edge. In addition, Patent Document 2 describes a technique for determining that it is normal when the corner folding is within the cutting area. When the printed sheet is thus determined to be abnormal, the abnormal printed sheet is discharged to a discharge destination different from that of the normal printed sheet. This prevents the abnormal printed sheet from being mixed into the normal printed sheet, and enables the operator to discard the abnormal printed sheet.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the case of the technology described in Patent Document 1, even if the amount of folding is small and the sheet is one that the user wishes to use, it may be judged as abnormal and discarded. Therefore, such corner folding detection does not conform to the user's intentions. Furthermore, in the case of the technology described in Patent Document 2, when printing a patch in the cutting area, the corner folding and the patch may overlap. It may also affect the stackability in the post-processing device and the linearity during cutting. Therefore, such corner folding detection is not suitable for print jobs or post-processing.

[0005] The present invention has been made in view of at least one of the above-mentioned problems and provides a novel mechanism that enables the inspection of printed sheets to conform to the user's intent, print job, or post-processing. [Means for solving the problem]

[0006] The present invention, for example, An acquisition unit that acquires an image including the sheet to be inspected, A setting unit for setting a reference point that serves as the basis for corner folding of the sheet to be inspected, The system includes a determination unit that determines whether or not the corners of the sheet to be inspected are bent, based on the aforementioned reference point and the contour of the sheet to be inspected in the aforementioned image. It is an inspection device. [Effects of the Invention]

[0007] According to the present invention, it is possible to perform inspection of printed sheets that conform to the user's intent, print job, or post-processing. [Brief explanation of the drawing]

[0008] [Figure 1] Overview diagram of a printing system according to one embodiment. [Figure 2] Block diagram of a printing system according to one embodiment. [Figure 3] Schematic diagram of an image forming apparatus according to one embodiment [Figure 4] Flowchart of the process according to one embodiment [Figure 5]Flowchart of the process according to an embodiment [Figure 6] Flowchart of the process according to an embodiment [Figure 7] Flowchart of the process according to an embodiment [Figure 8] Flowchart of the process according to an embodiment [Figure 9] Flowchart of the process according to an embodiment [Figure 10] Flowchart of the process according to an embodiment [Figure 11] Flowchart of the process according to an embodiment [Figure 12] Flowchart of the process according to an embodiment [Figure 13] Diagram showing a display screen according to an embodiment [Figure 14] Diagram showing a display screen according to an embodiment [Figure 15] Explanatory diagram of the calculation of corner folds according to an embodiment [Figure 16] Explanatory diagram of the calculation of edges according to an embodiment [Figure 17] Explanatory diagram of the definition of the amount of corner folds according to an embodiment [Figure 18] Explanatory diagram of the search algorithm for corner folds according to an embodiment [Figure 19] Flowchart of the process according to an embodiment [Figure 20] Diagram showing a display screen according to an embodiment [Figure 21] Diagram showing a display screen according to an embodiment [Figure 22] Flowchart of the process according to an embodiment [Figure 23] Explanatory diagram of the calculation of virtual vertices according to an embodiment

Mode for Carrying Out the Invention

[0009] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims. While the embodiments describe multiple features, not all of these features are essential to the invention, and the features may be combined in any way. Furthermore, in the attached drawings, identical or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0010] In this specification, the term "image forming apparatus" broadly includes devices that form (record) images on recording materials (recording media), such as single-function printers, copiers, multifunction printers, and commercial printing presses. Furthermore, an image forming apparatus may also be a system (image forming system) that connects an image forming apparatus main unit, which forms images on recording materials, with equipment such as a sheet processing device and a sheet feeding device.

[0011] In the following description, the external controller is also referred to as an image processing controller, digital front-end (DFE), or print server. The image forming apparatus is also referred to as a multifunction printer or multifunction peripheral (MFP).

[0012] The overall hardware configuration of the image forming system 100 will be explained using Figure 1. The image forming system 100 consists of an image forming apparatus 101 and an external controller 102. The image forming apparatus 101 and the external controller 102 are connected via an internal LAN 105 and a video cable 106 for communication. Note that the video cable 106 may not have its own configuration, and its function may be replaced by the internal LAN 105. The external controller 102 is connected via an external LAN 104 for communication with a client PC 103, and print commands are sent from the client PC 103 to the external controller 102.

[0013] The client PC 103 has a printer driver installed that has the function of converting print data into a print description language that can be processed by the external controller 102. Users who want to print can issue print commands from various applications via the printer driver. Based on the print command from the user, the printer driver sends print data to the external controller 102. When the external controller 102 receives a print command from the PC 103, it performs data analysis and rasterization processing, feeds the print data to the image forming apparatus 101, and issues a print command.

[0014] Next, the image forming apparatus 101 will be described. The image forming apparatus 101 is connected to multiple devices having different functions and is configured to enable complex printing processes such as bookbinding. These multiple devices having different functions include, for example, a printing device 107, an inserter 108, an inspection device 109, a large-capacity stacker 110, and a finisher 111.

[0015] The printing device 107 forms an image on the unprinted media (hereinafter also referred to as paper) transported from the paper feed section located at the bottom of the printing device 107 using toner. The paper on which the image has been formed will be referred to as a printed sheet below. The inserter 108 inserts the inserted paper into the printed material printed by the printing device 107. That is, the inserter 108 can insert paper between any sheets in a group of printed sheets that have been printed and transported by the printing device 107.

[0016] The inspection device 109 reads the image of the transported printed sheet and compares it with a pre-registered correct image to determine whether the printed image is normal or not. The large-capacity stacker 110 is a stacker capable of loading a large number of printed sheets. The finisher 111 applies finishing processing to the transported printed sheets. For example, the finisher 111 can perform finishing processing on the printed sheets, such as stapling, punching, or saddle stitching. The finisher 111 also discharges the finished printed sheets into the output tray.

[0017] The image forming system 100 is configured such that an external controller 102 is connected to the image forming apparatus 101, but it is not limited to this configuration. That is, the image forming system 100 may be configured such that the image forming apparatus 101 is connected to an external LAN 104, and processable print data is transmitted to the image forming apparatus 101 from a client PC 103 connected to the external LAN 104. Alternatively, the image forming system 100 may be configured to read print data from an HDD 221 (described later) inside the printing apparatus 107. In this case, data analysis and rasterization processing are performed in the image forming apparatus 101, and the printing process is executed.

[0018] Using Figure 2, the blocks of the image forming apparatus 101, external controller 102, and client PC 103 will be explained. First, the configuration of the printing apparatus 107 of the image forming apparatus 101 will be explained. The printing apparatus 107 of the image forming apparatus 101 is composed of a communication I / F 217, LAN I / F 218, video I / F 220, HDD 221, CPU 222, memory 223, operation unit 224, and display 225. Furthermore, the printing apparatus 107 is composed of a document reading unit 226, latent image unit 227, image formation unit 228, fixing unit 229, and paper feeding and transport unit 230. In addition, each component is connected to the others via a system bus 231.

[0019] The communication interface 217 includes a communication module and is connected to the inserter 108, inspection device 109, high-capacity stacker 110, and finisher 111 via a communication cable 254. This connection enables communication for the control of each device. The LAN interface 218 includes, for example, a Network Interface Card (NIC) and is connected to the external controller 102 via the internal LAN 105. This connection enables communication of print data and other data. The video interface 220 includes a module for communicating video signals and is connected to the external controller 102 via a video cable 106. This connection enables communication of image data and other data.

[0020] The HDD221 is a storage device where programs and data are stored. The CPU222 comprehensively controls image processing and printing based on the programs stored in the HDD221. The memory223 stores programs and image data necessary for the CPU222 to perform various processes and operates as a work area. The operation unit224 is, for example, a button that can be pressed and accepts input of various settings and operation instructions from the user. The display225 displays setting information of the printing device 107 and the processing status of print jobs. The printing device 107 may also be equipped with a touch panel display that combines the functions of the operation unit224 and the display225.

[0021] The document reading unit 226 includes an image sensor and an exposure lamp, and performs the process of reading a document when using the copy or scan function. Specifically, the document reading unit 226 captures an image with a CMOS image sensor while illuminating the print sheet placed by the user with the exposure lamp. In this way, the document reading unit 226 reads the document data. The latent image unit 227 is a device that performs primary charging and laser exposure in order to irradiate the photosensitive drum with laser light in order to develop the toner image. In the latent image unit 227, primary charging is first performed to charge the surface of the photosensitive drum to a uniform negative potential. Next, a laser driver irradiates the photosensitive drum with laser light, adjusting the reflection angle with a polygon mirror. In this way, an electrostatic latent image is formed.

[0022] The image-forming unit 228 is a device for transferring toner to paper and consists of a developing unit, a transfer unit, and a toner supply unit, etc., which transfer toner from the photosensitive drum to the paper. In the developing unit, negatively charged toner from the developing cylinder is attached to the electrostatic latent image on the surface of the photosensitive drum, making it a visible image. In the transfer unit, primary transfer is performed by applying a positive potential to the primary transfer roller to transfer the toner on the surface of the photosensitive drum to the transfer belt, and secondary transfer is performed by applying a positive potential to the secondary transfer outer roller to transfer the toner on the transfer belt to the paper. The fixing unit 229 is a device for dissolving and fixing the toner on the paper to the paper using heat and pressure, and consists of a heating heater, a fixing belt, and a pressure belt, etc. The paper feeding and transport unit 230 is a device for feeding paper and controls the paper feeding and transport operations using rollers and various sensors.

[0023] Next, the configuration of the inserter 108 of the image forming apparatus 101 will be described. The inserter 108 consists of a communication interface 232, a CPU 233, a memory 234, and a paper feed control unit 235. Each of these components is connected to the others via a system bus 236.

[0024] The communication interface 232 includes a communication module and is connected to the printing device 107 via a communication cable 254. This connection enables the communication necessary for control. The CPU 233 performs various controls necessary for paper feeding according to the control program stored in the memory 234. The memory 234 is a storage device in which the control program is stored. The paper feed control unit 235 controls the paper feeding section of the inserter 108 and the feeding and transport of paper transported from the printing device 107, while controlling the rollers and sensors based on instructions from the CPU 233.

[0025] Next, the configuration of the inspection device 109 of the image forming apparatus 101 will be described. The inspection device 109 consists of a communication interface 237, a CPU 238, a memory 239, an imaging unit 240, a display unit 241, an operation unit 242, and an HDD 255. Each component is connected to the others via a system bus 243. The communication interface 237 consists of a communication module and is connected to the printing apparatus 107 via a communication cable 254. Communication necessary for control is performed through this connection. The CPU 238 performs various controls necessary for inspection according to the control program stored in the memory 239. The memory 239 is a storage device in which the control program is stored. The imaging unit 240 consists of an imaging module and, based on instructions from the CPU 238, images the transported printed sheet.

[0026] The CPU 238 saves the image containing the inspection reference sheet as the correct image in memory 239. Furthermore, the CPU 238 compares the inspection image containing the printed sheet to be inspected (an example of the "sheet to be inspected") with the correct image saved in memory 239 to determine whether the inspection image is normal or not. The image containing the inspection reference sheet and the inspection image are both captured in advance by the imaging unit 240.

[0027] The display unit 241 includes a display and shows inspection results, setting items, etc. The operation unit 242 includes buttons that can be pressed by the user, for example, and receives operation instructions such as changing settings of the inspection device 109 or registering correct images. The inspection device 109 may also be provided with a touch panel display that combines the functions of the operation unit 242 and the display unit 241. The HDD 255 stores various setting information and images necessary for inspection. The stored setting information and images can be reused.

[0028] Next, the configuration of the large-capacity stacker 110 of the image forming apparatus 101 will be described. The large-capacity stacker 110 consists of a communication interface 244, a CPU 245, a memory 246, and a paper ejection control unit 247. Each component is connected to the others via a system bus 248. The communication interface 244 consists of a communication module and is connected to the printing apparatus 107 via a communication cable 254. Communication necessary for control is performed through this connection. The CPU 245 performs various controls necessary for paper ejection according to the control program stored in the memory 246. The memory 246 is a storage device in which the control program is stored. The paper ejection control unit 247 controls the transport of the conveyed printed sheets to the stack tray, escape tray, or subsequent finisher 111 based on instructions from the CPU 245.

[0029] Next, the configuration of the finisher 111 of the image forming apparatus 101 will be described. The finisher 111 consists of a communication I / F 249, a CPU 250, a memory 251, a paper output control unit 252, and a finishing processing unit 253. Each of these components is connected to the others via a system bus 256.

[0030] The communication interface 249 includes a communication module and is connected to the printing device 107 via a communication cable 254. This connection enables the communication necessary for control. The CPU 250 performs various controls necessary for finishing and paper ejection according to the control program stored in the memory 251. The memory 251 is a storage device in which the control program is stored. The paper ejection control unit 252 controls the transport and ejection of printed sheets based on instructions from the CPU 250. The finishing processing unit 253 controls finishing processes such as stapling, punching, and saddle stitching based on instructions from the CPU 250.

[0031] Next, the configuration of the external controller 102 will be described. The external controller 102 consists of a CPU 208, memory 209, HDD 210, keyboard 211, display 212, LAN I / F 213, LAN I / F 214, and video I / F 215. These components are connected to each other through the system bus 216.

[0032] The CPU 208 receives print data from the client PC 103 based on programs and data stored on the HDD 210. The CPU 208 also comprehensively performs processes such as RIP processing (Raster Image Processor) and transmission of print data to the image forming apparatus 101. Furthermore, it can perform RIP processing for ground truth image data. Specifically, in RIP processing for ground truth image data, for example, it converts a resolution of 600 dpi to 300 dpi to generate an image, while in RIP processing for print data, it generates an image without reducing the resolution.

[0033] Memory 209 stores programs and data necessary for the CPU 208 to perform various processes and operates as a work area. HDD 210 stores programs and data necessary for operations such as printing. Keyboard 211 is a device for inputting operation instructions for the external controller 102. Display 212 displays information such as the applications executed by the external controller 102 as still images and video signals.

[0034] LANI / F213 is configured, for example, to include a NIC and is connected to the client PC 103 via the external LAN 104. Communication such as print instructions is performed through this connection. LANI / F214 is configured, for example, to include a NIC and is connected to the image forming apparatus 101 via the internal LAN 105, and communication such as print instructions is performed through this connection. The external controller 102 can exchange various data with the printing apparatus 107, inserter 108, inspection apparatus 109, large-capacity stacker 110, and finisher 111 via the internal LAN 105 and communication cable 254. Video I / F215 is configured to include a module for communicating video signals and is connected to the image forming apparatus 101 via video cable 106. Communication such as print data is performed through this connection.

[0035] Next, the configuration of client PC 103 will be described. Client PC 103 consists of a CPU 201, memory 202, HDD 203, keyboard 204, display 205, and LAN I / F 206. Each component is connected to the others via a system bus 207.

[0036] CPU201 creates print data and executes print commands based on document processing programs and other data stored in HDD203. CPU201 also comprehensively controls each device connected to the system bus. Memory202 stores programs and data necessary for CPU201 to perform various processes and functions as a work area. HDD203 stores programs and data necessary for operations such as printing.

[0037] The keyboard 204 is a device for inputting operation instructions for the PC 103. The display 205 displays information such as the applications running on the client PC 103 as still images and video signals. The LANI / F206 is configured, for example, to include a NIC and is connected to the external LAN 104. Communication such as print instructions and the receipt of RIP images is performed through this connection. In this embodiment, the CPU 238 of the inspection device 109 calculates the amount of toner to be applied during printing. The CPU 201 may then receive this toner amount from the printing device 107 or the external controller 102 via the communication cable 254.

[0038] In the above description, the external controller 102 and the image forming apparatus 101 are connected via the internal LAN 105 and the video cable 106, but the configuration is not limited to this as long as it allows for the transmission and reception of data necessary for printing. For example, the external controller 102 and the image forming apparatus 101 may be connected only by the video cable. Also, memory 202, memory 209, memory 223, memory 234, memory 239, memory 246, and memory 251 may be any storage device for holding data or programs. For example, memory may be replaced with volatile RAM, non-volatile ROM, an internal HDD, an external HDD, or a USB memory stick.

[0039] The internal structure of the image forming apparatus 101 will be explained using Figure 3. The printing apparatus 107 is a device that forms images to be printed on paper, and consists of a paper feed deck (301, 302), a paper transport path 303, developing stations 304-307, an intermediate transfer belt 308, and a display 225. The printing apparatus 107 also consists of a fixing unit 311, paper transport paths (312, 314, 315), a second fixing unit 313, a paper reversal path 316, and a double-sided transport path 317.

[0040] Each paper feed deck (301, 302) can accommodate various types of paper. Furthermore, each paper feed deck (301, 302) can separate only the top sheet of paper and transport it to the paper transport path 303. Developing stations 304-307 use colored toners Y, M, C, and K respectively to form a toner image for color image formation. The toner image formed here is first transferred to the intermediate transfer belt 308. The intermediate transfer belt 308 rotates clockwise in Figure 3. Then, at the secondary transfer position 309, the toner image is transferred to the paper transported from the paper transport path 303. The display 225 displays information for the printing status and settings of the image forming apparatus 101.

[0041] The fuser unit 311 fixes the toner image to the paper. The fuser unit 311 consists of a pressure roller and a heating roller, and fixes the toner image by melting and pressing the toner onto the paper as it passes between the rollers. After passing through the fuser unit 311, the paper is transported through the paper transport path 312 to the paper transport path 315. If further melting and pressing is required for fixing depending on the type of paper, the paper is transported to the second fuser unit 313 using the paper transport path 314 located above the paper transport path 312 after passing through the fuser unit 311. The paper, which has undergone additional melting and pressing by the second fuser unit 313, is then transported through the paper transport path 314 to the paper transport path 315. If the image formation mode is double-sided, the paper is transported to the paper inversion path 316, where it is inverted, and then transported to the double-sided transport path 317, where the second image is transferred at the secondary transfer position 309.

[0042] The inserter 108 includes an inserter tray 321, and the paper placed on the inserter tray 321 is joined to the transport path via the paper transport path 322. In this way, the inserter 108 can insert paper at any interval in a series of sheets of paper transported from the printing device 107 and transport them to a subsequent device.

[0043] The printed sheet that has passed through the inserter 108 is transported to the inspection device 109. The inspection device 109 consists of an imaging unit 240, a display unit 241, and a printed sheet transport path 333. The imaging units 240 are arranged to face each other. The imaging units 240 are sensors for reading the front and back surfaces of the printed sheet, respectively. When the printed sheet transported on the printed sheet transport path 333 reaches a predetermined position, the inspection device 109 can read an image of the printed sheet using the imaging units 240 and determine whether the read image is normal or not. The display unit 241 displays the inspection results performed by the inspection device 109.

[0044] More specifically, the inspection device 109 inspects the received printed sheet image according to pre-set inspection items. In the following description, the printed sheet image refers to the printed sheet portion from which the background has been removed by the CPU 238 of the inspection device 109. The inspection of the printed sheet image is performed by comparing the received printed sheet image with a pre-set correct image. Methods for comparing images include comparing pixel values ​​at each image position, comparing the position of objects by edge detection, or extracting character data by OCR (Optical Character Recognition). One of the inspection items is corner folds on the printed sheet. In addition, other inspection items may include, for example, misalignment of the print position, image color, image density, streaks or smudges, or missing prints.

[0045] The high-capacity stacker 110 comprises a stack tray 341, a paper output reversal unit 342, a print sheet transport path (344, 345, 347, 348), and an escape tray 346. The high-capacity stacker 110 is a stacker capable of loading a large volume of print sheets.

[0046] The stack tray 341 is a tray for stacking printed sheets. Printed sheets that have passed through the inspection device 109 are input to the large-capacity stacker 110 via the printed sheet transport path 344. The printed sheets are then loaded onto the stack tray 341 via the printed sheet transport path 345 from the printed sheet transport path 344. The escape tray 346 is an output tray used to discharge printed sheets that have been determined to be abnormal by the inspection device 109. Printed sheets output to the escape tray 346 are transported via the printed sheet transport path 347 from the printed sheet transport path 344. When transporting printed sheets to a post-processing device downstream of the large-capacity stacker 110, the printed sheets are transported via the printed sheet transport path 348.

[0047] The paper output reversal unit 342 is used when loading print sheets onto the stack tray 341. The paper output reversal unit 342 reverses the print sheets input to the stack tray 341 so that the orientation of the print sheets at the time of input is the same as the orientation of the print sheets at the time of output. On the other hand, when transporting the print paper loaded onto the stack tray 341 to the escape tray 346 or a subsequent post-processing device, the paper output reversal unit 342 does not reverse the print sheets and discharges them as they are without flipping them during loading.

[0048] The finisher 111 comprises two output trays (351, 352), print sheet transport paths (353, 354, 357), processing units (355, 356), and a saddle-stitch binding tray 358. The finisher 111 applies finishing processing to the transported print sheets according to the functions specified by the user. Specifically, the finisher 111 has finishing functions such as stapling (single-point or double-point stapling), punching (two-hole or three-hole punching), and saddle-stitch binding.

[0049] The finished printed sheet is output to the output tray 351 via the printed sheet transport path 353. However, finishing processes such as stapling cannot be performed on the printed sheet transport path 353. If finishing processes such as stapling are required, the printed sheet is output to the output tray 352 via the printed sheet transport path 354, where the finishing function specified by the user is executed in the processing unit 355. The output trays (351 and 352) can be raised and lowered. That is, the output tray 351 can be lowered, and the output tray 351 can be operated to load the printed sheet that has been finished in the processing unit 355 onto the output tray 351.

[0050] If saddle stitching is specified, the saddle stitching processing unit 356 staples the center of the printed sheet and then folds the printed sheet in half. The saddle-stitched printed sheet is then output to the saddle stitching tray 358 via the printed sheet transport path 357. The saddle stitching tray 358 is configured as a belt conveyor. The bundles of saddle-stitched books stacked on the saddle stitching tray 358 are transported to the left side in Figure 3.

[0051] <Overall flow of the inspection process> Next, using Figure 4, the overall flow from pre-inspection work to inspection execution in the inspection device 109 will be explained. This process is achieved by the CPU 238 of the inspection device 109 reading and executing a program stored in memory 239.

[0052] In S401, the CPU 238 registers a reference image (hereinafter also referred to as the ground truth image) used to determine whether the printed sheet to be inspected is normal or not. Specifically, the CPU 238 uses a simulated image generated using print data, or scanned data generated by scanning the reference sheet in advance using the inspection device 109, as the ground truth image. The print data is received in advance using the communication I / F 237. The CPU 238 also acquires the feature points of the reference sheet included in the ground truth image and saves them in the HDD 255 along with the ground truth image data. The feature points are, for example, the top 8 feature points detected by Harris corner detection. However, Harris corner detection is just one example of feature point extraction, and the method of acquiring feature points is not limited to this. The feature points of the reference sheet included in the ground truth image will hereafter also be referred to as the reference feature points.

[0053] In S402, the CPU 238 receives user input to the operation unit 242 and sets the inspection level for streaks and smudges, the inspection type, or detailed inspection settings such as the inspection area. These settings are performed using the inspection UI (see Figure 13) displayed on the panel of the operation unit 242. The various settings in Figure 13 will be described later. In S403, the CPU 238 performs the inspection. The CPU 238 compares the correct image generated in S401 with the scanned image of the printed sheet to be inspected and performs an inspection to determine whether or not there are corner folds in the printed sheet based on the inspection settings set in S402. The CPU 238 also performs inspections other than corner folds as set in the inspection settings. Then the process ends.

[0054] <Inspection Process> Figure 5 illustrates a flowchart showing the flow of the inspection process in the inspection device 109. This process is achieved when the CPU 238 of the inspection device 109 reads and executes a program stored in the memory 239.

[0055] In S501, the CPU 238 acquires the correct image, reference feature point information, and inspection settings as described in S401 and S402 above. In S502, the CPU 238 transitions to a scan standby state. The inputs accepted in the scan standby state are two types of external inputs: image scan and inspection end instruction. In S503, the CPU 238 determines whether or not there is an external input. If the CPU 238 determines that it has received an inspection end instruction, the process ends. On the other hand, if the CPU 238 determines that it has received an image scan input, the process proceeds to S504.

[0056] In S504, the imaging unit 240 is always ready to scan the print sheet when the CPU 238 is in scan standby mode. When the printing device 107 starts printing and the print sheet passes through the imaging unit 240, the imaging unit 240 scans the print sheet. The CPU 238 (an example of an "acquisition unit") saves the scanned image read by the imaging unit 240 to the memory 239.

[0057] In S505, the CPU 238 performs alignment of the printed sheet image stored in memory 239 with respect to the ground truth image. That is, the CPU 238 performs alignment using affine transformation with respect to the feature points of each image (details will be described later in Figure 6). If a print misalignment threshold 1309 (Figure 13(A)) is set, alignment is performed if the misalignment between the ground truth image and the printed sheet image is greater than or equal to that threshold. In S506, the CPU 238 uses the coordinates and values ​​calculated during the alignment in S505 to calculate values ​​related to corner folding of the printed sheet (details will be described later in Figure 7). Corner folding refers to a state where the top of the printed sheet is bent, and the bent part overlaps with the paper and does not appear in the scanned image.

[0058] In S507, the CPU 238 uses the value calculated in S506 to determine whether there are corner folds exceeding a specified value. If the CPU 238 determines that there are corner folds exceeding a specified value, it processes the scanned image as an abnormal image and proceeds to S511. In S511, the CPU 238 instructs the high-capacity stacker 110 to eject the printed sheet as an abnormal printed sheet into the escape tray 346.

[0059] On the other hand, if CPU238 determines in S507 that there are no corner bends exceeding a specified value, the process proceeds to S508. In S508, CPU238 detects abnormalities in the pattern contained in the scanned image. That is, CPU238 obtains the image difference between the correct image and the alignment image generated in S505. The image difference includes, for example, the difference caused by the corner bend overlapping with the pattern. The image difference also includes, for example, the difference caused by dirt (so-called spots or streaks) adhering to the pattern.

[0060] In S509, if the CPU 238 determines that the image difference detected in S508 is greater than or equal to a specified value, it determines the scanned image to be an abnormal image and proceeds to S511. In S511, the CPU 238 instructs the high-capacity stacker 110 to eject the printed sheet to the escape tray 346 as an abnormal printed sheet. On the other hand, if the CPU 238 determines that the image difference is less than a specified value, the process proceeds to S510. In S510, the CPU 238 instructs the high-capacity stacker 110 to eject the printed sheet, which has been determined to be a normal image, to the stack tray 341. As a result, the inspection device 109 can perform alignment, check for corner folds and determine whether there are any abnormalities, and then separate the paper output destinations.

[0061] <Alignment> Figure 6 will be used to explain the details of the alignment process in S505. This process is achieved when the CPU 238 of the inspection device 109 reads and executes a program stored in memory 239.

[0062] The alignment process can be broadly divided into four steps. First, in S601, the CPU 238 estimates the vertex coordinates of the printed sheet contained in the scanned image. The flow of vertex coordinate estimation will be described later in Figure 8. In S602, the CPU 238 extracts feature points of the pattern formed in the printed sheet portion of the scanned image. The printed sheet portion of the scanned image is the region enclosed by lines connecting the vertices estimated in S601. Harris corner detection, for example, is used for feature extraction.

[0063] In S603, CPU238 calculates the amount of print misalignment. Details of the print misalignment and the calculation flow will be described later in Figure 9. In S604, CPU238 performs an affine transformation using the parameters obtained by vertex estimation and feature point extraction to perform alignment. Details of S604 will be described later in Figure 10. Then the process ends.

[0064] <Vertex Estimation> The process of estimating vertex coordinates in S601 will be explained using Figure 8. This process is implemented by the CPU 238 of the inspection device 109 reading and executing a program stored in memory 239.

[0065] In S801, CPU238 performs binarization of the scanned image. For binarization, for example, Otsu's binarization method is used. In this case, if the binarization threshold is calculated using the entire scanned image, the binarization threshold will be easily affected by the image printed on the printed sheet. Therefore, CPU238 sets the top 100 pixels of the scanned image as an ROI (Region of Interest) and determines the binarization threshold.

[0066] In S802, CPU238 obtains the outline of the printed sheet contained in the scanned image. The outline of the printed sheet can be obtained, for example, by a chain code algorithm. In S803, CPU238 obtains the vertices of the printed sheet from the group of pixels representing the obtained outline of the printed sheet. The obtained vertices will be referred to as provisional vertices from here on. An example of provisional vertices when the printed sheet is rectangular will be described later in Figure 11.

[0067] In S804, CPU238 extracts a point cloud (hereinafter also referred to as the edge point cloud) from the scanned image, representing at least some of the edges of each side of the printed sheet, using a hypothetical vertex. The edge extraction flow will be described later in Figure 12. In S805, CPU238 uses the extracted point cloud to estimate the edges of the printed sheet, assuming that there are no corner bends. Here, for example, the least squares method is used for edge estimation. In S806, CPU238 calculates the intersection points of each estimated edge. The calculated intersection points are then estimated to be the coordinates of the vertices of the printed sheet. These vertices will hereafter be referred to as estimated vertices (an example of a "second vertex"). Then the process ends. Note that this process uses algorithms such as Otsu's binarization, chain coding, and the least squares method, but is not limited to these and other algorithms may be used.

[0068] <Calculation of positional displacement> Figure 9 will be used to explain the details of the calculation process for the positional displacement in S603. This process is implemented by the CPU 238 of the inspection device 109 reading and executing a program stored in memory 239.

[0069] In S901, CPU238 determines whether there are three or more reference feature points. Here, the number of reference feature points refers to the number of feature points included in the ground truth image obtained in S401. This is to prevent the counting of more reference feature points than the actual number of feature points if the printed sheet has dirt or scratches when extracting feature points from an image that includes a printed sheet. If CPU238 determines that there are three or more reference feature points, the process proceeds to S902. On the other hand, if not, the process terminates because it is determined that the number of reference feature points in the printed pattern is too small to obtain the amount of positional displacement.

[0070] In S902, CPU238 calculates an affine matrix to align the vertex coordinates of the printed sheet contained in the scanned image with the vertex coordinates of the sheet contained in the ground truth image. Then, using this affine matrix, it calculates the absolute value of the movement vector of the vertices of the printed sheet contained in the scanned image to align with the ground truth image, and this absolute value is taken as the amount of displacement at the vertices of the printed sheet.

[0071] In S903, CPU238 uses the reference feature points of the ground truth image obtained in S401 and the feature points of the printed sheet portion of the scanned image calculated in S602 to calculate an affine matrix that aligns the pattern contained in the scanned image with the reference pattern contained in the ground truth image. Then, CPU238 uses this affine matrix to calculate the movement vector of each vertex of the printed sheet to align with the ground truth image, and the absolute value of this movement vector is taken as the amount of displacement at the feature points.

[0072] In S904, CPU238 calculates the print misalignment. The print misalignment is defined as the difference at the corner vertices of the movement vectors calculated in S902 and S903, respectively. The amount of print misalignment of the printed sheet is determined by the largest norm among these differences at each vertex. Then, the process ends.

[0073] <Details of alignment> The alignment process in S604 will be explained using Figure 10. This process is achieved when the CPU 238 of the inspection device 109 reads and executes a program stored in memory 239.

[0074] In S1001, CPU238 determines whether the number of feature points is three or more, similar to S901. The feature points counted at this time are the reference feature points of the ground truth image obtained in S401. If CPU238 determines that the number of reference feature points is three or more, the process proceeds to S1002; otherwise, the process proceeds to S1003.

[0075] In S1002, CPU238 performs alignment of the scanned image with the ground truth image using an affine matrix. This affine matrix is ​​the same as the matrix calculated in S903. Then the process ends. On the other hand, in S1003, CPU238 performs alignment of the scanned image with the ground truth image using a different affine matrix. This affine matrix is ​​the same as the matrix calculated in S902. Then the process ends.

[0076] <Estimation of false vertices> The process of estimating a provisional vertex in S803 will be explained using Figures 11 and 23. This process is implemented by the CPU 238 of the inspection device 109 reading and executing a program stored in memory 239.

[0077] In S1101, the CPU 238 sets the origin to, for example, the center of the read image. Then, in S802, the CPU 238 determines the point furthest from the origin coordinates among the point cloud of the print sheet contour (the center point of each pixel containing the contour) to be the provisional vertex A'2301 of the print sheet (see Figure 23, an example of the "first point"). In S1102, the CPU 238 determines the point furthest from provisional vertex A'2301 among the print sheet contour point cloud to be the provisional vertex B'2302 (an example of the "second point").

[0078] In S1103, CPU238 finds the line segment L connecting temporary vertex A'2301 and temporary vertex B'2302. In S1104, CPU238 determines that the point furthest from line segment L in the print sheet contour point cloud is temporary vertex C'2303 (an example of a "third point"). In S1105, CPU238 determines that the point furthest from temporary vertex C' in the print sheet contour point cloud is temporary vertex D'2304 (an example of a "fourth point"). In S1106, CPU238 designates the top-left temporary vertex of temporary vertices A', B', C', and D' as temporary vertex A. Then, proceeding clockwise, it designates them as temporary vertex B, temporary vertex C, and temporary vertex D. The temporary vertices selected in this way (an example of a "first vertex") are not the correct vertex coordinates if a corner bend has occurred, but rather vertex coordinates formed at a different position from their original position due to the corner bend. The process then ends.

[0079] <Edge point cloud extraction> Figure 12 illustrates the details of the edge point cloud extraction process of the printed sheet in S804. This process is implemented by the CPU 238 of the inspection device 109 reading and executing a program stored in memory 239.

[0080] In S1201, CPU238 determines whether the edge extraction for each side of the printed sheet has been completed. If CPU238 determines that the edge extraction is complete, the process ends; otherwise, the process proceeds to S1202. In S1202, CPU238 selects two adjacent points from the provisional vertices A, B, C, and D obtained in S803.

[0081] In S1203, CPU238 connects the two selected points with a line segment, creating a temporary side. In S1204, CPU238 determines whether this temporary side is a horizontal or vertical side. If CPU238 determines it is a horizontal side, the process proceeds to S1205; if CPU238 determines it is a vertical side, the process proceeds to S1207.

[0082] In S1205, the CPU 238 extracts points P (an example of a "fifth point") in the X direction at StepX intervals (an example of a "first interval") along the hypothetical edge (see Figure 16). In S1206, the CPU 238 sets points PR and PR, which are shifted in the Y direction perpendicular to the X direction, for each of the extracted points P. Then, it assumes a rectangle 1601 with points PR and PR as vertices and having a small width in the X direction. The CPU 238 then calculates the edges within the rectangle 1601 at multiple points (an example of a "sixth point") that are arranged at predetermined intervals (an example of a "second interval") between points PR. For example, a second-order differential filter is used for edge calculation. After that, the process proceeds to S1209.

[0083] On the other hand, if the process proceeds to S1207, the CPU 238 performs the processing in S1205 along the vertical edges. Then, in S1208, it performs the same processing as in S1206. After that, the process proceeds to S1209. In S1209, the CPU 238 sets Pn as the point where the strongest brightness difference is detected among the edges calculated within the rectangle 1601. The CPU 238 performs this setting of Pn for each point P set as a temporary edge.

[0084] In S1210, the CPU 238 selects edges and corresponding points Pn where the edge strength of each point Pn is greater than or equal to the threshold Edge_th (an example of "greater than or equal to the first threshold"). This selection process prevents the selection of points other than the print sheet contour as edges. That is, as shown in Figure 16, it prevents the selection of Pn in rectangle 1601 (the left rectangle 1601 in Figure 16) that does not overlap with the print sheet contour. In S1211, the CPU 238 adopts all the points Pn thus selected as points that constitute the edge point cloud. Then, the process returns to S1201, and a similar process is performed for other edges. Note that the print sheet is positioned assuming a rectangle, but the shape of the print sheet is not limited to a rectangle. For example, the print sheet may be hexagonal, like an envelope.

[0085] <Determination of whether a corner is bent or not> Using Figure 7, the process for determining whether or not there is a corner bend in steps S506 and S507 in Figure 5 will be explained. This process is implemented by the CPU 238 of the inspection device 109 reading and executing a program stored in memory 239.

[0086] In S701, CPU238 determines whether the determination of whether or not there are corner bends has been completed for all vertices. If CPU238 determines that the determination of whether or not there are corner bends has been completed for all vertices, the process ends. Otherwise, the process proceeds to S702, where the amount of corner bend is calculated.

[0087] In S702, CPU238 (an example of a "setting unit") sets the estimated vertex coordinates 1501 (see Figure 15), calculated in S601, as the reference point of the scanned image. Then, CPU238 calculates the distance between the reference point and the actual printed sheet. More specifically, CPU238 calculates the outline 1502 of the printed sheet in the flowchart shown in Figure 8. It then determines the distance between the center coordinates of each pixel forming the outline 1502 and the estimated vertex coordinates 1501. Finally, CPU238 extracts pixels whose distance is less than the threshold d_thresh.

[0088] In S703, CPU238 calculates the distance e_dist between the extracted pixel and the print sheet edge 1503 (hereinafter also referred to as the estimated edge) estimated when calculating the estimated vertex. In S704, CPU238 counts the number of pixels for which the distance e_dist is less than the distance threshold e_thresh. Then, CPU238 determines whether the number is less than the threshold n_thresh. If CPU238 determines that the number of pixels is less than n_thresh, CPU238 (an example of the "determination unit") determines that the vertex has a corner bend, and the process ends. On the other hand, if CPU238 determines that the number of pixels is n_thresh or greater, CPU238 determines that the vertex does not have a corner bend, and returns to S701. Note that this process uses the estimated vertex of the print sheet as the reference point, but the reference point is not limited to this. The reference point may be, for example, the origin of the image after alignment.

[0089] The effectiveness of the above method for calculating the amount of corner bend will be explained using Figures 15 and 18(B). When S702 is executed, the center coordinates of each pixel forming contour 1502 that is less than d_thresh from the estimated vertex 1501 (reference point) are selected. Then, S703 and S704 are executed. With this method, even if the position of the estimated vertex contains errors, the corner bend is determined according to the distance between the center coordinates of each pixel forming the selected contour 1502 and the estimated edge. Therefore, the influence of errors in the position of the estimated vertex on the accuracy of corner bend determination can be reduced. The threshold used in the process shown in Figure 7 is appropriately set by the CPU 238 using the corner bend width W and height H that have been pre-entered into the UI screen in Figure 13(A).

[0090] <Test Settings> Figure 13 illustrates the UI (User Interface) screen used to configure inspection settings (an example of a "setting screen where the reference value for corner folds can be set"). The screen 1301 shown in Figure 13(A) represents the inspection UI window. The CPU 238 of the inspection device 109 displays this inspection UI on the display unit 241. The user configures the inspection areas for priority areas, standard areas, and simplified inspection areas on this screen. Priority areas are areas where abnormality inspection is performed with particular emphasis compared to other areas, such as a person's face. Standard areas are areas where inspection is performed in a standard manner, such as the back of a printed sheet. Simplified inspection areas are areas where a simple inspection is sufficient, such as the back of a printed sheet. Although three methods for setting inspection areas are illustrated as examples, the area names and types are not limited to these.

[0091] Screen 1302 shows the preview screen. Here, the correct image registered in S401 is displayed. The aforementioned inspection area settings are set by drawing the area on this correct image. Area 1303 indicates the inspection area. The type of line surrounding the area in the figure is the same type of line as the frames of the four types of inspection area settings. For example, the area enclosed by the dotted line in area 1303 is the same type of line as the frame 1308 that surrounds the "Setting of Priority Area".

[0092] Button 1304 indicates the rotation function. By clicking it, the user can rotate the preview screen by 90°. As shown in the diagram, there are two buttons, and the user can choose between clockwise rotation and counterclockwise rotation.

[0093] Button 1305 is for saving the inspection area settings. Button 1306 is for discarding the inspection area settings. Inspection level setting 1307 is a UI for setting the inspection level. Within the frame of inspection level setting 1307, the three inspection area settings—namely, the focus area setting, the standard area setting, and the simplified area setting—have dropdown menus that allow adjustment of the respective inspection levels. The thresholds, which are the inspection criteria for dirt and scratches, are set according to the set inspection level. The types of lines used in the frames 1308 for the three inspection area settings are all different. The preview screen 1302 displays frames corresponding to these three frames 1308. For example, the frame 1308 for the focus area setting is formed by a wavy line, and a similar area 1303 enclosed by a wavy line is displayed at the top of the preview screen. This area 1303 indicates the focus area in the inspection.

[0094] Furthermore, Figure 13 will be used to explain how to set up the inspection. When setting up the inspection, the user specifies the area to be inspected on the preview screen 1302. Here, the user touches the buttons (1311-1314) displayed on the screen. In other words, the user touches the button corresponding to the type of inspection they want to set up, and can select the area 1303 on the preview screen using drag-and-drop. The border of the area at this time will be the same as the selected inspection type.

[0095] Next, the user sets the inspection threshold for each selected area. That is, the user can set the threshold for each area in the inspection level setting 1307. For example, if the user selects a priority area by touching button 1311, they set the inspection threshold for that priority area by operating the pull-down menu displayed in the area enclosed by frame 1308. In Figure 13(A), for example, for dots (point-like abnormalities printed on the printed sheet) and streaks (linear abnormalities printed on the printed sheet), the user can select the inspection level by selecting the corresponding pull-down menu.

[0096] The print misalignment threshold 1309 indicates the acceptable amount of misalignment when misalignment occurs. Users can check for misalignment by entering the maximum acceptable value in millimeters within the box. The misalignment check is performed only if the checkbox displayed to the left of the misalignment is checked; otherwise, the misalignment check is not performed. The method for calculating the amount of misalignment follows the flowchart shown in Figure 9.

[0097] The value entered in Corner Fold Display 1310 (an example of a "Corner Fold Standard Value") indicates the amount of corner fold that is acceptable and will pass inspection if it occurs. Users can inspect for corner folds by entering the maximum acceptable width W and height H of the corner fold in millimeters within the frame. The corner fold inspection is performed only if the checkbox displayed to the left of the corner fold is checked; otherwise, the corner fold inspection is not performed. The method for calculating the presence or absence of a corner fold is as shown in the flowchart in Figure 7.

[0098] Furthermore, as shown in the corner bend display 1310, the user can input a value for the corner bend, and the preview screen 1302 (for example, the upper right) will display a corner bend of the size indicated by the corner bend reference value. Also, as shown in Figure 13(B), if the user changes the value of the corner bend reference value, the size of the corner bend displayed on the preview screen 1302 will also change. This allows the user to set the degree of corner bend that is considered NG while visually confirming it. The user then completes the inspection setting by touching button 1305. The CPU 238 then accepts this inspection setting and saves it to the HDD 255.

[0099] <Presentation of Results> Using Figure 14, the UI screen for the inspection results displayed on the display unit 241 by the CPU 238 of the inspection device 109 will be explained. This UI can be viewed both during and after the inspection. The inspection results window 1401 is displayed on the UI screen. The user can check the results of the printed sheet after the inspection is complete on the inspection results window 1401.

[0100] The inspection results window 1401 includes the inspection results list 1402. The inspection results list 1402 displays the results of inspections that have already been completed. Users can view the details of the inspection by clicking on the print sheet (hereinafter also referred to as the abnormal print sheet) that has been determined to be abnormal in the inspection results list 1402. The inspection results list 1402 also displays a list of the causes of the abnormality. For example, for five types of abnormalities—vertical streaks, horizontal streaks, dots, misalignment, and corner folds—a "○" mark is displayed if the inspection is OK, and a "×" mark is displayed if the inspection is NG. Users can recognize the details of the abnormality in the print sheet by these marks.

[0101] Furthermore, the inspection results window 1401 includes a scan image display screen 1403, which is an area where images of scanned print sheets are displayed. Images of scanned print sheets, including abnormal print sheets, are pre-saved on the HDD 255. When the user clicks on an abnormal print sheet in the inspection results list 1402 to view details, the CPU 238 switches the scan image display screen 1403 to the image of the corresponding abnormal print sheet.

[0102] Furthermore, the inspection results window 1401 includes inspection results 1404. Inspection results 1404 displays the reason for the NG (Not Good) status of the selected abnormal print sheet. This reason for NG matches the details displayed in the inspection results list 1402. Examples of NG reasons include vertical streaks, horizontal streaks, dots, misalignment, or corner folds. Note that misalignment refers to the misalignment of the entire image being formed on the print sheet. If multiple causes are detected, all causes will be displayed.

[0103] The inspection results window 1401 also includes an inspection progress indicator 1405. The inspection progress indicator 1405 shows the current inspection progress. For example, the denominator displays the number of sheets that have been inspected, and the numerator displays the number of sheets that were determined to be abnormal during inspection. The inspection results window 1401 also includes an exit button 1406 in the lower right corner of the screen. When the user touches this button, the window is closed. If the user touches the exit button 1406 during inspection, the inspection will be performed on all print sheets that have already started printing at that time, and then the inspection will end and the window will be closed.

[0104] <One aspect of action / effect> According to the image forming system 100, the user can input criteria for determining the presence or absence of corner folds into an inspection UI as shown in Figure 13(A). Therefore, the user can set corner fold detection criteria that are suitable for stacking printed sheets into a stacker. Furthermore, if a finishing process is specified in the print job, the user can set corner fold detection criteria that are suitable for that finishing process. In addition, the user can set the criteria according to the size of the image formed on the sheet, etc. Thus, corner fold detection can be performed in accordance with the user's intentions.

[0105] Furthermore, the image forming system 100 estimates the vertices of the printed sheet and uses these estimated vertices as reference points to calculate the distance to pixels representing the contour of the printed sheet, thereby checking for corner folds. This type of inspection improves the accuracy of corner fold detection.

[0106] Furthermore, according to the image forming system 100, even if it is determined in S507 that there are no corner folds exceeding a specified value, it still determines in S508 and S509 that there is an abnormality in the pattern of the printed sheet. Therefore, even if the corner folds are small, it is possible to determine that a printed sheet in which the corner folds overlap the pattern is abnormal. With such an image forming system 100, it is possible to perform processing that takes into account the user's intentions.

[0107] (First variation) The criteria for determining a corner bend may differ from those of the above embodiment. More specifically, the execution of S702 shown in Figure 7 in the above embodiment takes into account the horizontal length W and vertical length H of the estimated side shown in Figure 17(A). On the other hand, Figures 17(B), 17(C), and 18(A) schematically illustrate the criteria for determining a corner bend according to the first modified example. Figures 17(B) and 18(A) show an example in which the shape of the corner bend is assumed to be a triangle and the height of that triangle is used as the criterion. Figure 17(C) shows an example in which the horizontal length W and vertical length H of the estimated side are calculated, and the shape of the corner bend is assumed to be a triangle and the product of W and H, i.e., the area of ​​the corner bend, is used as the criterion.

[0108] Using Figures 18(A) and 19, we will explain the process for determining corner kinks using the height of the triangle shown in Figure 17(B). This process is implemented by the CPU 238 of the inspection device 109 reading and executing a program stored in memory 239. Furthermore, the following explanation will focus on the differences from Figure 7.

[0109] In the flow shown in Figure 19, the process in S703 is not executed. That is, after the process in S702 is executed, in S1901 the CPU 238 of the inspection device 109 counts the number of points where the distance between the estimated vertex and the center coordinates of each pixel forming the contour of the printed sheet is less than the threshold d_thresh. The CPU 238 then determines whether that number is less than the threshold n_thresh. If the CPU 238 determines that the number is less than the threshold n_thresh, the CPU 238 determines that there is a corner fold in the printed sheet and terminates the process. Otherwise, the process returns to S701.

[0110] <One aspect of action / effect> According to the corner bend detection method shown in Figures 17(B) and 18(A), the S703 process is not executed compared to the process shown in Figure 7, thus improving processing speed. Furthermore, according to the corner bend detection method shown in Figure 17(C), the corner bend can be detected by calculating the area of ​​the bent corner from the estimated side lengths (W and H) without executing the process shown in Figure 7. Therefore, this detection method can also improve processing speed.

[0111] (Second variation) According to the above embodiment, corner folds exceeding a certain size can be flagged as NG (Not Accepted) according to user input. However, for example, when printing on a printing sheet, patches may be placed on the edges. In such cases, the user needs to set the size of the corner fold while being aware of the positional relationship between the patch and the corner fold in order to flag corner folds that are expected to overlap with the patch as NG. Also, for example, depending on the type of paper, if there are corner folds exceeding a certain size in a stack, the thickness of the folded part will be thicker than other printing sheets without corner folds, making it impossible to stack the original number of sheets that can be stacked. Therefore, the user needs to set the acceptable size of the corner fold while being aware of the hardware specifications. However, this size varies depending on the type of paper and the hardware, so the user's burden increases. Therefore, a second modified example that can reduce the burden on the user when setting the amount of corner folds in these cases will be described.

[0112] Figure 20 shows a settings screen (an example of the "first screen") that is displayed when, for example, a patch is formed on the print sheet. Patch 2001 shown on preview screen 1302 indicates a patch to be formed (an example of "print settings"). This patch is depicted on the correct image, and the CPU 238 of the inspection device 109 obtains the position of this patch on the correct image when the correct image is registered. The CPU 238 also obtains information such as the paper type (an example of "print settings") and calculates the acceptable amount of corner bending. The upper limit of the acceptable amount of corner bending is set to, for example, 3 mm based on the position of the patch (an example of "settable range of reference value").

[0113] The width setting frame 2002 in the paper inspection settings section is a frame where the user can input the width of the corner fold. Since patch 2001 is located 3mm from the edge of the paper, the user must set the width to 3mm or less. Therefore, if the user enters a value greater than 3mm, CPU 238 displays warning 2003 prompting the user to readjust the corner fold width setting so that it does not exceed 3mm. Although not illustrated in Figure 20, CPU 238 may also display warning 2003 if a value larger than the acceptable corner fold width is entered, depending on the specifications of the paper tray.

[0114] <One aspect of action / effect> According to the second modified image forming system 100, the burden on the user is reduced when setting the amount of corner folding so that the corner folding and the patch do not overlap, or when setting the amount of corner folding due to hardware limitations.

[0115] (Third variation) According to the above embodiment, corner folds exceeding a certain size can be flagged as NG (failed) based on user input. However, if factors such as temperature and humidity in the execution environment or severe paper curling occur together, continuous corner folds may occur. In such cases, if printing continues, the amount of wasted paper will increase. Therefore, in the third modified example, if continuous corner folds are detected, printing is automatically stopped to prevent an increase in wasted paper.

[0116] More specifically, if the CPU 238 of the inspection device 109 detects consecutive corner folds, it notifies the printing device 107 of a continuous NG signal via the communication cable 254. The CPU 222 of the printing device 107 receives the continuous NG signal using the communication interface 217. The CPU 222 then immediately interrupts printing, and the paper output control unit 247 ejects the paper currently in the machine to the escape tray.

[0117] Furthermore, Figure 21 illustrates the UI screen displayed by the CPU 238 of the inspection device 109 when corner folds occur consecutively. When the CPU 238 of the inspection device 109 detects that corner folds have occurred a predetermined number of times, it displays a warning 2101 on the UI screen when printing is interrupted. The CPU 238 notifies the user of the interruption of printing by displaying such a warning 2101. Warning 2101 includes a message prompting the user to check the paper to eliminate the cause. Note that warning 2101 is just an example and may include changes to the decurler settings, fixing conditions, transport conditions, etc.

[0118] <One aspect of action / effect> According to the third modified image forming system 100, when corner folds occur consecutively, it is possible to prevent further consecutive corner folds from occurring and increasing the number of sheets of paper.

[0119] (Fourth variation) According to the above embodiment, corner folds exceeding a certain size can be flagged as NG (failed) based on user input, using pre-registered correct images. However, the printed sheet itself included in the scanned image used as the correct image may have corner folds. In such cases, the inspection may mistakenly identify areas that are not actually abnormal as abnormal (hereinafter referred to as over-detection). Therefore, the fourth modified example prevents the inclusion of corner folds in the correct image.

[0120] The process of registering the correct image using the scanned image will be explained using Figure 22. This process is achieved when the CPU 238 of the inspection device 109 reads and executes a program stored in memory 239.

[0121] In S2201, the CPU 238 uses the imaging unit 240 to scan a print sheet (an example of a "candidate sheet"). In S2202, the CPU 238 determines whether or not there are any corner folds in the scanned image. The determination of whether or not there are corner folds here may be made by executing the flow shown in Figure 7 in the above embodiment. If the CPU 238 determines that there are no corner folds, the process proceeds to S2203; otherwise, the process proceeds to S2204.

[0122] In S2203, the CPU 238 determines that the scanned image is a normal correct image and saves it to memory 239 as the correct image. Through this process, the correct image is registered. Meanwhile, in S2204, the CPU 238 instructs the imaging unit 240 to scan another print sheet again and returns to S2201.

[0123] <One aspect of action / effect> According to the fourth modified image forming system 100, when a scanned image is adopted as the ground truth image, it is possible to prevent the ground truth image from containing corner distortions.

[0124] (Other variations) The scanning of the printed sheet in S504 may be performed outside the inspection device 109 (for example, on the printing device 107). The inspection device 109 may then acquire the scanned image of the printed sheet from the external device. Furthermore, the method for estimating the vertices of the printed sheet is not limited to the method shown in Figure 8. For example, the printed sheet may be approximated as a rectangle, and the vertices of the rectangle may be derived. Also, the criteria for determining corner folds are not limited to the forms shown in Figures 17(A) to 17(C).

[0125] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0126] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention.

[0127] The disclosures herein include the following inspection apparatus, a method for controlling the inspection apparatus, and a program. [Item 1] An acquisition unit that acquires an image including the sheet to be inspected, A setting unit for setting a reference point that serves as the basis for corner folding of the sheet to be inspected, The system includes a determination unit that determines whether or not the corners of the sheet to be inspected are bent, based on the aforementioned reference point and the contour of the sheet to be inspected in the aforementioned image. Inspection device. [Item 2] The aforementioned setting unit is, The contour of the sheet to be inspected is obtained, From the aforementioned contour, obtain the first vertex of the sheet to be inspected, Using the first vertex, an edge point cloud representing the edges of the sheet to be inspected is extracted. Using the aforementioned edge point cloud, obtain the edges of the sheet to be inspected that are not bent at the corners. Using the aforementioned edge, obtain the second vertex of the inspection target sheet in a state where the corners are not bent. The second vertex is set as the reference point. The inspection device described in item 1. [Item 3] The aforementioned setting unit is, Among the point cloud representing the outline of the sheet to be inspected, the first vertex is defined as the first point furthest from the origin, the second point furthest from the first point, the third point furthest from the line segment connecting the first and second points, and the fourth point furthest from the third point. The inspection device described in item 2. [Item 4] The aforementioned setting unit is, Connect adjacent first vertices to create a line segment, Extract the fifth point at the first interval along the aforementioned line segment, At the fifth point, set a sixth point at a second interval in a direction perpendicular to the line segment, At the sixth point, edges are extracted from the image including the sheet to be inspected. From the extracted edges, select the edge with the strongest strength. The sixth point corresponding to the edge having an intensity of 1 or higher among the strongest edges is defined as a point that constitutes the edge point group. An inspection device as described in any one of items 2 through 4. [Item 5] The system further includes a display unit that shows a setting screen on which the reference value for the aforementioned angle bend can be set. An inspection device as described in any one of items 1 through 4. [Item 6] The display unit further displays a first screen showing the range of configurable reference values ​​according to the print settings of the sheet to be inspected. The inspection device described in item 5. [Item 7] The display unit further displays a second screen indicating that the corner bend has occurred a predetermined number of times on the sheet being inspected. The inspection device described in item 5 or 6. [Item 8] The determination unit determines whether or not the sheet to be inspected has a corner bend, using the position of the corner bend of the corner bend of the sheet to be inspected before the bend and the outline of the sheet to be inspected. An inspection device as described in any one of items 1 through 7. [Item 9] The determination unit determines whether or not the sheet to be inspected has the corner bend using the distance between the reference point and the contour of the sheet to be inspected. An inspection device as described in any one of items 1 through 7. [Item 10] The determination unit determines whether or not the sheet to be inspected has a corner bend using the area formed by the position of the corner bend of the corner bend of the sheet to be inspected and the contour of the sheet to be inspected. An inspection device as described in any one of items 1 through 7. [Item 11] The determination unit further determines whether or not there is an abnormality in the sheet to be inspected using the difference between the image including the sheet to be inspected and the image including the reference sheet. An inspection device as described in any one of items 1 through 10. [Item 12] It also includes a reader unit to read the sheet, The reading unit reads the candidate sheet of the reference sheet, The determination unit further, The reading unit determines whether or not there are corner folds in the candidate sheet it has read, If it is determined that there are no corner folds in the candidate sheet, the candidate sheet is set as the reference sheet. If it is determined that the candidate sheet has a corner fold, the reading unit is instructed to read another candidate sheet. The inspection device described in item 11. [Item 13] The determination unit, before determining whether the corner of the sheet to be inspected is bent, The characteristic points of the pattern formed on the sheet to be inspected are detected, Using the detected feature points and the feature points of the pattern formed on the reference sheet, the inspection target sheet is aligned with the reference sheet. The inspection device described in item 11 or 12. [Item 14] A method for controlling an inspection device, The acquisition unit performs an acquisition process that acquires an image including the sheet to be inspected, The setting unit includes a setting step of setting a reference point that serves as the basis for corner folding of the sheet to be inspected, The determination unit includes a determination step of determining whether or not the corners of the sheet to be inspected are bent, based on the reference point and the contour of the sheet to be inspected in the image. A method for controlling an inspection device. [Item 15] A program for causing a computer to execute each step in a control method for an inspection device, wherein the control method is: The acquisition unit performs an acquisition process that acquires an image including the sheet to be inspected, The setting unit includes a setting step of setting a reference point that serves as the basis for corner folding of the sheet to be inspected, The determination unit includes a determination step of determining whether or not the corners of the sheet to be inspected are bent, based on the reference point and the contour of the sheet to be inspected in the image. program. [Explanation of Symbols]

[0128] 100: Image forming system, 101: Image forming apparatus, 102: External controller, 103: Client PC, 107: Printing device, 108: Inserter, 109: Inspection device, 110: High-capacity stacker, 111: Finisher, 238: CPU, 239: Memory, 240: Imaging unit

Claims

1. An acquisition unit that acquires an image including the sheet to be inspected, A setting unit for setting a reference point that serves as the basis for corner folding of the sheet to be inspected, The system includes a determination unit that determines whether or not the corners of the sheet to be inspected are bent, based on the aforementioned reference point and the contour of the sheet to be inspected in the aforementioned image. Inspection device.

2. The setting unit is, The contour of the sheet to be inspected is obtained, From the aforementioned contour, obtain the first vertex of the sheet to be inspected, Using the first vertex, an edge point cloud representing the edges of the sheet to be inspected is extracted. Using the aforementioned edge point cloud, obtain the edges of the sheet to be inspected that are not bent at the corners. Using the aforementioned edge, obtain the second vertex of the inspection target sheet in a state where the corners are not bent. The second vertex is set as the reference point. The inspection apparatus according to claim 1.

3. The setting unit is, Among the point cloud representing the outline of the sheet to be inspected, the first vertex is defined as the first point furthest from the origin, the second point furthest from the first point, the third point furthest from the line segment connecting the first and second points, and the fourth point furthest from the third point. The inspection apparatus according to claim 2.

4. The setting unit is, Connect adjacent first vertices to create a line segment, Extract a fifth point at a first interval along the aforementioned line segment, At the fifth point, set a sixth point at a second interval in a direction perpendicular to the line segment, At the sixth point mentioned above, edges are extracted from the image including the sheet to be inspected. From the extracted edges, select the edge with the strongest strength. The sixth point corresponding to the edge having an intensity of 1 or higher among the strongest edges is defined as a point that constitutes the edge point group. The inspection apparatus according to claim 2.

5. The system further includes a display unit that shows a setting screen on which the reference value for the aforementioned angle bend can be set. The inspection apparatus according to claim 1.

6. The display unit further displays a first screen showing the range of configurable reference values ​​according to the print settings of the sheet to be inspected. The inspection apparatus according to claim 5.

7. The display unit further displays a second screen indicating that the corner bend has occurred a predetermined number of times on the sheet being inspected. The inspection apparatus according to claim 5.

8. The determination unit determines whether or not the sheet to be inspected has a corner bend, using the position of the corner bend of the corner bend of the sheet to be inspected before the bend and the contour of the sheet to be inspected. The inspection apparatus according to claim 1.

9. The determination unit determines whether or not the sheet to be inspected has the corner bend using the distance between the reference point and the contour of the sheet to be inspected. The inspection apparatus according to claim 1.

10. The determination unit determines whether or not the sheet to be inspected has a corner bend using the area formed by the position of the corner bend of the corner bend of the sheet to be inspected and the contour of the sheet to be inspected. The inspection apparatus according to claim 1.

11. The determination unit further determines whether or not there is an abnormality in the sheet to be inspected using the difference between the image including the sheet to be inspected and the image including the reference sheet. An inspection apparatus according to any one of claims 1 to 10.

12. It also includes a reader unit to read the sheet, The reading unit reads the candidate sheet of the reference sheet, The determination unit further, The reading unit determines whether or not there are corner folds in the candidate sheet it has read, If it is determined that there are no corner folds in the candidate sheet, the candidate sheet is set as the reference sheet. If it is determined that the candidate sheet has a corner fold, the reading unit is instructed to read another candidate sheet. The inspection apparatus according to claim 11.

13. The determination unit, before determining whether the corner of the sheet to be inspected is bent, The characteristic points of the pattern formed on the sheet to be inspected are detected, Using the detected feature points and the feature points of the pattern formed on the reference sheet, the inspection target sheet is aligned with the reference sheet. The inspection apparatus according to claim 11.

14. A method for controlling an inspection device, The acquisition unit performs an acquisition process that acquires an image including the sheet to be inspected, The setting unit includes a setting step of setting a reference point that serves as the basis for corner folding of the sheet to be inspected, The determination unit includes a determination step of determining whether or not the corners of the sheet to be inspected are bent, based on the reference point and the contour of the sheet to be inspected in the image. A method for controlling an inspection device.

15. A program for causing a computer to execute each step in a control method for an inspection device, wherein the control method is: The acquisition unit performs an acquisition process that acquires an image including the sheet to be inspected, The setting unit includes a setting step of setting a reference point that serves as the basis for corner folding of the sheet to be inspected, The determination unit includes a determination step of determining whether or not the corners of the sheet to be inspected are bent, based on the reference point and the contour of the sheet to be inspected in the image. program.

Images