Inspection device, method for controlling inspection device, program, and inspection system

The inspection device automates the association of character strings and code images in bitmap format images by performing OCR and decoding, addressing the manual effort challenge and enhancing inspection efficiency.

JP2024162714AInactive Publication Date: 2024-11-21CANON KK
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023078538
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-11
Publication Date
2024-11-21
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing inspection systems struggle with associating character strings and code images in bitmap format images lacking coordinate information with their correct counterparts, necessitating manual effort.

Method used

An inspection device that performs OCR and code image decoding to extract character strings and coordinates, linking them to correct character strings, and sets inspection areas based on these coordinates, facilitating automated association.

Benefits of technology

Enables easy association of character strings and code images with their correct counterparts even in bitmap format images, reducing manual effort and enhancing inspection efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024162714000001_ABST
    Figure 2024162714000001_ABST
Patent Text Reader

Abstract

To facilitate settings of inspection areas on a printed sheet that are to be inspected.SOLUTION: An inspection device performs processing on blocks extracted from a reference image used for settings of inspection according to the types of the blocks, and thereby acquires a character string and the coordinates indicating the position of the character string for every block. The inspection device compares the character string acquired from the reference image with a correct character string to determine the character string acquired from the reference image corresponding to the correct character string, and links, to the correct character string, the coordinates indicating the position of the character string determined to correspond to the correct character string. The inspection device performs settings related to inspection areas on a printed sheet that are to be inspected on the basis of the coordinates linked to the correct character string.SELECTED DRAWING: Figure 5
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a technique for inspecting the print quality of a printed matter. [Background technology]

[0002] There is known an inspection system that inspects a printed sheet output from a printing device. The inspection includes a pattern inspection to check for stains on the pattern, and a data inspection to determine whether a character string or a character string represented by a code image is correct. When a plurality of character strings or code images to be subjected to the data inspection are present in the printed sheet, it is necessary to associate the character strings or code images in a reference image that serves as the basis for the inspection with the correct character strings. Conventionally, this association work has been performed manually by the user, which places a large burden on the user. In this regard, Patent Document 1 discloses a technology that makes it easy to perform the inspection even if the arrangement of the inspection object is changed by associating the coordinates embedded in a vector image as a reference image with the inspection object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-024223 Summary of the Invention [Problem to be solved by the invention]

[0004] The above Patent Document 1 is based on the premise that the data of the reference image is an image in vector format, and cannot be applied to bitmap image data that does not have coordinate information for each object in the data. In view of the above problem, the present disclosure aims to easily perform the task of associating character strings and code images in a reference image with their correct character strings, even if the reference image is image data in a format that does not have coordinate information for each object. [Means for solving the problem]

[0005] The inspection device according to the present disclosure is an inspection device for inspecting printed sheets, and is characterized in that it comprises: an acquisition means for acquiring a character string and coordinates indicating its position for each block extracted from a reference image used in inspection settings by performing processing according to the type of each block; a linking means for determining which character string acquired from the reference image corresponds to a correct string by comparing the character string acquired from the reference image with a correct string, and linking the coordinates indicating the position of the character string determined to correspond to the correct string with the correct string; and an inspection setting means for setting an inspection area to be inspected on the printed sheet based on the coordinates linked to the correct string by the linking means. Effect of the Invention

[0006] According to the technology of the present disclosure, even if the reference image is image data that does not have coordinate information for each object, it is easy to associate character strings or code images in the reference image with their respective correct character strings. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic diagram of an inspection system. [Diagram 2] FIG. 2 is a block diagram showing the hardware configuration of each device in the inspection system. [Diagram 3] FIG. 2 is a diagram showing the internal configuration of an image forming apparatus, an inspection unit, and a large-capacity stacker. [Figure 4] 11 is a flowchart showing an overview of an inspection process. [Diagram 5] 11 is a flowchart showing a process of linking a character string in a reference image with a correct character string. [Figure 6] 4 is a flowchart showing OCR processing for an image and decoding processing for a code image. [Figure 7] 13 is a diagram showing a correct character string and output coordinates based on the OCR result and the decoded result of the code image. [Figure 8] 13A and 13B are diagrams showing inspection setting screens for pattern inspection, character string inspection, and code image inspection. [Figure 9] 13 is a flowchart of an examination setting process. [Figure 10] 10 is a flowchart showing details of an inspection process. [Figure 11] 11 is a flowchart showing a process of linking a character string in a reference image with a correct character string. [Figure 12] 13 is a diagram showing a correct character string and output coordinates based on the OCR result and the decoded result of the code image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that the following embodiments do not limit the present disclosure, and not all combinations of features described in the embodiments are necessarily essential to the solution of the invention. In addition, each process (step) in the flow chart is indicated by a reference number beginning with "S".

[0009] [First embodiment] 1 is a schematic diagram of an inspection system according to the present embodiment. Note that, although the image forming apparatus according to the present embodiment will be described using an electrophotographic image forming apparatus, the image forming apparatus according to the present embodiment may be an image forming apparatus of a different type, such as an inkjet type or an offset type.

[0010] The image forming apparatus 101 is connected to an information processing apparatus 109 via a cable 112. The information processing apparatus 109 is connected to a client computer 110 and an inspection apparatus 108 that inspects print quality via a network 113. The image forming apparatus 101 includes a UI panel 102, a paper feed deck 103, and a paper feed deck 104. The image forming apparatus 101 further includes an option deck 105 consisting of three paper feed decks, and the UI panel 102. The UI panel 102 is a user interface that displays various information of the image forming apparatus 101 and accepts operations by a user, for example, with a capacitive touch panel. The image forming apparatus 101 includes an inspection unit 106 and a large-capacity stacker 107. The inspection unit is connected to the inspection apparatus 108 via a cable 114. The large-capacity stacker 107 includes a main tray and a top tray, and the main tray can hold several thousand sheets.

[0011] The client computer 110 generates a print job, and the generated job is transmitted to the information processing device 109 via the network 113 and managed by the information processing device 109. The information processing device 109 transmits the print job to the image forming device 101 via the cable 112. The image forming device 101 receives the print job and executes printing processing on the sheet. The print job may be generated in the information processing device 109. The information processing device 109 may transmit the print job to the image forming device 101 via the cable 112, and the print job may be managed by the image forming device 101. The client computer 110, the information processing device 109, and the inspection device 108 may be connected via the cable 112 and communicate with the image forming device 101. That is, the connection form of the image forming device 101, the information processing device 109, and the client computer 110 shown in this embodiment is an example, and the present embodiment may be implemented in various connection forms other than the connection form shown in this embodiment.

[0012] 2 is a block diagram showing the hardware configuration of each device in the inspection system according to this embodiment. A CPU (Central Processing Unit) 201 controls and performs calculations on each subunit in the image forming apparatus 101 via a system bus 212. The CPU 201 executes a program stored in a storage unit 205 and loaded into a RAM (Random Access Memory) 202. The RAM 202 is a type of general volatile storage device that can be directly accessed by the CPU 201, and is used as a work area for the CPU 201 or other temporary data storage area. The storage unit 205 stores the programs executed by the CPU 201, and also functions as a temporary storage area and work memory when the image forming apparatus 101 is in operation.

[0013] The engine I / F 209 communicates with the printer engine 210 and controls the printer engine 210. The paper feed deck I / F 204 communicates with the paper feed deck 211 and controls the paper feed deck 211. The paper feed deck 211 is a general term for the paper feed deck 103, the paper feed deck 104, and the option deck 105 as a hardware configuration. The network interface (hereinafter referred to as "NW I / F") 207 is connected to the NW I / F 238 of the information processing device 109 via a cable 213, and communicates between the information processing device 109 and the image forming device 101. In this embodiment, the interface of the image forming device 101 connected to the system bus 212 and the interface of the information processing device 109 connected to the system bus 239 are directly connected. The connection method is not limited to this, and the information processing device 109 and the image forming device 101 may be connected, for example, via a network. The video I / F 206 is connected to the video I / F 233 via a video cable 241, and communicates image data between the information processing device 109 and the image forming device 101. The connection interface between the information processing device 109 and the image forming device 101 may be a format that integrates the functions of the NW I / F 238 and the video I / F 233. The connection interface between the information processing device 109 and the image forming device 101 may be a format that integrates the functions of the NW I / F 207 and the video I / F 206. The accessory I / F 208 is connected to the accessory I / F 214 of the inspection unit 106 and the accessory I / F 220 of the large-capacity stacker 107 via a cable 225. That is, the image forming device 101 communicates with the inspection unit 106 and the large-capacity stacker 107 via the accessory I / Fs 208, 214, and 220.

[0014] The CPU 216 of the inspection unit 106 controls and performs calculations for each subunit in the inspection unit 106 via a system bus 219. The CPU 216 executes a program stored in a storage unit 247 and loaded into a RAM 217. The RAM 217 is a type of general volatile storage device that can be directly accessed by the CPU 216, and is used as a work area or other temporary data storage area for the CPU 216. The storage unit 247 stores programs executed by the CPU 216, and also functions as a temporary storage area and a work memory during operation of the inspection unit 106. The inspection device I / F 215 is connected to the inspection unit I / F 231 via a cable 249. That is, the inspection unit 106 communicates with the inspection device 108 via the inspection device I / F 215 and the inspection unit I / F 231. The photographing unit 218 has a photographing function equipped with, for example, a contact image sensor (hereinafter referred to as "CIS"), photographs the printed sheet passing through the inspection unit 106, and transmits the photographed inspection image to the inspection device 108 via the inspection device I / F 215. Note that the CIS is an example of a sensor, and does not limit the photographing method of the photographing unit 218. The sensor may be another type of sensor, such as a CCD image sensor.

[0015] A CPU 220 of the large capacity stacker 107 controls and performs calculations on each subunit in the large capacity stacker 107 via a system bus 224. The CPU 220 executes programs stored in a storage unit 248 and loaded into a RAM 222. The RAM 222 is a type of general volatile storage device that can be directly accessed by the CPU 221, and is used as a work area for the CPU 221 or other temporary data storage area. The storage unit 248 stores programs executed by the CPU 220, and also functions as a temporary storage area and work memory when the large capacity stacker 107 is in operation. A paper discharge unit 223 discharges paper to the main tray and the top tray, and monitors and controls the sheet loading status on each of the main tray and the top tray.

[0016] The CPU 226 of the inspection device 108 controls and performs calculations on each subunit in the inspection device 108 via the system bus 230. The CPU 226 executes a program stored in the storage unit 228 and loaded into the RAM 227. The RAM 227 is a type of general volatile storage device that can be directly accessed by the CPU 226, and is used as a work area or other temporary data storage area for the CPU 226. The storage unit 228 stores the program executed by the CPU 226, and also functions as a temporary storage area and work memory during operation of the inspection device 108. The PDL analysis unit 229 reads PDL data, such as PDF, PostScript, or PCL, received from the client computer 110 and the information processing device 109, and performs interpretation processing. The display control unit 245 is, for example, a touch panel connected to the inspection device 108. The display control unit 245 accepts user input to the inspection device 108, and displays the status of the inspection device 108.

[0017] The CPU 234 of the information processing device 109 controls and performs calculations on each subunit in the information processing device 109 via a system bus 239. The CPU 234 executes a program stored in the storage unit 236 and loaded into the RAM 235. The RAM 235 is a type of general volatile storage device that can be directly accessed by the CPU 234, and is used as a work area or other temporary data storage area for the CPU 234. The storage unit 236 stores programs executed by the CPU 234, and also functions as a temporary storage area and a work memory during operation of the information processing device 109. The NW I / F 237 is connected to the NW I / F 232 and the NW I / F 240 via a network. The information processing device 109 communicates with the inspection device 108 via the NW I / F 237 and the NW I / F 232. The information processing device 109 also communicates with the client computer 110 via the NW I / F 237 and the NW I / F 240.

[0018] The CPU 243 of the client computer 110 controls and performs calculations for each subunit in the client computer 110 via a system bus 246. The CPU 243 executes programs stored in the storage unit 244 and loaded into the RAM 242. The RAM 242 is a type of general volatile storage device that can be directly accessed by the CPU 243, and is used as a work area or other temporary data storage area for the CPU 243. The storage unit 244 stores programs executed by the CPU 243, and also functions as a temporary storage area and work memory when the client computer 110 is operating.

[0019] FIG. 3 is a diagram showing the internal configuration of the image forming apparatus 101, the inspection unit 106, and the large-capacity stacker 107. The image forming apparatus 101 accepts user input via the UI panel 102, and displays printing and device status on the UI panel 102. The paper feed deck 103 and the paper feed deck 104 can store various sheets. The paper feed deck 103 or the paper feed deck 104 can separate only the topmost sheet of the stored sheets and transport it to a sheet transport path 305. The developing stations 301, 302, 303, and 304 form toner images using color toners of Y, M, C, and K, respectively, to form a color image. The toner images formed in the developing stations 301, 302, 303, and 304 are primarily transferred to an intermediate transfer belt 306. The intermediate transfer belt 306 rotates clockwise in FIG. 3, and the toner image is transferred to a sheet transported from the sheet transport path 305 at a secondary transfer position 307. The fixing unit 308 includes a pressure roller and a heating roller, and the sheet passes between the rollers. The toner is melted and pressed onto the sheet, thereby fixing the toner image to the sheet. The sheet that has passed the fixing unit 308 is conveyed to 312 through a sheet conveying path 309. If the type of sheet requires further melting and pressing of the toner for fixing, the sheet is conveyed to a second fixing unit 310 using the upper sheet conveying path after passing through the fixing unit 308. After additional melting and pressing are performed in the second fixing unit 310, the sheet is conveyed to an end 312 through a sheet conveying path 311. If the image forming mode is double-sided, the sheet is conveyed to a sheet inversion path 313. The sheet is inverted in the sheet inversion path 313 and conveyed to a double-sided conveying path 314, where the image of the second side is transferred to a secondary transfer position 307.

[0020] In the inspection unit 106, the CIS 315 and the CIS 316 are arranged facing each other. The CIS 315 is a sensor for reading the upper surface of the printed sheet, and the CIS 316 is a sensor for reading the lower surface of the printed sheet. The inspection unit 106 scans the printed sheet using the CIS 315 and the CIS 316 when the sheet transported from the end portion 312 of the image forming apparatus 101 to the sheet transport path 317 reaches a predetermined position. The scanned image data is transmitted to the inspection device 108 via the inspection device I / F 215 and the inspection unit I / F 231. The CPU 226 of the inspection device 108 judges whether or not there is a printing defect in the received image, and notifies the inspection unit 106 of the judgment result again via the inspection unit I / F 231 and the inspection device I / F 215. The CPU 216 of the inspection unit 106 notifies the received judgment result to the large-capacity stacker 107 via the accessory I / F 214 and the accessory I / F 220.

[0021] The large-capacity stacker 107 can stack a large amount of printed sheets. The large-capacity stacker 107 has a main tray 324 as a tray for stacking printed sheets. The printed sheets that have passed through the inspection unit 106 pass through a sheet transport path 317 and enter the large-capacity stacker 107. The printed sheets are stacked on the main tray 324 via a sheet transport path 319 and a sheet transport path 322. The large-capacity stacker 107 also has a top tray 320 as a paper discharge tray. The CPU 221 of the large-capacity stacker 107 discharges printed sheets on which a printing defect has been detected by the inspection device 108 to the top tray 320. When discharging printed sheets to the top tray 320, the printed sheets are transported from the sheet transport path 319 to the top tray 320 via a sheet transport path 321. The inverting unit 323 is a unit for inverting printed sheets. This inverting unit 323 is used when stacking printed sheets on the main tray 324. When stacking on main tray 324, the printed sheet is inverted once by inversion unit 323 so that the orientation of the sheet entering large-capacity stacker 107 is the same as the orientation of the sheet when stacked on main tray 324. When transporting the printed sheet to top tray 320, the printed sheet is discharged to top tray 320 without flipping, so the inversion operation of the printed sheet is not performed by inversion unit 323.

[0022] The main processing of this embodiment will be described below with reference to a flowchart and an example of a UI screen displayed on the UI panel 102. There are two types of inspection: pattern inspection, which checks for stains on the pattern, and data inspection, which compares a character string obtained by decoding a code image (barcode or two-dimensional code) with a correct character string written in a correct file to check for the presence or absence of data defects. Since this disclosure mainly deals with processing related to data inspection, the following description will focus on data inspection.

[0023] The program of the image forming apparatus 101 according to this flow is stored in the storage unit 205 of the image forming apparatus 101, read into the RAM 202, and executed by the CPU 201. The program of the inspection apparatus 108 according to this flow is stored in the storage unit 228 of the inspection apparatus 108, read into the RAM 227, and executed by the CPU 226. The program of the information processing apparatus 109 according to this flow is stored in the storage unit 236 of the information processing apparatus 109, read into the RAM 235, and executed by the CPU 234. The program of the client computer 110 according to this flow is stored in the storage unit 244 of the client computer 110, read into the RAM 242, and executed by the CPU 243.

[0024] The overall flow from the start of inspection to the execution of inspection will be described with reference to the flowchart in Fig. 4. In S401, a reference image to be used in the printing process and the pattern inspection process as the reference for the inspection is registered in the inspection device 108 by the user. Specifically, the inspection device 108 starts the process of reading the reference image, and the client computer 110 executes a print job for registering the reference image. Based on the print job, the inspection unit 106 reads the printed sheet generated in the image forming device 101, and the inspection unit 106 transmits the read image data to the inspection device 108 as a reference image. The inspection device 108 reads the reference image transmitted from the inspection unit 106 and registers the reference image. When the reference image is registered in the inspection device 108, the process proceeds to S402.

[0025] In S402, a correct character string for a matching test is registered in the inspection device 108 by the user. The correct character string for a matching test is data stored in a reference file (correct file) for data inspection that is matched when performing a data inspection. The reference file is a file that the user should prepare in advance. The reference file lists correct character strings for string inspection and correct character strings encoded by code images. When performing a data inspection, the CPU 226 of the inspection device 108 compares the OCR result of the character string image in the character string inspection area and the character string resulting from decoding the code image with the correct character strings listed in the reference file. When the correct character string for a matching test is registered in the inspection device 108, processing proceeds to S403.

[0026] In S403, the CPU 226 of the inspection device 108 performs a process of linking the reference image registered in S401 with the correct answer character string registered in S402, and the process proceeds to S404. Details of this process will be described later with reference to the flowchart of FIG. 5. In S404, the user performs inspection setting, and the process proceeds to S405. The inspection setting refers to setting various inspection parameters, inspection area, inspection level, or serial number inspection using a UI screen. Details of the inspection setting will be described later. In S405, the inspection device 108 starts inspection, and a print job for inspection is executed in the client computer 110. Based on the print job, printing is executed in the image forming device 101. When the inspection unit 106 detects the conveyance of the printed sheet conveyed from the image forming device 101, the inspection unit 106 scans the printed sheet with the CIS 315 and CIS 316 to obtain an inspection image (scanned image). The inspection image is sent from the inspection unit 106 to the inspection device 108, and the inspection image is stored in the RAM 227 of the inspection device 108. The CPU 226 of the inspection device 108 outputs the inspection result by comparing the stored inspection image with the reference image. The inspection is performed based on the setting values ​​specified by the user in the inspection settings. When the inspection of the number of sheets specified in the print job is completed, the process of the flowchart shown in FIG. 4 ends.

[0027] Next, the process of linking the reference image with the correct character string (the process of S403) will be described with reference to the flowcharts shown in Fig. 5 and Fig. 6 and Fig. 7. Fig. 7(a) is a diagram showing an example of a correct character string in a matching test, an OCR result, and a decoded result of a code image. Fig. 7(b) is a diagram showing an example of a database of correct character strings and coordinates in a matching test (hereinafter referred to as a "correct character string database").

[0028] The process of linking the reference image and the correct character string will be described with reference to the flowchart of FIG. 5. In S501, the CPU 226 of the inspection device 108 executes a process of acquiring the character string and coordinates of the reference image, and the process proceeds to S502. In S502, the CPU 226 of the inspection device 108 compares the correct character string of the verification test with the character string of the OCR result and the decoded result of the code image acquired in S501 (character string extracted from the reference image). Specifically, the CPU 226 of the inspection device 108 determines whether the "correct character string of the verification test" shown in FIG. 7(a) is the same as the output character string in the "OCR result or decoded result of the code image" shown in FIG. 7(a). If the compared character strings are the same, the process proceeds to S503. If the compared character strings are not the same, the process returns to S502, and another "OCR result and decoded result of the code image" is compared with the correct character string.

[0029] In S503, the CPU 226 of the inspection device 108 registers the correct answer character string of the matching test and the coordinates of the "output character string" corresponding to the correct answer character string as a pair in the correct answer character string database 706 in the RAM 227 (see FIG. 7(b)). In this manner, the correct answer character string is linked to the coordinates indicating the position to be inspected. When linking, the correct answer character string may be directly associated and managed as shown in FIG. 7(b) or may be indirectly associated and managed via a predetermined parameter. In S504, the CPU 226 of the inspection device 108 determines whether or not the comparison of the correct answer character strings of all the matching tests has been completed. If the comparison of the correct answer character strings of all the matching tests has not been completed, the process returns to S502. If the comparison of the correct answer character strings of all the matching tests has been completed, the process of the flowchart shown in FIG. 5 ends.

[0030] The process of acquiring the character string and coordinates of the reference image will be described with reference to the flowchart of FIG. 6. In S601, the CPU 226 of the inspection device 108 extracts blocks for each content type from the reference image, and the process proceeds to S602. The loop process from S602 to S606 is repeated several times for the blocks extracted by the block extraction process executed in S601. In S602, the CPU 226 of the inspection device 108 cuts out an image of each block from the reference image. When the cutout of the image of each block is completed, the process proceeds to S603. In S603, the CPU 226 of the inspection device 108 determines the type of image for each cut-out block. When the type of image for each cut-out block is determined, the process proceeds from S604 to S606.

[0031] If the type is determined to be text in S603, the process proceeds to S604. In S604, the CPU 226 of the inspection device 108 performs character recognition processing on the image of each cut-out block, and the process proceeds to S606. The character recognition processing is performed by optical character recognition (OCR). The OCR processing obtains a character string and the area coordinates of the OCRed character string as the OCR result, and stores the OCR result in the RAM 227. Note that a known method is used for the OCR processing, and a detailed description thereof will be omitted.

[0032] If the type is determined to be an image in S603, the process proceeds to S605. In S605, the CPU 226 of the inspection device 108 performs a code image decoding process on the image of each cut-out block, and the process proceeds to S606. If a character string embedded in the code image is obtained as a result of the decoding process of the code image, the character string and the area coordinates of the code image are obtained, and the decoded result of the code image is stored in the RAM 227. Note that a known method is used for the decoding process of the code image, and therefore a detailed description is omitted. If the type is determined to be other than text or image in S603, the process proceeds to S606.

[0033] In S606, the CPU 226 of the inspection device 108 determines whether the process of S602 has been completed for all blocks. If the process of S602 has not been completed for all blocks, the process returns to S552. If the process of S602 has been completed for all blocks, the process of the flowchart shown in FIG. 6 ends.

[0034] Next, the details of the inspection setting will be described with reference to the diagram showing the UI screen shown in FIG. 8 and the flowchart shown in FIG. 9. When the user selects the reference image registered in the reference registration, the inspection setting process starts. FIG. 8 shows an example of an inspection setting screen that accepts the execution operation of the inspection setting from the user. The inspection setting screen is displayed on the UI panel 102 via the inspection unit 106 and the image forming device 101 from the inspection device 108. On the inspection setting screen, the user can arrange the inspection areas on the previewed reference image and set each inspection area individually. In this embodiment, as described later, the inspection area can be initially arranged based on information that associates the correct answer character string in FIG. 7(b) with the position coordinates of the area, so that the user can reduce the effort of manually arranging the inspection area. The user can correct the initially arranged inspection area or add a new inspection area as necessary.

[0035] The area 801 is a preview display screen where a reference image is displayed. In the case of a reference image for a multi-page print job, an operation to switch the reference image displayed in the preview can be accepted, and the reference image for each page can be displayed. The pattern inspection area 802, the character string inspection area 803, and the code image inspection area 804 are frames showing the inspection areas arranged on the preview. The inspection areas are arranged by the inspection area arrangement button 806, and the size and position can be changed by the user dragging the mouse. An example is shown in which a pattern is arranged in the pattern inspection area 802, a character string is arranged in the character string inspection area 803, and a barcode is arranged in the code image inspection area 804 in FIG. 8(a) to FIG. 8(c). The example of arranging a barcode is merely an example, and a two-dimensional code may be arranged in the code image inspection area 804.

[0036] The inspection area is selected by pressing the inspection area selection button 805. After the user presses the inspection area selection button 805, the user clicks the mouse in the pattern inspection area 802, the character string inspection area 803, and the code image inspection area 804, and the inspection area selection operation is accepted. After the user presses the inspection area placement button 806, the user selects the type of inspection area from the displayed pull-down menu, and selects the area on the preview display that the user wishes to specify as the inspection area by dragging the mouse, and the inspection area placement operation is accepted.

[0037] Area 807 is a setting item for the selected area, and displays the individual setting values ​​for the inspection area selected by the inspection area selection button 805. Area 808 is a setting item for the page range. The user presses each button to accept a specification operation for the page range on which inspection will be performed by the selected inspection area. If nothing is selected, the selected inspection area will be placed only on the page currently displayed on the preview display screen. If "Same side as current page" is selected, the selected inspection area will be placed on the page on the same side depending on whether the selected inspection area is placed on the front or back side of the sheet. If "All pages" is selected, the selected inspection area will be placed on all pages.

[0038] The area 809 is a setting item for each type of inspection area, and the setting item is displayed according to the type of inspection area selected by the inspection area selection button 805. FIG. 8(a) shows the setting items for the pattern inspection displayed when the pattern inspection area 802 is selected. The setting items for the pattern inspection include the item for what kind of defect is to be inspected when inspecting a printed matter, and the level of how fine the defect is to be detected. The inspection items include, for example, a round-shaped printing defect (a dot (spot)) and a linear printing defect (a streak). There are five inspection levels, for example, inspection level 1 to inspection level 5. Inspection level 5 detects printing defects more precisely than inspection level 1. In addition, the level can be set for each inspection item, such as inspection level 5 for dots (spots) and inspection level 4 for streaks. FIG. 8(b) shows the setting items for the character string inspection displayed when the character string inspection area 803 is selected. For each inspection area, the angle of the inspection area, the font of the character string, and the correct character string for the matching inspection are set. An item for selecting the font of the character string is displayed, and the font selection from the user is accepted. If the font is arbitrarily specified, the OCR recognition accuracy during inspection can be improved. FIG. 8(c) shows the setting items for code image inspection that are displayed when a code image inspection area 804 is selected. For each inspection area, the angle of the inspection area, the type of code, and the correct character string for the matching inspection are set. Note that in this embodiment, irrelevant items are not displayed by displaying setting items according to the type of the selected inspection area, but the display method is not limited to this. All items may be displayed and irrelevant items may be disabled, or irrelevant items may be collapsed and hidden.

[0039] The area 810 is a correct answer character string for the verification test. When the data inspection is performed, the OCR result or the decoded result of the code image is verified against the correct answer character string included in the area 810. The area 811 is an inspection setting completion button ("OK" button). When the user presses the "OK" button, the CPU 226 of the inspection device 108 completes the inspection setting, stores the inspection setting in the storage unit 228, and ends the inspection setting process. If there is an inconsistency in the settings, the CPU 226 of the inspection device 108 may disable the area 811, which is the inspection setting completion button, so that the user cannot press the inspection setting completion button. An example of an inconsistency in the settings is when an inspection area for performing the verification test exists, but the setting item of the data for the verification test is not set. The area 812 is an inspection setting cancellation button. When the user presses the cancellation button, the CPU 226 of the inspection device 108 discards the inspection setting and ends the inspection setting process.

[0040] 9 is a flowchart showing an inspection setting process executed by the CPU 226 of the inspection device 108 when an operation to start inspection setting is performed on the inspection device 108. That is, the CPU 226 of the inspection device 108 accepts an operation by a user from the inspection setting screen and the list of inspection areas displayed on the UI panel 102, and executes various processes for making inspection settings.

[0041] In S901, the CPU 226 of the inspection device 108 uses the correct character string and coordinates of the verification test acquired in S503 to initially place the inspection area and register the correct character string corresponding to each initially placed inspection area, and the process proceeds to S902. In this embodiment, the CPU 226 of the inspection device 108 initially places the character string inspection area 803 using the output coordinates 705, and further associates the initially placed character string inspection area 803 with the correct character string in the data of the verification test, based on the correct character string 702. In S902, the CPU 226 of the inspection device 108 accepts a setting operation of the data of the verification test from the user from the inspection setting screen, and the process proceeds to S903. In this embodiment, the setting operation of the data of the verification test is an operation related to the setting item of the data of the verification test by the user shown in the area 807 of FIG. 8(a). In S903, the CPU 226 of the inspection device 108 accepts an operation of placing an additional inspection area from the user from the inspection setting screen, and the process proceeds to S904. In this embodiment, the operation of arranging the inspection area refers to an operation by the user on the inspection area arrangement button 806 shown in FIG. 8(b) or FIG. 8(c).

[0042] In S904, the CPU 226 of the inspection device 108 accepts an inspection area selection operation from the user on the inspection setting screen, and the process proceeds to S905. In this embodiment, the inspection area selection operation refers to an operation by the user on the inspection area selection button 805 shown in FIG. 8(b) or FIG. 8(c). In S905, the CPU 226 of the inspection device 108 accepts a setting operation for each inspection area from the inspection setting screen, and the process proceeds to S906. In this embodiment, the setting operation for each inspection area refers to an operation of selecting setting items for each inspection area by the user shown in area 808 or area 809 in FIG. 8(b) or FIG. 8(c), and whether or not to perform a consecutive number inspection (not shown).

[0043] In S906, the CPU 226 of the inspection device 108 determines whether or not the user has completed the inspection settings. If the CPU 226 of the inspection device 108 determines that the inspection settings have been completed, that is, if the user has pressed the inspection settings completion button shown in area 811, the process proceeds to S907. If the CPU 226 of the inspection device 108 determines that the inspection settings have not been completed, the process returns to S902, and the user continues the inspection settings. In S907, the CPU 226 of the inspection device 108 saves the settings in the storage unit 228, and ends the inspection setting process shown in FIG. 9.

[0044] Next, details of the inspection process will be described with reference to the flowchart shown in Fig. 10. The inspection process shown in Fig. 10 is realized by the CPU 226 of the inspection device 108 executing various processes.

[0045] In S1001, the CPU 226 of the inspection device 108 reads out the inspection settings from the storage unit 228 to the RAM 227, transitions to a state waiting to receive a scanned image of the printed sheet, and processing proceeds to S1002. In S1002, the user executes a print job to be inspected on the client computer 110. When the print job is executed, the CPU 226 of the inspection device 108 receives the inspection image acquired by the inspection unit 106 scanning the printed sheet from the inspection unit 106 via the inspection device I / F 215 and the inspection unit I / F 231. When the inspection image is received, processing proceeds to S1003.

[0046] In S1003, the CPU 226 of the inspection device 108 executes the process of the flowchart of FIG. 6 on the inspection image, acquires a character string and its coordinates from the inspection image, and the process proceeds to S1004. In S1004, the CPU 226 of the inspection device 108 acquires an inspection area from the inspection settings loaded in the RAM 227, and the process proceeds to S1005. In S1005, the CPU 226 of the inspection device 108 compares the correct character string with the OCR result and the decoded result of the code image in the inspection area, and the process proceeds to S1006. If the comparison result is a mismatch, the CPU 226 of the inspection device 108 determines the inspection result of the inspection area as NG. If the comparison result is a match, the CPU 226 of the inspection device 108 determines the inspection result of the inspection area as OK. In S1006, the CPU 226 of the inspection device 108 stores the inspection result output in S1005 in the RAM 227 as a log, and the process proceeds to S1007. In S1007, the CPU 226 of the inspection device 108 determines whether the inspection of the inspection area specified in the inspection setting has been completed. If the CPU 226 of the inspection device 108 determines that the inspection has been completed, the process proceeds to S1008. If the CPU 226 of the inspection device 108 determines that the inspection has not been completed, the process returns to S1004.

[0047] In S1008, the CPU 226 of the inspection device 108 determines whether or not the inspection of all printed sheets to be inspected has been completed. If the CPU 226 of the inspection device 108 determines that the inspection of all printed sheets has not been completed, the process returns to step S1002. If the CPU 226 of the inspection device 108 determines that the inspection of all printed sheets has been completed, the process proceeds to step S1009. In S1009, the CPU 226 of the inspection device 108 ends the waiting state for receiving the inspection image of the printed sheet, and the process proceeds to step S1010. In S1010, the CPU 226 of the inspection device 108 transmits the log stored in the RAM 227 as the inspection result to the image forming device 101 via the inspection unit 106. When the CPU 201 of the image forming device 101 receives the inspection result and displays the inspection result on the UI panel 102, the CPU 226 of the inspection device 108 ends the inspection process shown in FIG. 10. In S1010, an example in which the inspection results are displayed on the UI panel 102 has been shown, but this is not limiting. The CPU 226 of the inspection device 108 may display the inspection results on the display control unit 245 of the inspection device 108. In addition, details of the inspection results of a printed sheet that has failed the inspection may be displayed on the UI panel 102 or the UI screen of the display control unit 245. At this time, a character string that does not match the correct character string and the character string on the printed sheet may be displayed in a different color from a character string that matches the correct character string.

[0048] As described above, according to this embodiment, before the test setting, the OCR process and the code image decoding process are performed on the reference image, and the character string in the reference image corresponding to the correct character string and the coordinates of the character string are obtained. This makes it possible to easily perform the work of associating the character string and the code image in the reference image with the correct character string, even if the reference image is image data that does not have coordinate information for each object.

[0049] [Second embodiment] In the first embodiment, an example was shown in which the OCR process and the code image decoding process are performed before the test setting is executed, and the coordinates of the process result are linked to the correct character string of the comparison test. However, in the above example, if the correct character string does not match the character string of the OCR result and the decoded result of the code image, the linking is not possible. Therefore, in this embodiment, even if the character string of the OCR result and the decoded result of the code image does not completely match the correct character string, the linking is performed. In other words, if the character string (output character string) of the OCR result or the decoded result is similar to the correct character string and can be estimated to be a character string corresponding to the correct character string, a linking process is performed to link the position coordinates of the output character string to the correct character string. Note that a description of the parts common to this embodiment and the first embodiment will be omitted.

[0050] The process S403 of linking a reference image and a correct answer character string in this embodiment will be described with reference to the flowchart in Fig. 11 and Fig. 12. Fig. 11 is a flowchart showing the process of linking a reference image and a correct answer character string for an inspection area in this embodiment. Fig. 12(a) is an example of a correct answer character string for a matching inspection in this embodiment, an OCR result, and a decoded result of a code image. Fig. 12(b) is an example of a correct answer character string database in this embodiment.

[0051] 11 is realized by the CPU 226 of the inspection device 108 executing various processes for linking the reference image with the correct answer character string. The process of S501 in the flowchart of FIG. 11 is the same as the process of S501 in the flowchart of FIG.

[0052] In S1101, the CPU 226 of the inspection device 108 obtains the Levenshtein distance between the correct character string of the verification inspection and all output character strings of the OCR result and the decoded result of the code image (all character strings extracted from the reference image). The Levenshtein distance is a distance that indicates how different two character strings are. That is, the shorter the Levenshtein distance between two character strings, the more similar the two character strings are. In other words, the Levenshtein distance indicates the similarity between two character strings. The Levenshtein distance indicates the minimum number of operations required to convert one character string into another. These operations include an operation of inserting one character into a character string, an operation of deleting one character from a character string, and an operation of replacing a character in a character string with another character. For example, the Levenshtein distance between the correct character string "abc" and the output character string "abo" in FIG. 12(a) is 1. Also, the Levenshtein distance between the correct character string "abc" and the output character string "efgh" is 4.

[0053] In S1102, the CPU 226 of the inspection device 108 determines whether the shortest Levenshtein distance acquired in S1101 is less than a threshold value. If the Levenshtein distance is less than the threshold value, it can be estimated that the output character string with the shortest Levenshtein distance is similar to the correct character string and corresponds to the correct character string. On the other hand, if the shortest Levenshtein distance is equal to or greater than the threshold value, it can be determined that there is no output character string that is similar to the correct character string by a predetermined amount. In this way, even if the output character string includes an error in OCR or an error in decoding the code image, it is possible to identify the output character string corresponding to the correct character string by determining whether the output character string is similar to the correct character string by a predetermined amount. If it is determined in S1102 that the shortest Levenshtein distance is less than the threshold value, the process proceeds to S1103. If it is determined in S1102 that the shortest Levenshtein distance is equal to or greater than the threshold value, the process returns to S1101. The threshold value may be set in advance, or may be set by the user inputting a value.

[0054] In S1103, the CPU 226 of the inspection device 108 registers the correct answer character string of the matching test and the coordinates of the output character string with the shortest Levenshtein distance as a pair, and the process proceeds to S1104. That is, the output coordinates 1203 of the output character string 1201 (the output character string with the shortest Levenshtein distance) that is most similar to the correct answer character string 702 is registered as the position coordinates corresponding to the correct answer character string 702. The above pair is registered in the correct answer character string database 1204 in the RAM 227. In S1104, the CPU 226 of the inspection device 108 determines whether or not the comparison of all the correct answer character strings of the matching test has been completed. If the comparison of all the correct answer character strings has not been completed, the process returns to S1101. If the comparison of all the correct answer character strings has been completed, the process of linking the position coordinates in the reference image with the correct answer character string is terminated.

[0055] In this embodiment, the Levenshtein distance is used to determine the output character string that is similar to the correct character string, but the method is not limited to the Levenshtein distance. For example, other methods capable of determining the similarity (e.g., Hamming distance, etc.) may be used to determine the character string that is similar to the correct character string.

[0056] According to this embodiment, even if the OCR result of the reference image or the decoded result of the code image executed before the inspection process is partially incorrect, it is possible to link the position coordinates of the output character string obtained from the reference image by the OCR process or the decoding process with the correct character string. As a result, even if the reference image is image data that does not have coordinate information of each object, it is possible to easily perform the work of associating the character string and the code image in the reference image with their correct character strings.

[0057] [Other embodiments] The present disclosure can also be realized by a process in which a program for implementing one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more functions.

[0058] The disclosure of the above-described embodiment includes the following configurations.

[0059] (Configuration 1) An inspection device for inspecting printed sheets, comprising: an acquisition means for acquiring a character string and coordinates indicating its position for each block by performing processing according to the type of each block extracted from a reference image used in inspection settings; a linking means for determining which character string acquired from the reference image corresponds to the correct string by comparing the character string acquired from the reference image with a correct string, and linking the coordinates indicating the position of the character string determined to correspond to the correct string with the correct string; and an inspection setting means for setting an inspection area to be inspected on the printed sheet based on the coordinates linked to the correct string by the linking means.

[0060] (Configuration 2) The inspection device described in Configuration 1, characterized in that the inspection setting means displays an inspection setting screen in which the inspection area is initially positioned on the reference image based on the coordinates linked to the correct string by the linking means, and performs settings regarding the inspection area based on user operation on the inspection setting screen.

[0061] (Configuration 3) An inspection device as described in configuration 1 or 2, further comprising an inspection means for reading a printed sheet to be inspected and inspecting an image obtained based on settings regarding the inspection area made by the inspection setting means.

[0062] (Configuration 4) The inspection device described in Configuration 1, characterized in that the linking means determines whether the character string obtained from the reference image is similar to the correct character string, and links coordinates indicating the position of the character string determined to be similar to the correct character string.

[0063] (Configuration 5) The inspection device according to Configuration 4, wherein the linking means determines whether the character string obtained from the reference image is similar to the correct character string using a Levenshtein distance between the character string obtained from the reference image and the correct character string, and links coordinates indicating a position of the character string determined to be similar to the correct character string.

[0064] (Configuration 6) The inspection device according to Configuration 5, wherein the linking means links coordinates indicating a position of the character string obtained from the reference image in which the Levenshtein distance is shortest and the Levenshtein distance is less than a threshold value, to the correct character string.

[0065] (Configuration 7) The inspection apparatus according to configuration 6, wherein the threshold value of the Levenshtein distance is set in advance.

[0066] (Configuration 8) The inspection device according to configuration 6, further comprising a display control means for displaying a UI screen for accepting the threshold value of the Levenshtein distance from a user.

[0067] (Configuration 9) The inspection device described in Configuration 3, further comprising a display control means for displaying details of the inspection result when the inspection result by the inspection means of the image obtained by reading the printed sheet to be inspected is a failure.

[0068] (Configuration 10) The inspection device according to configuration 1, wherein the correct character string is a character string that should be printed on the printed sheet.

[0069] (Configuration 11) The inspection device according to claim 1, characterized in that the correct character string is a character string that should be encoded by a code image to be printed on the printed sheet.

[0070] (Configuration 12) A control method for an inspection device for inspecting a printed sheet, comprising the steps of: acquiring a character string and coordinates indicating its position for each block extracted from a reference image used in inspection settings by performing processing according to the type of each block; determining which character string acquired from the reference image corresponds to the correct string by comparing the character string acquired from the reference image with a correct string, and linking the coordinates indicating the position of the character string determined to correspond to the correct string with the correct string; and setting an inspection area to be inspected on the printed sheet based on the coordinates linked to the correct string by the linking means.

[0071] (Configuration 13) An inspection system including a printing device that prints on a sheet based on image data specified in a print job, and an inspection device for inspecting the printed sheet, wherein the inspection device is equipped with: an acquisition means that acquires a character string and coordinates indicating its position for each block by performing processing according to the type of each block extracted from a reference image used to set up the inspection; a linking means that determines which character string acquired from the reference image corresponds to the correct string by comparing the character string acquired from the reference image with a correct string, and linking the coordinates indicating the position of the character string determined to correspond to the correct string with the correct string; an inspection setting means that sets an inspection area to be inspected on the printed sheet based on the coordinates linked to the correct string by the linking means; and an inspection means that inspects for defects in an image obtained by reading a sheet printed by the printing device, based on the setting for the inspection area made by the inspection setting means.

Claims

1. An inspection device for inspecting a printed sheet, comprising: an acquisition means for acquiring a character string and coordinates indicating a position of the character string for each block by performing processing according to the type of each block extracted from a reference image used for setting the inspection; a linking means for determining a character string obtained from the reference image that corresponds to the correct character string by comparing the character string obtained from the reference image with a correct character string, and linking coordinates indicating a position of the character string determined to correspond to the correct character string with the correct character string; an inspection setting means for setting an inspection area to be inspected on the printed sheet based on the coordinates linked to the correct character string by the linking means; Equipped with An inspection device characterized by:

2. The inspection device according to claim 1, characterized in that the inspection setting means displays an inspection setting screen in which the inspection area is initially positioned on the reference image based on the coordinates linked to the correct string by the linking means, and performs settings regarding the inspection area based on a user's operation on the inspection setting screen.

3. 2. The inspection apparatus according to claim 1, further comprising an inspection unit that performs an inspection on an image obtained by reading a printed sheet to be inspected based on the settings regarding the inspection area made by the inspection setting unit.

4. 2. The inspection device according to claim 1, wherein the linking means determines whether the character string obtained from the reference image is similar to the correct character string, and links coordinates indicating a position of the character string determined to be similar to the correct character string.

5. the linking means determines whether the character string obtained from the reference image is similar to the correct character string by using a Levenshtein distance between the character string obtained from the reference image and the correct character string, and links coordinates indicating a position of the character string determined to be similar to the correct character string.

5. The inspection apparatus according to claim 4.

6. the linking means links, with the correct character string, coordinates indicating a position of the character string acquired from the reference image in which the Levenshtein distance is shortest and is less than a threshold value; 6. The inspection apparatus according to claim 5.

7. 7. The inspection apparatus according to claim 6, wherein the threshold value of the Levenshtein distance is set in advance.

8. a display control unit for displaying a UI screen for receiving the Levenshtein distance threshold value from a user; 7. The inspection apparatus according to claim 6.

9. 4. The inspection device according to claim 3, further comprising a display control means for displaying details of the inspection result when the inspection result by the inspection means of the image obtained by reading the printed sheet to be inspected is a failure.

10. The correct answer character string is a character string to be printed on the printed sheet.

2. The inspection apparatus according to claim 1 .

11. The correct answer character string is a character string that should be encoded by a code image to be printed on the printed sheet.

2. The inspection apparatus according to claim 1 .

12. A method for controlling an inspection device for inspecting a printed sheet, comprising the steps of: A step of performing processing according to the type of each block extracted from the reference image used for setting the inspection, thereby obtaining a character string and coordinates indicating the position of the character string for each block; determining a character string obtained from the reference image that corresponds to a correct answer character string by comparing the character string obtained from the reference image with a correct answer character string, and associating coordinates indicating a position of the character string that corresponds to the correct answer character string with the correct answer character string; A step of setting an inspection area to be inspected on the printed sheet based on the coordinates linked to the correct character string by the linking means; Equipped with 23. A method for controlling an inspection apparatus comprising:

13. A program for causing a computer to function as the inspection device according to any one of claims 1 to 11.

14. An inspection system including a printing device that prints on a sheet based on image data specified in a print job, and an inspection device that inspects the printed sheet, The inspection device includes: an acquisition means for acquiring a character string and coordinates indicating a position of the character string for each block by performing processing according to the type of each block extracted from a reference image used for setting the inspection; a linking means for determining a character string obtained from the reference image that corresponds to the correct character string by comparing the character string obtained from the reference image with a correct character string, and linking coordinates indicating a position of the character string determined to correspond to the correct character string with the correct character string; an inspection setting means for setting an inspection area to be inspected on the printed sheet based on the coordinates linked to the correct character string by the linking means; an inspection means for inspecting an image obtained by reading a sheet printed by the printing device for defects based on the settings related to the inspection area made by the inspection setting means; Equipped with 1. An inspection system comprising:

Citation Information

Patent Citations

  • Information processing device and method for controlling the same

    JP2021024223A