Inspection apparatus

Abstract

To solve the problem that a large load is applied to an operator since an inspection area must be set on each of a front surface and a rear surface when setting the inspection area of front and rear coincidence inspection.SOLUTION: The inspection apparatus includes a storage unit configured to store a reference image, and an inspection setting unit configured to set a data inspection setting to be used in the matching inspection and a first region to be an inspection region in the matching inspection on the reference image by an operation of an operator, wherein the inspection setting unit automatically creates a second region corresponding to the first region on the reference image.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to an inspection device. [Background technology]

[0002] Traditionally, inspections (quality checks) to check whether printed materials are printed correctly have been performed manually, but in recent years, automatic inspection devices have been used as post-processing for printing presses. These inspection devices inspect variable data areas (variable data) such as character strings and barcodes in variable printing. Examples include data readability tests that check whether character strings and barcodes are readable, and data verification tests that verify the results of reading character strings and barcodes with the correct answer. In the following, data readability tests and data verification tests are referred to as data tests. In addition, one type of data test is a match test.

[0003] Patent Document 1 proposes a front / back consistency inspection method for checking whether the results of reading data inspection areas set on the front and back sides match. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-69482 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the method of Patent Document 1, when setting the inspection area for the coincidence inspection, the inspection area must be set on both the front and back sides, which poses a problem in that the operator has to work hard to set the coincidence inspection. [Means for solving the problem]

[0006] The inspection device of the present invention is an inspection device that extracts data from an image contained in multiple areas and performs a consistency test to check whether the extracted multiple pieces of data match, and is characterized in that it has a memory means for storing a reference image, and an inspection setting means that, by operation of an operator, sets a data inspection setting to be used in the consistency test and a first area that will be the inspection area in the consistency test for the reference image, and the inspection setting means automatically creates a second area corresponding to the first area for the reference image. [Effects of the Invention]

[0007] According to the present invention, it is possible to significantly reduce the load imposed on the setting of the back surface when setting the area for coincidence inspection. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of a system configuration including an inspection apparatus according to an embodiment of the present invention. [Figure 2] 1 is an example of an internal configuration diagram of an image forming apparatus 100 according to the present embodiment. [Figure 3] 1 is an example of an internal configuration diagram of an inspection device 110 according to this embodiment. [Figure 4] 1 is an example of a flowchart of the entire inspection process according to this embodiment. [Figure 5] 10 is an example of a job management UI screen according to this embodiment. [Figure 6] 1 is an example of a UI screen for test settings according to this embodiment. [Figure 7] 1 is a flowchart illustrating an example of the test setting (S403) according to the first embodiment. [Figure 8] An example of a match inspection area display in the first embodiment [Figure 9] 10 is a flowchart showing an example of a process for generating a match check area (S705) according to the second embodiment. [Figure 10] An example of setting a matching inspection area in the second embodiment [Figure 11] 10 is a flowchart showing an example of a process for generating a match check area (S705) according to the third embodiment. [Figure 12] An example of a UI when generating a match check area (S705) in the fourth embodiment [Figure 13] An example of a UI when generating a match check area (S705) in the fifth embodiment DETAILED DESCRIPTION OF THE INVENTION

[0009] The following detailed description of the embodiments of the present invention will be given with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention, and not all of the combinations of features described in the embodiments are necessarily essential to the solution of the present invention.

[0010] In the following description, the image forming apparatus may also be called a multifunction device, a multifunction peripheral, or an MFP (Multi Function Peripheral).

[0011] <Embodiment 1> 1 is a diagram showing a system configuration including an inspection device according to an embodiment of the present invention, in which 100 denotes an image forming device, 110 denotes an inspection device, 120 denotes a finisher, 130 denotes a client PC, 140 denotes a print server, and 150 denotes a network.

[0012] The image forming apparatus 100 performs print output based on various input data, for example, print data sent from a client PC 130 or a print server 140. In this embodiment, the image forming apparatus is described as an image forming apparatus, but the present invention is not limited to this and may be any apparatus that prints on a recording medium. For example, the image forming apparatus may be an apparatus that prints on metal.

[0013] The inspection device 110 inspects whether there are any defects in the printed matter that is sequentially conveyed from the image forming device 100. Here, a defect is something that reduces the quality of the printed matter, such as stains that occur when coloring material adheres to unintended locations during printing, or color loss that occurs when insufficient coloring material adheres to intended locations.

[0014] Furthermore, the inspection device 110 inspects variable area portions of variable prints that include variable area portions, such as one-dimensional codes such as character strings and barcodes, and two-dimensional codes such as QR Codes (registered trademark). For example, a data readability test is performed to check whether character strings and barcodes are readable, and a data matching test is performed to check whether the read results of the character strings and barcodes match the correct answers. Furthermore, a front-to-back consistency test is performed to check whether the read results of data inspection areas set on the front and back sides match. That is, the inspection device 110 performs print image inspection to detect abnormalities in the image portion of the printed material, and data inspection, including a data readability test and a data matching test. Note that the inspection device 110 does not necessarily need to have an inspection processing unit that performs print image inspection and data inspection within it. For example, an information processing device (not shown) such as an inspection PC communicably connected to the inspection device 110 may perform the inspection processing. Note that in this embodiment, barcode inspection refers to an inspection that can also inspect two-dimensional codes such as QR codes. Furthermore, the inspection system in this embodiment is assumed to be composed of at least the inspection device 110 that inspects printed materials.

[0015] The finisher 120 receives the output paper inspected by the inspection device 110, switches the paper discharge destination based on the inspection result of the inspection device 110, performs post-processing (binding, stapling, etc.) as necessary, and discharges the paper.

[0016] Image forming apparatus 100 is connected to client PC 130 and print server 140 via network 150, and is further connected to inspection apparatus 110 and finisher 120 via communication cables. In addition to image forming apparatus 100, inspection apparatus 110 is also connected to finisher 120 via communication cables. This embodiment will be described using, as an example, an inline inspection machine that performs image formation, inspection, post-processing, and paper discharge in an integrated manner, but this is not intended to limit the present invention.

[0017] [Image forming device configuration] 2 is a diagram showing the internal configuration of the image forming apparatus 100 in this embodiment. The controller 200 receives images and documents from the network 150 and converts the received images and documents into print data. The printer unit 210 generates a printout by printing the print data on a recording sheet (paper, sheet). The UI unit 220 displays the screen and accepts instructions from the operator to the image forming apparatus 100, such as the selection of paper information. The image forming apparatus 100 is made up of the above-mentioned controller 200, printer unit 210, and UI unit 220.

[0018] Reference numerals 201 to 208 are components of the controller 200. A network I / F (interface) unit 201 transmits and receives data to and from the client PC 130 and the print server 140 via the network 150. A CPU 202 controls the image forming apparatus 100 as a whole.

[0019] The RAM 203 is a work area when the CPU 202 executes various commands, and the ROM 204 stores program data executed by the CPU 202 at startup, setting data for the controller 200, etc. The image processing unit 205 performs RIP (Raster Image Processor) processing to convert image and document data received from the network 150 into print data.

[0020] In this embodiment, the RIP process does not need to be performed by the image processing unit 205, and may be performed by an information processing device (not shown) communicably connected to the image forming apparatus 100, for example.

[0021] An engine I / F (interface) unit 206 transmits print data to a printer unit 210. A communication I / F unit 207 communicates with the inspection device 110 and the finisher 120. Reference numeral 208 denotes an internal bus (system bus).

[0022] Images and documents created on client PC 130 or print server 140 on network 150 are transmitted as PDL data to image forming apparatus 100 via a network (e.g., a local area network). Note that print jobs for images, documents, etc. may be transmitted to an information processing device (not shown) via the network and managed by the information processing device. The print jobs may then be transmitted from the information processing device to image forming apparatus 100 via network 150, and image forming apparatus 100 may perform processing to print on paper.

[0023] The transmitted PDL data is stored in RAM 203 via network I / F unit 201. Print instructions from an operator on UI unit 220 are also stored in RAM 203 via internal bus 208. An example of a print instruction from an operator is a selection of a paper type.

[0024] The image processing unit 205 acquires the PDL data stored in the RAM 203 and performs image processing to convert it into print data. The image processing to convert it into print data involves, for example, rasterizing the PDL data, converting it into multi-value bitmap data, and then performing screen processing or the like to convert it into binary bitmap data. The binary bitmap data obtained by the image processing unit 205 is sent to the printer unit 210 via the engine I / F unit 206.

[0025] The printer unit 210 prints the received binary bitmap data on a recording sheet using color materials. The CPU 202 issues instructions to the printer unit 210 based on print instructions from an operator that are stored in the RAM 203. For example, if the operator instructs the printer unit 210 to print on coated paper, the CPU 202 instructs the printer unit 210 to output paper from a paper cassette (not shown) in the image forming apparatus 100 that stores coated paper. The CPU 202 controls various processes from receiving the PDL data to printing on paper, as described above, to form a full-color toner image on the paper.

[0026] [Internal configuration of inspection device 110] 3 is a diagram showing the internal configuration of the inspection device 110. An inspection control unit 300 controls the entire inspection device 110 and controls the inspection process to determine whether or not there are defects in printed matter.

[0027] The image reading unit 310 reads the printed material conveyed from the image forming apparatus 100. The image reading unit 310 generates a scanned image (read sheet) by reading the printed material.

[0028] The UI unit 320 is a UI unit that allows an operator to configure the inspection device 110 and display the inspection results to the operator. The configuration of the inspection device 110 configured by the operator refers to the type of defect to be inspected when inspecting a printed material. The inspection items include, for example, round defects (dots) and linear defects (streaks). The inspection device 110 is comprised of the inspection control unit 300, image reading unit 310, and UI unit 320 described above. In this embodiment, the UI unit 320 is comprised of a display unit that displays a screen and a display control unit that controls the screen displayed on the display unit. The configuration of the inspection device 110 and the display of the inspection results performed by the UI unit 320 may be displayed and instructions may be received by an external device such as the UI unit 220 of the image forming apparatus 100, an inspection PC (not shown), or an information processing device (not shown).

[0029] Reference numerals 301 to 306 are components of the inspection control unit 300. The communication I / F (interface) unit 301 transmits and receives data to and from the image forming apparatus 100, the finisher 120, and the inserter 160. The CPU 302 controls the entire inspection device 110. The RAM 303 is a work area when the CPU 302 executes various commands, and the ROM 304 stores program data executed by the CPU 302 at startup, setting data for the inspection control unit 300, and the like. The ROM 304 also stores a correct answer CSV file, which will be described later. The inspection processing unit 305 inspects whether or not there are defects in the printed matter. The internal bus 306 is a system bus.

[0030] (printed image inspection) An overview of the print image inspection performed by the inspection device 110 will be described. The inspection device 110 uses the image reading unit 310 to read the printed material conveyed from the image forming device 100 and acquire a scanned image (read image) of the inspection target. The acquired scanned image of the inspection target is stored in RAM 303. Next, the inspection device 110 uses the inspection processing unit 305 to acquire a difference value between the reference image, which has been stored in advance in RAM 303 as a correct image, and the scanned image of the inspection target.

[0031] Next, the inspection device 110 performs an inspection by comparing the calculated difference value with the inspection threshold value (contrast, size, etc.) for each inspection item for each pixel. The inspection results are saved in RAM 303, and include, for example, information on whether or not there is an abnormality in the printed matter, the type of abnormality detected (such as a dot or streak), and position information of the abnormality when displayed on the UI unit 320.

[0032] (Data inspection) An overview of the data inspection performed by the inspection device 110 will be described. The inspection device 110 uses the image reading unit 310 to read printed materials sequentially conveyed from the image forming device 100 and acquire a scanned image of the inspection target. The acquired scanned image of the inspection target is stored in RAM 303. Next, the inspection processing unit 305 of the inspection device 110 first performs an extraction process to extract data from character strings, barcodes, and QR codes in the scanned image. This is performed using a pre-set character font for optical character recognition (OCR) and barcode standards, and OCR processing is performed on characters. Decoding processing is performed on barcodes. The data inspection checks whether character strings and barcodes in the area set for data inspection can be read. If they can be read, an OK judgment is made; if they cannot be read, an NG judgment is made. A data matching test can also be performed to check whether the extracted data, such as the read character strings and barcodes, matches the corresponding data (correct data) in a pre-prepared correct CSV file. Here too, if the matching results show a match, an OK judgment is made; if they do not, an NG judgment is made. The results of the inspection are stored in RAM 303, including, for example, the results of the character strings and barcodes read from the printed material, the results of matching with the correct data, and the position information of the characters and barcodes read when displayed on the UI unit 320.

[0033] (Front and back matching test) An overview of the front and back match inspection performed by the inspection device 110 will be described. Similar to data inspection, the inspection device 110 reads the printed material and performs extraction processing to extract data from character strings, barcodes, and QR codes in the scanned image. In the front and back match inspection, it is determined whether the data extracted from the front and back match. If they match, an OK judgment is made, and if they do not match, an NG judgment is made. The results of the inspection are saved in RAM 303, and for example, the results of the character strings and barcodes read from the printed material, the results of whether the front and back match, and the position information of the read characters and barcodes when displayed on the UI unit 320 are saved.

[0034] After the above-described print image inspection, data inspection, and front / back consistency inspection are completed, the inspection device 110 instructs the UI unit 320 to display the inspection results stored in the RAM 303 via the CPU 302. The operator recognizes the inspection results by displaying them on the UI unit 320. Furthermore, if a defective printed product occurs or a certain number of defective products occur consecutively, the inspection device 110 causes the CPU 302 to send the above information to the image forming device 100 via the communication I / F unit 301.

[0035] Information that a defective printed matter has occurred is received by controller 200 via communication I / F unit 207. When controller 200 receives the information, CPU 202 instructs printer unit 210 to stop printing. Image forming apparatus 100 stops the printing operation when printer unit 210 is instructed to stop printing.

[0036] Furthermore, the inspection device 110 also transmits information to the finisher 120 via the communication I / F unit 301 based on the inspection results stored in the RAM 303 by the CPU 302. The information transmitted to the finisher 120 is information on whether or not there are defects in the printed matter. Using the received information, the finisher 120 discharges printed matter without defects to a normal paper discharge tray, and discharges printed matter with defects to a tray other than the normal paper discharge tray.

[0037] <Overall inspection flow> Next, the overall flow from the registration work before the start of an inspection in the inspection device 110 to the execution of the inspection will be described using the flowchart in Fig. 4. This flowchart is realized by the CPU 302 expanding the program code stored in the ROM 304 into the RAM 303, and reading and executing the program code expanded in the RAM.

[0038] In step S401, the CPU 302 registers a character font. The registered character font is used during data inspection. A character font is data that associates character images with character codes, which are required for optical character recognition (OCR) performed during data inspection. The character font creation procedure begins with the inspection device 110 waiting in character font image reading mode and receiving a print job for character font creation from the client PC 130. The inspection device 110 receives the character font job from the client PC 130 and reads the character font image. When printing is executed, the inspection device 110 detects the transport of a printed material from the image forming device 100, scans the printed material with the image reading unit 310, and stores the scanned image in RAM 303 of the inspection device 110. A character font can be created by extracting characters to be OCRed one by one from the scanned image and having the operator input the character code for the extracted character image. The created character font is stored in RAM 303 of the inspection device 110. While the method for creating character fonts in this embodiment has been described, any method that can create data in which character codes are associated with each character image extracted from a scanned image may be used. Note that there may be cases where data inspection is not performed and only print image inspection is performed. In this case, S401 is not performed, and the process proceeds to S402.

[0039] In step S402, the CPU 302 registers a reference image that will be the correct image for inspection. The inspection device 110 waits in a reference image reading mode, and a print job for registering the reference image is executed from the client PC 130. When printing is executed, the inspection device 110 detects the conveyance of the printed matter, scans the printed matter with the image reading unit 310, and the scanned image is saved in the RAM 303 of the inspection device 110 as the reference image. In this embodiment, the reference image is registered by scanning it with the image reading unit 310, but this is not limited to this. For example, an image RIP-processed by the print server 140 or the image processing unit 205 of the image forming device 100 may also be registered as the reference image.

[0040] In step S403, the CPU 302 accepts the inspection settings and stores various inspection setting values ​​such as the inspection area and inspection level in the RAM 303 of the inspection device 110. Details of S403 in this embodiment will be described later.

[0041] In step S404, the CPU 302 receives an inspection print job from the client PC 130, detects paper transport, scans the paper with the image reading unit 310, and saves the scanned image in the RAM 303 of the inspection device 110. Then, a pattern inspection is performed using the scanned image of the inspection job and the reference image registered in S402, using the inspection setting values ​​set in S403. Also, a data inspection is performed using the character font registered in step S401 and the inspection settings set in step S403. The processing of this flowchart ends.

[0042] FIG. 5 shows an example of a job management screen 500 displayed on the UI unit 320.

[0043] The job management screen 500 is displayed when the inspection device 110 is started up, or when an application is started up from the UI unit 320 by an operator operation.

[0044] From the job management screen 500, it is possible to transition to each of the processes of font registration, reference image registration, inspection setting, and inspection.

[0045] A button 501 is a button for clearing the display on the screen 500. A button 502 is a button for creating a new inspection job and registering a reference image.

[0046] Button 503 is a button for duplicating an inspection job that has already been created. This duplicates the inspection job selected in the inspection job list 508. By making a duplicate, the reference image and inspection settings can be duplicated, and a new inspection can be performed. Pressing button 503 will take you to the inspection settings screen shown in Figure 6.

[0047] Button 504 is a delete button, which deletes the inspection job selected in the inspection job list 508. Here, it is also possible to delete multiple inspection jobs simultaneously by selecting multiple inspection jobs and pressing button 504. Button 505 is an inspection setting button, which performs inspection settings for an inspection job for which reference image registration has been completed.

[0048] Button 506 is an inspection button, which inspects an inspection job for which reference image registration and inspection settings have been completed. Button 507 is a font registration button, which registers character fonts.

[0049] [Inspection Settings] Next, the test settings will be described with reference to Fig. 6. Fig. 6(a) shows an example of a test setting screen 600 displayed on the UI unit 320 of the testing device 110 for performing test settings.

[0050] Button 601 is a reference image change button, which is used when changing the reference image. Button 602 is an inspection area selection button, which is pressed by the operator when wanting to select an area that has already been set.

[0051] A button 603 is a button for deleting an inspection area, and is pressed by the operator when the selected area is to be deleted. A button 604 is a button for rotating the image displayed in an area 605 .

[0052] Area 605 is a display area that displays the loaded reference image. If there are multiple sheets to be loaded, the image to be displayed is switched using button 610. The front and back sides of the loaded sheets are also switched using button 610. Specifically, each time the right arrow button 610 is pressed, the pages are switched in the order of the front of the first sheet, the back of the first sheet, the front of the second sheet, and the back of the second sheet.

[0053] Button 611 is an OK button for saving the settings on screen 600 and transitioning to job management screen 500 shown in Fig. 5. Pressing button 611 may also transition to an inspection screen (not shown) where inspection can be performed. Button 612 is a cancel button for transitioning to job management screen 500 shown in Fig. 5 without saving the settings on screen 600.

[0054] Button 621 is pressed by the operator when creating a new print image inspection area. After pressing the button, the operator sets the inspection area for the reference image displayed in area 605. Area 606 shows an example of setting the print image inspection area.

[0055] Button 622 is pressed by the operator when creating a new area for character inspection or barcode inspection. After pressing the button, the operator sets the inspection area on the reference image displayed in area 605. Character area 607 shows an example of a set character inspection area. Area 608 shows an example of a set barcode inspection area. Note that in FIG. 6, areas 606, character area 607, and area 608 are drawn with the same black dotted line, but they may be displayed so that they are distinguishable as areas where different processes are performed. For example, the frames of areas where different processes are performed may be displayed in different colors or with dashed lines. The operator may also be able to select the display color of these frames.

[0056] Button 623 is pressed by the operator when creating a new area for serial number inspection. After pressing the button, the operator sets the inspection area for the reference image displayed in area 605. The serial number inspection is a data inspection performed based on predetermined rules. The predetermined rules include the start number, end number, and increment / decrement value.

[0057] Setting item 631 is a setting for misalignment inspection, and sets the amount of allowable misalignment of the print position from the reference image. In this embodiment, an example is shown in which the operator specifies a misalignment amount of 2 mm or more to be detected. In other words, the value specified by the operator here corresponds to the threshold value for misalignment detection. If a misalignment equal to or greater than the threshold value set here is detected, the inspection is judged to be NG.

[0058] The setting area 632 is a group of UIs for configuring the area currently selected in the area 605. The setting items 633 set the scope of application of the selected area. If nothing is selected, the selected inspection area will be placed only on the page currently displayed in the area 605. If "Same side as current page" is selected, the selected inspection area will be placed on the page on the same side depending on whether the selected inspection area is placed on the front or back side of the sheet. If "All pages" is selected, the selected inspection area will be placed on all pages.

[0059] Setting item 634 is for setting the detection level for round defects (dots) and linear defects (streaks). The detection level is a parameter set in stages to determine the size required for each feature of a detected defect to be deemed a defect. For example, there are five levels, from level 1 to level 5, with level 5 being able to detect thinner and smaller defects than level 1. Also, a level can be set for each inspection item, such as inspection level 5 for dots and inspection level 4 for streaks. Setting item 634 shows that the operator has selected level 4 for the inspection level setting for defects (dots) and level 4 for the inspection level setting for defects (streaks).

[0060] 6(b) is an example of an inspection setting screen when the currently selected setting area is a data area. The setting area 641 is a group of UIs for making settings for the data inspection area, which is displayed when a data inspection area is selected in area 605.

[0061] In this example, the currently selected area is assumed to be character area 607, but similar settings are made for character area 609. Note that area 605 displays selected areas so that they can be distinguished from unselected areas.

[0062] Settings area 642 sets the scope of the selected area. If nothing is selected, the selected inspection area will be placed only on the page currently displayed in area 605. If "Same Side as Current Page" is selected, the selected inspection area will be placed on the same side of the page, depending on whether the selected inspection area is on the front or back side of the sheet. If "All Pages" is selected, the selected inspection area will be placed on all pages.

[0063] The setting area 643 is an area for setting the correct CSV file to be compared when performing a matching test. A file is selected by pressing button 644, and the selected file name is displayed. The specification of the correct CSV file is common to both character and barcode tests. The setting area 645 is used to set the currently selected data test area. Here, the orientation, selection of the character area or barcode test area, font type, barcode type, whether or not to perform a matching test, and specification of the columns of the correct CSV file for the matching test are specified. These settings required for data testing are called test parameters.

[0064] Setting item 646 sets the character orientation in character area 607. Setting items 647 and 648 specify whether the selected area is to be inspected for a character string or a barcode. Setting item 649 sets the font used for OCR processing of character area 607. Setting item 650 sets the type of barcode when barcode inspection is selected in setting item 648.

[0065] Whether or not to perform a matching check is set in setting item 651. If matching check is set to "Yes," OCR processing is performed on the character area 607 using the correct answer CSV file specified in setting area 643 and the column number specified in setting item 652, and a matching check is performed between the read character string and the character string specified in the correct answer CSV.

[0066] Setting item 653 sets whether or not to perform a match check. A match check is a check to see if the results of reading two or more check areas match. For example, a match check area (not shown) for performing a match check is set on the opposite side of the currently selected character area 607, and it is checked whether the results of reading the areas on the front and back match. In particular, performing a match check between a front object and a back object may be called a front-back check or a front-back match check.

[0067] Here, examples of data inspections include a data readability inspection to check whether a character string or barcode can be read, a data verification inspection to check the results of reading a character string or barcode against the correct answer, and a front / back match inspection to check whether the results of reading the front and back sides match. However, the data inspection is not limited to these. For example, it may be a serial number inspection to check whether the read results are serial numbers, or a match inspection to check whether the results of reading multiple points on a surface match, as long as it reads a character string or barcode and performs the inspection.

[0068] <Test setting flow> Next, the inspection setting flow of S403 will be described using the flowchart in Fig. 7. This flowchart is realized by the CPU 302 of the inspection device 110 expanding the program code stored in the ROM 304 into the RAM 303, and reading and executing the program code expanded into the RAM 303.

[0069] In step S701, the CPU 302 receives a notification of an operator's UI operation from the UI unit 320.

[0070] In step S702, the CPU 302 determines whether an inspection setting has been made. Specifically, it determines whether a new data inspection area has been created or whether the setting of an already created data inspection area has been changed. If an inspection setting has been made (YES in step S702), the process proceeds to step S703. If an operation other than inspection setting has been made (NO in step S702), the process proceeds to step S707.

[0071] In step S703, CPU 302 determines whether or not a match check is set in the check setting in step S702. If a match check is set (YES in step S703), the process proceeds to step S704. If a match check is not set (NO in step S703), the process proceeds to step S707.

[0072] In step S704, CPU 302 determines whether a match check area has already been created. The match check area refers to the area where a match check is performed with the data check area set in step S702. If a new data check area has been created, the match check area has not yet been created (YES in step S703), so the process proceeds to step S705. If the setting of the data check area has been changed, the match check area has already been created (NO in step S703), so the process proceeds to step S706.

[0073] In step S705, CPU 302 generates a matching inspection area for the data inspection area set in step S702. The matching inspection area generated here is set with the same inspection settings, such as orientation and character type, as those set for the selected inspection area. The matching inspection area generated here may be generated within the same page as the sheet on which the data inspection area was set in S702, or may be generated on a different page. It may also be generated on a different page of the same sheet.

[0074] In step S706, the CPU 302 accepts a change to the setting of the inspection area by the operator. When changing the setting of the match inspection area, there may be restrictions on the items that can be changed, or confirmation may be required when making a change, but this will be described in detail later.

[0075] The generation of the match check area will be described in detail with reference to FIG. 8. FIG. 8(a) shows the area 605 when the data check area is set in step S702. Image 801 is a reference image, and the QR code in the image is set as the data check area 802. "Setting" refers to a state in which all required settings have been set in the setting area 641 in FIG. 6(b). Required settings include, for example, the orientation, selection of the character area or barcode check area, font type, and barcode type for a data readability check. For a data verification check, the required settings include the orientation, selection of the character area or barcode check area, font type, barcode type, whether or not to perform a verification check, and the column of the correct CSV file for the verification check. It is desirable to enable the match check for the data check area 802 after the required settings described above have been set. This is because the data check area settings are inherited when the match check area is automatically generated, thereby reducing the operator's setup burden. Therefore, the match check setting item 653 is grayed out until the required items are set for the data check area 802. Alternatively, if the matching check setting item 653 is enabled when the required items have not been set, a warning may be issued.

[0076] FIG. 8(b) shows the area 605 when the match check area is set in step S705. Image 803, displayed alongside image 801, is an image of the back side of image 801 and has the same QR code as image 801. When data check area 802 is selected and a match check is set, CPU 302 generates and displays match check area 804. At this time, the selected area shifts from data check area 802 to match check area 804. The settings of match check area 804 are all the same as data check area 802, except that it is located on the back side of data check area 802. Therefore, at the time of generation of match check area 804, match check area 804 does not match the position of the QR code in image 803. At this point, the operator changes the settings of match check area 804.

[0077] In this embodiment, as shown in Fig. 8(b), the front and back images may be displayed side by side in the area 605. The side-by-side display of the front and back images may be switched by pressing a display method switching button (not shown), or may be configured to be switched by pressing the matching check setting item 653.

[0078] FIG. 8(c) shows the match check area 805 after the setting change is complete. The setting change here is a change in the position of the match check area 804. The method for changing the position may be any method that allows the operator to appropriately change the setting of the check area, such as by dragging and dropping the match check area 804 or by the operator reselecting the location of the QR code. The operator may also change other setting items related to data check as needed. However, if a data matching check is set for the data check area 802, it may be possible to not accept changes to the correct answer CSV file in the setting area 643, whether or not to perform a matching check in the setting area 651, and the column number in the setting area 652. Methods for prohibiting setting changes may include graying out items for which setting changes are not accepted, or issuing a warning when a setting is changed, as long as the method notifies the user that the setting cannot be changed.

[0079] Even when changing the inspection settings after completing the setting of the matching inspection area, the CPU 302 displays the front and back images side by side as shown in Figure 8(c) to allow the inspection settings to be changed. When the settings of the data inspection area 802 are changed, whether to apply the setting changes to the matching inspection area 805 can be determined by confirming the change with a pop-up on the UI each time the settings are changed, or by determining this in advance in the initial settings. Also, in Figure 8(b), the data inspection area 802 and the matching inspection areas 804 and 805 are depicted with the same black dashed lines. For example, the automatically generated matching inspection areas 804 and 805 may be displayed with different line color, shape, thickness, action, etc. from the data inspection area 802, and then displayed in the same way as the data inspection area 802 after the settings are complete. If you want to perform a matching inspection on two locations, a character string and a QR code, as shown in Figure 8(d), for example, you would specify the data inspection area 802, then specify the matching inspection, and then generate the data inspection area 805. Furthermore, by specifying a data inspection area 806, then specifying a match inspection, and generating a data inspection area 807, it is possible to specify match inspections in two locations on one sheet.

[0080] Next, in step S707, CPU 302 determines whether the setting of the inspection area has been completed. Here, this determination is made based on whether or not the button (OK button) 611 shown in FIG. 6(a) has been pressed. If button 611 has been pressed (YES in step S707), CPU 302 determines that the inspection setting has been completed, saves the setting values ​​set on the inspection setting screen in RAM 303, and ends the inspection setting process. If the OK button has not been pressed (NO in step S707), the process returns to step S701, and CPU 302 waits for notification of a UI operation. This concludes the description of the inspection setting flow in step S403.

[0081] [Match Check] So far, the setting flow for the match check has been explained, but the contents of the match check and the display method will be explained.

[0082] As mentioned above, a match test is a type of data test that checks whether the data matches. Therefore, the data does not have to be displayed in the same format; it can be text strings, or a text string and a barcode, as long as the final data read results match.

[0083] There are several possible methods for the match check. Here, if a match check is set for data A and data B in certain print data, the check is performed for each of data A and data B using the check method described below. First, we will explain the case where only checking whether the read results of the areas for which the match check is set match will be explained. CPU 302 sequentially saves the read results of all areas for which the match check is set in RAM 303, compares the saved read results, and determines which data matches and which data does not match. If there is even one piece of data that does not match, the match check is NG.

[0084] Next, we will explain the case where all matching test areas are checked to see that the same CSV data is correct. The CPU 302 sequentially saves the results of the data matching test for all areas set for matching test in the RAM 303 and determines whether any result is NG. If there is even one NG result, the matching test is NG.

[0085] Next, we will explain the case where a data matching test is performed on only one matching test area, and the other matching test areas are checked to see if their read results match those of the test areas where the data matching test was performed. The CPU 302 saves the results of the data matching test and the read results of all matching test areas, compares the saved read results, and determines whether the data matches or does not match. If the data matching test is NG or there is even one mismatched data, the matching test is NG.

[0086] An example of a method for displaying the match check area will be described using Fig. 8(d). Fig. 8(d) shows an example in which a match check for data A and a match check for data B are set for the front and back of a printed material.

[0087] Characters 806 and 807 are the subject of a matching test separate from data A displayed in the QR code of the above-mentioned test areas 802 and 805. Also, icons 808 and 809 are icons added to the matching test areas. Adding the same icon to the matching test areas makes it easier to distinguish between the areas where a matching test is to be performed. Also, when multiple matching test areas are set on a page, they can be distinguished by changing the numbers on the icons. The method of displaying the matching test areas is not limited to this, and any method that makes it easy to recognize and distinguish between the areas where a matching test is to be performed, such as adding a mark or changing the pattern, thickness, or color of the enclosing line, may be used.

[0088] As described above, according to this embodiment, when the matching inspection is enabled, the front and back images are displayed side by side on the setting screen, and at the same time, a matching inspection area is automatically generated and displayed on the back, thereby significantly reducing the load involved in setting the matching inspection area.

[0089] <Embodiment 2> In the first embodiment, a method was described in which a matching inspection area for an area selected by an operator is automatically generated, and the operator then makes final setting changes to the automatically generated area. However, because the generated matching inspection area has the same settings as the data inspection area, the position of the matching inspection area may be misaligned or the character orientation may be different. Specifically, even if the area for which the matching inspection is performed within a page is the same on the front and back, the position of the inspection area may differ depending on the imposition during printing or the binding direction during double-sided printing. Alternatively, although the content cannot be positioned in the same position due to the configuration of the content within the page, they may be arranged with a certain degree of regularity, taking into account design and visibility.

[0090] In this embodiment, a method will be described in which the accuracy of setting the automatically generated coincidence inspection area is improved by taking the above-mentioned regularity into consideration, and manual setting by the operator is reduced.

[0091] The following describes the second embodiment, focusing on differences from the first embodiment. Note that parts not described in detail are the same as the first embodiment.

[0092] <Flow of generating a match check area> The inspection setting flow in step S403 in this embodiment is as shown in the flowchart in Fig. 7, but the flow of generating a match inspection area in step S705, which is a feature of this embodiment, will be described using the flowchart in Fig. 9. This flowchart is realized by the CPU 302 of the inspection device 110 expanding the program code stored in ROM 304 into RAM 303, and reading and executing the program code expanded in RAM 303.

[0093] In step S901, CPU 302 receives print settings of the print job executed for registering a reference image from RAM 303. The print settings are sent to image forming apparatus 100 via network 150 from client PC 130 or print server 140, on which the print job for registering a reference image was executed. The print settings are then sent to inspection apparatus 110 via communication I / F unit 301 and stored in RAM 303. Specifically, the print settings refer to the image orientation (portrait / landscape), imposition, and binding settings for double-sided printing.

[0094] In step S902, the CPU 302 generates a match check candidate area based on the print settings. Several examples of generating a match check candidate area will be described using FIG. 10. Here, it is assumed that the target of the match check is a QR code, and that the position within the page is the same. Images 1000 and 1001 in FIG. 10(a) show images of the front and back sides when the image orientation is portrait, the imposition is one page, and the document is bound on the short edge. If the print settings received in step S901 are in the above conditions, the match check candidate area for the data check area 1002 will be 1003. The setting of the match check candidate area 1003 is a position point-symmetrical to the data check area 1002, and because the page is upside down, the orientation setting is rotated 180 degrees.

[0095] Images 1004 and 1005 in Figure 10(b) show images of the front and back sides of a two-page print with portrait orientation and short-edge binding. If the print settings received in step S901 are as described above, the match inspection candidate area for data inspection area 1006 is 1007. The back side of the left page of image 1004 is the right page of image 1005. Therefore, the match inspection candidate area 1007 is set to a position shifted to the right of data inspection area 1006 by half the distance of the long side of the printing paper.

[0096] As described above, on the assumption that the areas to be subjected to the match test are located at the same position within the plane, the CPU 302 determines the estimated position and orientation according to the print settings to generate the match test candidate areas. Next, the CPU 302 checks the accuracy of the match test candidate areas.

[0097] In step S903, the CPU 302 analyzes the data inspection area and stores the analysis results in the RAM 303. The analysis here may be any analysis method that can extract features within an area and check the degree of match between areas, such as OCR, barcode analysis, or pixel value histogram analysis.

[0098] In step S904, the CPU 302 analyzes the match check candidate region and stores the analysis result in the RAM 303.

[0099] In step S905, the CPU 302 reads out the analysis results from step S903 and step S904 from the RAM 303 and compares them. If the results are OCR results or barcode analysis results, the contents are compared to determine whether they match. If the features are extracted based on pixel values ​​such as histograms, a threshold value for the degree of match (e.g., 90%) is set, and if the threshold value is met, it is determined that there is a match, even if there is not a perfect match (100%). If the analysis results match (YES in step S905), the CPU 302 ends the process of generating the match check area. If the analysis results do not match, the CPU 302 saves the coordinates of the match check candidate area and the mismatch result in the RAM 303, and repeats the process from step S904 onwards for other candidate patterns of predetermined coordinates.

[0100] The candidate coordinate patterns will be described with reference to FIG. 10(c). The candidate patterns are used to determine coordinate positions on the assumption that data is arranged with regularity. The inspection area 1008 is set in step S902, and is assumed to be the position where the analysis results are determined not to match in step S905, with its center coordinates being (x, y). The coordinates of the upper left corner of the image are (0, 0), and the coordinates of the lower right corner are (X, Y). The center coordinates of the first candidate pattern 1009 are (Xx, y), the center coordinates of the second candidate pattern 1010 are (Xx, Yy), and the center coordinates of the third candidate pattern 1011 are (x, Yy).

[0101] In step S906, the CPU 302 reads the results from the RAM 303 and determines whether there was a mismatch for all of the candidate patterns 1009, 1010, and 1011. If the results for all of the candidate patterns are a mismatch (YES in step S906), the CPU 302 ends the process of generating the match check area. At this time, the CPU 302 sets the match check candidate area first generated in step S902 as the match check area and ends the process. If there is a candidate pattern for which a mismatch has not yet been determined (NO in step S906), the CPU 302 changes the position of the match check candidate area in step S907.

[0102] In step S908, the CPU 302 determines whether or not an overlap error has occurred with the newly set match inspection candidate area with another inspection area. Specifically, if the print image inspection area and the data inspection area overlap, the CPU 302 determines the overlap and displays a warning (not shown) on the inspection setting screen 600. If no warning has been issued (NO in step S908), the CPU 302 returns to step S904 and performs processing. If a warning has been issued (YES in step S908), the CPU 302 saves the coordinates of the candidate pattern and the mismatch result in the RAM 303, and returns to step S906 and performs processing. This concludes the description of the flow for generating a match inspection area in embodiment 2.

[0103] According to this embodiment, by setting the match inspection area taking into account the print settings, it is possible to set it more accurately than in embodiment 1. Furthermore, by determining and analyzing candidate coordinates in consideration of the regularity of the design, the likelihood of correctly setting the match inspection area increases. As a result, it is possible to reduce the burden on the operator involved in setting the match inspection area.

[0104] <Embodiment 3> In the second embodiment, a method for generating a matching inspection area taking print settings and layout into consideration was described. In addition, analysis was performed when the matching inspection area was generated, making it possible to improve the accuracy of the setting. However, the method of the second embodiment can only check a limited area within the image.

[0105] In this embodiment, a method is described in which the accuracy of setting the matching inspection area is improved and manual setting by the operator is reduced by using image search to more flexibly search and set the matching inspection area from the entire image.

[0106] The following describes the third embodiment, focusing on differences from the first and second embodiments. Note that parts not described in detail are the same as the first and second embodiments.

[0107] <Flow of generating a match check area> The inspection setting flow in step S403 in this embodiment is as shown in the flowchart in Fig. 7, but the flow of generating a match inspection area in step S705, which is a feature of this embodiment, will be described using the flowchart in Fig. 11. This flowchart is realized by the CPU 302 of the inspection device 110 expanding the program code stored in ROM 304 into RAM 303, and reading and executing the program code expanded in RAM 303.

[0108] In step S1101, CPU 302 performs resolution conversion to lower the resolution of the image in the data inspection area and the back side, and stores the converted image in RAM 303. The resolution is lowered to reduce the load on the image search process and speed up the process. For example, if the resolution of the image registered as the reference image is 300 dpi, resolution conversion is performed to 100 dpi using the bicubic method. However, the resolution conversion method is not limited to this, and any common method may be used as long as it does not significantly impair the characteristics of the image. Note that since the bicubic method is a common method, a detailed explanation will be omitted.

[0109] Next, in step S1102, the CPU 302 reads the low-resolution image stored in RAM 303 and performs an image search. For example, the image search uses the SSIM (Structure SIMilarity) method to search the backside image for areas highly similar to the data inspection area. However, the image search method is not limited to this, and other methods, such as the PSNR (Peak Signal to Noise Ratio) method, may also be used. Since SSIM is also a common method, a detailed description is omitted. The specific image search method involves calculating SSIM evaluation values ​​while scanning the search area from the edge of the backside image, and sequentially saving the SSIM evaluation values ​​in an evaluation value list (not shown) generated in RAM 303. The evaluation value list associates and saves the coordinates of the search area in the low-resolution image with the evaluation value. When saving the evaluation value in the evaluation value list, the calculated value is compared with the value in the evaluation value list, and only if the calculated value is greater, the value in the evaluation value list is overwritten. When the entire image has been searched, the evaluation value list is saved in order of highest to lowest evaluation value, so that at least one data remains.

[0110] In step S1103, the CPU 302 analyzes the data inspection area of ​​the original image registered as the reference image. In step S1104, the CPU 302 calculates the coordinates of the 300 dpi image registered as the reference image from the coordinates in the evaluation value list, and analyzes the match inspection candidate area of ​​the original image registered as the reference image. Here, the coordinates read from the evaluation value list are those with the highest evaluation value.

[0111] The analysis and determination in step S1105 here are the same as steps S903 to S905 in the second embodiment, and therefore will not be described in detail. If the analysis result indicates a match (YES in step S1105), the CPU 302 ends the process of generating the match check area and deletes the evaluation value list from RAM 303. If the analysis result indicates a match (NO in step S1105), the CPU 302 determines in step S1106 whether all candidates stored in the evaluation value list have been analyzed. If all candidates have been analyzed (YES in step S1106), the CPU 302 ends the process of generating the match check area and deletes the evaluation value list from RAM 303. At this time, the CPU 302 sets the match check area at the same coordinates as the data check area and ends the process. If there are any candidates left to be analyzed (NO in step S1106), the processes of steps S1103 to S1105 are repeated for the coordinates with the next highest evaluation value in the evaluation value list. If the analysis results of all the candidates do not match, the process may return to step S1101 and the same process may be performed at a higher resolution (for example, 200 dpi). This concludes the description of the flow for generating a match check area in the third embodiment.

[0112] According to this embodiment, by using image search to search and set the matching inspection area from the entire image, the likelihood of correctly setting the matching inspection area is increased, thereby reducing the burden on the operator in setting the matching inspection area.

[0113] <Embodiment 4> In the third embodiment, a method for generating a match inspection area using image search has been described. However, image search and analysis take time, which may cause the operator to wait. There is a concern that the increased time required for setting up may actually increase the setting burden on the operator.

[0114] In this embodiment, we will explain a method for reducing the burden on the operator in setting the matching inspection area by having the operator specify the setting method at the beginning of setting the matching inspection area, thereby setting the matching inspection area more reliably and in a shorter time.

[0115] The following describes the fourth embodiment, focusing on differences from the first to third embodiments. Note that parts not described in detail are the same as the first embodiment.

[0116] <Flow of generating a match check area> The test setting flow in step S403 in this embodiment is as shown in the flowchart in FIG. 7, but the flow of generating the match test area in step S705, which is a feature of this embodiment, will be described with reference to FIG.

[0117] Reference numeral 1200 denotes a selection screen for setting the matching inspection area. Selection screen 1200 shows five setting methods, 1201 to 1205. Image 1206 with the letter "A" written on it is an image of the front side, and image 1207 with the letter "B" written on it is an image of the back side, and each is surrounded by a dotted frame indicating that the QR code is the inspection area.

[0118] 1201 indicates that the position and orientation of the matching inspection areas on the front and back sides are the same. 1202 indicates that the positions of the matching inspection areas on the front and back sides are symmetrical and the orientation is the same.

[0119] 1203 indicates that the positions of the matching inspection areas on the front and back sides are point-symmetric and the orientation is rotated by 180 degrees. 1204 indicates that the positions of the matching inspection areas on the front and back sides are manually set by the operator and the orientation is the same.

[0120] 1205 indicates that the positions of the coincidence inspection areas on the front and back sides are manually set by the operator, and the orientation is rotated by 180 degrees.

[0121] When generating a match check area in step S705, the CPU 302 first displays a selection screen 1200 on the test setting screen 600. The display method may be a method of displaying it as an image in the area 605, or a method of displaying it as a radio button in the setting area 641. After the operator selects one of the setting methods 1201 to 1205, the CPU 302 generates and displays a match check area according to the selected setting method. For example, when the setting method 1202 is selected, the CPU 302 displays FIG. 8(c) in the area 605. Alternatively, when the setting method 1204 is selected, the CPU 302 displays FIG. 8(b) in the area 605. The processing from step S706 onwards is the same as in the first embodiment.

[0122] The above is the description of the flow of generating the match check area in the fourth embodiment.

[0123] According to this embodiment, by having the operator specify the method for setting the matching inspection area, it is possible to set the matching inspection area more reliably and in a shorter time, thereby reducing the burden on the operator in setting the matching inspection area.

[0124] <Embodiment 5> In the first to fourth embodiments, a method for checking the coincidence between the front and back sides has been described. Also, the coincidence check area is one on the front side and one on the back side. However, in reality, coincidence checks may be performed in multiple locations on the front or back side, or in three or more locations, such as one location on the front side and two locations on the back side.

[0125] In this embodiment, a method will be described that is not limited to the front and back sides and also applies to cases where there are three or more coincidence inspection areas.

[0126] The following describes the fifth embodiment, focusing on differences from the first to fourth embodiments. Note that parts not described in detail are the same as the first embodiment.

[0127] <Flow of generating a match check area> The test setting flow in step S403 in this embodiment is as shown in the flowchart in FIG. 7, but the flow of generating the match test area in step S705, which is a feature of this embodiment, will be described with reference to FIG.

[0128] 1300 in FIG. 13(a) is a selection screen for selecting the type of match check, and 1306 in FIG. 13(b) is a detailed setting screen for setting the details of the match check.

[0129] The front and back inspection button 1301 is a selection button for setting one matching inspection area on the front side and one on the back side. The front and back inspection (multiple) button 1302 is a selection button for setting one matching inspection area on the front side and one on the back side, plus one or more additional matching inspection areas on the front side and / or back side.

[0130] The in-plane inspection button 1303 is a selection button for coincidence inspection that sets two or more coincidence inspection areas within either the front or back surface.

[0131] When generating the match inspection area in step S705, the CPU 302 first displays the selection screen 1300 on the inspection setting screen 600. The display method may be a method of displaying it as an image in the area 605 or a method of displaying it as a radio button in the setting area 641.

[0132] When the front / back inspection button 1301 is selected, the CPU 302 generates one match inspection area on the back side and proceeds to the process of step S706. At this time, the match inspection area may be generated by any of the methods described above.

[0133] When the front and back inspection (multiple) button 1302 is selected, the CPU 302 displays a detailed setting screen 1306 on the inspection setting screen 600 .

[0134] An area 1307 is a window that displays the number of matching inspection areas to be set on the front side, and the initial value is "1." An area 1308 is a window that displays the number of matching inspection areas to be set on the back side, and the initial value is "1."

[0135] Triangle button 1309 is a button for increasing the number of regions 1307, and each time it is pressed, CPU 302 increments the number of regions 1307 by 1 and redisplays them. Triangle button 1310 is a button for decreasing the number of regions 1307, and each time it is pressed, CPU 302 decrements the number of regions 1307 by 1 and redisplays them.

[0136] Triangle button 1311 is a button for increasing the number of regions 1308, and each time it is pressed, CPU 302 increments the number of regions 1308 by 1 and redisplays them. Triangle button 1312 is a button for decreasing the number of regions 1308, and each time it is pressed, CPU 302 decrements the number of regions 1308 by 1 and redisplays them.

[0137] An OK button 1313 is an OK button for saving the settings on the detailed setting screen 1306 in RAM 303 and transitioning to the inspection setting screen 600. After the OK button 1313 is pressed, the CPU 302 generates inspection areas according to the number of matching inspection areas on the front and back sides saved in RAM 303 and displays them in the area 605. In this case, the settings of the matching inspection areas may all be the same, but displaying them so that they do not overlap will make it easier for the operator to recognize multiple matching inspection areas.

[0138] When the intra-surface inspection button 1303 is selected, the CPU 302 displays a detailed setting screen 1306 on the inspection setting screen 600. At this time, if the surface displayed in area 605 is the front surface, area 1308, triangle button 1311, and triangle button 1312 used for setting the back surface are unnecessary and are therefore grayed out. If the surface displayed in area 605 is the back surface, area 1307, triangle button 1309, and triangle button 1310 used for setting the front surface are unnecessary and are therefore grayed out. After the number of inspection areas for the front or back surface has been set and the OK button 1313 has been pressed, the processing is as described above.

[0139] The above is the description of the flow of generating the match check area in the fifth embodiment.

[0140] According to this embodiment, it is possible to set the coincidence check for various cases, and also to reduce the burden on the operator in setting the coincidence check area.

[0141] (Other embodiments) While various examples and embodiments of the present invention have been shown and described above, the spirit and scope of the present invention are not limited to the specific descriptions within this specification.

[0142] This embodiment can also be realized by supplying a program that realizes one or more functions of the above-described embodiment to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

Claims

[Claim 1] An inspection device that extracts data from an image included in a plurality of regions and performs a coincidence inspection to inspect whether the extracted plurality of data matches, a storage means for storing a reference image; an inspection setting means for setting, in response to an operator's operation, a data inspection setting to be used in the coincidence inspection and a first area to be used as an inspection area in the coincidence inspection, for the reference image; The inspection device is characterized in that the inspection setting means automatically creates a second area corresponding to the first area in the reference image.

Images