Information processing device, storage medium, and information processing method

Abstract

An information processing device, a storage medium, and an information processing method, wherein the information processing device is characterized in that it includes a processor that performs the following processing: when there are multiple patterns of an original document area extracted from an input image obtained by photographing an original document, executing display control for displaying these multiple patterns as options on a screen; receiving selection of a correct pattern from the multiple patterns based on the display of the screen based on the display control; and extracting an image of the original document area contained in the pattern selected by the user from the input image.

Description

Technical Field

[0001] The present invention relates to an information processing device, a storage medium and an information processing method. Background Art

[0002] Some scanners and multifunction peripherals (i.e., devices that combine scanner, printer, and copier functions) have the ability to read multiple documents placed on a document table, also called a platen, and then cut out images of each document from the read images and convert them into digital form. This function is called a multi-cropping function.

[0003] Conventional devices have used methods such as covering a plurality of manuscripts placed on a manuscript table with black background paper to increase the contrast between the peripheral edges of the manuscripts and the background, thereby improving the accuracy of cropping images of each manuscript.

[0004] However, people often forget to cover the top of the manuscript group with black background paper. Multifunction printers and the like are equipped with a manuscript cover (which usually has an automatic document feeder built in) that can be opened and closed relative to the manuscript table, and the surface of the manuscript cover facing the manuscript table is usually white. If you forget to cover the multiple manuscripts on the manuscript table with black background paper, and close the manuscript cover as usual to read them, the image of the read result will show a state where multiple white manuscripts are arranged on a white background. The edges of the manuscripts are usually not clearly shown in the image of the read result. When the edges of the manuscripts are not clear, the cutting accuracy of the images of the individual manuscripts deteriorates. For example, errors such as cutting multiple different manuscripts as one large manuscript may occur.

[0005] Furthermore, even when using black background paper to improve cutting accuracy, errors can sometimes occur during cutting. For example, if multiple documents are neatly placed on the document table with no gaps or slightly overlapping, the multi-cut function may cut these documents as a single document.

[0006] Some conventional devices having a multi-cropping function have a function of presenting the cropping result to the user.

[0007] For example, in the apparatus disclosed in Patent Document 1, the area of each document is detected from an image obtained by pre-scanning the entire surface of the document table at a low resolution, and the main scan is performed after the detected area of the document is displayed (see paragraphs 0025, 0061, and 0079 of the document). Figure 2 ).

[0008] Furthermore, as documents showing conventional techniques related to multi-cutting, there are Patent Document 2 and Patent Document 3.

[0009] Patent Document 1: Japanese Patent No. 4574503

[0010] Patent Document 2: Japanese Patent Application Publication No. 2018-201174

[0011] Patent Document 3: Japanese Patent Application Publication No. 2019-092065

[0012] If the document area extracted from the input image obtained by photographing the document is simply displayed, the user can only confirm whether the displayed document area is correct. If the displayed document area is incorrect, the user can perform an operation such as resetting the document on the document table and re-photographing the document to extract the correct document area, but this operation is complicated for the user. Summary of the Invention

[0013] An object of the present invention is to reduce the user's operational burden for extracting a correct document region, compared to a method of simply displaying a document region extracted from an input image obtained by photographing a document.

[0014] The invention involved in Option 1 is an information processing device, characterized in that it includes a processor, which performs the following processing: when there are multiple patterns of the original document area extracted from the input image obtained by photographing the original document, the processor performs display control for displaying these multiple patterns as options on the screen; receives the selection of a correct pattern from the multiple patterns based on the display of the screen based on the display control; and extracts the image of the original document area contained in the pattern selected by the user from the input image.

[0015] The invention described in claim 2 is the information processing device described in claim 1, wherein the display control is a control for displaying an image showing the plurality of patterns simultaneously on the screen.

[0016] The invention described in Option 3 is the information processing device described in Option 2, characterized in that the image is an image that establishes a corresponding association with each of the multiple patterns and further displays a UI component for selecting the pattern, and in the process of receiving the selection, the pattern corresponding to the UI component selected by the user in the image displayed on the screen is used as the correct pattern.

[0017] The invention involved in Option 4 is an information processing device described in Option 2 or 3, wherein, when the number of patterns in the original area extracted from the input image is greater than a prescribed threshold, the processor performs display control different from the control for displaying an image of the plurality of patterns simultaneously on the screen.

[0018] The invention involved in Scheme 5 is an information processing device described in any one of Schemes 1 to 4, characterized in that the display control is a control for displaying an image showing the first pattern among the multiple patterns on the screen, and the processor also performs the following processing: receiving the selection of one or more correct original areas in the original area contained in the first pattern displayed in the image; executing a second receiving process, in which the designation of the original area contained in the part of the input image other than the one or more selected correct original areas is received by a method different from the method of selecting from the first pattern; and extracting each image of the one or more selected correct original areas and the image of each original area specified by the different method from the input image.

[0019] The invention according to claim 6 is the information processing device according to claim 5, wherein the second receiving process is a process of receiving selection of a correct pattern from among the plurality of patterns including the selected one or more correct document regions.

[0020] The invention involved in Scheme 7 is the information processing device described in Scheme 6, wherein, in the second receiving process, when the number of patterns including the one or more correct original areas selected is below a specified threshold value, images of these patterns including the one or more correct original areas are displayed simultaneously and an image of a UI component for selecting the pattern is displayed on the screen by establishing a corresponding association with each of the simultaneously displayed patterns, and the pattern corresponding to the UI component selected by the user in the image displayed on the screen is used as the correct pattern.

[0021] The invention according to claim 8 is the information processing apparatus according to claim 5, wherein in the second receiving process, a designation for specifying a vertex or an edge of a document area included in a portion other than the one or more selected correct document areas is received.

[0022] The invention involved in Scheme 9 is an information processing device described in any one of Schemes 1 to 4, characterized in that the display control is a control for displaying an image showing the first pattern among the multiple patterns on the screen, and the processor also performs the following processing: receiving the selection of one or more erroneous original manuscript areas in the original manuscript area contained in the first pattern displayed in the image; executing a second receiving process, in which the designation of the original manuscript area contained in the part of the input image corresponding to the selected one or more erroneous original manuscript areas is received by a method different from the method of selecting from the first pattern; and extracting from the input image the images of the remaining original manuscript areas other than the one or more erroneous original manuscript areas in the first pattern and the images of the original manuscript areas specified by the different methods.

[0023] The invention involved in Scheme 10 is the information processing device described in Scheme 9, wherein the second receiving process is a process of receiving the selection of a correct pattern from the patterns of the remaining manuscript areas in the plurality of patterns except for the one or more selected erroneous manuscript areas.

[0024] The invention involved in Scheme 11 is an information processing device described in Scheme 1, characterized in that the processor performs the following processing: receiving the designation of vertices or edges of at least one original area contained in the input image displayed on the screen; and extracting the image of the at least one original area determined by the designated vertices or edges from the input image.

[0025] The invention involved in Scheme 12 is a storage medium storing a program for causing a computer to operate in the following manner: when there are a plurality of patterns of a document area extracted from an input image obtained by photographing a document, display control is executed for displaying these plurality of patterns as options on a screen, selection of a correct pattern from the plurality of patterns is received based on the display of the screen based on the display control, and an image of the document area included in the pattern selected by the user is extracted from the input image.

[0026] The invention involved in Scheme 13 is an information processing method, which includes the following steps: when there are multiple patterns of the original area extracted from the input image obtained by photographing the original, performing display control for displaying these multiple patterns as options on the screen, receiving the selection of a correct pattern from the multiple patterns based on the display of the screen based on the display control, and extracting the image of the original area contained in the pattern selected by the user from the input image.

[0027] Effects of the Invention

[0028] According to the first, twelfth, or thirteenth aspect of the present invention, the user's operational burden for extracting a correct document region can be reduced compared to a method of simply displaying a document region extracted from an input image obtained by photographing the document.

[0029] According to the second aspect of the present invention, the number of screen switching times for selecting a correct pattern can be reduced compared to a method of switching and displaying a plurality of patterns.

[0030] According to the third aspect of the present invention, compared with a method of selecting correct document areas one by one, a correct pattern composed of correct document areas can be selected with fewer operations.

[0031] According to the fourth aspect of the present invention, when a plurality of patterns are displayed simultaneously and it is difficult to visually distinguish the patterns from each other, a display other than the simultaneous display can be performed.

[0032] According to the fifth aspect of the present invention, it is possible to accept selection of a correct document area among document areas within the first pattern in an image displaying the first pattern, and accept designation of an incorrect document area within the first pattern by another method.

[0033] According to the sixth aspect of the present invention, the number of operations for selecting a correct pattern can be reduced compared to selecting a correct pattern from all of a plurality of patterns.

[0034] According to the seventh aspect of the present invention, the number of screen switching operations required to select a correct pattern can be reduced compared to a method of switching and displaying a plurality of patterns. Compared to a method of selecting correct manuscript areas one by one, a correct pattern consisting of correct manuscript areas can be selected with fewer operations.

[0035] According to the eighth or eleventh aspect of the present invention, it is possible to receive from the user the identification of a document area other than the correct document area selected.

[0036] According to the ninth aspect of the present invention, it is possible to define document areas other than the erroneous document area selected from the first pattern as correct document areas, and receive designations of other document areas by other methods.

[0037] According to the tenth aspect of the present invention, the number of operations for selecting a correct pattern can be reduced compared to selecting a correct pattern from all of a plurality of patterns. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Embodiments of the present invention will be described in detail with reference to the following drawings.

[0039] Figure 1 This is a diagram illustrating the hardware configuration of a computer included in the information processing device;

[0040] Figure 2 A diagram illustrating a portion of processing steps performed by a processor of an information processing device;

[0041] Figure 3 A diagram illustrating a portion of processing steps performed by a processor of an information processing device;

[0042] Figure 4 A diagram illustrating a portion of processing steps performed by a processor of an information processing device;

[0043] Figure 5 A diagram illustrating a portion of processing steps performed by a processor of an information processing device;

[0044] Figure 6 FIG is a diagram showing an example of data representing a pattern extraction result;

[0045] Figure 7 FIG. 1 is a diagram showing an example of the arrangement of a group of original documents on a platen;

[0046] Figure 8 This is a diagram illustrating a confirmation UI screen;

[0047] Figure 9 FIG. 1 is a diagram showing another example of the arrangement of the document group on the platen;

[0048] Figure 10 This is a diagram illustrating a simultaneous display type UI screen;

[0049] Figure 11 This figure shows a state where one pattern is selected on the simultaneous display type UI screen;

[0050] Figure 12 FIG. 1 is a diagram showing another example of the arrangement of the document group on the platen;

[0051] Figure 13 This is a diagram illustrating a switching display type UI screen;

[0052] Figure 14 This is a diagram showing a state where one document area is selected on the switching display type UI screen;

[0053] Figure 15 This is a diagram showing a state where another pattern having one selected document area is displayed on the switching display type UI screen;

[0054] Figure 16 This figure shows a state where two document areas are selected on the switching display type UI screen;

[0055] Figure 17 This is a diagram showing a state where another pattern having two selected document areas is displayed on the switching display type UI screen;

[0056] Figure 18 This is a diagram showing a state where all displayed document areas are selected on the switching display type UI screen;

[0057] Figure 19 FIG. 1 is a diagram showing another example of the arrangement of the document group on the platen;

[0058] Figure 20 is an example of Figure 19 FIG shown is a diagram of the UI screen displayed when the platen is scanning;

[0059] Figure 21 This is a diagram illustrating a UI screen in which all document areas displayed are in a selected state in response to pressing of the all selection button;

[0060] Figure 22 This is a diagram illustrating a UI screen when one selection in all selected document areas is released;

[0061] Figure 23 This example shows that there are Figure 22 A diagram of another UI screen showing a common document area in a selected state;

[0062] Figure 24 This example selects Figure 23 FIG. 1 is a diagram of a UI screen showing two document areas in an unselected state;

[0063] Figure 25 This is a diagram illustrating a UI screen when the correct document area is not present among the extracted document areas;

[0064] Figure 26 This is a diagram illustrating a UI screen for relocation guidance;

[0065] Figure 27 This figure illustrates a UI screen for specifying a rectangle;

[0066] Figure 28 This figure illustrates a state where three vertices are selected on the rectangle designation UI screen;

[0067] Figure 29 This is a diagram illustrating a state in which one document area is specified on the rectangle designation UI screen.

[0068] Explanation of symbols

[0069] 10-Pressing plate, 12-Original, 102-Processor, 104-Memory, 106-Auxiliary storage device, 108-Input / output device, 110-Network interface, 112-Bus, 114-Scanner control circuit, 116-Printer control circuit, 118-Fax device, 200-UI screen, 210-Scanned image, 212-Original image, 214, 216-Original area, 220-Cut button, 222-Replace button, 224-Rectangle designation button, 226-"Select pattern 1" button, 228-"Select pattern 2" button, 230-Pattern quantity display bar, 232-Previous candidate button, 234-Next candidate button, 236-Select all button, 238-Release all button. DETAILED DESCRIPTION

[0070] <Hardware configuration example>

[0071] The hardware configuration of the information processing device of this embodiment is shown in FIG. Figure 1 . Figure 1 The example shown is an example of a so-called multifunction peripheral (MFP). A MFP is a device that combines the functions of a printer, scanner, copier, and fax machine. MFPs may also have the function of receiving requests from clients such as personal computers via a network such as a local area network or communicating with servers on the Internet.

[0072] In the information processing device, for example, Figure 2 As shown, the computer has the following circuit configuration: a processor 102 as hardware, a memory (primary storage device) 104 such as random access memory (RAM), a controller that controls an auxiliary storage device 106 such as a flash memory, an SSD (solid state drive), or an HDD (hard disk drive) as a non-volatile storage device, interfaces with various input / output devices 108, and a network interface 110 for controlling connections to a network such as a local area network, all connected via a data transmission path such as a bus 112. The input / output devices 108 include, for example, a display device that also serves as an input device, such as a touch panel, a voice output device, such as a speaker, and a card reader for user authentication. The circuit configuration described above can be the same as that of a general-purpose computer.

[0073] The information processing device also includes a scanner control circuit 114, a printer control circuit 116, and a facsimile device 118, all connected to its computer portion via a bus 112 or the like. These components are used to perform the various functions of the information processing device (in this example, a multifunction peripheral). The scanner control circuit 114 controls the scanner or automatic document feeder built into the multifunction peripheral, while the printer control circuit 116 controls the printer built into the multifunction peripheral. Furthermore, the facsimile device 118 performs the fax transmission and reception functions of the multifunction peripheral.

[0074] The computer portion of the information processing device performs information processing for UI (user interface) processing, data exchange control via a network, and control of various functional elements such as scanners, printers, and facsimile machines. Programs describing the contents of these various information processing operations are installed in the computer via a network, etc., and stored in auxiliary storage device 106. The information processing device of this embodiment is implemented by processor 102 executing the programs stored in auxiliary storage device 106 using memory 104.

[0075] Here, the processor 102 refers to a processor in a broad sense, including a general-purpose processor (e.g., CPU: Central Processing Unit, etc.) or a dedicated processor (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0076] Furthermore, the operations of the processor 102 may be performed not only by a single processor 102 but also by a plurality of processors 102 located in physically distant locations in cooperation. Furthermore, the operations of the processor 102 are not limited to the sequence described in the following embodiments and may be modified as appropriate.

[0077] The processing in this embodiment is for processing images captured by a camera mechanism (e.g., a scanner connected to the scanner control circuit 114) of the information processing device. Therefore, the information processing device may not include a printer and the printer control circuit 116 or the fax device 118 that controls them. Below, the case where the information processing device is implemented as a multifunction peripheral is described as the main example, but this is only one example. The information processing device can be any device as long as it has a camera mechanism, for example, it can be a scanner, an information kiosk terminal, a smartphone, a tablet terminal, a personal computer, etc.

[0078] <Example of Document Image Extraction Processing>

[0079] Below, reference Figures 2 to 5 An example of a process of extracting an original image from a scanned image obtained by scanning using a scanner will be described with reference to a flowchart of FIG.

[0080] The "multi-cut" process begins when a user places one or more documents on the platen (i.e., document table) of a scanner attached to an information processing device and instructs the information processing device to start executing the multi-cut process. Documents that can be multi-cut include, but are not limited to, business cards and receipts.

[0081] In this process, Figure 2 As shown in the example, the processor 102 first causes the scanner to perform scanning via the scanner control circuit 114 (S10). As a result, the processor 102 obtains an image of the result of scanning the entire platen (hereinafter referred to as a scanned image).

[0082] Next, processor 102 extracts one or more document areas from the scanned image (S12). The document area refers to the area of the scanned image containing the image of the document placed on the platen (i.e., the document image). For example, if the document is a receipt, the area representing the receipt image is the document area. The scanned image is an image containing one or more document images within a background image (e.g., an image of a cover covering the platen). In S12, the area containing the document images is extracted from this scanned image, i.e., the document area. Documents are often rectangular, so the document area is hereinafter referred to as a "rectangle" or "rectangular area."

[0083] The document area extraction in S12 is performed using conventionally known techniques. Examples of such techniques include a method of obtaining each document area based on edges extracted from a scanned image by edge extraction processing.

[0084] In the manuscript area extraction process of S12, there are often several possibilities in the manuscript area obtained based on the edge, etc. Figure 9 In the example shown, two rectangular manuscripts 12 of equal width are arranged close to each other on the pressure plate 10 in such a manner that the two side edges are arranged on the same straight line. When the manuscript area extraction process is performed on the scanned image obtained by scanning this state, not only the pattern consisting of the two manuscript areas corresponding to the two manuscripts 12 is recognized, but sometimes the pattern of the two manuscripts 12 being regarded as a continuous one manuscript area is also recognized. When the boundary between the two manuscripts 12 does not appear clearly on the scanned image, the possibility of recognizing the latter pattern is high. Therefore, in the manuscript area extraction process, a score representing the reliability of the pattern of the manuscript area recognized from the scanned image is calculated. In the calculation process of this score, the method used in the previous multi-cropping process can be used. In the previous multi-cropping process, the pattern with the highest score is automatically selected, and the images of each manuscript area shown by the pattern are extracted and output.

[0085] In contrast, in this embodiment, the pattern with the highest score is not automatically selected. Instead, when there are a plurality of patterns with relatively high scores, these patterns are presented to the user, and the user is asked to select the correct pattern.

[0086] Figure 6 , the result of processing S12 is shown in the figure, and the pattern extraction result data created by the processor 102 in the memory 104. The extraction result data in this example includes information on the pattern number, score, number of regions, region ID, and region shape for each pattern. The pattern number is the serial number of the pattern. The score is as described above. The number of regions is the number of manuscript regions included in the pattern, and is referenced, for example, in the determination of S16 described later. The region ID is the identification information of each manuscript region extracted in S12. The region shape is information that specifies the rectangular shape of the manuscript region indicated by the corresponding region ID. In the example shown, the rectangular shape information is a set of coordinates of the three vertices of the rectangular shape. Figure 6 The extraction result data shown is the same as that described later. Figure 10 The pattern number 1 shown in the example corresponds to the extraction result. The pattern number 1 shown in the example is a pattern consisting of two original areas with area IDs 1 and 2. Although not included in Figure 6 In the example shown, however, there are cases where the same document area belongs to a plurality of different patterns.

[0087] Processor 102 extracts patterns with scores above a predetermined score threshold from the one or more patterns obtained in S12 (S14). Conversely, patterns with scores below the score threshold have a significantly lower probability of being the correct answer and are therefore discarded at this stage.

[0088] Next, the processor 102 determines whether the number of patterns extracted in S14 is 1, 2 or more and T or less (T is an integer of 2 or more), or (T+1) or more ( S16 ).

[0089] Also refer to Figure 7 and Figure 8 , the processing when the judgment result of S16 is "1" is explained. Figure 7 The figure shows a state where two originals 12 are placed sufficiently apart from each other on a platen 10 of a scanning mechanism provided in an information processing apparatus. When the originals 12 are placed sufficiently apart from each other in this manner, the processing in S12 accurately extracts the original regions of all the originals 12, and narrows down the number of patterns in the original regions with scores exceeding a score threshold to one.

[0090] Thus, when the determination result of S16 is "1", the processor 102 Figure 8 The UI screen 200 shown in FIG. 1 is displayed on a display device as an example of the input / output device 108 (S18). Figure 8The UI screen 200 illustrated in FIG. 2 is called a confirmation UI screen. It displays a scanned image 210, a cut button 220, a reposition button 222, a rectangle designation button 224, and a pattern quantity display field 230. A confirmation UI screen displays only one found pattern and asks the user to confirm whether to extract the document image according to that pattern (indicated as "Cut" in the figure).

[0091] The scanned image 210 includes an image of the document 12 placed on the platen 10, namely, the document image 212. Furthermore, the scanned image 210 displays the document area 214 extracted in S12. In the illustrated example, the document area 214 is shown as a frame (a rectangular frame in the illustrated example) surrounding the outer perimeter of the document area. However, this is merely an example. Alternatively, for example, an image entirely filled with a specific color in the two-dimensional document area may be used as the document area 214.

[0092] The number of patterns display column 230 displays the number of patterns of the document area extracted from the scanned image 210 obtained in S10 . Figure 7 and Figure 8 In the example of , there is only one pattern that is somewhat reliable (ie, the score is equal to or greater than the score threshold), and therefore a value of “1” is displayed in the pattern number display field 230 .

[0093] The cut button 220 is a GUI (Graphical User Interface) component that receives an instruction to cut, i.e., extract, the image of each document area 214 from the scanned image 210. For example, when the display device provided by the information processing device is configured as a touch panel, the user presses the cut button 220 by touching the cut button 220 displayed on the touch panel with a finger. Figure 8 The arrows shown in an overlapping manner of the cut button 220 indicate that the cut button 220 is selected as the pressed object.

[0094] When the user re-places the set of originals 12 on the platen 10 for re-scanning, the user presses the re-place button 222 .

[0095] When the pattern of the document area extracted in S12 is discarded and the user manually specifies the document area (“rectangle” in this example), the rectangle specifying button 224 is pressed.

[0096] After displaying the UI screen 200 in S18 , the processor 102 determines which of the cut button 220 , the reposition button 222 , and the rectangle designation button 224 the user has pressed ( S20 ).

[0097] When the user presses the cut button 220 , the processor 102 extracts the images in the document areas 214 shown by the unique pattern displayed on the UI screen 200 , and outputs the extracted images as document images to a file ( S22 ).

[0098] Furthermore, if the user presses the re-placement button 222, the processor 102 displays the re-placement guidance screen on the display device (S24), and waits for the user to re-place the document according to the guidance of the re-placement guidance screen and instruct to start scanning (S26). A specific example of the re-placement guidance screen will be referred to later. Figure 26 Provide explanation.

[0099] If the user presses the rectangle designation button 224, the processor 102 displays the rectangle designation screen on the display device and receives the designation of the rectangle of the manuscript area from the user on the screen (S28). A specific example of the rectangle designation screen and the processing of S28 will be referred to later. Figures 27 to 29 Provide explanation.

[0100] When the determination result of S16 is "2 to T", the processor 102 enters Figure 3 In this case, the processor 102 causes the display device to display a simultaneous display type UI screen (S30). The simultaneous display type UI screen is a screen that simultaneously displays the patterns extracted in S14 with scores above the score threshold on one UI screen. If too many patterns are displayed on one screen, it will be difficult for a person to distinguish the patterns. Therefore, in this example, the upper limit of the number of patterns that a person can distinguish even if displayed on one screen is set as a threshold value T. If the number of patterns extracted in S14 is more than 2 and less than T, the simultaneous display type UI screen is used.

[0101] Figure 9 In the example shown, two rectangular documents 12 of equal width are placed adjacent to each other on the scanner's platen 10, with their sides aligned on the same straight line. As briefly explained, in this example, the document area extraction process not only recognizes a pattern consisting of two document areas corresponding to the two documents 12, but also sometimes recognizes a pattern in which these two documents 12 are treated as a single continuous document area.

[0102] Figure 10 In the example of scanning Figure 92 (i.e., a simultaneous display type UI screen). In this example, a pattern 1 consisting of a single manuscript area 214a formed by merging two manuscript images 212 and a pattern 2 consisting of two rectangular manuscript areas 214b substantially identical to each of the two manuscript images 212 are simultaneously displayed on the scanned image 210 within the UI screen 200. The manuscript areas belonging to pattern 1 and the manuscript areas belonging to pattern 2 are displayed in different display modes so that they can be distinguished from each other. In the illustrated example, the manuscript area 214a belonging to pattern 1 is represented by a solid frame line of a first color, and the manuscript areas 214b belonging to pattern 2 are represented by a dotted frame line of a second color different from the first color.

[0103] In this example, the number of patterns recognized from the scanned image is two, and this number of patterns “2” is displayed in the pattern number display field 230 .

[0104] Furthermore, on the UI screen 200, in addition to the Figure 8 In addition to the cut button 220, reposition button 222, and rectangle designation button 224 displayed on the UI screen 200, a "Select Pattern 1" button 226 and a "Select Pattern 2" button 228 are also displayed. "Select Pattern 1" button 226 and "Select Pattern 2" button 228 are GUI components for accepting user selections of Pattern 1 and Pattern 2, respectively. "Select Pattern 1" button 226 is displayed in a display mode corresponding to the display mode of the original area 214a of Pattern 1 on the scanned image 210, while "Select Pattern 2" button 228 is displayed in a display mode corresponding to the display mode of the original areas 214b of Pattern 2. For example, "Select Pattern 1" button 226 is displayed in the first color, which is the display color of the original area 214a of Pattern 1, while "Select Pattern 2" button 228 is displayed in the second color, which is the display color of the original areas 214b of Pattern 2.

[0105] Thus, in the illustrated UI screen 200, the display method of the manuscript area for each pattern and the display method of the button for selecting each pattern are set to be the same or correspond to each other. This makes it easy to visually distinguish the association between the pattern and the button. In addition to the colors exemplified above, other display methods for the pattern and the corresponding button include the color or line type (e.g., solid line, dashed line, etc.) of the border surrounding the manuscript area or button, and the presence or absence of a flashing display.

[0106] If multiple ( Figure 10 If there is a correct pattern among the patterns (in the example, 2), the user presses the button 226 or 228 corresponding to the correct pattern according to the message "Please select the correct pattern." on the UI screen 200.

[0107] When the user's operation on the UI screen 200 is detected, the processor 102 determines whether the operation is an operation of pressing the button 226 or 228 for selecting a pattern or an operation of pressing other buttons 220, 222 or 224 (S32). Figure 10 At the time of displaying the UI screen 200, the user has not yet selected any pattern from the multiple patterns displayed simultaneously. Therefore, even if the cut button 220 is pressed, it is unknown which of the multiple patterns the document image can be extracted from. Therefore, the cut button 220 can be disabled beforehand at the time of displaying S30.

[0108] If it is determined in S32 that the button 226 or 228 is pressed, the processor 102 displays the document area corresponding to the pressed button 226 or 228 on the scanned image 210 in a display mode indicating a "selected state" (i.e., a selected state) (S34). The display example of the UI screen 200 in S34 is shown in FIG. Figure 11 . This example is an example of a case where the user presses the "Select Pattern 2" button 228. In this case, a check mark 215 is displayed for each of the two manuscript areas 214b belonging to Pattern 2 in the scanned image 210. These check marks 215 indicate that these respective manuscript areas 214b are in a selected state. At the stage S34, since the user has selected a pattern, the cut button 220 is enabled (i.e., transitioned to a pressable state). The selected state can also be said to be a state indicating that the user has determined that the manuscript area is a correct manuscript area.

[0109] Thereafter, the processor 102 waits for an operation from the user, ie, pressing of the cut button 220 , the reposition button 222 , the rectangle designation button 224 , or the button 226 or 228 for selecting a pattern.

[0110] When the user operation is detected, the processor 102 determines whether the operation is pressing the cut button 220 (S36). If the result of this determination is yes, the processor 102 cuts the image in each selected document area from the scanned image 210 and outputs it to a file (S38).

[0111] When the determination result of S36 is no, the processor 102 determines whether the user's operation is to press the re-place button 222 (S40). When the determination result is yes, the process proceeds to Figure 2 When the determination result of S40 is no, the processor 102 determines whether the user's operation is to press the rectangular designation button 224 (S42). When the determination result is yes, the process proceeds to step S24. Figure 2If the determination result of S42 is negative, the processor 102 proceeds to S32 to determine whether the user's operation is to press the button 226 or 228 for selecting a pattern, and then executes the above-mentioned process.

[0112] When the result of determination in S16 is "(T+1) or more", the processor 102 enters Figure 4 The steps shown in Figure 4 In the step of , the user's instruction is received by using the switching display type UI screen. The following describes the case of T=2 as an example.

[0113] The switchable display type UI screen is a screen that displays only one pattern in the screen, and the type of the displayed pattern is sequentially switched by a user operation.

[0114] Figure 12 In the example shown, three originals 12 are placed on the platen 10, and the two originals 12 on the right are placed close to each other so that their two sides are aligned on the same straight line. Figure 2 The number of patterns extracted in S14 is 5. For example, serial numbers may be assigned to these 5 patterns in descending order of scores.

[0115] Figure 13 Scan Figure 12 2 (i.e., a switching display type UI screen). In this example, one of the five patterns (e.g., the pattern with the highest score) is displayed on the scanned image 210 within the UI screen 200. This pattern consists of a document area 216a covering the document image 212a on the left and a document area 216b continuously covering the two document images 212b and 212c on the right.

[0116] The five patterns extracted in S14 are candidates for switching the patterns displayed on the switching display type UI screen 200. These candidates are referred to as "switching candidates." The "5" to the right of the " / " in the display content "1 / 5" in the pattern quantity display field 230 represents the total number of switching candidate patterns, and the "1" to the left represents the number of the currently displayed pattern among these five switching candidates (for example, the aforementioned serial number).

[0117] The switchable display UI screen 200 displays a previous candidate button 232, a next candidate button 234, a select all button 236, and a cancel all button 238. The previous candidate button 232 is a button for receiving an instruction to switch the switch candidate pattern displayed on the UI screen 200 to the switch candidate immediately preceding the currently displayed pattern. The next candidate button 234 is a button for receiving an instruction to switch the switch candidate pattern displayed on the UI screen 200 to the switch candidate immediately following the currently displayed pattern. The select all button 236 is a button for receiving an instruction to select all document areas included in the currently displayed pattern, and the cancel all button 238 is a button for receiving an instruction to cancel the selection of all document areas included in the currently displayed pattern.

[0118] In this UI screen 200, if the user touches document area 216a or 216b, the touched document area 216a or 216b transitions to a selected state. Specifically, if the user finds a correct document area that fully matches the document image among the displayed document areas (i.e., rectangles), they touch that document area, thereby selecting it. Furthermore, if the user touches a selected document area 216a or 216b, the touched document area 216a or 216b is deselected. Consequently, the previously selected document area 216a or 216b transitions to a deselected state.

[0119] Back to Figure 4 When the result of S16 is "more than (T+1)", the processor 102 selects the pattern with a score greater than the score threshold extracted in S14 as a switching candidate (S50). Then, the processor 102 selects Figure 13 The UI screen 200 shown in FIG. 2 is displayed on the display device ( S52 ). At this point in time, all the document areas 216 a and 216 b belonging to the pattern shown on the scanned image 210 are in an unselected state.

[0120] Upon receiving a user operation on UI screen 200, processor 102 determines whether the operation is an operation to select or release the designated rectangle, i.e., document area 216a or 216b (S54). If the result of S54 is yes, processor 102 transitions the state of document area 216a or 216b to a selected state or an unselected state, corresponding to the selected or released operation. Furthermore, processor 102 reduces the candidate patterns displayed on UI screen 200 by pressing previous candidate button 232 or next candidate button 234 to only those patterns that include the currently selected document area (S56).

[0121] For example, in Figure 13In the UI screen 200 shown in FIG. 1 , both the manuscript areas 216a and 216b are in an unselected state. However, if the user touches the manuscript area 216a, Figure 14 As shown in FIG, the manuscript area 216a transitions to the selected state. Figure 14 In the UI screen 200 illustrated in FIG, a check mark 215 indicating a selected state is displayed on the document area 216a in the scanned image 210. Figure 13 In the state of , there are five switching candidates, but by selecting the document area 216a, the switching candidates are limited to patterns including the document area 216a. In the example shown in the figure, there are three such patterns.

[0122] If the determination result of S54 is negative, the processor 102 determines whether the user's operation is to press the next candidate button 234 (S58). If the determination result is positive, the processor 102 switches the pattern displayed on the scanned image 210 to the next switching candidate (S60) and returns to S52.

[0123] For example, if the user Figure 14 If the next candidate button 234 is pressed on the UI screen 200 illustrated in FIG. Figure 15 , the second candidate switching pattern is displayed on scanned image 210. This pattern consists of, in addition to selected document area 216a, document area 216c that completely covers document image 212b, and document area 216d that partially covers the lower portion of document image 212c. Document area 216d is likely the result of erroneous detection of an edge at the upper end of document area 216d, for example, due to the density distribution within document image 212c.

[0124] If the determination result of S58 is negative, the processor 102 determines whether the user's operation is to press the previous candidate button 232 (S62). If the determination result is positive, the processor 102 switches the pattern displayed on the scanned image 210 to the previous switching candidate (S64) and returns to S52.

[0125] If the result of the determination in S62 is negative, processor 102 determines whether the user's operation is to press all select button 236 (S66). If the result of this determination is positive, processor 102 transitions all rectangular document areas belonging to the displayed pattern to a selected state (S68) and returns to S52. Processor 102 then limits the switching candidates to only those patterns that include all of these document areas.

[0126] If the result of the determination in S66 is negative, the processor 102 determines whether the user's operation is to press the all-cancel button 238 (S70). If the result of this determination is positive, the processor 102 transitions all rectangular document areas belonging to the displayed pattern to an unselected state (S72) and returns to S52. The processor 102 also returns the set of switching candidates to the switching candidates at the time point selected in S50.

[0127] If the determination result of S70 is negative, Figure 5 As shown, the processor 102 determines whether the user's operation is to press the cut button 220 (S74). If the result of this determination is no, the processor 102 determines whether the user's operation is to press the re-place button 222 (S84). If the result of this determination is yes, the process proceeds to Figure 2 If the result of S84 is negative, the processor 102 determines whether the user's operation is to press the rectangular designation button 224 (S86). If the result of this determination is positive, the process proceeds to step S24. Figure 2 If the result of determination at S86 is negative, the processor 102 proceeds to S54 to determine whether the user's operation is an operation to select or release the document area, and then executes the above-described process.

[0128] If the result of the determination in S74 is YES, the processor 102 determines whether there is any unselected document area in the document area included in the displayed pattern (S76). If the result of the determination is NO (i.e., all document areas in the pattern are selected), the processor 102 extracts the image of each selected document area from the scanned image 210 and outputs it to a file (S82).

[0129] When the judgment result of S76 is yes, the processor 102 displays a dialog box on the UI screen 200 containing a warning indicating that the image of the unselected manuscript area is to be extracted without cutting and asking whether cutting is possible (S78). The processor 102 determines whether the answer entered by the user to the dialog box is "yes" (i.e., cutting is possible) (S80). When the result of this judgment is yes, the image in each manuscript area in the selected state is extracted from the scanned image 210 and output to a file (S82). S78 and S80 are processes that take into account the situation where there is a manuscript area that is unnecessary for the user in the manuscript area extracted by the information processing device. For example, due to dirt on the pressing plate 10, an image is generated in a part where there was no manuscript originally, and when the part containing this image is identified as the manuscript area, it is not necessary to extract the image of this manuscript area.

[0130] If the result of the determination at S80 is negative, the processor 102 returns to S52 and displays the UI screen 200 to receive further operations from the user.

[0131] exist Figure 4 and Figure 5 In the process of Figure 15 Select the manuscript area 216c on the UI screen 200 shown in FIG. Figure 16 As shown in FIG. 2 , a check mark 215 indicating that the document area 216c is in a selected state is displayed. As a result of the selection, the switching candidates are narrowed down to two patterns including the document areas 216a and 216c in the selected state. Figure 16 Press the next candidate button 234 on the UI screen 200, and the Figure 17 The next switching candidate pattern is exemplified in . This pattern includes, in addition to the document areas 216a and 216c described above, a document area 216e that completely covers the document image 212c.

[0132] If the user is Figure 17 Select the original area 216e on the UI screen 200, then Figure 18 As shown, a check mark 215 indicating a selected state is displayed on the document area 216e. Thereafter, if the user presses the cut button 220, the processor 102 cuts the images of the selected document areas 216a, 216c, and 216e in S82 and outputs each cut image to a file.

[0133] exist Figures 12 to 18 In the example shown, the user selects the correct document area from the displayed patterns to narrow down the candidate patterns, ultimately arriving at the correct pattern. However, if the number of candidate patterns is small, pressing the previous candidate button 232 and the next candidate button 234 to sequentially check all patterns will not take much time to find the correct pattern.

[0134] And, in Figure 16 and Figure 17 In the example shown, the switching display type UI screen 200 is displayed when the number of switching candidate patterns is reduced to a sufficiently small number of two. However, this is only an example. As another example, when the number of switching candidate patterns after reduction becomes less than T as a result of the user's selection of the document area group, the processor 102 may switch the displayed UI screen 200 to the simultaneous display type (see FIG. Figure 10 ) screen. At this point in time, the remaining candidate switching patterns have a common group of selected manuscript areas. Therefore, only the manuscript areas that are not selected can be Figure 10 The display modes are distinguished between the different patterns illustrated in FIG.

[0135] Next, an example of a usage scenario of the all selection button 236 will be described. Figure 19 As shown in the example, when a large number of originals 12 are arranged on the platen 10 in a relatively dense state, the number of patterns extracted in S14 may become a considerable number. Assume that the number of patterns is 15, and Figure 4 The UI screen 200 initially displayed after S50 is Figure 20 As described above, when the number of manuscripts or the number of switching candidates is large, it takes time to arrive at the correct pattern, whether using the method of sequentially checking all switching candidates using the next candidate button 234 or the method of sequentially selecting the correct manuscript area 218a or the like to narrow down the switching candidates.

[0136] Figure 20 In the example shown in FIG. 2 , the user visually inspects the respective manuscript areas 218a, 218b, ..., 218h, 218i and the respective manuscript images 212a, 212b, ..., 212i, 212j of the pattern displayed on UI screen 200. The user thereby determines that manuscript area 218i is incorrect because it continuously overlaps both manuscript images 212i and 212j. Furthermore, the user determines that the other manuscript areas 218a, 218b, ..., 218h completely match one manuscript image 212a, 212b, ..., 212h, respectively, and are therefore correct manuscript areas. The user then adopts manuscript images 212a, 212b, ..., 212h as the correct manuscript images and determines that only manuscript area 218i needs to be corrected.

[0137] Therefore, by pressing the all selection button 236, the user temporarily selects all the manuscript areas 218a, 218b, ..., 218h, 218i in the displayed pattern. As a result, the UI screen 200 changes to Figure 21 The screen shown. At this time, the number of switching candidates is 1. The user touches the manuscript area 218i on the UI screen 200 to release the selection state of the manuscript area 218i. As a result, the UI screen 200 changes to Figure 22 The screen shown. When the manuscript area 218i becomes unselected, the number of switching candidates increases to 2. The user presses the next candidate button 234 to confirm another switching candidate pattern. As a result, the UI screen 200 changes to Figure 23 The screen shown. Figure 23 The pattern shown in the UI screen 200 includes a document area 218j that completely matches the document image 212i and a document area 218k that completely matches the document image 212j. If the user selects these two document areas 218j and 218k, the UI screen 200 changes to Figure 24When the user presses the cut button 220 on the UI screen 200 , the processor 102 cuts the images in the selected document areas 218 a , 218 b , . . . , 218 h , 218 j , and 218 k and outputs them to files.

[0138] refer to Figures 19 to 24 In the example described, there is only one incorrect manuscript area, but when there are multiple incorrect manuscript areas in the manuscript area in the selected state, all of these incorrect manuscript areas may be selected. In this case, the processor 102 treats the portion of the scanned image 210 consisting of these multiple selected manuscript areas as an undetermined portion. Alternatively, the processor 102 may also treat the remaining portion of the scanned image other than "the manuscript area group that remains in the selected state (i.e., the correct manuscript area) after the incorrect manuscript area is selected" as an undetermined portion. Furthermore, the processor 102 receives a designation of the manuscript area within the undetermined portion from the user. For example, the processor 102 presents to the user a pattern that includes the manuscript image group that remains in the selected state among the patterns extracted in S14 as a switching candidate, and receives the user's selection from these switching candidates. In this case, the manuscript areas within the undetermined portion are different among the switching candidates presented to the user, but the manuscript areas in the selected state are the same. In this case, the document area within the undetermined portion displayed on the UI screen 200 is included in the pattern (i.e., the switching candidate) that includes the selected document area group. However, as another example, a document area that belongs to a pattern that does not include the selected document area group among the patterns extracted in S14 and that is included in the undetermined portion may be displayed on the UI screen 200 as a candidate for the correct document area, and a selection may be received from the user.

[0139] Furthermore, when the UI screen 200 is first displayed, all document areas of the pattern displayed therein may be set to a selected state (ie, an "all selected" state) so that the user can select an erroneous document area from this state.

[0140] Next, usage scenarios of the reposition button 222 and the rectangle designation button 224 are described.

[0141] Figure 25The example of UI screen 200 shown illustrates a situation where none of the document area patterns extracted in S12 and S14 are correct. Pattern 1 consists of a large rectangular document area 219a encompassing both document images 212a and 212b. Pattern 2 consists of document area 219b-1, which is a downward extension of document image 212a, and document area 219b-2, which partially covers document image 212b. The blurring of the edges of document images 212a and 212b, the density distribution of these images, and contamination of platen 10 may interact, leading to the extraction of document areas that do not match either document image 212a or 212b.

[0142] In this example, two patterns are extracted, so a simultaneous display type UI screen 200 is used, in which a "Select Pattern 1" button 226 and a "Select Pattern 2" button 228 are displayed. However, both patterns are incorrect, so even if these buttons are pressed, the correct pattern cannot be selected.

[0143] In this case, the user presses the re-place button 222 to re-place the document group on the platen 10 or presses the rectangle designation button 224 to directly designate a rectangle, ie, a document area, on the UI screen 200 .

[0144] When the user presses the reposition button 222, the processor 102 displays Figure 26 The relocation guidance UI screen 200A ( Figure 2 S24 of FIG. 24 ). The screen displays a guide message 240. The guide message 240 includes a message indicating the work that the user should do, "Please re-place the original and scan again." and a description of precautions for the work (for example, the distance between the originals should be kept at least a predetermined distance). The user refers to the guide message 240 to re-place the original on the platen 10 and instructs to execute the scan again. If the instruction to execute the scan is received ( Figure 2 If the result of S26 is yes), the processor 102 executes Figure 2 Steps after S10.

[0145] When the user Figure 25 When the rectangle designation button 224 is pressed on the UI screen 200 shown in FIG. 1 , the processor 102 displays Figure 27 The rectangle designation UI screen 200B shown in FIG. 1 receives an operation from the user and determines a rectangle, ie, a document area ( Figure 2 S28).

[0146] In the scanned image 210 of the UI screen 200B displayed immediately after the user presses the rectangle designation button 224, no document area is displayed. In addition, the UI screen 200B displays an explanation text 252 indicating the rectangle designation method, a rectangle number display bar 254, a cancel button 256, and a redo button 258. The rectangle designation method shown in the explanation text 252 is a method of selecting any three of the four vertices of the rectangle in clockwise order. The rectangle number display bar 254 displays the number of rectangles (i.e., document areas) that have been determined or selected corresponding to the user's operation. Figure 27 At the time point of the UI screen 200B, no rectangle has been determined, so the number of rectangles displayed in the rectangle number display field 254 is 0.

[0147] Cancel button 256 is a button for canceling a user's vertex selection. For example, if the user selects a second vertex after selecting the first vertex and immediately presses Cancel button 256, the selection of the second vertex is canceled. Pressing Cancel button 256 again also cancels the selection of the first vertex. Redo button 258 is a button for clearing all vertex selections on scanned image 210 and the document area definition associated with those selections.

[0148] The user observes the scanned image 210 to identify the document image 212a or 212b, and selects the vertices of the rectangle of the identified document image 212a or 212b one by one in a clockwise direction. This selection is performed, for example, by touching the vertices of the document image displayed in the scanned image 210 on the touch panel with a finger. The processor 102 stores the coordinates of the selected vertex in the scanned image 210, and displays a mark (circle in the figure) representing the selected vertex at the coordinates on the scanned image 210. Figure 27 At the stage shown in FIG, the user has selected the first vertex 250a of the document image 212a. Thereafter, the user selects the second and third vertices 250b and 250c of the document image 212a in a clockwise direction (refer to FIG. Figure 28 ). If the number of selected vertices becomes 3, the processor 102 will Figure 29 As shown in FIG, a rectangle uniquely determined by the coordinates of the three vertices is determined as one document area 219c, and information on the determined document area 219c is stored in the memory 104. Figure 29 On the UI screen 200B shown, the determined document area 219c is framed, and a check mark 215 indicating a selected state (ie, determined) is displayed within the area. Thereafter, the user sequentially selects three vertices of the rectangle representing the second document image 212b. Figure 29 It shows the state at the time when the first vertex 250d is selected.

[0149] In the above example, the document image rectangle is determined by the user selecting three vertices of the rectangle, but this is merely an example. Alternatively, the document image rectangle may be determined by the user tracing two adjacent sides of the document image on the UI screen 200B with a finger.

[0150] refer to Figures 25 to 29 The example described is an example of a case where there is no correct manuscript area among the manuscript areas extracted by the manuscript area extraction process. However, the rectangular designation is not limited to this case and can be executed at any time. For example, Figure 16 In the example shown in FIG, 2 , when two document areas 216a and 216c are selected, even if the previous candidate button 232 or the next candidate button 234 is pressed to check all remaining patterns and no pattern containing a document area that completely matches document image 212c is found, a rectangle representing the document area of document image 212c can be specified using rectangle designation. Specifically, in this case, processor 102 retains information about each selected document area in memory 104, identifies a new document area based on the vertex group selected by the user, and stores it additionally in memory 104.

[0151] The above describes an embodiment of the present invention. The above embodiment is merely an example. Various modifications and improvements are possible within the scope of the present invention. For example, it is also possible to use a smartphone camera to capture a surface on which multiple documents are arranged, and then use an application installed on the smartphone to perform the same processing as described above on the image thus obtained.

[0152] The above-described embodiments of the present invention are provided for the purpose of illustration and explanation. In addition, the embodiments of the present invention do not fully and exhaustively include the present invention, and do not limit the present invention to the disclosed embodiments. It is obvious that various modifications and variations are self-evident to those skilled in the art to which the present invention belongs. The present embodiment is selected and described in order to most easily explain the principles of the present invention and its application. Thus, other technical personnel in this field can understand the present invention through various modifications optimized for specific uses of the assumed various embodiments. The scope of the present invention is defined by the above claims and their equivalents.

Claims

1. An information processing device, characterized in that With processor, The processor performs the following processing: When there are a plurality of patterns in the document area extracted from an input image obtained by photographing the document, display control is executed for displaying the plurality of patterns as options on the screen; receiving, based on the display of the screen controlled by the display control, a selection of a correct pattern from among the plurality of patterns; extracting an image of the document area included in the pattern selected by the user from the input image, The display control is a control for displaying an image showing the plurality of patterns simultaneously on the screen. The processor executes display control different from control for displaying an image simultaneously displaying the plurality of patterns on the screen when the number of the patterns in the document area extracted from the input image is greater than a predetermined threshold.

2. The information processing device according to claim 1, wherein The image is an image that is associated with each of the plurality of patterns and further displays a UI component for selecting the pattern. In the process of receiving the selection, a pattern corresponding to the UI component selected by the user in the image displayed on the screen is determined as the correct pattern.

3. The information processing device according to claim 1 or 2, characterized in that The display control is a control for displaying an image showing a first pattern among the plurality of patterns on the screen. The processor also performs the following processing: receiving a selection of one or more correct original regions among the original regions included in the first pattern displayed on the image; executing a second receiving process in which a designation of a manuscript area included in a portion of the input image other than the one or more selected correct manuscript areas is received by a method different from a method selected from within the first pattern; The images of the one or more selected correct document regions and the images of the document regions specified by the different methods are extracted from the input image.

4. The information processing device according to claim 3, wherein: The second receiving process is a process of receiving selection of a correct pattern from among the plurality of patterns including the selected one or more correct document regions.

5. The information processing apparatus according to claim 4, wherein: In the second receiving process, when the number of patterns including the one or more correct original areas selected is below a specified threshold, images of these patterns including the one or more correct original areas are displayed simultaneously and an image of a UI component for selecting the pattern is displayed on the screen by establishing a corresponding association with each of the simultaneously displayed patterns, and the pattern corresponding to the UI component selected by the user in the image displayed on the screen is used as the correct pattern. The information processing apparatus according to claim 3 , wherein: In the second receiving process, a designation for specifying a vertex or a side of a document region included in a portion other than the one or more selected correct document regions is received.

7. The information processing device according to claim 1 or 2, characterized in that The display control is a control for displaying an image showing a first pattern among the plurality of patterns on the screen. The processor also performs the following processing: receiving a selection of one or more erroneous original regions among the original regions included in the first pattern displayed on the image; executing a second receiving process in which a designation of a document region included in a portion of the input image corresponding to the one or more selected erroneous document regions is received by a method different from the method selected from the first pattern; Images of the remaining document regions excluding the one or more erroneous document regions in the first pattern and images of the document regions specified by the different methods are extracted from the input image.

8. The information processing apparatus according to claim 7, wherein: The second receiving process is a process of receiving selection of a correct pattern from patterns including the remaining document regions other than the selected one or more erroneous document regions among the plurality of patterns.

9. The information processing device according to claim 1, wherein The processor performs the following processing: receiving a designation for determining a vertex or an edge of at least one document region included in the input image displayed on the screen; An image of the at least one document region specified by the designated vertex or edge is extracted from the input image.

10. A storage medium storing a program for causing a computer to operate in the following manner: When there are a plurality of patterns in the document area extracted from an input image obtained by photographing the document, display control is executed for displaying the plurality of patterns as options on the screen. receiving selection of a correct pattern from among the plurality of patterns based on the display of the screen under the display control, and extracting from the input image an image of the document area included in the pattern selected by the user, The display control is a control for displaying an image showing the plurality of patterns simultaneously on the screen. When the number of the patterns in the document area extracted from the input image is greater than a predetermined threshold, display control different from control for displaying an image simultaneously displaying the plurality of patterns on the screen is executed.

11. An information processing method comprising the following steps: When there are a plurality of patterns in the document area extracted from an input image obtained by photographing the document, display control is executed for displaying the plurality of patterns as options on the screen. receiving selection of a correct pattern from among the plurality of patterns based on the display of the screen under the display control, and extracting from the input image an image of the document area included in the pattern selected by the user, The display control is a control for displaying an image showing the plurality of patterns simultaneously on the screen. When the number of the patterns in the document area extracted from the input image is greater than a predetermined threshold, display control different from control for displaying an image simultaneously displaying the plurality of patterns on the screen is executed.

Images