Information processing system and its control method, program

Abstract

The present invention provides an information processing device, method, and program that enable appropriate OCR processing. [Solution] The information processing device includes: a receiving means for receiving a specification of the range to be processed for OCR processing for a document to be processed by OCR processing; an OCR means for performing OCR processing on the range received by the receiving means; a storage means for storing the type of document and information that can identify the range to be processed for OCR processing specified by the receiving means in association with each other; and an OCR range acquisition means for acquiring information that can identify the range to be processed for OCR processing from the storage means based on the type of a newly acquired document. The OCR means further performs OCR processing on the range identified by the information acquired by the OCR range acquisition means in the newly acquired document.

Description

Technical Field

[0001] The present invention relates to an information processing system for form analysis, its control method, and a program.

Background Art

[0002] Conventionally, it is known to perform optical character recognition (hereinafter referred to as OCR) to obtain string information described in form image data.

[0003] Patent Document 1 describes searching for item names related to item values to be extracted from a form image, and specifying, as an item value, information having a string type corresponding to the item value to be extracted from the strings in the vicinity thereof, and assisting in the confirmation work of whether the obtained item value is a correct value.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Disclosure of the Invention

Problems to be Solved by the Invention

[0005] However, in character detection, there may be cases where all character data on the image data cannot be comprehensively detected. Therefore, an object of the present invention is to realize appropriate OCR processing.

Means for Solving the Problems

[0006] Receiving means for receiving a range specification for a document, Determination means for determining whether the range received by the receiving means satisfies a predetermined criterion indicating a size suitable for OCR processing, Notification means for notifying the determination result by the determination means, characterized by comprising the above.

Effects of the Invention

[0007] According to the present invention, appropriate OCR processing can be achieved. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of the configuration of an OCR support system. [Figure 2] A block diagram showing an example of the hardware configuration of client terminal 101. [Figure 3] This flowchart shows an example of the process overview. [Figure 4] This flowchart illustrates an example of a process for correcting OCR results using the recorded correction range A. [Figure 5] This flowchart illustrates an example of a process that receives input for correction range B from the user and uses that correction range B to correct the OCR result. [Figure 6] This screen shows an example of OCR results before correction by user input and before correction by the recorded correction range A. [Figure 7] This diagram illustrates an example of the process for determining the type of document to be processed by OCR. [Figure 8] This figure shows an example of a list of missing items. [Figure 9] This screen shows an example of how a user can input a correction range B for OCR results. [Figure 10] This figure shows an example of data within correction range A. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described in detail below with reference to the drawings.

[0010] In this embodiment, the invention is described using processing related to OCR for non-standard forms whose layout cannot be grasped in advance (examples of forms include invoices, purchase orders, delivery notes, etc.). Note that the present invention is also applicable to standard forms whose layout can be grasped in advance.

[0011] Note that forms are an example of documents to be OCR'ed, and the present invention is also applicable to documents other than forms.

[0012] FIG. 1 is a diagram showing an example of the configuration of an OCR support system in an embodiment of the present invention.

[0013] In the information processing system of the present invention, a client terminal 101, a scanner 102, and a file server 103 are communicably connected via a network 100.

[0014] The client terminal 101 only needs to incorporate the functions shown in FIG. 2, and may be, for example, a personal computer (hereinafter referred to as a PC), a mobile terminal such as a smartphone, a tablet terminal, a wearable terminal such as a smartwatch, a PDA, a game machine, or the like.

[0015] The scanner 102 scans a form to create an image file and transmits it to the client terminal 101 via the network.

[0016] The network 100 can take forms such as a wired LAN, a wireless LAN, a USB, etc. according to the physical interface of the scanner 102.

[0017] The file server 103 can save the image file captured by the scanner or save the range related to the correction of OCR for the form.

[0018] Furthermore, when importing images scanned by scanner 102 to client terminal 101, either method is used: directly sending the images from scanner 102 to client terminal 101, or the image files imported by scanner 102 are temporarily stored in file server 103, and the client terminal 101 retrieves the image files from file server 103.

[0019] Figure 2 is a block diagram showing an example of the hardware configuration of a client terminal 101 (a client terminal is an example of an information processing device) in an embodiment of the present invention. The scanner 102 and file server 103 have a similar configuration.

[0020] As shown in Figure 2, each information processing device is connected via a system bus 204 to a CPU (Central Processing Unit) 201, ROM (Read Only Memory) 202, RAM (Random Access Memory) 203, input controller 205, video controller 206, memory controller 207, and communication I / F controller 208.

[0021] CPU201 provides comprehensive control over all devices and controllers connected to the system bus 204.

[0022] ROM202 or external memory211 holds the BIOS (Basic Input / Output System) and OS (Operating System), which are control programs executed by the CPU201, as well as computer-readable and executable programs and various necessary data (including data tables) for realizing this information processing method.

[0023] RAM203 functions as the main memory, work area, etc., of the CPU201. The CPU201 loads the necessary programs, etc., from ROM202 or external memory211 into RAM203, and then executes the loaded programs to perform various operations.

[0024] The input controller 205 controls input from input devices such as a keyboard 209, a mouse (not shown), a touchpad, and other pointing devices. If the input device is a touch panel, the user can give various instructions by pressing (touching with a finger, etc.) icons, cursors, or buttons displayed on the touch panel.

[0025] Furthermore, the touch panel may be a multi-touch screen or other touch panel capable of detecting the positions of multiple fingers touching it.

[0026] The video controller 206 controls the display to an external output device such as the display 210. The display may include the display of a notebook PC integrated with the main unit. The external output device is not limited to a display; for example, it may be a projector. Furthermore, for the aforementioned touch-enabled device, an input device is also provided.

[0027] In the flowchart-based explanation described later, the display destination will be display 210 unless otherwise specified.

[0028] The video controller 206 can control the video memory (VRAM) used for display control. It can utilize a portion of the RAM 203 as the video memory area, or it can provide a separate, dedicated video memory.

[0029] The memory controller 207 controls access to the external memory 211. The external memory can include an external storage device (hard disk), a flexible disk (FD), or a CompactFlash® memory connected to a PCMCIA card slot via an adapter, which stores boot programs, various applications, font data, user files, editing files, and other data.

[0030] The communication interface controller 208 connects to and communicates with external devices via the network 214 (for example, network 101 shown in Figure 1), and executes communication control processing on the network 214. For example, it can communicate using TCP / IP, telephone lines such as ISDN, and 3G, 4G, and 5G mobile phone lines.

[0031] The CPU 201 enables display on the display 210 by, for example, performing the process of expanding (rasterizing) outline fonts into the display information area in RAM 203. The CPU 201 also enables user input via a mouse cursor (not shown) on the display 210.

[0032] Next, the processes executed by the client terminal 101 in the embodiment of the present invention will be described using Figures 3 to 10.

[0033] Figures 3 to 5 are flowcharts showing the flow of processing performed by the client terminal 101 in an embodiment of the present invention.

[0034] The flowchart in Figure 3 shows the overall process when the CPU 201 of the client terminal 101 reads and executes a predetermined control program. Note that the processing of each step is executed by the CPU 201 of each device. When the client terminal 101 acquires an image to be OCR from the scanner 102 or file server 103, the process shown in Figure 3 begins.

[0035] In S301, CPU201 acquires images related to the form (hereinafter referred to as OCR target images). The OCR target images may be acquired directly from scanner102, or image files stored on file server103 may be acquired from file server103.

[0036] In S302, CPU201 performs preprocessing on the OCR target image acquired in S301 for subsequent processing. Preprocessing includes necessary operations for subsequent processing, such as binarization and noise reduction.

[0037] In S303, CPU201 determines the type of document related to the OCR target image. For example, when determining the type of document using the layout, this can be achieved by preparing a template image for each type of document, calculating the similarity of the layout between each template image and the OCR target image, and using this similarity to determine the type of document (so-called template matching).

[0038] Furthermore, the method for classifying documents by type is not limited to template matching; any method that can determine the type of document is acceptable. For example, the type can be determined by the title of the document, such as an invoice, receipt, or purchase order. In this case, the type is determined by the OCR result of the acquired document title. Therefore, this step may be executed after character recognition in S305. In addition, multiple classification methods may be used in combination.

[0039] Figure 7 shows an example of a process for determining the type of document related to the OCR target image.

[0040] Template image 701 is data stored on file server 103.

[0041] OCR target images 702 and 703 are the OCR target images to be processed.

[0042] Comparing template image 701 with OCR target image 702, we see that in template image 701, there are three slightly larger characters in the upper left corner of the document, and eight characters of similar size below them. Also, a table is included in the lower half of the document, and nothing is written below the table. Thus, the overall layout, including the size and placement of the characters and the placement of the table, is similar. Therefore, using template matching, both are determined to be of the same type. On the other hand, comparing template image 701 with OCR target image 703, in OCR target image 703, there are three slightly larger characters in the center of the document, and the table is also located almost in the center of the document. There is also a rectangular area below the table, and below that, there are smaller characters. Thus, since the layouts of template image 701 and OCR target image 703 are not similar, template matching will not determine that they are of the same type. Furthermore, the type can be determined by calculating the similarity between each stored template image 701 and the OCR target images 702 and 703, and then determining that it is the same type as the template image with the highest similarity. Alternatively, types with similarity exceeding a threshold can be presented, and the user can select the appropriate type.

[0043] Furthermore, when identifying the type based on the subject, for example, the largest string in the upper part of the document (which may be pre-defined as the upper half or upper third) can be obtained, and the type can be determined by whether the strings are identical (or by the degree of similarity between the strings). In template image 701, the string "Invoice" is obtained, and the string "Invoice" is also obtained from OCR target image 702. Since both are the same string, template image 701 and OCR target image 702 are determined to be of the same type. In the case of OCR target image 703, the string "Receipt" is obtained, so template image 701 and OCR target image 703 are determined to be of different types.

[0044] Furthermore, the system may be configured to combine template matching and title-based determination (combining multiple methods to determine the type). In this case, an evaluation value may be calculated by multiplying the similarity calculated by template matching with the result of the title-based determination, and the system may then determine whether the works belong to the same type based on the calculated evaluation value.

[0045] The information on the type of report obtained here is used to obtain the history of correction range A in S402 and to save correction range B in S508.

[0046] In this embodiment, the correction range refers to the area (a portion of the OCR target image) that the user has designated as an area where character detection was not performed in the OCR processing in S304 (OCR processing on the entire OCR target image) and where OCR processing should be performed again. Of the correction ranges, "Correction Range A" refers to the correction range specified by the user in a form that has been previously processed with OCR. Correction Range A is stored in the file server 103 in association with the form. "Correction Range B" refers to the correction range specified by the user in the OCR target image acquired in S301 (i.e., the OCR target image being processed).

[0047] CPU201 performs the first character detection process in S304, and subsequent character detection processes are performed from S307 onwards.

[0048] In S304, CPU201 performs character detection from the entire OCR target image. Specifically, CPU201 uses methods such as template matching and edge detection to find the outlines of characters and detect the areas containing characters. From the detected areas, CPU201 further divides them into individual regions, taking into account the distance between characters and the shape of the characters. From the detected individual regions, CPU201 extracts features necessary for character identification and performs character detection.

[0049] In the character detection steps from S307 onward, the same method as in S304 may be used, or a different detection method may be used.

[0050] In S305, CPU201 performs character recognition on the characters detected in S304. Specifically, CPU201 uses template matching, pattern recognition techniques, etc., to compare the shape and pattern of the characters, identify the correct characters, and recognize them as text. CPU201 then links the results of character recognition to each individual area.

[0051] In the character recognition steps from S307 onward, the same method as in S305 may be used, or a different recognition method may be used.

[0052] In this embodiment, the sequence of events from character detection in S304 to character recognition in S305 is referred to as OCR. In this embodiment, OCR includes OCR using AI (Artificial Intelligence) (hereinafter referred to as AI-OCR) and OCR without AI.

[0053] Figure 6 shows an example of a screen displaying the OCR results of an image targeted for OCR by CPU201.

[0054] Screen 600 displays the OCR results of a document that has undergone OCR processing.

[0055] Form 601 is the OCR target image acquired by S301. CPU201 displays the OCR results on the display unit. This screen also allows the user to input the correction range B.

[0056] In Figure 6, as shown in string 602, strings that have undergone OCR processing are displayed surrounded by a dashed rectangle. For example, the string "Invoice Amount: XXX yen" shown in string 603 is not surrounded by a dashed rectangle, indicating that it was not processed by OCR (character detection failed).

[0057] Furthermore, the method for identifying and displaying the OCR-processed text is not limited to enclosing it in a dashed rectangle. For example, it may also be enclosed in a rectangle with a solid line, wavy line, or dotted line, or it may be enclosed in a circle (including an ellipse) instead of a rectangle, or the area containing the text may be colored, such as by shading.

[0058] The End button 604 is a button that accepts the instruction to end the input of correction range B. The display of the End button 604 can be any display that indicates that the process has been completed, such as a button displaying "Complete," "Save," or "Confirm."

[0059] As shown in Figure 6, there are still strings that should be OCR'd but were not, such as string 603. The following explains how to efficiently perform OCR processing on strings that were missed during the OCR processes S304 and S305.

[0060] One reason why OCR may fail to detect certain characters is that the character size is either too large or too small relative to the area being processed. Specifically, if the area being processed is too wide, the relative character size (character size relative to the OCR area) becomes small, making it difficult to recognize the characters and increasing the likelihood of detection failure. On the other hand, if the area being processed is too narrow, the relative character size becomes large, also making it difficult to recognize the characters and increasing the likelihood of detection failure. This decrease in character detection and recognition accuracy due to improperly specified areas is due to the characteristics of OCR engines, especially machine learning models (trained models, AI) in general. Machine learning models typically resize the input image to a predetermined size before performing object detection and other processing. Therefore, it is necessary that the relative size, rather than the absolute size, of the characters in the input image is appropriate. As the character size also increases or decreases with resizing, it is necessary to select an area for OCR processing that is of a size suitable for character detection (not too large or too small). By specifying an appropriate size, characters become easier to recognize, and detection failures are reduced. Thus, the success or failure of OCR is influenced by the size of the area to be OCR'd. Therefore, in this embodiment, a process that allows the user to select a range of size suitable for OCR (where character detection is less likely to be missed) will be described below.

[0061] In S306, CPU201 determines whether a correction range has been specified in the past for a document of the same type as the OCR target document. Specifically, it determines this by checking whether correction range A is registered for a document of the same type as the OCR target document registered in file server 103. If a correction range has been specified in the past for a document of the same type as the OCR target document, the process transitions to S307; otherwise, it transitions to S308.

[0062] In S307, CPU201 performs re-OCR processing on document 601 based on correction range A stored in file server 103. The details of the S307 process are explained using the flowchart in Figure 4.

[0063] In S308, CPU201 accepts manual input from the user for specifying the correction range. The details of the S308 process are explained using the flowchart in Figure 5.

[0064] In S309, file server 103 saves the OCR-generated string from the process up to this point.

[0065] The above is an explanation of Figure 3.

[0066] Figure 4 is a flowchart showing the OCR processing flow using the recorded correction range A.

[0067] In S401, CPU201 obtains the missing item list 800. Items to be OCRed are defined in advance, and if OCR has not been performed on strings related to the defined items, CPU201 extracts the items for which OCR has not been performed and creates the missing item list 800.

[0068] Figure 8 shows an example of a missing item list.

[0069] The missing item list 800 in Figure 8 is a list of items that should be OCR processed but have not yet been processed at this stage, and consists of the item name 801 and a processing completion flag 802.

[0070] Item name 801 is the name of an item that should be OCR processed but has not yet been processed at this stage. In the missing item list 800 in Figure 8, items such as customer name, invoice amount, subtotal, and total amount are listed.

[0071] The processing completion flag 802 registers a flag indicating whether the OCR processing using correction range A in the flowchart of Figure 4 has been completed. In Figure 8, if the flag is set (meaning the OCR processing using correction range A has been completed), it is shown as "Processed". If it has not been completed, it is blank. In Figure 8, the fields "Customer Name" and "Invoice Amount" have had the OCR processing using correction range A completed, so these fields are registered as processed (the processing completion flag is set). The fields "Subtotal" and "Total Amount" have not had the OCR processing using correction range A completed, so they are not registered as processed.

[0072] In the missing items list 800, at the time of S401, all fields in the column for the processed flag 802 are blank.

[0073] In S402, CPU201 reads the correction range A for the same type of document as the one identified in S303 from file server 103. Then, steps S403 to S409 are repeated for the area in the OCR target image corresponding to the acquired correction range A. In S409, if CPU201 has performed OCR processing on all items registered in the missing item list 800, the loop process ends.

[0074] In S404, CPU201 extracts one item from the missing item list 800 obtained in S401 that has a blank processed flag 802.

[0075] In S405, CPU201 performs OCR processing on the area corresponding to the item acquired in S404. The specific method of OCR may be the same as in S304-S305, or a different method may be used.

[0076] As described above, by performing OCR processing using correction range A, which is a part of the image to be OCR-treated, rather than the entire image, the OCR processing is performed on relatively larger characters, thus improving the accuracy of the OCR processing. As a result, it becomes possible to perform OCR processing on characters that were missed when OCR processing was performed on the entire image to be OCR-treated. Furthermore, since correction range A is an area that has been specified by the user in the past for the same type of form, it can be said that this is an area that is prone to detection failures in that type of form. Since it becomes possible to perform OCR processing on such areas without receiving specification from the user each time, it is possible to reduce the effort required from the user and realize efficient OCR processing.

[0077] In S406, CPU201 sets a flag in the "Processing Completed Flag 802" column corresponding to the item for which re-OCR was performed.

[0078] In S407, CPU201 determines whether the re-OCR in S405 was successful. If the process is successful, it proceeds to S408. Otherwise, it skips the process in S408 and proceeds to S409.

[0079] In S408, CPU201 removes items that were processed successfully from the missing item list 800.

[0080] If re-OCR is performed on all items in the list, the process in this flowchart is terminated. If there are still items that have not been processed, the process from S404 onwards is performed on the next item. If all items in the missing item list 800 are deleted by repeating the process from S404 to S408, it means that all items that were missing from the OCR in the OCR target image have been successfully OCR'd. On the other hand, even if there are still missing items in the missing item list 800, if the "processed" flag is set for all items, it means that an attempt has been made to process all the missing items with OCR.

[0081] Furthermore, a limit on the number of times the OCR processing in S405 may fail (hereinafter referred to as the failure limit) may be set in advance, and if the number of OCR processing failures exceeds the failure limit, the loop processing may be terminated even if OCR processing has not been performed on the areas corresponding to all missing items. In other words, if the number of times the CPU201 fails to detect characters in the areas corresponding to the items acquired in S404 exceeds the failure limit, the process proceeds to receiving input for correction range B from the user and performing OCR processing using that correction range B (Figure 5). For example, if the failure limit is set to 2, and OCR processing is performed a total of 3 times on the areas corresponding to the three items "Customer Name", "Invoice Amount", and "Subtotal" in the missing item list 800, but no characters were detected, then in this case, although OCR processing has not yet been performed on the areas corresponding to the missing items from "Total Amount" onwards, the OCR processing has failed more than the failure limit of 2, so the CPU201 terminates the loop processing when the third OCR processing on the area corresponding to the "Subtotal" item fails. The reason for terminating the loop process midway is that if the OCR process fails multiple times, the likelihood of successful OCR processing for other missing items is low. By setting the number of attempts, the user can proceed to the process shown in Figure 5 without waiting for the OCR process to succeed (despite the low probability of success), thus reducing working time.

[0082] In the process shown in Figure 4, if corrections have previously been made to a document of the same type as the OCR target document, the correction can be performed using the recorded correction range A. This reduces the amount of manual correction required by the user, thereby easing the user's burden.

[0083] The above is an explanation of Figure 4.

[0084] Figure 5 is a flowchart showing an example of the OCR processing flow for the correction range B received from the user.

[0085] In S501, CPU201 displays the results of the OCR processing performed in S304-S307 on the OCR target image on the screen. An example of the screen displaying the OCR processing results is shown in Figure 6.

[0086] In S502, CPU201 accepts user operations to specify correction range B and operations to terminate the specification of correction range B.

[0087] In S503, CPU201 determines whether the user operation received in S502 is an operation to terminate the specification of correction range B. If it is an operation to terminate, the process moves to S508. Otherwise (if it is an operation to specify a correction range), the process proceeds to S504.

[0088] The operation to finish specifying the correction range is specifically the operation of clicking the finish button 604 on the screen shown in Figure 6.

[0089] Input for correction range B can be accepted, for example, by dragging with a mouse. Specifically, the drag operation accepts the instruction of the diagonal of the rectangle related to correction range B, and the rectangle with the diagonal from the start point to the end point of the drag operation is set as correction range B. Alternatively, the area related to a freely drawn shape created by the drag operation may be set as correction range B, or any range identified based on the user's operation may be set as correction range B. In this way, by allowing the user to specify / select the range that requires OCR processing, it becomes possible to perform OCR processing without omission in the range that requires it.

[0090] In S504, it is determined whether the correction range B received in S502 is a suitable size for OCR processing. If it is not a suitable size (S504: NO), the process moves to S505; if it is a suitable size (S504: YES), the process moves to S506.

[0091] One method for determining whether correction range B is a suitable size for OCR processing is to set a reference value in advance. If the size meets the reference value, it is considered a suitable size; if it does not, it is not considered a suitable size. As mentioned above, if the size of the characters to be OCRed is relatively small, OCR may fail. For example, a reference value A can be set as the upper limit of the size. If the size of correction range B is less than reference value A, it is considered a suitable size; if it is greater than or equal to reference value A, it is considered unsuitable. Alternatively, a reference value B can be set as the lower limit of the size. If the size of correction range B is greater than or equal to reference value B, it is considered suitable; if it is less than reference value B, it is considered unsuitable. Another method is to set both reference values ​​A and B, and determine that if the size of correction range B is less than reference value A and greater than or equal to reference value B, it is suitable; if it is greater than or equal to reference value A or less than reference value B, it is considered unsuitable.

[0092] In S505, CPU201 displays correction range B in a manner that allows recognition that it is unsuitable for OCR processing (a manner corresponding to the judgment result). Specifically, this can be done by, for example, displaying a border representing correction range B or displaying the area within correction range B in a predetermined color (e.g., red), or by displaying a notification with text such as "Inappropriate" or "NG" near correction range B. It may also display a prompt to adjust the size appropriately (for example, displaying text such as "Make it bigger" or "Make it smaller").

[0093] In S506, CPU201 displays correction range B in a way that allows it to recognize that it is suitable for OCR processing. Specifically, this can be done by, for example, by displaying a border representing correction range B, displaying the area within correction range B in a predetermined color (e.g., blue, a different color from the one used when it is unsuitable), or by displaying text such as "suitable" or "OK" near correction range B.

[0094] In this embodiment, the configuration is set up to determine whether something is suitable or unsuitable (a choice between two options). However, it is also possible to determine it in three stages, such as "optimal," "appropriate," and "inappropriate," or to express the degree of appropriateness numerically (for example, 1 to 100). If the above-mentioned color-based identification display is used, in the case of three stages, three colors such as "blue," "yellow," and "red" may be used. If the numerical representation is used, it is possible to display it in a way that makes it recognizable whether something is appropriate or not, such as by using a gradient from blue to red.

[0095] In this way, by providing a display that allows users to recognize whether the size of correction range B is appropriate, users will attempt to adjust the size of correction range B to make it the appropriate size, thus improving the accuracy of OCR processing for correction range B.

[0096] Furthermore, the method for notifying the user whether the size of correction range B is appropriate or inappropriate does not have to be displayed using color or text; for example, the user may be notified using sound. Specifically, if the size of correction range B is inappropriate, a voice notification such as "The selected range is too large. Please make it smaller." can be given, and if it is appropriate, a voice notification such as "The selected range is appropriate" can be given. Alternatively, an alert sound may be played if the size of correction range B is inappropriate. Any method is acceptable as long as it notifies the user whether the correction range B selected by the user is appropriate or inappropriate.

[0097] In S507, CPU201 performs OCR on correction range B. The specific method of OCR may be the same as in S304-S305, or a different method may be used. Note that if correction range B is determined to be of an appropriate size in 504, OCR will be performed. However, if correction range B is determined to be of an appropriate size, OCR may be performed as is despite the inappropriate size, or the user may be notified to reset correction range B without performing OCR. Alternatively, if OCR is performed, the user may be notified that accuracy may decrease and be given the option to choose whether or not to proceed with OCR.

[0098] In S508, the correction range B, where OCR has been performed, is saved to the file server 103. The items saved at this time are the coordinate information of correction range B, the type of image to be OCR'd, and the types of items that were recognized by OCR in correction range B. The correction range B saved by the file server 103 is used when OCR's the same type of document is performed in the future. That is, it is used as correction range A in the process shown in Figure 4.

[0099] As described above, the flowchart in Figure 5 shows a process where input for correction range B is received (S502), then it is determined whether correction range B is of an appropriate size (S504), and the determination result is displayed in a recognizable manner (S505, S506). However, it is also possible to perform the process by determining whether the size is appropriate while input for correction range B is being received (for example, while a drag operation is being performed, i.e., from the start to the end of the reception operation) (executing the process in S504), and displaying the determination result while input for correction range B is being received (executing the processes in S505, S506). By notifying the user whether the size is appropriate while input for correction range B is being received in this way, the user can recognize the appropriate size while inputting. Therefore, it is possible to prevent situations where the user has to re-enter the input after completing it because the size was not appropriate, thereby reducing the effort required for re-entry.

[0100] Furthermore, the OCR processing (S507) for correction range B may also be performed while input for correction range B is being received. Specifically, when input for correction range B is received by a drag operation, once the drag operation begins, the acquisition of the area identified by the operation is repeated at predetermined intervals (e.g., every 0.1 seconds) until the operation is completed. Then, each time an area is acquired, the system is controlled to perform OCR processing (or character detection processing alone, as described later). The area related to the string detected by the OCR processing is then displayed in an identifiable manner, such as by enclosing it in a rectangle. Alternatively, the result of the OCR processing (the string resulting from character recognition) may be displayed near the rectangular area.

[0101] Even when performing OCR in real time as described above, it is possible to perform OCR processing only if the size of correction range B is suitable for OCR processing, and not perform OCR processing if it is not suitable. For example, OCR processing may not be performed from the time input of correction range B is started until correction range B becomes suitable for OCR processing, but then OCR processing is performed once it becomes suitable. If the size of correction range B becomes unsuitable for OCR processing again, OCR processing may be withheld.

[0102] By displaying the OCR-processed text in a recognizable format in real time, users can understand whether OCR processing is being performed on the text they want to OCR (whether characters are being detected and whether appropriate character recognition is being achieved). As a result, users can set correction range B so that the text they want to OCR is detected and recognized, reducing the effort required to readjust correction range B.

[0103] Since OCR processing is a computationally intensive process, it may be possible to control the system to perform only character detection while accepting input for correction range B, and not character recognition, and then perform character recognition after input acceptance has finished. Alternatively, for the OCR processing performed while accepting input for correction range B, it may be possible to use an OCR engine with low detection and recognition accuracy but high processing speed and low processing load, and then use an OCR engine with high detection and recognition accuracy (but slow processing speed and heavy processing load) after correction range B has been determined.

[0104] Figure 9 shows the system accepting input for correction range B.

[0105] Figure 9 is an example of the input screen for correction range B, using the screen shown in Figure 6. Figure 9 is based on Figure 6, but some screens have been omitted for explanatory purposes.

[0106] Figure 9-1 shows the screen displaying the OCR target image 601 before accepting the user's specification of correction range B. This screen displays the results of the processing in steps S304 and S305 (as explained in Figure 6, the strings that have been OCR'd are indicated by dashed rectangles), and the user checks on this screen to confirm the strings on the form that have not been OCR'd.

[0107] Figure 9-2 shows how correction range B is specified by the user for a string that has not yet undergone OCR. A rectangle 901 indicating the specified correction range B is displayed on the screen. In Figure 9, the border 902 of rectangle 901 is displayed with a different shape from the border surrounding the string 602 that has already undergone OCR (in Figure 9, it is a dashed line, but other methods are also possible, such as differences in border thickness, spacing of dots in a dashed line, or using a solid or dashed line to distinguish it from the rectangle representing the string 602). By distinguishing the shape of the border, the user can recognize which part of the OCR target image 601 is being specified as correction range B without confusing it with the border surrounding the string 602 that has already undergone OCR or the border 903. In addition, the inside of the specified rectangle 901 may be displayed in any color. This allows the user to recognize which part of the form 601 is being specified as correction range B. The border line 903 is a dotted line used to display the result of the OCR processing (S507) performed on correction range B while input for correction range B is being received. The string enclosed by border line 903 is the string for which OCR processing was successful while input for correction range B was being received. The shape of border line 903 can be any shape that can distinguish it from border line 902 and the dotted line surrounding string 603, and it may be made more noticeable by blinking the border line or displaying a color on the border line. By distinguishing the shape of the border line, the user can recognize which string in the OCR target image 601 has been successfully processed for OCR without confusing it with the border line surrounding string 602 or border line 902, which have already been OCR processed.

[0108] Figure 9-3 is an example of a screen displaying the OCR results for correction range B using S507. For strings 904-906, the border line 903, which was dotted, has changed to a dashed line, indicating that these strings were successfully OCR'd.

[0109] Figure 10 shows an example of a data table for correction range A.

[0110] The data table 1000 in Figure 10 is a data table (S508) in which the correction range B for which OCR was successful is saved as correction range A for use when OCRing the same type of document in the future, and consists of the document type 1001, item name 1002, and coordinate information 1003.

[0111] Document type 1001 indicates which document type the items and coordinate information that were successfully processed by OCR in S507 were obtained from, and consists of a combination of the document name and an alphabet. Even for the same invoice, the coordinate information of the items differs depending on the type of invoice, so the documents are categorized by combining the document name and an alphabet, such as Invoice AAA, Invoice AAB, etc. Note that the code used to categorize documents does not have to be an alphabet; any other string, number, or symbol is acceptable as long as it identifies the type of document. In data table 1000 in Figure 10, "Customer Name," "Message from Person in Charge," and "Invoice Amount" are items that were OCR'd from Invoice AAA, while "Total Amount" and "Person in Charge Name" are items that were OCR'd from Receipt AAC.

[0112] Item name 1002 is the name of the item for which OCR was successful in S507. In data table 1000 in Figure 10, the customer name, message from the person in charge, and invoice amount were successfully OCR'd from invoice AAA, while the contact information and name of the person in charge were successfully OCR'd from receipt AAC.

[0113] Coordinate information 1003 is coordinate information indicating where the string corresponding to the item successfully processed by OCR in S507 was located on the OCR target image. Specifically, it is coordinate information indicating the rectangular area where the string existed, so it will be coordinate information for at least two points, such as the top-left and bottom-right vertices of the rectangular area. The coordinate information may also be expressed as a ratio (for example, (0.8, 0.15)) when the vertical and horizontal values ​​of the form are set to 1. Since the size of the image acquired by CPU201 in S301 is not constant, expressing it as a ratio allows it to handle cases where the size differs.

[0114] Note that depending on the correction range B entered by the user, there may be characters within correction range B that are not OCR-processed. In that case, the user may be able to correct the OCR by entering an additional correction range B that encloses the characters that were not OCR-processed. This process can be repeated until there are no more characters that were not OCR-processed on the form.

[0115] As described above, this embodiment makes it possible to reduce the user's burden in correcting OCR results.

[0116] The present invention can take the form of, for example, a system, apparatus, method, program, or recording medium. Specifically, it may be applied to a system consisting of multiple devices, or to an apparatus consisting of a single device.

[0117] Furthermore, the various controls described above, which are performed by CPU201, may be performed by a single piece of hardware, or multiple pieces of hardware (for example, multiple processors or circuits) may share the processing to control the entire device.

[0118] Furthermore, although the present invention has been described in detail based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention. Moreover, each of the embodiments described above is merely one embodiment of the present invention, and it is possible to combine each embodiment as appropriate.

[0119] Furthermore, although the above-described embodiments used the application of the present invention to a PC as an example, this is not limited to this example, and it can be applied to any device or software capable of performing OCR processing. For example, it can be applied to PDAs, mobile phone terminals (smartphones), wearable devices, tablet terminals, game consoles, and the like.

[0120] (Other embodiments) The present invention can also be realized by performing the following process: supplying software (program) that realizes the functions of the above-described embodiment to a system or device via a network or various storage media, and having the computer (or CPU or MPU, etc.) of that system or device read and execute the program code. In this case, the program and the storage media storing the program constitute the present invention.

Claims

1. A means for receiving the specification of the scope to be processed for OCR for documents subject to OCR processing, An OCR means that performs OCR processing on the range received by the aforementioned receiving means, A storage means that stores in association the type of document and information that can identify the scope of OCR processing specified by the receiving means, An OCR range acquisition means acquires information from the storage means that allows for the identification of the range to be processed by OCR based on the type of newly acquired document, Equipped with, The information processing apparatus is characterized in that the OCR means further performs OCR processing on a range specified by the information obtained by the OCR range acquisition means in a newly acquired document.

2. The information processing apparatus according to claim 1, characterized in that the OCR means performs OCR processing on a range specified by the information acquired by the OCR range acquisition means.

3. The aforementioned OCR means is further characterized by performing OCR processing on the entire document, The information processing apparatus according to claim 1, characterized in that the receiving means accepts a range specification on a screen that displays the results of OCR processing on the entire document by the OCR means.

4. The information processing apparatus according to claim 1, wherein the OCR means further performs OCR processing on the entire document, and if the string of characters relating to a predetermined item cannot be detected by the OCR processing on the entire document, the OCR processing is performed on the range specified by the information obtained by the OCR range acquisition means.

5. The information processing apparatus according to claim 1, further comprising means for identifying the type of document to be subjected to the OCR processing.

6. The information processing apparatus according to claim 1, characterized in that the OCR means performs OCR processing using AI.

7. The information processing device's receiving means includes a receiving step in which it receives a specification of the scope to be processed for OCR for a document that is subject to OCR processing, The OCR means of the information processing device performs an OCR process on the range received in the reception process, A control step in which the control means of the information processing device is controlled to store in the storage means the type of document and information that can identify the range to be processed by OCR processing, which has been specified in the reception step, in association with each other. The OCR range acquisition means of the information processing device acquires information from the storage means that allows for the identification of the range to be processed by OCR, based on the type of newly acquired document. The OCR means of the information processing device performs a second OCR processing step in which it performs OCR processing on a range specified by the information obtained in the OCR range acquisition step in a newly acquired document. An information processing method characterized by comprising:

8. A program for causing a computer to function as one of the means described in any one of claims 1 to 6.

Images