Information processing device, recording medium, information processing method, and computer program

Abstract

The present invention provides an information processing device, a recording medium, and an information processing method that can reduce the effort and time required to correct processing results, compared to a case where information regarding processing results related to local areas is not communicated to a preceding process. The information processing device includes a processor that performs the following processing: acquires image data from a preprocessor; calculates information regarding the accuracy of the processing result for each local area of ​​the image corresponding to the acquired image data; and communicates the calculated accuracy information to the preprocessor along with information identifying the corresponding local area.

Description

Technical Field

[0001] The present invention relates to an information processing device, a recording medium, and an information processing method. Background Art

[0002] A technology for converting characters printed on printed materials or handwritten character images into character codes that can be used by computers is known. This technology is called OCR (Optical Character Recognition). To obtain an image corresponding to a document containing characters, a so-called scanner or digital camera is used.

[0003] While it's possible to directly output image data captured by a scanner or digital camera to OCR processing, additional processing is sometimes performed beforehand to increase the accuracy of the character recognition results obtained through OCR processing (hereinafter referred to as "reliability"). For example, cleaning processing to remove noise or color blocks in the image is sometimes performed before OCR processing. Furthermore, the resolution of the image at the time of acquisition is sometimes set to a higher level. Hereinafter, processing performed before OCR processing is referred to as preprocessing.

[0004] Patent Document 1: Japanese Patent Application Laid-Open No. 2015-146075

[0005] Currently, cleanup processing is performed on the entire image. Similarly, the reliability of OCR processing is calculated for the entire image or data file being processed. Therefore, even if the OCR processing includes areas with reduced reliability, if the overall reliability is high, the processing results including the low-reliability areas are directly passed on to downstream processing. Therefore, if important information is included in the low-reliability areas, it may lead to reduced accuracy in downstream processing. Summary of the Invention

[0006] An object of the present invention is to reduce the effort and time required to correct a processing result, compared to a case where information on a processing result related to a local area is not notified to a preceding-stage process.

[0007] The invention described in Option 1 is an information processing device having a processor that performs the following processing: acquiring image data from preprocessing; calculating information related to the accuracy of the processing result for each local area of ​​the image corresponding to the acquired image data; and notifying the preprocessing of the information related to the calculated accuracy together with information determining the corresponding local area.

[0008] The invention described in Scheme 2 is an information processing device described in Scheme 1, wherein the processor calculates information related to the accuracy of the processing result for each of a plurality of small areas that further divide the local area, and notifies the preprocessing of the accuracy of the processing result calculated for each of the plurality of small areas together or separately.

[0009] The invention described in claim 3 is the information processing device described in claim 1 or 2, wherein the information related to the accuracy indicates that the accuracy of the processing result is lower than a predetermined threshold value.

[0010] The invention described in claim 4 is the information processing device described in claim 1, wherein the processor estimates a reason why the accuracy of the processing result is lower than a predetermined threshold value, and notifies the pre-processor of the estimated reason as information related to the accuracy.

[0011] The invention described in claim 5 is the information processing device described in claim 4, wherein the information related to the cause is information related to characters or background included in the partial area.

[0012] The invention described in claim 6 is the information processing device described in claim 4, wherein the processor estimates the cause including the accuracy of the processing result with respect to the entire image data.

[0013] The invention described in claim 7 is the information processing device described in claim 4, wherein the processor estimates the cause based on information on the processing content provided from the pre-processing.

[0014] The invention described in claim 8 is the information processing device described in claim 4, wherein the processor estimates the cause based on a history of causes estimated for other image data similar to the image data.

[0015] The invention described in claim 9 is the information processing device described in claim 4, wherein the processor estimates the cause based on the difference in accuracy between local areas of the same type.

[0016] The invention described in claim 10 is the information processing device described in claim 1, wherein the processor notifies the preprocessor of information indicating the content of the change in the preprocess.

[0017] The invention described in claim 11 is the information processing device described in claim 1, wherein the processor notifies the pre-processor of information indicating the processing content in response to the pre-processing request.

[0018] The invention described in claim 12 is the information processing device described in claim 11, wherein the information indicating the content of the pre-processing includes information indicating parameter values ​​used in the pre-processing.

[0019] The invention described in claim 13 is the information processing device described in claim 11, wherein the processor estimates the processing content for the pre-processing request based on a history of instructions for other image data similar to the image data.

[0020] The invention described in Scheme 14 is an information processing device described in Scheme 11, wherein the processor learns the content of the preprocessing notification and the accuracy of the result of processing the local area acquired again after the notification as teacher data, and infers the processing content of the preprocessing request.

[0021] The invention described in Scheme 15 is that in the information processing device described in Scheme 1, when the processor reacquires the image data that has been re-processed with different preprocessing, and when the processing results of each local area corresponding to the image data are higher than a predetermined threshold, the processor outputs the result of processing the re-acquired image data.

[0022] The invention described in Scheme 16 is an information processing device described in Scheme 1, wherein when the processor again obtains image data on which different pre-processing has been performed on a local area that has obtained an accuracy lower than a predetermined threshold, and the processing result thereof is higher than the threshold, the processing result of the local area is combined with the processing results of other local areas that obtained an accuracy higher than the threshold in the previous processing and outputs the combined data.

[0023] The invention described in claim 17 is the information processing device described in claim 1, wherein the processor shares information identifying the image data that is the target of the notification with the pre-processor.

[0024] The invention described in claim 18 is the information processing device described in claim 1, wherein the processor instructs the pre-processing for generating the image data to increase the resolution.

[0025] The invention described in claim 19 is the information processing device described in claim 1, wherein, when the accuracy of the processing result exceeds a predetermined threshold value for all local areas, the processor notifies the pre-processing of the end of the processing.

[0026] Solution 20 of the present invention is a recording medium, which records a program for enabling a computer to implement the following functions: a function of acquiring image data from preprocessing; a function of calculating information related to the accuracy of the processing result for each local area of ​​the image corresponding to the acquired image data; and a function of notifying the preprocessing of the information related to the calculated accuracy together with information determining the corresponding local area.

[0027] Solution 21 of the present invention is an information processing method, which includes the following steps: obtaining image data from preprocessing; calculating information related to the accuracy of the processing result for each local area of ​​the image corresponding to the obtained image data; and notifying the preprocessing together with the information related to the calculated accuracy and the information determining the corresponding local area.

[0028] Effects of the Invention

[0029] According to the first aspect of the present invention, compared with a case where information on the processing result regarding the local area is not notified to the preceding-stage process, the effort and time required to correct the processing result can be reduced.

[0030] According to the second aspect of the present invention, even in a local area, the accuracy can be notified in smaller area units.

[0031] According to the third aspect of the present invention, it is possible to selectively notify a region where a change in the content of pre-processing is required.

[0032] According to the fourth aspect of the present invention, the content of preprocessing on the preprocessing side can be changed efficiently.

[0033] According to the fifth aspect of the present invention, the content of preprocessing on the preprocessing side can be changed efficiently.

[0034] According to the sixth aspect of the present invention, it is possible to identify the cause caused by the preprocessing targeting the entire image.

[0035] According to the seventh aspect of the present invention, the accuracy of estimating the cause can be improved.

[0036] According to the eighth aspect of the present invention, the accuracy of estimating the cause can be improved.

[0037] According to the ninth aspect of the present invention, the accuracy of estimating the cause can be improved.

[0038] According to the tenth aspect of the present invention, the accuracy of the post-processing result can be changed.

[0039] According to the eleventh aspect of the present invention, the content of preprocessing on the preprocessing side can be changed efficiently.

[0040] According to the twelfth aspect of the present invention, the content of preprocessing on the preprocessing side can be changed efficiently.

[0041] According to the thirteenth aspect of the present invention, the accuracy of notification of instructions for pre-processing can be improved.

[0042] According to the fourteenth aspect of the present invention, the accuracy of notification of instructions for pre-processing can be improved.

[0043] According to the fifteenth aspect of the present invention, it is unnecessary to correct the processing result.

[0044] According to the sixteenth aspect of the present invention, processing results can be obtained efficiently.

[0045] According to the seventeenth aspect of the present invention, unnecessary processing can be omitted in both pre-processing and post-processing.

[0046] According to the eighteenth aspect of the present invention, the accuracy of the post-processing result can be effectively improved.

[0047] According to the nineteenth aspect of the present invention, it is possible to notify the pre-processor that the image data can be deleted.

[0048] According to the twentieth aspect of the present invention, compared with a case where information on the processing result regarding the local area is not notified to the preceding-stage process, the effort and time required to correct the processing result can be reduced.

[0049] According to the twenty-first aspect of the present invention, compared to a case where information on the processing result regarding the local area is not notified to the preceding-stage process, the effort and time required to correct the processing result can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Embodiments of the present invention will be described in detail with reference to the following drawings.

[0051] Figure 1 is a diagram showing a configuration example of an information processing system used in the embodiment;

[0052] Figure 2 This is a diagram illustrating an example of the hardware configuration of the OCR processing server used in Embodiment 1;

[0053] Figure 3 This is a diagram illustrating an overview of processing performed in the first embodiment;

[0054] Figure 4 This is a flowchart illustrating an example of processing performed by the OCR processing server in Embodiment 1;

[0055] Figure 5 A diagram illustrating an example of a document to be read by an image processing apparatus;

[0056] Figure 6 A diagram illustrating an example of an object separated from image data;

[0057] Figure 7 A diagram illustrating the relationship between the reliability calculated for a table region and the threshold value;

[0058] Figure 8 This is a diagram illustrating an overview of processing performed in the second embodiment;

[0059] Figure 9 This is a flowchart illustrating an example of processing performed by the OCR processing server in Embodiment 2;

[0060] Figure 10 This is a diagram illustrating an overview of processing performed in Embodiment 3;

[0061] Figure 11 This is a flowchart illustrating an example of processing performed by the OCR processing server in Embodiment 3;

[0062] Figure 12 This is a diagram illustrating an overview of processing performed in the fourth embodiment;

[0063] Figure 13 This is a flowchart illustrating an example of processing performed by the OCR processing server in Embodiment 4;

[0064] Figure 14 This is a diagram explaining an example of a cause estimated when the target data is a table area;

[0065] Figure 15 This is a diagram illustrating an overview of processing performed in Embodiment 5;

[0066] Figure 16 This is a flowchart illustrating an example of processing performed by the OCR processing server in Embodiment 5;

[0067] Figure 17 This is a flowchart illustrating an example of processing performed by the OCR processing server in Embodiment 6;

[0068] Figure 18 This is a diagram illustrating an overview of processing performed in Embodiment 7;

[0069] Figure 19 This is a diagram illustrating an overview of processing performed in the eighth embodiment;

[0070] Figure 20 This is a flowchart illustrating an example of processing performed by the OCR processing server in Embodiment 8;

[0071] Figure 21 This is a diagram for explaining an overview of the processing performed in the ninth embodiment.

[0072] Explanation of symbols

[0073] 1-Information processing system, 10-Image processing device, 20-OCR processing server, 21-Calculation unit, 21A-Processor, 22-Storage device, 23-Communication device, 30-Cloud network. DETAILED DESCRIPTION

[0074] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0075] <Implementation Method>

[0076] System Structure

[0077] Figure 1 It is a diagram showing a configuration example of an information processing system 1 used in the embodiment.

[0078] Figure 1 The illustrated information processing system 1 includes an image processing apparatus 10 , an OCR processing server 20 that recognizes characters included in image data supplied from the image processing apparatus 10 , and a cloud network 30 serving as a network environment.

[0079] The image processing apparatus 10 in this embodiment has functions such as creating a copy of a manuscript, printing a document or image on paper, optically reading a manuscript and creating image data, and transmitting and receiving faxes.

[0080] exist Figure 1 The upper portion of the main body of the image processing device 10 shown is equipped with a mechanism that feeds documents one by one to a position where the information is optically read. This mechanism is called, for example, an ADF (Auto Document Feeder). The ADF is used to read copied documents or faxed documents.

[0081] The function of printing a document or image on paper is also used in creating a copy. The data of the document or image is not only optically read in the image processing device 10, but also supplied from a recording medium connected to the main body or an external information terminal.

[0082] The functions provided in the image processing apparatus 10 are not limited to the functions described above. However, in the case of this embodiment, as long as the image processing apparatus 10 is provided with a function of optically reading document information and generating image data, other functions are optional.

[0083] In addition to paper with printed characters or images, the manuscript in this embodiment can also be a document or image with handwritten characters. The handwritten characters can be part of the document or image. That is, the characters in the document do not need to be all handwritten.

[0084] In this embodiment, handwritten documents are assumed to include, for example, applications, bills, delivery notes, invoices, and other handwritten forms. Handwritten forms contain characters written within pre-printed boxes. Handwritten documents are not limited to forms and can also include communication memos, circulated documents, postcards, sealed letters, and the like.

[0085] The image processing apparatus 10 in this embodiment also has a function of removing noise and color blocks from an image read from a document. In this embodiment, the image data after the noise and the like have been removed is sent to the OCR processing server 20 .

[0086] Figure 1 Although only one image processing apparatus 10 is shown in FIG, a plurality of image processing apparatuses 10 may constitute the information processing system 1. The image processing apparatus 10 in this embodiment is an example of an external apparatus.

[0087] The OCR processing server 20 in this embodiment is designed to perform OCR processing on image data received from the image processing device 10 and transfer the resulting text data to the image processing device 10. The image processing device 10, to which the text data is transferred, then performs post-processing on the received text data. This post-processing includes, for example, language processing, associating the text data with its correct administrative location, searching for documents related to the text data, and searching for routes used for delivering items. The post-processing details are determined based on the content of the document being read or the required processing.

[0088] Furthermore, the OCR processing server 20 of this embodiment is also provided with a function of feeding back information related to the reliability of the OCR processing results for each local area to the pre-processing process. This function is provided to improve the accuracy of the character recognition results obtained through the OCR process or to improve the quality or precision of the post-processing results.

[0089] The OCR processing server 20 in this embodiment is an example of an information processing device.

[0090] However, in this embodiment, the operator of the image processing apparatus 10 and the operator of the OCR processing server 20 may be the same or different.

[0091] In this embodiment, an OCR processing server 20 dedicated to OCR processing is used, but a general-purpose server capable of multiple functions may also be used. Furthermore, the computer performing OCR processing is not limited to a server. The computer performing OCR processing may be, for example, a desktop computer or laptop computer, or a smartphone or tablet terminal.

[0092] exist Figure 1In the case of , there is only one OCR processing server 20, but there may be a plurality of OCR processing servers 20 constituting the information processing system 1. The plurality of OCR processing servers 20 can perform distributed processing on one piece of image data.

[0093] In the present embodiment, the image processing apparatus 10 and the OCR processing server 20 communicate using the cloud network 30. However, the communication is not limited to communication via the cloud network 30. For example, a LAN (Local Area Network), 4G, or 5G mobile communication system may be used for communication between the image processing apparatus 10 and the OCR processing server 20.

[0094] <OCR Processing Server Structure>

[0095] Figure 2 This is a diagram illustrating an example of the hardware configuration of the OCR processing server 20 used in the first embodiment. Figure 2 The illustrated OCR processing server 20 includes a computing unit 21 for performing OCR processing, a storage device 22 for storing image data acquired from the image processing device 10, and a communication device 23 for external communication. Furthermore, a receiving device for receiving user instructions or a display device for displaying information to the user may be connected to the OCR processing server 20. The computing unit 21 and the various components are connected via a bus 24 or signal lines (not shown).

[0096] The arithmetic unit 21 in this embodiment includes a processor 21A, a ROM (read-only memory) (not shown) storing basic software and the BIOS (Basic Input Output System), and a RAM (random access memory) (not shown) used as a work area. The arithmetic unit 21 functions as a so-called computer. The aforementioned OCR processing is achieved by the processor 21A executing a program.

[0097] The storage device 22 is composed of a hard disk device or a nonvolatile rewritable semiconductor memory. For example, image data acquired from the image processing device 10 and text data obtained by OCR processing are stored in the storage device 22. Application programs can also be stored in the storage device 22.

[0098] The communication device 23 is composed of a module compliant with a wired or wireless communication standard. Examples of the communication device 23 include an EtherNet (registered trademark) module, a USB (Universal Serial Bus), a wireless LAN, and a fax modem.

[0099] <Processing Details>

[0100] Hereinafter, a process executed by cooperation between the image processing apparatus 10 and the OCR processing server 20 will be described.

[0101] <Overview of the Process>

[0102] Figure 3 This figure is a diagram for explaining the outline of the processing performed in the first embodiment. The processing in this embodiment is composed of five types of processing. The five types of processing are processing of acquiring image data of the original document, pre-processing of the acquired image data, OCR processing of the pre-processed image data, post-processing of the text data obtained as a result of the OCR processing, and storing the post-processed results in the storage device 22 (refer to FIG. 2 ). Figure 2 ) in the process.

[0103] In the case of this embodiment, the OCR processing server 20 (refer to Figure 1 ) performs OCR processing, and the other four types of processing are performed by the image processing device 10 (reference Figure 1 )implement.

[0104] In the case of this embodiment, as pre-processing, a cleaning process to remove noise or color blocks or a process of separating objects is performed. On the other hand, as post-processing, a process of extracting the value corresponding to the key or the key corresponding to the value is performed with reference to a dictionary in which a combination of values ​​(=value) corresponding to the key (=skey) is registered. The keys and values ​​in this embodiment correspond to characters or images. For example, when the key is a name, Fujitaro corresponds to the value. In other words, the key is a character or graphic representing an item, and the value is a character or graphic representing the specific content corresponding to the item.

[0105] In the case of this embodiment, information that specifies the processing target between the pre-processing and the OCR processing is notified from the pre-processing to the OCR processing. Figure 3 In the case of a pre-processed image data, a file name, which is information identifying the processing target, is notified as data attached to the pre-processed image data. In this embodiment, the file name is composed of the reading date and time, the name of the user who performed the reading operation, information that identifies the image processing device 10 used for reading, etc. However, the information that identifies the file name is not limited to this.

[0106] On the other hand, when the OCR process provides feedback to the preprocessor, it explicitly indicates the processing target, for example, using a file name. By notifying the preprocessor of this file name, the preprocessor and the OCR process can collaborate. For example, when the preprocessor and the OCR process process multiple image data in parallel, the file name can be used to distinguish the processing targets. Furthermore, as long as the processing target can be identified, the information from the preprocessor notification to the OCR process is not limited to the file name.

[0107] In the case of this embodiment, a process of separating image data into a plurality of objects is also performed in pre-processing.

[0108] In the case of this embodiment, four areas are used as objects: the area for characters (hereinafter referred to as the "character area"), the area for tables (hereinafter referred to as the "table area"), the area for graphics (hereinafter referred to as the "graphic area"), and the area for figures (hereinafter referred to as the "figure area").

[0109] For example, the area containing the title, characters, and numerical values ​​of the original document is cut as the character area. The table itself or the title attached to the table is cut as the table area. The area with a pattern such as a company name is cut as the graphic area or the image area. The rest of the area is the background. Each object is an example of a local area.

[0110] In addition, the background, graphic area, and image area are excluded from the OCR processing targets. Therefore, the image data corresponding to the character area and table area is sent as object data from the image processing device 10 to the OCR processing server 20. In addition, each object data is given information to identify each local area.

[0111] Information identifying a local area may include, for example, coordinates or a serial number indicating a location within the image data of the captured document. Coordinates may be assigned, for example, as one or more coordinate points defining the outer edge of the area. If the local area is rectangular, the coordinate point of, for example, the upper left corner of the local area may be used.

[0112] In the case of this embodiment, the pre-processed image data is sent from the pre-processing to the OCR processing in units of local regions.

[0113] Furthermore, information identifying the content of the pre-processing performed can be notified from the pre-processing to the OCR process. The pre-processing content can be used to estimate the cause of low reliability on the OCR process side.

[0114] In the case of this embodiment, information indicating that the reliability of the result of the OCR processing is low is fed back from the OCR processing to the pre-processing. The reliability is an example of information related to accuracy.

[0115] In this embodiment, the local area is used to represent the individual areas being clipped. In other words, if there are multiple character areas, different information may be fed back for each character area. The same applies to table areas. Furthermore, different information may be fed back for each row or column of a table area.

[0116] In this embodiment, information identifying each local area is fed from the pre-processing process to the OCR process. Consequently, the feedback from the OCR process to the pre-processing process includes information identifying each local area. However, in the OCR process, it is also possible to treat multiple local areas of the same object type as a single local area and calculate the reliability, and then feed back information indicating that the reliability is low along with the information identifying the multiple local areas for which the reliability was calculated.

[0117] In this embodiment, feedback is not performed for local areas whose reliability exceeds a predetermined threshold. Thus, for all local areas, if the reliability exceeds the predetermined threshold, feedback from OCR processing to pre-processing is not performed. This is because text data with high reliability is obtained.

[0118] In this embodiment, the average of the credibility calculated for each character extracted from the corresponding local area is calculated as the credibility of the local area. Characters herein also include numbers or symbols. Different weights can be used to calculate the average for each local area. For example, different weights can be used for character areas and table areas.

[0119] Furthermore, for each type of object corresponding to a local region, the threshold used in the credibility evaluation of the local region can be different or the same. For example, different weights can be used to calculate credibility for the character region and the table region. Furthermore, within the same type of local region, different weights can be used for the title and the text.

[0120] Furthermore, the feedback from the OCR process to the pre-processing may be feedback that does not specify a local area. In this case, the OCR process also identifies which local area has a high reliability and which local area has a low reliability.

[0121] Therefore, during OCR processing, only the local area with low reliability in the previous OCR process can be selected from the image data that has been newly pre-processed, and the change in reliability can be confirmed. In addition, during OCR processing, only text data with a reliability exceeding a threshold can be selectively output to post-processing.

[0122] <Processing performed by the OCR processing server>

[0123] Figure 4This is a flowchart for explaining an example of processing executed by the OCR processing server 20 in Embodiment 1. The symbol S shown in the figure indicates a step. Figure 4 The processing shown is performed by the processor 21A (refer to Figure 2 )implement.

[0124] In the present embodiment, the image processing apparatus 10 receives the image data which has been pre-processed and receives the instruction to read the document with OCR processing, and thus starts the process. Figure 4 In addition, each image data corresponds to each local area determined to be a character area or a table area. Each image data is given information for identifying the local area.

[0125] First, a document serving as a source of image data to be processed by the OCR processing server 20 will be described.

[0126] Figure 5 This is a diagram for explaining an example of a document to be read by the image processing apparatus 10 . Figure 5 The manuscript shown is titled "Quotation Sheet" and is supplemented with color blocks throughout the paper. Figure 5 The quotation sheet shown includes two tables: Table A at the upper level and Table B at the lower level. Figure 5 Tables A and B are each composed of three lines. The item names that form the headings of Tables A and B are printed with blank characters on a black background. Meanwhile, the second and third lines of Table A are printed with black characters on a white background. The second and third lines of Table B are printed with characters on a colored background. The characters can be black, blank, or colored. Furthermore, a shaded background is also possible.

[0127] Figure 6 This is a diagram illustrating an example of an object separated from image data. Figure 6 In this case, the region containing the character strings "quotation," "ABC Industry," "XYZ Chamber of Commerce," and "total amount 16,000 yen" is separated from the image data as a character region. Furthermore, the region containing the table corresponding to the characters in Table A and the table corresponding to the characters in Table B is separated from the image data as a table region. Furthermore, the logo located in the lower right corner of the image data is separated from the image data as a graphic region or an image region.

[0128] Preprocessing can be performed before or after object separation. In this embodiment, object separation is performed after preprocessing.

[0129] Return to Figure 4 Description.

[0130] The processor 21A receives the image data corresponding to each partial area determined to be a character area or a table area, and performs OCR processing on each image data corresponding to each partial area (step S1). The text data is extracted from each image data through the OCR processing.

[0131] In the present embodiment, a cleanup process is performed on the image data to be processed by OCR in the pre-processing to remove noise, color blocks, etc. However, there is no guarantee that the content of the cleanup process will be consistent with the content of the actual local area.

[0132] Furthermore, information for identifying each local area is added to each local area, and the information for identifying the local area is also associated with the extracted text data.

[0133] Next, the processor 21A calculates the credibility for each local area (step S2). For example, the credibility is calculated based on each local area corresponding to the character area or each local area corresponding to the table area. However, the credibility can also be calculated based on the area unit within the local area. For example, the credibility can also be calculated based on the row unit or the column unit.

[0134] Next, the processor 21A determines whether the calculated reliability is greater than or equal to a threshold value (step S3). As described above, the threshold value may be different or the same depending on the type of object corresponding to the local area.

[0135] Figure 7 This is a diagram illustrating the relationship between the reliability calculated for one table region and the threshold value. Figure 7 The example shown calculates the confidence level for each row in the table. Of course, it is also possible to compare a confidence level calculated without distinguishing rows with a threshold.

[0136] Figure 7 The examples shown assume that Figure 5 Table A or Table B in. Therefore, the number of rows is 3. Figure 7 In the case of , there are 8 combinations of credibility for one table area. Figure 7 In

[0045] , these are represented by combinations 1 to 8. The number of combinations depends on the number of rows constituting the table area or the number of credibility notified for one table area.

[0137] In addition, if Figure 7 Regarding the background colors of the rows corresponding to combinations 1 to 8, if the background colors of table A or table B are different in even and odd rows, the reliability can be calculated in units of odd and even rows even if the number of rows increases.

[0138] Combination 1 is a case where each reliability corresponding to the first to third rows is higher than the threshold value.

[0139] Combination 2 is a case where the respective reliability corresponding to the first and second rows is higher than the threshold, but the reliability corresponding to the third row is lower than the threshold.

[0140] Combination 3 is a case where the respective reliability corresponding to the first and third rows is higher than the threshold, but the reliability corresponding to the second row is lower than the threshold.

[0141] Combination 4 is a case where the reliability corresponding to the first row is higher than the threshold, but the reliability corresponding to the second row and the third row is lower than the threshold.

[0142] Combination 5 is a case where the respective reliability corresponding to the second and third rows is higher than the threshold, but the reliability corresponding to the first row is lower than the threshold.

[0143] Combination 6 is a case where the reliability corresponding to the second row is higher than the threshold, but the reliability corresponding to the first row and the third row is lower than the threshold.

[0144] Combination 7 is a case where the reliability corresponding to the third row is higher than the threshold, but the reliability corresponding to the first row and the second row is lower than the threshold.

[0145] Combination 8 is a case where each reliability corresponding to the first to third rows is lower than the threshold value.

[0146] Return to Figure 4 Description.

[0147] If a positive result is obtained in step S3, the processor 21A outputs the OCR processing result to the image processing device 10 for post-processing (step S4). This is because if a positive result is obtained in step S3, the reliability of all regions is higher than the threshold, and re-preprocessing is not required.

[0148] On the other hand, if a negative result is obtained in step S3, processor 21A feeds back information indicating that the reliability of the OCR processing result for the specific area is low to the pre-processing of image processing apparatus 10 (step S5). Low reliability means that the reliability is lower than a threshold.

[0149] Here, the feedback also includes information for identifying the local area with low reliability. When re-execution of the pre-processing is limited to the corresponding specific local area, the information for identifying the local area is required.

[0150] In the feedback to the preprocessing, information indicating that the confidence level of the entire table region is low can be fed back. However, Figure 7 As shown in the example, the rows constituting the table area can also be specified to feedback information indicating low reliability. By specifying a narrower range than the local area as the low reliability area, the accuracy of estimating the cause of the reduced reliability of the OCR processing result can be improved.

[0151] Furthermore, even if only information indicating a local area with low reliability is fed back, it can be estimated that the reliability of the pre-processing side is higher than that of other areas that have not been notified.

[0152] After executing step S5, the processor 21A returns to step S1. Specifically, it enters a state of waiting for reception of image data that has been re-preprocessed. The above-mentioned process is repeated until a positive result is obtained in step S3.

[0153] Thus, in this embodiment, during a period including a local area where the calculated reliability falls below a predetermined threshold, this information is fed back to the pre-processing process. Furthermore, for the pre-processing process that received feedback indicating low reliability, a different pre-processing content from the previous one is selectively executed, allowing the newly obtained image data to be submitted to the OCR process again. By changing the pre-processing content, the likelihood of obtaining a reliability level above the threshold is increased.

[0154] However, in this embodiment, only the fact that low reliability was obtained is fed back. Therefore, the reliability of the result of OCR processing of image data after performing pre-processing of another content is not necessarily limited to exceeding the threshold. On the contrary, the reliability may be lower than the previous time.

[0155] However, by repeatedly feeding back information indicating low confidence, the likelihood of obtaining high confidence for all local areas increases. As a result, text data with a confidence level exceeding the threshold is assigned during post-processing by the image processing device 10, improving the accuracy and reliability of the post-processing results. Furthermore, the effort and time required to manually confirm or correct recognized text data is reduced.

[0156] In addition, in the present embodiment, only information with low reliability is fed back, but information indicating a local area with high reliability may also be fed back.

[0157] <Implementation Method 2>

[0158] Figure 8 This is a diagram for explaining the outline of the processing executed in the second embodiment. Figure 8 The annotations in the middle correspond to Figure 3 The corresponding parts are shown with symbols.

[0159] In the case of this embodiment, the reliability calculated for each local area is fed back from the OCR process to the pre-processing.

[0160] Figure 9 This is a flowchart for explaining an example of processing executed by the OCR processing server 20 in the second embodiment. Figure 9 The annotations in the middle correspond to Figure 4The corresponding parts are shown with symbols.

[0161] exist Figure 9 In the case of , the content of step S5A executed when a negative result is obtained in step S3 is different from that in embodiment 1. In embodiment 1, information indicating that the reliability of the OCR processing result related to a specific area is low is fed back, but in this embodiment, the reliability itself is fed back.

[0162] Since the reliability itself is fed back, the determination of whether the reliability is higher than a threshold value is also performed on the pre-processing side.

[0163] <Implementation Method 3>

[0164] Figure 10 This is a diagram for explaining an overview of processing executed in the third embodiment. Figure 10 The annotations in the middle correspond to Figure 3 The corresponding parts are shown with symbols.

[0165] In the case of this embodiment, information requesting a change in the content of pre-processing for a specific local area is fed back from the OCR process to the pre-processing.

[0166] Figure 11 This is a flowchart for explaining an example of processing executed by the OCR processing server 20 in the third embodiment. Figure 11 The annotations in the middle correspond to Figure 4 The corresponding parts are shown with symbols.

[0167] exist Figure 11 In the case of a negative result in step S3, the content of step S5B executed is different from that in embodiment 1. In embodiment 1, information indicating that the reliability of the OCR processing result for a specific area is low is fed back, but in this embodiment, information requesting a change in the pre-processing content is fed back. In embodiments 1 and 2 above, information evaluating the OCR processing result is fed back, but in this embodiment, a request for pre-processing is fed back.

[0168] <Implementation Method 4>

[0169] Figure 12 This is a diagram for explaining an overview of processing executed in the fourth embodiment. Figure 12 The annotations in the middle correspond to Figure 3 The corresponding parts are shown with symbols.

[0170] In the case of this embodiment, the cause of the reduction in reliability is fed back from the OCR process to the pre-processing.

[0171] Figure 13This is a flowchart for explaining an example of processing executed by the OCR processing server 20 in the fourth embodiment. Figure 13 The annotations in the middle correspond to Figure 4 The corresponding parts are shown with symbols.

[0172] exist Figure 13 In the case of , the content of step S5C executed when a negative result is obtained in step S3 is different from that in embodiment 1.

[0173] When the processor 21A in the present embodiment obtains a negative result in step S3 , it estimates the cause of the decrease in reliability for the corresponding region and feeds back the estimated cause to the pre-processing of the image processing device 10 .

[0174] Processor 21A reads information such as the presence of color patches, background, font size, stains, folds, the relationship between background and character colors, and document type from image data and uses it to estimate the cause.

[0175] For the estimation in step S5C, for example, a correspondence table prepared in advance, a completed learning model updated by machine learning, or a determination program can be used.

[0176] The correspondence table stores combinations of features acquired from image data of a local area and, when the reliability is low, the presumed cause. However, the recommended preprocessing content for each combination may also be stored.

[0177] Furthermore, when the completed learning model is used, inputting image data corresponding to the local area into the completed learning model outputs the cause. However, it is also possible to output recommended pre-processing content when inputting image data corresponding to the local area into the completed learning model.

[0178] Furthermore, when using a determination program, the reason for the reduction in reliability is outputted by repeating the divergence caused by a single determination once or multiple times. In this case, instead of outputting the reason, the recommended pre-processing content may be outputted.

[0179] In the case of this embodiment, image data of a local area determined to have low reliability and the previous cause estimated for the corresponding image data are machine-learned as teacher data.

[0180] The learning unit is not limited to a local area, and may be a unit of object type, a unit of document type, or a unit of similar images.

[0181] In addition, even if the object types are the same, such as Figure 5If the background or character combination is different between Table A and Table B, the content of the preprocessing that helps improve the reliability may be different. Therefore, in this embodiment, machine learning is performed on a local area basis.

[0182] In this embodiment, reinforcement learning is used for machine learning. In reinforcement learning, learning is performed in a reward-based manner. Therefore, for preprocessed content that only obtains a confidence level below a threshold, no reward or only a low reward is given. On the other hand, if a positive result is obtained in step S3, a high reward is given for preprocessed content that obtains a confidence level above the threshold.

[0183] The results of machine learning are used to infer the cause of low credibility in subsequent iterations. When image data corresponding to a local region is applied to a completed learning model that has undergone reinforcement learning, the cause of low credibility in that local region is output. Improving the accuracy of reinforcement learning can also reduce the number of feedback cycles. Compared to randomly requesting changes to preprocessing content, this increases the likelihood that credibility will exceed the threshold. Learning can be performed on a local region, object, or document type basis.

[0184] Furthermore, the reliability of the entire image data can also be referenced when estimating the cause. The reliability of the entire image data is calculated based on all received local areas related to the same page. The same page is included in the information notified from the pre-processing.

[0185] The reliability of the entire image data can be calculated, for example, by setting the total number of texts extracted from all local areas received about the same page as the denominator and the number of texts with low reliability among the reliability calculated for each character as the numerator.

[0186] For example, if the reliability of the entire image data is high but the reliability of only a specific local area is low, a cause specific to that local area can be considered. On the other hand, if the reliability of not only the specific local area but also the image data as a whole is low, a common cause independent of the type of object can be inferred. For example, stains or creases can be inferred as possible causes.

[0187] Furthermore, if there is a history of preprocessing content used for similar or identical local areas and information related to their credibility, the cause can also be inferred based on the preprocessing content when a high degree of credibility is obtained. However, the preprocessing content must be communicated from the preprocessing side to the OCR processing side, and this information must be stored in association with the image data corresponding to the local area. Here, local areas being similar or identical in type means that the content of the image data corresponding to the local area is similar or identical in type.

[0188] Even without standardization, if a history of combinations of credibility corresponding to the inferred cause is stored, the cause with a high credibility can be used. Furthermore, if a history of combinations of credibility corresponding to preprocessing content is stored, the cause serving as the premise can be inferred based on the preprocessing content with a high credibility.

[0189] Furthermore, if there is a history of causes estimated for local areas of similar or identical types, the cause can be estimated by referring to the history of local areas of similar or identical types to the image data being processed this time.

[0190] Figure 14 This is a diagram explaining an example of a cause of estimation when the target data is a table area. Figure 14 In the table, (A) represents the combination of the calculated reliability for a table area, and (B) represents the estimated cause. Figure 14 The example is an example of calculating the reliability for each row. When the reliability is calculated in units of the entire table area, it is difficult to infer Figure 14 The detailed reasons are shown in Table (A). Figure 7 same.

[0191] Combination 1 is a case where each reliability corresponding to rows 1 to 3 is higher than the threshold value. In this case, since there is no problem with the content of the preprocessing performed last time, there is no need to estimate the cause of the low reliability.

[0192] Combination 2 is a case where the reliability corresponding to the first and second rows is higher than the threshold, but the reliability corresponding to the third row is lower than the threshold. In this case, the cause of the lowered reliability can be estimated to be that the value cell has a colored background.

[0193] Combination 3 is a case where the reliability corresponding to the first and third rows is higher than the threshold, while the reliability corresponding to the second row is lower than the threshold. In this case, the cause of the lowered reliability may be that the value cell is estimated to be a colored background.

[0194] Combination 4 is a case where the reliability corresponding to row 1 is higher than the threshold, while the reliability corresponding to rows 2 and 3 is lower than the threshold. In this case, it can be inferred that the reason for the lowered reliability is that only the value cells are shaded. Here, shaded areas also include color blocks.

[0195] Combination 5 is a case where the reliability corresponding to the second and third rows is higher than the threshold, while the reliability corresponding to row 1 is lower than the threshold. In this case, the cause of the lowered reliability can be inferred to be that the item name unit is a blank character.

[0196] Combination 6 is a case where the reliability corresponding to row 2 is higher than the threshold, but the reliability corresponding to rows 1 and 3 is lower than the threshold. In this case, the cause of the lowered reliability is presumably that the item name cell is a blank character and the value cell has a colored background.

[0197] Combination 7 is a case where the reliability corresponding to row 3 is higher than the threshold, but the reliability corresponding to rows 1 and 2 is lower than the threshold. In this case, the cause of the lowered reliability may be that the item name cell is a blank character and the value cell has a colored background.

[0198] Combination 8 is a case where the reliability corresponding to lines 1 to 3 is lower than the threshold. In this case, the reason for the lowered reliability can be estimated to be that the entire screen is a color block or the entire screen has a color background, and each character is a color character.

[0199] Furthermore, the above estimation focuses on the features of the original document image. Therefore, in order to estimate the reliability reduction caused by stains, folds, etc., other information is required. For example, the reliability of each page or information about the original image data is required.

[0200] <Implementation Method 5>

[0201] Figure 15 This is a diagram for explaining an overview of processing executed in the fifth embodiment. Figure 15 The annotations in the middle correspond to Figure 3 The corresponding parts are shown with symbols.

[0202] In the case of this embodiment, the content of the pre-processing requested for a specific local area is fed back from the OCR processing to the pre-processing.

[0203] Figure 16 This is a flowchart for explaining an example of processing executed by the OCR processing server 20 in the fifth embodiment. Figure 16 The annotations in the middle correspond to Figure 4 The corresponding parts are shown with symbols.

[0204] exist Figure 16 In the case of , the content of the processing executed when a negative result is obtained in step S3 is different from that in embodiment 1.

[0205] If the processor 21A in this embodiment obtains a negative result in step S3, it estimates the cause of the decrease in reliability for the corresponding area (step S6). The estimation details have been described in the fourth embodiment, so the description thereof will be omitted.

[0206] Next, the processor 21A feeds back the content of the preprocessing for eliminating the estimated cause to the preprocessing of the image processing apparatus 10 (step S7).

[0207] The requested processing details include, for example, the type of cleaning process, the intensity of the cleaning process, and parameter values ​​used in the cleaning process. Examples of the cleaning process include removing color blocks or shadows, removing stains, and removing background colors and converting blank or colored characters to black.

[0208] Here, in determining the content of pre-processing of the feedback, for example, a correspondence table prepared in advance, a completed learning model updated by machine learning, or a determination program can be used.

[0209] The correspondence table stores the estimated cause and the pre-processing content to eliminate the cause in a corresponding manner. However, by integrating the processing of steps S6 and S7, the recommended pre-processing content can be stored in a corresponding manner for the classification of images similar to the image data corresponding to the local area.

[0210] Furthermore, when using a completed learning model, if the estimated cause is input into the completed learning model, the pre-processing details for eliminating the cause are output. In this case, the processing of steps S6 and S7 can also be integrated, and if the image data corresponding to the local area is input into the completed learning model, the recommended pre-processing details can be output.

[0211] Furthermore, when using a determination program, pre-processing details that eliminate the cause of the perceived decrease in reliability can be output by repeating the differences caused by a single determination once or multiple times. In this case, the recommended pre-processing details can also be output by integrating the processing of steps S6 and S7 and repeating the determination corresponding to the features extracted from the image data corresponding to the local area.

[0212] In the case of this embodiment, the cause estimated last time and the preprocessing content of the preprocessing request are mechanically learned as teacher data. The learning unit is not limited to a local area, but can be a unit of object type, a unit of original type, or a unit of similar images. In addition, even if the object type is the same, such as Figure 5 If the background or character combination is different between Table A and Table B, the content of the preprocessing that helps improve the reliability may be different. Therefore, in this embodiment, machine learning is performed on a local area basis.

[0213] In this embodiment, reinforcement learning is used for machine learning. In reinforcement learning, learning is performed in a reward-based manner. Therefore, for preprocessed content that only obtains a confidence level below a threshold, no reward or only a low reward is given. On the other hand, if a positive result is obtained in step S3, a high reward is given for preprocessed content that obtains a confidence level above the threshold.

[0214] The results of machine learning are used to determine the content of subsequent preprocessing, which includes not only re-execution of preprocessing with feedback but also preprocessing of newly read document image data.

[0215] If a presumed cause is assigned to a completed learning model that has undergone reinforcement learning, the preprocessing details requested to eliminate the cause are output. Improving the accuracy of reinforcement learning can also reduce the number of feedback cycles. This also increases the likelihood that the confidence level of preprocessing details will exceed a threshold compared to randomly selecting preprocessing details.

[0216] After executing step S7, processor 21A returns to step S1 and waits to receive image data processed using the requested pre-processing content. As a result, text data with a credibility exceeding a threshold is assigned during post-processing by image processing device 10, improving the accuracy and reliability of the post-processing results. Furthermore, the effort and time required to manually verify or correct the recognized text data can be reduced.

[0217] <Implementation Method 6>

[0218] Here, other processing executed by the OCR processing server 20 will be described.

[0219] In the aforementioned embodiment, after the reliability of all local areas reaches or exceeds the threshold, the OCR processing results are collectively output to the post-processing of the image processing device 10. In other words, in the aforementioned embodiment, only the OCR processing results at the stage where a high reliability is finally obtained for each local area are collected and output to the post-processing of the image processing device 10.

[0220] In the case of the present embodiment, for each local area where the reliability is higher than the threshold, the OCR processing result, ie, text data, is output individually to the post-processing of the image processing device 10 .

[0221] Figure 17 This is a flowchart for explaining an example of processing executed by the OCR processing server 20 in the sixth embodiment. Figure 17 The annotations in the middle correspond to Figure 4 The corresponding parts are shown with symbols.

[0222] exist Figure 17 In the case of , after executing step S2, the processor 21A determines whether the reliability is greater than the threshold for each local area (step S3A). Incidentally, in the case of the above-mentioned other embodiments, whether the reliability is greater than the threshold is also determined for all local areas.

[0223] For the local area that received a negative result in step S3A, processor 21A executes the process of step S5. Specifically, processor 21A feeds back information indicating that the reliability of the OCR processing result for the specific area that received a negative result is low to the pre-processing of image processing apparatus 10.

[0224] In addition, as in the aforementioned embodiment 2, the credibility itself can be fed back, or as in the aforementioned embodiment 3, information requesting a change in the pre-processing content can be fed back, or as in the aforementioned embodiment 4, the reason for the reduction in credibility can be fed back, or as in the aforementioned embodiment 5, the pre-processing content requested of the image processing device 10 can be fed back.

[0225] On the other hand, for the local area that obtained a positive result in step S3A, processor 21A performs post-processing (step S4) to output the OCR processing result to image processing device 10. However, here, output is performed for each local area every time the reliability is confirmed to be greater than the threshold.

[0226] After executing step S4, the processor 21A determines whether the reliability in all local areas is equal to or greater than a threshold value (step S8).

[0227] If a negative result is obtained in step S8, the processor 21A returns to step S1. In this case, a local area with low reliability remains in the local area to be processed by OCR.

[0228] If a positive result is obtained in step S8 , the processor 21A ends the OCR process on the document being the processing target.

[0229] In the case of this embodiment, the text data obtained from the local areas with a credibility higher than a threshold through OCR processing is output to post-processing in sequence, but all text data can also be temporarily output to post-processing, and then only the text data obtained from the local areas with high credibility is output to post-processing as replacement or overwriting text data.

[0230] <Implementation Method 7>

[0231] In the case of the aforementioned embodiment, the image processing apparatus 10 separates the image data into objects, but the process of separating the image data into objects may also be performed on the OCR processing server 20 side.

[0232] Figure 18 This is a diagram for explaining an overview of processing performed in the seventh embodiment. Figure 18 The annotations in the middle correspond to Figure 3 The corresponding parts are shown with symbols.

[0233] exist Figure 18In this case, the file name used to distinguish the processing object and the pre-processed image data are sent from the pre-processing to the OCR processing. Here, the image data corresponds to the entire image data read from the original. This is because the processing of separating the objects is not performed.

[0234] On the other hand, in the feedback from the OCR process to the pre-processing, information indicating that the reliability of the OCR result is low is fed back in units of local regions, and the feedback also includes information identifying the local region.

[0235] In this embodiment, the image processing device 10 is unaware of the extent of each local area. Therefore, the information identifying a rectangular local area may include, for example, two coordinate points at diagonal positions defining the outer edge of the area. Furthermore, for example, the information identifying a rectangular local area may include the coordinate point of one corner of the area and the directions and lengths of two sides originating from the same coordinate point. The information identifying a local area is not limited to the two examples described above, as long as it can identify the feedback target, i.e., the local area.

[0236] In addition, in any of the second to sixth embodiments, the process of separating into objects can be performed by OCR processing.

[0237] <Implementation Method 8>

[0238] In the above-described embodiment, when the text data extracted from the character area or the table area having a reliability equal to or greater than a threshold value is output to post-processing, the OCR processing of the document as the processing target is completed.

[0239] In these cases, by detecting in the pre-process that text data is output from the OCR process to the post-process, or detecting in the pre-process that text data is received by the post-process, the end of the OCR process can be detected in the pre-process.

[0240] However, there are cases where it is desirable to explicitly notify the pre-processor of the completion of OCR processing. For example, this may involve deleting image data at a stage where pre-processing is no longer possible. Furthermore, if the image processing device 10 performing pre-processing is different from the image processing device 10 performing post-processing, it may be impossible to detect when text data has been output from OCR processing to post-processing, or when text data has been received through post-processing.

[0241] Therefore, in this embodiment, a function of notifying the pre-processor of the completion of the OCR process is provided during the OCR process.

[0242] Figure 19 This is a diagram for explaining an overview of the processing performed in the eighth embodiment. Figure 19 The annotations in the middle correspond to Figure 3 The corresponding parts are shown with symbols.

[0243] exist Figure 19 In the case where the local area with low credibility is included, information indicating the low credibility of the result of OCR processing is fed back from the OCR processing to the pre-processing. On the other hand, if the local area with low credibility disappears, the end of the OCR processing is notified.

[0244] Figure 20 This is a flowchart for explaining an example of processing executed by the OCR processing server 20 in the eighth embodiment. Figure 20 The annotations in the middle correspond to Figure 4 The corresponding parts are shown with symbols.

[0245] exist Figure 20 In this case, after executing step S4, processor 21A notifies the pre-processor of the completion of OCR processing to image processing apparatus 10 (step S9). Receiving the completion of OCR processing means that the pre-processor no longer needs to use the document image data for pre-processing. Therefore, the pre-processor can safely delete the target image data.

[0246] Furthermore, the function of notifying the completion of the OCR process can be combined with any of the aforementioned embodiments.

[0247] <Implementation Method 9>

[0248] Figure 21 This is a diagram for explaining an overview of the processing performed in the ninth embodiment. Figure 21 The annotations in the middle correspond to Figure 3 The corresponding parts are shown with symbols.

[0249] In the above-mentioned embodiment, the information is fed back only to the pre-processing. However, in the present embodiment, information is also fed back during the process of acquiring the image data of the document.

[0250] For example, if the resolution used to acquire the image data is too small for the size of the characters printed or written on the document, the reliability of the OCR processing results may be reduced. If the resolution discrepancy is the cause of the reduced reliability, even changing the pre-processing content will not improve the reliability.

[0251] Therefore, in this embodiment, when the font size included in the image data to be processed by OCR is considered to be a cause of low reliability, a process of feeding back the change in the resolution of the image data to the image data of the acquired document is executed. Figure 21 In the example of , the instruction changes from 200 dpi to 600 dpi. In addition, the technology of detecting the size of the font size is known.

[0252] The feedback described in this embodiment can also be combined with any of the above-mentioned embodiments.

[0253] <Other implementation methods>

[0254] While the embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the scope described in the aforementioned embodiments. It is clear from the description of the technical scope of the present invention that various modifications or improvements to the aforementioned embodiments are also included in the technical scope of the present invention.

[0255] (1) For example, in the aforementioned embodiment, the case where the combination of preprocessing and its subsequent-stage processing is a combination of cleaning processing and OCR processing is described. However, in other cases, the relationship between preprocessing and subsequent-stage processing can also be considered. For example, it can also be as follows: preprocessing extracts feature quantities for face recognition, and subsequent-stage processing is face recognition using the extracted feature quantities. In this case, the credibility is information indicating the accuracy of the result of face recognition, etc. In this way, in the aforementioned embodiment, it is assumed that the subsequent-stage processing is OCR processing, but the combination of preprocessing and its subsequent-stage processing can be arbitrary.

[0256] (2) In the case of the above embodiment, Figure 5 The illustrated manuscript is used as a premise, and the reliability is calculated in units of rows constituting the table. However, the present invention is also applicable when the reliability is calculated in units of columns.

[0257] (3) In the aforementioned embodiment, an image processing device 10 having a function of optically reading an original document and generating image data is illustrated as an example of a device for performing additional pre-processing on image data provided to the OCR processing server 20. However, an image scanner dedicated to reading image data corresponding to an original document may also be used as the image processing device 10. The image scanner may be provided with an ADF (Automatic Document Feeder).

[0258] Furthermore, as a device for performing additional pre-processing on the image data provided to the OCR processing server 20, in addition to a smartphone or digital camera for photographing the document, a computer that externally obtains the photographed document image data can also be used. Here, the computer is used for pre-processing and post-processing of the OCR-processed data, and does not necessarily have the function of photographing the document image or optically reading the document information.

[0259] (4) In the above embodiment, the image processing device 10 and the OCR processing server 20 are described as being independent devices. However, the OCR processing may be performed in the image processing device 10. In this case, all processes, including pre-processing, OCR processing, and post-processing, are performed within the image processing device 10.

[0260] (5) In the aforementioned embodiment, the case where the post-processing of the text data obtained by OCR processing is transferred to the image processing device 10 that has performed pre-processing is described, but the text data obtained by OCR processing can also be output to a processing device different from the image processing device 10 that has performed pre-processing.

[0261] (6) The processors in the aforementioned embodiments refer to processors in a broad sense, and include not only general-purpose processors (such as CPU (=Central Processing Unit)), but also dedicated processors (such as GPU, ASIC (=Application Specific Integrated Circuit), FPGA, programmable logic devices, etc.).

[0262] Furthermore, the operations of the processors in the aforementioned embodiments may be performed by a single processor alone, or may be performed collaboratively by multiple processors located in physically separate locations. Furthermore, the order in which the operations of the processors are performed is not limited to the order described in the aforementioned embodiments and may be independently modified.

[0263] The above-described embodiments of the present invention are provided for the purpose of illustration and explanation. In addition, the embodiments of the present invention do not fully and exhaustively include the present invention, and do not limit the present invention to the disclosed embodiments. It is obvious that various modifications and variations are self-evident to those skilled in the art to which the present invention belongs. The present embodiment is selected and described in order to most easily explain the principles of the present invention and its application. Thus, other technical personnel in this field can understand the present invention through various modifications optimized for specific uses of the assumed various embodiments. The scope of the present invention is defined by the above claims and their equivalents.

Claims

1. An information processing device comprising a processor, The processor performs the following processing: As a result of preprocessing performed before optical character recognition processing, image data of local area units cut out from an image to be processed and information identifying each local area are acquired; calculating information related to accuracy of an optical character recognition processing result for each image corresponding to the acquired image data; and notifying the pre-processing of the local area determined by the information together with the information identifying the corresponding local area of ​​the information about the calculated accuracy, There are multiple different combinations of credibility of multiple row positions in one of the local areas. The processor estimates a reason why the accuracy of the optical character recognition processing result is lower than a predetermined threshold value for each of the plurality of combinations, The estimated cause is notified to the pre-processing as information related to the accuracy.

2. The information processing device according to claim 1, wherein The processor calculates information related to the accuracy of the optical character recognition processing result for each of a plurality of small areas further dividing the local area, The accuracy of the optical character recognition processing result calculated for each of the plurality of small areas is reported to the pre-processing together or individually.

3. The information processing device according to claim 1 or 2, wherein: The information related to the accuracy indicates that the accuracy of the optical character recognition processing result is lower than a predetermined threshold value.

4. The information processing device according to claim 1, wherein: The information related to the cause is information related to the characters or background included in the partial area. The information processing device according to claim 1 , wherein: The processor estimates the cause including the accuracy of an optical character recognition processing result targeting the image data corresponding to all partial regions cut out from the image as a processing target. The information processing apparatus according to claim 1 , wherein: The processor estimates a cause based on information on the processing content provided from the pre-processing.

7. The information processing apparatus according to claim 1, wherein: The processor estimates a cause based on a history of causes estimated for other image data similar to the image data.

8. The information processing apparatus according to claim 1, wherein: The processor infers a cause based on a difference in accuracy between local areas of the same type.

9. The information processing apparatus according to claim 1, wherein: The processor notifies the preprocessor of information indicating content of the preprocess change.

10. The information processing apparatus according to claim 1, wherein: The processor notifies the preprocessor of information indicating content of processing for the preprocessing request. The information processing apparatus according to claim 10 , wherein: The information indicating the content of the preprocessing includes information indicating parameter values ​​used in the preprocessing.

12. The information processing apparatus according to claim 10, wherein: The processor estimates processing content in response to the pre-processing request based on instruction history for other image data similar to the image data in local area units.

13. The information processing apparatus according to claim 12, wherein: The processor learns the accuracy of the result of processing the content notified to the pre-processing and the local area acquired again after the notification as teacher data, The content of processing for the pre-processing request is estimated.

14. The information processing apparatus according to claim 1, wherein: When the processor reacquires the image data that has been re-processed with different preprocessing, and when the processing results of each local area corresponding to the image data are higher than a predetermined threshold, the processor outputs the result of optical character recognition processing on the re-acquired image data.

15. The information processing apparatus according to claim 1, wherein: When the processor again obtains image data on which different content pre-processing has been performed on a local area that has obtained an accuracy lower than a predetermined threshold, and its optical character recognition processing result is higher than the threshold, the processor combines and outputs the optical character recognition processing result of the local area with the optical character recognition processing results of other local areas that obtained an accuracy higher than the threshold in the previous processing.

16. The information processing apparatus according to claim 1, wherein: The processor increases the resolution of the pre-processing instruction for acquiring the image data to be processed.

17. The information processing apparatus according to claim 1, wherein: Regarding all the local areas, when the accuracy of the optical character recognition processing result exceeds a predetermined threshold value, the processor notifies the pre-processor of the end of the processing.

18. A recording medium having recorded thereon a program for causing a computer to implement the following functions: A function of acquiring, as a result of preprocessing performed before optical character recognition processing, image data of local area units cut out from an image to be processed and information identifying each local area; a function of calculating information related to the accuracy of an optical character recognition processing result for each image corresponding to the acquired image data; and notifying the function of pre-processing the local area determined by said information, together with said information identifying the corresponding local area, of information relating to said calculated accuracy, in, There are multiple different combinations of credibility of multiple row positions in one of the local areas, The program also enables the computer to implement the following functions: For each of the plurality of combinations, inferring a reason why the accuracy of the optical character recognition processing result is lower than a predetermined threshold; and The estimated cause is notified to the pre-processing as information related to the accuracy.

19. An information processing method comprising the following steps: As a result of preprocessing performed before optical character recognition processing, image data of local area units cut out from an image to be processed and information identifying each local area are acquired; calculating information related to accuracy of an optical character recognition processing result for each image corresponding to the acquired image data; and notifying the pre-processing of the local area determined by the information together with the information identifying the corresponding local area of ​​the information about the calculated accuracy, There are multiple different combinations of credibility of multiple row positions in one of the local areas. The information processing method further comprises the following steps: For each of the plurality of combinations, inferring a reason why the accuracy of the optical character recognition processing result is lower than a predetermined threshold; and The estimated cause is notified to the pre-processing as information related to the accuracy.

20. A computer program for causing a computer to implement the following functions: A function of acquiring, as a result of preprocessing performed before optical character recognition processing, image data of local area units cut out from an image to be processed and information identifying each local area; a function of calculating information related to the accuracy of an optical character recognition processing result for each image corresponding to the acquired image data; and notifying the function of pre-processing the local area determined by said information, together with said information identifying the corresponding local area, of information relating to said calculated accuracy, in, There are multiple different combinations of credibility of multiple row positions in one of the local areas, The computer program further enables the computer to implement the following functions: For each of the plurality of combinations, inferring a reason why the accuracy of the optical character recognition processing result is lower than a predetermined threshold; and The estimated cause is notified to the pre-processing as information related to the accuracy.

Images