Information processing apparatus, correction method, and program

Abstract

To suppress erroneous correction of a character string extracted from document data.SOLUTION: In the present invention, when the degree of coincidence between the second document data and the same document information is equal to or more than the threshold and when the character string extraction unit extracts the third character string that is the same character string regardless of the input document, when the difference between the position of the first character string and the position of the third character string is less than a threshold value, when the first character string and the third character string are determined to be the same based on a predetermined criterion, when the difference between the position of the second character string and the position of the third character string is less than a threshold value, when the second character string and the third character string are determined to be the same based on a predetermined criterion, in the second document data, when at least one of a case where a character string is present at the position of the second character string and a case where the second character string is present in the second document data is satisfied, the third character string is corrected on the basis of the document correction history.SELECTED DRAWING: Figure 21

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a correction method, and a program. [Background technology]

[0002] The following techniques are known for extracting character strings written in document data such as image data. 1. Technology for extracting text using templates corresponding to standard documents (standard OCR) 2. Technology that extracts character strings by finding item names and item values ​​according to definitions (quasi-standard OCR) 3. Technology to understand the meaning of documents and extract character strings (non-standard OCR) Documents used for standard OCR are those with a fixed format, such as application forms used within a company or cards used for personal authentication (e.g., driver's licenses).

[0003] The documents used for semi-standard OCR are mainly business-to-business forms such as invoices, receipts, purchase orders, delivery notes, and estimates. Documents vary widely because each company has its own document format. Furthermore, even the same document from the same company may have slightly different content, but the format may be similar.

[0004] Documents used for non-standard OCR are documents of any format, such as contracts, and since the document format differs for each contract, the documents for non-standard OCR are very diverse.

[0005] In recent years, many engines have been proposed as a way to recognize these documents, capable of extracting character strings with high accuracy by training AI (Artificial Intelligence) on large volumes of documents. However, even with these engines, it is difficult to correctly extract all character strings as intended by the user. In particular, the accuracy of recognition and character string extraction by information processing devices decreases when documents contain content specific to a company (e.g., the previous month's billing amount) or when documents contain content that makes it difficult for a human to identify the document type (e.g., January 10th, without a year).

[0006] A technology for recognizing the format of a document is already known (see, for example, Patent Document 1). Patent Document 1 discloses a technology for creating a template by learning items and their position information, and extracting character strings according to the template when a document identical to the learned document is input. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2022-127766 Summary of the Invention [Problem to be solved by the invention]

[0008] However, the technology described in Patent Document 1 has a problem in that character string correction based on correction history may result in erroneous correction. For example, according to learning data corresponding to certain document data, character string A included in document data of an input document should be corrected to character string B, but if correction is made without determining appropriate correction conditions, erroneous correction may occur.

[0009] In view of the above-mentioned problems, an object of the present invention is to suppress erroneous correction of character strings extracted from document data. [Means for solving the problem]

[0010] In view of the above problems, the present invention provides: An information processing device that extracts a character string from document data, an operation receiving unit that receives a correction of a first character string extracted as data of the first document data to a second character string and a correction of a position of the first character string to a position of the second character string; a document learning unit that stores the first character string before correction, the position of the first character string before correction, the second character string after correction accepted by the operation accepting unit, and the position of the second character string after correction as a document correction history, and that stores the first character string and the position of the first character string extracted as data of the first document data as identical document information; an acquisition unit for acquiring second document data of the input document; a character string extraction unit that extracts a third character string and a position of the third character string from the second document data acquired by the acquisition unit; an identical document determination unit that calculates a degree of match based on a result of comparing at least one of the third character string and the first character string in the identical document information, and the position of the third character string and the position of the first character string in the identical document information; If the degree of match is equal to or greater than a threshold value, and if the character string extraction unit extracts the third character string that is the same character string regardless of the input document, If the difference between the position of the first character string and the position of the third character string is less than a threshold value, If the first character string and the third character string are determined to be the same based on a predetermined criterion, If the difference between the position of the second character string and the position of the third character string is less than a threshold value, If the second character string and the third character string are determined to be the same based on a predetermined criterion, If a character string exists at the position of the second character string in the second document data, or If the second character string exists in the second document data, If at least one of the following holds, and a document correction section that corrects the third character string extracted by the character string extraction section based on the document correction history.

[0011] The present invention also provides An information processing device that extracts a character string from document data, an operation receiving unit that receives a correction of a first character string extracted as data of the first document data to a second character string and a correction of a position of the first character string to a position of the second character string; a document learning unit that stores the first character string before correction, the position of the first character string before correction, the second character string after correction accepted by the operation accepting unit, and the position of the second character string after correction as a document correction history, and that stores the first character string and the position of the first character string extracted as data of the first document data as identical document information; an acquisition unit for acquiring second document data of the input document; a character string extraction unit that extracts a third character string and a position of the third character string from the second document data acquired by the acquisition unit; an identical document determination unit that calculates a degree of match based on a result of comparing at least one of the third character string and the first character string in the identical document information, and the position of the third character string and the position of the first character string in the identical document information; When the degree of match is equal to or greater than a threshold value and the character string extraction unit extracts the third character string that is a character string that differs depending on the input document, If the difference between the position of the first character string and the position of the third character string is less than a threshold value, If the attributes of the first character string and the attributes of the third character string are the same, If the difference between the position of the second character string and the position of the third character string is less than a threshold value, If the attributes of the second character string and the attributes of the third character string are the same, If a character string exists at the position of the second character string in the second document data, or If the second character string exists in the second document data, If at least one of the following holds, and a document correction section that corrects the third character string extracted by the character string extraction section based on the document correction history. [Effects of the Invention]

[0012] It is possible to prevent erroneous correction of character strings extracted from document data. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 10 is a diagram illustrating a process of automatically learning only misrecognized documents, rather than learning all documents. [Figure 2] This figure explains the process of determining whether an input document and learning data have the same document format, using not only ruled lines and item names but also item values, table positions, table items, document titles, and document paragraphs as criteria for judgment. [Figure 3] This figure explains the process of deleting learning data when the engine's functionality has been enhanced to enable correct recognition by storing the engine name and version in the learning data. [Figure 4] FIG. 1 is a diagram illustrating an example of the overall configuration of an OCR processing system. [Figure 5] FIG. 1 illustrates a hardware configuration of an example of a character string extraction device. [Figure 6] FIG. 2 is an example of a functional block diagram illustrating functions of a scanner device and a character string extraction device, divided into blocks. [Figure 7] FIG. 1 is a flowchart illustrating an example of an overall flow of a character string extraction device extracting a character string from an input document. [Figure 8] FIG. 10 is a diagram illustrating an example of a form document. [Figure 9] FIG. 10 is a diagram illustrating an example of a semi-formal document. [Figure 10] FIG. 10 is a diagram showing an example of an unstructured document. [Figure 11] FIG. 10 is a diagram showing an example of character information recognized from a standard document. [Figure 12] FIG. 10 is a diagram showing an example of character information recognized from a semi-formal document. [Figure 13] FIG. 10 is a diagram showing an example of character information recognized from an unstructured document. [Figure 14] FIG. 10 is a diagram showing an example of a character string extracted from an invoice. [Figure 15] 10A and 10B are diagrams illustrating confirmation and correction of the character string extraction result by the user. [Figure 16] 10 is an example of a flowchart illustrating a determination by a document learning unit as to whether or not to generate learning data. FIG. [Figure 17] FIG. 10 is a diagram illustrating an example of identical document information. [Figure 18] 10 is a flowchart illustrating an example of a process performed by a document learning unit to generate identical document information. FIG. [Figure 19] FIG. 10 is a diagram illustrating an example of a document revision history. [Figure 20] FIG. 4 is a diagram illustrating an example of engine identification information. [Figure 21] FIG. 10 is an example of a flowchart illustrating document correction performed by a document correction unit. [Figure 22] FIG. 10 is a diagram showing an example of a determination result in which whether or not each character string matches is determined. [Figure 23] 10 is an example of a flowchart illustrating a method for calculating a degree of coincidence based on conditions 1 to 4. FIG. [Figure 24] FIG. 10 is a diagram illustrating a relative position with the inside of a schedule as an anchor. [Figure 25] FIG. 10 is a diagram illustrating the correction of a character string based on a correction condition (Case 1). [Figure 26] FIG. 10 is a diagram illustrating the correction of a character string based on a correction condition (Case 2). [Figure 27] FIG. 10 is a diagram illustrating the correction of a character string based on a correction condition (Case 3). [Figure 28] FIG. 10 is a diagram illustrating the correction of a character string based on a correction condition (Case 4). [Figure 29] FIG. 10 is a diagram illustrating the correction of a character string based on a correction condition (Case 5). [Figure 30] FIG. 10 is a diagram illustrating the correction of a character string based on a correction condition (Case 6). [Figure 31] FIG. 10 is a diagram illustrating the correction of a character string based on a correction condition (Case 7). [Figure 32] FIG. 10 is a diagram illustrating the correction of a character string based on a correction condition (Case 8). [Figure 33] FIG. 10 is a flowchart illustrating an example of automatic updating of learning data when an engine is changed. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, a character string extraction device and a correction method performed by the character string extraction device will be described as an example of an embodiment of the present invention.

[0015] <Outline of character string extraction in this embodiment> First, an outline of character string extraction according to this embodiment will be described with reference to FIGS.

[0016] The character string extraction device of this embodiment does not learn all document data in advance as in conventional fixed-form OCR, but automatically learns only misrecognized document data. This reduces the size of the learning data and the time it takes to determine whether an input document has the same format (same document) as the learning data. The format refers to the position where each piece of data, such as an item, is placed and the attributes of the data (how dates are written, etc.), and is also called a format or style.

[0017] When determining whether documents have the same document format (whether they are the same document), the string extraction device uses not only ruled lines and item names, but also the position of item values, item names in schedules, schedule layout, text, and document layout as criteria. This makes it possible to accurately determine whether input documents have the same format, even if they are diverse. A schedule is information presented in a tabular format. A tabular format is a format in which one or more columns are created horizontally and one or more rows are created vertically, and values ​​are entered in cells (fields, boxes) where the rows and columns correspond. A schedule is sometimes simply called a "table."

[0018] The character string extraction device does not judge whether the document formats are the same based on all conditions at once, but narrows down the learning data by document type and company name and then analyzes the detailed document structure. This makes it possible to quickly determine whether the document formats are the same even if there is learning data for a wide variety of documents.

[0019] By storing the engine name and version of each engine in the learning data, the character string extraction device deletes the learning data when the engine is modified due to functional enhancement (version upgrade, enhancement), etc., and character strings can be extracted correctly. This allows the learning data to be automatically updated to the correct state.

[0020] The character string extraction device saves a revision history of corrections made by the user when a character string is not extracted correctly. The character string extraction device further assumes cases in which the character string is extracted incorrectly and determines whether or not a revision condition prepared for each case is met. If the revision condition is met, the character string extraction device makes a revision using a revision method according to the revision condition. This allows the character string extraction device to suppress erroneous revisions when making revisions based on the document revision history. Furthermore, if a character string cannot be extracted correctly due to the same mistake as when the revision history was registered, the revision can be made.

[0021] FIG. 1 is a diagram illustrating a process for automatically learning only misrecognized document data, rather than learning all document data. FIG. 1 shows an example of a screen of a character string extraction device, with scanned image data 201 displayed on the left and an item name 202 and an item value 203, which is the result of extracting the character string, displayed on the right. The user can determine whether or not misrecognition occurred by comparing the image data 201 with the item value 203. If the user corrects the item value 203, it is determined that the document data was not extracted correctly. The character string extraction device learns the item values ​​of this document data and the position information of the item values ​​specified by the user. This process allows the character string extraction device to learn the minimum number of documents necessary, thereby reducing the size of the learning data and the time required for comparing identical document information with an input document.

[0022] FIG. 2 is a diagram explaining a process for determining whether an input document and learning data have the same document format, using not only ruled lines and item names but also the position of item values, item names in a schedule, the layout of the schedule, character information contained in rows, and the document layout as criteria for determination. In FIG. 2, the range enclosed by a rectangle is extracted as a single character string. Rectangle 206 is the document type, rectangle 207 is company information, and rectangle 208 is the schedule. By using these as criteria for determining whether documents are the same, the character string extraction device can accurately determine whether documents are the same, even among a wide variety of documents. Conventionally, whether a document has the same format was determined only based on the item names and ruled lines in a schedule.

[0023] FIG. 3 is a diagram explaining the process of deleting learning data when the engine name and version are stored in the learning data and the engine is modified due to functional enhancements or the like, making it possible to correctly extract character strings. The character string extraction device includes components such as a document recognition application 211, a character recognition engine 214, a document sorting engine 212, and a character string extraction engine 213. These engines are sold by various companies, and the best one is selected and implemented. When the engine is modified due to functional enhancements and character strings can be correctly extracted, the character string extraction device deletes the learning data. Thereafter, learning data is generated for this new engine, so the learning data can be automatically updated to the latest version. Furthermore, since the character string extraction device 2 is automatically enhanced in function, the accuracy of the character string extraction device can be automatically improved.

[0024] <Terminology> Document data includes various character strings. Character strings include sets such as item names and item values, character strings containing only values ​​such as company names and addresses, character strings in schedules, and character strings included in rows. Item values ​​change when the input document changes, but company names, addresses, and item names do not. Therefore, some character strings change from document to document and some do not. In this embodiment, character strings include, for example, document type, company information, item values, and item values ​​in schedules. Character strings include characters, numbers, symbols, and characters expressed in character codes such as the alphabet. In the case of document data in prose format such as a contract, this is also a character string.

[0025] Document data includes image data and data containing text. Data containing text is data in which the text is already represented by character code. In the case of image data, characters are recognized using OCR processing and the text is extracted. In the case of data containing text, OCR is not required because it contains text (character code).

[0026] <System configuration example> Next, an example of the configuration of the OCR processing system 1 will be described. Fig. 4 is a diagram illustrating an example of the overall configuration of the OCR processing system 1. As shown in Fig. 4, the OCR processing system 1 includes a character string extraction device 2, scanner devices 4A and 4B, and a terminal device 3, which can communicate with each other via a network 7. In this embodiment, any one of the scanner devices 4A and 4B will be referred to as a "scanner device 4."

[0027] The network 7 may be, for example, an in-house LAN. The network 7 may be configured using wireless communication such as Wi-Fi. Furthermore, if the character string extraction device 2 exists on the cloud, the network 7 may include a WAN or the Internet. For example, image data read by the scanner device 4 can be acquired by the terminal device 3 and transmitted to the character string extraction device 2. This image data is an example of document data. However, the document data may not be an image but may be data containing character codes formed by character strings (character codes).

[0028] The character string extraction device 2 may be directly connected to the scanner device 4 in a one-to-one wired manner via a USB cable or the like. Even in the case of one-to-one connection, the character string extraction device 2 and the scanner device 4 may communicate wirelessly. Known communication methods for this purpose include Wi-Fi Direct and Bluetooth (registered trademark).

[0029] The character string extraction device 2 may be a general-purpose information processing device. The character string extraction device 2 performs character recognition and data extraction on image data of an input document received from the scanner device 4, and allows the user to confirm or correct the extraction results. The character string extraction device 2 of this embodiment may perform character recognition using OCR technology on image data of an input document received from the scanner device 4.

[0030] Specifically, the character string extraction device 2 may be, for example, a PC (Personal Computer), a server device, a smartphone, or a tablet PC.

[0031] The scanner device 4 is an optical reading device that reads a document and generates image data and transmits it to the character string extraction device 2. In this embodiment, the scanner device 4 scans various documents as examples of input documents. Although FIG. 4 illustrates the scanner device 4, image data for character recognition may also be captured by a digital camera 8 or the like. The image data captured by the digital camera 8 may be transmitted via the network 7 or may be stored in a removable storage medium. When the user attaches the storage medium to the character string extraction device 2, the character string extraction device 2 can acquire the image data.

[0032] The scanner device 4 may also be a device called an MFP (Multifunction Peripheral) or a multifunction device, which may have a printer function, a copy function, a facsimile function, etc. in addition to a scanner function.

[0033] Furthermore, in FIG. 4, the scanner device 4 and the character string extraction device 2 are separate entities, but the scanner device 4 and the character string extraction device 2 may be integrated into one device (such as an MFP).

[0034] The terminal device 3 is a general-purpose information processing device such as a PC, a smartphone, or a tablet PC. When the character string extraction device 2 is a server device, the terminal device 3 can transmit document data received from the scanner device 4 to the character string extraction device 2. The terminal device 3 acquires character strings extracted by the character string extraction device 2. The terminal device 3 may acquire document data from the scanner device 4, or may store document data input by a user.

[0035] <Hardware configuration example> The hardware configuration of the character string extraction device 2 according to this embodiment will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the hardware configuration of the character string extraction device 2 according to this embodiment. As shown in Fig. 5, the character string extraction device 2 is constructed by a computer 500, and includes a CPU 501, a ROM 502, a RAM 503, a HD (Hard Disk) 504, an HDD (Hard Disk Drive) controller 505, a display 506, an external device connection I / F (Interface) 508, a network I / F 509, a bus line 510, a keyboard 511, a pointing device 512, a DVD-RW (Digital Versatile Disk Rewritable) drive 514, and a media I / F 516.

[0036] Of these, the CPU 501 controls the overall operation of the computer 500. The ROM 502 stores programs, such as an IPL, used to drive the CPU 501. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data, such as programs. The HDD controller 505 controls the reading and writing of various data from and to the HD 504 under the control of the CPU 501. The display 506 displays various information, such as a cursor, menus, windows, characters, or images. The external device connection I / F 508 is an interface for connecting various external devices. In this case, external devices include, for example, a USB (Universal Serial Bus) memory or a printer. The network I / F 509 is an interface for data communication using the network 7. The bus line 510 is an address bus, a data bus, or the like, for electrically connecting the components, such as the CPU 501, shown in FIG. 5.

[0037] The keyboard 511 is a type of input means having multiple keys used to input characters, numbers, various instructions, etc. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The optical drive 514 controls reading and writing of various data from an optical storage medium 513, which is an example of a removable storage medium. The optical storage medium 513 may be a CD, DVD, or Blu-Ray (registered trademark). The media I / F 516 controls reading and writing (storing) of data from a storage medium 515 such as a flash memory.

[0038] <About the function> Next, the functions of the character string extraction device 2 will be described in detail with reference to Fig. 6. Fig. 6 is a functional block diagram that explains the functions of the scanner device 4 and the character string extraction device 2 by dividing them into blocks.

[0039] <<Scanner device>> The scanner device 4 has a communication unit 41 and a reading unit 42. The reading unit 42 feeds documents such as forms one by one and generates image data with a predetermined resolution and gradation by scanning the document surface with a line sensor. Instead of the scanner device, a digital camera 8 or a device equipped with a camera function may acquire image data of an input document.

[0040] The communication unit 41 communicates with the character string extraction device 2 using a communication protocol such as SNMP (Simple Network Management Protocol), or communicates with the character string extraction device 2 via a dedicated line such as a USB cable. The communication unit 41 transmits the image data generated by the reading unit 42 to the character string extraction device 2.

[0041] If the document data contains character strings, the scanner device 4 is not necessary, and in this case the character string extraction device acquires the character string-containing data from an external device such as the terminal device 3.

[0042] <<String extraction device>> The character string extraction device 2 has an acquisition unit 11, a character recognition unit 12, a document sorting unit 13, a character string extraction unit 14, a same document determination unit 15, a display control unit 16, an operation reception unit 17, a document learning unit 18, a document correction unit 19, and an output unit 21. Each of these units of the character string extraction device 2 is a function or means realized by a CPU 501 of the character string extraction device 2 executing program instructions. This program may be, for example, a native application dedicated to the scanner device or a general-purpose native application. The program may also be a web application.

[0043] Each of these functions will be explained with reference to the flowchart in Fig. 7. Fig. 7 is a flowchart illustrating the overall flow of the character string extraction device 2 extracting a character string from an input document.

[0044] In step S1, the reading unit 42 controls the scanner device 4 or digital camera 8 to read the input document and create image data. If the image data is already digital data, such as a PDF, the reading unit 42 acquires the image data by visualizing and capturing the digital data. If the input document contains text, the scanner device 4 does not need to scan the original. The image data or text-containing data of the input document is called document data. The communication unit 41 transmits the document data to the character string extraction device 2. The acquisition unit 11 of the character string extraction device 2 acquires the document data. Alternatively, the acquisition unit 11 may receive the document data from the terminal device 3, or may read the document data from a memory card or the like. The acquisition unit 11 can also control the scanner device 4 via the network 7. A TWAIN (Tool Without An Interesting Name) driver is installed in the character string extraction device 2. TWAIN is a standard that defines technical specifications for controlling input devices such as image scanners from a scanner application running on a computer to capture image data.

[0045] In S2, the character recognition unit 12 acquires "character information" from the document data using the character recognition engine 214. The character information is the character code and position of each character. If the input document is image data, the character recognition unit 12 performs optical character recognition (OCR) on the image data.

[0046] In S3, the document sorting unit 13 uses the document sorting engine 212 to analyze the layout and character information of the document data, thereby sorting the document type.

[0047] In S4, the character string extraction unit 14 uses the character string extraction engine 213 prepared according to the document type to extract character strings necessary for the document from the character information as data of the document data. The necessary character strings are the amount, date, etc. for a slip, the billing amount, business partner, etc. for an invoice, and the name of the contractor, date, etc. for a contract.

[0048] In S5, the document correction unit 19 corrects the extraction result. The document correction unit 19 holds the correction content (learning data) of the document data from which the character string was extracted in the past. Details of the learning data will be described later. The document correction unit 19 determines whether the input document is in the same format when generating the learning data. The format is a description format such as document type, company information, item name, layout of the明细表, or document layout. If it is in the same format, it is presumed to be the same document. When it is determined to be in the same format, the document correction unit 19 corrects the character string based on the learning data generated for this document data.

[0049] In S6, the display control unit 16 displays the document data and the character string, and the user checks the extraction result and corrects it if there is a character string that has not been correctly extracted. The operation reception unit 17 receives the correction from the user.

[0050] In S7, the document learning unit 18 generates learning data based on the correction content corrected by the user. The learning data is saved in association with the same document (more specifically, the document format).

[0051] In S8, the output unit 21 outputs the extraction result of the character string obtained by the above processing. The output format may be a file such as csv or xml, or the data may be passed through the I / F of the related application and the application may output it.

[0052] <Details of Each Step> Hereinafter, each step of the flowchart in FIG. 7 will be described in detail.

[0053] <<S1 Reading of Document> As described above, there are mainly three types of documents: standard documents, semi-standard documents, and non-standard documents. FIGS. 8 to 10 show examples of standard documents, semi-standard documents, and non-standard documents, respectively. A standard document is a document whose format is uniquely determined by the document, such as application forms used within a company or cards used for personal authentication (e.g., driver's licenses and my number cards). A semi-standard document is a form used between companies, such as invoices, receipts, order forms, delivery notes, and estimates. Since the format varies from company to company, there are a wide variety of documents. Also, even within the same company, the content may change slightly, and the formats may be similar but slightly different. A non-standard document is a document whose format is not determined, such as a contract. Since the format varies for each contract, there are a wide variety of documents.

[0054] For these various types of documents, in the case of paper documents, the reading unit 42 reads the paper document and converts it into image data (e.g., file formats such as JPEG, PNG, BMP, PDF). In the case of digital documents (with character codes) such as Adobe's PDF and Microsoft's Word, the reading unit 42 performs file format conversion and converts it into image data (e.g., file formats such as JPEG, PNG, BMP, PDF). Therefore, both paper documents and digital documents can be processed in the same way. However, as will be described later, character information may be extracted by processing the digital document as it is. In the case of data with character codes, OCR is not required.

[0055] <<S2 Character Recognition>> When the document data is image data, the character recognition unit 12 performs optical character recognition (OCR) to obtain "character information including layout information". The character recognition unit 12 uses the character recognition engine 214 or is integrated with the character recognition engine 214. The layout is the position information of each character string,明细表, etc. In addition, photos and logos can also be extracted. Therefore, the positions of paragraphs and lines in the prose form become clear. When the document data is data with character codes, character recognition is not required.

[0056] In the case of data with a character code, the character recognition unit 12 extracts "character information including layout information" from the data with a character code.

[0057] Figs. 11 to 13 show examples of a regular document, a semi-regular document, and an irregular document from which character information has been extracted. In any of these documents, the character information is surrounded by a rectangle.

[0058] <<S3 Document Classification>> The document classification unit 13 determines the document type by analyzing "character information including layout information". For example, the document classification unit 13 holds the grid line information (the length and position of the grid line) in association with the document type, and determines whether it matches the grid line information of the input document. The document learning unit 18 classifies the document into the corresponding document type when they match. Also, the title name and the document type are stored in association in advance, and the document classification unit 13 determines whether the input document and the title name are the same to classify the document. Further, the document classification unit 13 may analyze the content described in the input document using AI or the like to classify the document type. Note that the document classification unit 13 uses the document classification engine 212 or is integrated with the document classification engine 212.

[0059] <<String Extraction>> The string extraction unit 14 extracts a string corresponding to the document type from "character information including layout information" as the data of the input document. Specifically, in the case of a document classified as a transfer slip exemplified in a regular document, the string extraction unit 14 extracts the document title, the creation date, the subject code, the amount, etc. Also, in the case of a document classified as an invoice exemplified in a semi-regular document, the string extraction unit 14 extracts the document title, the company information of the self, the company information of the customer, the invoice ID, the transaction details (product name, unit price, quantity, amount), and the invoice amount, etc. Also, in the case of a document classified as a contract exemplified in an irregular document, the string extraction unit 14 extracts information such as the document title, the company information of the self, the company information of the customer, and the contract date. The string extraction unit 14 uses the string extraction engine 213 or is integrated with the string extraction engine 213.

[0060] As a method for extracting a character string, since each company provides a character string extraction engine 213 learned for each document type, it is possible to use the character string extraction engine 213. In general, there are also methods of making a large amount of data learned by AI to extract the information to be recognized, and methods of extracting character strings by using existing rule-based techniques (such as what characters are at which positions, where grid lines are located, where a detailed table is located, etc.). Furthermore, when there are many documents with the same format such as fixed-form documents, it is also possible to extract character strings by previously learning positions to be recognized such as grid line information.

[0061] As a method of extracting based on rules, for example, in a document with a description of the character string "Claim ID 12345678", the developer creates a rule-based system in advance with the item name indicating the character string "Claim ID" and the numerical value "12345678" on the right side of "Claim ID" as the item value. The character string extraction unit 14 searches for "Claim ID" from the character information to extract the character string to be extracted on the right side of "Claim ID". Also, when only "To ○○ Co., Ltd." is described and the "company name of our own company" does not exist, the character string extraction unit 14 searches for the character string "Co., Ltd." and extracts it as the "company name of our own company" when detecting (for example, "To") described around the company name. In this way, the character string extraction unit 14 may perform character string extraction by obtaining the item value based on the item name, the position information of the item name, the item value, or the position information of the item value.

[0062] FIG. 14 shows an example of a character string extracted from a claim. The document type 221, the company name of our own company 222, the claim ID and character string 223, the name of the trading company 224, the detailed information 225, the claim amount 226, etc. are extracted.

[0063] <<S5 Document correction>> For document correction, learning data is used, so it will be described later after explaining the generation of learning data.

[0064] <<S6 Confirmation and correction by the user>> FIG. 15 is a diagram illustrating how a user can confirm and correct the character string extraction results. FIG. 15(a) shows a confirmation and correction screen 230 displayed by the character string extraction device 2. The confirmation and correction screen 230 displays document data 231 of the input document on the left side, and the extraction results 232 extracted by the character string extraction unit 14 on the right side. In the document data 231, the positions of the field values ​​of form type 233, document ID 234, customer 235, and invoice amount 236 extracted from the document data 231 are indicated by rectangles. Furthermore, these rectangles and the extraction results 232 are connected by lines. Although this line is shown for the purpose of explanation, it is also possible to display them by having the character string extraction unit 14 manage the correspondence between the two.

[0065] The item value "100 yen" is extracted for the billing amount 236. However, when the total amount 237 in the document data 231 is checked, it is "110 yen," so the correct value for the billing amount 236 is "110 yen." Therefore, the user corrects the item value for the billing amount 236.

[0066] Figure 15(b) shows the confirmation / correction screen 230 after the user has corrected the item value of the billing amount 236. The billing amount 236 has been corrected to "110 yen." The correction method will be explained below. The user uses a mouse or other device to circle the area 238 to be recognized in the document data 231. A mark 239 indicating that the user has corrected the area 238 is displayed in the area 238. The character recognition unit 12 performs OCR recognition of the character string in the specified area 238. Assume that the recognized character string is "110 yen." The document learning unit 18 corrects the item value of the billing amount 236 to "110 yen" using the recognized "110 yen." Note that the fact that the item value of the billing amount is in the area 238 specified by the user can be seen because the input document has been determined to be an invoice. Alternatively, the user may explicitly associate the item value of the billing amount 236 with the area 238 specified by the user. Furthermore, if the character recognition unit 12 makes a mistake in character recognition even when the user specifies the area 238, the user can input the correct item value in the billing amount 236 using the keyboard.

[0067] In addition, on the confirmation and correction screen 230, the document sorting and the extraction results of the character strings are displayed. However, if the user has preset the results to be obtained, only some of the extraction results specified by the user may be displayed.

[0068] The position of the area 238 corrected by the user is saved in the document correction history. The corrected item value "110 yen" is also saved in the document correction history. Whether to save it in the document correction history is predetermined (see Fig. 19).

[0069] When the user has confirmed and corrected all items, the process in Fig. 7 proceeds to the next step (S7) according to the user operation.

[0070] <<S7 Document Learning>> As shown in Fig. 16, when the user corrects the extraction results, the document learning unit 18 generates learning data. Fig. 16 is a flowchart diagram explaining the determination of whether the document learning unit 18 generates learning data.

[0071] The document learning unit 18 determines whether the user has corrected the extraction results in step S6 (S71). This determination is made based on whether the user has specified an area with the image data 241 or has corrected the extraction results 232. If the determination in step S71 is Yes, the process proceeds to step S72; if No, the process proceeds to step S8.

[0072] In step S72, the document learning unit 18 performs learning on the input document. Learning means generating and saving learning data (S72). The learning data includes "identical document information" for determining whether documents are identical, "document correction history" indicating the history of document corrections, and "engine identification information" for identifying the corrected recognition engine. These will be explained below.

[0073] Fig. 17 shows the identical document information. The identical document information is generated from the results obtained by character recognition, document sorting, and data extraction. The ID is identification information that identifies an item of document identity information. The description content is data in the input document extracted by the character string extraction unit 14 (all character strings extracted from the input document). The details are explanations of the character strings. The explanations of the character strings are prepared in advance. The details are written for the purpose of explaining this embodiment, and may not be included in the same document information. The string and location information in the storage items can be set to "save" or "not save" the string and location information. Location information is the coordinates of the string. For example, the document type, company information, item name, and table item values ​​do not change (they can be used to determine the same document), so both the string and location information are stored. The item values ​​change for each input document, so the string is not stored, but the location information is. The table item values ​​change for each input document, so the string is not stored, and the location information is not stored either, as it can change depending on the number of rows in the detailed table. For table layouts and document layouts, only location information is stored. Note that for document layouts, strings may also be stored.

[0074] A table layout is one or more of the position information that specifies the outer frame of a spreadsheet, the position information of all cells (also called fields or squares) that the spreadsheet has, and the position information of the ruled lines that form the cells. "All cells" refers to cells for which position information can be obtained, and it is not necessary to store the position information of some cells. A document layout is the position information of lines and paragraphs. A paragraph is an indented line or a block of lines with one or more lines of space between them. The number of lines and the number of paragraphs are also saved as part of the document layout. Even if the number of lines and the number of paragraphs are not saved, the positions of the lines and paragraphs are recorded, so they can be counted at any time.

[0075] The setting of "save" or "don't save" in Fig. 17 is predetermined by the data. In other words, the only character strings used to determine whether they are the same document are those extracted from the input document that are set to "save."

[0076] Note that the document type and company information are examples of character strings that do not change, the item names and table item names are examples of character strings that do not change, and the item values ​​and table item values ​​are examples of character strings that do change.

[0077] 18 is a flowchart illustrating the process of generating identical document information by the document learning unit 18. The process of FIG. 18 is executed, for example, in step S72 of FIG.

[0078] The document learning unit 18 acquires the document classification results and the data extraction results (S81).

[0079] The document learning unit 18 acquires the character string and its position set as "store" in the document identity information from the character information, and saves the character string and position information set as "store" in the document identity information as identity document information (S82). Note that the saved character string and position information are those before correction.

[0080] The document learning unit 18 repeats step S82 until all character strings predetermined according to the document type are acquired (S83).

[0081] Next, Figure 19 shows the document revision history. The document revision history stores extracted character strings, position information, and user revision results (character strings and position information). In other words, incorrectly extracted character strings and position information are also saved along with the user revision results. The ID is identification information that identifies an item in the document revision history. The description content is data in the input document extracted by the character string extraction unit 14 (all character strings extracted from the input document). The details are explanations of the character strings. The explanations of the character strings are prepared in advance. The details are provided for the purpose of explaining this embodiment, and may not be included in the document revision history. The string and location information items in the memory section define whether to "memorize" or "not memorize" each string when the user modifies it. In other words, if the string is set to "memorize" and the user modifies the string, it will be saved in the revision history. If the location information is set to "memorize" and the user modifies the recognized location, the string's location will be saved in the revision history. Whether to "memorize" or "not memorize" is predetermined based on the extracted data (description).

[0082] The document correction history is not only used to correct character strings extracted from the input document, but also for the document correction unit 19 to determine whether or not to correct character strings extracted from the input document (correction conditions (1) to (12) described below). The character strings of the document type, company information, item values, and table item values ​​used for this are stored. The position information of the document type, company information, item values, and table item values ​​is also stored. On the other hand, the character strings and position information of the item names, table item names, table layout, and document layout are not the character strings themselves that are the subject of extraction, and are therefore not used for correction or for determining whether or not to correct them. Therefore, the character strings and position information of the item names, table item names, table layout, and document layout are not stored.

[0083] When the character string is "memory", the character string to be memorized is the incorrectly extracted character string before correction and the corrected character string. When the location information is "memory", the location information to be memorized is the incorrectly extracted location information before correction and the corrected location information.

[0084] 20 shows engine specification information, which is information relating to the character recognition engine 214, the document classification engine 212, and the character string extraction engine 213. The ID is identification information that identifies an item of engine-specific information. The description indicates the engine function. -Details are engine names. · The engine name and engine version V indicate that the engine name and version are stored (preserved). If these engines are integrated, they may be managed collectively. In that case, only one engine is stored.

[0085] <<S8 Output>> The output unit 21 outputs the extraction result of the character string obtained by the above processing. The output format may be a file such as csv or xml, or data may be passed through the I / F of the related application, and the application may output it.

[0086] <<S5 Document Correction>> Subsequently, referring to FIG. 21 and the like, the document correction will be described. FIG. 21 is a flowchart diagram for explaining the document correction process performed by the document correction unit 19.

[0087] First, the document correction unit 19 reads learning data (identical document information, document correction history, engine identification information) (S51).

[0088] The identical document determination unit 15 determines whether the document type or company information has been corrected by referring to the document correction history (S52). When the user corrects the document type or company information, the document correction history stores the document type before correction, the document type after correction, the position of the document type before correction, and the document type after correction.

[0089] If the determination in step S52 is Yes, the process proceeds to step S55; if No, the process proceeds to step S53. If the determination in step S52 is Yes, there may be the same mistake as when the input document read by the scanner device 4 was corrected.

[0090] In step S53, the identical document determination unit 15 determines whether the document type or company information in the identical document information is the same as the document type or company information included in the extraction result (S53). If the determination in step S53 is Yes, the process proceeds to step S55; if No, the process proceeds to step S54.

[0091] As shown in step S53, the same document determination unit 15 first determines whether the document type and company information match between the same document information and the input document, and performs step S55 only if they match, thereby narrowing down the learning data and speeding up the determination of whether the documents are the same or not.

[0092] In step S54, the same document determination unit 15 determines whether or not all the learning data has been verified (S54). If the determination in step S54 is Yes, the process in Fig. 21 ends, and if No, the process proceeds to step S51.

[0093] In step S55, the same document determination unit 15 calculates the degree of match between the same document information and the input document to determine whether the read learning data can be used to correct the input document (S55). As will be described in detail later, the document correction unit 19 calculates the degree of match using information stored in the same document information, as shown in FIG. 17. Therefore, the document correction unit 19 calculates the degree of match by comparing the character strings and position information of the same document information with the character strings and position information of the input document for at least one of the following: document type (character string and position information), company information (character string and position information), item name (character string and position information), item value (position information only), table item name (character string and position information), table layout (position information only), and document layout (position information only). As shown in FIG. 17, only the character strings and position information set to "store" in the same document information are compared, so the time required to determine whether they match can be reduced.

[0094] Next, the same document determination unit 15 compares the degree of match with a threshold value to determine whether the documents can be determined to be the same (S56). If the degree of match is equal to or greater than the threshold value, the same document determination unit 15 determines that the documents are the same. If the determination in step S56 is Yes, the process proceeds to step S57; if the determination is No, the process proceeds to step S54.

[0095] In step S57, it can be determined that learning data learned from the input document has been found, so the document correction unit 19 corrects the extracted character string based on the document correction history (S57). Note that when a correction condition prepared in anticipation of cases in which a character string may be extracted incorrectly is met, the document correction unit 19 corrects the extracted character string according to the met correction condition, thereby suppressing erroneous corrections. The determination of whether or not to make corrections using the document correction history and the correction method will be described with reference to FIGS.

[0096] Next, a method for calculating the degree of match between the identical document information and the input document in step S55 will be described. Fig. 22 is a diagram illustrating the result of determining whether or not there is a match for each piece of data extracted from the input document. The document correction unit 19 compares the identical document information with the input document to calculate the degree of match. The document correction unit 19 classifies the conditions as follows and determines whether or not the conditions are met.

[0097] ·Condition 1 The document correction unit 19 determines whether the character strings and position information of "document type, company information, item name, table item name" are the same in the same document information and the input document. The character strings do not have to be completely identical, and differences between half-width and full-width characters are allowed. Furthermore, if a threshold value (e.g., 90%) or more of the character strings in the same document information are included in the character strings in the input document, they may be determined to be the same character strings. Similarly, the position of the character strings does not have to be completely identical, and if the difference between the position of the character string in the same document information and the position of the character string in the input document is less than a threshold value (e.g., half the character size), they may be determined to be in the same position. This determination method is similar to conditions 2 to 4.

[0098] ·Condition 2 In the case of "item values," the document correction unit 19 determines whether the position information is the same between the same document information and the input document. Whether the position information is the same may be determined by whether the area with the fewer recognized characters overlaps. Note that complete overlap is not required, as long as there is a certain percentage of overlap in area. Alternatively, if the difference between the position of the item value in the same document information and the position of the item value in the input document is less than a threshold value (e.g., half the character size), it may be determined that they are in the same position.

[0099] ·Condition 3 The document correction unit 19 determines whether the table layout is the same between the same document information and the input document. This may be determined based on the position of the schedule itself, or whether the positions of the cells or ruled lines match. A perfect match is not required for the positions. For example, a difference of about 5% in the schedule position between the same document information may be considered to be the same. A difference of about 50% in the cell position between the same document information may be considered to be the same. A difference of about 50% in the spacing between ruled lines between the same document information may be considered to be the same, and a difference of 10% to 20% in the length of the ruled lines may be considered to be the same. Alternatively, the document correction unit 19 may determine whether the difference in the number of cells is less than a threshold value.

[0100] ·Condition 4 The document correction unit 19 determines whether at least one of the number of lines and the number of paragraphs matches between the identical document information and the input document. A perfect match in the number of lines and the number of paragraphs is not required. For example, a difference of about 10% in the number of lines or the number of paragraphs in the identical document information may be considered to be identical. The document correction unit 19 also determines whether the character information contained in all lines matches between the identical document information and the input document. A perfect match in the character information is not required. For example, if a threshold value (e.g., 90%) or more of the character strings in the character information of the identical document information are included in the character information of the input document, it may be determined that they are the same document.

[0101] The document correction unit 19 counts whether conditions 1 to 4 are met for each piece of data, and calculates the degree of match by finding the rate of match. In FIG. 22, whether the character string of the document type and the position information match, whether the position of the item value matches, etc. are set. If it is determined that there is a match, the "Matched" item becomes "1", and if it is determined that there is no match, the "Matched" item becomes "0". The document correction unit 19 determines the degree of match as "the sum of the number of 1s in the matched items / the number of all items determined."

[0102] FIG. 23 is a flowchart illustrating a method for calculating the degree of coincidence based on the above conditions 1 to 4.

[0103] The document correction unit 19 determines whether the identical document information and the input document match with respect to specified character strings based on condition 1 (S55-1). The specified character strings are the document type, company information, item names, and table item names (condition 1).

[0104] The document correction unit 19 also determines whether the identical document information and the input document match with respect to the position of the specified character string based on condition 1 (S55-2). The specified character strings are the document type, company information, item names, and table item names (condition 1).

[0105] The document correction unit 19 determines whether the identical document information and the input document match with respect to the character positions of the item values, which are variable values, based on condition 2 (S55-3). That is, for items whose item values ​​change each time, only the character positions are compared (condition 2).

[0106] Next, the document correction unit 19 determines whether the identical document information and the input document match regarding the table layout based on condition 3 (S55-4). That is, the positions of the schedule, the positions of the cells, and the positions of the lines are compared (condition 3).

[0107] Next, the document correction unit 19 determines whether the identical document information and the input document match in terms of document layout based on condition 4 (S55-5). That is, one or more of the number of lines, the number of paragraphs, and the character information contained in the lines are compared (condition 4).

[0108] The document correction section 19 calculates the degree of match by summing up the results of the judgments on whether or not the data matches, which are obtained for each data, and dividing the sum by the number of all the data that have been judged (S55-6).

[0109] <<Increasing the accuracy of identity judgment>> As a method for comparing positional information such as character strings and detailed statements, a specific position within the document data may be used as an anchor (reference point) to allow for misalignment of the print position. For example, by using the average of one or more of the positions where the title, date, total amount, or client company name of an invoice is written as the anchor, identity can be determined with high accuracy as long as the relative position to the reference is the same even if the print position is misaligned.

[0110] Furthermore, to allow for document data with a variable number of rows in a schedule, the same document information and position information within the input document may be determined using a predetermined position in the schedule as an anchor. For example, the upper right corner, lower right corner, upper left corner, lower left corner, center, etc. of the schedule may be used as an anchor.

[0111] FIG. 24 is a diagram explaining the relative position when a predetermined position on the statement is used as anchor 109. If the predetermined position on the statement is not used as anchor 109, the coordinates of item value 209 on the statement are determined, for example, based on the top left corner of the invoice. In contrast, if a predetermined position on the statement (the top left corner of the statement in FIG. 24) is used as anchor 109, the coordinates of item value 209 on the statement are determined, for example, based on the top left corner of the statement. If the predetermined position on the statement is used as anchor 109, even if the printing position of the statement is shifted, the coordinates of item value 209 when viewed from the relative position are unlikely to change, making it easy to extract item value 209.

[0112] The ruled lines of a statement can be detected as straight lines of a certain length or more using a Hough transform or similar method. By using a specific position on the statement as an anchor, it is easy to determine that the statement is the same document even if the number of lines in the statement varies. This makes it possible to determine with high accuracy whether the statements are the same document or not.

[0113] In addition, in the case of a spreadsheet without ruled lines, the spreadsheet can be detected based on overlapping character strings in the row and column directions, and the top, bottom, left and right ends can be determined to be anchors.

[0114] <<Details of the fix>> Next, the correction in step S57 in Fig. 21 will be described. If it is determined that the input document is the same as the identical document information, the document correction unit 19 automatically corrects the incorrectly extracted character string based on the document correction history. However, if a correction condition prepared in anticipation of cases in which the character string may be extracted incorrectly is met, the document correction unit 19 makes a correction according to the met correction condition, thereby suppressing erroneous correction.

[0115] The document correction history includes four pieces of data: the character string before correction, the position of the character string before correction (hereinafter referred to as the position before correction), the character string after correction, and the position of the character string after correction (hereinafter referred to as the position after correction). The objects of comparison include two pieces of data: the character string extracted from the input document by the character string extraction unit 14 (hereinafter referred to as the recognized character string) and the position of the recognized character string.

[0116] The correction conditions are explained for the following eight cases. A. When the item you want to extract the string from is the same string regardless of the input document (e.g. company name) = Cases 1 to 4 B. When the item you want to extract strings from varies depending on the input document (date, amount, etc.) = Cases 5 to 8 In cases 1 to 8, the first image 251 described below is an image read when generating the document revision history (an example of first document data), and the second image 252 is an image generated by reading the input document (an example of second document data).

[0117] If the correction condition is met in cases 1 to 4, the document correction unit 19 reads a character string from the position after correction stored in the document correction history in the second image 252, and replaces the recognized character string with this character string. Alternatively, the document correction unit 19 replaces the recognized character string with the corrected character string stored in the document correction history.

[0118] If the correction conditions are met in cases 5 to 8, the document correction unit 19 reads the character string from the position after correction in the second image 252, and replaces the recognized character string with this character string.

[0119] ●Case 1 to Case 4 In Cases 1 to 4, the character string in the item "Customer" has been corrected by the user. The desired recognized character string in the second image 252 is "XX Trading Co., Ltd." and the position is (x3, y3)(x4, y4).

[0120] Case 1 is a case where the position of the character recognition in the first image 251 is incorrect, and the position of the character recognition in the second image 252 is also the same, resulting in erroneous recognition.

[0121] 25 is a diagram illustrating the situation corresponding to Case 1 and the correction of a character string based on the correction conditions. In Case 1, the character string before correction (an example of a first character string), the position before correction, the character string after correction (an example of a second character string), the position after correction, the recognized character string (an example of a third character string), and the position of the recognized character string are as follows: String before correction = ○○ Co., Ltd. Position before correction = (x1, y1) (x2, y2) Corrected string = □□ Trading Co., Ltd. Corrected position = (x3, y3) (x4, y4) Recognition string = ○○ Co., Ltd. Position of recognition string = (x1, y1) (x2, y2) The modified condition for detecting Case 1 is as follows: Correction condition (1): The position before correction and the position of the recognized character string are the same. Correction condition (2): If the string before correction and the recognized string are the same Therefore, the situation in FIG. 25 satisfies the modification conditions (1) and (2).

[0122] Note that "the same position" does not require a perfect match, but refers to the case where the difference between the position before correction and the position of the recognized character string is less than a threshold. Whether "character strings" are the same is also determined based on a predetermined standard. In other words, they do not require perfect identity, and differences between half-width and full-width characters are allowed. Also, if the number of character strings in the input document that are equal to or greater than a threshold (for example, 90%) of the character strings in the same document information are included in the character strings in the input document, they can be determined to be the same. The same applies to the following correction conditions.

[0123] In either case of correction condition (1) or (2), it can be determined that there is a high possibility that the same mistake that occurred when extracting the character string from the first image 251 occurred when extracting the character string from the second image 252.

[0124] The document correction unit 19 may perform correction when either correction condition (1) or (2) is satisfied, or may combine correction conditions (1) and (2) and perform correction when both correction conditions (1) and (2) are satisfied.

[0125] Case 2 Case 2 is a case where the character recognition position in the first image 251 is correct, and the character recognition position in the second image 252 is also correct, but the character recognition fails.

[0126] 26 is a diagram illustrating the situation corresponding to Case 2 and the correction of a character string based on the correction conditions. In Case 2, the character string before correction, the position before correction, the character string after correction, the position after correction, the recognized character string, and the position of the recognized character string are as follows: String before correction = XX Trading Co., Ltd. Position before correction = (x3, y3) (x4, y4) Corrected string = □□ Trading Co., Ltd. Corrected position = (x3, y3) (x4, y4) Recognition string = XX Trading Co., Ltd. Position of recognition string = (x3, y3) (x4, y4) The modified condition for detecting case 2 is as follows: Correction condition (1): The position before correction and the position of the recognized character string are the same. Correction condition (2): If the string before correction and the recognized string are the same based on the specified criteria Correction condition (3): When the corrected position is the same as the position of the recognized character string Correction condition (4): If the corrected string and the recognized string are the same based on the specified criteria Therefore, the situation in FIG. 26 satisfies the modification conditions (1) to (4).

[0127] Note that if correction condition (3) is satisfied alone, it is unclear whether character recognition has failed. If correction condition (4) is satisfied alone, it is unclear whether character recognition has failed. However, by making corrections when correction condition (3) or (4) is satisfied, it is possible to correct the possibility that character recognition has failed in the same way.

[0128] The document correction unit 19 may perform correction when any one of the correction conditions (1) to (4) is satisfied, or may perform correction when two or more of the correction conditions (1) to (4) are combined and two or more are satisfied.

[0129] Case 3 Case 3 is a case where the position of the character recognized in the first image 251 is incorrect, and the position of the character recognized in the second image 252 is correct.

[0130] 27 is a diagram illustrating the situation corresponding to Case 3 and the correction of a character string based on the correction conditions. In Case 3, the character string before correction, the position before correction, the character string after correction, the position after correction, the recognized character string, and the position of the recognized character string are as follows: String before correction = ○○ Co., Ltd. Position before correction = (x1, y1) (x2, y2) Corrected string = □□ Trading Co., Ltd. Corrected position = (x3, y3) (x4, y4) Recognition string = XX Trading Co., Ltd. Position of recognition string = (x3, y3) (x4, y4) The modified condition for detecting case 3 is as follows: Correction condition (3): When the corrected position is the same as the position of the recognized character string Correction condition (4): If the corrected string and the recognized string are the same Therefore, the situation in FIG. 27 satisfies the modification conditions (3) and (4).

[0131] Note that if correction condition (3) is satisfied alone, it is unclear whether or not a misrecognition has occurred. If correction condition (4) is satisfied alone, it is unclear whether or not a misrecognition has occurred. However, by making a correction when correction condition (3) or (4) is satisfied, it is possible to correct any possible misrecognition.

[0132] The document correction unit 19 may perform correction when either correction condition (3) or (4) is satisfied, or may combine correction conditions (3) and (4) and perform correction when both correction conditions (3) and (4) are satisfied.

[0133] Case 4 Case 4 is a case where a character string is erroneously recognized in the first image 251 and then mis-recognized in the second image 252 due to a mistake in position.

[0134] 28 is a diagram illustrating the situation corresponding to Case 4 and the correction of a character string based on the correction conditions. In Case 4, the character string before correction, the position before correction, the character string after correction, the position after correction, the recognized character string, and the position of the recognized character string are as follows: String before correction = XX Trading Co., Ltd. Position before correction = (x3, y3) (x4, y4) Corrected string = □□ Trading Co., Ltd. Corrected position = (x3, y3) (x4, y4) Recognition string = ○○ Shoji Co., Ltd. Position of recognition string = (x1, y1) (x2, y2) The modified condition for detecting case 4 is as follows: Correction condition (5): When a character exists in the corrected position in the second image 252 Correction condition (6): When the corrected character string exists in the second image 252 Therefore, the situation in FIG. 28 satisfies the modification conditions (5) and (6).

[0135] Note that if correction condition (5) is satisfied alone, it is unclear whether or not a misrecognition has occurred. If correction condition (6) is satisfied alone, it is unclear whether or not a misrecognition has occurred. However, by making a correction when correction condition (5) or (6) is satisfied, it is possible to correct any possible misrecognition.

[0136] The document correction unit 19 may perform correction when either correction condition (5) or (6) is satisfied, or may combine correction conditions (5) and (6) and perform correction when both correction conditions (5) and (6) are satisfied.

[0137] ●Case 5~Case 8 In cases 5 to 8, the character string in the item "Billing Date" is corrected by the user. The desired recognition character string in the second image 252 is "2023 / 1 / 10" and the position is (x8, y8)(x9, y9).

[0138] Case 5 Case 5 is a case in which the position of the character recognition in the first image 251 is incorrect, and the position of the character recognition in the second image 252 is also incorrect, resulting in erroneous recognition.

[0139] 29 is a diagram illustrating the situation corresponding to Case 5 and the correction of a character string based on the correction conditions. In Case 5, the character string before correction, the position before correction, the character string after correction, the position after correction, the recognized character string, the position of the recognized character string, the attribute of the character string before correction, and the attribute of the recognized character string are as follows: String before correction = 2023 / 5 / 30 Position before correction = (x6, y6)(x7, y7) Corrected string = 2023 / 1 / 10 Corrected position = (x8,y8)(x9,y9) Recognized string =2023 / 5 / 30 Position of recognition string = (x6, y6)(x7, y7) Attribute of the original string = Date Recognition string attribute = date The modified condition for detecting case 5 is as follows: Correction condition (7): The position before correction and the position of the recognized character string are the same. Correction condition (8): The attributes of the string before correction and the attributes of the recognized string are the same. Therefore, the situation in FIG. 29 satisfies the modification conditions (7) and (8).

[0140] If correction condition (7) is met, it is determined that there is a high possibility that the same mistake that occurred when extracting the character string from the first image 251 occurred when extracting the character string from the second image 252. If correction condition (8) is met, it is unclear whether or not a misrecognition occurred. However, by making a correction when correction condition (8) is met, it is possible to correct any possible misrecognition.

[0141] The attribute means that the date is written in a predetermined format, such as xxxx / xx / xx or xxxx year xx month xx, and the amount is written in \xx,xxx.

[0142] The document correction unit 19 may correct the document when either the correction condition (7) or (8) is satisfied, or may combine the correction conditions (7) and (8) and cause the document correction unit 19 to correct the document when both the correction conditions (7) and (8) are satisfied.

[0143] Case 6 Case 6 is a case where the character recognition position in the first image 251 is correct, and the character recognition position in the second image 252 is also correct, but the character recognition fails.

[0144] 30 is a diagram illustrating the situation corresponding to Case 6 and the correction of a character string based on the correction conditions. In Case 6, the character string before correction, the position before correction, the character string after correction, the position after correction, the recognized character string, the position of the recognized character string, the attributes of the character string before correction, the attributes of the character string after correction, and the attributes of the recognized character string are as follows: String before correction = 2023 / 1 / 10 Position before correction = (x8, y8) (x9, y9) Corrected string = 2023 / 1 / 10 Corrected position = (x8,y8)(x9,y9) Recognized string = 2023 / I / 10 (The number "1" was mistakenly recognized as the capital letter "I") Position of recognition string = (x8,y8)(x9,y9) Attribute of the original string = Date Corrected string attribute = date Recognition string attribute = date The modified condition for detecting case 6 is as follows: Correction condition (7): The position before correction and the position of the recognized character string are the same. Correction condition (8): The attributes of the string before correction and the attributes of the recognized string are the same. Correction condition (9): If the corrected position is the same as the position of the recognized character string Correction condition (10): The attribute of the corrected string is the same as the attribute of the recognized string. Therefore, the situation in FIG. 30 satisfies the modification conditions (7) to (10).

[0145] Note that if correction condition (9) is satisfied alone, it is unclear whether or not a misrecognition has occurred. If correction condition (10) is satisfied alone, it is unclear whether or not a misrecognition has occurred. However, by making a correction when correction condition (9) or (10) is satisfied, it is possible to correct any possible misrecognition.

[0146] The document correction unit 19 may perform correction when any one of the correction conditions (7) to (10) is satisfied, or may perform correction when two or more of the correction conditions (7) to (10) are combined and two or more are satisfied.

[0147] Case 7 Case 7 is a case where the position of the character recognized in the first image 251 is incorrect, and the position of the character recognized in the second image 252 is correct.

[0148] 31 is a diagram illustrating the situation corresponding to Case 7 and the correction of a character string based on the correction conditions. In Case 7, the character string before correction, the position before correction, the character string after correction, the position after correction, the recognized character string, the position of the recognized character string, the attribute of the corrected character string, and the attribute of the recognized character string are as follows: String before correction = 2023 / 5 / 30 Position before correction = (x6, y6)(x7, y7) Corrected string = 2023 / 1 / 10 Corrected position = (x8,y8)(x9,y9) Recognized string = 2023 / I / 10 (The number "1" was mistakenly recognized as the capital letter "I") Position of recognition string = (x8,y8)(x9,y9) Corrected string attribute = date Recognition string attribute = date The modified condition for detecting case 7 is as follows: Correction condition (9): If the corrected position is the same as the position of the recognized character string Correction condition (10): The attribute of the corrected string is the same as the attribute of the recognized string. Therefore, the situation in FIG. 31 satisfies the modified conditions (9) and (10).

[0149] Note that if correction condition (9) is satisfied alone, it is unclear whether or not a misrecognition has occurred. If correction condition (10) is satisfied alone, it is unclear whether or not a misrecognition has occurred. However, by making a correction when correction condition (9) or (10) is satisfied, it is possible to correct any possible misrecognition.

[0150] The document correction unit 19 may perform correction when either correction condition (9) or (10) is satisfied, or may combine correction conditions (9) and (10) and perform correction when both correction conditions (9) and (10) are satisfied.

[0151] Case 8 Case 8 is a case where a character is erroneously recognized in the first image 251, and then erroneously recognized in the second image 252 due to an error in the position.

[0152] 32 is a diagram illustrating the situation corresponding to Case 8 and the correction of a character string based on the correction conditions. In Case 8, the attributes of the character string before correction, the position before correction, the character string after correction, the position after correction, the recognized character string, the position of the recognized character string, and the corrected character string are as follows: String before correction = 2023 / 1 / 10 (The number "1" was mistakenly recognized as the capital letter "I") Position before correction = (x8, y8) (x9, y9) Corrected string = 2023 / 1 / 10 Corrected position = (x8,y8)(x9,y9) Recognized string =2023 / 5 / 30 Position of recognition string = (x6, y6)(x7, y7) Corrected string attribute = date The modified condition for detecting case 8 is as follows: Correction condition (11): When a character exists in the corrected position in the second image 252 Correction condition (12): When a character string with the same attribute as the attribute of the corrected character string exists in the second image 252 Therefore, the situation in FIG. 32 satisfies the modified conditions (11) and (12).

[0153] Note that if correction condition (11) is satisfied alone, it is unclear whether or not a misrecognition has occurred. If correction condition (12) is satisfied alone, it is unclear whether or not a misrecognition has occurred. However, by making a correction when correction condition (11) or (12) is satisfied, it is possible to correct any possible misrecognition. The document correction unit 19 may correct the document when either the correction condition (11) or (12) is satisfied, or may combine the correction conditions (11) and (12) and cause the document correction unit 19 to correct the document when both the correction conditions (11) and (12) are satisfied.

[0154] <Automatic update of learning data when engine is changed> Next, we will explain automatic deletion of learning data, in which document learning unit 18 automatically deletes learning data when various engines are changed and the same document can be correctly recognized (the intended character string can be extracted). If the same document cannot be correctly recognized, document learning unit 18 updates the learning data.

[0155] Fig. 33 is a flowchart illustrating automatic updating of learning data when an engine is changed. The process in Fig. 33 is executed, for example, when the engine is changed or when the user modifies the extraction results.

[0156] First, the document learning unit 18 reads the learning data (S101).

[0157] The document learning unit 18 calculates the degree of match between the identical document information and the input document (S102). The method of calculating the degree of match is the same as in FIG.

[0158] Next, the document learning unit 18 compares the degree of match with a threshold value to determine whether or not the documents are the same (S103). If the degree of match is equal to or greater than the threshold value, the document learning unit 18 determines that the documents are the same. If the determination in step S103 is Yes, the process proceeds to step S109; if the determination is No, the process proceeds to step S104.

[0159] In step S104, the document learning unit 18 determines whether or not all the learning data has been verified (S104). If the determination in step S104 is Yes, the process proceeds to step S105, and if No, the process proceeds to step S101.

[0160] In step S109, the document learning unit 18 determines whether the document revision history and the extraction result from the current input document are the same (S109). This determination may be made in the same manner as in one or more of the determination methods in cases 1 to 8.

[0161] If the determination in step S109 is Yes, the process proceeds to step S110, and if the determination is No, the process proceeds to step S111.

[0162] In step S110, the document correction unit 19 corrects the extracted result based on the document correction history (S110). The document correction unit 19 turns off the correction item flag (S112). The correction item flag is a flag that indicates whether or not a correction has been made based on the document correction history (OFF if corrected, ON if not corrected).

[0163] In step S111, the document learning unit 18 turns on the correction item flag because the document correction unit 19 has not made any corrections based on the document correction history (S111).

[0164] In step S105, the display control unit 16 displays the confirmation and correction screen 230 (S105), and the user corrects the extraction results as necessary (S105).

[0165] Next, the document learning unit 18 determines whether the user has corrected the extracted results based on the correction item flag (S106). If the determination in step S106 is Yes, the process proceeds to step S114, and if No, the process proceeds to step S107.

[0166] In step S114, since the user has made a correction, the document learning unit 18 updates the learning data (S114). That is, regardless of whether the engine has been changed, since the user has made a correction, it is necessary to update the learning data (identical document information, correction history information, engine identification information).

[0167] In step S107, the document learning unit 18 determines whether one or more engines have been changed (S107). Whether an engine has been changed can be determined by comparing the engine identification information with the current engine information. Engine changes include at least one of upgrading, downgrading, or replacing the engine itself. After this process, the engine identification information is updated with the current engine information. If the determination in step S107 is Yes, the process proceeds to step S113; if the determination is No, the process proceeds to step S108.

[0168] In step S113, if the correction item flag is ON, the document learning unit 18 deletes the learning data (S113). That is, it is assumed that the determination of No in step S109 is due to an engine change, and since no corrections have been made by the user, the performance of the new engine has improved, and it is determined that the old learning data is not valid. Therefore, the document learning unit 18 deletes the learning data. Also, if the correction item flag is OFF, the determination of Yes in step S109 is true, so it is determined that the old learning data is valid even if the engine has been changed, and it does not need to be deleted.

[0169] In step S108, the document learning unit 18 determines whether all check items have been completed (S108). If the determination in step S108 is Yes, the process in Fig. 33 ends, and if No, the process proceeds to step S106.

[0170] <Major Effects> When the character string extraction device 2 of this embodiment determines that the documents are the same, it further assumes cases in which the character string is erroneously extracted and determines whether the correction conditions prepared for each case are met. If the correction conditions are met, the character string extraction device 2 corrects the character string using a correction method according to the correction conditions. Therefore, the character string extraction device 2 can suppress erroneous corrections when correcting based on the document correction history. If the character string cannot be correctly extracted due to the same mistake as when the correction history was registered, it can be corrected.

[0171] Furthermore, when automatically generating learning data for a wide variety of documents, only the items modified by the user are reflected in the learning data, thereby reducing the size of the learning data and the time required to determine whether the documents are the same.

[0172] Furthermore, if a statement is present, a predetermined position on the statement is used as an anchor and character strings are extracted at relative positions from there, making it possible to determine with high accuracy whether or not the documents are the same.

[0173] Furthermore, when the engine is changed, the learning data is automatically deleted, so that the correct learning data can always be associated with the engine.

[0174] <Other application examples> The best mode for carrying out the present invention has been described above using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0175] For example, in this embodiment, the correction conditions (1) to (12) have been described for each of cases 1 to 8, but any combination of two or more of the correction conditions (1) to (12) may be used.

[0176] Furthermore, the processing described in the present embodiment may be executed by a server device. In this case, the terminal device 3 and the character string extraction device 2, which is a server device, communicate via a network, and the terminal device 3 executes a Web application. When the terminal device 3 transmits document data of a form to the character string extraction device 2 using the Web application, the character string extraction device 2 extracts character strings and transmits the extraction results to the terminal device 3. The character string extraction device 2, which is a server device, does not necessarily perform all of the processing from character recognition to document learning, and the terminal device 3 may execute some of the processing using the Web application.

[0177] In addition, the configuration examples in Fig. 6 and the like are divided according to main functions to facilitate understanding of the processing by the character string extraction device 2. The present invention is not limited by the way in which the processing units are divided or the names of the processing units. The processing by the character string extraction device 2 can also be divided into more processing units depending on the processing content. Furthermore, it can also be divided so that one processing unit includes more processes.

[0178] Furthermore, each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the above-described functions.

[0179] Embodiments of the present invention provide significant improvements in computer power and functionality. These improvements allow users to utilize computers that provide more efficient and robust interaction with tables, which are ways of storing and presenting information in information processing devices. Furthermore, embodiments of the present invention provide a better user experience through the use of more efficient, powerful, and robust user interfaces. Such user interfaces provide better interaction between humans and machines. [Explanation of symbols]

[0180] 1. OCR processing system 2 String extractor 4. Scanner device

Claims

1. An information processing device that extracts a character string from document data, an operation receiving unit that receives a correction of a first character string extracted as data of the first document data to a second character string and a correction of a position of the first character string to a position of the second character string; a document learning unit that stores the first character string before correction, the position of the first character string before correction, the second character string after correction accepted by the operation accepting unit, and the position of the second character string after correction as a document correction history, and that stores the first character string and the position of the first character string extracted as data of the first document data as identical document information; an acquisition unit that acquires second document data of the input document; a character string extraction unit that extracts a third character string and a position of the third character string from the second document data acquired by the acquisition unit; an identical document determination unit that calculates a degree of match based on a result of comparing at least one of the third character string and the first character string in the identical document information, and the position of the third character string and the position of the first character string in the identical document information; If the degree of match is equal to or greater than a threshold value and the character string extraction unit extracts the third character string, which is the same character string regardless of the input document, If the difference between the position of the first character string and the position of the third character string is less than a threshold value, If the first character string and the third character string are determined to be the same based on a predetermined criterion, If the difference between the position of the second character string and the position of the third character string is less than a threshold value, If the second character string and the third character string are determined to be the same based on a predetermined criterion, If a character string exists at the position of the second character string in the second document data, or If the second character string exists in the second document data, If at least one of the following holds: a document correction unit that corrects the third character string extracted by the character string extraction unit based on the document correction history; An information processing device having the above.

2. An information processing device that extracts a character string from document data, an operation receiving unit that receives a correction of a first character string extracted as data of the first document data to a second character string and a correction of a position of the first character string to a position of the second character string; a document learning unit that stores the first character string before correction, the position of the first character string before correction, the second character string after correction accepted by the operation accepting unit, and the position of the second character string after correction as a document correction history, and that stores the first character string and the position of the first character string extracted as data of the first document data as identical document information; an acquisition unit that acquires second document data of the input document; a character string extraction unit that extracts a third character string and a position of the third character string from the second document data acquired by the acquisition unit; an identical document determination unit that calculates a degree of match based on a result of comparing at least one of the third character string and the first character string in the identical document information, and the position of the third character string and the position of the first character string in the identical document information; When the degree of match is equal to or greater than a threshold value and the character string extraction unit extracts the third character string, which is a character string that differs depending on the input document, If the difference between the position of the first character string and the position of the third character string is less than a threshold value, If the attribute of the first character string and the attribute of the third character string are the same, If the difference between the position of the second character string and the position of the third character string is less than a threshold value, If the attribute of the second character string and the attribute of the third character string are the same, If a character string exists at the position of the second character string in the second document data, or If the second character string exists in the second document data, If at least one of the following holds: a document correction unit that corrects the third character string extracted by the character string extraction unit based on the document correction history; An information processing device having the above.

3. the document correction unit replaces the third character string with the second character string, or with a character string extracted from the position of the second character string in the second document data; The information processing device according to claim 1 .

4. the document correction unit replaces the third character string with a character string extracted from the position of the second character string in the second document data; The information processing device according to claim 2 .

5. the document learning unit saves the first character string, the position of the first character string, the second character string, and the position of the second character string as the document correction history only when the operation accepting unit accepts a correction of a first character string to a second character string and a correction of a position of the first character string to a position of the second character string; The information processing device according to claim 1 .

6. In addition to the first character string and the position of the first character string, the document learning unit storing, as the same document information, one or more of the table layout in the first document data, the number of lines or paragraphs in the first document data, or character information included in the lines or paragraphs; the same document determination unit calculates the degree of match based on a result of comparing at least one of the third character string and the first character string in the same document information, the position of the third character string and the position of the first character string in the same document information, the table layout in the first document data and the table layout in the second document data, the number of lines or paragraphs in the first document data and the number of lines or paragraphs in the second document data, or the character information in the first document data and the character information in the second document data. The information processing device according to any one of claims 1 to 5.

7. The same document information includes, for each data extracted by the character string extraction unit, whether to save the first character string and the position of the first character string; the same document determination unit compares the third character string with the first character string using only the first character string that is set to be stored in the same document information; comparing the position of the third character string with the position of the first character string using only the position of the first character string that is set to be saved in the same document information; comparing only the positions of the table layouts in the first document data and the second document data, which are set to be saved in the same document information; With respect to the number of lines or paragraphs in the first document data and the number of lines or paragraphs in the second document data that are set to be saved in the same document information, the numbers are counted based on the positions of the lines or paragraphs, and compared; or comparing character information included in the line or paragraph in the first document data, which is set to be stored in the same document information, with character information in the second document data; calculating the degree of match based on at least one comparison result; The information processing device according to claim 1 .

8. If a table is detected from the second document data acquired by the acquisition unit, using the position of the first character string relative to a reference point set in the table in the first document data and the position of the third character string relative to a reference point set in the table in the second document data, the same document determination unit compares the position of the third character string with the position of the first character string; The information processing device according to claim 1 .

9. When the document type or the company name of the third character string extracted as data by the character string extraction unit is the same as the document type or the company name included in the same document information as the first character string, the same document determination unit: the third character string other than the document type or the company name and the first character string, the position of the third character string other than the document type or the company name and the position of the first character string, a table layout in the first document data and a table layout in the second document data, the number of lines or paragraphs in the first document data and the number of lines or paragraphs in the second document data, or character information included in a line or the paragraph in the first document data and character information in the second document data; and calculating the degree of match based on the comparison result. The information processing device according to claim 1 .

10. the information processing device extracts the third character string and a position of the third character string using one or more engines that extract character strings from the document data; If one or more of said engines are modified, The document correction unit does not correct the third character string and the position of the third character string based on the document correction history, and When the operation acceptance unit does not accept the correction of the third character string to the second character string and the correction of the position of the third character string to the position of the second character string, the document learning unit deletes the identical document information and the document revision history. The information processing device according to claim 1 .

11. whether or not one or more of said engines have been modified; When the operation receiving unit receives a correction of the third character string to the second character string and a correction of the position of the third character string to the position of the second character string, the document learning unit updates the document correction history with the second character string after the correction accepted by the operation accepting unit and the position of the second character string after the correction; The information processing device according to claim 10.

12. A correction method performed by an information processing device that extracts a character string from document data, comprising: a process of correcting a first character string extracted as data of the first document data to a second character string and accepting the correction of the position of the first character string to the position of the second character string; a process of saving the first character string before correction, the position of the first character string before correction, the second character string after correction, and the position of the second character string after correction as a document correction history, and saving the first character string and the position of the first character string extracted as data of the first document data as identical document information; A process of obtaining second document data of the input document; extracting a third character string and a position of the third character string from the acquired second document data; a process of calculating a degree of match based on a result of comparing at least one of the third character string and the first character string in the same document information, and the position of the third character string and the position of the first character string in the same document information; If the degree of match is equal to or greater than a threshold value and the third character string is extracted, the third character string being the same character string regardless of the input document, If the difference between the position of the first character string and the position of the third character string is less than a threshold value, If the first character string and the third character string are determined to be the same based on a predetermined criterion, If the difference between the position of the second character string and the position of the third character string is less than a threshold value, If the second character string and the third character string are determined to be the same based on a predetermined criterion, If a character string exists at the position of the second character string in the second document data, or If the second character string exists in the second document data, If at least one of the following holds: correcting the third character string based on the document correction history; If the degree of match is equal to or greater than a threshold, and if the third character string, which is a character string that differs depending on the input document, is extracted, If the difference between the position of the first character string and the position of the third character string is less than a threshold value, If the attribute of the first character string and the attribute of the third character string are the same, If the difference between the position of the second character string and the position of the third character string is less than a threshold value, If the attribute of the second character string and the attribute of the third character string are the same, If a character string exists at the position of the second character string in the second document data, or If the second character string exists in the second document data, If at least one of the following holds: a process of correcting the third character string based on the document correction history; How to fix it.

13. An information processing device that extracts character strings from document data, an operation receiving unit that receives a correction of a first character string extracted as data of the first document data to a second character string and a correction of a position of the first character string to a position of the second character string; a document learning unit that stores the first character string before correction, the position of the first character string before correction, the second character string after correction accepted by the operation accepting unit, and the position of the second character string after correction as a document correction history, and that stores the first character string and the position of the first character string extracted as data of the first document data as identical document information; an acquisition unit that acquires second document data of the input document; a character string extraction unit that extracts a third character string and a position of the third character string from the second document data acquired by the acquisition unit; an identical document determination unit that calculates a degree of match based on a result of comparing at least one of the third character string and the first character string in the identical document information, and the position of the third character string and the position of the first character string in the identical document information; If the degree of match is equal to or greater than a threshold value and the character string extraction unit extracts the third character string, which is the same character string regardless of the input document, If the difference between the position of the first character string and the position of the third character string is less than a threshold value, If the first character string and the third character string are determined to be the same based on a predetermined criterion, If the difference between the position of the second character string and the position of the third character string is less than a threshold value, If the second character string and the third character string are determined to be the same based on a predetermined criterion, If a character string exists at the position of the second character string in the second document data, or If the second character string exists in the second document data, If at least one of the following holds: correcting the third character string extracted by the character string extraction unit based on the document correction history; When the degree of match is equal to or greater than a threshold value and the character string extraction unit extracts the third character string, which is a character string that differs depending on the input document, If the difference between the position of the first character string and the position of the third character string is less than a threshold value, If the attribute of the first character string and the attribute of the third character string are the same, If the difference between the position of the second character string and the position of the third character string is less than a threshold value, If the attribute of the second character string and the attribute of the third character string are the same, If a character string exists at the position of the second character string in the second document data, or If the second character string exists in the second document data, If at least one of the following holds: a document correction unit that corrects the third character string based on the document correction history; A program to function as a

Images