Document search device

Abstract

The present invention enhances the accuracy of document retrieval in a document including typographical errors. A document retrieval device according to one embodiment of the present invention comprises: an incorrect recognition table in which correctly recognized character strings and incorrectly recognized character strings are stored; and a document retrieval unit which acquires a retrieval character string, and retrieves the retrieval character string from both a document and character strings obtained by correcting the incorrectly recognized characters string included in the document into the correctly recognized character strings.

Description

Technical Field

[0004] , , , , , , ,

[0006] , , , ,

[0005] , , , , , ,

[0001] This application claims the priority of the basic application No. 2020-155507 filed with the Japan Patent Office on September 16, 2020, and incorporates all of its contents herein by reference.

[0002] The present invention relates to a document search device, a document search system, a document search program, and a document search method.

Background Art

[0003] Conventionally, document search for searching documents based on keywords specified by a user is known. In document search, a method called exact match search for extracting a character string that exactly matches the keyword is known. In exact match search, if a document contains a typo, there may be a failure to extract (also called search omission) what the user intended, or a search result different from what the user intended (also called search noise) may occur. Therefore, a method of searching after replacing a character assumed to be a typo with a correct character based on a prepared errata list is known (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, due to the replacement of characters based on such an errata list, a correct character may be mistaken for an incorrect character. That is, a character string that should have been extracted without replacing the character based on the errata list cannot be extracted.

[0006] Therefore, the present invention aims to improve the accuracy of document searches in documents containing typographical errors. [Means for solving the problem]

[0007] [1] A misrecognition table that stores correctly recognized strings and misrecognized strings, Get the search string, A document search unit searches for the search string from both the document and the string obtained by correcting the misrecognized string contained in the document to the correctly recognized string. A document retrieval device having the following features. [2] The document search device according to [1], wherein the correctly recognized string and the incorrectly recognized string are each strings consisting of one character. [3] The document retrieval device according to [1], wherein the correctly recognized string and the incorrectly recognized string are each morphemes consisting of multiple characters. [4] The document retrieval device according to [1], wherein a mistranslation table containing a plurality of pairs of correctly recognized and incorrectly recognized strings includes both cases in which the correctly recognized string and the incorrectly recognized string are strings consisting of one character each, and cases in which the correctly recognized string and the incorrectly recognized string are morphemes consisting of multiple characters each. [5] The system further has a document information table that stores the confidence level of the string obtained by correcting the misrecognized string contained in the document to the corrected string, and the confidence level of the string before correcting the misrecognized string contained in the document to the corrected string, The document search device according to any one of [1] to [4], wherein the document search unit excludes the string with low confidence from the search target. [6] The system further has a document information table that stores the confidence level of the string obtained by correcting the misrecognized string contained in the document to the corrected string, and the confidence level of the string before correcting the misrecognized string contained in the document to the corrected string, The document search unit lowers the priority of strings with low confidence levels. (Document search device according to any one of [1] to [4]) [7] The document search unit has a search result display unit, The search result display unit is a document search device according to any one of items [1] to [6] that displays the search results. [8] A document search system including a document search device and a user terminal, The aforementioned document search device, A misrecognition table that stores correctly recognized strings and misrecognized strings, Get the search string, A document search system comprising: a document search unit that searches for the search string from both a document and a string obtained by correcting the misrecognized string contained in the document to the correctly recognized string. [9] A computer having a misrecognition table in which correct and incorrect strings are stored, Get the search string, A document search program that functions as a document search unit, searching for the search string from both a document and a string obtained by correcting the misrecognized string contained in the document to the correctly recognized string.

[10] A method performed by a document retrieval device having a misrecognition table storing correctly recognized strings and misrecognized strings, Steps to obtain the search string, The steps of searching for the search string from both the document and the string obtained by correcting the misrecognized string contained in the document to the correctly recognized string, and A document search method that includes [specific document search methods]. [Effects of the Invention]

[0008] This invention can improve the accuracy of document searches in documents containing typographical errors. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows the overall system configuration including a document search device according to one embodiment of the present invention. [Figure 2] This figure shows the functional block of a document search device according to one embodiment of the present invention. [Figure 3]An example of data stored in the storage unit of the document search device according to an embodiment of the present invention. [Figure 4] A diagram for explaining document search (first embodiment) according to an embodiment of the present invention. [Figure 5] A diagram for explaining document search (second embodiment) according to an embodiment of the present invention. [Figure 6] A diagram for explaining an example of a method for creating a misrecognition table according to an embodiment of the present invention. [Figure 7] A diagram for explaining document search (third embodiment) according to an embodiment of the present invention. [Figure 8] A diagram for explaining an example of relative confidence according to an embodiment of the present invention. [Figure 9] A diagram for explaining accuracy verification according to an embodiment of the present invention. [Figure 10] A diagram for explaining accuracy verification according to an embodiment of the present invention. [Figure 11] A diagram for explaining accuracy verification according to an embodiment of the present invention. [Figure 12] A diagram for explaining accuracy verification according to an embodiment of the present invention. [Figure 13] A diagram for comparing accuracy verification according to an embodiment of the present invention. [Figure 14] A flowchart showing the process flow of document search according to an embodiment of the present invention. [Figure 15] A diagram showing the hardware configuration of a document search device and a user terminal according to an embodiment of the present invention.

Embodiments for Carrying Out the Invention

[0010] Hereinafter, each embodiment will be described with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions are omitted.

[0011] <Explanation of Terms> The present invention can be applied to document retrieval for searching any one or more documents. While this specification describes the case of documents converted from images of printed or handwritten documents using OCR (Optical Character Recognition), the present invention can be applied to document retrieval for searching any document, such as documents created using word processing software. "Document" may contain typographical errors.

[0012] <System Configuration> Figure 1 is a diagram showing the overall system configuration including a document search device 10 according to one embodiment of the present invention. As shown in Figure 1, the document search system 1 includes the document search device 10 and a user terminal 20. The document search device 10 can send and receive data with the user terminal 20 via any network. Each of these will be described below.

[0013] The document search device 10 is a device (for example, a server) that performs document search processing. The document search device 10 will be described in detail later with reference to Figure 2.

[0014] User terminal 20 is a terminal used by the user when searching for documents. For example, user terminal 20 may be a personal computer.

[0015] Furthermore, the document search device 10 may have some or all of the functions of the user terminal 20.

[0016] <Functional blocks of the document search device 10> Figure 2 is a diagram showing the functional blocks of a document search device 10 according to one embodiment of the present invention. As shown in Figure 2, the document search device 10 has a misrecognition table 202 and a document search unit 103. The misrecognition table will be described later. The document search device 10 may further have an OCR (Optical Character Recognition) processing unit 101, a source string candidate generation unit 102, and a storage unit 104. The document search unit 103 may further have a search result display unit 105. The storage unit 104 may have a document information table 201 and a misrecognition table 202. Furthermore, the document search device 10 having a misrecognition table can function as a document search unit 103 by executing a program. The document search device 10 having a misrecognition table can further function as an OCR processing unit 101, a source string candidate generation unit 102, and a search result display unit 105 by executing a program. Each of these will be described below.

[0017] The OCR processing unit 101 extracts string information (hereinafter also referred to as string information). Specifically, the OCR processing unit 101 acquires a document image (for example, an image of a handwritten document). The OCR processing unit 101 then performs OCR (Optical Character Recognition) processing and extracts string information from the acquired document image. In the case of a document created using word processing software, the string information within the document is used.

[0018] The original string candidate generation unit 102 generates original string candidates (for example, string candidates that are assumed to have existed in the original handwritten document) based on the misrecognition table 202. Specifically, the original string candidate generation unit 102 refers to the "correctly recognized" and "misrecognized" strings in the misrecognition table 202, corrects the "misrecognized" strings contained in the string information extracted by the OCR processing unit 101 (or, in the case of a document created using word processing software, the string information within the document) to "correctly recognized" strings, and generates original string candidates.

[0019] The document search unit 103 performs document search processing. Specifically, the document search unit 103 obtains the string to be searched for within the document (hereinafter also referred to as the search string) from the user terminal 20. The document search unit 103 also searches for the search string using both the string information extracted by the OCR processing unit 101 (or, in the case of a document created using document creation software, the string information within the document) and the original string candidate generation unit 102.

[0020] Furthermore, the search result display unit 105 of the document search unit 103 can display the results of the document search performed by the document search unit 103. The search results (for example, the document ID of the document containing the search string, the image of the handwritten document corresponding to the document containing the search string, etc.) are output to the user terminal 20.

[0021] Specifically, the search results display unit 105 can display the recognition results and corrected recognition results used in the search, although each item will be described later. The search results display unit 105 can also display the recognition confidence level and corrected confidence level used in the search. Furthermore, it can display a predetermined range of the document image that includes the recognition range coordinates corresponding to the search string.

[0022] The memory unit 104 stores the document information table 201 and the misrecognition table 202. The memory unit 104 will be described in detail below with reference to Figure 3.

[0023] Figure 3 shows an example of data stored in the storage unit 104 of a document retrieval device according to one embodiment of the present invention. As shown in Figure 3, the storage unit 104 stores a document information table 201 and a misrecognition table 202. Each of these will be described below.

[0024] <Document Information Table> The document information table 201 stores data for each document, including "Document ID," "Document Name," "Page," "Literacy Range Coordinates," "Literacy Result," "Literacy Confidence," "Correction Candidate," and "Correction Confidence."

[0025] A "document ID" is an identifier used to identify a document.

[0026] "Document name" is the name of the document.

[0027] In the case of a multi-page document, "page number" is information used to identify the page on which "literacy results" exist within the document.

[0028] The "literacy range coordinates" are coordinates that indicate the range in which "literacy results" exist.

[0029] The "literacy result" is the string information extracted from the document image by the OCR processing unit 101. In the case of documents created using word processing software, the string information within the document is used.

[0030] "Literacy confidence" is the degree of accuracy of the "literacy outcome." For details, please refer to Figure 8.

[0031] The "correction candidates" are candidates for the original string generated by the original string candidate generation unit 102. One or more correction candidates are stored.

[0032] The "correction confidence level" represents the degree of accuracy of each "candidate correction." For details, please refer to Figure 8.

[0033] <Misrecognition Table> The misrecognition table 202 stores data for pairs of "correctly recognized" and "incorrectly recognized" strings. As described above, the original string candidate generation unit 102 corrects the "incorrectly recognized" strings contained in the string information extracted by the OCR processing unit 101 (or, in the case of a document created using word processing software, the string information within the document) to "correctly recognized" strings, and generates a candidate for the original string (i.e., the "correction candidate" in the document information table 201 in Figure 3).

[0034] A "misrecognition" is a string of characters within a document that is assumed to be incorrect.

[0035] The "correct recognition" is the string corresponding to the string stored in the "incorrect recognition". Specifically, the "correct recognition" is the correct string that the string stored in the "incorrect recognition" should be corrected to.

[0036] Note that the string of "correct recognition" and the string of "incorrect recognition" may be a string consisting of one character, or may be a string consisting of multiple characters (for example, morphemes).

[0037] The incorrect recognition table 202 may be configured for each document, or may be configured in units of all the documents stored in the document information table 201. Also, the incorrect recognition table 202 may be configured for each of a plurality of similar documents such as those having a common technical field, or may be configured for each specific string composed of a plurality of morphemes including technical terms.

[0038] Also, the string of correct recognition and the string of incorrect recognition included in the incorrect conversion table 202 may each be a string consisting of one character, or may each be a morpheme consisting of multiple characters. When both the case where the string of correct recognition and the string of incorrect recognition are each a string consisting of one character and the case where they are each a morpheme consisting of multiple characters are included, it may be composed of a plurality of incorrect recognition tables 202.

[0039] Also, in the OCR processing unit 101, when performing OCR processing on the acquired image of the handwritten document, the number of characters may be incorrectly recognized. For example, when the radical and the character body are written separately by hand, "坪" may be incorrectly recognized as the two characters "土" and "平". Also, conversely, when two handwritten characters are written closely, for example, "山鳥" may be incorrectly recognized as the single character "嶋". Such cases of incorrect recognition may also be stored in the incorrect recognition table 202 as data of pairs of "incorrect recognition" and "correct recognition" respectively.

[0040] Hereinafter, a first embodiment of document search will be described while referring to FIG. 4, a second embodiment of document search will be described while referring to FIGS. 5 and 6, and a third embodiment of document search will be described while referring to FIGS. 7 and 8.

[0041] <First Embodiment> FIG. 4 is a diagram for explaining a document search (first embodiment) according to an embodiment of the present invention. The first embodiment is a case where the strings of "correct recognition" and "incorrect recognition" stored in the misrecognition table **202** are each a pair of strings consisting of one character. #1 and #2 in FIG. 4 are data of strings included in one document (in the case of document search for searching one document).

[0042] In the example of FIG. 4, assume that the "incorrect recognition" is "原" and the "correct recognition" is "厚", and the "incorrect recognition" is "原" and the "correct recognition" is "康". Also, assume that the "recognition result" is "原生労働省だけでなく" for #1 and "原生生物は、真菌類なども" for #2. Then, the "corrected recognition result (the "correction candidate" in the document information table **201** of FIG. 3)" is "厚生労働省だけでなく" and "康生労働省だけでなく" for #1, and "厚生生物は、真菌類なども" and "康生生物は、真菌類なども" for #2. Also, assume that the search strings for #1 are "厚生労働省", "厚生", and the search strings for #2 are "原生生物", "原生".

[0043] In #1, when the search target is only the recognition result, as shown in the "search result before correction" at the lower left of FIG. 4, the search strings "厚生労働省" and "厚生" cannot be extracted from the document (×). When the search target is both the recognition result and the corrected recognition result as in the present invention, as shown in the "search result after correction" at the lower right of FIG. 4, the search strings "厚生労働省" and "厚生" can be extracted from the document (HIT). However, in the first embodiment, when "原生" is input as the search string, "原生労働省だけでなく" will be extracted (false HIT).

[0044] When the search targets both the literacy result and the corrected literacy result, the search result display unit 105 displays a document image within a predetermined range, and the displayed document image includes the range in which the "literacy result" exists.

[0045] In #2, when the search target is only the literacy result, the search strings "protist" and "protist" can be extracted from the document (HIT), as shown in the "Search Results Before Correction" in the lower left of Figure 4. When the search target is both the literacy result and the corrected literacy result, as in the present invention, the search strings "protist" and "protist" can be extracted from the document (HIT), as shown in the "Search Results After Correction" in the lower right of Figure 4. However, in the first embodiment, when "health" is entered as the search string, "health organisms include fungi, etc." is extracted (false HIT).

[0046] When the search targets both the literacy result and the corrected literacy result, the search result display unit 105 displays a document image within a predetermined range, and the displayed document image includes the range in which the "literacy result" exists.

[0047] Thus, in the first embodiment, since both the document and the strings obtained by correcting the misrecognized strings within the document to the correctly recognized strings are searched, it is possible to avoid the situation where strings that could have been extracted without replacing the characters become unextractable.

[0048] <Second Embodiment> Figure 5 is a diagram illustrating a document search according to one embodiment of the present invention (second embodiment). In the second embodiment, the "correctly recognized" strings and "incorrectly recognized" strings stored in the misrecognition table 202 are pairs of strings consisting of multiple characters. Specifically, the "correctly recognized" strings and "incorrectly recognized" strings are not character units as in the first embodiment, but rather meaningful words (morphemes). #1 and #2 in Figure 5 are string data contained in a single document (in the case of a document search that searches a single document).

[0049] In the example in Figure 5, let's assume that the "misrecognition" was "Ministry of Proto-Labour and Labor" and the "correct recognition" was "Ministry of Health, Labour and Welfare," and that the "misrecognition" was "Kengen" and the "correct recognition" was "Kenko." Also, let's assume that the "literacy result" for #1 was "Not only the Ministry of Proto-Labour and Labor," and for #2 it was "Protists include fungi, etc." Then, the "corrected literacy result (the "correction candidate" in document information table 201 in Figure 3)" for #1 was "Not only the Ministry of Health, Labour and Welfare," and there was no corrected literacy result for #2. Also, let's assume that the search strings for #1 were "Ministry of Health, Labour and Welfare" and "Kosei," and the search strings for #2 were "Protists" and "Gensei."

[0050] In #1, when the search target is only the literacy result, as shown in the "Search Results Before Correction" in the lower left of Figure 5, the search strings "Ministry of Health, Labour and Welfare" and "Health" cannot be extracted from the document (×). When the search target is both the literacy result and the corrected literacy result, as in the present invention, as shown in the "Search Results with the Above Technology" in the lower right of Figure 5, the search strings "Ministry of Health, Labour and Welfare" and "Health" can be extracted from the document (HIT). However, in the second embodiment, when "Original" is entered as the search string, "Not only the Original Ministry of Labour" is extracted (False HIT).

[0051] When the search target includes both the reading result and the corrected reading result, the search result display unit 105 displays a document image within a predetermined range, and the displayed document image includes the image desired by the user.

[0052] In #2, when the search target is only the literacy result, the search strings "protist" and "protist" can be extracted from the document (HIT), as shown in the "Search Results Before Correction" in the lower left of Figure 5. When the search target is both the literacy result and the corrected literacy result, as in the present invention, the search strings "protist" and "protist" can be extracted from the document (HIT), as shown in the "Search Results Using the Above Technique" in the lower right of Figure 5.

[0053] When the search targets both the reading result and the corrected reading result, and the search result display unit 105 displays document images within a predetermined range, the displayed document images include only the images desired by the user.

[0054] Thus, in the second embodiment, even if "health" is entered as the search string, there is no incorrect corrected recognition result such as "health organisms include fungi, etc.," thus reducing search noise.

[0055] Figure 6 is a diagram illustrating an example of a method for creating a misrecognition table 202 according to one embodiment of the present invention. It will be explained in the order of (1), (2), (3), and (4). The original string is a string that exists in a handwritten document, and the OCR string is a string extracted by performing OCR processing on an image of the handwritten document.

[0056] First, in (1), the parts where differences exist between the original string and the OCR string are extracted using an existing algorithm such as SES (Shortest Edit Script). The parts in the OCR string where differences exist are considered misrecognized characters.

[0057] Next, in (2), morphological analysis is performed to divide the original string into morphemes.

[0058] Next, in (3), the morphemes divided in (2) are linked to the misrecognized characters extracted in (1).

[0059] Next, in (4), pairs of "correct recognition" and "incorrect recognition" are generated from the association in (3). Specifically, a morpheme (e.g., "medicine") is considered a "correct recognition". Also, a character (e.g., "study") corresponding to an incorrectly recognized character (e.g., "society") within the morpheme (e.g., "medicine") is replaced with the incorrectly recognized character (e.g., "society"), resulting in (e.g., "medical society") being considered an "incorrect recognition".

[0060] Thus, in the second embodiment, not only is it possible to avoid the inability to extract strings that could have been extracted without replacing characters, but by making the misrecognized strings and correctly recognized strings into morphemes consisting of multiple characters, it is possible to reduce search noise when creating candidates for the original string using the created misrecognition table.

[0061] Furthermore, the first and second embodiments may be combined. Specifically, the misrecognition table 202 may store both data for pairs of "correctly recognized" and "misrecognized" consisting of one character, and data for pairs of "correctly recognized" and "misrecognized" consisting of multiple characters (for example, morphemes). For example, among the strings corrected by pairs of "correctly recognized" and "misrecognized" consisting of one character, the strings of "correctly recognized" consisting of multiple characters (for example, morphemes) may be adopted as the target of the search.

[0062] <Third Embodiment> Figure 7 is a diagram illustrating a document search according to one embodiment of the present invention (third embodiment). In the third embodiment, the document search unit 103 can exclude low-confidence or modified-confidence results from the search, or assign a lower score to the search results. The third embodiment may be combined with the first embodiment, the second embodiment, or a combination of the first and second embodiments. #1 and #2 in Figure 7 are string data contained in a single document (in the case of a document search that searches a single document).

[0063] In the example in Figure 7, the "misrecognition" is "Protist Ministry of Health, Labour and Welfare" and the "correct recognition" is "Ministry of Health, Labour and Welfare," and the "misrecognition" is "Protist" and the "correct recognition" is "Health." Furthermore, the "literacy result" and its confidence level are assumed to be "Not only the Protist Ministry of Health, Labour and Welfare (literacy confidence: 0.5)" for #1 and "Protists include fungi, etc. (literacy confidence: 0.9)" for #2. Furthermore, the "corrected literacy result (the "correction candidate" in document information table 201 in Figure 3)" and its confidence level are assumed to be "Not only the Ministry of Health, Labour and Welfare (correction confidence: 0.8)" and "Not only the Health Ministry of Health, Labour and Welfare (correction confidence: 0.3)" for #1 and "Health organisms include fungi, etc. (correction confidence: 0.6)" for #2. Furthermore, the search strings for #1 are "Ministry of Health, Labour and Welfare" and "Health," and the search strings for #2 are "Protists" and "Protist."

[0064] In #1, when the search target is only the literacy result, as shown in the "Search Results Before Correction" in the lower left of Figure 7, the search strings "Ministry of Health, Labour and Welfare" and "Health" cannot be extracted from the document (×). In the present invention, when the search target is both the literacy result and the corrected literacy result, and the literacy result with low confidence or the corrected literacy result is excluded from the search target, as shown in the "Search Results with the Above Technology" in the lower right of Figure 7, the search strings "Ministry of Health, Labour and Welfare" and "Health" can be extracted from the document (HIT). Furthermore, in the third embodiment, even if "Original" is entered as the search string, low-confidence phrases such as "Not only the original Ministry of Labour" are not found, thus reducing search noise.

[0065] If the search target excludes low-confidence or corrected-literacy results from both the regular and corrected-literacy results, then when the search result display unit 105 displays a predetermined range of document images, the displayed document images include the images desired by the user.

[0066] In #2, when the search target is only the literacy results, the search strings "protist" and "protist" can be extracted from the document (HIT), as shown in the "Search Results Before Correction" in the lower left of Figure 7. When the search target is both the literacy results and the corrected literacy results, with low confidence levels and corrected literacy results excluded from the search target, as in the "Search Results Using the Above Technique" in the lower right of Figure 7, the search strings "protist" and "protist" can be extracted from the document (HIT).

[0067] If the search target excludes low-confidence or corrected-literacy results from both the regular and corrected-literacy results, then when the search result display unit 105 displays a predetermined range of document images, the displayed document images include the images desired by the user.

[0068] Figure 8 illustrates an example of relative confidence in one embodiment of the present invention. The method for calculating confidence in Figure 8 utilizes the fact that unnatural sentences containing sequences of characters that do not make sense in Japanese are divided into many morphemes. This will be explained in detail below.

[0069] For example, suppose the OCR string (literacy result) is "(0) In Japan's Shio Business Act, a rotating substance containing 4 / 100 or more sodium chloride." Note that the OCR string is a string extracted by performing OCR processing on an image of a handwritten document. Also, suppose the "misrecognition" is "Shio Business Act" and the "correct recognition" is "Shio Business Act," the "misrecognition" is "rotating substance" and the "correct recognition" is "solid substance," the "misrecognition" is "10" and the "correct recognition" is "10g." In Figure 8, #1 to #3 are the correct / incorrect pairs stored in the misrecognition table.

[0070] In the example shown in Figure 8, if we use one of the correct / incorrect pairs #1 to #3 to generate correction candidates for the OCR string, the following correction candidates will be generated: "(1) In the case of Japan's Salt Business Act, a shaped substance containing 4 / 100 or more sodium chloride", "(2) In the case of Japan's Indication Business Act, a solid substance containing 4 / 100 or more sodium chloride", and "(3) In the case of Japan's Indication Business Act, a shaped substance containing 4 / 10g or more sodium chloride".

[0071] Next, we perform morphological analysis on (0), (1), (2), and (3) above. Morphological analysis divides sequences of characters that do not make sense as Japanese into more morphemes. Specifically, let's assume that morphological analysis of (0) divides it into 22 morphemes, morphological analysis of (1) divides it into 21 morphemes, morphological analysis of (2) divides it into 21 morphemes, and morphological analysis of (3) divides it into 23 morphemes. In the method for calculating confidence shown in Figure 8, the more morphemes an OCR string (literacy result) and correction candidate have, the lower the confidence level of that OCR string (literacy result) and correction candidate can be.

[0072] Furthermore, the confidence level may be calculated using methods other than the morphological analysis method shown in Figure 8 (for example, methods using natural language processing). Also, the confidence level may be calculated on a line-by-line basis, a sentence-by-sentence basis, or a paragraph-by-paragraph basis within the document.

[0073] When string information is obtained from a document image, one method for calculating confidence levels on a character-by-character basis is to use character-image similarity.

[0074] When confidence levels are calculated on a sentence-by-sentence basis, one method for calculating confidence levels is to use part-of-speech order analysis, which focuses on the order of parts of speech, such as determining whether a particular order of nouns and verbs is natural.

[0075] Thus, in the third embodiment, not only is it possible to avoid the inability to extract strings that could have been extracted without replacing characters, but search noise can also be reduced by excluding or lowering the priority of items with low confidence levels. Furthermore, in the third embodiment, the document is modified at the character or word (morpheme) level, and the degree of accuracy of the modification is judged at the line, sentence, and paragraph level within the document, so the appropriateness of the modification can be judged for each line, sentence, and paragraph within the document.

[0076] <Verification of search accuracy> Figure 9 is a diagram illustrating the accuracy verification according to one embodiment of the present invention. Four accuracy verifications were performed: "Accuracy Verification #1 (Original Document)", "Accuracy Verification #2 (OCR Data)", "Accuracy Verification #3 (OCR Data with Corrected Misrecognition (Multiple Candidates Retained))", and "Accuracy Verification #4 (OCR Data with Corrected Misrecognition (Multiple Candidates Retained, Confidence Level Retained))".

[0077] The specific procedure involved first using the text shown in the upper left of Figure 9 as the source text, and then creating a handwritten-style document image (shown in the upper center of Figure 9) containing handwritten characters using a crude handwritten font from the source text. For accuracy verification #1, the source text (source document) used was the text information from the "Salt" entry (Japanese site) on the multilingual internet encyclopedia site Wikipedia®, which consists of approximately 13,000 characters.

[0078] Next, an OCR processing tool was used to extract text information from the document image, obtaining OCR data consisting of the OCR text shown in the upper right of Figure 9. The text information contained in the OCR text includes misrecognitions as a result of the recognition process.

[0079] Following the method for creating the misrecognition table described above, a misrecognition table was generated from the original text and OCR text, as shown in the lower left of Figure 9. For the item "salt," approximately 900 correct / incorrect pairs were created.

[0080] Accuracy verification #1 is a verification of the accuracy of the search in the original document.

[0081] In accuracy verification #2, a search for the search string was performed using OCR data that still contained misrecognitions, without using a misrecognition table. Accuracy verification #2 will be explained with reference to Figure 10.

[0082] In accuracy verification #3, the search string was searched from both the document and the strings in which the misrecognized strings contained in the document were corrected to the correctly recognized strings. In other words, the search was performed from both the OCR data, which is the OCR correction candidate data, and the multiple original string candidates, which are the OCR correction candidate data in which the misrecognized strings contained in the OCR data were corrected to the correctly recognized strings. Accuracy verification #3 will be explained with reference to Figure 11.

[0083] In accuracy validation #4, low-confidence or corrected literacy results are excluded from the search. Accuracy validation #4 will be explained with reference to Figure 12.

[0084] In accuracy verification #1 to #4, "Precision," "Recall," and "F1 value" were calculated, respectively.

[0085] "Precision" (also called accuracy) is a value calculated by dividing the number of correct strings found in a search by the total number of strings found in the search, and can be considered an indicator of the accuracy of the search. The higher the "Precision," the less search noise there is. Precision is calculated as TP = TP / (TP + FP) (TP: True Positive, FP: False Positive).

[0086] "Recall" (also called recall rate or sensitivity) is a value calculated by dividing the number of correct strings found in a search by the total number of correct strings. It can be considered an indicator of the comprehensiveness of the search. A higher "Recall" value indicates that more correct strings have been found. Recall is calculated as Recall = TP / (TP + FN) (TP: True Positive, FN: False Negative).

[0087] The "F1 score" is calculated using the harmonic mean of "Precision" and "Recall," and can be considered an indicator of the balance between search accuracy and comprehensiveness. The F1 score is calculated as: F1 = 2Recall·Precision / (Recall+Precision).

[0088] Figure 10 is a diagram illustrating accuracy verification (accuracy verification #2) according to one embodiment of the present invention. In accuracy verification #2, the "ground truth data" (i.e., the data from the original document) shown in the upper left of Figure 10 was compared with the "OCR data" (i.e., the data from the OCR document) shown in the upper right of Figure 10. The procedure for verifying accuracy will be explained in detail. Each word (for example, "sodium chloride," "rock salt," etc.) was counted as TP if it was present in the first line of the ground truth data and also in the first line of the OCR data, as FN if it was present in the first line of the ground truth data but not in the first line of the OCR data, and as FP if it was not present in the first line of the ground truth data but was present in the first line of the OCR document. This was repeated for the second line, ..., and so on. Then, "Precision," "Recall," and "F1 value" were calculated. In this way, accuracy verification #2 shows how much the search accuracy is inferior in the OCR-processed recognition results.

[0089] Figure 11 is a diagram illustrating accuracy verification (accuracy verification #3) according to one embodiment of the present invention. In accuracy verification #3, the correct data (i.e., the data of the original document) was compared with the "OCR data with misrecognition correction (holding multiple candidates)" shown at the top of Figure 11. Specifically, the method for creating the OCR data with misrecognition correction (holding multiple candidates) will be explained. For each row of the OCR data (i.e., the data of the OCR document), correction candidates are generated based on the misrecognition table (in the example in Figure 11, "salt-containing" is corrected to "rock salt", "salt" is corrected to "", etc.). Then, the OCR data and the generated correction candidates are concatenated into one line with a delimiter (in the example in Figure 11, "@@@") in between. The procedure for accuracy verification is the same as in accuracy verification #2, so the explanation will be omitted.

[0090] Figure 12 is a diagram illustrating accuracy verification (accuracy verification #4) according to one embodiment of the present invention. In accuracy verification #4, the correct data (i.e., the data of the original document) was compared with the "OCR data with misrecognition correction (multiple candidates retained, confidence level retained)" shown at the top of Figure 12. Specifically, the method for creating the OCR data with misrecognition correction (multiple candidates retained, confidence level retained) will be explained. Similar to accuracy verification #3, correction candidates are generated for each row of the OCR data (i.e., the data of the OCR document) based on the misrecognition table (in the example in Figure 12, "salt-containing" is corrected to "rock salt", "salt" is corrected to "", etc.). Then, confidence levels are assigned to the OCR data and the generated correction candidates. After that, the OCR data and the generated correction candidates are concatenated into a single line separated by a delimiter (in the example in Figure 12, "@@@"). The procedure for verifying accuracy is the same as in accuracy verification #2, so the explanation will be omitted (in accuracy verification #4, items with low confidence are excluded from the search (×)).

[0091] Figure 13 is a diagram for comparing accuracy verifications according to one embodiment of the present invention. Figure 13 shows the "Precision," "Recall," and "F1 value" for the above-mentioned accuracy verification #1 ("#1 Original Document" in Figure 9), accuracy verification #2 ("#2 OCR Data" in Figure 10), accuracy verification #3 ("#3 Misrecognition Corrected OCR Data (Multiple Candidates Retained)" in Figure 11), and accuracy verification #4 ("#4 Misrecognition Corrected OCR Data (Multiple Candidates Retained, Confidence Level Retained)" in Figure 12). Note that 800 pairs were used from the data stored in the misrecognition table (approximately 900 correct / incorrect pairs for the item "salt" in this example), excluding those where "misrecognition" is a single character (×). The confidence level was calculated using a method employing morphological analysis as shown in Figure 8.

[0092] As shown in Figure 13, the F1 values ​​of the first and second embodiments, "#3 Misrecognition Corrected OCR Data (Multiple Candidates Retained)," were higher than the F1 value of "#2 OCR Document." Furthermore, the F1 value of the third embodiment, "#4 Misrecognition Corrected OCR Data (Multiple Candidates Retained, Confidence Level Retained)," was higher than the F1 value of "#3 Misrecognition Corrected OCR Data (Multiple Candidates Retained)."

[0093] <Processing method> Figure 14 is a flowchart showing the processing flow for searching for documents related to one embodiment of the present invention.

[0094] In step 11 (S11), the OCR processing unit 101 extracts string information. Specifically, the OCR processing unit 101 acquires a document image (for example, an image of a handwritten document). Next, the OCR processing unit 101 performs OCR (optical character recognition) processing and extracts string information from the acquired document image. In the case of a document created using word processing software, the string information within the document is used.

[0095] In step 12 (S12), the original string candidate generation unit 102 generates original string candidates based on the misrecognition table 202. Specifically, the original string candidate generation unit 102 refers to the "correctly recognized" and "misrecognized" strings in the misrecognition table 202, corrects the "misrecognized" strings included in the string information of S11 to "correctly recognized" strings, and generates original string candidates.

[0096] In step 13 (S13), the document search unit 103 scans the string information from S11 and the candidate original strings generated in S12. Specifically, the document search unit 103 searches for the search string obtained from the user terminal 20 using the string information from S11 and the candidate original strings from S12.

[0097] In step 14 (S14), the search result display unit 105 of the document search unit 103 outputs the search results (for example, the document ID of the document containing the search string, an image of the handwritten document corresponding to the document containing the search string, etc.) to the user terminal 20.

[0098] <Effects> Thus, this invention searches both the document and the strings obtained by correcting misrecognized strings within the document to correctly recognized strings. Therefore, it is possible to avoid situations where strings that could have been extracted without replacing the characters are not extracted. Furthermore, by converting the misrecognized strings and correctly recognized strings into morphemes, search noise can be reduced. Additionally, search noise can be reduced by excluding or lowering the priority of items with low confidence levels.

[0099] <Hardware Configuration> Figure 15 is a hardware configuration diagram of a document search device 10 and a user terminal 20 according to one embodiment of the present invention. The document search device 10 and user terminal 20 have a CPU (Central Processing Unit) 1001, a ROM (Read Only Memory) 1002, and a RAM (Random Access Memory) 1003. The CPU 1001, ROM 1002, and RAM 1003 form a so-called computer.

[0100] Furthermore, the document search device 10 and user terminal 20 may include an auxiliary storage device 1004, a display device 1005, an operating device 1006, an I / F (Interface) device 1007, and a drive device 1008.

[0101] The document search device 10 and the user terminal 20 are interconnected via bus B.

[0102] The CPU 1001 is a computing device that executes various programs installed on the auxiliary storage device 1004.

[0103] ROM1002 is non-volatile memory. ROM1002 functions as a main memory device that stores various programs, data, etc., necessary for the CPU1001 to execute the various programs installed on the auxiliary storage device 1004. Specifically, ROM1002 functions as a main memory device that stores boot programs such as BIOS (Basic Input / Output System) and EFI (Extensible Firmware Interface).

[0104] RAM1003 is a volatile memory such as DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory). RAM1003 functions as a main memory device that provides a working area that is expanded when various programs installed on the auxiliary storage device 1004 are executed by the CPU 1001.

[0105] The auxiliary storage device 1004 is an auxiliary storage device that stores various programs and information used when various programs are executed.

[0106] The display device 1005 is a display device that displays the internal status of the document search device 10 and the user terminal 20.

[0107] The operating device 1006 is an input device used by the operator of the document search device 10 and the user terminal 20 to input various instructions to the document search device 10 and the user terminal 20.

[0108] The I / F device 1007 is a communication device that connects to a network and communicates with other devices.

[0109] The drive device 1008 is a device for setting the recording medium 1009. The recording medium 1009 here includes media that record information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, and magneto-optical disks. The recording medium 1009 may also include semiconductor memory that records information electrically, such as EPROM (Erasable Programmable Read Only Memory) and flash memory.

[0110] The various programs to be installed on the auxiliary storage device 1004 are installed, for example, when the distributed recording medium 1009 is set in the drive device 1008 and the various programs recorded on the recording medium 1009 are read by the drive device 1008. Alternatively, the various programs to be installed on the auxiliary storage device 1004 may be installed by downloading them from the network via the I / F device 1007.

[0111] Although embodiments of the present invention have been described in detail above, the present invention is not limited to the specific embodiments described above, and various modifications and changes are possible within the scope of the gist of the present invention as described in the claims. [Explanation of symbols]

[0112] 1. Document Search System 10 Document search device 20 User Terminals 101 OCR Processing Unit 102 Original string candidate generator 103 Document Search Department 104 Storage section 105 Search Results Display Section 201 Document Information Table 202 Misrecognition Table 1001 CPU 1002 ROM 1003 RAM 1004 Auxiliary storage device 1005 Display device 1006 Operating device 1007 I / F device 1008 Drive unit 1009 Recording media

Claims

1. A misrecognition table that stores correctly recognized strings and misrecognized strings, Get the search string, A document search unit searches for the search string from both the document and the string obtained by correcting the misrecognized string contained in the document to the correctly recognized string. It has, The document search unit searches using the confidence level of the string obtained by correcting the misrecognized string contained in the document to the corrected string, and the confidence level of the string contained in the document before correcting the misrecognized string to the correct string.

2. The document retrieval device according to claim 1, wherein the correctly recognized string and the incorrectly recognized string are each strings consisting of one character.

3. The document retrieval device according to claim 1, wherein the correctly recognized string and the incorrectly recognized string are each morphemes consisting of multiple characters.

4. The document retrieval device according to claim 1, wherein the misconversion table, which includes a plurality of pairs of correctly recognized and incorrectly recognized strings, includes both cases where the correctly recognized string and the incorrectly recognized string are each strings consisting of one character, and cases where they are each morphemes consisting of multiple characters.

5. The system further includes a document information table that stores the confidence level of the string obtained by correcting the misrecognized string contained in the document to the corrected string, and the confidence level of the string before correcting the misrecognized string contained in the document to the corrected string. The document search device according to any one of claims 1 to 4, wherein the document search unit excludes the string with low confidence from the search target.

6. The system further includes a document information table that stores the confidence level of the string obtained by correcting the misrecognized string contained in the document to the corrected string, and the confidence level of the string before correcting the misrecognized string contained in the document to the corrected string. The document search unit lowers the priority of strings with low confidence levels, as described in any one of claims 1 to 4.

7. The document search unit has a search result display unit, The document search device according to any one of claims 1 to 6, wherein the search result display unit displays the search results.

8. A document search system including a document search device and a user terminal, The aforementioned document search device, A misrecognition table that stores correctly recognized strings and misrecognized strings, Get the search string, The system includes a document search unit that searches for the search string from both the document and the string obtained by correcting the misrecognized string contained in the document to the correctly recognized string, The document search unit is a document search system that searches using the confidence level of the string obtained by correcting the misrecognized string contained in the document to the corrected string, and the confidence level of the string before correcting the misrecognized string contained in the document to the correct string.

9. A computer having a misrecognition table that stores correctly recognized strings and misrecognized strings, Get the search string, It functions as a document search unit that searches for the search string from both the document and the string obtained by correcting the misrecognized string contained in the document to the correctly recognized string. The document search unit is a document search program that searches using the confidence level of the string obtained by correcting the misrecognized string contained in the document to the corrected string, and the confidence level of the string before correcting the misrecognized string contained in the document to the correct string.

10. A method performed by a document retrieval device having a misrecognition table that stores correctly recognized strings and misrecognized strings, Steps to obtain the search string, The steps of searching for the search string from both the document and the string obtained by correcting the misrecognized string contained in the document to the correctly recognized string, and Includes, A document search method comprising the steps of searching, wherein the search is performed using the confidence level of the string obtained by correcting the misrecognized string contained in the document to the corrected string, and the confidence level of the string before correcting the misrecognized string contained in the document to the correct string.

Images