Document image processing system

Abstract

To reduce an extraction mistake for a character-string aggregate having a specific relevancy in a plurality of character strings in a document image.SOLUTION: A document image processing unit 22 extracts a plurality of character strings included in a document image with character recognition processing, and identifies each position of the plurality of character strings. A data extraction unit 24 extracts a character-string aggregate having a specific relevancy among the plurality of character strings. A document data management unit 27 accumulates the character-string aggregate extracted in the document image and the position data of the character string in the character-string aggregate in a document database 12. An extraction data evaluation unit 25 uses the document database to evaluate validity of the character-string aggregate extracted from the document image to be processed on the basis of the character-string aggregate for a similar document image to the document image to be processed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a document image processing system. [Background technology]

[0002] One system (a) extracts a set of character images from a document image and generates text data (character strings) from the set of character images, (b) generates a feature vector corresponding to the text data of the set of character images, and (c) detects attribute label candidates and attribute value candidates for a specific attribute from the text data, and sets these pairs as pair candidates. The attribute label candidates are detected based on the feature vectors of the text data of the set of character images, and pairs of attribute labels and attribute values are identified based on, for example, the degree of association between them (see, for example, Patent Document 1). This allows the attribute value of a certain attribute to be accurately identified without using template data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-178723 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above system, pairs of attribute labels and corresponding attribute values are extracted from multiple character strings in a document image based on the relevance. Therefore, if the relevance is accurately derived, inappropriate pairs will not be extracted. However, when extracting pairs of attribute labels and attribute values from document images with various (arbitrary) layouts, there is a possibility that pairs will be extracted incorrectly (such as pairs that should be extracted not being extracted).

[0005] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a document image processing system that reduces errors in extracting sets of character strings that have a specific correlation among multiple character strings in a document image. [Means for solving the problem]

[0006] The document image processing system of the present invention comprises a document image processing unit that extracts multiple character strings included in a document image through character recognition processing and identifies the position of each of the multiple character strings; a data extraction unit that extracts a set of character strings having a specific correlation from the multiple character strings; a document data management unit that accumulates, for each document image, the set of character strings extracted from the document image in at least a document database; and an extracted data verification unit that uses the document database to verify the validity of the set of character strings extracted from the document image to be processed based on the set of character strings for document images similar to the document image to be processed.

[0007] The document image processing method according to the present invention comprises the steps of extracting a plurality of character strings contained in a document image by character recognition processing and identifying the position of each of the plurality of character strings; extracting a set of character strings having a specific correlation from the plurality of character strings; storing, for each document image, the set of character strings extracted from the document image in at least a document database; and using the document database to verify the validity of the set of character strings extracted from the document image to be processed based on the set of character strings for document images similar to the document image to be processed.

[0008] A document image processing program according to the present invention causes a computer to function as the document image processing unit, the data extraction unit, the document data management unit, and the extracted data verification unit. [Effects of the Invention]

[0009] According to the present invention, a document image processing system is provided that reduces errors in extracting a set of character strings having a specific correlation among a plurality of character strings in a document image.

[0010] The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing the configuration of a document image processing system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating a data flow in the document image processing system shown in FIG. [Figure 3] FIG. 3 is a diagram showing an example of a document image. [Figure 4] FIG. 4 is a diagram illustrating character strings (parts) extracted from the document image shown in FIG. 3 and their associations (connections). [Figure 5] FIG. 5 is a diagram showing an example of an operation screen for assisting the user in manual correction by the extracted data manual correction unit 26 in the second embodiment. [Figure 6] FIG. 6 is a diagram showing an example of an operation screen for assisting the user in manual correction by the extracted data manual correction unit 26 in the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0013] Embodiment 1

[0014] Fig. 1 is a block diagram showing the configuration of a document image processing system according to an embodiment of the present invention. Fig. 2 is a diagram illustrating the data flow of the document image processing system shown in Fig. 1. The document image processing system shown in Fig. 1 is configured with one information processing device (personal computer, server, etc.), but the processing units described below may be distributed among multiple information processing devices that can communicate data with each other. Furthermore, such multiple information processing devices may include GPUs (Graphics Processing Units) that process specific operations in parallel.

[0015] The document image processing system shown in FIG. 1 includes a storage device 1, a communication device 2, an image reading device 3, a display device 4, an input device 5, and a processing device 6.

[0016] The storage device 1 is a non-volatile storage device such as a flash memory or a hard disk, and stores various data and programs. Here, the storage device 1 stores a document image processing program 11, and also stores system setting data (such as coefficient setting values of a learning device such as a neural network used in each processing unit described below) as needed. The document image processing program 11 may be stored on a portable computer-readable recording medium such as a CD (Compact Disk). In this case, for example, the document image processing program 11 is installed from the recording medium into the storage device 1. The document image processing program 11 may be a single program or a collection of multiple programs.

[0017] The communication device 2 is a device capable of data communication, such as a network interface, a peripheral device interface, or a modem, and performs data communication with other devices as needed. The image reading device 3 optically reads a document image from a document and generates image data (e.g., raster image data) of the document image. The communication device 2 and the image reading device 3 are provided as needed.

[0018] The display device 4 is a device (such as a liquid crystal display) that displays various information to the user. The input device 5 is a device (such as a keyboard, mouse, or touch panel) that detects user operations.

[0019] The arithmetic processing device 6 is a computer equipped with a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc., and operates as various processing units by loading programs from the ROM, storage device 1, etc. into the RAM and executing them on the CPU.

[0020] Here, by executing the document image processing program 11, the processing device 6 operates as a document image acquisition unit 21, a document image processing unit 22, a document type determination unit 23, a data extraction unit 24, an extracted data verification unit 25, an extracted data manual correction unit 26, a document data management unit 27, a data output unit 28, and a machine learning processing unit 29.

[0021] The document image acquisition unit 21 acquires a document image as image data such as raster image data. The document image may be an image of a specific document to be submitted, such as a receipt, invoice, delivery note, or other form, or a customs clearance document, and may or may not be classified by document type. Here, the document image is an image of a document that includes attribute labels (character strings such as headings) and attribute values (character strings such as numbers, one or more words, etc.) for one or more attributes (description items, etc.). For example, the document image acquisition unit 21 reads out a document image as image data stored in the storage device 1, acquires a document image as image data received by the communication device 2 via a communication path such as a network, or acquires a document image as image data generated by the image reading device 3.

[0022] The document image processing unit 22 extracts multiple character strings contained in the acquired document image (document image to be processed) through character recognition processing. In this embodiment, the document image processing unit 22 identifies the multiple character strings as well as their respective positions. Specifically, the position of each character string's bounding box (such as the center or a predetermined corner of the bounding box) is identified as the position of the character string. Specifically, character objects (raster images) are extracted from the document image using existing technology (e.g., region separation processing or a machine-learned deep neural network), and the character objects are converted into character strings (text data) through character recognition processing. The position (position data) of each of the multiple character strings is also identified.

[0023] In this embodiment, the document image processing unit 22 also generates a document vector for the document image to be processed. Specifically, the document vector is generated by embedding the image and character strings in the document image using an existing method. Here, the document vector for a certain document image is generated so as to increase the cosine similarity (i.e., so as to shorten the cosine distance) with the document vector of a document with similar content to the document image. Note that the document vector may also be generated using methods such as BOW (Bag of Words) and BERT (Bidirectional Encoder Representations from Transformers).

[0024] The document type determination unit 23 determines the document type of the document image. In this embodiment, the document type determination unit 23 (a) identifies the document type of the similar document image corresponding to the document image to be processed in the document database 12, and (b) sets the identified document type as the document type of the document image to be processed.

[0025] In this embodiment, similar document images are detected in the document database 12 based on the similarity (e.g., cosine similarity or cosine distance) to the document vector of the document image to be processed. For example, among the document images registered in the document database 12 (such as character strings, position data, and document vectors), a document image whose similarity to the document vector of the document image to be processed is equal to or greater than a predetermined value and whose document vector has the highest similarity is detected as a similar document image. Note that the document database 12 does not necessarily have to store raster image data of similar document images.

[0026] When detecting similar document images in the document database 12, the document type determination unit 23 may derive the centroid of the document vectors registered in the document database 12 for each document type in the document database 12 and register it in the document database 12, and determine that a similar document image exists if the distance from the document vector of the document image to be processed to any of the registered centroids is less than a predetermined threshold, and otherwise determine that a similar document image does not exist.

[0027] In this case, a classifier (a machine-learned learning device) is provided that identifies the document type of a document image from its document vector, and if the document type determination unit 23 determines that a similar document image exists in the document database 12, it detects the similar document image in the document database 12 as described above, and if it determines that a similar document image does not exist in the document database 12, it uses the classifier to identify the document type based on the document vector.

[0028] The document type determination unit 23 may derive the similarity (e.g., cosine distance) between the document vector of the document image to be processed and only for the document vector of the document type with the closest center of gravity (i.e., excluding document types other than the document type with the closest center of gravity), and detect the document image of the document vector with the highest derived similarity (e.g., the shortest cosine distance) as a similar document image. In this way, fewer document vectors need to be compared, and similar document images can be detected in a short time.

[0029] The data extraction unit 24 extracts a set of character strings having a specific relationship (for example, a relationship between an attribute label and an attribute value) from the extracted multiple character strings. Here, the data extraction unit 24 identifies a set of character strings corresponding to the extracted multiple character strings and the above-mentioned position data. In this embodiment, the data extraction unit 24 extracts the set of character strings having the specific relationship using a machine-learned graph neural network (GNN) in which the character string vectors of each of the multiple character strings in the document image are used as nodes and the specific relationship between the character strings is used as a connection between the nodes. The character string vectors are obtained by embedding character strings (text data) using a predetermined method.

[0030] In this embodiment, the data extraction unit 24 extracts a set of character strings having a specific correlation using a learner (such as the above-mentioned GNN) that has been machine-learned for each document type. Specifically, the data extraction unit 24 extracts a set of character strings having a specific correlation using a learner (such as the above-mentioned GNN) that has been machine-learned for the document type identified for the document image to be processed.

[0031] The data extraction unit 24 extracts a set of character strings having a specific correlation from a document image without using template data that indicates the position of a character string in the set of character strings, which means that the user does not need to prepare template data.

[0032] The extracted data verification unit 25 uses the document database 12 to verify the validity of the set of character strings extracted from the document image to be processed, based on the set of character strings for the document image similar to the document image to be processed.

[0033] For example, the extracted data verification unit 25 verifies whether or not the set of character strings extracted from the document image to be processed includes all of the sets of character strings that should be extracted from the document image to be processed, based on the sets of character strings for the similar document images. In other words, it verifies whether or not there are any sets of character strings that have not been extracted.

[0034] In this embodiment, the extracted data verification unit 25 automatically corrects a set of character strings that is determined to be invalid from a set of character strings extracted from a document image to be processed with a character string selected from a plurality of character strings based on the set of character strings and position data for a similar document image.

[0035] Specifically, for example, if a string set is a pair of an attribute label and an attribute value, if a string set having the same attribute label as an attribute label in the string set for a similar document image is not extracted from the document image to be processed, it is determined that a string set including that attribute label has not been extracted. In this case, if the attribute label is found in the strings extracted from the document image to be processed, an attribute value paired with the attribute label in the document image to be processed is selected from the strings extracted from the document image to be processed based on, for example, position data of the attribute label and attribute value in the string set for the similar document image (i.e., the relative position of the attribute value with respect to the attribute label), and the pair of the attribute label and its attribute value is added to the string set for the document image to be processed.

[0036] Furthermore, for example, if multiple overlapping sets of character strings are extracted from a document image to be processed, the most highly related set of overlapping character strings may be retained and the remaining sets of character strings may be deleted. Here, this relevance may be, for example, the degree of connection of the GNN.

[0037] In this embodiment, similar document images are identified from document images in the document database 12 that have the same document type as the document image to be processed, and the extracted data verification unit 25 verifies the validity of the set of character strings extracted from the document image to be processed based on the set of character strings for the similar document images in the document database 12.

[0038] The extracted data manual correction unit 26 (a) presents the extracted set of character strings to the user on the display device 4, and (b) when a user operation indicating an instruction to correct a set of character strings that the user has determined to be invalid is detected on the input device 5, corrects the set of character strings in accordance with the user operation. For example, the document image to be processed and the extracted set of character strings are displayed on the display device 4, and the user visually checks the document image and the set of character strings to see if there are any errors in the extracted set of character strings and if there are any sets of character strings that have not been extracted, and when a user operation to correct the set of character strings that has an error or to add a set of character strings that has not been extracted is detected on the input device 5, the set of character strings that has an error or the set of character strings that has not been extracted is corrected or the set of character strings that has not been extracted is added in accordance with the user operation.

[0039] For each acquired document image, the document data management unit 27 stores at least a set of character strings extracted from the document image in the document database 12. In this embodiment, the document data management unit 27 stores, for each document image, a set of character strings, position data, a document vector, and a document type in the document database 12.

[0040] When the set of character strings is corrected by the extracted data verification unit 25, the document data management unit 27 stores the automatically corrected set of character strings as a set of character strings extracted from the document image in the document database 12. When the set of character strings is corrected by the extracted data manual correction unit 26 in accordance with a user operation, the document data management unit 27 stores the set of character strings corrected in accordance with the user operation in the document database 12 as a set of character strings extracted from the document image.

[0041] The data output unit 28 stores the document type, character strings, and associated data for each document image in a predetermined data format in the storage device 1, or transmits them via the communication device 2. Note that the character strings and associated data may be array data that individually indicates a set (e.g., a pair) of character strings (text data).

[0042] The machine learning processing unit 29 executes machine learning processing for performing machine learning on a learner such as the above-mentioned GNN. Note that the machine learning processing unit 29 is not essential and may be provided as needed, and if the machine learning on the above-mentioned learner has been completed, the machine learning processing unit 29 may not be provided.

[0043] Next, the operation of the document image processing system according to this embodiment will be described.

[0044] When the document image acquisition unit 21 acquires a document image to be processed, the document image processing unit 22 generates character string data (character string, character string position data, and document vector) of the document image. Fig. 3 is a diagram showing an example of a document image. The document image shown in Fig. 3 is a document image whose document type is INVOICE, and is one of multiple document types (document images) submitted together as customs documents.

[0045] Then, as described above, the document type determination unit 23 determines the document type of the document image to be processed, and the data extraction unit 24 uses, for example, the GNN of that document type to extract string sets having specific relationships (for example, bonds in the GNN) based on the string data, and generates relationship data indicating the string sets (for each string set, a data array of strings (or string vectors) that are elements of the string set, etc.). For example, as shown in Figure 4, strings 61 and 62 (bounding boxes 61a and 62a) indicated by dashed lines are extracted, and bond 63 indicated by a solid line is identified. Here, strings 61 and 62 connected by bond 63 are detected as a string set (here, a pair).

[0046] Thereafter, the extracted data verification unit 25 verifies the validity of the set of character strings extracted from the document image to be processed (here, the above-mentioned relevance data) based on the set of character strings for the similar document image to the document image to be processed, as described above, and corrects the set of character strings for the document image to be processed, as necessary. Note that if the similar document image is not in the document database 12, the extracted data verification unit 25 does not perform this verification.

[0047] Furthermore, the extracted data manual correction unit 26 then presents the set of character strings to the user on the display device 4, and when a user operation indicating a correction instruction for the set of character strings that the user has determined to be invalid is detected on the input device 5, the extracted data manual correction unit 26 corrects the set of character strings in accordance with the user operation.

[0048] Then, the document data management unit 27 associates the character string data, document vector, and document type of the document image to be processed (corrected automatically and / or manually as necessary) with each other and registers them in the document database 12.

[0049] In this way, character string data, document vectors, and document types are registered in the document database 12 for each document image, and are used to determine the character type of document images to be subsequently processed, automatically verify and correct extracted data, and so on.

[0050] Furthermore, the data output unit 28 outputs the document type, a set of character strings (here, character strings and relation data), and the like for the document image to be processed.

[0051] As described above, according to the first embodiment, the document image processing unit 22 extracts multiple character strings included in a document image through character recognition processing and identifies the positions of each of the multiple character strings. The data extraction unit 24 extracts a set of character strings having a specific correlation from the multiple character strings. The document data management unit 27 stores, for each document image, the set of character strings extracted from the document image in at least the document database 12. The extracted data verification unit 25 uses the document database to verify the validity of the set of character strings extracted from the document image to be processed, based on the set of character strings for document images similar to the document image to be processed.

[0052] As a result, without using template data for document images of various layouts, the extracted data verification unit 25 can identify sets of character strings that should be extracted but were not properly extracted, thereby reducing extraction errors for sets of character strings that have a specific correlation among multiple character strings in a document image.

[0053] Furthermore, even if, for example, irregular document images of multiple document types are mixed, the document types of the document images are identified in order, and a set of character strings having a specific association in each document image is appropriately extracted.

[0054] Embodiment 2

[0055] FIG. 5 is a diagram showing an example of an operation screen for assisting the user in manual correction by the extracted data manual correction unit 26 in the second embodiment.

[0056] In the second embodiment, the specific association is an association between an attribute label and an attribute value, and the set of character strings includes character strings of the attribute labels and character strings of the attribute values.

[0057] In the second embodiment, as shown in Fig. 5, for example, on one operation screen 101, the extracted data manual correction unit 26 (a) displays on the display device 4 a document image list 111 and a verification result key 112 of a document type key for each document type, and (b) displays on the display device 4 a document image 121 selected from the document image list 111, and (c) displays on the display device 4 a list 122 of pairs of attribute label character strings and attribute value character strings extracted from the document image 121 by the data extraction unit 24. While such an operation screen 101 is displayed, the extracted data manual correction unit 26 (a) identifies pairs 131 specified by a user operation (click, tap, etc.) in the above-mentioned pair list 122, and (b) identifies image positions 132 (here, area specification) specified by a user operation in the displayed document image 121. Then, the extracted data manual correction unit 26 (a) identifies a character string corresponding to the image position 132 based on the positions of the multiple character strings already identified by the document image processing unit 22 as described above, and (b) updates and displays the character string of the attribute value in the identified pair 131 with the identified character string. In this way, the extracted data manual correction unit 26 corrects the set of character strings in accordance with user operations. Note that these user operations are detected by the input device 5.

[0058] In other words, such an operation screen 101 allows the user to visually check the extraction results by the data extraction unit 24, and if there is an error in the extraction results, the user can correct the incorrect attribute value by specifying the image position 132 on the GUI as described above (without entering text).

[0059] The other configurations and operations of the document image processing system according to the second embodiment are the same as those of the first embodiment, and therefore the description thereof will be omitted.

[0060] Embodiment 3

[0061] FIG. 6 is a diagram showing an example of an operation screen for assisting the user in manual correction by the extracted data manual correction unit 26 in the third embodiment.

[0062] In the third embodiment, the specific association is an association between an attribute label and an attribute value for a certain attribute item, and the set of character strings includes character strings of the attribute label and character strings of the attribute value.

[0063] In the third embodiment, the document type determination unit 23 determines the document type of each document image in a set of document images, each of which has a plurality of document types. For example, this set of document images is a set of document images to be submitted to customs. Such a set of documents to be submitted to customs includes documents of document types such as an invoice, a packing list, a bill of lading, an arrival notice, and a shipping label.

[0064] In the third embodiment, the document image acquisition unit 21 acquires such a set of multiple document images, and the document image processing unit 22 extracts multiple character strings contained in each document image of such a set of multiple document images through character recognition processing, and identifies the position of each of the multiple character strings.

[0065] In the third embodiment, the extracted data manual correction unit 26 (a) identifies, for each attribute item, a character string of an attribute value corresponding to the attribute item contained in each of the above-mentioned set of multiple document images, (b) determines whether or not the multiple attribute value character strings thus identified corresponding to the multiple document images match each other, and (c) displays the determination result of whether or not the multiple attribute value character strings match each other on the display device 4.

[0066] In this case, a single attribute label may be associated with a certain attribute item, or one or more attribute labels may be associated with a certain attribute item in advance and stored in the storage device 1 as setting data. By referring to the setting data, if any of the associated attribute labels is detected in the set of document images, the attribute value string corresponding to the detected attribute label is used as the attribute value string corresponding to the attribute item, and the above-mentioned match / mismatch determination of the attribute value strings for the attribute item among the set of document images may be performed. For example, the attribute item "Exporter" in FIG. 6 is associated with attribute labels such as "BILL TO," "Buyer," "Buyer:," and "CONSIGNEE." If any of these attribute labels is detected in a document image, the attribute value string corresponding to the detected attribute label is used as the attribute value string for "Exporter" in the document image.

[0067] Specifically, as shown in FIG. 6, the extracted data manual correction unit 26 displays on the display device 4 on the operation screen 101 lists 211, 221 of character strings of multiple attribute values identified for each attribute item 201a, 201b, and also displays the above-mentioned judgment results 212, 222 on the display device 4.

[0068] On the operation screen 101, each attribute value string in the attribute value string lists 211, 221 is represented as a soft key, and the extracted data manual correction unit 26 identifies a document image corresponding to an attribute value string (soft key) specified by a user operation in the string lists 211, 221. Then, in response to the user operation on the attribute value string (soft key) (by switching the screen), the extracted data manual correction unit 26, for the specified document image, (a) displays a list of pairs of attribute label strings and attribute value strings on the display device 4, (b) specifies the pair specified by the user operation in the list of pairs, (e) specifies an image position specified by the user operation in the displayed document image, (f) specifies a string corresponding to the image position based on the positions of the multiple strings specified in the document image by the document image processing unit 22, and (e) updates the attribute value string in the above-mentioned pair with the specified string, as described in the second embodiment.

[0069] In other words, if the judgment results 212, 222 are inconsistent, the user can operate the character strings 211, 221 of the attribute values of the document image to be edited to display the operation screen 101 (Figure 5) for that document image as described in embodiment 2, and can edit the character strings 211, 221 on that operation screen.

[0070] The other configurations and operations of the document image processing system according to the third embodiment are the same as those of the first or second embodiment, and therefore the description thereof will be omitted.

[0071] It should be noted that various changes and modifications to the above-described embodiments will be apparent to those skilled in the art. Such changes and modifications may be made without departing from the spirit and scope of the subject matter and without diminishing its intended advantages. In other words, it is intended that such changes and modifications be included within the scope of the claims.

[0072] For example, in the above embodiment, a document image of a customs document is exemplified as a document image, but the document image is not limited to this. [Industrial Applicability]

[0073] The present invention is applicable to, for example, recognition processing of document images with various layouts. [Explanation of symbols]

[0074] 11 Document image processing program 12 Document Database 22 Document image processing unit 23 Document type determination unit 24 Data Extraction Section 25 Extracted Data Verification Unit 26 Manual correction of extracted data

Claims

1. a document image processing unit that extracts a plurality of character strings included in a document image by character recognition processing and identifies the positions of the plurality of character strings; a data extraction unit that extracts a set of character strings having a specific correlation from the plurality of character strings; a document data management unit that stores, for each of the document images, the set of character strings extracted from the document image in at least a document database; an extracted data verification unit that verifies the validity of the set of character strings extracted from the document image to be processed based on the set of character strings for document images similar to the document image to be processed, using the document database; A document image processing system comprising:

2. the document image processing unit generates a document vector for the document image; the document data management unit accumulates the document vector together with the set of character strings in the document database for each of the document images; the similar document images are detected in the document database based on similarity to the document vector of the document image to be processed; 2. The document image processing system according to claim 1, wherein:

3. 2. The document image processing system according to claim 1, wherein the extracted data verification unit verifies, based on the character string sets for the similar document images, whether or not the character string sets extracted from the document image to be processed include all of the character string sets that should be extracted from the document image to be processed.

4. the document data management unit accumulates, for each document image, the set of character strings together with position data of character strings in the set of character strings in the document database; the extracted data verification unit automatically corrects a set of character strings determined to be invalid from the set of character strings extracted from the document image to be processed with a character string selected from the plurality of character strings based on the set of character strings and the position data for the similar document image; the document data management unit stores the automatically corrected set of character strings in the document database as the set of character strings extracted from the document image; 2. The document image processing system according to claim 1, wherein:

5. A display device; An input device; and an extracted data manual correction unit, the extracted data manual correction unit (a) presents the extracted set of character strings to the user on the display device; and (b) when a user operation indicating an instruction to correct the set of character strings determined by the user to be invalid is detected on the input device, corrects the set of character strings in accordance with the user operation; the document data management unit stores the set of character strings corrected in accordance with the user operation in the document database as the set of character strings extracted from the document image; 2. The document image processing system according to claim 1, wherein:

6. the specific relationship is a relationship between an attribute label and an attribute value; the set of strings includes strings of the attribute labels and strings of the attribute values; The extracted data manual correction unit (a) displays the document image on the display device, and also displays a list of pairs of character strings of the attribute labels and character strings of the attribute values on the display device; (b) identifies the pairs specified by the user through a user operation in the list of pairs; (c) identifies an image position specified by the user through a user operation in the displayed document image; (d) identifies a character string corresponding to the image position based on the positions of the plurality of character strings specified by the document image processing unit; and (e) updates the attribute value character string in the pair with the identified character string.

6. The document image processing system according to claim 5, wherein:

7. a document type determination unit that determines the document type of each document image in a set of a plurality of document images, each having a plurality of document types; the specific relationship is a relationship between an attribute label and an attribute value for a certain attribute item; the set of strings includes strings of the attribute labels and strings of the attribute values; the document image processing unit extracts a plurality of character strings included in each of the set of a plurality of document images by character recognition processing, and identifies the positions of each of the plurality of character strings; the extracted data manual correction unit (a) specifies, for each attribute item, a character string of the attribute value that corresponds to the attribute item included in each of the plurality of document images; (b) determines whether or not the character strings of the attribute values specified corresponding to the plurality of document images match each other; and (c) displays, on the display device, a determination result of whether or not the character strings of the plurality of attribute values match each other.

6. The document image processing system according to claim 5, wherein:

8. The extracted data manual correction unit (a) displays on the display device a list of character strings of the plurality of attribute values identified for each of the attribute items, (b) identifies a document image corresponding to a character string of an attribute value designated by a user operation by the user in the list of character strings of the plurality of attribute values, (c) displays on the display device the identified document image, and also displays on the display device a list of pairs of character strings of the attribute labels and character strings of the attribute values for the identified document image, (d) identifies the pair designated by a user operation by the user in the list of pairs, (e) identifies an image position designated by a user operation by the user in the displayed document image, (f) identifies a character string corresponding to the image position based on the positions of the plurality of character strings identified by the document image processing unit, and (e) updates the attribute value character string in the pair with the identified character string.

8. The document image processing system according to claim 7, wherein:

9. 2. The document image processing system according to claim 1, wherein the data extraction unit extracts a set of character strings having the specific relationship using a machine-learned graph neural network in which the character string vectors of each of the plurality of character strings are defined as nodes and the relationship is defined as a connection between the nodes.

10. a document type determination unit for determining a document type of the document image; the document data management unit stores the document type together with the set of character strings and the document vector for each of the document images in the document database; the document type determination unit (a) identifies a document type of the similar document image in the document database, and (b) sets the identified document type as the document type of the document image to be processed; the data extraction unit extracts a set of character strings having the specific association using a learning device that has been machine-learned for each of the document types; the extracted data verification unit verifies the validity of the set of character strings extracted from the document image to be processed based on the set of character strings for the similar document image; 3. The document image processing system according to claim 2, wherein:

11. 11. The document image processing system according to claim 10, wherein the document type determination unit (a) derives the center of gravity of document vectors registered in the document database for each document type in the document database, (b) determines that the similar document image exists in the document database if the distance from the document vector of the document image to be processed to any of the centers of gravity is less than a predetermined threshold, detects the similar document image in the document database, and sets the document type of the similar document image to the document type of the document image to be processed, and (c) determines that the similar document image does not exist in the document database if the distance from the document vector of the document image to be processed to any of the centers of gravity is not less than the predetermined threshold, and identifies the document type based on the document vector of the document image to be processed using a machine-learned classifier.

12. 12. The document image processing system according to claim 1, wherein the data extraction unit extracts the set of character strings having the specific correlation without using template data indicating the position of a character string in the set of character strings.

13. extracting a plurality of character strings included in a document image by character recognition processing and identifying the positions of the plurality of character strings; extracting a set of character strings having a specific correlation from the plurality of character strings; storing, for each of the document images, the set of character strings extracted from the document image in at least a document database; using the document database to verify the validity of the set of character strings extracted from the document image to be processed based on the set of character strings for document images similar to the document image to be processed; A document image processing method comprising:

14. Computer, a document image processing unit that extracts a plurality of character strings included in a document image by character recognition processing and identifies the positions of the plurality of character strings; a data extraction unit that extracts a set of character strings having a specific correlation from the plurality of character strings; a document data management unit that stores, for each of the document images, the set of character strings extracted from the document image in at least a document database; an extracted data verification unit that verifies the validity of the set of character strings extracted from the document image to be processed based on the set of character strings for document images similar to the document image to be processed, using the document database; A document image processing program characterized by causing the program to function as:

Images