Document Image Processing System

Abstract

This reduces errors in extracting character strings from document images. [Solution] A local OCR processing unit 22 performs character recognition processing on a document image to generate text data of a character string described in the document image. An LLM-OCR processing unit 23 inputs the document image along with a prompt to a large-scale language model, and acquires text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt. An integration processing unit 24 determines the validity of the text data generated by the LLM-OCR processing unit 23 based on a predetermined similarity between the text data generated by the local OCR processing unit 22 and the text data generated by the LLM-OCR processing unit 23, and outputs the text data generated by the LLM-OCR processing unit 23 if it determines that the text data generated by the LLM-OCR processing unit 23 is valid.

Description

[Technical Field]

[0001] The present invention relates to a document image processing system. [Background technology]

[0002] One system (a) extracts a set of character images from a document image and generates text data (character strings) from the set of character images, (b) generates a feature vector corresponding to the text data of the set of character images, and (c) detects attribute label candidates and attribute value candidates for a specific attribute from the text data, and sets these pairs as pair candidates. The attribute label candidates are detected based on the feature vectors of the text data of the set of character images, and pairs of attribute labels and attribute values ​​are identified based on, for example, the degree of association between them (see, for example, Patent Document 1). This allows the attribute value of a certain attribute to be accurately identified without using template data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-178723 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above system, pairs of attribute labels and corresponding attribute values ​​are extracted from multiple character strings in a document image based on the relevance. Therefore, if the relevance is accurately derived, inappropriate pairs will not be extracted. However, when extracting pairs of attribute labels and attribute values ​​from document images with various (arbitrary) layouts, there is a possibility that pairs will be extracted incorrectly (such as pairs that should be extracted not being extracted).

[0005] The present invention has been made in view of the above-mentioned problems, and has as its object to provide a document image processing system or the like that reduces errors in extracting character strings from within a document image. [Means for solving the problem]

[0006] The document image processing system of the present invention comprises a local OCR processing unit that performs character recognition processing on a document image and generates text data of a character string described in the document image; an LLM-OCR processing unit that inputs the document image along with a prompt to a large-scale language model and acquires text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt; and an integration processing unit that determines the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data acquired by the LLM-OCR processing unit, and outputs the text data acquired by the LLM-OCR processing unit if it is determined that the text data acquired by the LLM-OCR processing unit is valid. The system further includes any one of the following configurations (A) to (G): (A) the prompt includes text data generated by the local OCR processing unit; (B) when the integration processing unit determines that the text data acquired by the LLM-OCR processing unit is invalid, the integration processing unit adds the text data generated by the local OCR processing unit to the prompt, and the LLM-OCR processing unit inputs the added prompt and the document image to the large-scale language model, and acquires text data of a character string identified by the large-scale language model in accordance with the prompt from the large-scale language model; (C) the local OCR processing unit identifies, together with the text data, a position of a character string in the document image corresponding to the text data; and the integration processing unit identifies a position of a character string in the document image corresponding to the text data acquired by the LLM-OCR processing unit, based on the text data identified by the local OCR processing unit and the position of the character string; (D) the LLM-OCR processing unit selects a prompt corresponding to the document type of the document image from prompts of multiple document types, inputs the selected prompt and the document image to the large-scale language model, and acquires text data of a character string identified by the large-scale language model in accordance with the selected prompt from the large-scale language model. (E) The LLM-OCR processing unit generates a document vector for the above-mentioned document image, selects a prompt corresponding to the generated document vector from among prompts associated with multiple document vectors, inputs the selected prompt and the above-mentioned document image into a large-scale language model, and obtains, from the large-scale language model, text data of the character string identified by the large-scale language model for the item specified by the selected prompt.(F) The local OCR processing unit generates text data of character strings described in the above-mentioned document image by associating text data of character strings as labels for specific items and text data of character strings as values ​​for those items, the LLM-OCR processing unit inputs the above-mentioned document image to a large-scale language model along with a prompt specifying the above-mentioned specific items, and obtains from the large-scale language model text data of character strings identified by the large-scale language model for the specific items specified by the prompt, and the integration processing unit determines the validity of the text data obtained by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data obtained by the LLM-OCR processing unit for the specific items. (G) The LLM-OCR processing unit inputs the above-mentioned document image along with the above-mentioned prompt to a plurality of different large-scale language models, and acquires text data of the character string identified by the large-scale language model from each of the plurality of large-scale language models in accordance with the prompt. The integration processing unit (a) acquires text data of the character string in the above-mentioned document image by synthesizing the text data acquired from each of the plurality of large-scale language models, and (b) determines the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data obtained by synthesizing the text data acquired from each of the above-mentioned plurality of large-scale language models, and outputs the text data acquired by the LLM-OCR processing unit if it is determined that the text data acquired by the LLM-OCR processing unit is valid.

[0007] The document image processing method according to the present invention comprises: On your computer, Perform character recognition processing on document images, On your computer, a local OCR processing step for generating text data of character strings described in the document image; On your computer, Inputting document images with prompts to a large-scale language model On your computer, an LLM-OCR processing step of obtaining text data of the character string identified by the large-scale language model according to the prompt from the large-scale language model; On your computer, The validity of the text data acquired in the LLM-OCR processing step is determined based on a predetermined similarity between the text data generated in the local OCR processing step and the text data acquired in the LLM-OCR processing step, and if it is determined that the text data acquired in the LLM-OCR processing step is valid, On your computer, and an integration processing step for outputting the text data acquired in the LLM-OCR processing step. The method further includes any one of the following configurations (A) to (G): (A) the prompt includes text data generated in the local OCR processing step; (B) in the integration processing step, if it is determined that the text data acquired in the LLM-OCR processing step is invalid, the text data generated in the local OCR processing step is added to the prompt; and in the LLM-OCR processing step, the prompt after addition and the document image are input to a large-scale language model, and text data of a character string identified by the large-scale language model in accordance with the prompt is acquired from the large-scale language model; (C) in the local OCR processing step, the position of the character string in a document image corresponding to the text data is identified together with the text data; and in the integration processing step, the position of the character string in a document image corresponding to the text data acquired in the LLM-OCR processing step is identified based on the text data and the position of the character string identified in the local OCR processing step. (D) In ​​the LLM-OCR processing step, a prompt corresponding to the document type of the above-mentioned document image is selected from multiple document type prompts, the selected prompt and the above-mentioned document image are input to a large-scale language model, and text data of a character string identified by the large-scale language model in accordance with the selected prompt is obtained from the large-scale language model. (E) In the LLM-OCR processing step, a document vector of the above-mentioned document image is generated, a prompt corresponding to the generated document vector is selected from prompts associated with the multiple document vectors, the selected prompt and the above-mentioned document image are input to the large-scale language model, and text data of a character string identified by the large-scale language model for the item specified by the selected prompt is obtained from the large-scale language model.(F) In the local OCR processing step, text data of character strings described in the document image is generated by associating text data of character strings as labels for specific items with text data of character strings as values ​​for those items; in the LLM-OCR processing step, the document image described above is input to a large-scale language model along with a prompt specifying a specific item, and text data of character strings identified by the large-scale language model for the specific item specified by the prompt is obtained from the large-scale language model; and in the integration processing step, the validity of the text data obtained in the LLM-OCR processing step is determined based on a predetermined similarity between the text data generated in the local OCR processing step and the text data obtained in the LLM-OCR processing step for that specific item. (G) In the LLM-OCR processing step, the document image is input to a plurality of different large-scale language models along with the prompt, and text data of the character string identified by the large-scale language model is obtained from each of the plurality of large-scale language models in accordance with the prompt. In the integration processing step, (a) the text data of the character string in the document image is obtained by synthesizing the text data obtained from the plurality of large-scale language models, and (b) the validity of the text data obtained in the LLM-OCR processing step is determined based on a predetermined similarity between the text data generated in the local OCR processing step and the text data obtained by synthesizing the text data obtained from the plurality of large-scale language models, and if it is determined that the text data obtained in the LLM-OCR processing step is valid, the text data obtained in the LLM-OCR processing step is output.

[0008] A document image processing program according to the present invention causes a computer to function as the above-described local OCR processing unit, the above-described LLM-OCR processing unit, and the above-described integrated processing unit. [Effects of the Invention]

[0009] According to the present invention, a document image processing system that reduces errors in extracting character strings from within a document image can be obtained.

[0010] The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing the configuration of a document image processing system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram showing an example of a prompt. [Figure 3] FIG. 3 is a flowchart illustrating the operation of the document image processing system according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating prompt data according to the second embodiment. [Figure 5] FIG. 5 is a block diagram showing a configuration of a document image processing system according to the third embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0013] Embodiment 1

[0014] Fig. 1 is a block diagram showing the configuration of a document image processing system according to an embodiment of the present invention. The document image processing system 1 shown in Fig. 1 is configured with one information processing device (personal computer, server, etc.), but the processing units described below may be distributed among multiple information processing devices that can communicate data with each other. Furthermore, such multiple information processing devices may include a GPU (Graphics Processing Unit) that processes specific operations in parallel.

[0015] The document image processing system 1 shown in FIG. 1 includes a storage device 11, a communication device 12, and a processing device 13.

[0016] The storage device 11 is a non-volatile storage device such as a flash memory or a hard disk, and stores various data and programs. In this example, the storage device 11 stores a document image processing program 11a and prompt data 11b. In addition, system setting data (such as coefficient setting values ​​for a learning device such as a neural network used in the local OCR processing unit 22, which will be described later) is stored in the storage device 11 as needed.

[0017] The document image processing program 11a may be stored in a portable computer-readable recording medium such as a CD (Compact Disk). In this case, for example, the document image processing program 11a is installed from the recording medium into the storage device 11. The document image processing program 11a may be a single program or a collection of multiple programs.

[0018] The prompt data 11 b includes one or more prompts to send to the LLM server 3 .

[0019] The communication device 12 is a device capable of data communication, such as a network interface or modem, and performs data communication with other devices (such as the LLM server 3 and the user terminal device 4) via the network 2.

[0020] The LLM server 3 is a server that receives input data including prompts, executes processing specified by the prompts in the input data using a large-scale language model (LLM), and transmits the processing results as output data. Examples of LLMs that can be used include GPT-4 and Claude3.5sonnet. The LLM server 3 may be included in the document image processing system 1.

[0021] The user terminal device 4 is a device such as a personal computer or smartphone, which transmits a document image specified by the user to the document image processing system 1 in accordance with user operation, and receives specific text data (i.e., the result of character recognition processing (OCR)) obtained from the document image by the document image processing system 1 from the document image processing system 1. The transmitted document image may be raster image data obtained by a scanner or camera, or may be PDF data. The received text data may be displayed by the user terminal device 4 or saved in the user terminal device 4.

[0022] The arithmetic processing device 13 is a computer equipped with a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and operates as various processing units by loading programs from the ROM, the storage device 11, etc. into the RAM and executing them on the CPU.

[0023] Here, by executing the document image processing program 11a, the arithmetic processing device 13 operates as a front-end processing unit 21, a local OCR processing unit 22, an LLM-OCR processing unit 23, and an integration processing unit 24.

[0024] The front-end processing unit 21 uses the communication device 12 to communicate data with the user terminal device 4, (a) receive document data from the user terminal device 4, and convert the document data into a document image (raster image) as necessary, and (b) transmit the results of character recognition processing on the document image to the user terminal device 4.

[0025] The local OCR processor 22 performs character recognition processing on the document image described above, and generates text data of the character strings written in the document image.

[0026] Here, the local OCR processor 22 performs character recognition processing without using template data, and generates text data of the character strings written in the document image.

[0027] The local OCR processing unit 22 may generate text data of character strings described in a document image by associating text data of character strings as labels for specific items and text data of character strings as values ​​for those items. In this case, the local OCR processing unit 22 may generate text data of character strings described in a document image, for example, using a method described in Japanese Patent Application Laid-Open No. 2022-178723. Specifically, in this case, (a) a set of character images within the document image is extracted and text data of the set of character images is generated, (b) a feature vector corresponding to the text data of the set of character images is generated by embedding, (c) attribute label candidates and attribute value candidates for a specific attribute are detected from the text data, and these pairs are set as pair candidates. The attribute label candidates are detected based on the feature vectors of the text data of the set of character images, and (d) pairs of attribute labels (text data of labels for specific items described below) and attribute values ​​(text data of values ​​for the specific items) are identified based on the feature vectors of the attribute label candidates and the feature vectors of the attribute value candidates.

[0028] The local OCR processor 22 may perform character recognition processing using template data. The template data is data that indicates the position where the value of each item is written in the document image.

[0029] The LLM-OCR processing unit 23 uses the communication device 12 to input a document image to the large-scale language model (LLM server 3) along with a prompt, and obtains text data of a character string identified by the large-scale language model (LLM server 3) from the large-scale language model (LLM server 3) in accordance with the prompt.

[0030] In this embodiment, in particular, the LLM-OCR processing unit 23 uses the communication device 12 to input a document image to the LLM server 3 along with a prompt specifying the above-mentioned specific item, and obtains from the LLM server 3 text data of the character string specified by the LLM server 3 for the specific item specified by the prompt.

[0031] Fig. 2 is a diagram showing an example of a prompt. The prompt shown in Fig. 2 is an example of a prompt for extracting values ​​of the items "quote number," "quote date," "supplier name," "quote expiration date," and "details" from a document image of a "quotation" (i.e., character strings indicating specific values ​​for the items, rather than character strings for the items themselves). In the prompt, for example, an output data format for values ​​and extraction rules may be additionally set for the items, such as "quote date" in Fig. 2.

[0032] For example, the output data format of the LLM server 3 (JSON, CSV, XML, etc.) may be specified in the prompt. In this case, the output of the LLM server 3 includes a pair of the item name (label) and the value of that item for each item. For example, in the case of JSON format, the output of the LLM server 3 is written as {"Quote number":"11111", "Quote date":"20241010", "Supplier name":"XXX Co., Ltd.", ...}.

[0033] In this embodiment, the LLM-OCR processing unit 23 selects a prompt corresponding to the document type of the document image from the prompts for multiple document types included in the prompt data 11b, inputs the selected prompt and the document image to the LLM server 3, and obtains from the LLM server 3 text data of the character string identified by the LLM server 3 in accordance with the selected prompt.

[0034] The integration processing unit 24 determines the validity of the text data acquired by the LLM-OCR processing unit 23 based on a predetermined similarity between the text data generated by the local OCR processing unit 22 and the text data acquired by the LLM-OCR processing unit 23, and if it determines that the text data acquired by the LLM-OCR processing unit 23 is valid, it outputs the text data acquired by the LLM-OCR processing unit 23. Specifically, if the similarity exceeds a predetermined threshold, the text data is determined to be valid, and if not, the text data is determined to be invalid.

[0035] Here, for example, the above-mentioned similarity is defined as the proportion of words contained in the text data generated by the local OCR processing unit 22 that match words contained in the text data generated by the LLM-OCR processing unit 23 (i.e., the ratio between the total number of words contained in the text data generated by the local OCR processing unit 22 and the number of words that match words contained in the text data generated by the LLM-OCR processing unit 23).

[0036] Alternatively, the similarity may be the cosine similarity of the feature vectors of the text data obtained by embedding.

[0037] Furthermore, when the local OCR processing unit 22 generates text data for each character string through character recognition processing as described above, it may identify attributes of the character string (item (label), value for the item, etc.) and associate them with the text data, and the similarity described above may be the similarity for text data of character strings with a specific attribute (e.g., item label, etc.) (i.e., in this case, character strings with other attributes are excluded from the derivation of the similarity.) That is, for example, the integration processing unit 24 determines the validity of the text data acquired by the LLM-OCR processing unit 23 based on the similarity described above between the text data generated by the local OCR processing unit 22 and the text data acquired by the LLM-OCR processing unit 23 for the specific item.

[0038] Furthermore, in this embodiment, if the integration processing unit 24 determines that the text data acquired by the LLM-OCR processing unit 23 is invalid, it adds the text data (here, a pair of an item label and an item value) generated by the local OCR processing unit 22 to the prompt, and then the LLM-OCR processing unit 23 inputs the added prompt and document image to the large-scale language model (LLM server 3), and acquires from the large-scale language model (LLM server 3) the text data of the character string identified by the large-scale language model in accordance with the prompt.

[0039] Furthermore, in this embodiment, the local OCR processing unit 22 identifies the position in the document image of the character string corresponding to the text data (specifically, a bounding box indicating the position and size of the character string) together with the text data through character recognition processing, and the integration processing unit 24 identifies the position of the character string in the document image corresponding to the text data acquired by the LLM-OCR processing unit 23 based on the text data and the position of the character string identified by the local OCR processing unit 22.

[0040] Next, a description will be given of the operation of the document image processing system according to Embodiment 1. Fig. 3 is a flowchart illustrating the operation of the document image processing system according to Embodiment 1.

[0041] The front-end processing unit 21 receives an OCR request and document data sent from the user terminal device 4 (step S1). At this time, if the document data is PDF data, the front-end processing unit 21 converts the document data into raster image data for each page. At this time, if the PDF data includes images of multiple pages, the front-end processing unit 21 identifies an image range of a specific document from the images of the multiple pages, and sets the images in that image range (for example, page images of multiple pages) as the document image to be subjected to OCR processing.

[0042] Next, the local OCR processor 22 performs character recognition processing on the document image, and generates text data of the character strings written in the document image (step S2).

[0043] In addition, the LLM-OCR processing unit 23 selects a prompt in the prompt data 11b (step S3), uses the communication device 12 to send the prompt and the above-mentioned document image to the LLM server 3 (step S4), and receives the processing result of the LLM server 3 (i.e., the text data that is the OCR result) from the LLM server 3 in response to them (step S5).

[0044] The process of step S2 and the processes of steps S3 to S5 may be performed in parallel.

[0045] Then, the integration processing unit 24 derives the similarity between the text data obtained by the local OCR processing unit 22 and the text data obtained by the LLM-OCR processing unit 23, and determines whether the text data obtained by the LLM-OCR processing unit 23 is valid based on the derived similarity (step S6).

[0046] If it is determined that the text data obtained by the LLM-OCR processing unit 23 is valid, the integration processing unit 24 transmits the text data as an OCR result to the user terminal device 4 (step S7).

[0047] At this time, since the OCR result from the LLM-OCR processing unit 23 does not include position information of the character string (coordinate values ​​at which the character string in the text data is written in the document image), the integration processing unit 24 may use the position information included in the OCR result from the local OCR processing unit 22 to add the position information to the OCR result from the LLM-OCR processing unit 23. Specifically, the integration processing unit 24 adds the position information of the character string in the OCR result from the local OCR processing unit 22 to the character string in the OCR result from the LLM-OCR processing unit 23 that matches the character string in the OCR result from the local OCR processing unit 22.

[0048] Based on the OCR result to which the position information has been added in this way, the integration processing unit 24 may transmit, for example, a document image with a character string superimposed at the position indicated by the position information as the OCR result to the user terminal device 4 and display it on the user terminal device 4. This makes it easier for the user to check the OCR result.

[0049] On the other hand, if it is determined that the text data obtained by the LLM-OCR processing unit 23 is invalid, the integration processing unit 24 adds the text data as the OCR result of the local OCR processing unit 22 to the above-mentioned prompt (step S8), sends the added prompt and the above-mentioned document image to the LLM server 3 (step S9), and receives the processing result of the LLM server 3 (i.e., the text data as the OCR result) from the LLM server 3 in response to them (step S10).The integration processing unit 24 then sends the text data as the OCR result to the user terminal device 4 (step S7).

[0050] In addition, the validity of the OCR result of the LLM server 3 in response to a prompt to which the OCR result of the local OCR processing unit 22 has been added may also be determined as described above, and if it is valid, the text data may be sent to the user terminal device 4 as the OCR result, and if it is not valid, a warning such as an error determination may be sent to the user terminal device 4.

[0051] As described above, according to the first embodiment, the local OCR processing unit 22 performs character recognition processing on a document image to generate text data of character strings described in the document image. The LLM-OCR processing unit 23 inputs the document image described above along with a prompt to the large-scale language model and acquires, from the large-scale language model, text data of character strings identified by the large-scale language model for an item specified by the prompt. The integration processing unit 24 determines the validity of the text data acquired by the LLM-OCR processing unit 23 based on a predetermined similarity between the text data generated by the local OCR processing unit 22 and the text data acquired by the LLM-OCR processing unit 23. If it determines that the text data acquired by the LLM-OCR processing unit 23 is valid, the integration processing unit 24 outputs the text data acquired by the LLM-OCR processing unit 23.

[0052] This reduces errors in extracting character strings from document images that occur due to hallucination of LLM, etc. In particular, it reduces errors in extracting sets of character strings that have a specific correlation among multiple character strings in a document image.

[0053] Embodiment 2

[0054] Fig. 4 is a diagram illustrating prompt data in embodiment 2. In embodiment 2, for example, as shown in Fig. 4, prompt data 11b includes prompt sets 41-i (i = 1,...,N) for each document type, and each prompt set 41-i includes a default prompt 51 and, as needed, individual prompts 52 for specific document formats.

[0055] A document vector 52a of a specific document is associated with the individual prompt 52. The document vector 52a is a vector obtained by embedding a document image of a specific document format. Note that the individual prompt 52 is a prompt that is individually generated to obtain a valid OCR result when the default prompt 51 does not obtain a valid OCR result for a document image of a specific document format, and is added together with the document vector 52a as necessary.

[0056] In embodiment 2, the LLM-OCR processing unit 23 generates a document vector of a document image by embedding, selects an individual prompt 52 corresponding to the generated document vector from the individual prompts 52 associated with multiple document vectors 52a, inputs the selected individual prompt 52 and the document image to the LLM server 3, and obtains from the LLM server 3 text data of the character string identified by the LLM server 3 for the item specified by the selected individual prompt 52.

[0057] Specifically, the LLM-OCR processor 23 selects the individual prompt 52 corresponding to the generated document vector from the individual prompts 52 associated with the multiple document vectors 52a in the prompt set 41-i of the document type of the document image. For example, the prompt shown in Figure 2 is a prompt for the document type "quote."

[0058] Specifically, an individual prompt 52 of a document vector 52a having the highest similarity to the generated document vector and exceeding a predetermined threshold is selected, and if there is no document vector 52a having a similarity to the generated document vector that exceeds a predetermined threshold, a default prompt 51 is selected, and the default prompt 51 and the document image are input to the LLM server 3.

[0059] The other configurations and operations of the document image processing system according to the second embodiment are the same as those of the first embodiment, and therefore the description thereof will be omitted.

[0060] Embodiment 3

[0061] FIG. 5 is a block diagram showing a configuration of a document image processing system according to the third embodiment of the present invention.

[0062] In embodiment 3, the LLM-OCR processing unit 23 inputs a document image along with the above-mentioned prompt to a plurality of different large-scale language models (here, LLM servers 3-1 to 3-N), and obtains text data of character strings identified by each of the plurality of large-scale language models in accordance with the prompt.

[0063] Furthermore, in the third embodiment, the integration processing unit 24 (a) acquires text data of a character string in a document image by synthesizing the text data acquired from each of the multiple large-scale language models, and (b) determines the validity of the text data acquired by the LLM-OCR processing unit 23 based on a predetermined similarity between the text data generated by the local OCR processing unit 22 and the text data obtained by synthesizing the text data acquired from each of the multiple large-scale language models. If it is determined that the text data acquired by the LLM-OCR processing unit 23 is valid, it outputs the text data acquired by the LLM-OCR processing unit 23.

[0064] In particular, in embodiment 3, the LLM-OCR processing unit 23 inputs the same prompt and document image to each of multiple different large-scale language models (here, LLM servers 3-1 to 3-N), and obtains text data of character strings identified by each of the multiple different large-scale language models (here, LLM servers 3-1 to 3-N) for the item specified by the above-mentioned prompt from the multiple different large-scale language models (here, LLM servers 3-1 to 3-N).

[0065] In this case, in the third embodiment, if a character string corresponding to a certain item is not obtained in the text data obtained from one of the multiple large-scale language models (here, LLM server 3-i) among the multiple large-scale language models (here, LLM servers 3-1 to 3-N), the integration processing unit 24 complements the character string corresponding to the item in the text data obtained from another large-scale language model (here, LLM server 3-j), thereby synthesizing the text data obtained from the multiple large-scale language models.

[0066] In addition, in this case, in the third embodiment, when multiple character strings corresponding to a certain item are obtained in the text data respectively acquired from multiple large-scale language models (here, LLM servers 3-1 to 3-N), the integration processing unit 24 determines the character string corresponding to the item by majority vote, thereby synthesizing the text data respectively acquired from the multiple large-scale language models.

[0067] The other configurations and operations of the document image processing system according to the third embodiment are the same as those of the first or second embodiment, and therefore the description thereof will be omitted.

[0068] As described above, according to the third embodiment, text data obtained by synthesizing text data obtained from multiple different large-scale language models is used, which makes it easier to extract character strings in document images more accurately.

[0069] It should be noted that various changes and modifications to the above-described embodiments will be apparent to those skilled in the art. Such changes and modifications may be made without departing from the spirit and scope of the subject matter and without diminishing its intended advantages. In other words, it is intended that such changes and modifications be included within the scope of the claims.

[0070] For example, in the above-mentioned embodiments 1 and 2, text data as the OCR result generated by the local OCR processing unit 22 may be added to the prompt initially input to the LLM server 3, and a prompt including the text data as the OCR result generated by the local OCR processing unit 22 may be input to the LLM server 3.

[0071] In addition, in the above-mentioned embodiments 1 and 2, the document image processing system 1 may generate template data for the document type of the document image based on the OCR results (extracted text data) obtained from the LLM server 3 after location information has been added.

[0072] Furthermore, in the above-mentioned first and second embodiments, the document image processing system 1 may generate training data for a learning device (such as a deep neural network) for character recognition processing that does not use template data in the local OCR processing unit 22, etc., based on the OCR results (extracted text data) obtained from the LLM server 3 after location information has been added (i.e., annotation may be performed automatically).

[0073] Furthermore, in the above-described first and second embodiments, the local OCR processor 22 identifies the position of the identified character string. Therefore, when the integration processor 24 determines that the document image contains a table, it may identify each row of the table based on the position, and for each character string contained in that row, determine whether the text data in both OCR results match, as described above, to determine the validity of the OCR result by the LLM server 3. That is, in this case, for multiple rows of the table, it is determined whether the text data in both OCR results match, row by row (along the sub-scanning direction of the document image). In this case, it may also be possible to individually add the above-described position information to the text data of the OCR result by the LLM server 3, row by row.

[0074] In addition, in the above embodiments 1 to 3, the local OCR processing unit 22 or the like may identify an area (left side, lower half, etc.) in the text image where the value of a specific item is written, and then extract the item from the identified area.

[0075] In addition, in the above-mentioned embodiment 2, a default prompt set 41-j may be included in the prompt data 11b, and if the prompt set 41-k for the document type of the document image to be OCR processed is not included in the prompt data 11b, the prompt to be input to the LLM server 3 may be selected from the default prompt set 41-j.

[0076] In the first to third embodiments, the local OCR processing unit 22 may perform the character recognition process described above using a learning device that has been machine-trained using training data for each document type. In this case, the local OCR processing unit 22 may associate a document vector corresponding to each document type with the document type in advance and store it in a storage device, and automatically identify the document type of the document image based on the document vector of the document image from the similarity between the two, and then perform the character recognition process using a learning device for the identified document type. [Industrial Applicability]

[0077] The present invention is applicable to, for example, recognition processing of document images with various layouts. [Explanation of symbols]

[0078] 1. Document image processing system 3,3-1~3-N LLM Server 11a Document image processing program 13 Processing Unit (an example of a computer) 22 Local OCR processing section 23 LLM-OCR Processing Section 24 Integrated Processing Unit

Claims

1. A local OCR processing unit that performs character recognition processing on a document image and generates text data of a character string described in the document image; an LLM-OCR processing unit that inputs the document image to a large-scale language model together with a prompt, and acquires text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt; an integration processing unit that determines the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data acquired by the LLM-OCR processing unit, and outputs the text data acquired by the LLM-OCR processing unit when it is determined that the text data acquired by the LLM-OCR processing unit is valid; The document image processing system according to claim 1, wherein the prompt includes the text data generated by the local OCR processor.

2. A local OCR processing unit that performs character recognition processing on a document image and generates text data of a character string described in the document image; an LLM-OCR processing unit that inputs the document image to a large-scale language model together with a prompt, and acquires text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt; an integration processing unit that determines the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data acquired by the LLM-OCR processing unit, and outputs the text data acquired by the LLM-OCR processing unit when it is determined that the text data acquired by the LLM-OCR processing unit is valid; the integration processing unit, when it is determined that the text data acquired by the LLM-OCR processing unit is invalid, adds the text data generated by the local OCR processing unit to the prompt; the LLM-OCR processing unit inputs the prompt and the document image after the addition to the large-scale language model, and acquires, from the large-scale language model, text data of a character string identified by the large-scale language model in accordance with the prompt; A document image processing system characterized by:

3. A local OCR processing unit that performs character recognition processing on a document image and generates text data of a character string described in the document image; an LLM-OCR processing unit that inputs the document image to a large-scale language model together with a prompt, and acquires text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt; an integration processing unit that determines the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data acquired by the LLM-OCR processing unit, and outputs the text data acquired by the LLM-OCR processing unit when it is determined that the text data acquired by the LLM-OCR processing unit is valid; the local OCR processing unit identifies the text data and the position of the character string in the document image corresponding to the text data; the integration processing unit specifies a position of a character string in the document image corresponding to the text data acquired by the LLM-OCR processing unit, based on the text data and the position of the character string specified by the local OCR processing unit; A document image processing system characterized by:

4. A local OCR processing unit that performs character recognition processing on a document image and generates text data of a character string described in the document image; an LLM-OCR processing unit that inputs the document image to a large-scale language model together with a prompt, and acquires text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt; an integration processing unit that determines the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data acquired by the LLM-OCR processing unit, and outputs the text data acquired by the LLM-OCR processing unit when it is determined that the text data acquired by the LLM-OCR processing unit is valid; a document image processing system characterized in that the LLM-OCR processing unit selects a prompt corresponding to the document type of the document image from among a plurality of document type prompts, inputs the selected prompt and the document image to the large-scale language model, and obtains, from the large-scale language model, text data of a character string identified by the large-scale language model in accordance with the selected prompt.

5. A local OCR processing unit that performs character recognition processing on a document image and generates text data of a character string described in the document image; an LLM-OCR processing unit that inputs the document image to a large-scale language model together with a prompt, and acquires text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt; an integration processing unit that determines the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data acquired by the LLM-OCR processing unit, and outputs the text data acquired by the LLM-OCR processing unit when it is determined that the text data acquired by the LLM-OCR processing unit is valid; The LLM-OCR processing unit generates a document vector for the document image, selects a prompt corresponding to the generated document vector from among prompts associated with a plurality of document vectors, inputs the selected prompt and the document image to the large-scale language model, and obtains, from the large-scale language model, text data of a character string identified by the large-scale language model for an item specified by the selected prompt.

6. A local OCR processing unit that performs character recognition processing on a document image and generates text data of a character string described in the document image; an LLM-OCR processing unit that inputs the document image to a large-scale language model together with a prompt, and acquires text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt; an integration processing unit that determines the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data acquired by the LLM-OCR processing unit, and outputs the text data acquired by the LLM-OCR processing unit when it is determined that the text data acquired by the LLM-OCR processing unit is valid; the local OCR processing unit generates text data of character strings described in the document image, the text data of character strings as labels for specific items and the text data of character strings as values ​​for the items, by associating them with each other; the LLM-OCR processing unit inputs the document image to the large-scale language model together with a prompt specifying the specific item, and acquires, from the large-scale language model, text data of a character string identified by the large-scale language model for the specific item specified by the prompt; the integration processing unit determines the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data acquired by the LLM-OCR processing unit for the specific item; A document image processing system characterized by:

7. A local OCR processing unit that performs character recognition processing on a document image and generates text data of a character string described in the document image; an LLM-OCR processing unit that inputs the document image to a large-scale language model together with a prompt, and acquires text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt; an integration processing unit that determines the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data acquired by the LLM-OCR processing unit, and outputs the text data acquired by the LLM-OCR processing unit when it is determined that the text data acquired by the LLM-OCR processing unit is valid; the LLM-OCR processing unit inputs the document image together with the prompt to a plurality of large-scale language models different from each other, and acquires text data of character strings identified by the large-scale language models from each of the plurality of large-scale language models in accordance with the prompt; the integration processing unit (a) acquires text data of a character string in the document image by synthesizing the text data acquired from the plurality of large-scale language models, respectively; (b) determines the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data obtained by synthesizing the text data acquired from the plurality of large-scale language models, and outputs the text data acquired by the LLM-OCR processing unit when it is determined that the text data acquired by the LLM-OCR processing unit is valid; A document image processing system characterized by:

8. the LLM-OCR processing unit acquires, from the plurality of large-scale language models, text data of character strings identified by the large-scale language models for the items specified by the prompt; when a character string corresponding to the item cannot be obtained in the text data acquired from one of the plurality of large-scale language models, the integration processing unit complements the text data acquired from another of the plurality of large-scale language models with a character string corresponding to the item, thereby synthesizing the text data acquired from each of the plurality of large-scale language models; 8. The document image processing system according to claim 7, wherein:

9. the LLM-OCR processing unit acquires, from the plurality of large-scale language models, text data of character strings identified by the large-scale language models for the items specified by the prompt; when a plurality of character strings corresponding to the item are obtained in the text data respectively acquired from the plurality of large-scale language models, the integration processing unit determines the character string corresponding to the item by majority vote, thereby synthesizing the text data respectively acquired from the plurality of large-scale language models; 8. The document image processing system according to claim 7, wherein:

10. A local OCR processing step in which a computer performs character recognition processing on a document image and generates text data of a character string described in the document image on the computer; an LLM-OCR processing step of inputting the document image to a large-scale language model along with a prompt, and obtaining text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt, in the computer; an integration processing step in which the computer determines the validity of the text data acquired in the LLM-OCR processing step based on a predetermined similarity between the text data generated in the local OCR processing step and the text data acquired in the LLM-OCR processing step, and when it is determined that the text data acquired in the LLM-OCR processing step is valid, the computer outputs the text data acquired in the LLM-OCR processing step; the prompt includes the text data generated in the local OCR processing step; A document image processing method comprising:

11. A local OCR processing step in which a computer performs character recognition processing on a document image and generates text data of a character string described in the document image on the computer; an LLM-OCR processing step of inputting the document image to a large-scale language model along with a prompt, and obtaining text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt, in the computer; an integration processing step in which the computer determines the validity of the text data acquired in the LLM-OCR processing step based on a predetermined similarity between the text data generated in the local OCR processing step and the text data acquired in the LLM-OCR processing step, and when it is determined that the text data acquired in the LLM-OCR processing step is valid, the computer outputs the text data acquired in the LLM-OCR processing step; In the integration processing step, if it is determined that the text data acquired in the LLM-OCR processing step is invalid, the text data generated in the local OCR processing step is added to the prompt; in the LLM-OCR processing step, the prompt after addition and the document image are input to the large-scale language model, and text data of a character string identified by the large-scale language model in accordance with the prompt is obtained from the large-scale language model; A document image processing method comprising:

12. A local OCR processing step in which a computer performs character recognition processing on a document image and generates text data of a character string described in the document image on the computer; an LLM-OCR processing step of inputting the document image to a large-scale language model along with a prompt, and obtaining text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt, in the computer; an integration processing step in which the computer determines the validity of the text data acquired in the LLM-OCR processing step based on a predetermined similarity between the text data generated in the local OCR processing step and the text data acquired in the LLM-OCR processing step, and when it is determined that the text data acquired in the LLM-OCR processing step is valid, the computer outputs the text data acquired in the LLM-OCR processing step; In the local OCR processing step, the text data and the position of the character string in the document image corresponding to the text data are identified; In the integration processing step, a position of a character string in the document image corresponding to the text data acquired in the LLM-OCR processing step is identified based on the text data and the position of the character string in the local OCR processing step; A document image processing method comprising:

13. A local OCR processing step in which a computer performs character recognition processing on a document image and generates text data of a character string described in the document image on the computer; an LLM-OCR processing step of inputting the document image to a large-scale language model along with a prompt, and obtaining text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt, in the computer; an integration processing step in which the computer determines the validity of the text data acquired in the LLM-OCR processing step based on a predetermined similarity between the text data generated in the local OCR processing step and the text data acquired in the LLM-OCR processing step, and when it is determined that the text data acquired in the LLM-OCR processing step is valid, the computer outputs the text data acquired in the LLM-OCR processing step; in the LLM-OCR processing step, selecting a prompt corresponding to the document type of the document image from among a plurality of document type prompts, inputting the selected prompt and the document image to the large-scale language model, and acquiring, from the large-scale language model, text data of a character string identified by the large-scale language model in accordance with the selected prompt; A document image processing method comprising:

14. A local OCR processing step in which a computer performs character recognition processing on a document image and generates text data of a character string described in the document image on the computer; an LLM-OCR processing step of inputting the document image to a large-scale language model along with a prompt, and obtaining text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt, in the computer; an integration processing step in which the computer determines the validity of the text data acquired in the LLM-OCR processing step based on a predetermined similarity between the text data generated in the local OCR processing step and the text data acquired in the LLM-OCR processing step, and when it is determined that the text data acquired in the LLM-OCR processing step is valid, the computer outputs the text data acquired in the LLM-OCR processing step; In the LLM-OCR processing step, a document vector of the document image is generated, a prompt corresponding to the generated document vector is selected from among prompts associated with a plurality of document vectors, the selected prompt and the document image are input to the large-scale language model, and text data of a character string identified by the large-scale language model for an item specified by the selected prompt is obtained from the large-scale language model; A document image processing method comprising:

15. A local OCR processing step in which a computer performs character recognition processing on a document image and generates text data of a character string described in the document image on the computer; an LLM-OCR processing step of inputting the document image to a large-scale language model along with a prompt, and obtaining text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt, in the computer; an integration processing step in which the computer determines the validity of the text data acquired in the LLM-OCR processing step based on a predetermined similarity between the text data generated in the local OCR processing step and the text data acquired in the LLM-OCR processing step, and when it is determined that the text data acquired in the LLM-OCR processing step is valid, the computer outputs the text data acquired in the LLM-OCR processing step; In the local OCR processing step, text data of character strings described in the document image is generated by associating text data of character strings as labels for specific items with text data of character strings as values ​​for the items, and In the LLM-OCR processing step, the document image is input to the large-scale language model together with a prompt specifying the specific item, and text data of a character string identified by the large-scale language model for the specific item specified by the prompt is obtained from the large-scale language model; in the integration processing step, determining the validity of the text data acquired in the LLM-OCR processing step based on a predetermined similarity between the text data generated in the local OCR processing step and the text data acquired in the LLM-OCR processing step for the specific item; A document image processing method comprising:

16. A local OCR processing step in which a computer performs character recognition processing on a document image and generates text data of a character string described in the document image on the computer; an LLM-OCR processing step of inputting the document image to a large-scale language model along with a prompt, and obtaining text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt, in the computer; an integration processing step in which the computer determines the validity of the text data acquired in the LLM-OCR processing step based on a predetermined similarity between the text data generated in the local OCR processing step and the text data acquired in the LLM-OCR processing step, and when it is determined that the text data acquired in the LLM-OCR processing step is valid, the computer outputs the text data acquired in the LLM-OCR processing step; In the LLM-OCR processing step, the document image is input to a plurality of large-scale language models different from each other together with the prompt, and text data of character strings identified by the large-scale language models is acquired from each of the plurality of large-scale language models in accordance with the prompt; In the integration processing step, (a) the text data acquired from each of the plurality of large-scale language models is synthesized to acquire text data of a character string in the document image, and (b) the validity of the text data acquired in the LLM-OCR processing step is determined based on a predetermined similarity between the text data generated in the local OCR processing step and the text data acquired by synthesizing the text data acquired from each of the plurality of large-scale language models, and when it is determined that the text data acquired in the LLM-OCR processing step is valid, the text data acquired in the LLM-OCR processing step is output. A document image processing method comprising:

17. Computer, a local OCR processing unit that performs character recognition processing on a document image and generates text data of a character string described in the document image; an LLM-OCR processing unit that inputs the document image to a large-scale language model together with a prompt, and acquires text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt; and determining the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data acquired by the LLM-OCR processing unit, and when determining that the text data acquired by the LLM-OCR processing unit is valid, causing the LLM-OCR processing unit to function as an integrated processing unit that outputs the text data acquired by the LLM-OCR processing unit; the prompt includes the text data generated by the local OCR processor; A document image processing program characterized by:

18. A computer, a local OCR processing unit that performs character recognition processing on a document image and generates text data of a character string described in the document image; an LLM-OCR processing unit that inputs the document image to a large-scale language model together with a prompt, and acquires text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt; and determining the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data acquired by the LLM-OCR processing unit, and when determining that the text data acquired by the LLM-OCR processing unit is valid, causing the LLM-OCR processing unit to function as an integrated processing unit that outputs the text data acquired by the LLM-OCR processing unit; the integration processing unit, when it is determined that the text data acquired by the LLM-OCR processing unit is invalid, adds the text data generated by the local OCR processing unit to the prompt; the LLM-OCR processing unit inputs the prompt and the document image after the addition to the large-scale language model, and acquires, from the large-scale language model, text data of a character string identified by the large-scale language model in accordance with the prompt; A document image processing program characterized by:

19. A computer, a local OCR processing unit that performs character recognition processing on a document image and generates text data of a character string described in the document image; an LLM-OCR processing unit that inputs the document image to a large-scale language model together with a prompt, and acquires text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt; and determining the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data acquired by the LLM-OCR processing unit, and when determining that the text data acquired by the LLM-OCR processing unit is valid, causing the LLM-OCR processing unit to function as an integrated processing unit that outputs the text data acquired by the LLM-OCR processing unit; the local OCR processing unit identifies the text data and the position of the character string in the document image corresponding to the text data; the integration processing unit specifies a position of a character string in the document image corresponding to the text data acquired by the LLM-OCR processing unit, based on the text data and the position of the character string specified by the local OCR processing unit; A document image processing program characterized by:

20. A computer, a local OCR processing unit that performs character recognition processing on a document image and generates text data of a character string described in the document image; an LLM-OCR processing unit that inputs the document image to a large-scale language model together with a prompt, and acquires text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt; and determining the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data acquired by the LLM-OCR processing unit, and when determining that the text data acquired by the LLM-OCR processing unit is valid, causing the LLM-OCR processing unit to function as an integrated processing unit that outputs the text data acquired by the LLM-OCR processing unit; the LLM-OCR processing unit selects a prompt corresponding to the document type of the document image from among a plurality of document type prompts, inputs the selected prompt and the document image to the large-scale language model, and acquires, from the large-scale language model, text data of a character string identified by the large-scale language model in accordance with the selected prompt; A document image processing program characterized by:

21. A computer, a local OCR processing unit that performs character recognition processing on a document image and generates text data of a character string described in the document image; an LLM-OCR processing unit that inputs the document image to a large-scale language model together with a prompt, and acquires text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt; and determining the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data acquired by the LLM-OCR processing unit, and when determining that the text data acquired by the LLM-OCR processing unit is valid, causing the LLM-OCR processing unit to function as an integrated processing unit that outputs the text data acquired by the LLM-OCR processing unit; the LLM-OCR processing unit generates a document vector for the document image, selects a prompt corresponding to the generated document vector from among prompts associated with a plurality of document vectors, inputs the selected prompt and the document image to the large-scale language model, and acquires, from the large-scale language model, text data of a character string identified by the large-scale language model for an item specified by the selected prompt; A document image processing program characterized by:

22. A computer, a local OCR processing unit that performs character recognition processing on a document image and generates text data of a character string described in the document image; an LLM-OCR processing unit that inputs the document image to a large-scale language model together with a prompt, and acquires text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt; and determining the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data acquired by the LLM-OCR processing unit, and when determining that the text data acquired by the LLM-OCR processing unit is valid, causing the LLM-OCR processing unit to function as an integrated processing unit that outputs the text data acquired by the LLM-OCR processing unit; the local OCR processing unit generates text data of character strings described in the document image, the text data of character strings as labels for specific items and the text data of character strings as values ​​for the items, by associating them with each other; the LLM-OCR processing unit inputs the document image to the large-scale language model together with a prompt specifying the specific item, and acquires, from the large-scale language model, text data of a character string identified by the large-scale language model for the specific item specified by the prompt; the integration processing unit determines the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data acquired by the LLM-OCR processing unit for the specific item; A document image processing program characterized by:

23. A computer, a local OCR processing unit that performs character recognition processing on a document image and generates text data of a character string described in the document image; an LLM-OCR processing unit that inputs the document image to a large-scale language model together with a prompt, and acquires text data of a character string identified by the large-scale language model from the large-scale language model in accordance with the prompt; and determining the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data acquired by the LLM-OCR processing unit, and when determining that the text data acquired by the LLM-OCR processing unit is valid, causing the LLM-OCR processing unit to function as an integrated processing unit that outputs the text data acquired by the LLM-OCR processing unit; the LLM-OCR processing unit inputs the document image together with the prompt to a plurality of large-scale language models different from each other, and acquires text data of character strings identified by the large-scale language models from each of the plurality of large-scale language models in accordance with the prompt; the integration processing unit (a) acquires text data of a character string in the document image by synthesizing the text data acquired from the plurality of large-scale language models, respectively; (b) determines the validity of the text data acquired by the LLM-OCR processing unit based on a predetermined similarity between the text data generated by the local OCR processing unit and the text data obtained by synthesizing the text data acquired from the plurality of large-scale language models, and outputs the text data acquired by the LLM-OCR processing unit when it is determined that the text data acquired by the LLM-OCR processing unit is valid; A document image processing program characterized by:

Images