Information processing device, information processing method, and program

Abstract

The present invention provides an information processing device, method, and program that perform highly accurate item extraction while reducing computational complexity, even when item extraction is performed on documents where the evaluation values ​​of multiple candidates for item names or item values ​​are approximately the same, or on documents that satisfy given conditions. [Solution] The method involves performing character recognition processing on a document image to obtain character information about the strings contained in the document image, extracting candidate strings corresponding to at least one of the item names and item values ​​to be extracted from the character information, obtaining an evaluation value indicating the likelihood of the candidates, and if the evaluation value of the candidates satisfies predetermined conditions, extracting the strings corresponding to at least one of the item names and item values ​​contained in the document image using a large-scale conversational language model (LLM).

Description

Technical Field

[0001] This disclosure relates to a technique for extracting necessary information from document images.

Background Art

[0002] There is a technique for identifying character strings corresponding to item names and item values included in a document image obtained by reading a document such as a receipt with a scanner or the like, and extracting information regarding the item names and item values included in the document using the relationship between the character strings. Hereinafter, the extraction process of information regarding item names and item values will be described as "item extraction" or "item extraction process". In Patent Document 1, when the accuracy required for the results of inspection processing based on rules or the like is not satisfied, the processing accuracy is improved by combining the results of processing by a rule-based method and a learned model obtained as a result of machine learning or the like.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Item extraction is performed by calculating an evaluation value based on character strings corresponding to item names and item values identified in a document image and the arrangement of these character strings in the document image. However, there may be a plurality of candidates for item names or item values or candidates for pairs of item names and item values (hereinafter referred to as "candidate pairs"), and the evaluation values for two or more of these candidates or candidate pairs may be substantially the same or may satisfy a given condition. In such a case, conventional item extraction methods may not be able to perform item extraction correctly.

[0005] In contrast, the technology disclosed in Patent Document 1 improves the accuracy of the judgment process for determining good products in the inspection process by combining rule-based judgment with judgment using a trained model, rather than relying solely on rule-based judgment. However, this method has the problem of increasing the amount of computation because it is necessary to perform both rule-based judgment and judgment using a trained model.

[0006] This disclosure aims to provide a technology that enables highly accurate item extraction while reducing the amount of computation, even when the evaluation values ​​for multiple candidates are nearly identical or satisfy given conditions in the item extraction process. [Means for solving the problem]

[0007] The information processing device according to this disclosure includes: a string acquisition means for acquiring string information relating to strings contained in a document image by performing character recognition processing on the document image; a first extraction means for extracting candidate strings corresponding to at least one of the item names and item values ​​to be extracted from the string information; an evaluation value acquisition means for acquiring an evaluation value indicating the likelihood of the candidate; and a second extraction means for extracting strings corresponding to at least one of the item names and item values ​​contained in the document image using a large-scale dialogue language model, wherein the second extraction means extracts strings corresponding to at least one of the item names and item values ​​when the evaluation value of the candidate satisfies predetermined conditions. [Effects of the Invention]

[0008] According to this disclosure, even when item extraction is performed on documents where the evaluation values ​​for multiple candidates are approximately the same or on documents that satisfy given conditions, it is possible to perform highly accurate item extraction while reducing the amount of computation. [Brief explanation of the drawing]

[0009] [Figure 1]This is a block diagram showing an example of the configuration of an information processing system. [Figure 2] This block diagram shows an example of the functional configuration of an information processing device. [Figure 3] This is a flowchart showing an example of the processing flow of an information processing device. [Figure 4] This figure shows an example of a scanned image. [Figure 5] This figure shows an example of a table related to extraction rules. [Figure 6] This figure shows examples of candidate item names and candidate item values. [Figure 7] This figure shows an example of a prompt. [Figure 8] This is a flowchart showing an example of the processing flow of an information processing device. [Modes for carrying out the invention]

[0010] The best mode for implementing the technology described herein will be described below with reference to the drawings. Note that the following embodiments are not intended to limit the technology described herein, and not all combinations of features described in these embodiments are necessarily essential for solving the problems of this disclosure.

[0011] [First Embodiment] <Configuration of the Information Processing System> Figure 1 is a block diagram showing an example of the configuration of an information processing system 100 according to the first embodiment. The information processing system 100 includes an image input device 110, an information processing device 120, and an external information processing server 130. The image input device 110 has a hardware configuration consisting of a reading unit 111 and a communication unit 112. The image input device 110 is configured as a scanner or a multifunction peripheral (MFP). The reading unit 111 generates a scanned image by reading a document. The communication unit 112 communicates with external devices such as the information processing device 120 via a network. The scanned image data generated by the reading unit 111 is transmitted to the information processing device 120 via the network.

[0012] <Configuration of the information processing device> The information processing device 120 has a hardware configuration consisting of a CPU 121, ROM 122, RAM 123, secondary storage device 124, input unit 125, display unit 126, and communication unit 127. The information processing device 120 is configured using a PC (Personal Computer) or the like.

[0013] The CPU 121 controls the entire information processing device 120 by reading and executing control programs stored in the ROM 122. The RAM 123 is used as the CPU 121's main memory and temporary storage area such as the work area. The secondary storage device 124 is a storage device that stores various data and programs, and is composed of an HDD (Hard Disk Drive) or SSD (Solid State Drive), etc. The processing in the functional configuration of the information processing device 120, which will be described later, is realized by the CPU 121 reading programs stored in the ROM 122 or the secondary storage device 124, etc., and executing them.

[0014] The communication unit 127 communicates with external devices such as the image input device 110 and the external information processing server 130 via a network or the like. The input unit 125 is composed of an input device such as a touch panel, keyboard, or mouse, and receives user operations and transmits a signal corresponding to the operation to the CPU 121. The display unit 126 is composed of a display device such as a liquid crystal display and displays a user interface screen such as a GUI (Graphical User Interface) according to signals transmitted from the CPU 101. The CPU 121 also operates as an input control unit that controls the input unit 125 and a display control unit that controls the display unit 126. In this embodiment, the input unit 125 and the display unit 126 are described as being located inside the information processing device 120, but at least one of the input unit 125 and the display unit 126 may be located outside the information processing device 120 as a separate device.

[0015] The external information processing server 130 provides an API (Application Programming Interface) for executing an interactive large language model (LLM; Large Language Model). The interactive LLM generates a response sentence for a given string called a prompt using the interactive LLM, and ChatGPT (registered trademark) and Bard (registered trademark) are known examples. In this embodiment, the external information processing server 130 is described as being configured as a device different from the information processing device 120, but the external information processing server 130 may be a configuration existing inside the information processing device 120.

[0016] FIG. 2 is a block diagram showing an example of the functional configuration of the information processing device 120 according to the first embodiment. The information processing device 120 includes, as functional components, an image acquisition unit 201, a character string acquisition unit 202, a first extraction unit 203, a score acquisition unit 204, a prompt generation unit 205, a second extraction unit 206, and an output unit 207. The processing of each unit included in the information processing device 120 as a functional component is realized by the CPU 121 reading and executing a program stored in the ROM 122 or the secondary storage device 124.

[0017] <Operation of the information processing device> Referring to FIG. 3, the operation of the information processing device 120 will be described. FIG. 3 is a flowchart showing an example of the processing flow of the information processing device 120 according to the first embodiment. The processing of this flowchart is realized by the CPU 121 reading and executing a program stored in the ROM 122 or the secondary storage device 124.

[0018] First, in S301, the image acquisition unit 201 acquires the data of the scanned image. Specifically, for example, the image acquisition unit 201 receives a signal indicating an instruction regarding image acquisition from the input unit 125 by the user, and transmits a signal regarding a scan request to the image input device 110 via the communication unit 127. When the image input device 110 receives the signal, it generates a scanned image by executing a document reading process using the reading unit 111. After generating the scanned image, the image input device 110 transmits the data of the generated scanned image to the information processing device 120. The data of the scanned image transmitted from the image input device 110 is acquired by the image acquisition unit 201. The image acquisition unit 201 stores the acquired data of the scanned image in the secondary storage device 124.

[0019] Note that in the above description, the image input device 110 has been described as receiving a signal regarding a scan request from the information processing device 120, but it is not limited thereto. For example, the image input device 110 may execute a document reading process based on an operation input by the user on an operation input unit such as a button included in the image input device 110. Also, in the above description, the information processing device has been described as acquiring the data of the scanned image transmitted from the image input device 110, but it is not limited thereto. For example, the information processing device 120 may acquire the data of the scanned image generated and saved in advance by reading it from an external device.

[0020] Figure 4 shows an example of a scanned image 400 according to the first embodiment. The scanned image 400 is, for example, an image obtained by reading an invoice, which is one form of a document. The scanned image 400 includes string information indicating the total amount of the invoice, and string information regarding the issuer of the invoice. The information processing device 120 performs an item extraction process (item extraction process) to extract string information related to item names such as "Total" and string information related to item values ​​such as "¥1,210" corresponding to the item names from the scanned image 400. In this embodiment, the document is described as an invoice as an example, but the document is not limited to an invoice and may be of other forms. Also, in this embodiment, the target of the item extraction process by the information processing device 120 is described as a scanned image obtained as a result of reading, but it is not limited to this. The target of the item extraction process can be any image that contains some string information.

[0021] Following S301, in S302, the string acquisition unit 202 performs optical character recognition (OCR) processing on the scanned image acquired in S301. The string information obtained as a result of the OCR processing, coordinate information indicating the position of the string image in the scanned image, and confidence level information regarding the extraction of the string are stored in RAM 123 or secondary storage device 124. Hereinafter, the string information obtained as a result of the OCR processing and the coordinate information indicating the position of the string image in the scanned image will be collectively referred to as OCR information. Note that the string referred to here is not limited to those composed of multiple characters, but also includes those composed of a single character. Furthermore, the characters referred to here are not limited to the alphabet, hiragana, katakana, and kanji, but may also include numbers and symbols.

[0022] Next, in S303, the first extraction unit 203 acquires information regarding the extraction rules. Specifically, for example, the first extraction unit 203 acquires this information by reading information regarding the extraction rules that is pre-stored in the ROM 122 or the secondary storage device 124.

[0023] Figure 5 shows an example of Table 501 relating to the extraction rules according to the first embodiment. Table 501 shows a list of extraction rules and includes the Acquisition Target Item column 502, Item Name Format column 503, Rule column 504, and Item Value Format column 505. The Item Name Format column 503 stores string information about item names that may be candidates for extraction.

[0024] The item name format column 503 includes the word column 506, the property column 507, and the score column 508, etc. The word column 506 stores regular expressions, etc., that represent words that can be used as item names. The information contained in the word column 506 is not limited to regular expressions, but may also be expressed as strings or word vector representations, etc. The property column 507 stores information indicating the properties (also called "attributes") of words that can be used as item names, such as font size and font weight. The score column 508 stores information indicating the likelihood that the extracted string corresponds to the desired item name, for example, as an evaluation score, when the extracted string matches each of the conditions stored in the word column 506 and the property column 507. Hereinafter, strings that match these conditions will be referred to as "item name candidates" and explained accordingly.

[0025] The rule column 504 stores information for searching for candidate strings that could be the corresponding item value, based on the image region from which the candidate string for the item name was extracted. The rule column 504 includes the direction column 509 and the weighting column 510, etc. The direction column 509 stores information indicating which image regions to search based on the position of the image region from which the candidate string for the item name was extracted. Specifically, the direction column 509 stores information indicating the direction of the search based on the position of the image region from which the candidate string for the item name was extracted. For example, in Figure 5, as an example, the direction column 509 stores information indicating only the direction, such as "right". The information stored in the direction column 509 is not limited to this, and the direction column 509 may also store information indicating a specific range, such as a numerical value that can be determined based on the row height or row width of the candidate string for the item name.

[0026] The weighting column 510 stores information indicating which distances of item values ​​(hereinafter referred to as "item value candidates") located at the position specified in the direction column 509 should be prioritized as candidates. For example, if the information "neighborhood priority" is stored, then in the rule-based evaluation process in the S305 process described later, item values ​​that are close to each other in the document information as item value candidates and item name candidates will be prioritized as item name candidates.

[0027] The item value format column 505 stores information indicating the format of item values ​​that could be candidates for item values. Similar to the item name format column 503, the item value format column 505 includes a word column, a property column, and a score column. Also, in a similar structure to the item name format column 503, information indicating the likelihood that a string matching each condition stored in the word column and property column corresponds to the desired item value is stored, for example, as an evaluation score in the score column.

[0028] Following S303, in S304, the first extraction unit 203 determines candidate item names and candidate item values. Specifically, for example, the first extraction unit 203 determines candidate item names and candidate item values ​​by determining whether each string obtained by the extraction process in S302 meets the conditions stored in the item name format column 503 and the item value format column 505.

[0029] Figure 6 shows an example of item name candidates and item value candidates determined by the first extraction unit 203 according to the first embodiment. Specifically, Figure 6 shows, as an example, regions 601 to 605 containing strings that have been determined by the first extraction unit 203 as item name candidates or item value candidates from among the strings extracted from the scan image 400 shown in Figure 4. Region 601 contains the string (word) "Total", and this string matches the regular expression conditions stored in the word column 506 and the conditions stored in the property column 507 in the item name format column 503. Therefore, region 601 is a region containing the string image that could be an item name candidate. Each of regions 602 to 605 contains the string image that includes a number, and this string matches each of the conditions stored in the item value format column 505. Therefore, each of regions 602 to 605 is a region containing the string image that could be an item value candidate.

[0030] After S304, in S305, the score acquisition unit 204 evaluates each of the one or more item name candidate and item value candidate pairs (candidate pairs) determined in S304 according to the rules, and obtains an evaluation score, which is the final evaluation value for each candidate pair. For example, in Figure 6, if the characters contained in area 601 are a medium font, which is a normal font of 20 pixels or more, then according to Table 501 shown in the rules of Figure 5, the evaluation score for the item name candidate corresponding to area 601 is 18 points. Similarly, the evaluation scores for each item value candidate corresponding to areas 602 and 605 are also 18 points. The score acquisition unit 204 identifies candidate pairs that satisfy the rule from among these item name candidate and item value candidate. In the case of Figure 6, "Total," which is the item name candidate corresponding to area 601, and "¥1,210," which is the item value candidate corresponding to area 605, are identified as candidate pairs that satisfy the rule.

[0031] For example, if the evaluation score is considered to be the sum of the evaluation points of the item name candidates and the evaluation points of the item value candidates, the highest evaluation score would be 18 + 18 = 36 points. However, the method of calculating the evaluation score is not limited to this, and the evaluation score can be calculated by any method, such as the sum of each evaluation point, the product of each evaluation point, or by multiplying each evaluation point by a predetermined constant and then calculating the sum of those. In this embodiment, as an example, the evaluation score was used as the sum of the evaluation points of the item name candidates and the evaluation points of the item value candidates, but other factors such as the reliability of the OCR processing results or the weighting of the rules may also be quantified and taken into consideration.

[0032] After S305, in S306, the prompt generation unit 205 determines whether the evaluation scores of two or more candidate pairs that have the highest evaluation score among the evaluation scores of each candidate pair are close to each other, if there are multiple candidate pairs. For example, if the highest evaluation score among the multiple candidate pairs obtained in S305 is 36 points, the prompt generation unit 205 determines that the evaluation scores of two or more candidate pairs with an evaluation score of 36 points are close to each other.

[0033] Furthermore, the determination of whether evaluation scores are close to each other is not limited to whether they are identical, but may also be made by determining whether the evaluation scores are approximately identical. Here, "approximately identical" means, for example, when multiple evaluation scores have the same number of digits and the most significant digit is approximately identical, or when multiple evaluation scores are above a predetermined threshold that can be calculated based on the average value of the evaluation scores, etc. Also, for strings where there are multiple correct answers as an item value, such as telephone numbers, if there are a predetermined number or more candidate pairs whose evaluation scores are above the threshold, all of these evaluation scores may be determined to be approximately identical. Also, if there are multiple candidate pairs of questionable evaluation scores, all of these evaluation scores may be determined to be approximately identical. Here, a questionable evaluation score is, for example, when the evaluation scores are clustered into a higher score group and a lower score group, and the statistical value such as the average value of the evaluation scores in the higher score group is approximately identical to the statistical value such as the average value of the evaluation scores in the lower score group. In this case, for example, all evaluation scores in the higher score group may be determined to be approximately identical.

[0034] If it is determined in S306 that the evaluation scores are not close to each other, the information processing device 120 executes the process in S310. The process in S310 will be described later. If it is determined in S306 that the evaluation scores are close to each other, in S308 and S309, the information processing device 120 executes item extraction processing using the interactive LLM. It is known that by passing OCR information to the interactive LLM, it is possible to automatically extract important information from the document. First, in S308, the prompt generation unit 205 generates a prompt for performing item extraction processing using the interactive LLM.

[0035] Figure 7 shows an example of a prompt 700 generated by the prompt generation unit 205 according to the first embodiment. The prompt 700 includes, for example, an extraction target specification statement 701 and a document string 702.

[0036] The extraction target specification statement 701 is a statement that describes the items to be extracted from the entire text of the prompt 700. This statement is generated, for example, based on the information in the acquisition target item column 502 in Table 501 shown in Figure 5. In the example shown in Figure 7, a standard phrase, "We will extract information from the document regarding the following items," is added at the beginning. Subsequently, a string indicating the items to be extracted is added based on the strings stored in the acquisition target item column 502. Next, a statement describing the format of the strings extracted from the document is added. Note that, as described above, the extraction target specification statement 701 includes the strings stored in the acquisition target item column 502, but is not limited to this. For example, the extraction target specification statement 701 may include, in place of or in addition to, the strings stored in the acquisition target item column 502, information stored in the item name format column 503 and the item value format column 505.

[0037] The document string 702 is a text string containing OCR information for the document used to extract items from the entire text of the prompt 700. In the example shown in Figure 7, the document string 702 is edited line by line, containing the coordinates in the scanned image for each block of strings obtained as a result of the OCR process, and the string information contained within each block of strings. In the above description, each line in the document string 702 is represented in the format "{array of coordinate information}:{string obtained as a result of OCR processing}", but the representation method of the document string 702 is not limited to this. For example, each line in the document string 702 may be represented in a format that includes information such as the font of the string obtained as a result of the OCR process, in addition to the information described above.

[0038] After S308, in S309, the second extraction unit 206 sends the prompt 700 generated in S308 to the external information processing server 130 via the communication unit 127 and obtains the result of the item extraction process performed by the external information processing server 130. The result of the item extraction process performed by the external information processing server 130 is provided, for example, in JSON (JavaScript Object Notation) format. The second extraction unit 206 parses the result of the item extraction process in JSON format or the like provided by the external information processing server 130 to obtain, for example, information such as a string indicating the name of a pair of items and a string indicating the value of the item. Note that the method of communication with the external information processing server 130 is not limited to the method described above. The communication method may, for example, use an input / output method such as the Function Calling API in ChatGPT (registered trademark) to set a function that provides the item value in advance, and the result of the item value is returned as an argument. After S309, the information processing device 120 executes the process in S310.

[0039] In S310, the output unit 207 outputs the results of the item extraction process. Specifically, if it was determined in S306 that the evaluation scores were not close to each other, the output unit 207 outputs the candidate pair with the highest evaluation score obtained in S305 as the result of the item extraction process. On the other hand, if it was determined in S306 that the evaluation scores were close to each other, the output unit 207 outputs the string pairs of item names and item values ​​obtained as a result of the item extraction process by the external information processing server 130 in S309 as the result of the item extraction process.

[0040] The output unit 207 outputs, for example, a display image showing the results of the extracted item processing to the display unit 126 of the information processing device 120, causing the display image to be displayed on the display unit 126. Note that the output method by the output unit 207 is not limited to outputting a display image to the display unit 126; the output unit 207 may also output information showing the results of the extracted item processing to the secondary storage device 124, storing this information in the secondary storage device 124. Furthermore, the output unit 207 may output information showing the results of the extracted item processing to the image input device 110 via the communication unit 127. After S310, the information processing device 120 completes the processing shown in the flowchart in Figure 3.

[0041] As described above, the information processing device 120 is configured to execute an interactive LLM-based item extraction process if the evaluation scores of the items to be extracted are close to each other as a result of the rule-based item extraction process. With the information processing device 120 configured in this way, pairs of strings to be extracted can be extracted from a document at high speed and with high accuracy.

[0042] [Second Embodiment] In the second embodiment, descriptions of the configuration and processing similar to that of the first embodiment will be omitted, and only the differences from the first embodiment will be described. Therefore, configurations and processing not specifically mentioned are assumed to be the same as in the first embodiment.

[0043] In the first embodiment, the information processing device 120 embedded all of the OCR information into the prompt and then performed item extraction processing using the interactive LLM. In the second embodiment, a configuration in which the information processing device 120 embeds only a portion of the OCR information into the prompt passed to the interactive LLM will be described.

[0044] <Operation of the Information Processing Device> Figure 8 is a flowchart showing an example of the processing flow of the information processing device 120 according to the second embodiment (hereinafter simply referred to as "information processing device 120"). Note that for steps in Figure 8 that perform the same processing as those in Figure 3, the corresponding step reference numerals in Figure 3 are used, and their explanation is omitted. First, the information processing device 120 executes the processes from S301 to S306. If it is determined in S306 that the evaluation scores are not close to each other, the information processing device 120 executes the process in S310. If it is determined in S306 that the evaluation scores are close to each other, in S807, S308, and S309, the information processing device 120 executes item extraction processing using the interactive LLM.

[0045] In S807, the prompt generation unit 205 selects the OCR information to be included in the prompt passed to the interactive LLM. Here, the selection of OCR information means selecting only the OCR information for the pages containing the candidate item name or candidate item value determined in S304, when the document targeted for item extraction processing consists of multiple pages. Note that the selection of OCR information when executing item extraction processing by the interactive LLM does not necessarily have to be done on a page-by-page basis; for example, it may be done within a predetermined range, such as within 100 characters surrounding the candidate item name or candidate item value. Furthermore, this range may be specified or changed as necessary.

[0046] After S807, the information processing device 120 executes the process in S308. In this embodiment, the prompt generation unit 205 creates a prompt as shown in Figure 7 as an example, and the document string 702 includes the OCR information selected in S807 converted into text. The prompt generation unit 205 may also generate a prompt in the document string 702 that includes, in addition to the OCR information selected in S807, characters or symbols indicating page breaks in the document, and information indicating page numbers. After S308, the information processing device 120 executes the processes in S309 and S310, and then terminates the process of the flowchart shown in Figure 8.

[0047] As described above, the information processing device 120 is configured to limit the OCR information included in the prompt when performing item extraction processing using the interactive LLM. With the information processing device 120 configured in this way, it becomes possible to perform item extraction processing using the interactive LLM while reducing the amount of prompts. As a result, with the information processing device 120, for example, when using an API provided from an external source, the usage fee for the interactive LLM can be reduced. In addition, with the information processing device 120, for example, when operating the interactive LLM internally, it is expected that processing will be sped up by reducing the amount of computation.

[0048] [Other embodiments] In the information processing system 100 according to the above embodiment, the image input device 110 and the information processing device 120 were described as separate devices, but they may also be configured as a single device that possesses all the functions of the image input device 110 and the information processing device 120. Specifically, for example, the image input device 110 may have the information processing device 120 inside it, or the information processing device 120 may have the image input device 110 inside it.

[0049] Furthermore, this disclosure can also be implemented by supplying a program that implements one or more of the functions of the embodiments described above to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be implemented by a circuit (e.g., an ASIC) that implements one or more functions.

[0050] Within the scope of this disclosure, it is possible to freely combine the embodiments, modify any component of each embodiment, or omit any component in each embodiment.

[0051] <Structure of this disclosure> The technology disclosed herein includes the following configurations, methods, and programs.

[0052] <Configuration 1> A string acquisition means that obtains string information relating to strings contained in a document image by performing character recognition processing on the document image, A first extraction means extracts candidate strings from the string information that correspond to at least one of the item name and item value to be extracted, An evaluation value acquisition means for obtaining an evaluation value indicating the likelihood of the aforementioned candidate, A second extraction means that uses a large-scale dialogue language model to extract a string corresponding to at least one of the item name and item value contained in the document image, It has, The second extraction means extracts a string corresponding to at least one of the item name and the item value when the candidate evaluation value satisfies predetermined conditions. An information processing device characterized by the following.

[0053] <Configuration 2> The second extraction means extracts a string corresponding to at least one of the item name and the item value when the difference between two or more candidate evaluation values ​​is less than a predetermined threshold. An information processing device as described in Configuration 1, characterized by the above.

[0054] <Structure 3> The second extraction means extracts a string corresponding to at least one of the item name and the item value when the evaluation values ​​of two or more candidates are equal to or greater than a predetermined threshold. An information processing device according to configuration 1 or 2, characterized by the above.

[0055] <Structure 4> The second extraction means generates input information to be input to the large-scale dialogue language model using the string information and the string information corresponding to the item name, thereby extracting a string corresponding to at least one of the item name and the item value. An information processing device according to any one of configurations 1 to 3 characterized by the above.

[0056] <Composition 5> The second extraction means, when generating the input information, selects information to be included in the input information from the string information based on the evaluation value of the candidates. An information processing device as described in configuration 4, characterized by the above.

[0057] <Composition 6> The first extraction means extracts candidate strings corresponding to the item name and candidate strings corresponding to the item value that is paired with the item name from the string information as candidate pairs. The system further includes an evaluation value acquisition means for acquiring an evaluation value indicating the likelihood of the aforementioned candidate pair, The second extraction means extracts a string corresponding to at least one of the item name and the item value when the evaluation value of the candidate pair satisfies predetermined conditions. An information processing device according to any one of configurations 1 to 5 characterized by the above.

[0058] <Composition 7> The evaluation value acquisition means acquires the candidate evaluation value of the string corresponding to the item name and the candidate evaluation value of the string corresponding to the item value, and acquires the evaluation value of the candidate pair based on the acquired candidate evaluation value of the string corresponding to the item name and the candidate evaluation value of the string corresponding to the item value. An information processing device according to configuration 6, characterized by the above.

[0059] <Structure 8> The second extraction means extracts a string corresponding to at least one of the item name and the item value when the difference between the evaluation values ​​of two or more candidate pairs is less than a predetermined threshold. An information processing apparatus according to configuration 6 or 7, characterized by the above.

[0060] <Composition 9> The second extraction means extracts a string corresponding to at least one of the item name and the item value when the evaluation values ​​of two or more candidate pairs are equal to or greater than a predetermined threshold. An information processing device according to any one of configurations 6 to 8 characterized by the above.

[0061] <Composition 10> The second extraction means generates input information to be input to the large-scale dialogue language model using the string information and the string information corresponding to the item name, thereby extracting a string corresponding to at least one of the item name and the item value. The second extraction means, when generating the input information, selects information to be included in the input information from the string information based on the evaluation value of the candidate pair. An information processing device according to any one of configurations 6 to 9 characterized by the above.

[0062] <Composition 11> The first extraction means performs the extraction of candidates based on predetermined extraction conditions. An information processing device according to any one of configurations 1 to 10 characterized by the above.

[0063] <Method> A string acquisition step involves performing character recognition processing on a document image to obtain string information relating to the strings contained in the document image, A first extraction step involves extracting candidate strings from the string information that correspond to at least one of the item name and item value to be extracted. An evaluation value acquisition step is to obtain an evaluation value that indicates the likelihood of the aforementioned candidate, A second extraction step involves using a large-scale dialogue language model to extract a string corresponding to at least one of the item name and item value contained in the document image, Includes, The second extraction step involves extracting a string corresponding to at least one of the item name and the item value if the candidate evaluation value satisfies predetermined conditions. An information processing method characterized by the following.

[0064] <Program> A program to cause a computer to function as an information processing device described in any one of configurations 1 through 11. [Explanation of Symbols]

[0065] 120 Information Processing Devices 202 String acquisition part 203 1st extraction part 204 Score Acquisition Section 206 Second extraction part

Claims

1. A string acquisition means that obtains string information relating to strings contained in a document image by performing character recognition processing on the document image, A first extraction means extracts candidate strings from the string information that correspond to at least one of the item name and item value to be extracted, An evaluation value acquisition means for obtaining an evaluation value indicating the likelihood of the aforementioned candidate, A second extraction means that uses a large-scale dialogue language model to extract a string corresponding to at least one of the item name and item value contained in the document image, It has, The second extraction means extracts a string corresponding to at least one of the item name and the item value when the candidate evaluation value satisfies predetermined conditions. An information processing device characterized by the following.

2. The second extraction means extracts a string corresponding to at least one of the item name and the item value when the difference between two or more candidate evaluation values ​​is less than a predetermined threshold. The information processing apparatus according to claim 1, characterized by the following:

3. The second extraction means extracts a string corresponding to at least one of the item name and the item value when the evaluation values ​​of two or more candidates are equal to or greater than a predetermined threshold. The information processing apparatus according to claim 1, characterized by the following:

4. The second extraction means generates input information to be input to the large-scale dialogue language model using the string information and the string information corresponding to the item name, thereby extracting a string corresponding to at least one of the item name and the item value. The information processing apparatus according to claim 1, characterized by the following:

5. The second extraction means, when generating the input information, selects information to be included in the input information from the string information based on the evaluation value of the candidates. The information processing apparatus according to claim 4, characterized by the above.

6. The first extraction means extracts candidate strings corresponding to the item name and candidate strings corresponding to the item value that is paired with the item name from the string information as candidate pairs. The system further includes an evaluation value acquisition means for acquiring an evaluation value indicating the likelihood of the aforementioned candidate pair, The second extraction means extracts a string corresponding to at least one of the item name and the item value when the evaluation value of the candidate pair satisfies predetermined conditions. The information processing apparatus according to claim 1, characterized by the following:

7. The evaluation value acquisition means acquires the candidate evaluation value of the string corresponding to the item name and the candidate evaluation value of the string corresponding to the item value, and acquires the evaluation value of the candidate pair based on the acquired candidate evaluation value of the string corresponding to the item name and the candidate evaluation value of the string corresponding to the item value. The information processing apparatus according to claim 6, characterized by the following:

8. The second extraction means extracts a string corresponding to at least one of the item name and the item value when the difference between the evaluation values ​​of two or more candidate pairs is less than a predetermined threshold. The information processing apparatus according to claim 6, characterized by the following:

9. The second extraction means extracts a string corresponding to at least one of the item name and the item value when the evaluation values ​​of two or more candidate pairs are equal to or greater than a predetermined threshold. The information processing apparatus according to claim 6, characterized by the following:

10. The second extraction means generates input information to be input to the large-scale dialogue language model using the string information and the string information corresponding to the item name, thereby extracting a string corresponding to at least one of the item name and the item value. The second extraction means, when generating the input information, selects information to be included in the input information from the string information based on the evaluation value of the candidate pair. The information processing apparatus according to claim 6, characterized by the following:

11. The first extraction means performs the extraction of candidates based on predetermined extraction conditions. The information processing apparatus according to claim 1, characterized by the following:

12. A string acquisition step involves performing character recognition processing on a document image to obtain string information relating to the strings contained in the document image, A first extraction step involves extracting candidate strings from the string information that correspond to at least one of the item name and item value to be extracted. An evaluation value acquisition step is to obtain an evaluation value that indicates the likelihood of the aforementioned candidate, A second extraction step involves using a large-scale dialogue language model to extract a string corresponding to at least one of the item name and item value contained in the document image, Includes, The second extraction step involves extracting a string corresponding to at least one of the item name and the item value if the candidate evaluation value satisfies predetermined conditions. An information processing method characterized by the following.

13. A program for causing a computer to function as an information processing device according to any one of claims 1 to 11.

Images