Information processing apparatus, method for controlling information processing apparatus, program, and information processing system

Abstract

To provide an information processing apparatus that, even when an item value of an item to be extracted is not described on a document image, infers the item value of the item to be extracted, a method for controlling the same, a program, and an information processing system.SOLUTION: An information processing system includes an information processing server 102 for extracting character strings corresponding to predetermined items from a document image. The information processing server comprises: an instruction generation unit 353 that generates instruction sentences used for inference of character strings according to the format of item values corresponding to the predetermined items; a character string acquisition unit 354 that inputs the instruction sentences generated by the instruction generation unit 353 to a large-scale language model to acquire character strings according to the format; and an output unit 355 that outputs the character strings according to the format acquired by the character string acquisition unit 354 as the item values.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a control method for an information processing device, a program, and an information processing system for extracting information from a document image. [Background technology]

[0002] There is a technology for extracting character strings (hereinafter referred to as "item values") corresponding to predetermined items (e.g., title, document number, issue date, company name, amount, etc., hereinafter referred to as "item names") written in an input document image. For example, Patent Document 1 discloses a technology for extracting item values ​​corresponding to each item name from a document image and displaying on a UI screen the positions of each combination of the extracted item name and item value on the document image so that the user can see them. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2021-131593 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, if the item name is a deadline, there are cases where a specific date is written, such as "Estimated deadline: October 12, 2023," and cases where no specific date is written, such as "Estimated deadline: 30 days." When a specific date is written, the item value in the document image can be extracted using the technology of Patent Document 1. However, when a specific date is not written, it is not possible to extract a specific date from the information written in the document image. Therefore, the user had to calculate the actual estimate deadline date using information other than "Estimated deadline: 30 days," such as "Issue date: October 12, 2023." The present disclosure has been made in consideration of these points. [Means for solving the problem]

[0005] The information processing device according to the present disclosure is an information processing device for extracting a character string corresponding to a specified item from a document image, and is characterized by comprising: a generation means for generating an instruction sentence used to infer a character string conforming to a format of an item value corresponding to the specified item; an acquisition means for acquiring the character string conforming to the format by inputting the instruction sentence generated by the generation means into a large-scale language model; and an output means for outputting the character string conforming to the format acquired by the acquisition means as the item value. [Effects of the Invention]

[0006] According to the technology of the present disclosure, even when an item value corresponding to an item name to be extracted is not written in a document image, it is possible to obtain the item value appropriately. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing a system configuration of an information processing system. [Figure 2] FIG. 2 is a diagram showing the hardware configuration of an information processing device and an information processing server. [Figure 3] FIG. 2 is a diagram showing the software configuration of an information processing apparatus and an information processing server. [Figure 4] 6 is a flowchart showing a process of inferring extraction items from a document image and outputting the results in the first embodiment. [Figure 5] 10A and 10B are diagrams showing specific examples of an input document image and a displayed document image. [Figure 6] FIG. 3 is a diagram showing the results of OCR processing performed on a document image in the first embodiment. [Figure 7] 6 is a flowchart showing a process of inferring an item value from an OCR result in the first embodiment. [Figure 8] FIG. 3 is a diagram showing an example of a directive sentence to be input to a large-scale language model in the first embodiment. [Figure 9] 10 is a flowchart showing a process of inferring an item value from an OCR result in the second embodiment. [Figure 10]FIG. 11 is a diagram showing an example of a directive sentence to be input to a large-scale language model in the second embodiment. [Figure 11] FIG. 11 is a diagram showing an example of a directive sentence to be input to a large-scale language model in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that the following embodiments do not limit the present disclosure, and not all combinations of features described in the embodiments are necessarily essential to the solutions of the present disclosure. Note that identical components will be described with the same reference numerals. Furthermore, each process (step) in a flowchart will be indicated with a reference numeral beginning with "S."

[0009] [First embodiment] <System configuration of information processing system> FIG. 1 is a diagram showing an example of the configuration of an information processing system according to this embodiment. As shown in FIG. 1, the information processing system 100 includes an information processing device 101, an information processing server 102, and an external information processing server 103. The devices are connected to each other via a network 104. In the information processing system 100, the information processing device 101, the information processing server 102, and the external information processing server 103 may be configured to be connected to the network 104 in multiple rather than single connections. For example, the information processing server 102 may include a first server device having high-speed computing resources and a second server device having large-capacity storage, and the server devices may be connected to each other via the network 104. Furthermore, in this embodiment, the information processing device 101 and the information processing server 102 are described as being separate entities, but the information processing device 101 and the information processing server 102 may be integrated.

[0010] <Hardware configuration of each device> 2 is a diagram showing an example of the hardware configuration of the information processing device 101 and the information processing server 102 of the information processing system 100 according to this embodiment. FIG. 2(a) is a diagram showing the hardware configuration of the information processing device 101. As shown in FIG. 2(a), the information processing device 101 includes a CPU 201, a ROM 202, a RAM 204, a printer device 205, a scanner device 206, a storage 208, and an external interface 211. The devices are connected to each other via a data bus 203.

[0011] The CPU 201 is a control unit for controlling the overall operation of the information processing device 101. The CPU 201 starts up the system of the information processing device 101 by executing a startup program stored in the ROM 202. The CPU 201 also executes a control program stored in the storage 208 to realize functions of the information processing device 101, such as printing, scanning, or faxing.

[0012] The ROM 202 is a non-volatile memory and is a storage unit for storing a startup program that starts up the information processing device 101. The data bus 203 is a communication unit for transmitting and receiving data between devices that make up the information processing device 101. The RAM 204 is a volatile memory and is a storage unit used as a work memory when the CPU 201 executes a control program.

[0013] The printer device 205 is an image output device and a processing unit for printing and outputting a document image inside the information processing device 101 on a sheet of paper or the like. The scanner device 206 is an image input device and a processing unit for optically reading a sheet of paper or the like on which characters, diagrams, etc. are printed and acquiring the document image. The document transport device 207 is realized by an ADF (Auto Document Feeder) or the like and is a processing unit for detecting documents placed on a platen and transporting the detected documents one by one to the scanner device 206.

[0014] The storage 208 is realized by a hard disk drive (HDD) or the like, and is a storage unit for storing the control programs and document images described above. The storage 208 is not limited to an HDD, but may be a solid state drive (SSD) or a flash memory.

[0015] The input device 209 is realized by a touch panel, hard keys, or the like, and is a processing unit for accepting operation input from a user who uses the information processing device 101. The display device 210 is realized by a liquid crystal display, or the like, and is a display unit for displaying output results of a setting screen of the information processing device 101 to the user. By using a touch panel, the input device 209 and the display device 210 may be integrated. The external interface 211 is an interface unit that connects the information processing device 101 and the network 104, receives FAX data transmitted from a FAX transmitter (not shown), and transmits a document image to the information processing server 102.

[0016] Fig. 2(b) is a diagram showing the configuration of the information processing server 102. As shown in Fig. 2(b), the information processing server 102 includes a CPU 261, a ROM 262, a RAM 264, a storage 265, an input device 266, a display device 267, and an external interface 268. The devices are connected to each other via a data bus 263.

[0017] The CPU 261 is a control unit for controlling the overall operation of the information processing server 102. The CPU 261 starts up the system of the information processing server 102 by executing a boot program stored in the ROM 262. The CPU 261 also executes an information processing program stored in the storage 265 to perform information processing such as character recognition and information extraction using OCR (Optical Character Recognition) or the like.

[0018] The ROM 262 is a non-volatile memory and is a storage unit for storing a boot program that starts up the information processing server 102. The data bus 263 is a communication unit for transmitting and receiving data between devices that make up the information processing server 102. The RAM 264 is a volatile memory and is a storage unit used as a work memory when the CPU 261 executes the information processing program. The storage 265 is realized by an HDD or the like and is a storage unit for storing the information processing program, document images, item extraction results, extraction results of information used for inference, etc. The storage 265 is not limited to an HDD and may be an SSD or flash memory.

[0019] The input device 266 is realized by a mouse, a keyboard, or the like, and is a processing unit for accepting input operations to the information processing server 102 from a user who uses the information processing server 102 or an engineer who controls the information processing server 102. The display device 267 is realized by a liquid crystal display or the like, and is a display unit for displaying a setting screen of the information processing server 102 and output results to a user who uses the information processing server 102 or an engineer who controls the information processing server 102. The input device 266 and the display device 267 may be integrated by using a touch panel. The external interface 268 is an interface unit that connects the information processing server 102 and the network 104 and receives document images from the information processing device 101.

[0020] <Software configuration of each device> 3A and 3B are diagrams showing an example of software functional blocks of the information processing device 101 and the information processing server 102 of the information processing system 100 according to this embodiment. Fig. 3A is a diagram showing software functional blocks of the information processing device 101, and Fig. 3B is a diagram showing software functional blocks of the external information processing server 103.

[0021] The information processing device 101 is realized by an MFP (Multi Function Peripheral) or the like having multiple functions such as printing, scanning, and faxing, and functions as an image acquisition unit 301, a first data transmission / reception unit 302, and a display control unit 303.

[0022] The image acquisition unit 301, for example, optically reads an original document printed on a sheet such as paper and performs predetermined scan image processing to generate a document image. The image acquisition unit 301 also receives, for example, fax data transmitted from a fax transmitter (not shown) and generates a document image by performing predetermined fax image processing. The first data transmission / reception unit 302 transmits the generated document image to the information processing server 102. The first data transmission / reception unit 302 also transmits and receives other data to and from the information processing server 102. The display control unit 303, for example, controls the display of image data of the document image and other information received from the information processing server 102 on the display device 210. The display control unit 303, for example, displays a confirmation screen or the like shown in FIG. 9 (described later) on the display device 210.

[0023] The information processing device 101 may be configured to be realized by a PC (Personal Computer), a tablet terminal, a smartphone, etc., other than the MFP having the above-mentioned scan and FAX functions. Specifically, for example, a document image such as PDF or JPEG generated using a document creation application running on a PC serving as the information processing device 101 may be transmitted to the information processing server 102 via the first data transmission / reception unit 302.

[0024] The information processing server 102 functions as a second data transmission / reception unit 351, a character string extraction unit 352, an instruction generation unit 353, a character string acquisition unit 354, and an output unit 355. Specifically, the information processing server 102 has a function of receiving a document image as input, processing the image via the external information processing server 103, and transmitting the processed result to the information processing device 101.

[0025] The second data transmission / reception unit 351 transmits and receives data such as a document image generated by the information processing device 101 to and from the information processing device 101. The character string extraction unit 352 performs OCR processing on the document image received from the information processing device 101, acquires multiple character strings in the document image, and extracts character strings corresponding to predetermined items from the document image.

[0026] If the extracted character string does not conform to the format of the item value corresponding to a predetermined item, the instruction generation unit 353 generates an instruction including an instruction sentence for inferring a character string conforming to the format. Specifically, the instruction generation unit 353 applies multiple character strings included in the document image and the item type to be extracted to a pre-prepared instruction template, and generates an instruction to input the template to the large-scale language model 151. Here, the pre-prepared instruction template may be one prepared by an engineer as a preset, one prepared by a user, or one modified or added to by the system or the user. Thereafter, the information processing server 102 transmits the generated instruction to the external information processing server 103 via the network 104. The instruction generation unit 353 also generates an instruction including an instruction sentence for inferring a candidate item value using the instruction template.

[0027] The character string acquisition unit 354 receives a response result to the instruction from the external information processing server 103, and acquires a character string according to the above format or a character string according to an instruction statement that infers a character string of a candidate item value, etc. The output unit 355 displays the character string according to the format acquired by the character string acquisition unit 354 on the display device 267. The output unit 355 also transmits the character string according to the format and other data to the information processing device 101 via the second data transmission / reception unit 351. The display control unit 303 of the information processing device 101 displays the character string according to the format and other data received via the first data transmission / reception unit 302 on the display device 210.

[0028] The network 104 is realized by a LAN, a WAN, or the like, and is a communication unit that connects the information processing device 101, the information processing server 102, and the external information processing server 103 to one another and transmits and receives data between the devices.

[0029] The external information processing server 103 functions as a large-scale language model 151. The large-scale language model 151 is accessed by an API (Application Programming Interface) via the network 104, and outputs a response result to a command received from the information processing server 102 to the information processing server 102. The external information processing server 103 may be configured to exist on a different system of the same vendor, or may be configured to exist on a system of an external vendor. The external information processing server 103 including the large-scale language model 151 may be configured to exist within the information processing server 102. Alternatively, the information processing server 102 may be configured so that some of the functions or some of the devices of the external information processing server 103 exist.

[0030] <Item value output processing> 4 is a flowchart illustrating a flow in which the information processing server 102 performs document analysis on a document image received from the information processing device 101, and infers and outputs item values ​​to be extracted. Note that the execution program for each step shown in FIG. 3 is stored in the ROM 262, RAM 264, or storage 265 of the information processing server 102, and is executed by the CPU 261 of the information processing server 102.

[0031] In S401, the character string extraction unit 352 performs character recognition processing on the document image to acquire one or more character strings. Specifically, OCR processing is performed on the document image to acquire one or more character strings included in the document image and area information for the character strings. A method for acquiring one or more character strings and area information will be described with reference to FIGS. 5 and 6. FIG. 5 is a diagram showing a document image 500 according to this embodiment, which is an example of a document image. FIG. 6 is a diagram showing an OCR result 600 obtained by performing OCR processing on the document image 500 of FIG. 5. The OCR result 600 includes one or more pieces of character information (each line in FIG. 6). The one or more pieces of character information further include a character string 601 of the OCR result and area information 602 indicating the position of the character string acquired by performing the OCR processing. The character string 601 is a character string extracted from the document image 500, and the area information 602 is position information of the character string 601 on the document image 500. Note that the above-described method for acquiring a character string is merely an example and is not limited thereto.

[0032] In S402, the character string acquisition unit 354 infers an item value using the character string in the document image 500 acquired in S401. Detailed processing of S402 will be described later. In S403, the output unit 355 outputs the item value inferred in S402 to a display device.

[0033] <First inference process for item values> 7 is a flowchart showing a first inference process of item values ​​in which the information processing server 102 infers item values ​​from OCR results. Note that an execution program for each step shown in FIG. 7 is stored in the ROM 262, RAM 264, or storage 265 of the information processing server 102, and is executed by the CPU 261 of the information processing server 102.

[0034] In S701, the character string extraction unit 352 extracts an item value using the character string acquired in S401. A known method is used to extract the item value. For example, the method disclosed in Patent Document 1 may be used.

[0035] In S702, the character string extraction unit 352 determines whether the item value acquired in S701 is in an appropriate format. If the item value is in an appropriate format, the first inference process for the item value ends. If the item value is not in an appropriate format, the process proceeds to S703. An item value being in an appropriate format means that the format of the acquired item value matches a predefined item value format. For example, an item value related to a date such as an issue date or expiration date would be "YYYY-MM-DD" or "YYYY / MM / DD" (where Y, M, and D are numbers from 0 to 9). An item value related to an amount such as a total amount or payment amount would be "¥" or "¥〇" (where 〇 is any number). Other formats include the format of an item value for a document name or document number. It is desirable to store the definition of the item value format in the ROM 262 in advance, but it may also be input when inferring the item value.

[0036] In S703, the instruction generation unit 353 generates an instruction including an instruction statement for inferring an item value. FIG. 8 shows an example of an instruction 810 generated when inferring an item value for an extracted item name "Deadline." The instruction 810 includes instruction content 811, an output format entry field 812, and text information 813. While the description shown in FIG. 8 is used in this embodiment, other instructions may be used as long as they include the above three components and convey the intent of the instruction. For example, the text information 813 may be written at the beginning of the instruction 810. The instruction 810 includes an "instruction to infer an item value" (instruction statement 821) and an "instruction regarding an item value inference method" (instruction statement 822). The "instruction to infer an item value" (instruction statement 821) is a sentence that includes the extracted item 801 and may contain content instructing the inference of the item value of the extracted item 801. The "instruction regarding an item value inference method" (instruction statement 822) describes how to infer the item value of the extracted item 801 and specifies the output format. In this embodiment, the instruction is to use the value corresponding to the extracted item "deadline" and the information learned by the large-scale language model 151, but this is not limited to this. Any expression may be used as long as the instruction content similar to the instruction content described above is described. For example, an instruction that combines instruction statement 821 and instruction statement 822 may be used. In other words, an instruction may be used that includes an instruction to infer an item value, an instruction for inferring an item value, and a specification of an output format for the item value.

[0037] The output format entry field 812 includes extracted items 801 and an output format 802. Any expression may be used as long as it describes the output format instruction. The text information 813 is composed of text content 803 that infers the item value of the extracted items 801. In this embodiment, the text content 803 describes the OCR result 600 of the document image 500.

[0038] In S704, the character string acquisition unit 354 inputs the generated instruction 810 into the large-scale language model 151 and acquires a character string of the item value that conforms to the format specified in the instruction statement 822. When the character string of the item value that conforms to the format is acquired, the following item value display process is executed.

[0039] If the item value extracted in S701 is in an appropriate format, the output unit 355 displays the character string of the extracted item value on the display device 267 of the information processing server 102. The character string of the extracted item value is also transmitted to the information processing device 101 via the second data transmission / reception unit 351 and the first data transmission / reception unit 302. The display control unit 303 of the information processing device 101 displays the character string of the extracted item value on the display device 210.

[0040] If the item value extracted in S701 is not in an appropriate format, the output unit 355 displays a character string of the item value according to the format acquired in S704 on the display device 267 of the information processing server 102. The character string of the item value according to the format is transmitted to the information processing device 101 via the second data transmission / reception unit 351 and the first data transmission / reception unit 302. The display control unit 303 of the information processing device 101 displays the character string of the item value according to the format on the display device 210.

[0041] As described above, according to this embodiment, even if an item value corresponding to an item name to be extracted is not written in a document image, it is possible to obtain an appropriate item value. Therefore, it is not necessary for the user to calculate the item value of an item that is not written by the user.

[0042] [Second embodiment] In this embodiment, only the differences from the first embodiment will be described, and the same configuration or procedure as the first embodiment will be used unless otherwise specified. In the first embodiment, a publicly known item value extraction method such as that described in Patent Document 1 is used to extract item values. For example, there are various ways to write the item value of the extraction item "deadline," so there is a possibility that extraction will be missed using a publicly known item extraction method. In this embodiment, a method will be described in which item values ​​are extracted using a large-scale language model 151 and parameters of the large-scale language model 151 are controlled.

[0043] <Second inference process for item values> 9 is a flowchart showing the second inference process of inferring item values ​​from OCR results according to this embodiment. The processes of S905, S906, and S907 are the same as those of S702, S703, and S704, and therefore detailed descriptions of each step will be omitted.

[0044] In S901, the instruction generation unit 353 uses the character string acquired in S401 to generate a first instruction including an instruction statement to extract an item value of an item to be extracted. In the following description, the item to be extracted is assumed to be "deadline." FIG. 10 shows an example of a first instruction generated when extracting an item value of an extraction item name "deadline." The first instruction 1010 includes instruction content 1011 and text information 1012. The instruction content 1011 is composed of an "instruction to extract an item value" (instruction statement 1021 (first instruction statement)), and is a text including the extraction item 1001. The text information 1012 includes text content 1002 to extract the item value of the extraction item 1001. In this embodiment, the text content 1002 describes the OCR result 600 of the document image 500.

[0045] In S902, the character string acquisition unit 354 inputs the generated first instruction 1010 to the large-scale language model 151. In this embodiment, when inputting an instruction to the large-scale language model, a value for Temperature is also set. Temperature is one of the parameters of the large-scale language model 151, and can control the randomness of the output. The higher the Temperature value, the greater the diversity of sentences output by the AI ​​(Artificial Intelligence). The lower the Temperature value, the more consistent the response obtained. The Temperature in the large-scale language model 151 used in this embodiment can be set in the range from 0 to 1. In S902, the value of Temperature is set to 0.8. Furthermore, by inputting the first instruction 1010 multiple times to the large-scale language model 151, the item value for the extraction item "deadline" is acquired 10 times, and the acquired character string is used as a candidate value for the item value. Note that the value of Temperature may be other than 0.8. In order to extract a variety of item values ​​without omission, it is desirable that the value of Temperature be 0.5 or greater.

[0046] In S903, the instruction generation unit 353 generates a second instruction including an instruction statement for selecting an optimal item value from the candidate values ​​of the item value obtained in S902. FIG. 11 shows an example of a second instruction 1110 including an instruction statement for selecting an optimal item value from candidate values ​​of the extracted item name "deadline." The second instruction 1110 includes instruction content 1111, a list of deadline candidates 1112, and text information 1113. The instruction content 1111 is composed of "an instruction to select an optimal item value from candidate item values" (instruction statement 1121 (second instruction statement)) and is a text including the extracted item 1101. The list of deadline candidates 1112 has multiple deadline candidates 1122 and includes the results obtained by inputting the instruction statement multiple times in S902. The text information 1113 is composed of text content 1103 that is referenced to select an optimal item value for the extracted item 1101. In this embodiment, the text content 1103 describes the OCR result 600 of the document image 500.

[0047] In S904, the character string acquisition unit 354 inputs the second instruction 1110 generated in S903 to the large-scale language model 151. In S904, the value of Temperature is set to 0. In the processing of S904, it is necessary to output a consistent result rather than a diverse result, so it is desirable that the value of Temperature be as low as possible. When the processing of S904 is completed, the character string acquisition unit 354 acquires a character string with an optimal item value based on the second instruction 1110.

[0048] In S905, the character string acquisition unit 354 determines whether the item value acquired in S904 is in an appropriate format, and the following process is executed. If the item value extracted in S904 is in an appropriate format, the output unit 355 displays the character string of the extracted item value on the display device 267 of the information processing server 102. In addition, the extracted character string of the item value is transmitted to the information processing device 101 via the second data transmission / reception unit 351 and the first data transmission / reception unit 302. The display control unit 303 of the information processing device 101 displays the extracted character string of the item value on the display device 210.

[0049] If the item value extracted in S904 is not in an appropriate format, in S906 the instruction generation unit 353 generates a third instruction including an instruction statement (third instruction statement) for inferring the item value of the item to be extracted using the document image 500 and the item to be extracted. In S907, the character string acquisition unit 354 inputs the generated third instruction to the large-scale language model 151 to acquire a character string of the item value that conforms to the format of the item to be extracted. The output unit 355 displays the character string of the item value that conforms to the format on the display device 267 of the information processing server 102. In addition, the character string of the item value that conforms to the format is transmitted to the information processing device 101 via the second data transmission / reception unit 351 and the first data transmission / reception unit 302. The display control unit 303 of the information processing device 101 displays the character string of the item value that conforms to the format on the display device 210.

[0050] As described above, according to this embodiment, even if the item values ​​corresponding to the item names to be extracted that are not written in the document image are written in various expressions, it is possible to obtain appropriate item values. Therefore, it is no longer necessary for the user to calculate the item values ​​of items that are not written in the document image by themselves.

[0051] The present disclosure can also be realized by providing a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0052] The disclosure of the above-described embodiment includes the following configurations.

[0053] (Configuration 1) An information processing device for extracting a character string corresponding to a specified item from a document image, comprising: a generation means for generating an instruction sentence used to infer a character string conforming to a format of an item value corresponding to the specified item; an acquisition means for acquiring a character string conforming to the format by inputting the instruction sentence generated by the generation means into a large-scale language model; and an output means for outputting the character string conforming to the format acquired by the acquisition means as the item value.

[0054] (Configuration 2) The information processing device according to Configuration 1, further comprising an extraction means for extracting a character string corresponding to the predetermined item from the document image, wherein the generation means generates the instruction sentence when the character string extracted by the extraction means does not conform to the format of the item value corresponding to the predetermined item.

[0055] (Configuration 3) The information processing device according to configuration 1 or 2, wherein the format of the item value corresponding to the predetermined item includes a date, an amount, a document name, or a document number.

[0056] (Configuration 4) The information processing device according to any one of configurations 1 to 3, further comprising a display device for displaying an output result, wherein the output means outputs the character string according to the format to the display device.

[0057] (Configuration 5) A control method for an information processing device for extracting a character string corresponding to a specified item from a document image, comprising: a step of generating an instruction statement used to infer a character string conforming to a format of an item value corresponding to the specified item; a step of acquiring a character string conforming to the format by inputting the instruction statement generated in the generating step into a large-scale language model; and a step of outputting the character string conforming to the format acquired in the acquiring step as the item value.

[0058] (Configuration 6) A program for causing a computer to function as the information processing device according to any one of configurations 1 to 4.

[0059] (Configuration 7) An information processing system including a first information processing device that transmits a document image and predetermined items for the document image and receives item values ​​corresponding to the predetermined items, and a second information processing device that receives the document image and the predetermined items for the document image and transmits item values ​​corresponding to the predetermined items, wherein the second information processing device is characterized in that it comprises: a generation means that generates an instruction statement used to infer a string that conforms to the format of the item value corresponding to the predetermined item; an acquisition means that acquires the string that conforms to the format by inputting the instruction statement generated by the generation means into a large-scale language model; and an output means that outputs the string that conforms to the format acquired by the acquisition means as the item value.

[0060] (Configuration 8) The information processing system described in Configuration 7, wherein the second information processing device further has an extraction means for extracting a character string corresponding to the specified item from the document image, and the generation means generates the instruction sentence when the character string extracted by the extraction means does not conform to the format of the item value corresponding to the specified item.

[0061] (Configuration 9) The information processing system described in configuration 7 or 8, characterized in that the first information processing device includes a first display device, and the output means outputs a character string conforming to the format to the first display device.

[0062] (Configuration 10) An information processing system described in any one of configurations 7 to 9, characterized in that the second information processing device further includes a second display device, and the output means outputs a character string conforming to the format to the second display device.

[0063] (Configuration 11) An information processing device for extracting a character string corresponding to a specified item from a document image, comprising: a generation means for generating a first instruction sentence using the document image and the specified item to be used in inferring a character string corresponding to the specified item; an acquisition means for acquiring candidate character strings corresponding to the specified item by inputting the first instruction sentence generated by the generation means and a first parameter that controls randomness of a large-scale language model into the large-scale language model multiple times; and an acquisition means for acquiring the most suitable character string for the specified item by inputting a second instruction sentence used to infer a most suitable character string from the candidate character strings corresponding to the specified item using the document image and the candidate character strings corresponding to the specified item and a second parameter having a value smaller than the first parameter into the large-scale language model; and an output means for outputting the most suitable character string as the item value when the most suitable character string acquired by the acquisition means conforms to the format of an item value corresponding to the specified item.

[0064] (Configuration 12) The information processing device described in Configuration 11, characterized in that if the most suitable string acquired by the acquisition means does not conform to the format of the item value corresponding to the specified item, the generation means generates a third instruction sentence using the document image and the specified item to be used to infer a string conforming to the format of the item value corresponding to the specified item, the acquisition means inputs the third instruction sentence into the large-scale language model to acquire a string conforming to the format, and the output means outputs the string conforming to the format as the item value.

Claims

1. An information processing device for extracting character strings corresponding to predetermined items from a document image, a generating means for generating a directive used to infer a character string according to a format of an item value corresponding to the predetermined item; an acquisition means for acquiring a character string conforming to the format by inputting the instruction sentence generated by the generation means into a large-scale language model; an output means for outputting the character string according to the format acquired by the acquisition means as the item value; Equipped with 1. An information processing device comprising:

2. further comprising an extraction unit for extracting a character string corresponding to the predetermined item from the document image; 2. The information processing apparatus according to claim 1, wherein the generating means generates the instruction statement when the character string extracted by the extracting means does not conform to a format of an item value corresponding to the predetermined item.

3. 2. The information processing apparatus according to claim 1, wherein the format of the item value corresponding to the predetermined item includes a date, an amount, a document name, or a document number.

4. Further provided is a display device for displaying the output result, 2. The information processing apparatus according to claim 1, wherein said output means outputs said character string in accordance with said format to said display device.

5. A method for controlling an information processing device for extracting character strings corresponding to predetermined items from a document image, comprising: generating an instruction statement to be used for inferring a character string according to the format of an item value corresponding to the predetermined item; a step of inputting the instruction sentence generated in the generating step into a large-scale language model to obtain a character string conforming to the format; a step of outputting the character string according to the format acquired in the acquiring step as the item value; Equipped with 2. A method for controlling an information processing apparatus comprising:

6. A program for causing a computer to function as the information processing device according to any one of claims 1 to 4.

7. An information processing system including: a first information processing device that transmits a document image and predetermined items for the document image and receives item values ​​corresponding to the predetermined items; and a second information processing device that receives the document image and the predetermined items for the document image and transmits the item values ​​corresponding to the predetermined items, The second information processing device a generating means for generating a directive used to infer a character string according to a format of an item value corresponding to the predetermined item; an acquisition means for acquiring a character string conforming to the format by inputting the instruction sentence generated by the generation means into a large-scale language model; an output means for outputting the character string according to the format acquired by the acquisition means as the item value; Equipped with An information processing system comprising:

8. the second information processing device further includes an extraction unit for extracting a character string corresponding to the predetermined item from the document image; 8. The information processing system according to claim 7, wherein the generating means generates the instruction statement when the character string extracted by the extracting means does not conform to a format of an item value corresponding to the predetermined item.

9. the first information processing device includes a first display device; 8. The information processing system according to claim 7, wherein said output means outputs a character string conforming to said format to said first display device.

10. the second information processing device further includes a second display device; 8. The information processing system according to claim 7, wherein said output means outputs a character string conforming to said format to said second display device.

11. An information processing device for extracting character strings corresponding to predetermined items from a document image, a generating means for generating a first instruction sentence to be used for inferring a character string corresponding to the predetermined item using the document image and the predetermined item; acquiring character string candidates corresponding to the predetermined item by inputting the first directive generated by the generating means and a first parameter that controls randomness of the large-scale language model into the large-scale language model a plurality of times; an acquisition means for acquiring the most suitable character string for the predetermined item by inputting a second directive used to infer the most suitable character string from the candidate character strings corresponding to the predetermined item and a second parameter having a value smaller than the first parameter into the large-scale language model using the document image and candidate character strings corresponding to the predetermined item; an output means for outputting the most suitable character string as the item value when the most suitable character string obtained by the obtaining means conforms to a format of an item value corresponding to the predetermined item; Equipped with 1. An information processing device comprising:

12. If the most suitable character string acquired by the acquisition means does not conform to the format of the item value corresponding to the predetermined item, the generating means generates a third directive used to infer a character string according to a format of an item value corresponding to the predetermined item, using the document image and the predetermined item; the acquiring means acquires a character string conforming to the format by inputting the third directive sentence into the large-scale language model; the output means outputs a character string conforming to the format as the item value.

12. The information processing apparatus according to claim 11,

Images