Information processing apparatus, information processing method, and program

Abstract

To provide an information processing apparatus, an information processing method, and a program which extract a desired item from an informal document without preparing various pieces of learning data.SOLUTION: In a method, an information processing apparatus performs: an acquisition step of acquiring image data from a terminal; a text request step of transmitting the acquired image data to OCR means to request the OCR means to extract a text from the image data; an item request step of generating a prompt for item extraction which gives an instruction to output at least one extracted item from the text extracted in response to the request, and outputting the prompt for item extraction to generative AI means; and a generation step of acquiring an extracted item generated by processing the prompt in the generative AI means and generating output information based on the extracted item.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] A technology (also called non-standard data analysis technology) has been developed that uses computer processing to extract desired items from non-standard documents (such as non-standard forms and non-standard contracts) in which the positions of items within the document are not uniquely determined (see, for example, Patent Document 1). Here, non-standard forms are forms whose format (or layout) is company-dependent and whose format (or layout) varies from form to form. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-009598 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional unstructured data analysis techniques have the problem that various training data must be prepared in order to extract desired items from documents in various formats.

[0005] The present invention has been made in consideration of the above-mentioned problems, and aims to provide an information processing device, an information processing method, and a program that enable desired items to be extracted from unstructured documents without preparing various learning data. [Means for solving the problem]

[0006] An information processing device according to a first aspect of the present invention is an information processing device capable of exchanging information with an OCR means that extracts text from an image by optical character recognition and a generation AI means that executes a generation AI that outputs text data in response to input, and is equipped with at least one processor, wherein the at least one processor executes an acquisition procedure that acquires image data from a terminal, a text request procedure that sends the acquired image data to the OCR means and requests the OCR means to extract text from the image data, an item request procedure that generates an item extraction prompt that instructs the OCR means to output at least one extracted item from the text extracted by the request and outputs the item extraction prompt to the generation AI means, and a generation procedure that acquires the extracted items output after the prompt is processed by the generation AI means and generates output information based on the extracted items.

[0007] According to this configuration, the generation AI means outputs the desired items from the text extracted from the unstructured document by the OCR means, so that the desired items can be extracted from the unstructured document without preparing various training data.

[0008] An information processing device according to a second aspect of the present invention is an information processing device according to the first aspect, wherein the at least one processor is capable of referencing a storage device in which at least one extraction item is stored in association with a document type, and before requesting the extraction of the extraction item, further executes a document type request procedure in which the extracted text is output to a generation AI means and a request is made to determine the document type from the text, and in the item request procedure, the storage device is referenced to read an extraction item corresponding to the determined document type, and a prompt for item extraction is generated which includes a prompt instructing the read extraction item to be output from the text, and the prompt for item extraction is output to the generation AI means.

[0009] According to this configuration, appropriate extraction items can be extracted depending on the type of document.

[0010] An information processing device according to a third aspect of the present invention is an information processing device according to the first aspect, wherein the at least one processor is capable of referencing a storage device in which at least one extraction item is stored in association with a document type, and before requesting the extraction of the extraction item, further executes a document discrimination procedure that determines the document type from the extracted text by pattern matching, and when the item request procedure requests the extraction of an extraction item, references the storage device to read out the extraction item corresponding to the determined document type, generates the item extraction prompt including a prompt instructing the device to output the read extraction item from the text, and outputs the item extraction prompt to a generation AI means.

[0011] According to this configuration, appropriate extraction items can be extracted depending on the type of document.

[0012] An information processing device according to a fourth aspect of the present invention is an information processing device according to any one of the first to third aspects, wherein the at least one processor is capable of referencing a storage device in which customer identification information and a customer-specific prompt instructing the output of additional data are associated and stored, and in the acquisition procedure, the information processing device acquires customer identification information in addition to the image data, and in the item request procedure, the information processing device refers to the storage device to read out the customer-specific prompt corresponding to the acquired customer identification information, generates the item extraction prompt to include the customer-specific prompt, and outputs the item extraction prompt to the generation AI means.

[0013] According to this configuration, a prompt for item extraction is generated that includes a customer-specific prompt according to the customer identification information, so that the additional extraction items can be changed depending on the customer.

[0014] An information processing device according to a fifth aspect of the present invention is an information processing device according to the fourth aspect, wherein the customer-specific prompt is a prompt that instructs the device to search the text to see if an object exists, and if so, to output the search results.

[0015] According to this configuration, it is possible to search the text to see if an object exists, and if it does, output the search results.

[0016] An information processing device according to a sixth aspect of the present invention is an information processing device according to any one of the first to fifth aspects, wherein the at least one processor, in the text request procedure, requests extraction of coordinates of each character string contained in the text when requesting extraction of the text, and in the generation procedure, generates a pair of the extracted item and the coordinate as the output information by comparing the extracted item processed and output by the generation AI means with the coordinate extracted by the OCR means.

[0017] According to this configuration, data sets of extracted items and their coordinates are obtained, so that processing can be performed on the image area of ​​the target extracted item (for example, extraction processing of this image area, masking of this image area, etc.).

[0018] An information processing device according to a seventh aspect of the present invention is an information processing device according to the sixth aspect, wherein the at least one processor executes a process of extracting an image area of ​​the target extraction item from the image data using the coordinate data associated with the target extraction item.

[0019] According to this configuration, it is possible to obtain an image in which the image area of ​​the extracted item is marked, or to obtain image data in which the image area of ​​the extracted item is cut out.

[0020] An information processing device according to an eighth aspect of the present invention is an information processing device according to the sixth aspect, wherein the at least one processor generates image data in which the extracted items to be masked are masked from the image data using coordinates associated with the extracted items to be masked.

[0021] According to this configuration, it is possible to acquire image data in which the image area of ​​the extraction item to be masked is masked.

[0022] An information processing method according to a ninth aspect of the present invention includes an acquisition procedure for acquiring image data from a terminal, a text request procedure for sending the acquired image data to an OCR means and requesting the OCR means to extract text from the image data, an item request procedure for generating an item extraction prompt that instructs the OCR means to output at least one extracted item from the text extracted by the request and outputting the item extraction prompt to a generation AI means, and a generation procedure for acquiring the extracted items output when the prompt is processed by the generation AI means and generating output information based on the extracted items.

[0023] A program according to a tenth aspect of the present invention is a program for causing a computer to execute the following steps: an acquisition procedure for acquiring image data from a terminal; a text request procedure for sending the acquired image data to an OCR means and requesting the OCR means to extract text from the image data; an item request procedure for generating an item extraction prompt that instructs the OCR means to output at least one extracted item from the text extracted by the request and outputting the item extraction prompt to a generation AI means; and a generation procedure for acquiring the extracted items output when the prompt is processed by the generation AI means and generating output information based on the extracted items. [Effects of the Invention]

[0024] According to one aspect of the present invention, the generation AI means outputs the desired items from the text extracted from the unstructured document by the OCR means, so that the desired items can be extracted from the unstructured document without preparing various training data. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a schematic configuration diagram of an information processing system according to an embodiment of the present invention. [Figure 2] 1 is a diagram illustrating an example of a schematic configuration of an information processing apparatus according to an embodiment of the present invention. [Figure 3]1 is an example of a prompt dictionary stored in a storage device of an information processing device. [Figure 4] FIG. 2 is a schematic diagram showing an outline of a processing flow in the first embodiment. [Figure 5] 1 is a flowchart illustrating an example of a processing flow in the first embodiment. [Figure 6A] 1 is an example of an item extraction prompt. [Figure 6B] 10 is an example of extracted item data. [Figure 6C] 10 is an example of output information. [Figure 7] 10 is an example of a table stored in a storage device in the second embodiment. [Figure 8] 10 is a flowchart illustrating an example of a processing flow in the second embodiment. [Figure 9] 11 is a flowchart illustrating an example of a processing flow in a third embodiment. [Figure 10A] 10 is an example of data of pairs of extracted items and their coordinates generated by the information processing device. [Figure 10B] An image in which extracted items are marked. [Figure 11] 10 is a flowchart showing an example of a processing flow in the fourth embodiment. [Figure 12A] 13 is an example of an item extraction prompt in the fourth embodiment. [Figure 12B] 13 is an example of extracted item data in the fourth embodiment. [Figure 12C] 13 is an example of output information in the fourth embodiment. [Figure 12D] 13 is an example of an image after a masking process in the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0026] Hereinafter, each embodiment will be described with reference to the drawings. However, unnecessary detailed description may be omitted. For example, detailed description of well-known matters or redundant description of substantially the same configuration may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art.

[0027] First Embodiment Fig. 1 is a schematic diagram of an information processing system according to the present embodiment. As shown in Fig. 1, an information processing system S according to the first embodiment includes, as an example, terminals 1-1 to 1-N (N is a natural number), an information processing device 2 connected to each of the terminals 1-1 to 1-N via a communication network CN, an OCR means 3 connected to the information processing device 2 via the communication network CN, and a generation AI means 4 connected to the information processing device 2 via the communication network CN.

[0028] Terminals 1-1 to 1-N are used by different users and are, for example, mobile phones such as multi-function mobile phones (so-called smartphones), tablets, notebook computers, desktop computers, etc. Hereinafter, terminals 1-1 to 1-N will also be collectively referred to as terminal 1.

[0029] The information processing device 2 is used by, for example, an administrator who manages the information processing system S according to this embodiment, and is, for example, a server. The information processing device 2 provides information in response to a request from a terminal 1-i (i is an index and is an integer from 1 to N). The information processing device 2 may be a single computer or may be composed of multiple computers.

[0030] The OCR means 3 extracts text from an image using optical character recognition. The OCR means 3 is, for example, Azure AI Vision. The generation AI means 4 executes a generation AI that outputs text data in response to input. The generation AI means 4 is, for example, Azure OpenAI Service.

[0031] Fig. 2 is an example of a schematic configuration diagram of an information processing device in this embodiment. As shown in Fig. 2, the information processing device 2 includes, as an example, an input interface 21, a communication module 22, a storage device 23, a memory 24, an output interface 25, and a processor 26. Note that, in this embodiment, the information processing device 2 is described as including one processor 26, but is not limited to this, and may include multiple processors, as long as it includes at least one processor.

[0032] The input interface 21 receives an input from a manager of the information processing device 2 (for example, an employee of a management organization), and outputs an input signal to the processor 26 in accordance with the received input. The communication module 22 is connected to the communication network CN and communicates with the terminals 1-1 to 1-N, the OCR means 3, and the generating AI means 4. This communication may be wired or wireless, but the following description will be given assuming that it is wired.

[0033] The storage device 23 stores programs and various data that the processor 26 reads and executes. The memory 24 temporarily stores data and programs and is a volatile memory, such as a random access memory (RAM). The output interface 25 can be connected to an external device and can output a signal to the external device.

[0034] The processor 26 loads a program from the storage device 23 into the memory 24 and executes a series of instructions contained in the program.

[0035] Figure 3 shows an example of a prompt dictionary stored in the storage device of an information processing device. As shown in prompt dictionary T1 in Figure 3, pairs of keys and prompts are stored in storage device 23. For example, a key "common to all customers" is associated with a prompt that "instructs to search the contents to see if an in-house control number exists, and if so, to return all search results in a specified format." This allows the above corresponding prompt to be executed commonly for all customers. Similarly, for example, a key such as "Customer ID_0001" is associated with a prompt that instructs "to search the contents to see if there are any stamped areas, and if so, to return all search results in a specified format." Here, the customer ID is an example of customer identification information that identifies a customer. This allows the corresponding prompt to be executed when the customer ID is "0001."

[0036] For example, a key called "common for automatic extraction" is stored in association with a corresponding prompt, which allows the corresponding prompt to be executed when marking or extracting the image area of ​​the target extracted item from the image data of the target unformatted document. For example, a key called "common masking" is stored in association with a corresponding prompt, so that when masking a target extracted item from image data of a target unstructured document, the corresponding prompt can be executed.

[0037] Example 1 FIG. 4 is a schematic diagram showing an overview of the processing flow in Example 1. The OCR means 3 extracts characters from an image S1 of a form (specifically, a receipt), which is an example of a document, by performing OCR on the image S1. As a result, a character string S2 is extracted. FIG. 4 shows an example of extracted items S3 output by the generation AI means 4 after processing a prompt instructing the generation AI means 4 to output at least one extracted item (specifically, for example, the document name, transaction date, client, and amount) from the character string S2. These extracted items are then input into text boxes B1 to B4 of the form S4 by the information processing device 2.

[0038] FIG. 5 is a flowchart showing an example of the processing flow in the first embodiment. FIG. 6A is an example of a prompt for item extraction. FIG. 6B is an example of extracted item data. FIG. 6C is an example of output information. The processing flow will be described below along FIG. 5 with reference to FIGS. 6A to 6C. In the first embodiment, the type of document is known, and the information processing device 2 processes the document assuming that the document type is a receipt, for example. In addition, all the flowcharts below describe the processing of the processor, and the description of the processor is omitted.

[0039] (Step S111) First, the terminal 1-i receives from the user an API request operation for an API (Application Programming Interface) that executes automatic input of items.

[0040] (Step S112) Next, the terminal 1-i makes an API request using the customer ID along with image data of the target unstructured document (here, a receipt as an example).

[0041] (Step S211) When the information processing device 2 receives an API request, it requests the OCR means 3 to extract text from image data.

[0042] (Step S311) When the OCR means 3 receives a request from the information processing device 2, it extracts text from the image data by OCR processing and transmits the extracted text data to the information processing device 2.

[0043] (Step S212) The information processing device 2 reads out a customer-specific prompt corresponding to the customer ID received in step S112 from the prompt dictionary stored in the storage device 23. Here, the customer-specific prompt is a prompt that instructs the output of additional data. Specifically, for example, the customer-specific prompt is a prompt that instructs the search to be performed to search for the presence of an object (e.g., a stamped area) from the text (extracted by the OCR means 3) and, if present, to output the search results (e.g., in a specific format). Here, as an example, if the customer ID is "0001," the prompt shown in Figure 3 is read out, which instructs the search to "search for the presence of a stamped area from the content and, if present, to return all search results in a specific format."

[0044] (Step S213) Next, the information processing device 2 generates an item extraction prompt that includes a basic prompt that instructs the device to extract basic extraction items from the text and a customer-specific prompt. This generates, for example, the item extraction prompt shown in FIG. 6A. While the item extraction prompt shown in FIG. 6A includes the above text as an example, the text may be transmitted separately from the information processing device 2 before or after the item extraction prompt. Since the basic extraction items change depending on the document type (e.g., the type of form), the basic prompt changes depending on the document type (e.g., the type of form), and as a result, the item extraction prompt changes. In the example of FIG. 6A, a receipt is the target, and the basic extraction items are, for example, the form classification, transaction date, client, and amount. While four basic extraction items are shown here as an example, at least one extraction item is sufficient.

[0045] (Step S214) Next, the information processing device 2 transmits the generated item extraction prompt to the generation AI means 4.

[0046] (Step S411) Next, when the generation AI means 4 receives an item extraction prompt from the information processing device 2, it processes the item extraction prompt to generate extracted item data. As a result, for example, extracted item data shown in Fig. 6B is generated.

[0047] (Step S412) Next, the generation AI means 4 transmits the generated extracted item data to the information processing device 2.

[0048] (Step S215) When the information processing device 2 receives the extracted item data, it converts the extracted item data into output information in a predetermined format, thereby generating output information in a JSON format, for example, as shown in Fig. 6C.

[0049] (Step S216) Next, the information processing device 2 transmits the generated output information to the terminal 1-i as an API response.

[0050] (Step S113) When the terminal 1-i receives the output information, it inserts the output information into the form items, thereby automatically filling in the form items.

[0051] As described above, in the first embodiment, the information processing device 2 can exchange information with the OCR means 3 that extracts text from an image by optical character recognition and the generation AI means 4 that executes the generation AI that outputs text data in response to an input.

[0052] The information processing device 2 has at least one processor, which executes an acquisition procedure for acquiring image data from terminal 1-i, a text request procedure for sending the acquired image data to OCR means 3 and requesting the OCR means 3 to extract text from the image data, an item request procedure for generating an item extraction prompt that instructs the OCR means 3 to output at least one extracted item from the text extracted by the request and outputting the item extraction prompt to generation AI means 4, and an output procedure for acquiring the extracted items output after the prompt is processed by the generation AI means 4, and outputting output information based on the extracted items.

[0053] According to this configuration, the generation AI means 4 outputs the desired items from the text extracted from the unstructured document by the OCR means 3, so that the desired items can be extracted from the unstructured document without preparing various learning data.

[0054] In the first embodiment, at least one processor can refer to a storage device in which customer identification information (e.g., a customer ID) and a customer-specific prompt instructing to output additional extracted items are stored in association with each other. In the acquisition step, the at least one processor acquires the customer identification information in addition to the image data, and in the item request step, it refers to the storage device 23 to read the customer-specific prompt corresponding to the acquired customer identification information, generates the item extraction prompt so as to include the customer-specific prompt, and outputs the item extraction prompt to the generation AI means 4.

[0055] According to this configuration, a prompt for item extraction is generated that includes a customer-specific prompt according to the customer identification information, so that the additional extraction items can be changed depending on the customer.

[0056] <Example 2> Next, a description will be given of Example 2. In Example 1, the type of document is known, and the information processing device 2 processes the document on the assumption that the document type is a receipt, for example. In Example 2, the type of document is unknown, and the information processing device 2 changes the extracted items depending on the document type (for example, a contract, an invoice, a receipt, etc.).

[0057] Fig. 7 is an example of a table stored in the storage device in Example 2. As shown in Fig. 7, document types and at least one extracted item are stored in association with each other in the storage device 23. This allows the processor 26 to refer to the storage device 23 and read out the extracted item corresponding to the document type.

[0058] 8 is a flowchart showing an example of the flow of processing in the second embodiment. Processing different from the processing in the flowchart of the first embodiment shown in FIG. 5 will be described.

[0059] Steps S111, S112, S211, and S311 in the second embodiment of FIG. 8 are the same as those in the first embodiment of FIG. 5, and therefore the description thereof will be omitted.

[0060] (Step S221) The information processing device 2 transmits to the AI ​​generation means 4 a prompt requesting discrimination of the type of document from the text of the text data received in step S311.

[0061] (Step S421) When the generation AI means 4 receives a prompt, it processes the prompt and outputs the document type.

[0062] (Step S422) The generation AI means 4 transmits the type of the output document to the information processing device 2.

[0063] Step S212 in the second embodiment of FIG. 8 is the same as that in the first embodiment of FIG. 5, and therefore the description thereof will be omitted.

[0064] (Step S221) Next, the information processing device 2 refers to the storage device 23 and reads out the extraction items corresponding to the determined document type as basic extraction items. Then, the information processing device 2 generates an item extraction prompt. This item extraction prompt includes a basic prompt that instructs the device to output the read basic extraction items from the text, and the customer-specific prompt read out in step S212.

[0065] Steps S214, S411, S412, S215, S216, and S113 in the second embodiment of FIG. 8 are the same as those in the first embodiment of FIG. 5, and therefore will not be described again.

[0066] As described above, in the second embodiment, at least one processor can refer to a storage device in which at least one extraction item is stored in association with a document type. Before requesting extraction of the extraction item, the at least one processor further executes a document type request procedure in which the extracted text is output to the generation AI means 4 and a document type is determined from the text. In the item request procedure, the at least one processor refers to the storage device 23 to read an extraction item corresponding to the determined document type, generates a prompt for item extraction including a prompt to output the read extraction item from the text, and outputs the prompt for item extraction to the generation AI means 4.

[0067] <Modification of Example 2> In the second embodiment, the generation AI means 4 determines the document type, but this is not limiting and the determination may be made by pattern matching. Specifically, for example, at least one processor may execute a document determination procedure that determines the document type from the extracted text by pattern matching before requesting the extraction of the extraction items. In this case, when the item request procedure requests the extraction of extraction items, the at least one processor may refer to the storage device 23 to read the extraction items corresponding to the determined document type, and request the generation AI means 4 to extract the read extraction items from the text.

[0068] Example 3 Next, a third embodiment will be described. In the third embodiment, a process is executed to extract an image area of ​​a target extraction item from image data. This process involves, for example, marking the image area on the image data, or cutting out the area and generating image data of only that image area. Here, an example of marking the image area on the image data will be described as an example.

[0069] Fig. 9 is a flowchart showing an example of a processing flow in the third embodiment. A process different from the process of the flowchart in the first embodiment shown in Fig. 5 will be described. Fig. 10A is an example of data of pairs of extracted items and their coordinates generated by an information processing device. Fig. 10B is an image in which extracted items are marked. An example of the processing flow will be described below using FIG. 9 while also referring to FIGS. 10A and 10B.

[0070] (Step S131) ​​First, the terminal 1-i receives from the user an API request operation for an API that executes automatic extraction of an item.

[0071] (Step S112) Next, the terminal 1-i makes an API request using the customer ID along with image data of the target unstructured document (here, a receipt as an example).

[0072] (Step S231) When the information processing device 2 receives the API request of step S112, it requests the OCR means 3 to extract the text and the coordinates of each character string contained in the text from the image data. Here, the coordinates of each character string are, for example, the coordinates of two diagonal corners of the four corners when the character string is enclosed in a square.

[0073] (Step S331) When the OCR means 3 receives a request from the information processing device 2, it extracts text and the coordinates of each character string contained in the text from the image data by OCR processing, and transmits the extracted text data and coordinate data to the information processing device 2.

[0074] Steps S211, S213, S213, S214, S411, and S412 in the third embodiment of FIG. 9 are the same as those in the first embodiment of FIG. 5, and therefore the description thereof will be omitted.

[0075] (Step S232) Next, when the information processing device 2 receives extracted item data from the generation AI means 4, it compares the received extracted item data with the coordinate data received in step S331 and generates data of pairs of the extracted items of the received extracted item data and their coordinates as output information. As a result, data of pairs of the extracted items and their coordinates is generated, for example, as shown in FIG. 10A.

[0076] (Step S233) Next, the information processing device 2 generates image data in which image areas of the extracted items are marked from the image. As a result, an image in which the extracted items are marked is generated, for example, as shown in Fig. 10B. Here, in Fig. 10B, image areas R1 to R5 of the extracted items are marked with dashed squares, as an example.

[0077] (Step S234) Next, the information processing device 2 transmits the generated image data to the terminal 1-i as an API response.

[0078] (Step S131) ​​When the terminal 1-i receives image data, for example, it displays the image data. As a result, an image with the extracted items marked is displayed, for example, as shown in Fig. 10B. As the extracted items are marked, the user of the terminal (for example, an operator who inputs or confirms the extracted items) can easily confirm the extracted items.

[0079] As described above, in the third embodiment, at least one processor requests extraction of the coordinates of each character string included in the text when requesting extraction of the text in the text request procedure, as described in step S231 of Fig. 9. Then, in the generation procedure, the at least one processor generates pairs of the extracted items and their coordinates as the output information by comparing the extracted items processed and output by the generation AI means 4 with the coordinates extracted by the OCR means 3. This provides data of pairs of the extracted items and their coordinates, allowing processing to be performed on the image area of ​​the target extracted items (e.g., extraction processing of this image area, masking of this image area, etc.).

[0080] Then, at least one processor executes a process of extracting an image area of ​​the target extracted item from the image data using the coordinate data associated with the target extracted item, thereby obtaining an image in which the image area of ​​the extracted item is marked, or obtaining image data in which the image area of ​​the extracted item is cut out.

[0081] Example 4 Next, a description will be given of Example 4. In Example 4, the target extraction items are subjected to masking processing.

[0082] FIG. 11 is a flowchart showing an example of a processing flow in the fourth embodiment. Processing different from the processing of the flowchart in the third embodiment in FIG. 9 will be described. FIG. 12A is an example of a prompt for item extraction in the fourth embodiment. FIG. 12B is an example of extracted item data in the fourth embodiment. FIG. 12C is an example of output information in the fourth embodiment. FIG. 12D is an example of an image after masking processing in the fourth embodiment.

[0083] (Step S141) First, the terminal 1-i receives from the user an API request operation for an API that executes masking processing on an image. Here, the masking processing is, for example, masking processing on a specific extraction item (for example, personal information).

[0084] (Step S112) Next, the terminal 1-i makes an API request using the customer ID along with image data of the target unstructured document (here, a receipt as an example).

[0085] Steps S211 and S331 in the fourth embodiment of FIG. 11 are the same as those in the third embodiment of FIG. 9, and therefore the description thereof will be omitted.

[0086] (Step S241) Next, the information processing device 2 refers to a customer-specific prompt corresponding to the customer ID in the prompt dictionary. Here, as an example, the following description will be given assuming that there is no customer-specific prompt corresponding to the target customer ID in the prompt dictionary.

[0087] (Step S242) Next, the information processing device 2 generates a prompt for item extraction. As a result, for example, the prompt for item extraction shown in Fig. 12A is generated. Here, in the example of Fig. 12A, it is assumed that the extraction items are predetermined as name, address, and presence or absence of a facial photograph in the API that executes this masking process.

[0088] Steps S214, S411, S412, and S232 in Example 4 of Fig. 12 are the same as those in Example 3 of Fig. 9, and therefore will not be described again. Here, by the processing of step S411, for example, extracted item data as shown in Fig. 12B is generated. In this way, each extracted item, name, address, and presence or absence of a facial photograph, is obtained. Furthermore, by the processing of step S232, for example, output information as shown in Fig. 12C is generated. In this way, each extracted item, name, address, and presence or absence of a facial photograph, and a set of their coordinates are obtained.

[0089] (Step S243) Next, the information processing device 2 generates image data in which the extraction item to be masked is masked from the image data, using the coordinates associated with the extraction item to be masked.

[0090] (Step S244) Next, the information processing device 2 transmits the generated image data to the terminal 1-i as an API response.

[0091] (Step S141) When the terminal 1-i receives image data, the terminal 1-i displays the image data, for example. As a result, an image in which the target extracted items (for example, name, address, and facial photo) are masked is displayed, as shown in FIG. 12D.

[0092] As described above, in the fourth embodiment, in the text request step, at least one processor requests extraction of the coordinates of each character string included in the text when requesting extraction of the text. Then, in the generation step, the at least one processor compares the extracted items processed and output by the generation AI means with the coordinates extracted by the OCR means to generate pairs of the extracted items and the coordinates as the output information. Then, the at least one processor uses the coordinates associated with the extracted items to be masked to generate image data in which the extracted items to be masked are masked from among the image data. This makes it possible to obtain image data in which the image areas of the extracted items to be masked are masked.

[0093] At least a part of the information processing device 2 described in the above embodiment may be configured with hardware or software. If configured with software, a program that realizes at least a part of the functions of the information processing device 2 may be stored in a computer-readable recording medium and read and executed by a computer. The recording medium is not limited to removable media such as magnetic disks and optical disks, but may also be fixed recording media such as hard disk drives and memories.

[0094] In addition, a program that realizes at least a part of the functions of the information processing device 2 may be distributed via a communication line (including wireless communication) such as the Internet. Furthermore, the program may be encrypted, modulated, or compressed and distributed via a wired line or wireless line such as the Internet, or stored on a recording medium.

[0095] Furthermore, the information processing device 2 may be functioned by one or more information devices. When multiple information devices are used, at least one of the devices may be a computer, and the computer may execute a predetermined program to realize the functions as at least one means of the information processing device 2.

[0096] In the method invention, all processes (steps) may be realized by automatic control using a computer. Alternatively, each process may be performed by a computer, with progress control between processes being performed manually. Furthermore, at least some of the processes may be performed manually.

[0097] As described above, the present invention is not limited to the above-described embodiments, and the components can be modified and embodied in practice without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined. [Explanation of symbols]

[0098] 1-1, ..., 1-N terminals 2. Information processing equipment 21 Input Interface 22 Communication Module 23 Storage device 24 memory 25 Output Interface 26 processors 3 OCR means 4 Generation AI means S Information Processing System

Claims

1. An information processing device capable of exchanging information with an OCR means for extracting text from an image by optical character recognition and a generation AI means for executing a generation AI that outputs text data in response to an input, at least one processor; The at least one processor: An acquisition procedure for acquiring image data from a terminal; a text request step of transmitting the acquired image data to an OCR means and requesting the OCR means to extract text from the image data; an item request procedure for generating an item extraction prompt instructing output of at least one extracted item from the text extracted by the request, and outputting the item extraction prompt to the generating AI means; a generation procedure for acquiring extracted items output by the prompt processing by the generation AI means, and generating output information based on the extracted items; An information processing device that executes the above.

2. the at least one processor is capable of referencing a storage device in which at least one extracted item is stored in association with a document type; Before requesting the extraction of the extracted items, a document type request procedure is further executed to output the extracted text to a generating AI means and request that the document type be determined from the text; In the item request step, the storage device is referenced to read out extracted items corresponding to the determined document type, a prompt for item extraction is generated including a prompt for instructing to output the read extracted items from the text, and the prompt for item extraction is output to the generation AI means. The information processing device according to claim 1 .

3. the at least one processor is capable of referencing a storage device in which at least one extracted item is stored in association with a document type; before requesting extraction of the extracted items, further executing a document identification procedure for identifying a document type from the extracted text by pattern matching; When the item request procedure requests extraction of an extraction item, the method refers to the storage device to read out an extraction item corresponding to the determined document type, generates a prompt for item extraction including a prompt to instruct outputting the read-out extraction item from the text, and outputs the prompt for item extraction to the generation AI means. The information processing device according to claim 1 .

4. the at least one processor is operable to access a storage device that stores customer identification information and associated customer-specific prompts for outputting additional data; In the acquisition step, customer identification information is acquired in addition to the image data, In the item request step, the storage device is referenced to read out a customer-specific prompt corresponding to the acquired customer identification information, the item extraction prompt is generated so as to include the customer-specific prompt, and the item extraction prompt is output to the generation AI means. The information processing device according to claim 1 .

5. The customer-specific prompt is a prompt that instructs the user to search the text for the presence of an object and, if present, to output the search results. The information processing device according to claim 4 .

6. In the text requesting step, the at least one processor requests extraction of coordinates of each character string included in the text when requesting extraction of the text; In the generation procedure, the extracted items processed and output by the generation AI means are compared with the coordinates extracted by the OCR means, and a pair of the extracted items and the coordinates is generated as the output information. The information processing device according to claim 1 .

7. The at least one processor: A process of extracting an image area of ​​the target extraction item from the image data is executed using the coordinate data associated with the target extraction item. The information processing device according to claim 6 .

8. The at least one processor: Using the coordinates associated with the extracted item to be masked, image data is generated in which the extracted item to be masked is masked from the image data. The information processing device according to claim 6 .

9. An acquisition procedure for acquiring image data from a terminal; a text request step of transmitting the acquired image data to an OCR means and requesting the OCR means to extract text from the image data; an item request procedure for generating an item extraction prompt instructing output of at least one extracted item from the text extracted by the request, and outputting the item extraction prompt to the generating AI means; a generation procedure for acquiring extracted items output by the prompt processing by the generation AI means, and generating output information based on the extracted items; An information processing method comprising:

10. On the computer, An acquisition procedure for acquiring image data from a terminal; a text request step of transmitting the acquired image data to an OCR means and requesting the OCR means to extract text from the image data; an item request procedure for generating an item extraction prompt instructing output of at least one extracted item from the text extracted by the request, and outputting the item extraction prompt to the generating AI means; a generation procedure for acquiring extracted items output by the prompt processing by the generation AI means, and generating output information based on the extracted items; A program to execute.

Images