LLM utilization system, LLM service provision device, and LLM service provision method

Abstract

The present invention involves: a prompt generation unit 12 that divides original data including confidential information into a plurality of divided pieces of data, replaces the confidential information with other information, and generates a plurality of prompts respectively for the plurality of divided pieces of data; and a target data generation unit 13 that uses a plurality of sets of output information, which are obtained by entering the plurality of prompts into an LLM 200, and association information, which is stored in an association information storage unit 16 when the confidential information is replaced with said other information, to restore the original confidential information before the replacement with respect to the plurality of sets of output information or the plurality of divided pieces of data, and generates target data using the restored information. The confidential information of the original data is not contained in the prompts generated using the original data, and furthermore a prompt is generated for each of the plurality of divided pieces of data obtained by dividing the original data into pieces, thereby making it possible to prevent the confidential information from leaking due to each prompt entered into the LLM 200 on the Internet.

Description

LLM utilization system, LLM service providing device, and LLM service providing method

[0001] The present invention relates to an LLM utilization system, an LLM service providing device, and an LLM service providing method, and is particularly suitable for use in a system that uses an LLM to provide a predetermined data processing service.

[0002] Conventionally, systems have been provided that perform natural language processing such as question-and-answering using large language models (LLMs). LLMs are natural language processing models trained using large amounts of text data, and take sentences as input and output sentences. When LLMs are applied to a question-and-answering system, when a question is input to the LLM, an answer is output from the LLM. In an LLM, the input sentence is called a "prompt."

[0003] Since highly accurate LLMs are large models, they require numerous expensive GPU computing resources to process them without delay. Therefore, it is difficult to process them on a local personal computer, and it is common to send prompts to an LLM service built on the Internet to obtain answers.

[0004] However, one of the most common risks associated with using LLM services is the risk of personal information and trade secrets being leaked. It is undesirable to disclose highly confidential information, such as personal information and trade secrets, to the public. Therefore, it has been difficult to generate prompts containing confidential information and use LLM services on the Internet.

[0005] In addition, there is a known technology that prevents the leakage of patient personal information in a system in which electronic receipt data created on a local terminal is sent to an external audit center connected via a network, allowing receipt checking to be performed externally (see, for example, Patent Document 1).

[0006] In the system described in Patent Document 1, the terminal conceals information recorded in the electronic medical receipt data that can identify individual patients by converting it into different information according to a specified algorithm. At this time, the terminal records the concealed personal information in a storage means in association with an identifier that indicates which data unit of the electronic medical receipt data the information was recorded in. The terminal then reads the personal information from the storage means according to the identifier recorded in the audit result data received from the audit center, and restores the personal information accordingly.

[0007] Japanese Patent Application Laid-Open No. 2006-134077

[0008] The present invention has been made to solve the above-mentioned problems, and has as its object to prevent the leakage of confidential information when using LLM services on the Internet.

[0009] In order to solve the above-mentioned problems, the present invention divides original data containing secret information into multiple pieces, replaces the secret information with other information, and generates multiple prompts for each of the multiple pieces of split data.Then, using multiple pieces of output information obtained by inputting the multiple prompts into a large-scale language model and linking information stored when replacing the secret information with other information, the secret information before replacement is restored for the multiple pieces of output information or the multiple pieces of split data, and target data is generated using the restored information.

[0010] According to the present invention configured as described above, confidential information of the original data is not included in the prompts generated using the original data, and prompts are generated for each of the fragmented data pieces obtained by dividing the original data into multiple pieces. Therefore, even when using an LLM service on the Internet, it is possible to prevent confidential information from being leaked by the individual prompts input into the large-scale language model.

[0011] 1 is a diagram showing an example of the overall configuration of an LLM utilization system according to this embodiment. FIG. 2 is a block diagram showing an example of the functional configuration of a service providing server according to this embodiment. FIG. 3 is a diagram illustrating an example of a portion of electronic medical record data input by an original data input unit according to this embodiment. FIG. 4 is a diagram illustrating an example of divided data generated from electronic medical record data by a prompt generation unit according to this embodiment. FIG. 5 is a diagram illustrating an example of exemplary information stored in an exemplary information storage unit according to this embodiment. FIG. 6 is a diagram illustrating an example of one prompt generated from one divided data by a prompt generation unit according to this embodiment. FIG. 7 is a diagram illustrating an example of information output when the prompt illustrated in FIG. 6 is input to an LLM. FIG. 8 is a diagram illustrating the result of restoring diagnostic information before replacement for structured data elements included in output information from the LLM shown in FIG. 9. FIG. 10 is a diagram illustrating an example of structured data generated by a target data generation unit according to this embodiment. FIG. 11 is a flowchart illustrating an example of the operation of a service providing server according to this embodiment.

[0012] An embodiment of the present invention will be described below with reference to the drawings. Fig. 1 is a diagram showing an example of the overall configuration of an LLM utilization system according to this embodiment. As shown in Fig. 1, the LLM utilization system of this embodiment includes a service providing server 100, an LLM (large-scale language model) 200, and a user terminal 300. The service providing server 100, the LLM 200, and the user terminal 300 are connected via a communication network 400 such as the Internet and / or a mobile phone network.

[0013] The service providing server 100 corresponds to an LLM service providing device in the claims, and provides an LLM service to a user terminal 300. The service providing server 100 of this embodiment receives original data including confidential information from the user terminal 300, generates a prompt based on the original data, inputs the prompt into the LLM 200, and generates target data using the information output from the LLM 200 as a result, and provides the target data to the user terminal 300.

[0014] Confidential information includes, for example, personal information, trade secrets of companies, and transaction information of companies and financial institutions. Personal information includes personal attribute information that can identify a specific individual, such as name, date of birth, age, gender, and address, as well as personal health information and patient information of medical institutions. Trade secrets of companies include confidential information related to products, technology, and sales. Transaction information of companies and financial institutions includes commercial transaction information between companies, and customer transaction and account information at banks, stock exchanges, etc. Note that the confidential information listed here is merely an example and is not limited to these.

[0015] In the following, we will explain the case where the confidential information is personal information as an example, particularly when the original data is electronic medical record data that contains the patient's personal attribute information and diagnosis information (patient information including medical records and diagnosis results by doctors, measurement results by testing equipment, etc.).

[0016] In the following, an example will be described in which the service providing server 100 inputs unstructured data from an electronic medical record as source data from the user terminal 300, converts the unstructured data into structured data using the LLM 200, and provides this as target data to the user terminal 300. For example, the service providing server 100 extracts multiple pieces of diagnostic information, such as body temperature, pulse rate, heart rate, respiratory rate, blood pressure, blood glucose level, and blood oxygen saturation, recorded in the electronic medical record data input from the user terminal 300, formats the information into structured data, and provides the generated structured data to the user terminal 300.

[0017] In this way, generating structured data from electronic medical record data and providing it to the user terminal 300 can be useful for searching for candidates to be recruited for a clinical trial. That is, the user terminal 300 can use the provided structured data to easily search for candidates who could become subjects for a clinical trial, using the conditions that must be met as search keys.

[0018] The user terminal 300 is, for example, a terminal used to handle electronic medical records in a medical institution, such as an electronic medical record (EMR) terminal, an electronic health record (EHR) terminal, or an electronic medical record terminal. Alternatively, the user terminal 300 may be a general-purpose terminal such as a personal computer connected to these terminals via a local area network (LAN). The user terminal 300 transmits the patient's electronic medical record data to the service providing server 100 to request processing, and obtains structured data that structures the patient's diagnostic information from the service providing server 100 as the processing result.

[0019] The LLM 200 is an AI (artificial intelligence) using a large-scale language model connected to the communication network 400, receives a prompt from the service providing server 100, executes processing according to the instructions written in the prompt, and outputs the execution results to the service providing server 100. The LLM 200 can utilize a publicly known system that is generally available on the Internet.

[0020] 2 is a block diagram showing an example of the functional configuration of the service providing server 100 according to this embodiment. As shown in Fig. 2, the service providing server 100 according to this embodiment includes, as its functional configuration, an original data input unit 11, a prompt generation unit 12, a target data generation unit 13, and a target data provision unit 14. The service providing server 100 according to this embodiment also includes, as storage media, an example information storage unit 15 and a linking information storage unit 16.

[0021] The functional blocks 11 to 14 execute the processes described below through cooperation between hardware and software. For example, the processes of the functional blocks 11 to 14 are executed by the operation of a program stored in a storage medium such as RAM, ROM, a hard disk, or a semiconductor memory under the control of a processor of a microcomputer including a CPU, RAM, ROM, etc. In addition to the microcomputer, a DSP (Digital Signal Processor) or the like may also be included.

[0022] The original data input unit 11 inputs original data from the user terminal 300 via the communication network 400. In this embodiment, as an example, electronic medical record data is input. In this embodiment, electronic medical record data for one patient may be input, or electronic medical record data for multiple patients may be input. When the original data input unit 11 inputs multiple electronic medical record data, the service providing server 100 executes the processing described below for each electronic medical record data, generates structured data for each patient, and provides the structured data to the user terminal 300. Alternatively, the structured data generated for each patient may be further structured and shaped into a single structured data.

[0023] The prompt generator 12 generates prompts to be input into the LLM 200 using original data containing confidential information (electronic medical record data containing the patient's personal attribute information and diagnosis information) (first process). The prompt generator 12 divides the input electronic medical record data into multiple pieces, replaces the confidential information with other information, and generates multiple prompts for each of the multiple pieces of divided data. The prompt generator 12 then generates linking information linking the confidential information before the replacement with the other information after the replacement, and stores the linking information in the linking information storage unit 16. The linking information may include information linking the multiple pieces of divided data together.

[0024] Here, the other information to be replaced may be null information. In other words, deleting secret information is also included as a form of replacement. Hereinafter, replacements other than "deletion" may be referred to as "modification." Furthermore, the original data may be divided so that the multiple divided data do not include overlapping portions, or so that overlapping portions are included. For example, if the original data includes information a, information b, and information c, it is possible to divide the data without overlapping, such as into first divided data including only information a, second divided data including only information b, and third divided data including only information c. Alternatively, the data may be divided while allowing overlapping, such as into first divided data including information a and b and second divided data including information b and c.

[0025] For example, the prompt generation unit 12 extracts multiple nouns and numerical values ​​by performing natural language analysis on the text information recorded in the electronic medical record data, and identifies personal attribute information such as name, date of birth, age, sex, and address, as well as diagnostic information such as medical history, symptoms, body temperature, pulse rate, heart rate, respiratory rate, blood pressure, blood glucose level, and blood oxygen saturation, and then replaces the identified personal attribute information and diagnostic information with other information.

[0026] For example, the prompt generation unit 12 deletes at least a portion of the personal attribute information according to a predetermined algorithm, while changing the remaining personal attribute information and diagnostic information to information that has the same data attributes as the deleted information and is significant for the data attributes. The confidential information to be deleted is information that can be used to generate the desired structured data using the LLM 200 even if it is not included in the prompt. For example, proper nouns or unique information such as the patient's name and address can be deleted. On the other hand, the confidential information to be changed is information that may be required to generate the desired structured data using the LLM 200. For example, information related to classifications such as gender or numerical values ​​can be changed. Note that names, addresses, etc. may also be changed to different information.

[0027] Information that is significant for a data attribute means information that can be assumed as a medical evaluation for that data attribute. For example, in the case of blood pressure, the blood pressure value extracted from the electronic medical record data is changed to any value that is medically possible for blood pressure. At this time, it may be changed to any value whose difference from the original blood pressure falls within a predetermined value. Also, in the case of body temperature, the body temperature value extracted from the electronic medical record data is changed to any value whose difference from the original body temperature falls within a predetermined value. The same applies to other diagnostic information.

[0028] The prompt generation unit 12 generates a plurality of prompts, each including example information of a process to be executed in the LLM 200 and divided data generated by dividing the original data, i.e., electronic medical record data, and replacing confidential information with other information. The example information is pre-stored in the example information storage unit 15, and the prompt generation unit 12 reads the example information from the example information storage unit 15 to generate a prompt. That is, the prompt generation unit 12 generates a prompt for performing so-called few-shot prompting for each divided data.

[0029] The target data generation unit 13 generates target data using output information acquired by inputting the prompts generated by the prompt generation unit 12 to the LLM 200 (second process). Here, the target data generation unit 13 restores the secret information before replacement for the output information using the plurality of pieces of output information acquired by inputting the plurality of prompts generated for each piece of divided data to the LLM 200 and the linking information stored in the linking information storage unit 16 by the prompt generation unit 12, and generates target data using the restored information.

[0030] In this embodiment, the information output from the LLM 200 includes element information (structured data elements) for generating structured data. For example, the structured data elements are information corresponding to individual diagnostic information such as body temperature, pulse rate, heart rate, respiratory rate, blood pressure, blood glucose level, and blood oxygen saturation. The target data generation unit 13 restores the confidential information (personal attribute information and diagnostic information) before replacement for the multiple structured data elements output from the LLM 200 for each of the multiple prompts, and generates structured data as target data using the multiple structured data elements from which the confidential information has been restored.

[0031] The target data providing unit 14 provides the target data (structured data) generated by the target data generating unit 13 to the user terminal 300 .

[0032] The processing performed by the service providing server 100 configured as described above will be described below with reference to a specific example. Fig. 3 is a diagram showing an example of part of the electronic medical record data input by the original data input unit 11. The electronic medical record data shown in Fig. 3 includes personal attribute information such as name and age, as well as current medical history, past medical history, and current diagnosis information. The diagnosis information includes various numerical values ​​that represent the diagnosis results.

[0033] Fig. 4 is a diagram showing an example of divided data generated from electronic medical record data by the prompt generation unit 12. That is, Fig. 4 shows an example of a plurality of divided data generated by dividing the electronic medical record data shown in Fig. 3 into a plurality of pieces and replacing (including changing and deleting) the personal attribute information and the diagnostic information with other information.

[0034] In the example of Figure 4, the electronic medical record data is divided into multiple pieces at any line break from the top. The number of line breaks at which the data is divided may be determined randomly. Alternatively, the electronic medical record data may be divided into multiple pieces at every predetermined number of line breaks. Alternatively, the electronic medical record data may be divided into multiple pieces by heading unit such as current medical history, past medical history, or current symptoms. Note that the data fragmented by the division may be semantically interpretable (for example, as information indicating part of the electronic medical record), or may not be semantically interpretable. As an example of the latter division, further division may be performed within a single heading unit.

[0035] In the example of FIG. 4 , in the first divided data, the name is deleted and the age is changed from "56" to "58." Furthermore, the value "two years ago," indicating the time when the symptom occurred, is changed to "five years ago." In the second divided data, the value "two hours later," indicating the time when the symptom occurred, is changed to "four hours later." In the third divided data, the age is changed from "56" to "58." The fourth divided data does not contain the personal attribute information and diagnostic information to be replaced, and therefore no replacement is performed. In the fifth divided data, the values ​​of body temperature, respiratory rate, pulse rate, and blood pressure are each changed to different values. At this time, the prompt generation unit 12 generates linking information linking the confidential information before replacement with the different information after replacement, and stores the linking information in the linking information storage unit 16.

[0036] FIG. 5 is a diagram showing an example of exemplary information stored in the exemplary information storage unit 15. The exemplary information shown in FIG. 5 shows part of an execution example of a process for generating structured data elements based on unstructured information recorded in electronic medical record data. The first exemplary information shows a processing example in which, when information corresponding to blood pressure is included in the electronic medical record data (split data), the information "blood pressure present, certainty factor: 1.0, 111.89 (value extracted from the electronic medical record data)" is output. The second exemplary information shows a processing example in which, when information corresponding to blood pressure is not included in the electronic medical record data (split data), the information "blood pressure not present, certainty factor: 0.0, null" is output.

[0037] 5 only shows an example of a process for outputting information related to blood pressure, and does not show examples of other processes. The example information in FIG. 5 may also include examples of a process for outputting other diagnostic information such as body temperature, pulse rate, heart rate, respiratory rate, blood glucose level, and blood oxygen saturation. The types of examples of processes to be included can be designed according to the target data to be generated.

[0038] 6 is a diagram showing an example of one prompt generated from one piece of divided data by the prompt generation unit 12. The prompt shown in Fig. 6 shows a prompt generated by the prompt generation unit 12 including a plurality of pieces of example information 61 read out from the example information storage unit 15 and a first piece of divided data 62 divided from electronic medical record data. This prompt instructs the execution of a process similar to the process example shown in the plurality of pieces of example information 61 on the divided data 62 and output of the execution result.

[0039] Although not shown in the figure, the prompt generation unit 12 generates second, third, ... prompts by including the same example information as the example information 61 shown in Figure 6 and the second, third, ... divided data segments obtained by dividing the electronic medical record data. In this way, the prompt generation unit 12 generates multiple prompts for each of the multiple divided data segments. The target data generation unit 13 inputs the multiple prompts generated by the prompt generation unit 12 as described above to the LLM 200 and obtains output information from the LLM 200. In this case, the target data generation unit 13 does not need to be concerned about the order of the divided data segments, and may input the multiple prompts to the LLM 200 in any order.

[0040] 7A and 7B are diagrams showing an example of information output from the LLM 200 corresponding to the two pieces of example information from above when the prompt shown in FIG. 6 is input to the LLM 200. FIG. 7A shows information output from the LLM 200 when a prompt generated using the first to fourth pieces of divided data shown in FIG. 4 is input to the LLM 200. FIG. 7B shows information output from the LLM 200 when a prompt generated using the fifth piece of divided data shown in FIG. 4 is input to the LLM 200. Of the output information shown in FIG. 7B, the numerical values ​​"191,98" are structured data elements.

[0041] If the example information 61 shown in Figure 6 also includes examples of processes for outputting other diagnostic information such as body temperature, pulse rate, heart rate, respiratory rate, blood glucose level, and blood oxygen saturation, information containing that diagnostic information as structured data elements is output from LLM200.

[0042] The target data generation unit 13 restores the pre-replacement confidential information for the structured data elements included in the plurality of pieces of output information using the output information from the LLM 200 and the linking information stored in the linking information storage unit 16 by the prompt generation unit 12, and generates structured data using the restored information. Figure 8 shows the results of restoring the pre-replacement diagnostic information for the structured data elements included in the output information from the LLM 200 shown in Figure 7. Here, since the output information shown in Figure 7(a) does not contain a value corresponding to blood pressure, restoration to the pre-replacement diagnostic information has not been performed. On the other hand, since the output information shown in Figure 7(b) contains a value corresponding to blood pressure, restoration to the pre-replacement diagnostic information has been performed.

[0043] The target data generator 13 generates structured data using a structured data element included in the output information with the highest confidence level among the multiple pieces of output information output from the LLM 200 for each of the multiple prompts. Figure 9 is a diagram showing an example of structured data generated by the target data generator 13. The structured data shown in Figure 9 shows an example of one piece of structured data generated using multiple structured data elements included in multiple pieces of output information obtained by inputting multiple prompts to the LLM 200.

[0044] Figure 9 shows an example of structured data generated from the electronic medical record data of one patient shown in Figure 3. However, as described above, multiple pieces of structured data generated from the electronic medical record data of multiple patients may be integrated to generate one piece of structured data.

[0045] 10 is a flowchart showing an example of the operation of the service providing server 100 according to this embodiment. Here, an example of the operation when the service providing server 100 processes one piece of original data input from the user terminal 300 is shown.

[0046] First, the original data input unit 11 inputs original data from the user terminal 300 via the communication network 400 (step S1). Next, the prompt generation unit 12 divides the input original data into multiple pieces and replaces the secret information included in the original data with other information to generate multiple pieces of divided data. Then, the prompt generation unit 12 generates multiple prompts using the multiple pieces of divided data and the example information stored in the example information storage unit 15 (step S2). At this time, the prompt generation unit 12 generates linking information that links the secret information before the replacement with the other information after the replacement, and stores the linking information in the linking information storage unit 16 (step S3).

[0047] Next, the target data generator 13 inputs one of the multiple prompts generated for each data segment to the LLM 200 and acquires information output from the LLM 200 (step S4). The target data generator 13 then determines whether all of the multiple prompts generated by the prompt generator 12 have been processed (step S5). If an unprocessed prompt remains, the process returns to step S4, and the target data generator 13 inputs the next prompt to the LLM 200 and acquires output information from the LLM 200.

[0048] After processing all of the prompts generated by the prompt generation unit 12, the target data generation unit 13 restores the original secret information for the output information using the output information acquired from the LLM 200 and the linking information stored in the linking information storage unit 16 by the prompt generation unit 12, and generates target data using the restored information (step S6). The target data provision unit 14 then provides the target data generated by the target data generation unit 13 to the user terminal 300 (step S7). This completes the processing of the flowchart shown in FIG. 10.

[0049] As described above in detail, in this embodiment, the prompt generation unit 12 divides original data including secret information into multiple pieces, replaces the secret information with other information, and generates multiple prompts for each of the multiple pieces of divided data. The target data generation unit 13 then restores the secret information before replacement for the multiple pieces of output information using multiple pieces of output information obtained by inputting the multiple prompts to the LLM 200 and the linking information stored in the linking information storage unit 16 by the prompt generation unit 12 when replacing the secret information with other information, and generates target data using the restored information.

[0050] As a result, according to this embodiment, confidential information of the original data is not included in the prompts generated using the original data, and a prompt is generated for each of the fragmented data pieces obtained by dividing the original data into multiple pieces. Therefore, even when using an LLM service on the Internet, it is possible to prevent confidential information from being leaked by individual prompts input into LLM 200.

[0051] In this embodiment, when the prompt generator 12 generates a prompt, it replaces the confidential information with information that has the same data attributes as the confidential information and is significant to the data attributes. This allows the LLM 200 to correctly interpret and extract structured data elements from the prompt based on the example information included in the prompt, even if the prompt is generated by including information that replaces the confidential information.

[0052] In the above embodiment, electronic medical record data is used as an example of unstructured data, and an example of a process for generating structured data from electronic medical record data is described, but the present invention is not limited to this. In other words, the present embodiment can be applied to all processes for generating structured data from unstructured data that includes confidential information.

[0053] Furthermore, in the above embodiment, an example of a process for generating structured data from unstructured data has been described, but the present invention is not limited to this. That is, this embodiment can also be applied to cases where unstructured data is generated from unstructured data, where structured data is generated from structured data, and where unstructured data is generated from structured data. That is, this embodiment can be applied to all processes where a prompt is generated using source data containing confidential information, and target data is generated using output information obtained by inputting the prompt into the LLM 200.

[0054] In the above embodiment, an example of searching for recruiting candidates for a clinical trial has been described as a use case of generating structured data from unstructured data, but the present invention is not limited to this. For example, the present embodiment can be used for other use cases, such as marketing analysis, acquiring data to be used in writing research papers, and inputting data into clinical trial data.

[0055] In the above embodiment, an example has been described in which the secret information before substitution is restored from the output information from the LLM 200 and the target data is generated using the restored information, but the present invention is not limited to this. For example, instead of or in addition to the output information from the LLM 200, the secret information before substitution may be restored from the divided data generated by the prompt generation unit 12, and the target data may be generated using the output information from the LLM 200 and the divided data.

[0056] Furthermore, in the above embodiment, an example has been described in which a prompt for executing Few-Shot prompting is generated, but this is merely an example, and the present invention is not limited to this.

[0057] Furthermore, the above-described embodiments are merely examples of specific embodiments for carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereby. In other words, the present invention can be carried out in various forms without departing from the gist or main characteristics thereof.

[0058] REFERENCE SIGNS LIST 11 Original data input unit 12 Prompt generation unit 13 Target data generation unit 14 Target data provision unit 15 Example information storage unit 16 Linking information storage unit 100 Service provision server (LLM service provision device) 200 LLM (Large Scale Language Model) 300 User terminal

Claims

1. A system using an LLM, comprising: a prompt generation unit that generates prompts to be input into a large-scale language model using original data including secret information; and a target data generation unit that generates target data using output information obtained by inputting the prompts generated by the prompt generation unit into the large-scale language model, wherein the prompt generation unit divides the original data into multiple pieces and replaces the secret information with different information to generate multiple prompts for each of the multiple pieces of divided data, and stores linking information that links the secret information before the replacement with the different information after the replacement, and the target data generation unit uses multiple pieces of output information obtained by inputting the multiple prompts generated by the prompt generation unit into the large-scale language model and the linking information stored by the prompt generation unit to restore the secret information before replacement for the multiple pieces of output information or the multiple pieces of divided data, and generates the target data using the restored information.

2. The LLM utilization system described in claim 1, characterized in that the prompt generation unit replaces the secret information with information that has the same data attributes as the secret information and is significant for the data attributes, according to a predetermined algorithm.

3. The LLM utilization system described in claim 1 or 2, characterized in that the original data is unstructured data, the target data is structured data, the prompt generation unit generates a plurality of prompts including example information of a process to be executed in the large-scale language model and the divided data generated by dividing the original data and replacing the secret information with the other information, and the target data generation unit restores the secret information before replacement for a plurality of structured data elements output from the large-scale language model for each of the plurality of prompts, and generates the structured data using the plurality of structured data elements from which the secret information has been restored.

4. The LLM utilization system described in claim 3, characterized in that the unstructured data is electronic medical record data, the confidential information includes the patient's personal attribute information and diagnostic information, the prompt generation unit generates multiple prompts by replacing the personal attribute information and the diagnostic information with the other information, and the target data generation unit restores the other information included in the output information from the large-scale language model to the personal attribute information or the diagnostic information before replacement, thereby generating the structured data.

5. An LLM service providing device comprising: a prompt generation unit that generates prompts to be input into a large-scale language model using original data including secret information; and a target data generation unit that generates target data using output information obtained by inputting the prompts generated by the prompt generation unit into the large-scale language model, wherein the prompt generation unit divides the original data into multiple pieces and replaces the secret information with different information to generate multiple prompts for each of the multiple pieces of divided data, and at that time stores linking information that links the secret information before the replacement with the different information after the replacement, and the target data generation unit uses multiple pieces of output information obtained by inputting the multiple prompts generated by the prompt generation unit into the large-scale language model and the linking information stored by the prompt generation unit to restore the secret information before replacement for the multiple pieces of output information or the multiple pieces of divided data, and generates the target data using the restored information.

6. A method for providing an LLM service, comprising: a processor of an LLM service providing device executing a first process for generating prompts to be input into a large-scale language model using original data including secret information; and a second process for generating target data using output information obtained by inputting the prompts generated by the first process into the large-scale language model; wherein the first process divides the original data into multiple pieces and replaces the secret information with different information to generate multiple prompts for each of the multiple pieces of divided data, and stores linking information linking the secret information before the replacement with the different information after the replacement; and wherein the second process uses multiple pieces of output information obtained by inputting the multiple prompts generated by the first process into the large-scale language model and the linking information stored during the first process to restore the secret information before replacement for the multiple pieces of output information or the multiple pieces of divided data, and generates the target data using the restored information.

Images