Information processing device, information processing method, and information processing program

Abstract

To provide a technique for prompt generation for causing an LLM to generate large-scale content.SOLUTION: A prompt generation unit comprises a sharing instruction generation unit for generating sharing instruction information to instruct shared generation of answers, and a takeover instruction generation unit for generating takeover instruction information to instruct takeover matters to be referred to by a language model when generating answers.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Conventionally, systems have been provided that perform natural language processing such as question-answering using large-scale language models (LLMs) as language models. LLMs are natural language processing models trained using large amounts of text data, and take sentences as input and output sentences. When LLMs are applied to a system for question-answering, when a question is input into the LLM, an answer is output from the LLM. In LLMs, the sentences that are input are called "prompts."

[0003] A prompt is a sentence written in natural language that describes the task that the artificial intelligence should perform. In order to obtain an appropriate answer from the language model, a prompt is generated that includes not only the question but also the conditioning and information to be referenced to guide the language model to give an appropriate answer. By inputting this generated prompt into the language model, the language model outputs an appropriate answer.

[0004] Patent Document 1 discloses, with regard to adding information to a prompt, adding an instruction sentence including an explanation about the chatbot character to the prompt, adding a character dialogue sample to the prompt, adding a dialogue scene obtained from a dialogue storage database in relation to the dialogue topic to the prompt, and replacing words contained in the question sentence with synonyms or the like and adding them to the prompt.

[0005] Patent Document 2 discloses generating another question based on a history of past answers, and Patent Document 3 discloses generating another question by replacing words included in a question with reduced terms. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2022-180282 A [Patent Document 2] Special Publication No. 2001-519559 [Patent Document 3] Special Publication No. 2002-508863 Summary of the Invention [Problem to be solved by the invention]

[0007] In this way, inputting a prompt generated by adding reference information to a question into an LLM is effective in that it is easier to obtain a suitable answer than if no reference information is added. However, because the number of tokens (number of characters) is limited in LLM, it may not return a long answer or the sentence may be cut off midway. Therefore, it is not suitable for generating large-scale content such as large documents or source code.

[0008] The present disclosure has been made to solve such problems, and aims to provide a prompt generation technique for enabling an LLM to generate large-scale content. [Means for solving the problem]

[0009] In order to solve the above problems, the information processing device of the present disclosure has a prompt generation unit that generates a prompt to be input to a language model based on a question, and the prompt generation unit is equipped with a sharing instruction generation unit that generates sharing instruction information that instructs that the answer should be generated in a shared manner, and a handover instruction generation unit that generates handover instruction information that instructs the language model to refer to handover items when generating the answer.

[0010] In addition, the information processing method disclosed herein includes a prompt generation step of generating a prompt to be input to a language model based on a question, and the prompt generation step includes a sharing instruction generation step of generating sharing instruction information that instructs that the answer should be generated in a sharing manner, and a handover instruction generation step of generating handover instruction information that instructs the language model to refer to handover items when generating the answer.

[0011] In addition, the information processing program disclosed herein is an information processing program that causes a computer to execute a prompt generation procedure that generates a prompt to be input to a language model based on a question, and the prompt generation procedure includes a sharing instruction generation procedure that generates sharing instruction information that instructs that an answer should be generated in a sharing manner, and a handover instruction generation procedure that generates handover instruction information that instructs the language model to refer to handover items when generating a new answer following the answer. Effect of the Invention

[0012] According to the present disclosure, a prompt generation technique can be provided for allowing an LLM to generate large-scale content. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 illustrates an example of an information processing system according to the present disclosure. [Diagram 2] FIG. 2 is a diagram illustrating an example of a sentence generation unit of an information processing device according to the present disclosure. [Diagram 3] FIG. 13 is a diagram illustrating an example of generating a first prompt. [Figure 4] FIG. 13 illustrates an example of the generation of a second prompt in response to an LLM's response to a first prompt. [Diagram 5] FIG. 13 illustrates an example of the generation of a third prompt in response to an LLM's response to a second prompt. [Figure 6] FIG. 13 is a diagram showing an image of registering instruction symbols. [Figure 7] FIG. 1 is a flow diagram illustrating an example of an information processing method according to the present disclosure. [Figure 8] FIG. 13 illustrates an example of generating a review prompt. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. However, more detailed explanations than necessary, such as detailed explanations of already well-known matters and duplicate explanations of substantially the same configurations, may be omitted.

[0015] It should be noted that the following description and the drawings referred to are provided to enable those skilled in the art to understand the present disclosure, and are not intended to limit the scope of the claims of the present disclosure.

[0016] (Information Processing System in This Disclosure) First, an information processing system according to an embodiment of the present disclosure will be described with reference to FIG.

[0017] An information processing system 1 according to an embodiment of the present disclosure includes a terminal device 100 and a large-scale language model (hereinafter, referred to as "LLM") 200. The terminal device 100 and the LLM 200 communicate with each other via a communication network NW. The network NW includes, for example, the Internet, a LAN (Local Area Network), a wireless base station, a provider device, and the like.

[0018] The terminal device 100 is, for example, a personal computer, a smartphone, a tablet terminal, etc. The terminal device 100 is an information processing device including a CPU, a storage, a ROM, a RAM, an input / output I / F, a communication unit, a display unit, etc. Note that the information processing device is a well-known technology, and therefore a detailed description thereof will be omitted.

[0019] The terminal device 100 functions as a sentence generation unit 110. The sentence generation unit 110 generates a prompt to be input as a question to the LLM 200 by executing a sentence generation program by a processor such as a CPU. Also, in response to an answer output from the LLM 200, the sentence generation unit 110 generates subsequent prompts to be input as questions to the LLM 200.

[0020] The LLM200 is a large-scale language model that inputs (receives) a question from the terminal device 100, generates an answer to the question, and outputs (transmits) it to the user terminal 200. The LLM200 is a language model trained with a large amount of text data. In particular, by enlarging the "calculation amount," "data amount," and "number of model parameters," accurate and natural text can be generated. When training the LLM200, pre-training and fine-tuning using a large amount of training data are repeatedly performed. In pre-training, a large amount of text data is collected, and the appearance patterns of the words and phrases are learned. For language models that have completed a certain level of training, tests are performed with verification data to fine-tune the parameters and determine the validity of the model, and fine-tuning is performed to optimize the model. Note that an example of such a large-scale language model is GPT-4 provided by OpenAI, Inc. in the United States.

[0021] (Text generation unit of terminal device in the present disclosure) Next, the sentence generation unit according to the embodiment of the present disclosure will be described with reference to FIG.

[0022] The sentence generation unit 110 is composed of a prompt generation unit 120, an answer processing unit 130, a transmission / reception processing unit 140, and the like. The sentence generation unit 110 also includes a sentence generation database (not shown). When the prompt generation unit 120 generates a prompt, the prompt is generated so as to obtain an appropriate answer from the LLM 200 by using the sentence generation database. The sentence generation unit 110 also includes an external information acquisition unit (not shown). When it is necessary to add a result of a web search to the prompt when generating the prompt, the external information acquisition unit starts a web search tool to acquire the necessary information. When it is necessary to add a document stored in an external server such as a cloud storage to the prompt when generating the prompt, the external information acquisition unit searches for the document stored in the cloud storage and extracts the necessary part as a reference sentence.

[0023] The prompt generation unit 120 has a function of generating a prompt, and includes a question processing unit 121, a sharing instruction generation unit 122, an answer condition instruction generation unit 123, a token upper limit number instruction generation unit 124, a handover item instruction generation unit 125, a chapter instruction generation unit 126, an instruction correction unit 127, and the like.

[0024] The question processor 121 executes a process of accepting input of a question from a user and a process of interpreting the content of the input question.

[0025] The sharing instruction generation unit 122 judges whether or not there is a possibility that the answer to the current question will not be completed with just one answer by the LLM 200, based on the interpretation of the question content by the question processing unit 121. If the sharing instruction generation unit 122 judges that there is a possibility that the answer to the current question will not be completed with just one answer by the LLM 200, it generates a sentence instructing the LLM 200 that the answer to the current question is a shared task between multiple members (multiple LLMs 200 = the LLM 200 performs multiple answers)) and adds this sentence to the prompt.

[0026] When the sharing instruction generating unit 122 generates a sentence indicating that it is a shared task, the answer condition instruction generating unit 123 generates a sentence indicating the conditions for answering the current question to the LLM 200. The answer condition instruction generating unit 123 generates a sentence indicating that there are matters to be handed over from the previous member (the answer to the previously executed LLM 200), and also indicates the rules (handover rules) for writing down the handed over matters, and adds the generated sentence to the prompt.

[0027] The token upper limit instruction generating unit 124 generates a sentence that indicates the upper limit (token upper limit) of the amount of text (number of tokens) that can be replied to the LLM 200, and adds the sentence to the prompt.

[0028] When a sentence indicating that it is a shared task is generated by the shared generation instruction process, the handover item instruction generator 125 generates a sentence to instruct the LLM 200 to generate handover items for the next member in the answer if the answer cannot be completed with the current answer work, and adds the sentence to the prompt. Also, the handover items to be added to the next prompt are extracted from the answer of the LLM 200, and a sentence instructing the handover items is generated and added to the prompt.

[0029] When a sentence indicating that the task is a shared task is generated by the sharing generation instruction process, the chapter instruction generation unit 126 generates a sentence instructing the LLM 200 that the steps leading up to completing the answer may be divided into chapters (chapters, sections, paragraphs), and adds this sentence to the prompt.

[0030] The instruction correction unit 127 gives the user an opportunity to correct the handover items described in the prompt to be answered in the current task, extracted by the answer condition instruction process. The instruction correction unit 127 displays to the user a sentence instructing the handover items generated by the handover item instruction generation unit 125. If the user wishes to correct the sentence of the handover items, he or she can add or correct it here.

[0031] The prompt generation unit 110 checks whether the prompt generated by executing each of the above-mentioned processes based on the question entered by the user is within the limit on the number of characters that can be input to the LLM 200, and if the number of input characters exceeds the limit, adjusts the number of input characters so that it is within the limit.

[0032] The answer processing unit 130 has a function of processing answers output from the LLM 200. The answer processing unit 130 is composed of a text editing unit 131, a token number counter 132, and the like.

[0033] The text editing unit 131 executes a text editing process in which it extracts text related to the user's question from the answer output from the LLM 200 and pastes it into a word processing software or a text editor that generates a document file. Note that it may be possible for the user to extract a desired text and edit it themselves using a word processing software or a text editor.

[0034] The token number counter 132 counts the number of characters (token number) in the answer output from the LLM 200 , and outputs the current number of tokens to the prompt generation unit 120 .

[0035] The transmission / reception processor 140 executes a process for transmitting a prompt generated by the prompt generator 120 and a process for receiving a response transmitted from the LLM 200 .

[0036] (Example of generating prompts) Next, examples of prompt generation by the prompt generation unit 120 will be described with reference to Figs. 3 to 5. Fig. 3 is a diagram showing an example of generating a first prompt. Fig. 4 is a diagram showing an example of generating a second prompt in response to the LLM's answer to the first prompt. Fig. 5 is a diagram showing an example of generating a third prompt in response to the LLM's answer to the second prompt.

[0037] First, an example of generating the first prompt will be described with reference to Fig. 3. In this example, it is assumed that the user is currently attempting to generate a book on the theme of global warming.

[0038] When the sentence generation program is started by a user operation, a sentence generation screen is displayed. First, the prompt generation unit 120 accepts input of a question (question sentence) through the question processing unit 121. The user inputs a question, for example, "Please write a book about global warming" on the sentence generation screen. Based on the input of this question, the prompt generation unit 120 executes a first prompt generation process to generate a first prompt Q001.

[0039] First, the question processor 121 executes question processing to interpret the contents of the input question. Then, if the question processor 121 determines through its interpretation of the question contents that it is for the generation of large-scale content (that there is a possibility of generating large-scale content), it executes the sharing instruction generation processing by the sharing instruction generator 122. The sharing instruction generator 122 generates a sentence to allow the LLM 200 to interpret that the work is shared and that there is a handover sentence, such as "You are writing a book with multiple members sharing the work, and the handover sentence has been handed down to you," and adds this sentence to the prompt.

[0040] Next, the answer condition instruction generating unit 123 executes an answer condition instruction generating process. The answer condition instruction generating unit 123 generates a sentence such as "The handover sentence contains handover items in addition to the sentence written by the previous member" and adds it to the prompt. This allows the LLM 200 to interpret that the handover sentence from the previous member (the answer to the previously executed LLM 200) is written.

[0041] Furthermore, the answer condition instruction generating unit 123 generates a sentence such as "The items to be handed over are written in the form ##...##. Please create or modify the text according to this instruction" and adds it to the prompt. This allows the LLM 200 to interpret that the items to be handed over are written between "##" and "##", that the rule is to write the items to be handed over between "##" and "##", and that the answer this time is to be given to the items to be handed over that are written between "##" and "##". Note that the rule for the items to be handed over may be clearly stated in a sentence explaining the rule, such as "The rule is to always write the items to be handed over between ## and ##."

[0042] Next, the token upper limit instruction generation unit 124 executes a token upper limit instruction process. For example, the token upper limit instruction generation unit 124 generates a sentence such as "The upper limit of the amount of text you can write is 4000 characters" and adds it to the prompt. This allows the LLM 200 to interpret that the upper limit of the number of tokens in the current answer is 4000 tokens (4000 characters). The upper limit number instructed by the token upper limit instruction generation unit 124 may be instructed based on the upper limit number of answer tokens determined by the LLM 200, or the user may be allowed to set the numerical value of the upper limit number instructed by the token upper limit instruction generation unit 124 to a desired numerical value in advance.

[0043] Next, the handover item instruction generating unit 125 executes the handover item instruction process. The handover item instruction generating unit 125 generates a sentence such as "If there is any part that cannot be written in this work, leave that part as a handover item for the next member" and adds it to the prompt. This allows the LLM 200 to understand that it is not necessary to complete the answer (summarize the answer) within the token upper limit of 4000 tokens, but that if it cannot be written, it must be left as a handover item for the next member.

[0044] The handover item instruction generating unit 125 also generates a sentence in which the question entered by the user is inserted between "##" and "##" as the initial handover item according to the handover item description rules determined by the answer condition instruction generating unit 123. Then, this handover item is registered (stored) in the topmost hierarchy of handover items as instruction code 00.

[0045] Next, the chaptering instruction generation unit 126 executes a chaptering instruction process. The chaptering instruction generation unit 126 generates a sentence such as "If it is necessary to insert a step such as creating a chapter, please do so" and adds it to the prompt. This allows the LLM 200 to interpret that it is okay to create a chapter.

[0046] Through the above processing, the first prompt generation processing generates the first prompt Q001. The generated first prompt Q001 is transmitted from the terminal device 100 to the LLM 200 in response to a transmission operation by the user.

[0047] Next, an example of generating a second prompt in response to the LLM's response to the first prompt will be described with reference to FIG.

[0048] When LLM200 receives the first prompt Q001, it generates and returns an answer (answer text) A001. The answer A001 from LLM200 is in a format that conforms to the instructions and rules of the first prompt Q001.

[0049] The prompt generation unit 120 executes a second prompt generation process to generate a second prompt Q002 based on the answer A002. In the second prompt generation process, similar to the first prompt generation process described above, a share generation instruction process, an answer condition instruction process, an upper limit number of tokens instruction process, and a chapter instruction process are executed. Note that if the prompt generation unit 120 determines that it is not necessary to instruct chapters at this stage, it does not execute the chapter instruction process.

[0050] In the second prompt generation process, the handover item instruction generation unit 125 checks whether the answer A001 contains a sentence (instruction symbol) surrounded by "##" and "##". The answer A001 is divided into four chapters, such as "1. Looking back at global warming with facts", "2. International agreements", "3. Efforts by each company", and "4. What can we do", and the handover items for each chapter are written between "##" and "##". Therefore, the handover item instruction generation unit 125 extracts, for example, the handover items for chapter 1 as instruction symbol 10, the handover items for chapter 2 as instruction symbol 20, the handover items for chapter 3 as instruction symbol 30, and the handover items for chapter 4 as instruction symbol 40 from the answer A001, and registers them in a lower hierarchy following instruction symbol 00.

[0051] Furthermore, the handover item instruction generating unit 125 extracts instruction symbol 10, which is registered at the top of the hierarchy below instruction symbol 00, as the second handover item, and adds it to the prompt. Here, "##Please add content that matches "Looking back on global warming with facts" registered as instruction symbol 10. We will try to introduce objective facts about global warming as much as possible. ##" is added to the prompt.

[0052] Next, in the second prompt generation process, the instruction correction unit 127 executes the instruction correction process. The instruction correction unit 127 displays the text of the instruction symbol 10 that has been added to the prompt as a handover item. The user checks the text of the instruction symbol 10, and if there is no need for correction, executes an operation to display the generated second prompt Q002. On the other hand, if there is a need for correction, the user makes the correction here. In this example of generation of the second prompt Q002, the user adds a sentence to the text of the instruction symbol 10, saying, "First, create a detailed section." After the user finishes correcting the instruction symbol 10, the user executes an operation to display the second prompt Q002. At this time, the handover item instruction generation unit 125 overwrites the instruction symbol 10 with the corrected instruction symbol 10 and registers it.

[0053] In the second prompt generation process, the second prompt Q002 is generated by the above-described process. The generated second prompt Q002 is transmitted from the terminal device 100 to the LLM 200 in response to a transmission operation by the user.

[0054] Next, an example of generating a third prompt Q003 in response to an answer A002 from the LLM to the second prompt Q002 will be described with reference to FIG.

[0055] When the LLM 200 receives the second prompt Q002, it generates and returns an answer A002. The answer A002 from the LLM 200 is in a format that conforms to the instructions and rules of the second prompt Q002.

[0056] The prompt generation unit 120 executes a third prompt generation process to generate a third prompt Q003 based on the answer A002. In the third prompt generation process, similar to the first prompt generation process described above, a share generation instruction process, an answer condition instruction process, a token upper limit instruction process, and a chapter instruction process are executed. Note that if the prompt generation unit 120 determines that it is not necessary to instruct chapters at this stage, it does not execute the chapter instruction process.

[0057] In the third prompt generation process, the handover item instruction generation unit 125 checks whether the answer A002 has an instruction symbol. In this answer A002, chapter 1 is divided into three sections, namely, "1.1 The Importance of Facts," "1.2 Changes in Global Temperature," and "1.3 Changes in Greenhouse Gas Concentration," and the handover items in each section are written as instruction symbols. Therefore, the handover item instruction generation unit 125 extracts, for example, the handover items for section 1.1 as instruction symbol 11, the handover items for section 1.2 as instruction symbol 12, and the handover items for section 1.3 as instruction symbol 13 from the answer A002, and registers them in a lower hierarchical level following instruction symbol 10.

[0058] Furthermore, the handover item instruction generating unit 125 extracts instruction symbol 11, which is registered at the top of the hierarchy below instruction symbol 10, as a third handover item, and adds it to the prompt.

[0059] Here, the prompt will be added with "##Please add content that matches "The importance of facts", which is registered as instruction symbol 11. Please mention that in the debate over global warming, there are opinions that question the veracity of global warming to begin with, but that calm judgment based on scientific facts is necessary. ##"

[0060] Here, another generation method by the handover item instruction generating unit 125 will be described.

[0061] As another generation method, the handover item instruction generation unit 125 interprets the contents of the instruction symbol and judges the necessity of referring to external information. For example, if the instruction symbol XX states "Search for XX information and write based on it," the handover item instruction generation unit 125 adds an instruction such as "Search for XX information using external information such as a web search and write based on the search results" to the instruction symbol 11. Alternatively, the handover item instruction generation unit 125 activates a web search tool or accesses cloud storage using the external information acquisition unit to acquire a description related to XX information and add it to the instruction symbol.

[0062] Here, the handover item instruction generating unit 125 may add a comment by referring to another part of the current text. In such a case, for example, an instruction such as "Write about XX in consideration of the contents of the previous chapter" or "Write about XX in consideration of the contents of Chapter 1, Section 2" is added. This allows the LLM 200 to interpret that a text in another chapter is to be referred to when generating an answer. Note that any description part may be referred to, not limited to the previous chapter. In such a case, the handover item instruction generating unit 125 executes a process of quoting (extracting) the contents of the previous chapter or any description part, and adds the quoted information to the prompt by including it in the reference text. Note that when adding the quoted information to the prompt by including it in the reference text, the quoted information may be appropriately summarized to reduce the number of tokens of the quoted information.

[0063] The handover instruction generator 125 may also instruct the LLM 200 to search more broadly. In such a case, for example, the handover instruction generator 125 may add an instruction such as, "Find all descriptions of XX, summarize their assertions, and generate a consistent statement."

[0064] The other generation method by the handover item instruction generator 125 described above is to generate an instruction for the LLM 200 to generate a sentence related to XX. However, this is not limited to this, and an instruction to rewrite the reference points referred to by the LLM 200 may be given. For example, an instruction may be given such as "Find all the places where XX is written, and rewrite them so that they do not contradict the assertion △△."

[0065] Next, in the third prompt generation process, the instruction correction unit 127 executes the instruction correction process. The instruction correction unit 127 displays the text of the instruction symbol 11 that has been added to the prompt as a handover item. The user checks the text of this instruction symbol 11, and if there are no corrections, the user executes an operation to display the generated third prompt Q003. If there are corrections, the user makes the corrections here. In this example of generating the third prompt, since there are no corrections made by the user, the user executes an operation to display the third prompt Q003. If there are corrections, the handover item instruction generation unit 125 overwrites the instruction symbol 11 with the corrected instruction symbol 11 and registers it.

[0066] In the third prompt generation process, the third prompt Q003 is generated by the above-described process. The generated third prompt Q003 is transmitted from the terminal device 100 to the LLM 200 in response to a transmission operation by the user.

[0067] When LLM200 receives the third prompt Q003, it generates and replies with answer A003. Answer A003 from LLM200 is in a format that conforms to the instructions and rules of the third prompt Q003, and answer A003 contains a sentence that corresponds to instruction symbol 11.

[0068] Thereafter, the prompt generation unit 120 repeatedly executes the above-described prompt generation process, such as using instruction symbol 12 for the fourth prompt Q004, instruction symbol 13 for the fifth prompt Q005, instruction symbol 20 for the sixth prompt Q006, etc. By repeatedly executing the prompt generation process by the prompt generation unit 120 in this manner, a large-scale text is generated through the shared work of the LLM 200.

[0069] 6 is a diagram showing an image of the registration of instruction symbols executed by the handover item instruction process. As described above, the handover item instruction generating unit 125 registers a question from a user as instruction symbol 00, and then registers chapters as instruction symbols 10, 20..., sections following instruction symbol 10 as instruction symbols 11, 12..., sections following instruction symbol 20 as instruction symbol 21..., sections following instruction symbol 30 as instruction symbol 31..., and sections following instruction symbol 40 as instruction symbol 41... in a hierarchical manner. Then, the sentences related to each instruction symbol are registered in association with each instruction symbol.

[0070] (Information Processing Method in This Disclosure) Next, an information processing method according to an embodiment of the present disclosure will be described with reference to FIG.

[0071] First, the user inputs a question to be answered by the LLM 200. When the question is input, the sentence generator 110 generates the above-mentioned first prompt Q001. The generated first prompt Q001 is transmitted from the terminal device 200 to the LLM 200 by the transmission / reception processor 140, thereby instructing the LLM 200 to generate a sentence (step S10).

[0072] When the LLM 200 receives the first prompt Q001, it generates an answer (answer A001) to this first prompt. The generated answer is transmitted to the terminal device 200 in units of several to several tens of characters.

[0073] When the transmission / reception processing unit 140 receives a reply, the reply processing unit 130 displays (adds) the reply in units of several to several tens of characters each time it is received (step S20).

[0074] Next, the answer processor 130 determines whether the answer to the first prompt Q001 has been completed (answer A001 has been completed) (step S21).

[0075] Next, if it is determined that the answer to the first prompt Q001 has not been completed, the number of tokens (number of characters) in the received answer is counted by the token counter. Here, it is determined whether the number of tokens in the received answer has reached a limit (step S22). Note that in this embodiment, the upper limit for the number of tokens is 4000 tokens, so the limit here is set to, for example, 3800 tokens.

[0076] If it is determined that the number of tokens in the received answer does not reach the limit, the process returns to step S21. On the other hand, if it is determined that the number of tokens in the received answer reaches the limit, a sentence correction instruction is executed (step S23).

[0077] The text correction instruction is executed by the prompt generation unit 120. When the current number of tokens in the token number counter reaches the token limit number, the prompt generation unit 120 generates a correction instruction prompt, for example, a sentence instructing, for example, "The token length is likely to be long, so please create a detailed chapter structure," and instructs the LLM 200 to correct the text.

[0078] When the number of tokens reaches the token limit, the prompt generation unit 120 may generate a sentence such as "Instead of generating everything in this step, switch to a policy of further dividing and generating. To do so, generate mainly with instruction symbols" as a correction instruction prompt and instruct the LLM 200 again to abort the generation of the answer when the number of tokens reaches the token limit. When the number of tokens reaches the token limit, the prompt generation unit 120 may send a prompt to the LLM 200 instructing the LLM 200 to abort the generation of the answer when the number of tokens reaches the token limit, and after stopping the generation of the answer, may write an instruction symbol in the form of "##Create the next sentence by referring to the previous generated sentence##" in the handover instruction process in the next prompt generation process. At that time, the sentence continuing from the previous answer may be added as a sentence to be referred to when the LLM 200 generates the answer.

[0079] When a correction instruction prompt is sent by a sentence correction instruction, the LLM 200 generates an answer in response to the correction instruction prompt.

[0080] If it is determined that the answer to the first prompt has been completed, a text editing process is executed (step S30). In the text editing process, a text related to the current prompt is extracted from the answer, and a document file is generated. Note that at this time, the text can also be edited by copying or cutting a desired description from the answer and pasting it into a word processing software or text editor used by the user.

[0081] When the process of step S30 is completed, the sentence generator 110 generates the second prompt Q002 described above. The generated second prompt Q002 is transmitted from the terminal device 200 to the LLM 200 by the transmission / reception processor 140, thereby instructing the LLM 200 to generate a sentence (step S40).

[0082] When the LLM 200 receives the second prompt Q002, it generates an answer (answer A002) to this second prompt Q002.

[0083] The document generator 110 repeats the above process until the generation of the large-scale content is completed.

[0084] As described above, according to an embodiment of the present disclosure, the prompt generated by the sentence generator 110 includes instructions to the LLM 200 that it is a shared task and that the answer should be generated with reference to the handover items. Therefore, when large-scale content is generated by the LLM 200, it is possible to generate a suitable prompt.

[0085] Furthermore, the prompt generated by the sentence generator 110 includes an instruction to the LLM 200 that chapters may be divided. Therefore, it is possible to generate a prompt suitable for generating large-scale content.

[0086] Furthermore, a symbol that enables the LLM 200 to recognize the location of the handover information to be referred to is added to the prompt generated by the sentence generator 110. The LLM 200 is then instructed to add the same symbol to the location of the handover information to be referred to by the next LLM 200. Therefore, when large-scale content is generated by multiple responses from the LLM 200, the LLM 200 can interpret the handover information to be referred to this time, and the next LLM 200 can also interpret the handover information to be referred to, so that the handover information can be clearly communicated.

[0087] Furthermore, the prompt generated by the sentence generator 110 includes, as an instruction, the upper limit of the number of tokens that can be used in a response to the LLM 200. Therefore, the LLM 200 can generate the current response within the range of the instructed upper limit of the number of tokens.

[0088] In addition, since the generated handover items can be modified by the user, it is possible to reflect the user's intentions in the handover items.

[0089] Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the embodiments, and various modifications and changes are possible within the scope of the gist of the present disclosure described in the claims.

[0090] (Example of generating review prompts) In an embodiment of the present disclosure, as described above, the sentence generator 110 causes the LLM200 to generate large-scale content by the process of generating first prompt Q001 → generating answer A001 → generating second prompt Q002 → generating answer A002 → ... However, after the generation of the large-scale content is completed or during the generation of the large-scale content, the LLM200 may be caused to review the content (answer sentence) generated by the LLM200.

[0091] 8 is a diagram showing an example of generating a review prompt. Here, an example will be described in which the prompt generation unit 120 generates a review prompt Q010 that instructs the LLM 200 to review the text S001 of the first chapter, “1. Looking back on global warming with facts.”

[0092] When a sentence S001 about "1. Looking back at global warming with facts" in chapter 1 of the title "Global Warming" is generated based on a response from the LLM 200, the prompt generating unit 120 executes a review prompt generating process.

[0093] First, the review instruction generation unit executes a review instruction generation process. The review instruction generation unit generates a sentence such as, "You are an editor. Please review the following sentence submitted by the author," and adds it to the prompt. This allows the LLM 200 to understand that you are an editor and that you will review the following sentence.

[0094] Next, the agreed instruction information generation unit executes an agreed instruction information generation process. The agreed instruction information generation unit generates a sentence such as "The specific instructions that were agreed upon were as follows. #The specific instructions that were agreed upon" and adds it to the prompt. Then, it extracts an instruction symbol that corresponds to "#The specific instructions that were agreed upon". Here, since the instruction in this case is to review Chapter 1, the agreed instruction information generation unit extracts instruction symbol 10 and adds it to the prompt. This allows the LLM200 to interpret that there are instructions that have been agreed upon with the LLM200 and the specific contents of the instructions.

[0095] Next, the draft extraction unit executes a draft extraction process. The draft extraction unit extracts the sentence S001 and adds it to the prompt. This allows the LLM 200 to interpret the sentence to be reviewed.

[0096] Next, the correction instruction generation unit executes a correction instruction generation process. The review instruction generation unit generates a sentence such as "If corrections are necessary, please add instructions to the author in the form of ##...##" and adds it to the prompt. This allows the LLM 200 to interpret that the sentence can be corrected, and that if corrections are required, instructions to the author should be entered between ## and ##.

[0097] In the review prompt generation process, the review prompt Q010 is generated by the above-mentioned process. The generated review prompt Q010 is transmitted from the terminal device 100 to the LLM 200.

[0098] When the LLM 200 receives the review prompt Q010, it generates and returns an answer A010. The answer A010 from the LLM 200 is in a format that conforms to the instructions and rules of the review prompt Q010.

[0099] Here, if answer A010 contains a sentence enclosed by "##" and "##", handover item instruction generation unit 125 registers it in the form of instruction symbols 101, 102... as additional handover items for chapter 1, in association with instruction symbol 10. If instruction symbols 101, 102... and the like are registered, prompt generation unit 120 extracts these instruction symbols and generates a prompt in the next or subsequent prompt generation process. Note that if answer A010 contains a sentence enclosed by "##" and "##", handover item instruction generation unit 125 may have the user confirm the sentence, and register it as an additional handover item if the user determines it is necessary.

[0100] In this way, the prompt generator 120 can generate a review prompt and send it to the LLM 200, thereby causing the LLM 200 to compare the sentence generated by the LLM 200 with the instruction symbols given when generating this sentence. This allows the LLM 200 to self-check whether a given objective has been achieved. Furthermore, by having the LLM 200 self-check, the performance of the LLM 200 can be improved.

[0101] In addition, the correction instruction generation unit may generate a sentence such as "If corrections are necessary, please edit the sentence" in the review prompt Q001, thereby instructing the LLM200 that it may directly edit the sentence, thereby improving the content of sentence S001.

[0102] According to an embodiment of the present disclosure, the above-described process can be repeatedly executed to cause the LLM 200 to execute the assigned work. Therefore, even if a question from a user is related to the generation of large-scale content such as a large document or source code, the large-scale content can be generated based on multiple answers by the LLM 200.

[0103] However, there are problems with answers given by LLM200. Answers output by LLM200 are not necessarily immediately in line with the user's intentions, so the user must give detailed correction instructions. This requires the user to repeatedly give correction instructions to LLM200 through trial and error. For the user, this task of repeatedly giving correction instructions through trial and error is a difficult task that requires patience. To solve these problems, it is desirable to be able to give correction instructions to LLM200 on a sentence-by-sentence basis as finely as possible.

[0104] To address this issue, according to an embodiment of the present disclosure, it is possible to instruct large-scale content to be generated through shared work by multiple members (to be completed through multiple responses to LLM 200), rather than being generated in a single response to LLM 200 (to be completed through the current response to LLM 200). Therefore, it is possible to issue correction instructions to LLM 200 on a sentence-by-sentence basis, as small as possible.

[0105] Furthermore, according to an embodiment of the present disclosure, the LLM 200 can first generate chapters (chapters, sections, and paragraphs), and then generate detailed parts (sentences, source code, etc.) corresponding to each chapter, etc.

[0106] Furthermore, according to an embodiment of the present disclosure, when the LLM 200 generates a portion corresponding to each chapter, etc., the LLM 200 generates an answer including handover items (handover notes) for the next LLM 200, and generates subsequent prompts according to this answer. Therefore, the subsequent LLM 200 generates the current answer while referring to the handover items generated by the LLM 200 itself, so that consistency can be maintained in the subsequent answers obtained from the LLM 200.

[0107] Furthermore, according to an embodiment of the present disclosure, the user is given the opportunity to modify handover items generated by the LLM 200 itself, so that the user can modify only the handover items.

[0108] In addition, in the embodiment of the present disclosure, when the content of the instruction symbol is interpreted, for example, "##···Describe the content of XX. In that case, delve deeper into the point XX.##", the LLM 200 may simply proceed to generate an answer according to the content of the instruction symbol. On the other hand, when the content of the instruction symbol is interpreted, for example, "##···Search for XX information and write based on that.##", an external tool such as a web search tool is used, and the search results are added to the prompt, and then the LLM 200 proceeds to generate an answer. However, this is not limited to this, and various tools other than web search may be used. For example, if a function for linking with cloud storage is provided, an answer can be generated while citing reference documents stored in advance in cloud storage. In addition, if a function for linking with a calendar is provided, an email text containing an appropriate free time can be written by referring to the calendar.

[0109] Furthermore, the prompt generation unit 120 may delete the instruction symbols after the generation of the sentence is completed, or may save (store) them without deleting them. If the instruction symbols are saved rather than deleted, a record may be kept to the effect that the generation process has been completed. By saving the instruction symbols, the user can use them again (or the next time or for a new sentence) to generate a sentence by modifying the instruction content corresponding to the instruction symbols, which has the advantage of reducing the burden on the user when generating a sentence again and allowing more accurate instructions to be given to the LLM 200.

[0110] Furthermore, the prompt generation unit 120 may add all of the answers (sentences) generated by the LLM 200 from the first prompt generated by the prompt generation unit 120 to the present time as a series of sentences within the upper limit number of tokens. Furthermore, the prompt generation unit 120 may summarize and add the answers (sentences) generated by the LLM 200 from the first prompt generated by the prompt generation unit 120 to the present time within the upper limit number of tokens. Furthermore, when summarizing, the prompt generation unit 120 may not summarize the prompt, but may appropriately summarize and add the answers (sentences) generated by the LLM 200 to the present time. Note that the summarization method by the prompt generation unit 120 is not limited to this. For example, sentences that are to be emphasized (or referred to) when the LLM 200 generates an answer based on the current prompt may not be summarized, and other sentences may be summarized. Furthermore, sentences that do not need to be referred to by the LLM 200 may be appropriately deleted.

[0111] In the embodiment of the present disclosure, the transmission of the prompt to the LLM 200 is executed in response to a prompt transmission operation by the user. However, the present disclosure is not limited to this, and the transmission may be automatic without a transmission operation by the user. Also, the part to be automatically transmitted and the part to be transmitted by a transmission operation by the user may be separated. In such a case, for example, automatic transmission is performed until the chapter is completed, a transmission operation by the user is required when proceeding from the chapter to the next, automatic transmission is performed until the generation of the text of the first chapter is completed, and a transmission operation by the user is required when proceeding from the text of the first chapter to the next. In such a case, the instruction correction process by the instruction correction unit 127 is not performed in the part to be automatically transmitted. [Explanation of symbols]

[0112] 100 Terminal Equipment 110 Sentence generation section 120 Prompt Generation Unit 130 Answer processing section 140 Transmission and reception processing unit 200 Large-scale language models

Claims

1. a prompt generator for generating a prompt to be input to a language model based on a question; The prompt generation unit is a sharing instruction generating unit that generates sharing instruction information that instructs the generation of the answer to be shared; 13. An information processing apparatus comprising: a handover instruction generating unit configured to generate handover instruction information that indicates handover items to be referred to by the language model when generating the answer.

2. 2 . The information processing apparatus according to claim 1 , wherein the handover item instruction generating unit adds a symbol that enables the language model to recognize a description location of the handover item when generating the handover instruction information.

3. 3. The information processing apparatus according to claim 1, wherein the handover instruction information generated by the handover item instruction generating unit can be modified by a user.

4. The information processing device according to claim 1 or 2, characterized in that, when a plurality of handover items exist in the answer generated by the language model, the handover item instruction generation unit extracts one handover item and generates the handover instruction information.

5. 3. The information processing apparatus according to claim 1, wherein the handover item instruction generating unit generates external information reference permission information that permits the language model to refer to external information when generating the handover instruction information.

6. 3. The information processing apparatus according to claim 1, wherein the takeover instruction generating unit is capable of generating instruction information for any one of search, summary, and rewrite when generating the takeover instruction information.

7. The information processing apparatus according to claim 1 , wherein the prompt generating unit stores the takeover instruction information without erasing it.

8. The prompt generation unit is The information processing apparatus according to claim 1 , further comprising a chaptering instruction generating unit for generating chaptering instruction information for instructing that the answer should be divided into chapters.

9. 1. A computer-implemented information processing method, comprising: a prompt generation step for generating a prompt based on the question for input to the language model; The prompt generating step includes: a sharing instruction generating step of generating sharing instruction information for instructing that the answer be generated in a sharing manner; and generating handover instruction information that indicates handover items to be referred to by the language model when generating the answer.

10. An information processing program that causes a computer to execute a prompt generation procedure for generating a prompt to be input to a language model based on a question, the program comprising: The step of generating a prompt comprises: a sharing instruction generating step for generating sharing instruction information for instructing that the answer is to be generated in a sharing manner; and a takeover instruction generating step for generating takeover instruction information that indicates takeover items to be referred to by the language model when generating a new answer following the answer.

Images