Sentence generation device and sentence generation method

Abstract

To generate a prompt that includes valid sentences as reference information for an input question sentence within a determined character limit.SOLUTION: A sentence generation device comprises: an additional sentence generation unit 11 that generates additional sentences related to a question sentence based on a question sentence input by a user, with a total number of characters including the number of characters in the question sentence not exceeding the character limit that can be input to an answer generation unit 20 of a large-scale language model (LLM) 1; and a prompt generation unit 12 that generates a prompt by adding the additional sentences generated by the additional sentence generation unit 11 to the input question sentence as reference information. The sentence generation device generates the additional sentences so that the number of characters does not exceed the character limit that can be input to the LLM 1 as a prompt, thereby making it possible to generate a prompt that adds valid sentences to the input question sentence as reference information within the determined character limit.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a sentence generation device and a sentence generation method, and more particularly to a technique for generating sentences to be input as prompts to a large-scale language model. [Background technology]

[0002] Conventionally, systems have been provided that use large-scale language models (LLMs) to perform natural language processing such as question-answering. LLMs are natural language processing models trained using large amounts of text data, and take sentences as input and output sentences. When LLMs are applied to a system that performs question-answering, when a question is input into the LLM, the LLM outputs an answer. In LLMs, the sentences that are input are called "prompts."

[0003] In recent years, the use of LLMs has progressed, but there are major issues regarding the accuracy of answers. In other words, the behavior of LLMs, such as confidently outputting incorrect answers, is viewed as problematic. As a means of solving this, prompts are generated by adding information to the question. When there is additional information, the LLM uses the additional information as a reference to generate an answer to the question. For example, as shown in Figure 13, by creating a prompt by adding reference information in addition to the question, it becomes easier to obtain an appropriate answer from the LLM that takes the reference information into consideration.

[0004] Conventionally, a method that combines the Completion model and the Embedding model has been proposed as a technology for generating prompts using additional information. The Completion model is a model that inputs a prompt containing a question sentence or a reference sentence, and outputs an answer sentence generated based on that. The Embedding model is a model that converts the input sentence into a numerical vector based on its meaning. When the meanings of two sentences are similar, the parameters of the model are adjusted so that the vectors generated from them are similar. The method that combines these models uses the vectors of the Embedding model to search for information closely related to the question sentence, reflects the search results in the prompt, and then executes the Completion model.

[0005] A technology for generating a prompt by adding reference information to an input question sentence, and generating an answer sentence by inputting the generated prompt into a language model is also disclosed in Patent Document 1. Patent Document 1 discloses, with regard to adding information to a prompt, adding an instruction sentence including an explanation about a chatbot character to the prompt, adding a character dialogue sample to the prompt, adding a dialogue scene acquired from a dialogue storage database in relation to the dialogue subject to the prompt, replacing words included in the question sentence with synonyms or the like and adding them to the prompt, etc.

[0006] Patent Documents 2 and 3 disclose generating another question from an input question and adding it to a prompt. Patent Document 2 discloses generating another question based on a history of past answers. Patent Document 3 discloses generating another question by replacing words included in a question with reduced terms or the like. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2022-180282 A [Patent Document 2] Special Publication No. 2001-519559 [Patent Document 3] Special Publication No. 2002-508863 Summary of the Invention [Problem to be solved by the invention]

[0008] As mentioned above, adding reference information to a question and inputting the generated prompt into the LLM is effective in that it makes it easier to obtain an appropriate answer than if no reference information is added.

[0009] In response to this, the present invention aims to generate a prompt that adds valid sentences as reference information to an input question sentence, using a method different from the methods described in Patent Documents 1 to 3. [Means for solving the problem]

[0010] In order to solve the above-mentioned problems, in the present invention, based on an input question text, candidate sentences related to the question text are obtained, and the additional sentences generated based on the obtained candidate sentences are added to the question text as reference information, thereby generating a prompt for input into a large-scale language model. Effect of the Invention

[0011] According to the present invention configured as described above, when generating a prompt by adding a sentence related to the input question, a candidate sentence related to the input question is obtained, and rather than using the candidate sentence as is as the added sentence, the additional sentence is generated based on the candidate sentence, so that a prompt can be generated in which a sentence that is valid for the input question is added as reference information. [Brief description of the drawings]

[0012] [Figure 1]FIG. 1 is a diagram illustrating an example of the configuration of a language model system including a sentence generation device according to an embodiment of the present invention. [Diagram 2] 2 is a block diagram showing an example of a functional configuration of a sentence generation unit (sentence generation device) according to the present embodiment. FIG. [Diagram 3] 11 is a block diagram showing an example of a functional configuration of a candidate sentence acquisition unit that implements a first candidate sentence acquisition method. FIG. [Figure 4] FIG. 13 is a diagram showing an example of a prompt constructed using candidate sentences respectively obtained from three sentence DBs. [Diagram 5] 13 is a block diagram showing an example of a functional configuration of a candidate sentence acquisition unit that implements a second candidate sentence acquisition method. FIG. [Figure 6] 13 is a block diagram showing an example of a functional configuration of a candidate sentence acquisition unit that implements a third candidate sentence acquisition method. FIG. [Figure 7] FIG. 13 is a diagram showing an example of a prompt and another question sentence generated by applying the third candidate sentence acquisition method. [Figure 8] 13A and 13B are diagrams illustrating other examples of prompts and alternative question sentences generated by applying the third candidate sentence acquisition method. [Figure 9] 13 is a block diagram showing an example of a functional configuration of a candidate sentence acquisition unit that implements a fourth candidate sentence acquisition method. FIG. [Figure 10] FIG. 13 shows example prompts for generating question-answer pairs for supervised learning. [Figure 11] FIG. 13 is a diagram showing two examples of candidate sentences adjusted by a first candidate sentence adjustment method. [Figure 12] 4 is a flowchart showing an example of the operation of a sentence generation unit (sentence generation device) according to the present embodiment. [Figure 13] This figure shows an example of the content of a prompt generated by adding reference information to a question sentence and inputting the generated prompt into an LLM to obtain an answer sentence. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] An embodiment of the present invention will be described below with reference to the drawings. Fig. 1 is a diagram showing an example of the configuration of a language model system 100 equipped with a sentence generation device of this embodiment. The language model system 100 of this embodiment is configured with a large-scale language model (LLM) 1 and a sentence database (DB) 2.

[0014] The LLM1 is connected to a user terminal 200 via a communication network 300 such as the Internet or a mobile phone network, inputs a question sentence from the user terminal 200, generates an answer sentence to the question sentence, and outputs it to the user terminal 200. When generating the answer sentence, the LLM1 generates an additional sentence related to the question sentence, and generates a prompt including the question sentence and the additional sentence.

[0015] The text DB2 is a database that the LLM1 uses when generating additional information based on a question entered from the user terminal 200. Note that, although Fig. 1 shows a configuration in which the text DB2 is connected to the LLM1, this is not limiting. For example, a configuration in which part or all of the text DB2 is connected to the communication network 300 may also be used. In this case, the LLM1 accesses the text DB2 via the communication network 300 and acquires the necessary information.

[0016] LLM1 is software that performs natural language processing according to a model trained using a large amount of text data, and includes a sentence generation unit 10 and an answer generation unit 20 as functional components realized by the software. The sentence generation unit 10 corresponds to the sentence generation device of this embodiment, and its functions are realized by a program stored in a storage medium such as a computer's RAM, ROM, hard disk, or semiconductor memory operating under the control of a CPU. The functions of the answer generation unit 20 are similarly realized.

[0017] The sentence generation unit 10 of this embodiment generates a prompt for input to the LLM 1 by adding reference information to the question sentence input from the user terminal 200. The prompt for input to the LLM 1 is specifically a prompt for input to the answer generation unit 20. In other words, the sentence generation unit 10 generates additional sentence based on the question sentence input from the user terminal 200, and further generates sentence by adding this additional sentence as reference information for the question sentence, and inputs this to the answer generation unit 20 as a prompt.

[0018] The answer generation unit 20 inputs the prompt generated by the sentence generation unit 10, and generates an answer sentence to the question sentence included in the prompt by using the additional sentence included in the prompt as reference information. In other words, the answer generation unit 20 generates an answer sentence that is more suitable for the reference information.

[0019] Fig. 2 is a block diagram showing an example of a functional configuration of the sentence generation unit 10. As shown in Fig. 2, the sentence generation unit 10 of this embodiment includes, as its functional configuration, an additional sentence generation unit 11 and a prompt generation unit 12. The additional sentence generation unit 11 includes, as its more specific functional configuration, a candidate sentence acquisition unit 111 and a sentence adjustment unit 112.

[0020] The additional sentence generation unit 11 generates additional sentences related to the question sentence input from the user terminal 200, with the total number of characters, including the number of characters in the question sentence, not exceeding the limit on the number of characters that can be input to the answer generation unit 20 of the LLM 1, based on the question sentence input from the user terminal 200. The prompt generation unit 12 generates a prompt by adding the additional sentences generated by the additional sentence generation unit 11 as reference information to the question sentence input from the user terminal 200.

[0021] For example, as shown in Fig. 13, when a question such as "Is Hiroshima a good place to hang out laundry tomorrow?" is input from the user terminal 200, the additional sentence generation unit 11 generates three additional sentences indicated by #Reference facts, and the prompt generation unit 12 generates a prompt by adding these additional sentences to the question. In the example of Fig. 13, a sentence such as "Please answer the following #Question. When answering, please refer to the contents written in #Reference facts and answer based on that" is also added to the beginning of the prompt. At this time, the additional sentence generation unit 11 generates the additional sentences so that the total number of characters of the prompt consisting of the input question and additional sentences does not exceed the limit on the number of characters that can be input to the LLM1.

[0022] Here, the candidate sentence acquisition unit 111 of the additional sentence generation unit 11 acquires candidate sentences related to the question sentence based on the question sentence input from the user terminal 200. In order for the answer generation unit 20 to generate a more appropriate answer sentence for the question sentence, it is important to generate appropriate additional sentences for the question sentence. In order to generate appropriate additional sentences, it is important to acquire appropriate candidate sentences from the sentence DB 2. To this end, the candidate sentence acquisition unit 111 acquires candidate sentences using one or more of the methods described below.

[0023] The sentence adjustment unit 112 generates additional sentences by adjusting the candidate sentences acquired by the candidate sentence acquisition unit 111. When the number of characters in the candidate sentences acquired by the candidate sentence acquisition unit 111 is large, if the candidate sentences are added as they are to the question sentence to generate a prompt, the total number of characters may exceed the limit on the number of characters that can be input to the LLM 1. In such a case, the sentence adjustment unit 112 adjusts the candidate sentences using one or more of the methods described below, so that the total number of characters falls within the character limit.

[0024] The following is a detailed description of a method for acquiring candidate sentences by the candidate sentence acquiring unit 111. The candidate sentence acquiring unit 111 can acquire candidate sentences by applying any one or more of the methods described below.

[0025] <First method for obtaining candidate sentences> 3 is a block diagram showing an example of the functional configuration of the candidate sentence acquisition unit 111 that implements the first candidate sentence acquisition method. In the example shown in FIG. 3, the candidate sentence acquisition unit 111 includes a keyword extraction unit 111a and a keyword search unit 111b.

[0026] The keyword extraction unit 111a extracts search keywords from a question sentence input from the user terminal 200. For example, the keyword extraction unit 111a extracts words as search keywords from the input question sentence by morphological analysis. Here, the words extracted may be limited to specific parts of speech (e.g., nouns, verbs, adjectives, adjectival verbs, etc.), or may be extracted excluding specific parts of speech (e.g., particles, auxiliary verbs, adverbs, conjunctions, conjunctions, interjections, etc.). One or more words may be extracted, but when multiple words are extracted and used as search keywords, it is possible to search for and acquire sentences that are more closely related to the question sentence.

[0027] The keyword search unit 111b uses the search keyword extracted by the keyword extraction unit 111a to search the sentence DB2 in which multiple sentences are recorded, and acquires sentences containing the search keyword as candidate sentences from the sentence DB2. When multiple search keywords are extracted by the keyword extraction unit 111a, the keyword search unit 111b acquires sentences containing all of the multiple search keywords as candidate sentences from the sentence DB2. If no matching sentences are found, the search may be re-executed by omitting one of the search keywords.

[0028] When there are multiple sentences that match the search keyword, the keyword search unit 111b may acquire all of the multiple sentences. If all of the multiple sentences that match the search keyword are acquired, the total number of characters is likely to exceed the limit on the number of characters that can be input to the LLM1. Therefore, a predetermined number of sentences that match the search keyword most closely may be acquired as candidate sentences.

[0029] Here, there may be multiple types of sentence DB2 to be searched, as shown in Fig. 3. In Fig. 3, three are shown: fact information DB2a, persona information DB2b, and correct answer example information DB2c (hereinafter, the three may be collectively referred to as sentence DB2a-2c). The keyword search unit 111b searches for sentences using the words extracted by the keyword extraction unit 111a as search keywords, targeting one or more of these three sentence DB2a-2c. In order to improve the accuracy of the answer sentences generated by the answer generation unit 20, it is preferable to search multiple databases.

[0030] The fact information DB2a is a database that holds, in a referable state, for example, a web page of a specific company, Wikipedia (registered trademark), or files such as txt, csv, word, excel, and pdf uploaded by a user using the user terminal 200. The persona information DB2b is a database that holds a history of past dialogue between a specific user and LLM1, or information extracted therefrom that can estimate the literacy of the user (for example, the level of prior knowledge in a specific technical field, past travel history, demographic attributes such as age, sex, and family structure, etc.). The correct answer example information DB2c is a database that holds information in which a person manually corrects the history of past dialogue between a user and LLM1, such as "LLM1 should have originally responded like this."

[0031] Fig. 4 is a diagram showing an example of a prompt configured using candidate sentences acquired from three sentence DBs 2a to 2c. In Fig. 4, "#referenceable information" is a candidate sentence acquired from fact information DB2a, "#persona of questioner" is a candidate sentence acquired from persona information DB2b, and "#helpful answer example" is a candidate sentence acquired from correct answer example information DB2c. In the example shown in Fig. 4, since the total number of characters of all the sentences included in the prompt does not exceed the limit of the number of characters that can be input to LLM1, it is not necessary to execute the process of sentence adjustment unit 112, and the prompt shown in Fig. 4 is input to answer generation unit 20.

[0032] <Second method for obtaining candidate sentences> Fig. 5 is a block diagram showing an example of a functional configuration of the candidate sentence acquisition unit 111 that implements the second candidate sentence acquisition method. In the example shown in Fig. 5, the candidate sentence acquisition unit 111 includes a vector calculation unit 111c and a vector search unit 111d.

[0033] The vector calculation unit 111c calculates a feature vector of a sentence from a question sentence input from the user terminal 200. For example, the vector calculation unit 111c includes a small-scale language model of an Embedding model, and converts the input question sentence into a numerical vector using the Embedding model. The small-scale language model is a natural language processing model trained based on less data than LLM1. Note that the method of calculating the feature vector is not limited to this, and other known techniques may be applied. For example, words may be extracted from the input question sentence by morphological analysis, and the meanings of the words and the relationships between the words may be vectorized.

[0034] In order to apply the second candidate sentence acquisition method, feature vectors are calculated in advance for the multiple sentences recorded in the sentence DBs 2a to 2c, and the multiple sentences are recorded in advance in association with their respective feature vectors. The vector search unit 111d searches the sentence DBs 2a to 2c using the feature vector calculated by the vector calculation unit 111c for the input question sentence, and acquires, as candidate sentences, sentences associated with the feature vector that satisfies a predetermined condition for the similarity with the feature vector calculated by the vector calculation unit 111c from the sentence DBs 2a to 2c.

[0035] The predetermined condition for acquiring sentences can be determined arbitrarily. For example, it is possible to acquire a predetermined number of sentences starting from those with the highest similarity of feature vectors. Alternatively, sentences with feature vector similarity greater than a predetermined threshold value may be acquired. In this case, all sentences with feature vector similarity greater than a predetermined threshold value may be acquired, or if the number of sentences with feature vector similarity greater than the predetermined threshold value exceeds a predetermined number, a predetermined number of sentences starting from those with the highest similarity may be acquired.

[0036] <Third method for obtaining candidate sentences> Fig. 6 is a block diagram showing an example of a functional configuration of the candidate sentence acquisition unit 111 that implements the third candidate sentence acquisition method. In the example shown in Fig. 6, the candidate sentence acquisition unit 111 further includes a question sentence generation unit 111e in addition to the vector calculation unit 111c and the vector search unit 111d shown in Fig. 5. Moreover, the sentence DB2 further includes an example question sentence DB2d that records a plurality of example question sentences.

[0037] The question generation unit 111e generates another question from the question input from the user terminal 200. The vector calculation unit 111c calculates a feature vector of the question input from the user terminal 200, as well as a feature vector of the other question generated by the question generation unit 111e. The vector search unit 111d searches the sentence DBs 2a to 2c using the feature vector calculated from the question input from the user terminal 200 and the feature vector calculated from the other question generated by the question generation unit 111e, and acquires candidate sentences.

[0038] Here, the question generation unit 111e has a small-scale language model generated by applying few-shot learning, for example, and generates a prompt by adding an example question to the input question, and inputs the example question to the small-scale language model, thereby outputting another question from the small-scale language model. The example question is acquired from the example question DB2d. FIG. 7 is a diagram showing an example of a prompt and another question generated by the candidate sentence acquisition unit 111 in this case. In the example of FIG. 7, a prompt is generated by adding an example question extracted from the example question DB2d using "Okinawa trip" contained in the input question as a keyword, and another question is generated as shown in the example question.

[0039] Furthermore, the question generation unit 111e may dynamically generate a reference question generation example to configure a prompt in response to a question sentence input from the user terminal 200. For example, the question generation unit 111e can extract an example question sentence that matches the question sentence input from the user terminal 200 from the example question sentence DB 2d, and generate a reference question generation example from the extracted example question sentence to configure a prompt. In this example, the example question sentence that matches the input question sentence can be, for example, an example question sentence associated with a feature vector whose similarity with a feature vector calculated from the input question sentence is equal to or greater than a threshold value.

[0040] FIG. 8 is a diagram showing an example of a prompt and another question sentence generated by the candidate sentence acquisition unit 111 in this case. In the example of FIG. 8, a plurality of example question sentences matching the input question sentence are extracted from the example question sentence DB2d, and a reference question generation example is generated from these to configure the prompt. The small-scale language model dynamically generates another question sentence using the plurality of question generation examples included in this prompt. In this example, the small-scale language model is generated by training using a past conversation history, and instead of generating another question sentence according to the question generation example, the question generation example and the conversation history are input to generate another question sentence.

[0041] <The fourth method for obtaining candidate sentences> Fig. 9 is a block diagram showing an example of the functional configuration of a candidate sentence acquisition unit 111 that implements the fourth candidate sentence acquisition method. In the example shown in Fig. 9(a), the candidate sentence acquisition unit 111 includes a question example generation unit 111f and a question and answer generation unit 111g. In addition, the sentence DB2 does not include sentence DB2a to 2c, but includes an example question sentence DB2d.

[0042] The question example generating unit 111f generates a reference question example from a question sentence input from the user terminal 200. The method of generating a reference question example by the question example generating unit 111f may be the same as the method of generating a reference question example by the question sentence generating unit 111e illustrated in FIG. 6 and FIG. 8. That is, the question example generating unit 111f extracts an example question sentence matching the question sentence input from the user terminal 200 from the example question sentence DB2d, and generates a reference question example from the extracted example question sentence. Alternatively, since the question and answer generating unit 111g is a small-scale language model with flexible reading ability, the question input from the user terminal 200 may be directly input to the question and answer generating unit 111g without bothering to generate a question example. In this case, the question example generating unit 111f and the example question sentence DB2d may be deleted from the configuration.

[0043] The question and answer generation unit 111g inputs reference question examples generated by the question example generation unit 111f to the small-scale language model, and obtains combinations of question sentences and answer sentences as candidate sentences from the small-scale language model.

[0044] The small-scale language model provided in the question and answer generation unit 111g can be, for example, a model obtained by fine-tuning LLM1 to suit a specific task. Although the small-scale language model is a small-scale model, it is practically difficult to fine-tune for each user and to always store multiple small-scale language models generated by this in the server memory. To solve this problem, a well-known additional learning method called LoRA may be used.

[0045] LoRA adds an ultra-small language model called an Adapter to an original small language model for learning, and can perform fine tuning efficiently with low memory. The original small language model is usually several tens of GBytes, whereas the Adapter is usually only several tens of MBytes. Therefore, even if the small language model is made to reside in memory and only the Adapter is loaded on demand in response to a user request, the load time is negligible. Alternatively, it is possible to place a large number of Adapters in a limited memory. This method can be used not only in the question and answer generation unit 111g but also in general in situations where a language model is used.

[0046] Either self-supervised learning or supervised learning may be used to fine-tune the small-scale language model of the question and answer generation unit 111g. When using self-supervised learning, it is possible to learn by extracting input and output pairs from a back-and-forth dialogue of the past. On the other hand, when using supervised learning, it is possible to generate questions and answers by another LLM that refers to the past dialogue history through prompts, and it is possible to learn these question and answer pairs. An example of a prompt for obtaining question and answer pairs is shown in FIG. 10.

[0047] In the example shown in FIG. 9(b), the candidate sentence acquisition unit 111 includes a question sentence generation unit 111h and an answer sentence acquisition unit 111j. The question sentence generation unit 111h generates another question sentence from a question sentence input from the user terminal 200. The method of generating another question sentence by the question sentence generation unit 111h may be the same as the method of generating another question sentence by the question sentence generation unit 111e illustrated in FIGS. 6 to 8. Alternatively, since the answer sentence acquisition unit 111j is a small-scale language model with flexible reading ability, the question sentence input from the user terminal 200 may be input directly to the answer sentence acquisition unit 111j without bothering to generate a question sentence. In this case, the question sentence generation unit 111h and the example question sentence DB2d may be deleted from the configuration.

[0048] The answer sentence acquisition unit 111j inputs another question sentence generated by the question sentence generation unit 111h to the small-scale language model, and acquires an answer sentence to the other question sentence as a candidate sentence from the small-scale language model. In the example shown in Fig. 9(b), a combination of the question sentence generated by the question sentence generation unit 111h and the answer sentence acquired by the answer sentence acquisition unit 111j is output as a candidate sentence from the candidate sentence acquisition unit 111. In this case, the question sentence generation unit 111h and the answer sentence acquisition unit 111j constitute a question and answer generation unit in the claims.

[0049] When the candidate sentence acquisition unit 111 acquires candidate sentences using the method described above, the number of acquired candidate sentences is often multiple, and the acquired candidate sentences may be redundant. Therefore, if the candidate sentences acquired by the candidate sentence acquisition unit 111 are added directly to a question sentence to generate a prompt, the total number of characters in the prompt may exceed the limit on the number of characters that can be input by the LLM1. In such a case, processing by the sentence adjustment unit 112 is required.

[0050] The following describes in detail how the sentence adjuster 112 adjusts the candidate sentences. The sentence adjuster 112 can adjust the candidate sentences by applying one or more of the techniques described below.

[0051] <First candidate sentence adjustment method> The sentence adjustment unit 112 inputs the question sentence input from the user terminal 200 and the candidate sentence acquired by the candidate sentence acquisition unit 111 into the small-scale language model, and generates an additional sentence with a reduced number of characters by using a sentence portion of the candidate sentence related to the question sentence. The small-scale language model provided in the sentence adjustment unit 112 may be, for example, the LLM1 of this embodiment that has been independently fine-tuned for the above task, or may be a small-scale language model generated for the above task.

[0052] Fig. 11 is a diagram showing two examples of candidate sentence adjustment by the first candidate sentence adjustment method. In the example shown in Fig. 11, the candidate sentence acquired by the candidate sentence acquisition unit 111 is "When you think of a trip to Okinawa, it's the sea in summer and the aquarium in winter!", and in order to adjust it and reduce the number of characters, this candidate sentence and the question sentence input from the user terminal 200 are input to the small-scale language model.

[0053] Figure 11(a) shows an example of candidate sentence adjustment when the input question is "I want to go on a trip to Okinawa in winter. What is a recommended plan for 3 nights and 4 days?" Since the question is about a trip to Okinawa in winter, the small-scale language model evaluates the part of the candidate sentence "In summer, the sea" as being less relevant to the question, and generates the additional sentence "For a trip to Okinawa, the aquarium is recommended" using the part of the candidate sentence that is related to the question, "In winter, the aquarium."

[0054] In contrast, Figure 11(b) shows an example of candidate sentence adjustment when the input question is "I want to go on a trip to Okinawa in the summer. What is a recommended plan for three nights and four days?" Since the question is about a summer trip to Okinawa, the small-scale language model evaluates the relevance of the part of the candidate sentence, "In winter, the aquarium," to the question as low, and generates the additional sentence, "For a trip to Okinawa, the sea is recommended," using the part of the candidate sentence, "In summer, the sea," which is related to the question.

[0055] For the sake of explanation, an example of adjusting one candidate sentence is shown here, but adding only the one candidate sentence shown here to the question sentence will not exceed the character limit that can be input to the LLM1. In reality, if there are multiple candidate sentences and adding all of them to the question sentence would cause the total number of characters to exceed the character limit, the above-mentioned adjustment process is performed on each candidate sentence. This also applies to the second candidate sentence adjustment method described below.

[0056] <Second candidate sentence adjustment method> The sentence adjustment unit 112 may generate additional sentences with a reduced number of characters by summarizing the candidate sentences acquired by the candidate sentence acquisition unit 111. The candidate sentences may be summarized using a small-scale language model generated for the task, or using a known algorithm.

[0057] Furthermore, the sentence adjustment unit 112 may generate additional sentences with a reduced number of characters by summarizing key points related to the question sentence input from the user terminal 200 for the candidate sentences acquired by the candidate sentence acquisition unit 111. In this case, the sentence adjustment unit 112 can be configured to include a small-scale language model that receives as input the question sentence input from the user terminal 200 and the candidate sentences acquired by the candidate sentence acquisition unit 111, and is trained to output the summarized candidate sentences.

[0058] Fig. 12 is a flowchart showing an example of the operation of the sentence generation unit 10 according to this embodiment configured as described above. The flowchart shown in Fig. 12 shows an overview of the procedure of processing executed for one question sentence input from the user terminal 200 to the LLM 1.

[0059] First, the additional sentence generation unit 11 of the LLM1 inputs a question sentence from the user terminal 200 (step S1). When the additional sentence generation unit 11 inputs the question sentence, the candidate sentence acquisition unit 111 applies any one or more of the first to fourth candidate sentence acquisition methods to acquire one or more candidate sentences related to the input question sentence (step S2).

[0060] Next, the sentence adjustment unit 112 determines whether the total number of characters, which is the sum of the number of characters in the question sentence input from the user terminal 200 and the number of characters in one or more candidate sentences acquired by the candidate sentence acquisition unit 111, exceeds the limit on the number of characters that can be input to the LLM 1 (step S3). If it is determined that the total number of characters exceeds the limit on the number of input characters, the sentence adjustment unit 112 applies either one or both of the first and second candidate sentence adjustment methods to reduce the number of characters in the candidate sentences so that the total number of characters falls within the limit on the number of input characters (step S4).

[0061] Here, the sentence adjustment unit 112 adjusts, for example, all the candidate sentences acquired by the candidate sentence acquisition unit 111 to reduce the number of characters, and re-determines whether the total number of characters after the reduction exceeds the limit on the number of input characters. If the total number of characters still exceeds the limit on the number of input characters, the sentence adjustment unit 112 adjusts the candidate sentences again to reduce the number of characters. Alternatively, it may be possible to delete some of the candidate sentences acquired by the candidate sentence acquisition unit 111. By repeating such processing, the total number of characters is set within the limit on the number of input characters. One or more candidate sentences adjusted in this manner are determined as additional sentences for the question sentence.

[0062] Once additional sentences for the question have been generated by adjusting the candidate sentences by the sentence adjustment unit 112 in the above manner, the prompt generation unit 12 generates a prompt by adding the additional sentences generated by the sentence adjustment unit 112 in step S4 as reference information to the question input from the user terminal 200 in step S1 (step S5).

[0063] If it is determined in step S3 above that the total number of characters does not exceed the limit on the number of input characters, the processing of the sentence adjustment unit 112 in step S4 is not performed, and the candidate sentence acquired by the candidate sentence acquisition unit 111 is determined as an additional sentence to the question sentence as is. In this case, the prompt generation unit 12 generates a prompt by adding the candidate sentence (=additional sentence) acquired by the candidate sentence acquisition unit 111 in step S2 as reference information to the question sentence input from the user terminal 200 in step S1 (step S5).

[0064] The prompt generator 12 then inputs the generated prompt to the answer generator 20 (step S6). The answer generator 20 uses the additional sentence included in the prompt input by the sentence generator 10 as reference information to generate an answer sentence to the question sentence included in the prompt, and outputs the generated answer sentence to the user terminal 200 (step S7). This ends the processing of the flowchart shown in FIG.

[0065] As described in detail above, in this embodiment, based on a question text inputted to the LLM1 from the user terminal 200, additional text related to the question text is generated, the total number of characters including the number of characters in the question text does not exceed the character limit that can be inputted to the answer generation unit 20 of the LLM1, and the generated additional text is added to the question text as reference information, thereby generating a prompt to be inputted to the answer generation unit 20.

[0066] According to this embodiment configured as described above, when generating a prompt by adding a sentence related to the input question, the additional sentence is generated so that the number of characters does not exceed the character limit that can be input to the answer generation unit 20 as a prompt. Therefore, a prompt can be generated that adds valid sentence for the input question as reference information within the set character limit.

[0067] In the above embodiment, the LLM 1 is configured with the sentence generation unit 10, but the present invention is not limited to this. For example, the sentence generation unit 10 may be configured separately from the LLM 1.

[0068] In the above embodiment, the user may be allowed to select the output format of the answer sentence output by the answer generation unit 20. For example, the tone of the answer sentence may be selected. This can be realized by specifying, in an additional sentence, "answer in a tone of voice like XX" when the prompt is generated by the sentence generation unit 10. As a property of the LLM, when reference information is described in the prompt, the specification of the tone of voice tends to be ignored and the tone of the reference information takes precedence. To solve this, an instruction may be given to the LLM to output in two stages. That is, an instruction to "first write a normal answer following 'normal answer:', and then write a normal answer following 'XX character:' in the tone of voice of XX character" may be included as an additional sentence of the prompt. In this case, by outputting the part of the output of the LLM after "XX character:", only the part that imitates the character's tone of voice can be presented to the user. Also, the format of the answer sentence may be selected. For example, the answer may be output according to chapters such as "summary", "business content", "application items", and "additional notes" in accordance with the format of government declaration documents. This can also be achieved by specifying additional text in the prompt.

[0069] In addition, the sentence DBs 2a to 2d or the small-scale language models used in the above embodiment can be sold to companies. A lump-sum payment or a subscription type may be used. When selling the sentence DBs 2a to 2d or the small-scale language models, it is necessary to remove personal information and confidential company information. This work may be done manually or by using an LLM generated for the task. The output format of the answer sentences described above may also be made available for sale to companies.

[0070] In addition, the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited thereby. In other words, the present invention can be implemented in various forms without departing from the gist or main characteristics thereof. [Explanation of symbols]

[0071] 1 Large-scale Language Models (LLM) 2 Text DB 2a Factual Information Database 2b Persona information DB 2c Correct Answer Information DB 10 Sentence generation unit (sentence generation device) 11 Additional text generation section 12 Prompt Generation 20 Answer generation part 111 Candidate sentence acquisition part 111a Keyword Extraction Section 111b Keyword Search Section 111c Vector calculation section 111d Vector Search Section 111e Question generator 111f Question generation example generator 111g Question / Answer Generator 111h Question generation unit (question / answer generation unit) 111j Answer text acquisition unit (question / answer generation unit) 112 Text Adjustment Department

Claims

1. A sentence generation device that generates a prompt for input to a large-scale language model by adding reference information to an input question sentence, a candidate sentence acquisition unit that acquires, based on the input question sentence, candidate sentences related to the question sentence, the candidate sentences being used for generating additional sentences to be added to the question sentence as the reference information; a prompt generating unit that generates the prompt by adding, as the reference information, the additional sentence generated based on the candidate sentence acquired by the candidate sentence acquiring unit to the input question sentence. A sentence generation device comprising:

2. The candidate sentence acquisition unit is a vector calculation unit for calculating a feature vector of a sentence from the input question sentence; and a vector search unit that searches a sentence database in which a plurality of sentences are recorded in advance in association with each of the feature vectors by using the feature vector calculated by the vector calculation unit, and acquires, from the sentence database, a sentence associated with a feature vector that satisfies a predetermined condition regarding a similarity with the feature vector calculated by the vector calculation unit, as the candidate sentence.

2. The text generating apparatus according to claim 1,

3. The candidate sentence acquisition unit is A question generation unit that generates another question from the input question, the vector calculation unit calculates a feature vector of the input question sentence and also a feature vector of the separate question sentence generated by the question sentence generation unit; The vector search unit searches a text database using the feature vector calculated from the input question sentence and the feature vector calculated from the different question sentence.

3. The text generating apparatus according to claim 2.

4. The candidate sentence acquisition unit is a keyword extraction unit that extracts search keywords from the input question; a keyword search unit that searches a sentence database in which a plurality of sentences are recorded, using the search keyword extracted by the keyword extraction unit, to obtain sentences including the search keyword from the sentence database as the candidate sentences.

2. The text generating apparatus according to claim 1,

5. The text generation device according to claim 1, characterized in that the candidate text acquisition unit includes a question and answer generation unit that generates, based on the input question text, a combination of another question text and an answer text using a small-scale language model as the candidate text.

6. A method for generating a prompt for input to a large-scale language model by adding reference information to an input question sentence, comprising: a first step of acquiring, by a candidate sentence acquisition unit of a computer, candidate sentences related to the input question sentence, which are used for generating additional sentences to be added to the question sentence as the reference information, based on the input question sentence; a second step in which a prompt generating unit of the computer generates the prompt by adding, as the reference information, the additional sentence generated based on the candidate sentence acquired by the candidate sentence acquiring unit to the input question sentence. A sentence generation method comprising:

Images