Document generation system and operation method thereof

Abstract

To provide a document generation system using a language model and an operation method thereof.SOLUTION: Provided are a document generation system having a first component and an operation method thereof, wherein the first component has the functions of: receiving a question sentence; making a language model predict the number of tokens of an answer sentence to the question sentence; executing a similarity search on a database on the basis of the question sentence; creating a prompt from the question sentence and the result of similarity search; calculating the sum of the number of tokens of the prompt and the predicted number of tokens of the answer sentence; determining that the sum is smaller than or equal to a prescribed value; and making the language model generate an answer sentence to the prompt when it is determined that the sum is smaller than or equal to a prescribed value.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a document generation system that uses a language model, typically a large language model (LLM), and an operating method thereof.

[0002] The above technical field is one embodiment of the present invention, and the present invention is not limited to the above technical field. Other embodiments of the present invention can include, for example, a semiconductor device, a display device, a light-emitting device, a power storage device, a memory device, an electronic device, a lighting device, an input device (for example, a touch sensor), an input / output device (for example, a touch panel), a driving method thereof, or a manufacturing method thereof. [Background technology]

[0003] An example of a conversational AI service is ChatGPT, which uses LLMs such as GPT-3 (Generative Pre-trained Transformer 3) and GPT-4 (Generative Pre-trained Transformer 4).

[0004] The language models used in conversational AI services are natural language processing models that use generative AI. Various studies are underway to improve the accuracy of language models in answering questions. In order to improve the accuracy of answers from LLMs that have only learned general information, a system has been proposed in which natural language processing is performed on questions, information highly relevant to the question is searched for in a database or the like, and the highly relevant information is passed to the LLM together with the question to allow it to answer. Non-Patent Document 1 discloses a specific system for providing information other than training data to the LLM. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Patrick Lewis et.al.,”Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”, 12 Apr 2021 Summary of the Invention [Problem to be solved by the invention]

[0006] LLM processes text data in units called tokens. LLM has a limit on the maximum number of tokens that can be processed at one time. However, the system in Non-Patent Document 1 does not consider the number of tokens at all.

[0007] The present invention has been made in view of the above problems, and an object of one embodiment of the present invention is to provide a novel document generation system and an operation method thereof. Another object of one embodiment of the present invention is to provide a method for operating a document generation system using a language model with the number of tokens set in the language model or less.

[0008] The present invention does not necessarily solve all of these problems. Furthermore, the description of these problems does not preclude the existence of other problems of the present invention. For example, problems other than these can be extracted from the description of the specification, drawings, and claims. [Means for solving the problem]

[0009] In view of the above problems, one aspect of the present invention is a document generation system having a first component, which has the following functions: accepting a question; having a language model predict the number of tokens in an answer to the question; performing a similarity search on a database based on the question; creating a prompt from the question and the results of the similarity search; calculating the sum of the number of tokens in the prompt and the number of tokens in the predicted answer; determining whether the sum is less than or equal to a predetermined value; and, if it is determined that the sum is less than or equal to the predetermined value, having the language model generate an answer to the prompt.

[0010] Another aspect of the present invention is a document generation system having a first component that has the following functions: accepting a question; having a language model predict the number of tokens in an answer to the question; performing a similarity search on a database based on the question; creating a first prompt from the question and the results of the similarity search; calculating a first sum of the number of tokens in the first prompt and the number of tokens in the predicted answer; determining whether the first sum is less than or equal to a predetermined value; and, if the first sum is determined to be less than or equal to the predetermined value, having the language model summarize the results of the similarity search; creating a second prompt from the question and the summary of the results of the similarity search; calculating a second sum of the number of tokens in the second prompt and the number of tokens in the predicted answer; determining whether the second sum is less than or equal to a predetermined value; and, if it is determined that the second sum is less than or equal to the predetermined value, having the language model generate an answer to the second prompt.

[0011] Another aspect of the present invention is a document generation system having a first component that has the following functions: accepting a question; having a language model predict the number of tokens in an answer to the question; performing a similarity search on a database based on the question; creating a first prompt from the question and the results of the similarity search; calculating a first sum of the number of tokens in the first prompt and the number of tokens in the predicted answer; determining whether the first sum is less than or equal to a predetermined value; deleting the result with the lowest similarity from the results of the similarity search if the first sum exceeds the predetermined value; creating a second prompt from the question and the results of the similarity search after the deletion; calculating a second sum of the number of tokens in the second prompt and the number of tokens in the predicted answer; determining whether the second sum is less than or equal to a predetermined value; and causing the language model to generate an answer to the second prompt if it is determined that the second sum is less than or equal to the predetermined value.

[0012] Another aspect of the present invention is a document generation system having a first component that has the following functions: accepting a question; having a language model predict the number of tokens in an answer to the question; performing a similarity search on a database based on the question; creating a first prompt from the question and the results of the similarity search; calculating a first sum of the number of tokens in the first prompt and the number of tokens in the predicted answer; determining whether the first sum is less than or equal to a predetermined value; causing the language model to summarize the results of the similarity search if the first sum exceeds the predetermined value; deleting the result with the lowest similarity from the summarized results of the similarity search; creating a second prompt from the question and the results of the similarity search after the deletion; calculating a second sum of the number of tokens in the second prompt and the number of tokens in the predicted answer; determining whether the second sum is less than or equal to a predetermined value; and causing the language model to generate an answer to the second prompt if it is determined that the second sum is less than or equal to the predetermined value.

[0013] Another aspect of the present invention is a method for operating a document generation system, comprising the steps of accepting a question sentence, having a language model predict the number of tokens in an answer sentence to the question sentence, performing a similarity search on a database based on the question sentence, creating a prompt from the question sentence and the results of the similarity search, calculating the sum of the number of tokens in the prompt and the number of tokens in the predicted answer sentence, determining whether the sum is less than or equal to a predetermined value, and, if it is determined that the sum is less than or equal to the predetermined value, having the language model generate an answer sentence to the prompt.

[0014] Another aspect of the present invention is a method for operating a document generation system, comprising the steps of: accepting a question; having a language model predict the number of tokens in an answer to the question; performing a similarity search on a database based on the question; creating a first prompt from the question and the results of the similarity search; calculating a first sum of the number of tokens in the first prompt and the number of tokens in the predicted answer; determining whether the first sum is less than or equal to a predetermined value; and, if the first sum exceeds the predetermined value, having the language model summarize the results of the similarity search; creating a second prompt from the question and the summary of the results of the similarity search; calculating a second sum of the number of tokens in the second prompt and the number of tokens in the predicted answer; determining whether the second sum is less than or equal to a predetermined value; and, if it is determined that the second sum is less than or equal to the predetermined value, having the language model generate an answer to the second prompt.

[0015] Another aspect of the present invention is a method for operating a document generation system, comprising the steps of: accepting a question; having a language model predict the number of tokens in an answer to the question; performing a similarity search on a database based on the question; creating a first prompt from the question and the results of the similarity search; calculating a first sum of the number of tokens in the first prompt and the number of tokens in the predicted answer; determining whether the first sum is less than or equal to a predetermined value; deleting the result with the lowest similarity from the results of the similarity search if the first sum exceeds the predetermined value; creating a second prompt from the question and the results of the similarity search after the deletion; calculating a second sum of the number of tokens in the second prompt and the number of tokens in the predicted answer; determining whether the second sum is less than or equal to a predetermined value; and, if it is determined that the second sum is less than or equal to the predetermined value, having the language model generate an answer to the second prompt.

[0016] Another aspect of the present invention is a method for operating a document generation system, comprising the steps of: accepting a question; having a language model predict the number of tokens in an answer to the question; performing a similarity search on a database based on the question; creating a first prompt from the question and the results of the similarity search; calculating a first sum of the number of tokens in the first prompt and the number of tokens in the predicted answer; determining whether the first sum is less than or equal to a predetermined value; if the first sum exceeds the predetermined value, having the language model summarize the results of the similarity search; deleting the result with the lowest similarity from the summarized results of the similarity search; creating a second prompt from the question and the results of the similarity search after the deletion; calculating a second sum of the number of tokens in the second prompt and the number of tokens in the predicted answer; determining whether the second sum is less than or equal to a predetermined value; and, if it is determined that the second sum is less than or equal to the predetermined value, having the language model generate an answer to the second prompt. [Effects of the Invention]

[0017] According to one aspect of the present invention, it is possible to provide a novel method for operating a document generation system. Also, according to another aspect of the present invention, it is possible to provide a method for operating a document generation system using a language model with the number of tokens set in the language model or less.

[0018] The present invention does not necessarily have to have all of these effects. Furthermore, the description of these effects does not preclude the existence of other effects of the present invention. For example, other effects can be extracted from the description of the specification, drawings, and claims. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 1 is a flowchart showing an example of a method for operating the document generation system of the present invention. [Figure 2] FIG. 2 is a flowchart showing an example of a method of operation of the document generation system of the present invention. [Figure 3] FIG. 3 is a diagram showing an example of a document generation system according to the present invention. [Figure 4] FIG. 4 is a diagram showing an example of a document generation system according to the present invention. [Figure 5] FIG. 5 is a diagram showing an example of a document generation system according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0020] The following description will discuss embodiments of the present invention with reference to the accompanying drawings. However, it will be readily understood by those skilled in the art that the present invention can be modified in various ways without departing from the spirit and scope of the present invention. Therefore, the present invention should not be construed as being limited to the following description of the embodiments.

[0021] In the drawings, the position, size, range, etc. of each component may not accurately represent the actual components, and therefore the position, size, range, etc. of each component are not necessarily limited to the position, size, range, etc. disclosed in the drawings.

[0022] In this specification, a language model is based on the Transformer architecture and is incrementally trained to become a conversational (also called dialog-based) model. In other words, a conversational generative model is a lower-level concept of a language model. A language model is also generally called a large-scale language model (LLM).

[0023] In this specification, the words "first" and "second" are used for the convenience of understanding the technical content or to identify each component. Therefore, the words "first" and "second" do not limit the number of each component. Furthermore, the words "first" and "second" do not limit the order of each component. Furthermore, the words "first" and "second" or identifying symbols used in this specification may not match the words or identifying symbols in the claims.

[0024] In this specification, a document refers to a written expression of human intent using characters or symbols. A document may be divided into multiple chapters or multiple paragraphs. A chapter may also consist of multiple paragraphs.

[0025] (Embodiment) In this embodiment, an example of an operation method of a document generation system according to one embodiment of the present invention will be described with reference to a flowchart. The operation method of a document generation system may be referred to as a sentence generation method.

[0026] <Example of how the document generation system works> An example of an operation method of a document generation system according to an aspect of the present invention includes Steps 1 to 11 as shown in Figures 1 and 2. Steps 1 to 5 are explained in Figure 1, and Steps 6 to 11 are explained in Figure 2.

[0027] <step1> A user sends a question to the document generation system, which then accepts the question. The question is preferably written in a natural language, and the user can write the question in one of the languages ​​selected from English, Japanese, and other languages. There are no restrictions on the language of the question, and the above languages ​​may be mixed.

[0028] <step2> The document generation system predicts the number of tokens using a language model, etc. For example, it is preferable for the document generation system to predict the number of tokens in an answer to a question using a language model, etc. The number of tokens in a question can be accurately obtained, so there is no need to predict it.

[0029] When an LLM is used as a language model, the LLM has appropriate values ​​for the number of input tokens and the number of output tokens depending on the learning process, algorithm, or hardware constraints. The number of input tokens and the number of output tokens are collectively referred to as the number of tokens that the LLM can process at one time. The maximum number of tokens that can be processed at one time is set in the LLM. If the number of tokens exceeds the appropriate value, the answer sentence will become significantly distorted, making it difficult to obtain an appropriate answer. Therefore, document generation systems typically predict the number of tokens in the answer sentence to the question sentence, so that the number of tokens does not exceed the appropriate value, that is, the number of tokens is set to an appropriate volume. This step of predicting the number of tokens allows the document generation system to obtain an appropriate answer sentence.

[0030] We will now explain a different form from the above. Apart from the maximum number of tokens set in the LLM, an appropriate number of tokens can be set in the document generation system. Such a restriction can result in an appropriate answer sentence. Furthermore, it is possible to reduce the amount of calculation in the document generation system and also reduce the processing time.

[0031] Next, the method by which the LLM generates the predicted number of tokens (sometimes referred to as the predicted number of tokens) is not limited in any way, but an example will be described.

[0032] <Example 1> LLM can use GPT-3, GPT-3.5, GPT-4, LaMDA (Language Model for Dialogue Applications), PaLM (Pathways Language Model), Llama2, etc. Because LLM understands the context of each token, it can at least separate the question into tokens (called tokenization). Tokenization can be done using word tokenization, character tokenization, or subword tokenization. In this way, LLM can obtain an accurate value for the number of tokens in the question. On the other hand, the number of tokens in the answer must be predicted. For example, a prompt such as "Please predict the number of tokens in the answer to the question" can be created by the document generation system or prepared as a template by the document generation system, and a prompt containing the question and the instruction can be input to LLM. This method does not generate an answer to the question, so LLM generates a predicted number of tokens.

[0033] BERT (Bidirectional Encoder Representations from Transformers) and T5 (Text-to-Text Transfer Transformer) can be used as language models. Using BERT to generate the predicted number of tokens is preferable because it requires less load and takes less time than using GPT-3 and GPT-4 to generate the predicted number of tokens.

[0034] <Example 2> As another example, the LLM may be made to generate an answer to a question and generate the correct number of tokens in the answer. For example, a document generation system may create an instruction such as "Please tell me the answer to the question. Then, calculate the number of tokens in the answer," or the document generation system may prepare it as a standard text, and a prompt including the question and the instruction may be input to the LLM. This method will result in obtaining the answer and its correct number of tokens. In this method, the LLM may be made to tokenize the answer. This method makes it possible to calculate the correct number of tokens in the answer.

[0035] <Example 3> Another example will now be described. As in Example 2, an answer to a question is generated by the LLM. As an example, the document generation system creates an instruction such as "Please tell me the answer to the question," or the document generation system prepares it as a template, and a prompt including the question and the instruction is input to the LLM. The document generation system receives the answer generated by the LLM and performs tokenization of the answer. This method also makes it possible to calculate the exact number of tokens in the answer.

[0036] <Additional features> In relation to Step 2, the document generation system may have the following additional function. As an additional function, it may be possible to determine whether it is necessary to proceed to the process (Step 3) related to RAG (Retrieval-Augmented Generation), which will be described later. Depending on the question, RAG may not be necessary to obtain the answer. If RAG is not necessary, Step 3 can be omitted. Therefore, determining whether RAG is necessary as an additional function related to Step 2 is an efficient procedure in this step.

[0037] <step3> The document generation system can use RAG. To explain RAG in more detail, the document generation system searches a database in a search engine based on a question sentence, and generates information (reference information) specific to the question sentence from the results. The document generation system can use similarity search as the search method. As will be explained later, when building a database, it is preferable to set an upper limit on the number of data tokens to be saved.

[0038] A search engine is a system that can output results in descending order of similarity (also called score) with the question. Similarity indicates the strength of the relationship with the question. To improve the accuracy or efficiency of similarity searches, it is preferable that each piece of information stored in a database is assigned a search index. A search index is an index held by the database within a search engine.

[0039] The reference information includes a list of information specific to the question sentence, and the list is preferably sorted in descending or ascending order of similarity.

[0040] When using RAG, the user can specify a database. Using a specified database is preferable because it makes it easier to obtain appropriate reference information. The document generation system can also build a new database of reference information.

[0041] The database can contain internal company information. Internal company information includes information that exists within a specific organization (typically a company), and specifically includes documents known as confidential documents or special-purpose documents. Examples of confidential documents or special-purpose documents include manuals. Examples of confidential documents or special-purpose documents include circuit design data. Documents such as manuals and circuit design data are important documents that can improve business efficiency or the level of an organization. For business purposes, users can obtain appropriate answers by specifying a database containing internal company information.

[0042] Operating a document generation system with an appropriate number of tokens is also useful for document generation systems that use LLMs published on the cloud. This allows the document generation system to process data efficiently and obtain appropriate answers. Furthermore, since LLMs can incur costs depending on the number of tokens, this can help reduce costs.

[0043] It is also preferable that the database contains publicly known information. For example, the database may contain official documents, patent publications, patent bulletins, books, or literature. In a document generation system that applies a published LLM, it is preferable to specify a database containing publicly known information.

[0044] If the reference information is appropriate in Step 3, RAG needs to be executed once, but if the reference information is inappropriate or if you want to revise the reference information, you can execute RAG two or more times. In other words, there is no limit to the number of times you can generate a document using RAG.

[0045] It is preferable to perform a keyword search or a phrase search as a similarity search. A keyword search is a search that collects characters entered in the search box of a search engine by breaking them down into characters before and after spaces. A phrase search is a search that collects characters that include spaces in the search box. Such keyword searches and phrase searches have a low load and can obtain results in a short time. Keyword searches and phrase searches are also preferable because they are highly readable. The document generation system can also create characters to be entered in the search box based on the question text.

[0046] As another form of similarity search, vector search is preferable. Vector search converts data, including text and images, into numerical vectors and calculates the similarity between the vectors. Vector search allows for faster processing than keyword search and can yield results with higher relevance. In other words, vector search is preferable because it can yield results close to what the user intended, even if common words cannot be identified.

[0047] As yet another form of similarity search, if an answer sentence is generated in Step 2, a similarity search may be performed using the answer sentence. As described above, the similarity search may be performed using a keyword search, a phrase search, or a vector search.

[0048] <step4> The document generation system creates a new prompt. Specifically, it is preferable that the document generation system creates the prompt based on the question and the information (reference information) specific to the question generated in Step 3. In Figure 1, the above prompt is shown as the first prompt. The first prompt may include, for example, an instruction to the LLM to generate an answer to the question using the reference information.

[0049] <step5> The document generation system calculates the sum of the number of tokens in the first prompt and the expected number of tokens obtained in Step 2. As will be explained later, the number of tokens in the first prompt may be reduced. Therefore, the expected number of tokens is not included in the first prompt in Step 5.

[0050] Step 6 and subsequent steps will be explained using FIG.

[0051] <step6> The document generation system determines the number of tokens. Specifically, the document generation system determines whether the sum calculated in Step 5 is less than or equal to a predetermined value. If an LLM is used as the language model, the predetermined value is the maximum number of tokens that the LLM can process at one time. If the result of the determination is YES, proceed to Step 11; if the result is NO, proceed to Step 7.

[0052] <step7> The document generation system uses a language model, typically an LLM, to summarize the similar search results obtained in Step 3. The document generation system prepares a prompt for summarization. The prompt includes, for example, an instruction to have the LLM summarize the similar search results. If there are multiple similar search results, it is preferable to summarize each of them. Step 7 allows the number of tokens corresponding to the similar search results to be reduced. In this way, the number of tokens in the first prompt can be reduced.

[0053] <step8> The document generation system determines the number of tokens. Specifically, the document generation system recalculates the sum of the number of tokens in the prompt containing the similar search results summarized in Step 7 and the expected number of tokens, and determines whether the calculated sum is equal to or less than a predetermined value. If the result of this determination is that the sum is equal to or less than the predetermined value (YES), proceed to Step 10; if the sum exceeds the predetermined value (NO), proceed to Step 9.

[0054] <step9> The document generation system deletes the results with the lowest similarity from the similar search results. In order to reduce the number of tokens, deleting some of the similar search results is more effective than summarizing the similar search results.

[0055] <step8> After obtaining the similar search results after deletion, the process returns to Step 8 and determines the number of tokens. Specifically, the document generation system recalculates the sum of the number of tokens in the prompt including the similar search results after deletion that have gone through Step 9 and the expected number of tokens, and determines whether the calculated sum is equal to or less than a predetermined value. After Step 9, deletion of similar search results is repeated until the determination of the number of tokens becomes YES, and the determination of Step 8 is performed each time.

[0056] <step10> If the answer to the question in Step 8 is YES, the document generation system creates a new prompt. In Figure 2, this prompt is shown as the second prompt. The second prompt can be said to have fewer tokens of reference information than the first prompt because similar search results have been summarized and some of the similar search results have been deleted.

[0057] <step11> The document generation system causes the LLM to generate a response sentence to the first prompt or the second prompt.

[0058] The user can then check the answer. It is preferable that the answer be written in the same language as the question, but the language of the answer is not limited. For example, if the answer is written in a language different from the question, the document generation system can translate the answer.

[0059] This type of operation is preferable because it allows for continuous response and further improves accuracy.

[0060] Next, an example of the configuration of a document generation system will be described.

[0061] <Document generation system configuration example 1> The document generation system of this embodiment includes at least a first information processing device 10 as shown in FIG. 3. Furthermore, as a system for receiving the services of the document generation system, in addition to the first information processing device 10, the system also includes a second information processing device 40, a third information processing device 50, and an information terminal 20. The first information processing device 10, the second information processing device 40, and the information terminal 20 can each be called a component. The first information processing device 10 is a component that enables the above-described operating method, the second information processing device 40 is a component that enables execution of a language model, and the information terminal 20 is a component used by a user. The third information processing device 50 is a component that includes a database that enables RAG. Furthermore, the above-described RAG may be executed in a database provided in the first information processing device 10 or in a database provided in the information terminal 20.

[0062] 3, the information terminal 20 is connected to a first information processing device 10 via a network 31. The first information processing device 10 is also connected to a third information processing device 50 via the network 31. The first information processing device 10 is also connected to a second information processing device 40 via the network 30.

[0063] <Document generation system configuration example 2> As shown in FIG. 4, the document generation system of this embodiment does not use a first information processing device 10, but has at least a second information processing device 40. In addition to the second information processing device 40, a system for receiving the services of the document generation system also has a third information processing device 50 and an information terminal 20. The second information processing device 40 is a component that enables the execution of the above-mentioned operating method and language model, and the information terminal 20 is a component used by the user. The third information processing device 50 is a component that includes a database that enables RAG. The above-mentioned operating method may be executed on the information terminal 20. The above-mentioned RAG may also be executed on the information terminal 20.

[0064] As shown in FIG. 4, the information terminal 20 is connected to a second information processing device 40 and a third information processing device 50 via a network 32.

[0065] <<Configuration Example of First Information Processing Device 10>> An example of the configuration of the first information processing device 10 when configuration example 1 is applied will be described with reference to FIG. 5. As shown in FIG. 5, the first information processing device 10 has an input unit 110, a storage unit 120, a processing unit 130, an output unit 140, a transmission path 150, and a display 160. In addition to the first information processing device 10, FIG. 5 also shows an information terminal 20, a second information processing device 40, and a third information processing device 50, with arrows indicating data transmission and reception. The input unit and output unit may be collectively referred to as a communication unit. The communication unit enables the first information processing device 10 to transmit and receive data to and from the outside. Note that if the first information processing device 10 does not need to display information to the user, the display 160 may not be included. The first information processing device 10 may have a microphone and a speaker instead of the display 160.

[0066] [Input section 110] The first information processing device 10 has an input unit 110. The input unit 110 can have a function of accepting data from outside. For example, the input unit 110 can accept a question from the information terminal 20. Furthermore, the input unit 110 can accept data from the second information processing device 40. The input unit 110 can also accept data from another terminal or information processing device.

[0067] The input unit 110 can supply the received data to one or more selected from the storage unit 120 , the processing unit 130 and the display 160 via a transmission path 150 .

[0068] [Storage section 120] The first information processing device 10 has a storage unit 120. The storage unit 120 is a memory area and can store programs and / or data. A representative example of the program is a program executed by the processing unit 130. The data includes data generated by the processing unit 130 (for example, calculation results, analysis results, and inference results). The data also includes data received by the input unit 110.

[0069] The storage unit 120 may also have a database. The storage unit 120 has a function of storing the database and can further manage it. Managing the database includes deleting unnecessary data as appropriate. The database can also be used as a database for the RAG. The third information processing device 50 can be omitted from a system for receiving the services of the document generation system.

[0070] The database may not be included in the storage unit 120, or may be a database that exists outside the first information processing device 10 in addition to the storage unit 120. A third information processing device 50 may be provided as a database that exists outside the first information processing device 10.

[0071] The storage unit 120 has at least one of a volatile memory and a nonvolatile memory. Examples of volatile memories include DRAM (Dynamic Random Access Memory) and SRAM (Static Random Access Memory). Examples of nonvolatile memories include ReRAM (Resistive Random Access Memory, also called resistance change memory), PRAM (Phase change Random Access Memory), FeRAM (Ferroelectric Random Access Memory), MRAM (Magnetoresistive Random Access Memory, also called magnetoresistive memory), and flash memory. The storage unit 120 can be configured with SiLSI (circuits using silicon transistors).

[0072] The storage unit 120 may also include at least one of NOSRAM (registered trademark) and DOSRAM (registered trademark). The storage unit 120 may also include a recording media drive. Examples of recording media drives include a hard disk drive (HDD) and a solid state drive (SSD).

[0073] NOSRAM is an abbreviation for "Nonvolatile Oxide Semiconductor Random Access Memory (RAM)." NOSRAM memory cells are either two-transistor (2T) or three-transistor (3T) gain cells, and are made of metal oxides (also known as OS transistors) with a channel region made of metal oxide. OS transistors have extremely low leakage current, the current that flows between the source and drain when they are off. NOSRAM can be used as a nonvolatile memory by retaining a charge corresponding to data within the memory cell using its extremely low leakage current. NOSRAM is particularly suitable for computational processing, which requires repeated large amounts of data read operations, because it can read stored data without destroying it (nondestructive read). NOSRAM memory cells can be stacked. This increases data capacity, allowing it to be used as a large-scale cache memory or main memory storage memory, thereby improving performance.

[0074] DOSRAM is an abbreviation for "Dynamic Oxide Semiconductor RAM" and refers to RAM with one transistor (1T) one capacitor (1C) type memory cells. DOSRAM is a DRAM formed using OS transistors, and is a memory that temporarily stores information sent from outside. DOSRAM is a memory that takes advantage of the small off-current of OS transistors.

[0075] In this specification and the like, a metal oxide refers to an oxide of a metal in a broad sense. Metal oxides are classified into oxide insulators, oxide conductors (including transparent oxide conductors), oxide semiconductors (also referred to as oxide semiconductors or simply as OSs), and the like. For example, when a metal oxide is used in a semiconductor layer of a transistor, the metal oxide may be referred to as an oxide semiconductor.

[0076] The metal oxide in the channel formation region preferably contains indium (In). An OS transistor using a metal oxide containing indium in the channel formation region has high carrier mobility (electron mobility). The metal oxide in the channel formation region is preferably an oxide semiconductor containing an element M, which will be described later, instead of or in addition to In. The element M is preferably at least one of aluminum (Al), gallium (Ga), and tin (Sn). Other elements that can be used for the element M include boron (B), silicon (Si), titanium (Ti), iron (Fe), nickel (Ni), germanium (Ge), yttrium (Y), zirconium (Zr), molybdenum (Mo), lanthanum (La), cerium (Ce), neodymium (Nd), hafnium (Hf), tantalum (Ta), and tungsten (W). A combination of the elements listed as the element M may be used in the metal oxide. The element M is an element having a high bond energy with oxygen, and the bond energy with oxygen is higher than the bond energy between oxygen and indium. Furthermore, the metal oxide contained in the channel formation region is preferably a metal oxide containing zinc (Zn) instead of or in addition to In. Metal oxides containing zinc may be more likely to crystallize.

[0077] The metal oxide contained in the channel formation region is not limited to the above-mentioned elements, typically, metal oxides containing indium. The metal oxide contained in the channel formation region may be, for example, metal oxides containing zinc but not indium, such as zinc tin oxide and gallium tin oxide, metal oxides containing gallium, or metal oxides containing tin.

[0078] [Processing section 130] The first information processing device 10 has a processing unit 130. The processing unit 130 has a function of performing processes such as calculation, analysis, and inference. Typically, data is supplied from one or both of the input unit 110 and the storage unit 120, and the processing unit 130 can perform processes such as calculation, analysis, and inference using the supplied data. Data can also be acquired from the storage unit 120, and the acquired data can be used to perform processes such as calculation, analysis, and inference.

[0079] The processing unit 130 can typically perform processes related to Steps 2 to 11. Typically, the processing unit 130 has a function of having a language model predict the number of tokens in an answer sentence to a question sentence. That is, the processing unit 130 has a function of creating an appropriate prompt for the prediction or selecting an appropriate prompt or instruction sentence from a plurality of pre-prepared prompts or instruction sentences. The processing unit 130 also has a function of performing a similarity search on a database based on the question sentence. The third information processing device 50 can be provided as the database. The processing unit 130 also has a function of creating a prompt from the question sentence and the results of the similarity search. A command sentence selected from a plurality of pre-prepared command sentences may be used as the prompt. The processing unit 130 also has a function of calculating the sum of the number of tokens in the prompt and the number of tokens in the predicted answer sentence. The processing unit 130 also has a function of determining whether the sum is equal to or less than a predetermined value. If it is determined that the sum is equal to or less than the predetermined value, the processing unit 130 also has a function of having a language model create an answer sentence to the prompt. A command sentence selected from a plurality of pre-prepared command sentences may be used as the prompt.

[0080] The processing unit 130 may further have a function of having a language model summarize the results of the similarity search if the first sum exceeds a predetermined value. That is, the processing unit 130 has a function of creating an appropriate prompt for the summarization or selecting an appropriate prompt or instruction from a plurality of pre-prepared prompts or instructions. The processing unit 130 may also have a function of deleting the least similar result from the summarized similarity search results. The processing unit 130 may also have a function of creating a second prompt from the question sentence and the similarity search results after the deletion. The second prompt may be an instruction selected from a plurality of pre-prepared instructions. The processing unit 130 may also have a function of calculating a second sum of the number of tokens in the second prompt and the number of tokens in the predicted answer sentence. The processing unit 130 may also have a function of determining whether the second sum is equal to or less than a predetermined value. The processing unit 130 may also have a function of generating an answer sentence to the second prompt from the language model if it is determined that the second sum is equal to or less than the predetermined value.

[0081] The processing unit 130 can supply data (e.g., calculation results, analysis results, inference results) to one or both of the storage unit 120 and the output unit 140. For example, the data can be supplied to the storage unit 120 via a transmission path 150. The data supplied to the storage unit 120 is stored as existing data, and the processing unit 130 can refer to the existing data. Furthermore, the processing unit 130 can determine the similarity between the existing data and the current data. If the similarity is high, the processing unit 130 can determine to process without using RAG. Furthermore, if the similarity is low, the processing unit 130 may determine to process without using RAG.

[0082] The processing unit 130 has at least an arithmetic circuit to realize the above functions. The arithmetic circuit may include, for example, a central processing unit (CPU). The CPU includes an arithmetic unit, a primary cache memory, a secondary cache memory, etc. The processing unit 130 may also include a graphics processing unit (GPU) in addition to or instead of the CPU. The GPU includes an arithmetic unit, a primary cache memory, a secondary cache memory, etc. The switches and the like of the CPU or GPU may include one or both of an OS transistor (a transistor using an oxide semiconductor layer as a channel) and a Si transistor (a transistor using a semiconductor layer containing silicon as a channel).

[0083] The processing unit 130 may have a register and a main memory in addition to a CPU. The register and main memory may also be said to be owned by the CPU. The main memory is capable of sending and receiving data to and from a secondary cache, etc. The main memory has at least one of a volatile memory such as RAM (Random Access Memory) and a non-volatile memory such as ROM (Read Only Memory). The main memory may also have at least one of NOSRAM and DOSRAM. The main memory can have one or both of OS transistors and Si transistors. The configuration of the register and main memory can be understood by replacing CPU in this paragraph with GPU.

[0084] Examples of RAM include DRAM and SRAM. The DRAM or SRAM can be used as a working space for the processing unit 130 by virtually allocating memory space thereto. The operating system, application programs, program modules, program data, lookup tables, and the like stored in the storage unit 120 are loaded into the RAM immediately before execution. The operating system, application programs, program modules, program data, and lookup tables loaded into the RAM can each be accessed by the processing unit 130.

[0085] ROM can store systems that do not require rewriting. Examples of systems that do not require rewriting include BIOS (Basic Input / Output System) and firmware. Examples of ROM include mask ROM, OTPROM (One-Time Programmable Read Only Memory), and EPROM (Erasable Programmable Read Only Memory). Examples of EPROM include UV-EPROM (Ultra-Violet Erasable Programmable Read Only Memory), which allows stored data to be erased by exposure to ultraviolet light, EEPROM (Electrically Erasable Programmable Read Only Memory), and flash memory.

[0086] The processing unit 130 may have a microprocessor such as a DSP (Digital Signal Processor) in addition to a CPU or GPU. Since a DSP is specialized for digital signal processing, it is preferable to install it to control peripheral circuits of the CPU or GPU. The microprocessor may be implemented by a PLD (Programmable Logic Device) that operates on hardware such as an FPGA (Field Programmable Gate Array) or an FPAA (Field Programmable Analog Array). The processing unit 130 may also have a quantum processor. The processing unit 130 can interpret instructions from various programs using a processor such as a quantum processor and can perform various data processing and program control. Programs that can be executed by the processor are stored in at least one of the memory area of ​​the processor and the storage unit 120.

[0087] Because the OS transistors described above have an extremely small off-state current, using them as switches for retaining charge (data) flowing into a capacitor element can ensure long-term data retention. By utilizing this characteristic in at least one of the register and cache memory of the processing unit 130, the processing unit can be operated only when necessary, and in other cases, the signal input or power supply to the processing unit 130 can be turned off by saving the information from the previous processing in the corresponding memory element. In other words, the OS transistors enable normally-off computing, enabling the document generation system to consume less power.

[0088] Furthermore, by using a CPU or the like capable of high-speed operation in the processing unit 130, some of the processing executed by the first information processing device 10 can be performed using AI. The first information processing device 10 may be provided with an artificial neural network (ANN, hereinafter also simply referred to as a neural network) to enable processing using AI. Since a neural network is realized by a circuit (hardware) or a program (software), the first information processing device 10 preferably has the above circuit or the above program in addition to a CPU capable of high-speed operation.

[0089] In this specification, a neural network refers to a general model that imitates the neural circuit network of a living organism, determines the connection strength between neurons through learning, and has problem-solving capabilities. A neural network has an input layer where information is input, an output layer where information is output, and an intermediate layer (hidden layer) between the input and output layers, and optimizes the weights for the input data to obtain the correct output result.

[0090] In this specification and the like, when discussing neural networks, determining neurons and their weight coefficients from existing information may be referred to as "learning."

[0091] In this specification and the like, constructing a neural network using weighting coefficients obtained by learning and deriving a new conclusion from it may be referred to as "inference."

[0092] Furthermore, if the first information processing device 10 has a display 160, it is preferable that the processing unit 130 also has a function of constructing display data. It is preferable that the processing unit 130 constructs the display data so that the layout is easy for the user to check on the display 160. Of course, it is also possible for the display 160 to have a layout according to user settings.

[0093] [Output section 140] The first information processing device 10 has an output unit 140. The output unit 140 has a function of outputting calculation results and the like to the outside. For example, the output unit 140 can output calculation results and the like in the processing unit 130 to the outside of the first information processing device 10. The outside can be one or more selected from the second information processing device 40 and the information terminal 20. For example, the user can check the answer sentence output from the output unit 140 to the information terminal 20.

[0094] [Transmission Line 150] The first information processing device 10 has a transmission path 150. The transmission path 150 has a function of transmitting data. Data can be transmitted and received between the input unit 110, the storage unit 120, the processing unit 130, the output unit 140, and the display 160 via the transmission path 150.

[0095] <<Configuration Example of Second Information Processing Device>> Next, an example of the configuration of the second information processing device 40 will be described.

[0096] The second information processing device 40 can process the received data and transmit the processing results. For example, the second information processing device 40 can perform processing such as calculations using the data received from the first information processing device 10. Furthermore, the second information processing device 40 can transmit the processing results to the first information processing device 10. This can reduce the calculation load on the first information processing device 10.

[0097] The second information processing device 40 can perform processing using a language model. For example, it can execute processing using a natural language processing model such as BERT or T5 (Text-to-Text Transfer Transformer). The second information processing device 40 can execute processing using a general-purpose language processing model that can perform various natural language processing tasks.

[0098] Furthermore, the second information processing device 40 can perform processing using a model that uses an LLM (such as a document generation model or a dialogue model). For example, GPT-3, GPT-3.5, GPT-4, LaMDA (Language Model for Dialogue Applications), PaLM (Pathways Language Model), Llama2, etc. can be used as the LLM. Of course, a model built in-house can be used as the LLM.

[0099] The second information processing device 40 can perform the processing that the language model should perform in Steps 2 to 11. Typically, the second information processing device 40 can predict the number of tokens and create a response sentence to a prompt.

[0100] In the document generation system, the provider of the document generation system does not necessarily need to own the second information processing device 40. For example, a service provider can use part of a service provided by another business or the like using the second information processing device 40.

[0101] <<Configuration Example of Third Information Processing Device>> Next, the database of the third information processing device 50 will be described. When constructing the database, it is preferable to set an upper limit on the number of data tokens to be saved. Specifically, if the upper limit is set to 400 tokens, when a document of 1000 tokens is saved in the database, the document is divided into 400 tokens or less. The divided documents are preferably divided into groups of the document (for example, items). It is also preferable to ensure that the divided documents do not break up into sentences. The document generation system can acquire the divided documents as a result of a similarity search.

[0102] <<Configuration Example of Information Terminal 20>> The information terminal 20 is operated by a user and includes at least the functions of inputting a question and outputting a response. For the function of inputting a question, the information terminal 20 has an input device such as a keyboard or a speech device including a microphone. For the function of outputting a response, the information terminal 20 has an output device such as a display or a speech device including a speaker. The information terminal 20 can also be called a client computer. It is preferable that the information terminal 20 has an environment in which dedicated application software or a web browser can run. This allows the user to enjoy services using the document generation system of one aspect of the present invention.

[0103] 3 and 4 show a desktop computer as an example, but a notebook computer, smartphone, or tablet computer may also be used as the information terminal 20. A tablet computer can be folded over and placed on top of the main body, with the housing having an input unit (typically a keyboard). Some tablet computers also allow the housing to be separated from the main body. Because notebook computers, smartphones, and tablet computers are easy to carry, they are suitable for use when a user is away from the home and uses the document generation system, which is one aspect of the present invention.

[0104] The information terminal 20 can provide the user with data output by the document generation system or the LLM. In other words, in the document generation system, it is preferable that the user operates the information terminal 20, but does not operate the first information processing device 10 or the second information processing device 40. This configuration can improve security.

[0105] For example, when a provider of a service using a document generation system and a user who receives the service belong to the same organization such as a company, it is preferable that data transmission and reception between the information terminal 20 and the first information processing device 10 be performed using, for example, a network 31 established within the organization. This allows data to be transmitted and received between the information terminal 20 and the second information processing device 40 more securely than when data is transmitted over the Internet. It is also possible to prevent internal company information from leaking to the outside. Alternatively, data transmission and reception between the information terminal 20 and the first information processing device 10 may be performed using a network 30 (for example, the Internet).

[0106] Network 30 The network 30 connects the first information processing device 10 and the second information processing device 40. This allows input data and processed data to be transmitted and received between the two devices. It also distributes the load associated with information processing.

[0107] In this embodiment, the network 30 is mainly described as a computer network that is larger in scale than the network 31. For example, a global network can be used for the network 30. Specifically, the Internet, which is the foundation of the World Wide Web (WWW), can be used.

[0108] Network 31 The network 31 connects the information terminal 20 and the first information processing device 10. This allows data to be transmitted and received between them. It also distributes the load related to information processing. Furthermore, a service provider can provide a service using a document generation system to a user via the network 31, for example.

[0109] For example, a local network can be used for the network 31. Also, an intranet or an extranet can be used for the network 31. Also, a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a global area network (GAN), etc. can be used for the network 31.

[0110] When performing wireless communication, communication standards such as the fourth generation mobile communication system (4G), fifth generation mobile communication system (5G), and sixth generation mobile communication system (6G), or specifications standardized by IEEE such as Wi-Fi (registered trademark) and Bluetooth (registered trademark), can be used as communication protocols or communication technologies.

[0111] Network 32 The network 32 connects the information terminal 20 and the second information processing device 40. This allows input data and processed data to be transmitted and received between them. It also makes it possible to distribute the load related to information processing. The network 32 can be the same as the network 30 or the network 31 described above.

[0112] According to one aspect of the present invention, it is possible to provide a novel method for operating a document generation system. Also, according to another aspect of the present invention, it is possible to provide a method for operating a document generation system using a language model with the number of tokens set in the language model or less. [Explanation of symbols]

[0113] 10 First information processing device 20 Information terminal 30 Network 31 Network 32 Network 40 Second information processing device 50 Third information processing device 110 Input section 120 Storage section 130 Processing section 140 Output section 150 Transmission Line 160 display

Claims

1. a first component; The first component has a function of accepting a question sentence; a function of predicting the number of tokens in an answer to the question using a language model; a function of performing a similarity search on a database based on the query; a function of creating a prompt from the question and the results of the similarity search; a function for calculating the sum of the number of tokens in the prompt and the number of tokens in the predicted answer sentence; determining whether the sum is equal to or less than a predetermined value; and a function of causing the language model to generate a response sentence to the prompt when it is determined that the sum is equal to or less than a predetermined value.

2. a first component; The first component has a function of accepting a question sentence; a function of predicting the number of tokens in an answer to the question using a language model; a function of performing a similarity search on a database based on the query; a function of generating a first prompt from the question sentence and the result of the similarity search; a function for calculating a first sum of the number of tokens in the first prompt and the number of tokens in the predicted answer sentence; determining whether the first sum is less than or equal to a predetermined value; a function of causing the language model to summarize the results of the similarity search if the first sum exceeds a predetermined value; a function of generating a second prompt from the question and a summary of the results of the similarity search; a function for calculating a second sum of the number of tokens in the second prompt and the number of tokens in the predicted answer sentence; determining whether the second sum is equal to or less than a predetermined value; If it is determined that the second sum is equal to or less than a predetermined value, the document generation system has a function of causing the language model to generate an answer sentence to the second prompt.

3. a first component; The first component has a function of accepting a question sentence; a function of predicting the number of tokens in an answer to the question using a language model; a function of performing a similarity search on a database based on the query; a function of generating a first prompt from the question sentence and the result of the similarity search; a function for calculating a first sum of the number of tokens in the first prompt and the number of tokens in the predicted answer sentence; determining whether the first sum is less than or equal to a predetermined value; a function of deleting the result with the lowest similarity from the results of the similarity search when the first sum exceeds a predetermined value; a function of generating a second prompt from the question sentence and the result of the similar search after deletion; a function for calculating a second sum of the number of tokens in the second prompt and the number of tokens in the predicted answer sentence; determining whether the second sum is equal to or less than a predetermined value; and when it is determined that the second sum is equal to or less than a predetermined value, causing the language model to generate a response sentence to the second prompt.

4. a first component; The first component has a function of accepting a question sentence; a function of predicting the number of tokens in an answer to the question using a language model; a function of performing a similarity search on a database based on the query; a function of generating a first prompt from the question sentence and the result of the similarity search; a function for calculating a first sum of the number of tokens in the first prompt and the number of tokens in the predicted answer sentence; determining whether the first sum is less than or equal to a predetermined value; a function of causing the language model to summarize the results of the similarity search if the first sum exceeds a predetermined value; a function of removing the least similar results from the summarized similarity search results; a function of generating a second prompt from the question sentence and the result of the similar search after deletion; a function for calculating a second sum of the number of tokens in the second prompt and the number of tokens in the predicted answer sentence; determining whether the second sum is equal to or less than a predetermined value; and when it is determined that the second sum is equal to or less than a predetermined value, causing the language model to generate a response sentence to the second prompt.

5. accepting a question; a step of predicting the number of tokens of an answer sentence to the question sentence using a language model; performing a similarity search on a database based on the query; creating a prompt from the question and the results of the similarity search; calculating the number of tokens in the prompt plus the number of tokens in the predicted answer sentence; determining whether the sum is less than or equal to a predetermined value; and if it is determined that the sum is equal to or less than a predetermined value, causing the language model to generate an answer sentence to the prompt.

6. accepting a question; a step of predicting the number of tokens of an answer sentence to the question sentence using a language model; performing a similarity search on a database based on the query; creating a first prompt from the question sentence and the results of the similarity search; calculating a first sum of the number of tokens in the first prompt and the number of tokens in the anticipated answer sentence; determining that the first sum is less than or equal to a predetermined value; if the first sum exceeds a predetermined value, summarizing the results of the similarity search using the language model; generating a second prompt from the question and a summary of the results of the similarity search; calculating a second sum of the number of tokens in the second prompt and the number of tokens in the anticipated answer sentence; determining whether the second sum is less than or equal to a predetermined value; and if it is determined that the second sum is equal to or less than a predetermined value, causing the language model to generate an answer sentence to the second prompt.

7. accepting a question; a step of predicting the number of tokens of an answer sentence to the question sentence using a language model; performing a similarity search on a database based on the query; creating a first prompt from the question sentence and the results of the similarity search; calculating a first sum of the number of tokens in the first prompt and the number of tokens in the anticipated answer sentence; determining that the first sum is less than or equal to a predetermined value; If the first sum exceeds a predetermined value, deleting the result with the lowest similarity from the results of the similarity search; creating a second prompt from the question sentence and the results of the similarity search after deletion; calculating a second sum of the number of tokens in the second prompt and the number of tokens in the anticipated answer sentence; determining whether the second sum is less than or equal to a predetermined value; and if it is determined that the second sum is equal to or less than a predetermined value, causing the language model to generate an answer sentence to the second prompt.

8. accepting a question; a step of predicting the number of tokens of an answer sentence to the question sentence using a language model; performing a similarity search on a database based on the query; creating a first prompt from the question sentence and the results of the similarity search; calculating a first sum of the number of tokens in the first prompt and the number of tokens in the anticipated answer sentence; determining that the first sum is less than or equal to a predetermined value; if the first sum exceeds a predetermined value, summarizing the results of the similarity search using the language model; removing the least similar results from the summarized similarity search results; creating a second prompt from the question sentence and the results of the similarity search after deletion; calculating a second sum of the number of tokens in the second prompt and the number of tokens in the anticipated answer sentence; determining whether the second sum is less than or equal to a predetermined value; and if it is determined that the second sum is equal to or less than a predetermined value, causing the language model to generate an answer sentence to the second prompt.

Images