Large Language Model System

Abstract

Construct multiple retrieval augmented generation (RAG) databases for different related fields as needed to increase the recording capacity of additional information. By referring to the entire text or image of the original document page where relevant content is described based on the chunks obtained from RAG retrieval, eliminate reference omissions. At the same time, reflect the background information and related information that may be described around the retrieved chunks in the context of the question text. Databaseize the records of the question text and the answer text to make them searchable, and provide a large language model using retrieval augmented generation that eliminates the need for costly and time-consuming regenerating answers. 【Solution means】The RAG database includes a page image acquisition means, a page text database recording means, a related feature vector extraction means, and a feature vector database recording means.

Description

Technical Field

[0001] The present invention relates to Large-scale language model system using search expansion generation.

Background Art

[0002] In recent years, the development of machine learning has been remarkable, and in particular, the spread of large language models (LLMs) has been progressing. Hundreds of billions to trillions of neural network parameters are learned using large amounts of data in the terabytes, and have been handling tasks such as translation, speech and image recognition, and text summarization. In addition, generative AI that generates images, music, and documents based on instructions (prompts) is also being put into practical use. As it has become known that performance improves by scaling the model (scaling laws), the scaling of the model has been rapidly progressing.

[0003] As the application range of large language models expands, knowledge data in various fields is required, and with the progress of social situations and technologies in each field, the incorporation of up-to-date knowledge data is constantly demanded. However, learning a large amount of data requires large-scale computing resources, enormous power, and high costs. Therefore, the reconstruction of large language models cannot be performed frequently. When asking a large language model a question, there is a known phenomenon called hallucination where, even when the knowledge necessary to answer the question is not internally recorded, an answer that is not based on facts is generated. This is the reason why the spread is restricted in fields such as medicine where mistakes can lead directly to accidents.

[0004] For the on-site utilization of large language models based on the latest information, additional acquisition of the latest information is required. Currently, two types of approaches are being carried out. One is what is called additional learning (fine-tuning) or transfer learning, where learning is performed on a part of the output layer (fine-tuning) or only the final layer (transfer learning) of an existing large language model using additional information to create a specialized large language model. Since it becomes a domain-specific large language model, its usefulness is high. However, although it is not as costly as training a large language model from scratch, costs and technical skills are still required for training.

[0005] The other approach is the Retrieval-Augmented Generation (RAG) approach used in the present invention. The model of the large language model itself is not changed. Additional information is separately stored in a database (RAG database). Information necessary for solving the question text is retrieved from the RAG database, and the obtained information is added to the question text to obtain an answer from the large language model. Even if the amount of data with additional potential is large, the additional information required for answering a certain question text is limited. Therefore, only the limited additional information is retrieved from the RAG database, and the extracted additional information (context) is added to the question text in the question input field of the large language model. In this approach, no changes are made to the large language model itself through learning. Only context information is added to the question input field, so the cost and technical hurdles are low. Furthermore, it is highly responsive, such as being able to immediately reflect highly urgent information such as emergency side effect information of drugs.

[0006] Here, the RAG database cuts the document information to be added into fine fragments (chunks), vectorizes each chunk (chunk vector), and forms a database. When asking a question, the question text itself is also vectorized, and a chunk vector with a high similarity to the vector of the question text is retrieved from the RAG database, and the content of the obtained group of chunks is added to the question text. As a result, it becomes possible to handle specific fields and the latest information without performing additional learning of a costly large language model, and by limiting the basis for the answer to the question text to the RAG database, hallucinations can be prevented. The following are the prior art documents related to this application.

Prior Art Documents

Non-Patent Documents

[0007]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0008] When actually constructing and searching a RAG database, there are still many unsolved problems, such as: (1) the RAG database itself becomes huge and relatively more data unrelated to the question text increases; (2) since the content to be searched is not necessarily appropriately included in the chunks, a search omission of data related to the question text occurs; (3) background information and related information that may be described around the searched chunks are left out; (4) image recognition and search by current large-scale language models are still insufficient. Also, repeating RAG searches for the same question is a waste of resources such as cost and time.

[0009] The present invention has been made to solve such conventional problems, and its object is to construct a plurality of RAG databases for each related field as needed to increase the recording capacity of additional information, and to improve efficiency by narrowing down the target of RAG search. By referring to the text or image of the entire original document page of the part where the related content is described based on the chunk group obtained by RAG search, not only is the reference omission eliminated, but also the background information and related information that may be described around the searched chunks are reflected in the context of the question text. Presenting image data that is difficult to textify to the user and enabling additional description in the question text, databaseizing the records of the question text and the answer text so that they can be searched, and eliminating the need for costly and time-consuming re-answer generation, etc., by using Large-scale language model system search expansion generation.

Means for Solving the Problem

[0010] As a means for achieving the above object, in the Large-scale language model system described in claim 1, (1) page image image acquisition means for acquiring the image of each page of the information source to be referred to separately from the large-scale language model during inference, together with the reference address to the image; (2) page text database recording means for extracting the text of the character string from the page image image and recording it together with the reference address; (3) The feature vector database recording means that divides the extracted text into small segments (chunks), calculates feature vectors, and records them together with the reference address. (4) The related feature vector extraction means that extracts a group of feature vectors highly related to the question sentence together with the reference address for the feature vector of the question sentence given to the large language model. (5) The duplicate reference address removal means that removes duplicate reference address groups from the extracted related feature vectors. (6) After extracting the page text specified by the reference address from the page text database, the related text transcription means that transcribes it into the question input field together with the question sentence, and an answer sentence is obtained by the operations from (1) to (6) above.

[0011] According to claim 2 Large-scale language model system In the case of Large-scale language model system It is characterized in that it comprises page image database recording means that records the obtained individual page image images together with the reference address to the page image images.

[0012] According to claim 3 Large-scale language model system In the case of Large-scale language model system According to claim 1 or 2

[0013] According to claim 4 Large-scale language model system In the case of As recited in claim 2 of Large-scale language model system In the large language model, after extracting the page image image specified by the reference address from Page image database recording means comments are input for charts with insufficient text conversion, and it is characterized in that it comprises comment transcription means that transcribes them into the question input field together with the question sentence.

[0014] According to claim 5 Large-scale language model system In the case of As recited in claim 2 of Large-scale language model systemIn this case, the page image image acquisition means, the page text database recording means for the page image image database recording means, the above for the page text database recording means Feature vector database recording means is provided with a plurality of the Feature vector database recording means , and is characterized by including a plurality of RAG database search means for searching and extracting related feature vectors for any of the feature vector recording means.

[0015] According to claim 6 Large-scale language model system In the case of claim 1 Large-scale language model system includes a question-and-answer recording means for recording the question text and the obtained answer text, and for a new question text, first searches in the question-and-answer recording means to see if there is a question text similar to the new question text. If there is a similar question text, it is characterized by including an F&Q database that uses the answer record for the similar question text as the answer to the question.

Advantages of the Invention

[0016] According to claim 1 Large-scale language model system Since it includes an image acquisition means, it acquires the image of each page of the information source to be referred to separately from the large language model during inference, together with the reference address to the image. Since it includes a page text database recording means, it extracts the text of the string from the page image and records it together with the reference address. Since it includes a feature vector database recording means, it divides the extracted text into small segments (chunks) to calculate feature vectors and records them together with the reference address. Since it includes a related feature vector extraction means, for the feature vector of the question text given to the large language model, it extracts a group of feature vectors highly related to the question text together with the reference address. Since it includes a duplicate reference address removal means, it removes the group of duplicate reference addresses among the extracted related feature vectors. Since it is equipped with related text transcription means, after extracting the page text specified by the reference address from the page text database, it is transcribed into the question input field together with the question sentence.

[0017] According to claim 2 Large-scale language model system In this case, since it is equipped with page image database recording means, the acquired individual page images are recorded together with the reference address to the page image.

[0018] According to claim 3 Large-scale language model system In this case, since it is equipped with page image viewing means, the page image specified by the reference address is provided for display and viewing.

[0019] According to claim 4 Large-scale language model system In this case, since it is equipped with comment transcription means, in the large language model, after extracting the page image specified by the reference address Page image database recording means comments are input for charts with insufficient text conversion, and are transcribed into the question input field together with the question sentence.

[0020] According to claim 5 Large-scale language model system In this case, it is equipped with a plurality of page image acquisition means, page text database recording means for the page image database recording means, and feature vector recording means for the page text database recording means, and a plurality of RAG database search means for searching and extracting related feature vectors for any of the feature vector recording means.

[0021] According to claim 6 Large-scale language model system In this case, it is equipped with question-answer recording means for recording the question sentence and the obtained answer sentence. For a new question sentence, first, it searches in the question-answer recording means to see if there is a question sentence similar to the new question sentence. If there is a similar question sentence, it has an F&Q database that uses the answer record for the similar question sentence as the answer to the question.

Brief Description of the Drawings

[0022]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Mode for Carrying Out the Invention

[0023] Figure 1 is a typical system configuration of the present invention. Large-scale language model system It is composed of huge data, a large number of CPUs (Central Processing Units), GPUs (Graphics Processing Units), and a high-speed network connecting them. Therefore, it is constructed in a huge server such as a cloud data center and provided through the WEB. In companies, hospitals, etc., a large number of PC terminals are connected via a LAN (Local Area Network) connected to the WEB. There are also servers in companies and hospitals that operate in-house databases and electronic medical records. In recent years, the use of mobile terminals such as smartphones and tablets to access cloud services such as large language models, in-house databases, and electronic medical records from outside companies and hospitals has also increased.

[0024] A server, a terminal, a mobile terminal, all of which are composed of a memory for recording programs and data, a recording medium such as a hard disk for permanently recording the programs and data as required, a CPU for reading and processing the programs and data, a GPU for performing parallel processing at high speed as appropriate, a communication module, etc. As cloud services have become more stable and less expensive, the number of cases of migrating in-house databases and some or all of the electronic medical records to the cloud has been increasing. Conversely, there has also been a movement to move some or all of the large language models to the terminal side with increased processing capabilities and storage capacity (edge computing). In addition, the development of small-scale language models with reduced numbers of parameters of the language model, etc., is also progressing. Even if it is called small-scale, it is sufficiently large compared to before the emergence of large language models, and any embodiment including this form is included in the present invention.

[0025] Figure 2 is an example of a user interface in a large language model (LLM). LLMs are currently being developed rapidly, and a large number of models have been developed, including ChatGPT (a registered trademark of OpenAI), Bard, LaMDA (a registered trademark of Google), LLaMA (a registered trademark of Meta), etc. Naturally, the user interfaces are different, but typically, as shown in Figure 2, there is a frame for inputting a prompt for instructing and querying the LLM (prompt input frame), a frame for displaying the answer to the prompt (answer display frame), and at the same time, a frame for displaying the history of the prompt and the answer as a usage log (usage history frame).

[0026] Recently, in addition to the use of the above-mentioned LLM alone, the LLM itself is equipped with an API (Application Programming Interface), and the cases of using the functions of the LLM from external software have been increasing. In this case, since the prompt, answer, and history are input and output between the external software via the API, the display format is not restricted to Figure 2 and is controlled by the external software.

[0027] In large language models, to represent a certain vocabulary, a one-hot vector, which is a long vector consisting of zeros with the same number of dimensions as the types of vocabulary used and has only one 1 at the position corresponding to the vocabulary, is used. All the vocabularies of a large number of documents are replaced with vectors in this format, and the association (Attention) between each vocabulary vector is obtained using deep learning. For the query sentence (prompt), the vocabulary with a high probability of coming next in the query sentence and the already generated intermediate answer sentence is generated and added one by one to create the answer sentence. If the information for generating the answer is included in the large number of documents, it is expected that a reasonable, highly useful, and correct sentence will be generated. However, when the information for generating the answer sentence is not included in the large number of documents, since the vocabulary with a high probability is mechanically adopted to proceed with the answer sentence generation, it is known that a baseless and false answer sentence (Hallucination) will be generated. If this hallucination occurs in fields such as medicine, it may endanger the lives of patients, which is one of the reasons why the application of large language models to core business operations has not advanced.

[0028] For the learning of large language models, a large number of documents are read, and in order to obtain the relationship between vocabularies through deep learning, a large server equipped with a large number of parallel computing devices (GPUs), a large amount of electrical resources, and costs are required. Although new documents are generated every day, it is not realistic to reflect all of them in the large language model without delay. In addition, the large number of documents to be read mainly consists of publicly available documents published on the WEB and the like, and since they do not include confidential information such as corporate internal documents and hospital electronic medical records, it is said that the collectible documents are only a very small part of all the documents existing on the ground.

[0029] To utilize large language models based on the latest information in the field, additional up-to-date information acquisition is required, and currently two types of approaches are being taken. One is what is called additional learning (fine-tuning) or transfer learning, which uses additional information to perform learning on a part of the output layer (fine-tuning) or only the final layer (transfer learning) of an existing large language model to create a specialized large language model. Although it becomes a field-specific large language model and thus has high usefulness, although it is not as much as training a large language model from scratch, the costs and technical skills required for training are still quite necessary. In addition, since sensitive information such as personal information and descriptions of medical conditions included in the additional information is used for training, there is a risk of being referenced outside the organization. To prevent this, it is necessary to exclusively build and operate a large language model that has undergone additional training within one's own company or hospital.

[0030] The other approach is the Retrieval-Augmented Generation (RAG) approach used in the present invention. The model itself of the large language model is not changed. Additional information is separately stored in a database (RAG database), the information necessary for solving the question sentence is retrieved from the RAG database, and the obtained chunk of text information is added to the question sentence to obtain an answer sentence from the large language model. Even if the amount of data with additional potential is large, the additional information necessary for answering a certain question sentence is limited. Therefore, only the limited additional information is retrieved from the RAG database, and the extracted additional information is appended to the question sentence (context) in the question input field of the large language model. In this approach, no changes are made to the model itself of the large language model, and only context information is added to the question input field, so the costs and technical hurdles are low. Furthermore, it is excellent in terms of immediacy, such as being able to immediately reflect highly urgent information such as emergency side effect information of drugs.

[0031] The processing flow is shown in Figure 3. A broker program such as a chat app receives a question text sent from a user (1), and searches the RAG database for information related to the question text (2). The broker program asks the LLM about the obtained search results (3) together with the question text (4), and receives an answer from the LLM (5). The received answer is presented to the user (6). However, although the length of the text string of the chunk is variable in settings, there are certain limitations, and there may be cases where the necessary information cannot be fully included. Conversely, if the chunk is too long, there is a risk that the focus of the feature vector will become blurred and it will be impossible to appropriately respond to the search. Also, if keywords are added at the joints between chunks, appropriate feature vectorization cannot be performed and the search becomes difficult. In addition, scanned images such as images and illustrations may not yet have their contents fully recognized and texturized, and there is a possibility that they will not be used for search. Furthermore, when attempting to build a RAG database at a level that can withstand practical use, it will become large-scale based on a large number of document chunks. The required document groups vary greatly depending on the area of interest. For example, the required document groups in the medical field are very different from those in the history and literature fields. Constructing a RAG database from document groups covering all fields would result in a redundant configuration and risk imposing an unnecessary load on database construction and search.

[0032] Figure 4 shows the configuration of the RAG database. Scan images of books, image images of PDF-converted web documents, etc., which are information sources (information resources) that may be added, are recorded in the database by assigning reference addresses such as "information resource name + page" to each page image image (page image image). (Page image image database). Subsequently, text is extracted from each page image image (page text), and the page text is recorded together with the reference address (page text database). The text of each page is cut into small fragments (chunks), a feature vector is calculated for each chunk, and the reference address is assigned and recorded in the feature vector database. In addition, if there are strict restrictions on the recording capacity, etc., the page text may be compressed and recorded using the summarization function of a large language model as needed.

[0033] Here, as shown in FIG. 5, the reference address is numbered in units of book title + page, which is familiar in books etc. However, for documents on the WEB etc., unlike books with physical restrictions, there is no need to be particular about the display. Depending on the granularity of the information, it may be numbered in units of paragraphs, or conversely, in larger units of sections or chapters. The page text may be unstructured plain text, but in the case of complex content, a tagged notation such as XML, JSON, or Markdown is desirable to clarify the structure of the document. Recommendations vary depending on the software for calculating the feature vector. In the present invention, the JSON notation is used, but any notation may be used.

[0034] Regarding images, videos, illustrations, etc. included in the page image, at present, it is not possible to represent the text in a perfect form. Therefore, ultimately, judgment and understanding by human eyes are required. It is also useful to have the user or the like judge the above-mentioned images, videos, illustrations, etc. and add explanatory text to the corresponding page text as needed. In this way, the page image database after extracting the page text is not essential, but it may be useful, and the page image may be referred to as needed.

[0035] FIG. 6 shows the search procedure of the RAG database. When a question sentence is input into the prompt input box, the broker program calculates the feature vector of the question sentence and extracts a list of approximate feature vectors from the vector database (RAG database). The degree of approximation between feature vectors is often represented by the magnitude of the inner product between the vectors (cosine similarity), but the Manhattan distance obtained by summing the absolute values of the differences of each element of the two vectors may also be used. The magnitude of the threshold for the degree of approximation used as the extraction criterion is variable depending on the situation. If the extracted list is too large, the criterion can be raised to narrow it down, and if the list is too small, the criterion can be lowered to increase the size of the list. In some cases, it may be set to the top 10, etc., and the criterion may be automatically adjusted so that the number of extracted elements becomes the set value.

[0036] When a list of feature vectors corresponding to a plurality of chunks is extracted, in the conventional RAG, the text of the chunk from which each feature vector originated was added to the question sentence. However, as described above, there is a limit to the length of the text string of the chunk, and there are cases where it cannot contain all the necessary information. Also, when keywords were added at the seams between chunks, appropriate feature vectorization could not be performed. In addition, scanned images such as images and illustrations may not be sufficiently texturized. To compensate for this drawback, the present invention takes the following steps.

[0037] Organize the reference addresses assigned to each feature vector, remove duplicate reference addresses, and transcribe the entire page text indicated by the remaining reference addresses together with the question sentence into the prompt input box, and prompt the large language model to answer based on this. By this procedure, the problem of the seam that the chunk could not contain all the necessary information, which was a problem of the above-mentioned fragmented chunks, is solved, and at the same time, the text between chunks can also be added to the question sentence. This function is useful for promoting deep understanding by showing not only the solution to a specific problem but also the information that is the background of the problem during education for students and the like.

[0038] When it comes to images, videos, illustrations, etc. where text conversion is insufficient, search for the image of the page from the page image database, display it to the user, and the user adds a description about the image, video, illustration, etc. to the question text. If necessary, it may also be added to the page text. In this way, it is also possible to utilize information such as images, videos, and illustrations that have not been fully utilized in the past.

[0039] When attempting to build a RAG database that can withstand practical use, it will be large-scale based on a large number of document chunks. For example, in the medical field, even in just large fields such as internal medicine, surgery, and obstetrics and gynecology, there are easily over 50, and furthermore, there is a tendency towards further subdivision as medicine develops. If all are to be aggregated into one RAG database, it will cause a huge load on data storage and retrieval. Here, the document groups required vary greatly depending on the area of interest. For example, even within the medical field, the document groups of abdominal surgery and psychiatry, for instance, have many non-overlapping parts. Furthermore, the document groups required in medicine are very different from those in the fields of history and literature. Therefore, as shown in Figure 7, by constructing multiple RAG databases for each area of interest and having a broker program search for 1 to multiple RAG databases related to the question text as needed, it is possible to avoid the inefficiency of retrieval due to the enlargement of the RAG database. Of course, if the RAG database is overly subdivided, it becomes necessary to execute a large number of searches and the efficiency of retrieval deteriorates. Needless to say, it is necessary to aggregate documents in highly relevant fields to build a RAG database.

[0040] In the educational field, when generating an explanatory text for the answer to a test question, for students who have answered a certain question incorrectly and whose grades are not considered good, an explanatory text based on a general RAG database is appropriate. However, in that case, there is a risk that students with good grades will not have their intellectual curiosity satisfied. In this case, it is also useful to construct a RAG database containing more advanced and fundamental content in advance, and generate and provide explanatory texts of advanced and in-depth content from it.

[0041] The question text and the corresponding answer text may be created each time, but it requires a certain amount of computing resources and costs. As shown in FIG. 8, a pair of a question text and an answer text for the question is used as a FAQ database and converted into a RAG database. When a question text is input, first, an answer text highly relevant to the question is searched from the FAQ database. Only when the content of the searched answer text is unsatisfactory, an answer text is newly created according to the procedure of the present invention, and the result is also registered in the FAQ database. When registering in this FAQ database, either a method of registering only the feature vector of the question text in the RAG database and linking it to the answer text, or a method of chunking the pair of the question text and the answer text and registering the feature vector, may be used.

[0042] Note that since there is a risk of information leakage if the content input to the prompt is used for the learning of the large language model, it is useful to explicitly declare a learning prohibition in the prompt, or to use a paid version of the large language model that is guaranteed not to be used for learning.

[0043] As described above, the embodiments have been explained. However, the specific configuration of the present invention is not limited to the above embodiments, and design changes and the like within the scope not departing from the gist of the invention are also included in the present invention. For example, in the present invention, the feature vector database of chunks has been mainly described. However, as long as it is information useful for generation, information extracted from a normal relational database by SQL statements or reference information from the WEB, etc., may be appropriately added to the prompt in addition to the feature vector database, and it is also included in the present invention.

Claims

1. (1) a page image acquisition means for acquiring an image of each page of an information source to be referenced separately from the large-scale language model during inference, together with a reference address of the image; (2) a page text database recording means for extracting text of a character string from the page image and recording the text together with the reference address; (3) comprising a feature vector database recording means for dividing the extracted text into small sections (chunks), calculating feature vectors, and recording the feature vectors together with the reference addresses; (4) comprising a related feature vector extraction means for extracting a group of feature vectors that are highly related to a question sentence given to the large-scale language model together with the reference address, (5) comprising a duplicate reference address removing means for removing duplicate reference addresses from the extracted related feature vectors; (6) a related text transcription means for extracting a page text designated by the reference address from the page text database and then transcribing the page text together with the question text into a question input field; A large-scale language model system that obtains answer sentences by the above operations (1) to (6).

2. 2. The large-scale language model system according to claim 1, further comprising a page image database recording means for recording each of the acquired page images together with a reference address to the page image.

3. 3. A large scale language model system according to claim 1, further comprising a page image viewing means for displaying and viewing the page image designated by said reference address.

4. 3. The large scale language model system according to claim 2, further comprising a comment transcription means for inputting a comment for a diagram or table which has not been sufficiently converted into text after extracting the page image designated by the reference address from the page image database recording means, and transcribing the comment together with the question in a question input field.

5. 3. The large scale language model system according to claim 2, further comprising: a plurality of page image acquisition means, a plurality of page text database recording means for said page image database recording means, and a plurality of feature vector database recording means for said page text database recording means, and further comprising a plurality of RAG database search means for searching for and extracting related feature vectors for any of said feature vector recording means.

6. 2. The large-scale language model system according to claim 1, further comprising a question and answer recording means for recording the question sentence and the obtained answer sentence, and for a new question sentence, a F&Q database is further provided in which the question and answer recording means is first searched for a question sentence similar to the new question sentence, and if a similar question sentence is found, an answer record to the similar question sentence is used as the answer to the question.

Images