Method and system for enhancing large language model generation by using network search

Abstract

The invention discloses a method and system for enhancing generation of a large language model by using network search, and the method provides related data for the large language model by using network search related content, provides more information for the output of the large language model, and enhances the accuracy and timeliness of the answer of the large model. Meanwhile, by using the method, the huge cost of large model pre-training can be saved. Meanwhile, by using the method, the cost can be reduced, and the huge cost of large model pre-training is saved. According to the method, an information source can be found for output of a large model, and the situation of'illusion 'output of the large oracle model is effectively reduced. And finally, the query response time can be remarkably accelerated through a designed cache data mechanism.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and system for enhancing generation of a large language model using network search. Background Art

[0002] Large language models (LLMs, which are generative models in nature), such as ChatGPT (Chat Generative Pre-trained Transformer, a chatbot program developed by OpenAI), can generate fluent human-like responses for many downstream tasks (such as task-oriented dialogue and question answering). However, it is still challenging to apply LLMs to task-solving applications in the real world, mainly because they may produce replies that are inconsistent with reality and cannot use external knowledge as a reference for reply generation. For example, for questions such as "What is the weather in city A now", "What is the exchange rate of currency B to currency C", and "How long does it take to get from location C to location D", real-time information is required when answering, and this real-time information (such as real-time time, real-time exchange rate, real-time road conditions, and real-time traffic information, etc.) cannot be generated simply by relying on the knowledge contained in the LLM parameters, and all need to rely on external knowledge to generate accurate results.

[0003] There is also the problem of hallucinations in large language models. For example, in the academic field, higher accuracy of answers is required, and it is hoped that relevant citation information can be found for the answers. However, the current large language models may even fabricate some citation information and cannot be applied to question and answer in the academic field. Summary of the invention

[0004] In view of the deficiencies of the prior art, the present invention proposes a method and system for enhancing the generation of a large language model using network search.

[0005] The object of the present invention is achieved by the following technical solution: A method for enhancing the generation of a large language model using network search, comprising:

[0006] The input question is judged by the judgment model to determine whether it needs to be searched;

[0007] If no retrieval is required, the large language model directly generates answers based on the input question;

[0008] If a search is required, determine whether the input question exists in the cache database;

[0009] If present in the cache database, searching for cached questions and / or information fragments related to the input question by calculating similarity distance in the vector space;

[0010] If it does not exist in the cache database, a network search is performed;

[0011] Reorder the information fragments obtained by searching the cache database or the information fragments obtained by searching the network according to their relevance to the input question, and select the first n information fragments;

[0012] The input question and the selected information fragments are fed into the large language model to generate the answer.

[0013] Furthermore, the network search includes:

[0014] Split the input problem into multiple sub-problems for web search; and perform web search on the input problem and the sub-problems.

[0015] Furthermore, the input question is split into multiple sub-questions for network search through a sub-question splitting model; the sub-question splitting model is a large language model.

[0016] Furthermore, it also includes: vectorizing the input question and the information fragments obtained by the network search to obtain vector data, and adding the input question and the information fragments obtained by the network search and their corresponding vector data to the cache database.

[0017] Furthermore, through the re-ranking model, the information fragments obtained by searching the cache database or the information fragments obtained by searching the network are re-ranked according to their relevance to the input question.

[0018] Furthermore, the re-ranking model is trained as follows: academic data is evenly selected by subject, questions are generated based on the academic data to form question-answer pairs, i.e., positive samples; the most relevant text set outside the positive samples is retrieved based on the generated questions as negative samples; the positive and negative samples are sent together to the re-ranking model for training, wherein the logits score of the positive samples is greater than that of the negative samples in a batch.

[0019] Furthermore, the step of inputting the input question and the selected information fragment into the large language model to generate an answer further includes:

[0020] The input question, selected information fragments and instruction prompts are assembled into prompt words and sent to the large language model to ensure that the large language model answers based on the currently provided information and combined with inherent knowledge.

[0021] The present invention also provides a system for enhancing generation of a large language model using network search, comprising:

[0022] The intention understanding module is used to determine whether the input question needs to be searched through the evaluation model; if it needs to be searched, it determines whether the input question exists in the cache database;

[0023] a cache query module for searching cache questions and / or information fragments related to the input question by calculating similarity distances in a vector space if present in the cache database;

[0024] A network search module, used to perform a network search if the search does not exist in the cache database;

[0025] A reordering module is used to reorder the information fragments obtained by searching the cache database or the information fragments obtained by searching the network according to their relevance to the input question, and select the information fragments ranked in the top n;

[0026] The large language model generation module is used to generate answers based on the input question directly if no retrieval is required; if retrieval is required, the input question and the selected information fragments are input into the large language model to generate answers.

[0027] The present invention also provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the above-mentioned method of using network search to enhance the generation of a large language model is implemented.

[0028] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the method for enhancing the generation of a large language model using network search is implemented.

[0029] The beneficial effects of the present invention are as follows:

[0030] The traditional large language model provides relevant information to the large language model by searching for relevant content on the Internet, providing more information for the large language model output and enhancing the accuracy and timeliness of the large model's answers. At the same time, this method can reduce costs and save the huge cost of pre-training large models. This method can find the source of information for the output of the large model and effectively reduce the situation of "hallucination" in the output of the large prediction model. Finally, the designed cache data mechanism can significantly speed up the query response time. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0032] Figure 1 The present invention is a flowchart of a method for enhancing generation of a large language model using network search according to an embodiment of the present invention.

[0033] Figure 2This is a deployment architecture diagram of a network search enhanced large language model system of the present invention.

[0034] Figure 3 It is a schematic diagram of the computer device structure of an embodiment of the present application.

[0035] Figure 4 A system module diagram for enhancing large language model generation using network search according to an embodiment of the present invention.

[0036] Figure 5 A structure of a cache database stored in a vector database is provided in an embodiment of the present invention.

[0037] Figure 6 A reordering model training scheme is provided in an embodiment of the present invention.

[0038] Figure 7 An example of a network enhanced answer provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0040] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0041] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0042] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0043] A method of enhancing a large language model using network search of the present invention comprises:

[0044] The input question is judged by the judgment model to determine whether it needs to be searched;

[0045] If no retrieval is required, the large language model directly generates answers based on the input question;

[0046] If a search is required, determine whether the input question exists in the cache database;

[0047] If present in the cache database, searching for cached questions and / or information fragments related to the input question by calculating similarity distance in the vector space;

[0048] If it does not exist in the cache database, a network search is performed;

[0049] Reorder the information fragments obtained by searching the cache database or the information fragments obtained by searching the network according to their relevance to the input question, and select the first n information fragments;

[0050] The input question and the selected information fragments are fed into the large language model to generate the answer.

[0051] The present invention also provides a system for enhancing the generation of a large language model using network search, including an intention understanding module, a cache query module, a network search module, a re-ranking module and a large model generation answer module. Figure 4 , the modules are as follows:

[0052] 1. The intent understanding module belongs to the model service, which is used to determine whether the input question needs to be retrieved through the evaluation model, that is, whether external knowledge needs to be retrieved to enhance the answer; if retrieval is required, it determines whether the input question exists in the cache database; the evaluation model is a large language model.

[0053] 1.1 Evaluation Model

[0054] Data Collection:

[0055] Use a large language model to generate a corpus of reflective symbols, and then refine its knowledge into an internal judgment model C

[0056] Construct a certain amount of training data set D_critic (usually > 100,000), whose format is: input x, output y, y is composed of reflection symbols: {Yes Retrieval|No Retrieval,Continue}

[0057] Generate data example:

[0058] {"instruction":"When provided with instruction,please evaluate whether seeking additional information from external sources such as the web (eg, Google) aids in producing a more comprehensive response.Respond witheither[Yes Retrieval]or[No Retrieval]or[Continue].",

[0059] "input":"Task instruction:Find the longest word in the sentence\u201cGreat minds think alike.\u201d\n",

[0060] "output":"[No Retrieval]",

[0061] "task":"retrieval"}

[0062] Model training:

[0063] Initialize C using the pre-trained language model and train the critic model with the goal of maximizing likelihood. It is hoped to maximize the expected value of D_critic, where the expected value is the logarithm of the conditional probability of the reflection tokens.

[0064]

[0065] in, is the expected value of the D_critic training data set, that is, the expected value of p c (y|x) find the expectation; p c (y|x) is the accuracy of using the large language model C when inputting x and outputting y (accuracy is determined based on the model output value and the original annotation)

[0066] Evaluation of the judging model:

[0067] Construct another batch of data sets in the same format and in the same way as the training set. Judge the consistency with the GPT-4 output prediction on the retrieval category, and the training goal is usually to reach more than 90%.

[0068] 2. A cache query module, used to search for cache questions and / or information fragments related to the input question by calculating similarity distances in the vector space if they exist in the cache database;

[0069] The cache database belongs to a vector database, which puts the retrieved questions into the cache database.

[0070] Figure 5 A structure of a cache database stored in a vector database is provided in an embodiment of the present invention.

[0071] The stored data is divided into two structures:

[0072] Store query-answer pairs, such as Google (or Bing, Baidu, etc.) search engines, expert-annotated question and answer sets, etc. Store the query it retrieves and the corresponding related search results as answers, and make good associations between the question and answer pairs when storing.

[0073] During retrieval, related questions are searched in the database through a new question. Generally, the number of top ranked questions (for example, 3) and the similarity threshold greater than the vector similarity threshold (for example, normalized similarity of 0 to 1, with a threshold of 0.9) are limited. The cached answers associated with the found questions are returned.

[0074] Only the structure of the answer is stored. For example, Wikipedia entries, dictionaries, textbooks, OA papers, etc., use a semantic segmentation algorithm with a limited length (such as 512 tokens) to segment the text content into text fragments.

[0075] The text fragments are vectorized through the vector embedding model (see 3.2 Vector Embedding Model Description) and the related text fragment vector data and content are stored in the cache vector database.

[0076] During retrieval, the similarity between the query and the answer is calculated (see 3.2 Similarity calculation formula), and the relevant text fragments are returned by limiting the number of top rankings (for example, 64) and satisfying a vector similarity threshold.

[0077] Through the second storage form, more relevant knowledge content can be cached, which can serve as a supplement to the content of online search questions and answers.

[0078] The goal of designing a cache database is to speed up the performance of retrieval and question answering. By vectorizing and storing data in a local database through a cache mechanism, the time for network API calls and crawling and parsing can be saved. In practice, the average delay of network search calls can be reduced from 15S to less than 2S.

[0079] As the system goes online, the content of the cache database will gradually increase, and more and more professional knowledge will be accumulated, which will cover more common problems and greatly improve the user experience of waiting delays in network searches.

[0080] The cached content should be updated after a certain period of time. This update can be performed every 3 to 6 months. This can fix the problem by calling the search engine in batches and comparing the search results, updating the updated content returned by the search engine service to keep the cache database data timely.

[0081] Cache query of questions can be performed using direct query or text similarity algorithm search.

[0082] Direct query, i.e., exact string matching query, can quickly obtain query matching results by caching the query statements in the database and using mechanisms such as index encoding.

[0083] The text similarity algorithm uses a string matching algorithm. If the text similarity is higher than a certain threshold (for example, a normalized similarity of 0 to 1 distribution, with a configurable threshold of 0.9), the most similar matching result is taken. For example, the normalized edit distance can be used as a text similarity metric.

[0084] Edit distance, also known as Levenshtein distance, is a quantitative measure of the difference between two strings. It represents the minimum number of edit operations required to convert one string into another, including inserting, deleting, or replacing a character. Edit distance definition

[0085]

[0086] i and j represent the subscripts of string a and string b respectively. The subscripts start from 1.

[0087] Normalized edit distance definition:

[0088] r = (sum – lev) / sum

[0089] Where sum refers to the sum of the lengths of str1 and str2, and lev is the edit distance.

[0090] The cache query module is used to search the cache database for the input question. If the question is already cached in the cache library, the search results of the answers related to the question are directly returned. If there is no matching result in the search, the network search module is entered.

[0091] In one embodiment, a vector embedding model module is also included, and the vector embedding model module belongs to the model service. The retrieval and ranking algorithms are roughly divided into two categories, sparse retrieval based on words (such as inverted index algorithms such as TF-IDF and BM25) and dense retrieval based on text segment embedding vector space. Dense retrieval is different from sparse retrieval. Dense retrieval embeds queries and documents into a continuous vector space according to specific criteria, such as semantic similarity. Because it can be trained, dense retrieval methods show greater flexibility and potential in adaptability.

[0092] The vector embedding model is a technology that maps discrete variables (such as words) to a continuous vector space. In this vector space, semantically similar objects are mapped to adjacent positions, allowing computers to better understand and process the relationship between these discrete variables. Embedding technology is widely used in natural language processing, especially in the field of retrieval, and is one of the important cornerstones of modern artificial intelligence.

[0093] In this embodiment, Embedding is such a mapping function:

[0094] f:V→R d

[0095] Among them, V represents the word list sequence, R dRepresents a d-dimensional real vector space. The current retrieval vector model d is generally 1024 or 4096 dimensions. The goal of the Embedding model is to learn this mapping function f so that in the mapped vector space, similar objects are as close as possible and dissimilar objects are as far away as possible.

[0096] Large-scale specialized pre-training further improves the performance of the Embedding model on knowledge-intensive tasks. Based on the encoder-only Transformer-like sequence modeling, by developing a self-supervised learning method with an attention mechanism, it is usually pre-trained on large-scale unaligned documents through contrastive learning, so it performs well in terms of versatility and can better adapt and generalize to new fields or tasks.

[0097] The MTEB leaderboard (https: / / huggingface.co / spaces / mteb / leaderboard) spans 8 embedding tasks, covering 58 datasets and 112 languages.

[0098] Through the MTEB list, you can select the top-ranked large-scale pre-trained Emedding models for English or Chinese, which can be used for fast coarse recall in vector embedding retrieval tasks. The existing models available for selection are BGE (https: / / github.com / FlagOpen / FlagEmbedding),

[0099] Models such as SFR-Mistral (https: / / huggingface.co / Salesforce / SFR-Embedding-Mistral).

[0100] After the text content is calculated by the vector embedding model, it becomes a vector of fixed length. The relevance of the question or answer text is calculated using the following similarity calculation formula for sorting.

[0101] For example, if cosine similarity is used, the input variable is a Float embedding vector in the text field. Cosine similarity calculates the angle between two sets of data (assuming that they both start from [0,0,…] to [A1,…,A n ] and [B1,...,B n ]), the calculation formula is as follows,

[0102]

[0103] Its value range is [-1,1], and the larger the value, the higher the similarity.

[0104] Other similar similarity distances available include L2 square, inner product, etc., as listed below:

[0105]

[0106] 3. The web search module belongs to the external search engine API and is used to perform web search when the input question does not exist in the cache database;

[0107] Specifically, the network search module includes three sub-modules: sub-problem splitting model, network search engine call, and web page content collection.

[0108] 3.1 Sub-problem splitting model

[0109] Due to the way users express their input questions, potential information missing, etc., it is necessary to break down the questions to obtain better search results.

[0110] The sub-question splitting model uses a large language model to split the input question Query into multiple sub-questions that can be directly searched by network search engines.

[0111] Model splitting example:

[0112] Question: In reflected wave seismic exploration, the most accurate statement regarding stackingis().

[0113] A.The more stacking,the better

[0114] B.Stacking is a low-pass filter

[0115] C. The fewer stacking, the better

[0116] D.Stacking is a high-pass filter

[0117] Sub-question 1: effect of stacking on signal quality in seismic exploration

[0118] Sub-problem 2: stacking as a filter in seismic data processing

[0119] Subproblem 3: optimal stacking number seismic exploration

[0120] You can use the prompt project to call a large language model, such as GPT-4, Llama3, Qwen, etc., to obtain sub-problem splitting. An example of a prompt word splitting 3 sub-problems is as follows:

[0121] Write 3google search queries to search online that form an objectiveopinion from the following:"{question}".Use the current date if needed.You must respond with a list of strings in the following format:["query 1","query2","query 3"].'

[0122] 3.2 Internet search engine calls

[0123] Through the API services provided by the top network search engines, such as Google, Bing, Baidu and other search engine service providers, you can access the Internet search content. Enter the question and sub-questions to obtain the most relevant search results (for example, 16) and the sorting sequence for each question.

[0124] Taking Google search as an example, briefly describe its configuration and calling process.

[0125] 1) Register a Google account in the Google Cloud Console at https: / / console.cloud.google.com / apis / .

[0126] 2) Create a project and follow the instructions to get an API Key.

[0127] 3) Configure Google Programmable Search Engine and obtain the search engine ID, i.e. CX ID. 4) Configure the above two information 2 and 3 and call it through the API interface. The interface address is

[0128] https: / / www.googleapis.com / customsearch / v1?key={YOUR_KEY}&q={SEARCH_WORDS}&cx={YOUR_CX}&start={16}&num={16}

[0129] The returned sample is as follows:

[0130] {

[0131] "source":"google search",

[0132] "id":"https: / / www.sciencebase.gov / catalog / item / 583f4114e4b04fc80e3c4a1a",

[0133] "index":1,

[0134] "section":"",

[0135] "subsection":"",

[0136] "title":"An Open Repository of Earthquake-Triggered Ground-Failure...",

[0137] "text":"An Open Repository of Earthquake-Triggered Ground-FailureInventories Dates Citation Summary Summary Child Items(2)View children inFolder Search children Contacts Poin t of Contact:gs-haz_groundfailure@usgs.gov,Robert Schmitt,Kate E Allstadt Metadata Contact:U.S.GeologicalSurvey,Geologic Hazards Science Center Distributor:U.S.Geological Survey-ScienceBase Originator:Robert G Schmitt,Hakan Tanyas,Anna Nowicki Jessee,JingZhu,Kat herine M Biegel,Kate E Allstadt,Randall W Jibson,Eric M Thompson,CeesJ van Westen,Hiros hi Sato,David J Wald,Jonathan W Godt,Tolga Gorum,Chong XuUSGS Mission Area:Natural Hazards SDC Data Owner:Earthquake Hazards ProgramAttached Files sorting Metadata First Recent First Alphabetical Click ontitle to download individual files attached to this item.Related ExternalResources Purpose Tags Prove nance Additional Information IdentifiersCommunity Exten sion Item Actions View Item as...Save Item as...View Item..."

[0138] }

[0139] The API programming interface services provided by other search engines are similar.

[0140] 3.3 Web Content Collection

[0141] The results returned through the search engine service provider's API interface only contain an overview of the web page. The index content processed by the search engine is called a snippet. The full text content is required to accurately answer the question. At this time, it is necessary to crawl the full text content through a crawler and a web page parsing program, and a PDF parsing engine when necessary (if there is a PDF document). If the content is too long, it needs to be compressed through a summary module.

[0142] The crawler program can be implemented by writing relevant programs in Python language, and can return the full text content of the web page analysis for the input URL.

[0143] PDF parsing is mainly based on OCR optical character recognition technology. For example, the open source program Nougat (https: / / github.com / facebookresearch / nougat) can be used to parse PDF documents.

[0144] After parsing the full text, since the text may be too long (for example, more than 32,768 words), a summary module is used to extract key information related to the question. This step can be achieved through a large model. An example of a prompt word is as follows:

[0145] {data}\n Using the above text,summarize it based on the followingtask or query:"{query}".\n If the query cannot be answered using the text,YOUMUST summarize the text in short.\n Include all factual information such as numbers,stats,quotes,etc if available.

[0146] After obtaining the web page search content, the web search results are entered into the vector embedding model module, for example, 16 texts each including the original question and 3 sub-questions, and a total of 64 texts are recalled.

[0147] 4. The re-ranking module belongs to the model service. The re-ranking model (such as the Rerank model) is used to re-rank the relatively large number of text fragment results (for example, 64 or 256) recalled in the rough ranking, so that the final target result is a more streamlined Top-K (for example, K=8), so that the results that best meet the search requirements are returned in the front row, achieving the goal of selective ranking.

[0148] The training of the re-ranking model mainly includes supervised data preparation, generation of hard negative samples, and model training and tuning. 1) Supervised data preparation includes data generation and open source data acquisition. Data generation involves obtaining data through QA of the domain text question-answering pairs in the data generation step. Other supervised data pairs can also be obtained through channels such as public datasets in similar fields and expert-annotated datasets.

[0149] 2) After obtaining supervised data, in order to obtain more accurate ranking results, we construct a difficult negative sample set based on the existing data to improve the model accuracy. First, we use the full text data as the search library. For each positive sample question, we retrieve the most relevant text set outside the positive sample as a negative sample to increase the difficulty of model distinction. Through multiple training and learning, the model accuracy is improved. 3) After the training data is constructed, the query text and the answer text are input into the language model through cross-coding to jointly encode and learn the relevance score. See Figure 6 .

[0150] The question-answer text pairs are cross-encoded to learn similarity scores. The training method is to feed positive and negative samples into the model together, where the logits score of the positive samples is greater than that of the negative samples in a batch.

[0151] Using cross entropy as the loss function, the Reranker loss function is the following Multiple NegativeRanking Loss.

[0152]

[0153] The advantages of this training scheme are real-time cross-coding and high accuracy.

[0154] In summary, this example constructs a BERT-like (Bidirectional Encoder Representations from Transformers) model that performs cross-encoding learning training on multilingual ranking pair data, question-answer pair data, and semantic matching pair data sets.

[0155] In one embodiment, for question-answering in the academic field, the re-ranking model is trained as follows: academic data is evenly selected by subject, questions are generated based on the academic data, and question-answer pairs, i.e., positive samples, are formed; the most relevant text sets outside the positive samples are retrieved based on the generated questions as negative samples; the positive and negative samples are sent together to the re-ranking model for training, wherein the logits score of the positive samples is greater than that of the negative samples in a batch.

[0156] 5. The large language model generation module belongs to the large language model API. It obtains the Top-K (for example, K=8) text fragments after re-ranking through network search or local cache database retrieval, and assembles them into prompts together with questions and instruction prompts and sends them to the large model, so that the large model can distinguish useful information based on the content and answer it based on inherent knowledge, thereby achieving the purpose of RAG (Retrieval Augment Generation). The base large language models that can be selected include GPT-4, Llama3, Qwen, etc. A prompt for reference is as follows:

[0157] {

[0158] "promptRagInput":"Answer the professional[question]based on the[references],which are certain fundamental entities with corresponding explanations which may be relevant to the question at hand or might beunrelated.Use the[references]only if they are highly relevant to the question.If the[references]lack relevance is strong,disregard them and answeraccording to your own expertise.\n\n[references]are as follows:\n{ragReference}\n[question]:{input}The Answer is"

[0159] }

[0160] See also Figure 7 , which is an example of a network enhanced answer provided by an embodiment of the present invention.

[0161] System deployment architecture: A deployment architecture for a network search enhanced large language model system is shown in the attached Figure 2 As shown, the system deployment is implemented using cloud services (taking Alibaba Cloud as an example, AWS / Google Cloud / Tencent Cloud are similar) as the carrier.

[0162] 1. Application Services

[0163] The application service is the central module of the web search enhanced large language model system. It carries the dialogue questions sent by the user's client (laptop, desktop or smart phone), connects other modules through the web search enhanced service (Web RAG), crawls the network content through the crawler service, and stores text and PDF files through the OSS file storage service. This module is mainly based on the back-end service implemented by Python and JAVA programs, with front-end interactive pages. Usually, several (for example, 2 or 4) cloud server (for example, Alibaba Cloud ECS elastically scalable cloud host virtual server) instances are configured according to the needs of the request traffic to achieve load balancing.

[0164] 2. External Search Engine API

[0165] This module corresponds to 2.2 web search engine call. By configuring the API of external search engines (Google / Bing / Baidu), the latest relevant information can be instantly queried on the Internet according to the query request, and the relevant URL and summary content can be returned.

[0166] 3. Model service (GPU server)

[0167] The model services that need to be deployed to the GPU server include vector embedding model, reordering model and PDF parsing service. The models in this embodiment each require a GPU server instance with a single card Nvidia V100 or above. It is recommended to use NvidiaA100 for better performance. After each model is deployed, the Http restful reasoning interface is exposed internally and called through the Web RAG hub module.

[0168] 4. Vector Database

[0169] For vector databases used to cache data, for better performance and production practicality, you can choose Zilliz Cloud or Alibaba Cloud vector search service products. Vector databases are used to store vectorized cached text data and provide ms-level query responses during coarse recall retrieval. Open source vector databases can also be used here, such as Milvus / Faiss / Pinecone / weaviate, etc., which are distributed and deployed through multiple cloud hosts. Deploying open source vector databases requires more configuration and operation and maintenance work to achieve good performance and stable operation.

[0170] 5. Large Language Model API

[0171] For the base large language model, you can choose OpenAI's GPT-4 or the large model service provided by cloud service providers (such as the Tongyi Qianwen and Llamma3 model services provided by Alibaba Cloud Bailian / Lingji large model products). You can pay to use it after opening an account. Web RAG calls the relevant large model in the step of generating answers from the large language model to achieve enhanced answers to network search information.

[0172] In the above modules, application services and model services belong to the system's own services; external search engine API, vector database service and large language model API belong to external cloud services. The above deployment modules are connected in series through application services as the hub to form a complete network search enhanced large language model system.

[0173] The present invention uses the network to search for relevant content and then provides relevant information to the large language model, providing more information for the large language model output, and enhancing the accuracy and timeliness of the large model's answers. At the same time, the use of this method can reduce costs and save the huge cost of pre-training of large models. This method can find the source of information for the output of the large model, effectively reducing the situation of "hallucination" in the output of the large prediction model. In addition, the query response time can be significantly accelerated through the designed cache data mechanism.

[0174] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, a method for enhancing generation of a large language model using network search in any of the above embodiments is implemented. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and related programs for enhancing a large language model using network search. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner. When the computer program is executed by the processor, an event recognition method based on graph node embedding is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0175] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0176] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in any of the above-mentioned message pushing method or message forwarding method embodiments are implemented.

[0177] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, a method for enhancing generation of a large language model using network search in any of the above embodiments is implemented.

[0178] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0179] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0180] Those skilled in the art can understand that the above are only preferred examples of the invention and are not intended to limit the invention. Although the invention is described in detail with reference to the above examples, those skilled in the art can still modify the technical solutions recorded in the above examples or replace some of the technical features therein with equivalents. Any modification, equivalent replacement, etc. made within the spirit and principle of the invention shall be included in the protection scope of the invention.

Claims

1. A method for enhancing large language model generation using web search, characterized in that: include: The input question is judged by the judgment model to determine whether it needs to be searched; If no retrieval is required, the large language model directly generates answers based on the input question; If a search is required, determine whether the input question exists in the cache database; If present in the cache database, searching for cached questions and / or information fragments related to the input question by calculating similarity distance in the vector space; If it does not exist in the cache database, a network search is performed; Reorder the information fragments obtained by searching the cache database or the information fragments obtained by searching the network according to their relevance to the input question, and select the first n information fragments; The input question and the selected information fragments are fed into the large language model to generate the answer.

2. The method according to claim 1, characterized in that The network search includes: Split the input problem into multiple sub-problems for web search; and perform web search on the input problem and the sub-problems.

3. The method according to claim 2, characterized in that An input question is split into multiple sub-questions for network search through a sub-question splitting model; the sub-question splitting model is a large language model.

4. The method according to claim 1, characterized in that: Also includes: The input question and the information fragments obtained by the network search are vectorized to obtain vector data, and the input question and the information fragments obtained by the network search and their corresponding vector data are added to the cache database.

5. The method according to claim 1, characterized in that Through the re-ranking model, the information fragments obtained by searching the cache database or the information fragments obtained by searching the network are re-ranked according to their relevance to the input question.

6. The method according to claim 1, characterized in that The re-ranking model is trained as follows: academic data is evenly selected by subject, questions are generated based on the academic data to form question-answer pairs, i.e., positive samples; the most relevant text set outside the positive samples is retrieved based on the generated questions as negative samples; the positive and negative samples are sent together to the re-ranking model for training, wherein the logits score of the positive samples is greater than that of the negative samples in a batch.

7. The method according to claim 1, characterized in that The step of inputting the input question and the selected information fragment into the large language model to generate an answer further includes: The input question, selected information fragments and instruction prompts are assembled into prompt words and sent to the large language model to ensure that the large language model answers based on the currently provided information and combined with inherent knowledge.

8. A system for enhancing the generation of large language models using web search, characterized in that: include: The intention understanding module is used to judge whether the input question needs to be searched through the judgment model; If a search is required, determine whether the input question exists in the cache database; a cache query module for searching cache questions and / or information fragments related to the input question by calculating similarity distances in a vector space if present in the cache database; A network search module, used to perform a network search if the search does not exist in the cache database; A reordering module is used to reorder the information fragments obtained by searching the cache database or the information fragments obtained by searching the network according to their relevance to the input question, and select the information fragments ranked in the top n; A large language model generation module is used to generate answers directly based on the input question if no retrieval is required; If a search is required, the input question and the selected information fragments are input into the large language model to generate the answer.

9. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the method for enhancing the generation of a large language model using network search as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a method for enhancing generation of a large language model using network search is implemented as described in any one of claims 1-7.

Images