Timing information question and answer method and system based on retrieval enhancement and large language model

Abstract

The invention discloses an aging information question and answer method and system based on retrieval enhancement and a large language model, and the method comprises the steps: obtaining and analyzing a target document, and uploading the analyzed text data to a database; taking the content of the text data as a reference content context, and generating a training data set by using a large language model; using the training data set to train the Embedding model; inputting the text data into the trained Embedding model to obtain a feature code corresponding to each piece of text data, and associating the feature code with the corresponding text data; receiving a user question, converting the user question into a user question vector, performing similarity comparison on the feature code and the user question vector, and selecting N pieces of text data with the highest similarity as target data; and inputting the target data and the user question into the large language model, and outputting a target answer. According to the method, the adaptation cost can be reduced, and the accuracy of the whole large language model system is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of artificial intelligence technology, and specifically to a time-sensitive information question-answering method and system based on retrieval enhancement and a large language model. Background Art

[0002] The current general large language model knowledge mainly comes from the data set used in training, while specific knowledge in fields such as news and finance is timely and accurate. The general large language model cannot complete the task well when faced with tasks with relevant timely knowledge, which requires supplementing knowledge for the general large language model.

[0003] Currently, the methods for supplementing general large language models with additional knowledge can be roughly divided into two categories: one is to convert knowledge into model weight parameters through training and embed them into the model; the other method uses knowledge as the context of the conversation and inputs it into the large language model in each conversation. However, both the training method and the context method are affected by many factors, which makes supplementing large language models with additional knowledge to complete time-sensitive tasks in fields such as news and finance a complex and difficult problem.

[0004] There are currently two main methods in the industry, one is the training method, and the other is the RAG (retrieval enhanced generation) method.

[0005] The training method is characterized by strong model specificity and high knowledge compression. Since there is no additional context during reasoning, the consumption during reasoning is small. This method first needs to collect knowledge question and answer pairs to form a training data set, and then use full parameter fine-tuning, lora fine-tuning, p-tuning and other methods to train the general large language model. Generally speaking, the larger the amount of data and the larger the number of parameters for model training, the better the effect of the fully trained model. However, this method has obvious disadvantages. Training requires a large amount of data, time cost, resource cost, etc. The time cost of training a model is often calculated on a monthly basis, which leads to the fact that the speed of training updates in fields such as finance and news often cannot keep up with the speed of professional knowledge updates. And the resource cost of training once is often very huge. Therefore, using training methods to deal with the problem of time-sensitive knowledge is a method that has a certain effect but is not economical and practical.

[0006] The RAG (Retrieval-Augmented Generation) method is a method for retrieving relevant knowledge from texts such as questions and using the knowledge as a context to enhance the model to generate responses. This method first uses an Embedding model to retrieve knowledge text related to the user's question, then inputs the retrieved knowledge as context along with the user's question into a general large language model, and finally the general large language model enhances the quality of the generated response after referring to the knowledge. This method does not require changing the large language model, but only needs to update the retrieved knowledge base. Compared with the training method, it greatly reduces the time cost and resource cost, and the update speed of its local knowledge base can be consistent with the update speed of knowledge. The key link of this method is that the Embedding model retrieves knowledge. However, due to the fact that professional fields such as finance and news contain more professional terms and professional scenarios, the current Embedding model is often unsatisfactory in terms of accuracy for fields such as finance and news, and cannot be well adapted to application scenarios, resulting in low accuracy and reliability of the general large language model response. Summary of the invention

[0007] In order to overcome the shortcomings of the prior art, the purpose of an embodiment of the present invention is to provide a time-sensitive information question and answer method and system based on retrieval enhancement and a large language model, which automatically generates a training data set by utilizing the generalization ability of a large language model, realizes the training of the Embedding model, reduces the adaptation cost, and increases the accuracy of the entire large language model system.

[0008] To solve the above problems, the first aspect of the embodiment of the present invention discloses a time-sensitive information question answering method based on retrieval enhancement and a large language model, including:

[0009] Obtaining a target document, performing text parsing on the target document, obtaining parsed text data, and uploading the text data to a database;

[0010] Using the content of the text data as a reference content context, and using a large language model to generate a training data set;

[0011] Using the training data set to train the Embedding model;

[0012] Input the text data into the trained Embedding model to obtain the feature code corresponding to each text data, upload the feature code to the database, and associate the feature code with the corresponding text data;

[0013] Receive user questions and input the user questions into the trained Embedding model to obtain a user question vector, perform a similarity comparison between the feature code and the user question vector, and select text data corresponding to the N feature codes with the highest similarity as target data, where N≥1;

[0014] The target data is used as context reference material and input into a large language model together with the user question, so that the large language model outputs a target answer.

[0015] As a preferred embodiment, in a first aspect of the embodiment of the present invention, the content of the text data is used as a reference content context, and a training data set is generated using a large language model, including:

[0016] Input the content of the text data into a large language model context list as a reference article for the large language model;

[0017] A prompt instruction is set so that the large language model performs the task of generating the training data set, and the prompt instruction includes a query field, a positively related knowledge field, and a negatively related knowledge field.

[0018] As a preferred embodiment, in the first aspect of the embodiment of the present invention, using the training data set to train the Embedding model includes:

[0019] The training data set is input into the Embedding model for comparative learning training, which includes:

[0020] The InfoNCE function is used as the contrastive learning loss function, and its formula is:

[0021]

[0022] Wherein, Ls(.) represents the loss function, q represents the vector after the question or knowledge in the inquiry field in the training data set is encoded, p* represents the vector after the positively correlated content in the positively correlated knowledge field in the training data set is encoded, and p′ represents the vector after the negatively correlated content in the negatively correlated knowledge field in the training data set is encoded; τ is the temperature coefficient, and its size is set according to needs; s(q,p*) represents the similarity between q and p*, and s(q,p) represents the similarity between q and p;

[0023] By minimizing the loss function, the parameters of the Embedding model are fine-tuned using the BGE-M3 pre-trained model as the fine-tuning target model to complete the comparative learning training of the Embedding model.

[0024] As a preferred embodiment, in the first aspect of the embodiment of the present invention, the text data is input into the trained Embedding model to obtain the feature code corresponding to each text data, the feature code is uploaded to the database, and the feature code is associated with the corresponding text data, including:

[0025] Input the text data into the trained Embedding model for feature encoding to obtain feature encoding corresponding to each text data, wherein the feature encoding is a 1×1024-dimensional feature vector;

[0026] A feature vector field is added to the database, and the feature code is stored in the feature vector field corresponding to each text data.

[0027] As a preferred embodiment, in the first aspect of the embodiment of the present invention, the user question is input into the trained Embedding model to obtain a user question vector, and the feature code is compared with the user question vector for similarity, and the text data corresponding to the N feature codes with the highest similarity are selected as the target data, including:

[0028] Input the user question into the trained Embedding model to vectorize the user question to obtain a user question vector, which is also a 1×1024-dimensional feature vector;

[0029] Compare the user question vector with each feature code for similarity, and sort them from large to small according to the similarity value;

[0030] The text data corresponding to the top N feature codes are selected as the target data.

[0031] As a preferred embodiment, in the first aspect of the embodiment of the present invention, the database is an Elasticsearch database.

[0032] As a preferred embodiment, in the first aspect of the embodiment of the present invention, the similarity adopts cosine similarity, or / and, and the large language model is any one of GPT-3, GPT-4, PaLM, LLaMA and Qwen-max.

[0033] A second aspect of an embodiment of the present invention discloses a time-sensitive information question-answering system based on retrieval enhancement and a large language model, including:

[0034] A parsing unit, used to obtain a target document, perform text parsing on the target document, obtain parsed text data, and upload the text data to a database;

[0035] A generating unit, configured to use the content of the text data as a reference content context and generate a training data set using a large language model;

[0036] A training unit, used to train the Embedding model using the training data set;

[0037] An encoding unit, used for inputting the text data into the trained Embedding model, obtaining a feature code corresponding to each text data, uploading the feature code to the database, and associating the feature code with the corresponding text data;

[0038] A comparison unit is used to receive user questions, input the user questions into the trained Embedding model, obtain a user question vector, compare the feature code with the user question vector for similarity, and select text data corresponding to N feature codes with the highest similarity as target data, where N≥1;

[0039] The output unit is used to input the target data as context reference material together with the user question into the large language model so that the large language model outputs a target answer.

[0040] A third aspect of an embodiment of the present invention discloses an electronic device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, the steps of the time-sensitive information question and answer method based on retrieval enhancement and a large language model disclosed in the first aspect of the embodiment of the present invention are implemented.

[0041] A fourth aspect of an embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the steps of the time-sensitive information question-answering method based on retrieval enhancement and a large language model disclosed in the first aspect of the embodiment of the present invention.

[0042] A fifth aspect of an embodiment of the present invention discloses a computer program product. When the computer program product runs on a computer, the computer executes the steps of the time-sensitive information question and answer method based on retrieval enhancement and a large language model disclosed in the first aspect of the embodiment of the present invention.

[0043] A sixth aspect of an embodiment of the present invention discloses an application publishing platform, which is used to publish a computer program product. When the computer program product runs on a computer, the computer executes the steps of the time-sensitive information question and answer method based on retrieval enhancement and a large language model disclosed in the first aspect of the embodiment of the present invention.

[0044] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0045] The embodiment of the present invention utilizes the generalization ability of the large language model to automatically generate a training data set, and can realize the training of the Embedding model with a small amount of data, so that the RAG system can accurately retrieve and match professional problems, terms, etc. in professional fields such as finance and news, without adjusting the general large language model, thereby reducing the adaptation cost and increasing the accuracy of the entire large language model system. When faced with problems with high timeliness and high accuracy requirements, a large amount of information can be cited to provide more in-depth, accurate and valuable answers, thereby improving the reliability of the generated text. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flowchart of a time-sensitive information question-answering method based on retrieval enhancement and a large language model provided by an embodiment of the present invention;

[0047] Figure 2 is a schematic diagram of a financial news document provided by an embodiment of the present invention;

[0048] Figure 3 It is a schematic diagram of text data entered into a database provided by an embodiment of the present invention;

[0049] Figure 4 is a schematic diagram of a training data set generated according to an embodiment of the present invention;

[0050] Figure 5 is a schematic diagram of a loss reduction process provided by an embodiment of the present invention;

[0051] Figure 6 It is a schematic diagram of field settings of an ES database provided by an embodiment of the present invention;

[0052] Figure 7 It is a schematic diagram of the result of similarity matching between text data and user questions provided by an embodiment of the present invention;

[0053] Figure 8 is a schematic diagram of a target answer output by a large language model provided by an embodiment of the present invention;

[0054] Fig. 9 It is a structural diagram of a time-sensitive information question-answering system based on retrieval enhancement and a large language model provided by an embodiment of the present invention;

[0055] Fig.10 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] This specific implementation manner is merely an explanation of an embodiment of the present invention, and it is not a limitation of the embodiment of the present invention. After reading this specification, a person skilled in the art may make non-creative modifications to the embodiment as needed, but as long as it is within the scope of the claims of the embodiment of the present invention, it is protected by the patent law.

[0057] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments in the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the embodiments of the present invention.

[0058] The term "comprise" and any variations thereof in the specification and claims of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0059] In the embodiments of the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0060] The embodiment of the present invention utilizes the generalization ability of the large language model to automatically generate a training data set, and can realize the training of the Embedding model with a small amount of data, so that the RAG system can accurately retrieve and match professional problems, terms, etc. in professional fields such as finance and news, without adjusting the general large language model, thereby reducing the adaptation cost and increasing the accuracy of the entire large language model system. When faced with problems with high timeliness and high accuracy requirements, a large amount of information can be cited to provide more in-depth, accurate and valuable answers, thereby improving the reliability of the generated text, which is described in detail below in conjunction with the accompanying drawings.

[0061] Embodiment 1

[0062] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of a method for answering time-sensitive information based on retrieval enhancement and a large language model disclosed in an embodiment of the present invention. Figure 1 As shown, the time-sensitive information question answering method based on retrieval enhancement and large language model includes:

[0063] S110, obtaining a target document, performing text parsing on the target document, obtaining parsed text data, and uploading the text data to a database.

[0064] There are many ways to obtain the target document. For example, the required documents can be crawled on relevant websites through crawler software or plug-ins. In a preferred embodiment of the present invention, the target document is time-sensitive data, corresponding to time-sensitive information, including but not limited to relevant materials in the fields of news, policy, finance, and financial management.

[0065] The target document can exist in multiple formats and file types. The corresponding parsing method can be determined according to the file type, and then the target document is parsed. The parsing result, i.e., text data, is entered into the Elasticsearch (ES) database to facilitate query and search in subsequent steps. The text data is also referred to as knowledge in the embodiment of the present invention. For example, for XML files, any one of the DOM parsing method, the SAX parsing method, and the JDOM parsing method can be used for parsing.

[0066] For example, you can Figure 2 We process some of the financial news documents obtained from the .txt file. First, we determine the document file type, then perform text parsing based on the file type to obtain text data, and finally enter the text data into the Elasticsearch database to facilitate subsequent query and search. The entry results are as follows: Figure 3 shown.

[0067] S120: Using the content of the text data as a reference content context, and using a large language model to generate a training data set.

[0068] The purpose of using time-sensitive text data as the reference content context of the large language model and inputting it into the large language model to generate a training data set is twofold. First, a small number of samples can be used to complete the training of the subsequent Embedding model, saving costs. Second, since a small number of samples are all time-sensitive information, the RAG (Retrieval-Augmented Generation) system can accurately retrieve and match professional issues and terms in professional fields such as finance and journalism, so that the entire large language model system process can be effectively, accurately and economically applied to the financial and journalism fields.

[0069] The large language model can adopt a general large language model, for example, it can be any one of GPT-3, GPT-4, PaLM, LLaMA and Qwen-max, and of course it can also be other general large language models.

[0070] The text data obtained in step S110 is input into the large language model context list as a reference article for the model, and the prompt instruction is set to: "Build several questions based on the above article for training, the format is as follows: {"query":"","pos":[""],"neg":["","","","","","",""]}. Among them, in the prompt instruction training data set format, the query field represents knowledge or user inquiry, also known as the query field, the pos field represents the positively related knowledge fragment in the reference article, also known as the positively related field, and the neg field represents the negatively related knowledge fragment in the reference article, also known as the negatively related field.

[0071] Then a general large language model is used to generate a training dataset, such as Figure 4 As shown in the figure. Finally, a total of 1,300 training data sets were generated, including 1,000 news knowledge data and 300 policy knowledge data. In order to better verify and test the model performance, 157 real user questions and answers were manually collected to test the trained Embedding model.

[0072] S130: Use the training data set to train the Embedding model.

[0073] The training data set generated in step S102 is used to perform comparative learning training on the Embedding model. Specifically, the BGE-M3 pre-trained model can be used as the fine-tuning target model. The fine-tuning parameters are: epoch is set to 10 rounds, batch size is set to 10, loss uses InfoNCE Loss, learning rate is set to 1e-5, learning rate drop temperature system τ is set to 0.02, and the loss drop process is as follows: Figure 5 shown.

[0074] The InfoNCE function is used as the contrastive learning loss function, which is represented by Ls(.), and its formula is as follows:

[0075]

[0076] Among them, Ls(.) represents the loss function, q represents the vector after the question or knowledge in the inquiry field in the training data set is encoded, p* represents the vector after the positively correlated content in the positively correlated knowledge field in the training data set is encoded, and p′ represents the vector after the negatively correlated content in the negatively correlated knowledge field in the training data set is encoded; τ is the temperature coefficient, and its size is set according to needs; s(q,p*) represents the similarity between q and p*, and s(q,p) represents the similarity between q and p.

[0077] Similarity can be calculated using cosine similarity, for example:

[0078]

[0079] During the training process, by minimizing the loss function, the Embedding model learns the paradigm of the encoded query so that the query has a high similarity with the positively related content knowledge and a low similarity with the negatively related content knowledge.

[0080] It can be understood that when each training sample is input into the Embedding model for training, the Embedding model can output the above-mentioned q, p* and p′, and then calculate the comparative learning loss through the InfoNCE function, and then use the BGE-M3 pre-trained model to adjust the parameters of the Embedding model until the loss drops to the preset threshold or the training data set is trained a preset number of times.

[0081] S140, inputting the text data into the trained Embedding model to obtain a feature code corresponding to each text data, uploading the feature code to the database, and associating the feature code with the corresponding text data.

[0082] After training, the Embedding model can perform feature encoding on text data and user questions, that is, vectorize text data and user questions.

[0083] This step is to feature encode the text data obtained in step S110. Each text data is represented by a 1×1024-dimensional vector, and a feature vector "dense_vector" field (i.e., feature vector field) is added to the ES database, which corresponds to the text data as a text feature vector, to facilitate indexing and querying in subsequent steps. Figure 6 As shown, the _id field is the index id, _index is the ES database index name, context is the text data, dense_vector is a 1*1024 feature vector, and metadata.source is the source document identifier.

[0084] S150, receiving user questions, and inputting the user questions into the trained Embedding model to obtain a user question vector, performing a similarity comparison between the feature coding and the user question vector, and selecting text data corresponding to N feature codings with the highest similarity as target data, N≥1.

[0085] The questions asked by users (i.e. user questions) are vectorized, and vector matching is performed on the knowledge in the database to filter out the knowledge with the highest similarity.

[0086] Specifically, the Embedding model trained in step S130 can be deployed and interfaced. When a user asks a question, the Embedding model first performs feature encoding and vectorization of the user's question into a 1×1024 vector, and then performs similarity calculation with the feature vector field added in step S140 in the ES database (cosine similarity can also be used), and finally selects the top N text data with the highest similarity. The specific value of N can be set as needed, for example, it can be set to 5, or other values, and the call result is as follows: Figure 7 shown.

[0087] S160: Input the target data as context reference material together with the user question into a large language model so that the large language model outputs a target answer.

[0088] The N text data highly relevant to the user's question selected in step S150, i.e., the target data, are used as context reference materials and input into the general large language model together with the user's question. The large language model responds to the user based on the target data, and the response content is recorded as the target answer. Figure 8 The response results of the large language model are shown.

[0089] In order to verify the effectiveness of the present invention, a test set of 157 pieces of knowledge in the financial and news fields was prepared to evaluate the accuracy and reliability of the model, including 100 pieces of financial news knowledge and 57 pieces of financial policy knowledge. The open source models bge-m3 and bge-large-zh-v1.5 were used as baseline models for comparison. The trained model of the present invention is called Sfc-m3. The evaluation indicators used were the commonly used comparison indicators of the Embedding model: MRR (mean reciprocal rank) and Recall (recall rate). The evaluation results are shown in Tables 1-3:

[0090] Table 1. Financial news knowledge assessment results

[0091]

[0092] Table 2. Financial policy knowledge assessment results

[0093]

[0094] Table 3. Evaluation results of financial news and policy hybrid knowledge

[0095]

[0096] In the above standards, MRR@1 reflects whether the first piece of knowledge found by the Embedding model is correct knowledge, emphasizing the order and accuracy of the knowledge found by the model. Similarly, MRR@5 and MRR@10 respectively indicate whether the first five pieces of knowledge and the first ten pieces of knowledge found meet the test set labels. The Recall indicator reflects the coverage of the Embedding model for related knowledge, emphasizing the completeness of the knowledge found by the model.

[0097] It can be seen from the above table that the model trained by the present invention has certain advantages in all indicators in the 100 financial news knowledge test set, the 57 financial policy knowledge test set and the 157 mixed test set, indicating that it has greater advantages and better effects in solving problems in the financial and news fields.

[0098] Embodiment 2

[0099] See also Fig. 9 , Fig. 9 Schematic diagram of the structure of a time-sensitive information question answering system based on retrieval enhancement and large language model disclosed in an embodiment of the present invention. Fig. 9 As shown, the time-sensitive information question answering system based on retrieval enhancement and large language model may include:

[0100] The parsing unit 210 is used to obtain a target document, perform text parsing on the target document, obtain parsed text data, and upload the text data to a database;

[0101] A generating unit 220, configured to use the content of the text data as a reference content context and use a large language model to generate a training data set;

[0102] A training unit 230, configured to train an Embedding model using the training data set;

[0103] The encoding unit 240 is used to input the text data into the trained Embedding model, obtain the feature code corresponding to each text data, upload the feature code to the database, and associate the feature code with the corresponding text data;

[0104] The comparison unit 250 is used to receive user questions, input the user questions into the trained Embedding model, obtain a user question vector, compare the feature code with the user question vector for similarity, and select text data corresponding to the N feature codes with the highest similarity as target data, where N≥1;

[0105] The output unit 260 is used to input the target data as context reference material together with the user question into the large language model so that the large language model outputs a target answer.

[0106] Preferably, the generating unit 220 may include:

[0107] Input the content of the text data into a large language model context list as a reference article for the large language model;

[0108] A prompt instruction is set so that the large language model performs the task of generating the training data set, and the prompt instruction includes a query field, a positively related knowledge field, and a negatively related knowledge field.

[0109] Preferably, the training unit 230 may include:

[0110] The training data set is input into the Embedding model for comparative learning training, which includes:

[0111] The InfoNCE function is used as the contrastive learning loss function, and its formula is:

[0112]

[0113] Wherein, Ls(.) represents the loss function, q represents the vector after the question or knowledge in the inquiry field in the training data set is encoded, p* represents the vector after the positively correlated content in the positively correlated knowledge field in the training data set is encoded, and p′ represents the vector after the negatively correlated content in the negatively correlated knowledge field in the training data set is encoded; τ is the temperature coefficient, and its size is set according to needs; s(q,p*) represents the similarity between q and p*, and s(q,p) represents the similarity between q and p;

[0114] By minimizing the loss function, the parameters of the Embedding model are fine-tuned using the BGE-M3 pre-trained model as the fine-tuning target model to complete the comparative learning training of the Embedding model.

[0115] Preferably, the encoding unit 240 may include:

[0116] Input the text data into the trained Embedding model for feature encoding to obtain feature encoding corresponding to each text data, wherein the feature encoding is a 1×1024-dimensional feature vector;

[0117] A feature vector field is added to the database, and the feature code is stored in the feature vector field corresponding to each text data.

[0118] Preferably, the comparison unit 250 may include:

[0119] Input the user question into the trained Embedding model to vectorize the user question to obtain a user question vector, which is also a 1×1024-dimensional feature vector;

[0120] Compare the user question vector with each feature code for similarity, and sort them from large to small according to the similarity value;

[0121] The text data corresponding to the top N feature codes are selected as the target data.

[0122] Embodiment 3

[0123] See also Fig.10 , Fig.10 A schematic diagram of an electronic device that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present invention described herein or / and claimed.

[0124] like Fig.10 As shown, the electronic device includes at least one processor 310, and a memory connected to the at least one processor 310, such as a ROM (read-only memory) 320, a RAM (random access memory) 330, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 310 can perform various appropriate actions and processes according to the computer program stored in the ROM 320 or the computer program loaded from the storage unit 380 to the random access memory RAM 330. In the RAM 330, various programs and data required for the operation of the electronic device can also be stored. The processor 310, the ROM 320, and the RAM 330 are connected to each other via a bus 340. An I / O (input / output) interface 350 is also connected to the bus 340.

[0125] A number of components in the electronic device are connected to the I / O interface 350, including: an input unit 360, such as a keyboard, a mouse, etc.; an output unit 370, such as various types of displays, speakers, etc.; a storage unit 380, such as a disk, an optical disk, etc.; and a communication unit 390, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 390 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet or / and various telecommunication networks.

[0126] The processor 310 may be a variety of general or / and special processing components with processing and computing capabilities. Some examples of the processor 310 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 310 executes one or more steps of a time-sensitive information question-answering method based on retrieval enhancement and a large language model described in the first embodiment above.

[0127] In some embodiments, a method for timely information question and answer based on retrieval enhancement and a large language model may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 380. In some embodiments, part or all of the computer program may be loaded into or / and installed on an electronic device via ROM 320 and / or a communication unit 390. When the computer program is loaded into RAM 330 and executed by processor 310, one or more steps of a method for timely information question and answer based on retrieval enhancement and a large language model described in the first embodiment above may be executed. Alternatively, in other embodiments, processor 310 may be configured to execute a method for timely information question and answer based on retrieval enhancement and a large language model by any other appropriate means (e.g., by means of firmware).

[0128] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, or / and combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed or / and interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0129] The computer programs for implementing the methods of the embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart or / and the block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0130] In the context of an embodiment of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0131] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0132] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0133] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0134] The above is a detailed introduction to a method and system for timely information question and answer based on retrieval enhancement and a large language model disclosed in the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A time-sensitive information question answering method based on retrieval enhancement and large language model, characterized in that: It includes: Obtaining a target document, performing text parsing on the target document, obtaining parsed text data, and uploading the text data to a database; Using the content of the text data as a reference content context, and using a large language model to generate a training data set; Using the training data set to train the Embedding model; Input the text data into the trained Embedding model to obtain the feature code corresponding to each text data, upload the feature code to the database, and associate the feature code with the corresponding text data; Receive user questions and input the user questions into the trained Embedding model to obtain a user question vector, perform a similarity comparison between the feature code and the user question vector, and select text data corresponding to the N feature codes with the highest similarity as target data, where N≥1; The target data is used as context reference material and input into a large language model together with the user question, so that the large language model outputs a target answer.

2. The time-sensitive information question answering method based on retrieval enhancement and large language model according to claim 1 is characterized in that: The content of the text data is used as a reference content context, and a training data set is generated using a large language model, including: Input the content of the text data into a large language model context list as a reference article for the large language model; A prompt instruction is set so that the large language model performs the task of generating the training data set, and the prompt instruction includes a query field, a positively related knowledge field, and a negatively related knowledge field.

3. The time-sensitive information question answering method based on retrieval enhancement and large language model according to claim 2 is characterized in that: The Embedding model is trained using the training data set, including: The training data set is input into the Embedding model for comparative learning training, which includes: The InfoNCE function is used as the contrastive learning loss function, and its formula is: Wherein, Ls(.) represents the loss function, q represents the vector after the question or knowledge in the inquiry field in the training data set is encoded, p* represents the vector after the positively correlated content in the positively correlated knowledge field in the training data set is encoded, and p′ represents the vector after the negatively correlated content in the negatively correlated knowledge field in the training data set is encoded; τ is the temperature coefficient, and its size is set according to needs; s(q,p*) represents the similarity between q and p*, and s(q,p) represents the similarity between q and p; By minimizing the loss function, the parameters of the Embedding model are fine-tuned using the BGE-M3 pre-trained model as the fine-tuning target model to complete the comparative learning training of the Embedding model.

4. The time-sensitive information question answering method based on retrieval enhancement and large language model according to claim 1 is characterized in that: Inputting the text data into the trained Embedding model to obtain feature codes corresponding to each text data, uploading the feature codes to the database, and associating the feature codes with the corresponding text data, including: Input the text data into the trained Embedding model for feature encoding to obtain feature encoding corresponding to each text data, wherein the feature encoding is a 1×1024-dimensional feature vector; A feature vector field is added to the database, and the feature code is stored in the feature vector field corresponding to each text data.

5. The time-sensitive information question answering method based on retrieval enhancement and large language model according to claim 4 is characterized in that: Inputting the user question into the trained Embedding model to obtain the user question vector, and performing a similarity comparison between the feature code and the user question vector, and selecting text data corresponding to the N feature codes with the highest similarity as target data, including: Input the user question into the trained Embedding model to vectorize the user question to obtain a user question vector, which is also a 1×1024-dimensional feature vector; Compare the user question vector with each feature code for similarity, and sort them from large to small according to the similarity value; The text data corresponding to the top N feature codes are selected as the target data.

6. The time-sensitive information question answering method based on retrieval enhancement and large language model according to any one of claims 1 to 5, characterized in that: The database is an Elasticsearch database.

7. The time-sensitive information question answering method based on retrieval enhancement and large language model according to any one of claims 1 to 5, characterized in that: The similarity adopts cosine similarity, or / and, and the large language model is any one of GPT-3, GPT-4, PaLM, LLaMA and Qwen-max.

8. A time-sensitive information question answering system based on retrieval enhancement and large language model, characterized in that: It includes: A parsing unit, used to obtain a target document, perform text parsing on the target document, obtain parsed text data, and upload the text data to a database; A generating unit, configured to use the content of the text data as a reference content context and generate a training data set using a large language model; A training unit, used to train the Embedding model using the training data set; An encoding unit, used for inputting the text data into the trained Embedding model, obtaining a feature code corresponding to each text data, uploading the feature code to the database, and associating the feature code with the corresponding text data; A comparison unit is used to receive user questions, input the user questions into the trained Embedding model, obtain a user question vector, compare the feature code with the user question vector for similarity, and select text data corresponding to N feature codes with the highest similarity as target data, where N≥1; The output unit is used to input the target data as context reference material together with the user question into the large language model so that the large language model outputs a target answer.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein when the processor executes the computer program, the steps of the time-sensitive information question-answering method based on retrieval enhancement and a large language model are implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: It stores a computer program, wherein the computer program enables the computer to execute the steps of the time-sensitive information question-answering method based on retrieval enhancement and a large language model as described in any one of claims 1 to 7.

Images