Caching method for large language model questions and answers

Abstract

The invention discloses a caching method for questions and answers of a large language model. The method comprises the following steps: obtaining multiple rounds of dialogue results of the large model according to questions input by a user; inputting multiple rounds of dialogue results into the large model for identification, removing unsatisfied answers of the user and returning correct questions and answers; inputting the obtained questions into a large language model for keyword extraction, and obtaining keywords with the same semantics as the questions; taking the keyword as a key and the answer as a value, writing the keyword and the answer into a distributed redis cache, and setting the validity period and the writing time of the cache; obtaining a new round of questions of the user and judging whether cache information can be searched from the cache or not; if yes, the cached answer is returned to serve as the reply of the current user question; otherwise, if not found, the large model generates a new answer according to the business process. The response speed is increased, the load of the server is reduced, external interface calling is reduced, and the use cost of the external interface is reduced.

Description

Technical Field

[0001] The present invention relates to the application field of large models of automobile intelligent cockpits, and in particular to a caching method for large language model question and answer. Background Art

[0002] AI big models are the best way to open the smart cockpit. By introducing big models, the task processing capabilities of the smart cockpit can be greatly improved, and deep interaction capabilities can be achieved. The smart cockpit big model can provide a higher quality interactive experience through more accurate and smooth voice recognition functions, as well as richer knowledge reserves and semantic understanding capabilities. At the same time, the big model can comprehensively process multiple types of data such as voice, vision, and touch, provide deep multimodal interaction, and provide a more comfortable driving experience. As one of the core areas of technological innovation and competition in smart cars, the computing processing power, generation, and multimodal interaction capabilities behind the big model can significantly improve the intelligence level of the smart cockpit.

[0003] In the application scenario of the smart cockpit big model, since the smart cockpit big model dialogue has high requirements for the efficiency of service return, the caching mechanism can greatly reduce the waiting time for service return; since a large amount of model inference calculation is required in the smart cockpit big model dialogue, the caching mechanism can greatly reduce the server load; since the smart cockpit big model dialogue requires multiple calls to the external source interface, such as music, weather, navigation, travel, etc., the caching mechanism can reduce the cost of using the external source interface.

[0004] Chinese patent publication number CN202410118530.X discloses an online intelligent question-answering method and device based on instruction fine-tuning and retrieval enhancement generation. The invention discloses an online intelligent question-answering method and device based on instruction fine-tuning and retrieval enhancement generation, including: obtaining user online question information, refining user online question information based on a large language model, and expanding the refined questions with similar questions; vectorizing user online questions and expanded similar questions based on a trained semantic vector model to obtain online question text vectors; based on a two-way retrieval recall strategy, the online question text vector is retrieved and compared with the data in the preset knowledge base to obtain similar data, and then multiple target text blocks corresponding to the similar data are recalled from the corresponding cache area based on the similar data and the TopK strategy; the recalled text blocks are reordered based on the reordering strategy to obtain reordered text blocks; based on a large language model and prompt words, the user's online questions, recalled text blocks and reordered text blocks are polished and summarized to obtain online answer results. The accuracy of intelligent question-answering is improved.

[0005] Currently, most large-model question-answering optimization methods are based on improving the accuracy of question-answering, but are unable to increase response speed and reduce server load pressure. Summary of the invention

[0006] The purpose of the present invention is to provide a caching method for large language model questions and answers. By implementing a question and answer caching mechanism, answers to frequently accessed identical or very similar questions can be stored, so that subsequent requests for identical or similar questions can quickly obtain results directly from the cache, thereby greatly improving the response speed and reducing the server load, while reducing external interface calls and reducing the cost of using external interfaces.

[0007] The technical solution adopted by the present invention is: A caching method for large language model question answering includes the following steps: Step 1: Multi-round dialogue with a large model: In a common business process, the multi-round dialogue results of the large model are obtained based on the questions input by the user; Step 2: Obtain the correct questions and answers through the big model: Input the results of multiple rounds of conversations into the big model for recognition, remove the answers that the user is not satisfied with, and return the correct questions and answers, which include questions and answers; Step 3: Extract question keywords through the big model: Input the obtained question into the big language model for keyword extraction, remove invalid text, and obtain keywords with the same semantics as the question; Step 4: Cache Q&A: Use the keyword as the key and the answer as the value to write into the distributed redis cache, and set the cache validity period and write time; Step 5: Use cache in big model dialogue: Get a new round of questions from the user and determine whether the cached information can be found in the cache; if so, return the cached answer as the answer to the current user's question; otherwise, if not found, the big model generates a new answer according to the business process.

[0008] Furthermore, in step 2, the role of the large language model is set as an "information recognition expert", which uses information analysis, semantic recognition, and logical reasoning capabilities to identify the results of multiple rounds of dialogue.

[0009] Furthermore, step 2 specifically includes the following steps: Step 2-1, read and understand the input multi-round dialogue results; Step 2-2, through semantic analysis, based on the user's feedback on the big model's answer, identify whether the big model's answer is an answer that the user is satisfied with; Step 2-3, based on the judgment result, remove the answers that the user is not satisfied with and retain the answers that the user is satisfied with; Step 2-4, return to the question and answer, which includes the user's questions and the answers that the user is satisfied with.

[0010] Furthermore, in step 3, the role of the large language model is set as an "information screening expert", and keyword extraction is performed using information analysis, key point recognition, and logical reasoning capabilities.

[0011] Furthermore, step 3 specifically includes the following steps: Step 3-1, read and understand the user's question; Step 3-2: Determine whether the preconditions in the question are of substantial help to the answer; Step 3-3, based on the judgment result, remove the information that is not of substantial help to the answer, and retain the information that is of substantial help to the answer; In step 3-4, according to the semantics of the question, the retained information is further processed by natural language understanding, intent recognition, synonym and near-synonym standardization, standardized output template matching, sentence normalization, semantic vector standardization, rule engine, etc., to further extract and convert effective and standardized keywords.

[0012] Furthermore, in step 5, keywords are extracted from the user's question in the large model dialogue according to the method of step 3, and the cache information is searched from the cache using the keywords.

[0013] Furthermore, it also includes the following steps: Step 6: Cache cleaning: To solve the problem of outdated cache information, redis automatically cleans up the expired cache according to the set validity period.

[0014] Furthermore, in step 6, an LRU (least recently used) replacement strategy is adopted to automatically clean up data that has not been accessed for a long time.

[0015] The present invention adopts the above technical solution, by caching the answers, for the same or similar questions repeatedly asked by multiple users, only one business process calculation is needed, which can greatly improve the response speed and reduce the server load, while reducing the cost of using external interfaces; using distributed cache technology such as Redis tools to help manage and synchronize data caches across multiple nodes. The present invention does not directly use the question as the key for caching, but uses the keyword of the question as the key for caching. For similar questions, the same keyword is generated, avoiding repeated recording of multiple caches for similar questions; by implementing a regular refresh strategy, an event-driven update mechanism, and a regular invalidation mechanism, the problem of outdated information in the cache is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments; Figure 1 The figure is a flow chart of a large language model question and answer caching method of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0018] like Figure 1 As shown, the present invention discloses a caching method for large language model question and answer, which comprises the following steps: Step 1: Multi-round dialogue with a large model: In a common business process, the multi-round dialogue results of the large model are obtained based on the questions input by the user; Step 2: Obtain the correct questions and answers through the big model: Input the results of multiple rounds of conversations into the big model for recognition, remove the answers that the user is not satisfied with, and return the correct questions and answers, which include questions and answers; Furthermore, in step 2, the role of the large language model is set as an "information identification expert", and the results of multiple rounds of dialogue are identified using information analysis, semantic recognition, and logical reasoning capabilities. The role of the large language model is set as an "information identification expert", and its information analysis, semantic recognition, and logical reasoning capabilities are used to input the results of multiple rounds of dialogue in step 1 into the large model for identification, and the large model is used to remove answers that the user is not satisfied with and return the correct questions and answers; the specific process is as follows: Step 2-1, read and understand the input multi-round dialogue results; Step 2-2, through semantic analysis, based on the user's feedback on the big model's answer, identify whether the big model's answer is an answer that the user is satisfied with; Step 2-3, based on the judgment result, remove the answers that the user is not satisfied with and retain the answers that the user is satisfied with; Step 2-4, return to the question and answer, which includes the user's questions and the answers that the user is satisfied with.

[0019] Step 3: Extract question keywords through the big model: Input the obtained question into the big language model for keyword extraction, remove invalid text, and obtain keywords with the same semantics as the question; further, in step 3, the role of the big language model is set as an "information screening expert", and keyword extraction is performed using information analysis, key point recognition, and logical reasoning capabilities. The specific process is as follows: Step 3-1, read and understand the user's question; Step 3-2: Determine whether the preconditions in the question are of substantial help to the answer; Step 3-3, based on the judgment result, remove the information that is not of substantial help to the answer, and retain the information that is of substantial help to the answer; In step 3-4, according to the semantics of the question, the retained information is further processed by natural language understanding, intent recognition, synonym and near-synonym standardization, standardized output template matching, sentence normalization, semantic vector standardization, rule engine, etc., to further extract and convert effective and standardized keywords.

[0020] 5) Return the refined keywords; Step 4: Cache Q&A: Use the keyword as the key and the answer as the value to write into the distributed redis cache, and set the cache validity period and write time; Step 5: Use cache in big model dialogue: Get the user's new round of questions and determine whether the cached information can be found in the cache; if so, return the cached answer as the answer to the current user's question; otherwise, if it is not found, the big model generates a new answer according to the business process. Furthermore, in step 5, the keywords are extracted from the user's questions in the big model dialogue according to step 3, and the keywords are used to find the cached information from the cache.

[0021] Furthermore, it also includes the following steps: Step 6: Cache cleaning: To solve the problem of outdated cache information, redis automatically cleans up the expired cache according to the set validity period.

[0022] Specifically, in step 6, an LRU (least recently used) replacement strategy is adopted to automatically clean up data that has not been accessed for a long time.

[0023] The present invention adopts standardized processing of synonyms and near-anonymous words, processing of different expressions with the same meaning, standardized output template matching, sentence normalization processing, rule engine, etc.; the technology of obtaining correct questions and answers through the big model sets the role of the big language model as an "information recognition expert", and uses its information analysis, semantic recognition, and logical reasoning capabilities to input the results of multiple rounds of dialogue into the big model for recognition, and uses the big model to remove answers that the user is not satisfied with, and return the correct questions and answers.

[0024] The present invention adopts the above technical solution, by caching the answers, for the same or similar questions repeatedly asked by multiple users, only one business process calculation is needed, which can greatly improve the response speed and reduce the server load, while reducing the cost of using external interfaces; using distributed cache technology such as Redis tools to help manage and synchronize data caches across multiple nodes. The present invention does not directly use the question as the key for caching, but uses the keyword of the question as the key for caching. For similar questions, the same keyword is generated, avoiding repeated recording of multiple caches for similar questions; by implementing a regular refresh strategy, an event-driven update mechanism, and a regular invalidation mechanism, the problem of outdated information in the cache is solved.

[0025] Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of the present application is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians of the art without making creative work are within the scope of protection of the present application.

Claims

1. A caching method for large language model question answering, characterized by: It includes the following steps: Step 1: Multi-round dialogue with a large model: In a common business process, the multi-round dialogue results of the large model are obtained based on the questions input by the user; Step 2: Obtain the correct questions and answers through the big model: Input the results of multiple rounds of conversations into the big model for recognition, remove the answers that the user is not satisfied with, and return the correct questions and answers, which include questions and answers; Step 3: Extract question keywords through the big model: Input the obtained question into the big language model for keyword extraction, remove invalid text, and obtain keywords with the same semantics as the question; Step 4: Cache Q&A: Use the keyword as the key and the answer as the value to write into the distributed redis cache, and set the cache validity period and write time; Step 5: Use cache in large model dialogue: Get a new round of questions from the user and determine whether the cached information can be found in the cache; If so, return the cached answer as the answer to the current user's question; Otherwise, if not found, the big model generates a new answer according to the business process.

2. The method for caching a large language model question and answer according to claim 1, characterized in that: In step 2, the role of the large language model is set as an "information recognition expert", which uses information analysis, semantic recognition, and logical reasoning capabilities to identify the results of multiple rounds of dialogue.

3. The method for caching a large language model question and answer according to claim 2, characterized in that: Step 2 specifically includes the following steps: Step 2-1, read and understand the input multi-round dialogue results; Step 2-2, through semantic analysis, based on the user's feedback on the big model's answer, identify whether the big model's answer is an answer that the user is satisfied with; Step 2-3, based on the judgment result, remove the answers that the user is not satisfied with and retain the answers that the user is satisfied with; Step 2-4, return to the question and answer, which includes the user's questions and the answers that the user is satisfied with.

4. The method for caching a large language model question and answer according to claim 1, characterized in that: In step 3, the role of the large language model is set as "information screening expert", and keyword extraction is performed using information analysis, key point recognition, and logical reasoning capabilities.

5. The method for caching a large language model question and answer according to claim 4, characterized in that: Step 3 specifically includes the following steps: Step 3-1, read and understand the user's question; Step 3-2: Determine whether the preconditions in the question are of substantial help to the answer; Step 3-3, based on the judgment result, remove the information that is not of substantial help to the answer, and retain the information that is of substantial help to the answer; Step 3-4: Based on the semantics of the question, the retained information is further extracted and converted into effective and standardized keywords.

6. The method for caching a large language model question and answer according to claim 5, characterized in that: Further extraction of retained information includes natural language understanding, intent recognition, synonym and near-synonym standardization, standardized output template matching, sentence normalization, semantic vector standardization, and rule engine processing.

7. The method for caching a large language model question and answer according to claim 1, characterized in that: In step 5, keywords are extracted from the user's question in the large model dialogue according to the method of step 3, and the keywords are used to search the cache information from the cache.

8. The method for caching a large language model question and answer according to claim 1, characterized in that: It also includes the following steps: Step 6: Cache cleaning: Redis automatically cleans up the expired cache according to the set validity period.

9. The method for caching a large language model question and answer according to claim 8, characterized in that: In step 6, the least recently used (LRU) replacement strategy is used to automatically clean up data that has not been accessed for a long time.

Images