Method for improving LLM question answering accuracy based on target weight

Abstract

The invention relates to the technical field of computers, in particular to a method for improving LLM question answering accuracy based on target weight, which comprises the following steps: S1, matching an explanation target, S2, constructing a target knowledge base and generating knowledge base weight: defining the type of the knowledge base according to a target user group and a demand, establishing a database, and according to matched user ID (Identity), establishing a knowledge base weight; generating a group of knowledge base weights; s3, context retrieval: retrieving a context related to a user question through question word embedding and knowledge vector library word embedding slices according to the requirement of the user; s4, weighting context retrieval according to the weight and re-sequencing: weighting the context retrieved in the step S3 according to the knowledge base weight in the step S2, and re-sequencing the retrieved context to obtain a target knowledge context; and S5, generating a question result: taking the target knowledge context reordered in the step S4 and the user question as the input of LLM, and generating a question answer according to a user prompt.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method for improving the accuracy of LLM question answers based on target weights. Background Art

[0002] With its powerful text generation and comprehension capabilities, LLM has shown great potential in multiple application scenarios such as question-answering systems, dialogue generation, and text creation. However, in actual applications, digital human question and answering of industrial manufacturing parts, digital human question and answering of agricultural crops, digital human explanations and question and answering of cultural museums and tourist attractions, and digital human explanations and question and answering of shopping guides are not particularly accurate and comprehensive.

[0003] In the prior art, LLM mainly relies on the knowledge and patterns learned during its training process when answering questions. However, due to the diversity and complexity of training data, LLM often has difficulty accurately capturing user intent when dealing with specific fields or complex problems, resulting in a deviation between the generated answers and the actual needs. In addition, when dealing with questions containing multiple related contexts, LLM may generate inconsistent or redundant answers due to the lack of an effective context management mechanism.

[0004] Therefore, the present application develops a method for improving the accuracy of answering LLM questions based on target weights to solve the problems existing in the prior art. Summary of the invention

[0005] The purpose of the present invention is to provide a method for improving the accuracy of LLM question answers based on target weights, so as to solve the problem in the prior art that when LLM answers user questions or conducts Q&A with customers, its answers are inaccurate.

[0006] The technical solution of the present invention is: a method for improving the accuracy of LLM question answers based on target weights, comprising the following steps:

[0007] S1: Matching explanation target: Matching the user sensor and the target sensor so that the target sensor receives the signal of the user sensor;

[0008] S2: Build the target knowledge base and generate knowledge base weights: Define the type of knowledge base according to the target user group and needs, and build a database. Generate a set of knowledge base weights based on the matching user IDs;

[0009] S3: Context retrieval: According to the user's needs, the context related to the user's question is retrieved through the question word embedding and the knowledge vector library word embedding slices;

[0010] S4: weighting and reordering the context retrieval by weight: weighting the context retrieved in step S3 according to the knowledge base weight in step S2, and reordering the retrieved context to obtain the target knowledge context;

[0011] S5: Generate question results: Take the target knowledge context and user questions reordered in step S4 as inputs to the LLM, and generate answers to the questions based on user prompts.

[0012] Preferably, generating the knowledge base weight in step S2 includes the following sub-steps:

[0013] S2.1: Clarify the weight assessment criteria;

[0014] S2.2: Method for calculating selection weights;

[0015] S2.3: Assign weights based on the importance and relevance of the content.

[0016] Preferably, in step S2, a knowledge relevance vector library is constructed according to the database, as shown below:

[0017]

[0018] x i ,x j is a single sample in the knowledge base;

[0019] n is the total number of knowledge base samples;

[0020] x ij For x i With x j of relevance.

[0021] Preferably, step S4 includes the following sub-steps:

[0022] S4.1: Read the input context;

[0023] S4.2: Generate multiple candidate results, each representing a different possible output;

[0024] S4.3: Extract the features of relevance and confidence in the candidate results;

[0025] S4.4: Assign a weight to each candidate result based on the relevance and confidence of the candidate result and the consistency of the context according to a preset rule or model;

[0026] S4.5: Weight the candidate results using the assigned weights, calculate the weighted scores, and sort the candidate results according to the weighted scores, select the result with the highest score as the final output, and obtain the re-sorted target knowledge context.

[0027] Preferably, the reordering in step S4 is performed in the following manner:

[0028] Integrate the knowledge retrieval similarity in step S3 with the knowledge relevance vector in step S2. When the matching target ID is m,

[0029]

[0030] simSo i Retrieve similarity for original knowledge;

[0031] simS wi Retrieve similarity for the integrated knowledge;

[0032] f pos (x) = 2 × sigmoid (x) - 1

[0033]

[0034] score_max = 1;

[0035] score_min = 0;

[0036] alpha=[0,10].

[0037] Compared with the prior art, the advantages of the present invention are:

[0038] (1) By weighting and re-ranking the contextual results, LLM can prioritize the most relevant and important information to the query, making the generated answers more accurate;

[0039] (2) Through weighting and re-ranking, LLM can eliminate irrelevant or redundant information and focus on features relevant to the question, thereby improving the accuracy of the answer;

[0040] (3) By assigning weights to the target knowledge base, LLM can prioritize the content that is most relevant to the user's question, thereby generating more accurate answers;

[0041] (4) The weight ranking can be adjusted according to user needs or historical issues to provide responses that better meet user needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0043] Figure 1 The present invention is a flowchart of a method for improving the accuracy of answering LLM questions based on target weights. DETAILED DESCRIPTION

[0044] The present invention is further described in detail below in conjunction with specific embodiments:

[0045] like Figure 1 As shown, a method for improving the accuracy of LLM question answering based on target weights includes the following steps:

[0046] S1: Matching explanation target: Matching the user sensor and the target sensor so that the target sensor receives the signal of the user sensor, wherein the user sensor is a device such as Bluetooth, camera, QR code, etc. that can be connected to the target sensor;

[0047] First, clarify the sensor application scenarios, such as digital human Q&A for agricultural crops, digital human explanations and Q&A at cultural museums and tourist attractions, and digital human explanations and Q&A for shopping guides, etc. Determine the performance requirements and functional requirements of the sensors through application scenarios, ensure that the user sensors and target sensors are consistent in terms of measurement targets and parameters, so as to achieve seamless connection and fusion of data. According to the type of sensor and application scenario, select appropriate matching algorithms, such as Kalman filtering and particle filtering, to improve the accuracy and stability of data fusion. Then, realize the synchronous collection of sensor data through timestamps, event triggers, etc., to ensure the real-time and relevance of the data, and use matching algorithms to fuse data from different sensors to generate more comprehensive and accurate target parameters.

[0048] S2: Build the target knowledge base and generate knowledge base weights: Define the type of knowledge base according to the target user group and needs, and build a database. Generate a set of knowledge base weights based on the matching user IDs;

[0049] First, clearly define the goals and specific requirements for building a knowledge base. Based on the goals and requirements, collect and organize relevant knowledge resources, design the knowledge base structure, including creating directories, folders, and subfolders to organize and classify knowledge resources, and write knowledge base content based on the knowledge base structure, including creating new knowledge materials, writing detailed instructions and explanations, etc. According to the functions and characteristics of different tools, select a knowledge base tool that meets the needs and is easy to use and maintain. Import the collected knowledge resources into the knowledge base tool one by one, and add appropriate tags and keywords to each resource to facilitate users to find the required knowledge more quickly and improve the efficiency of using the knowledge base;

[0050] After completing the construction of the target knowledge base, determine the weight factors based on the importance, timeliness, frequency of use, accuracy and other factors of the knowledge. By comprehensively considering these factors, assign corresponding weight values ​​to each knowledge point in the knowledge base. According to the determined weight factors, formulate specific weight allocation rules. According to the formulated weight allocation rules, assign corresponding weight values ​​to each knowledge point in the knowledge base.

[0051] Generating the knowledge base weight in step S2 includes the following sub-steps:

[0052] S2.1: Clarify the weight evaluation criteria, judge the authenticity and reliability of knowledge items by evaluating the degree of direct relevance between knowledge items and knowledge base topics or user needs, and consider the update frequency and currency of knowledge items. Evaluate based on the influence of knowledge items in specific fields or scenarios, and analyze the frequency of users accessing and using knowledge items to determine the weight ratio of different knowledge items;

[0053] S2.2: Select the weight calculation method. Different calculation methods can be selected according to actual needs as shown below:

[0054] Expert scoring method: Invite domain experts to conduct subjective evaluation of knowledge items and give corresponding weight values;

[0055] Statistical analysis method: weights are calculated by statistical analysis based on historical data and usage records;

[0056] AHP: A comprehensive quantitative method that decomposes complex decision factors into different components and determines the relative importance of the factors in the hierarchy by comparing them two by two.

[0057] Weighted average method: According to the preset weight distribution ratio, multiple evaluation indicators of knowledge items are weighted averaged.

[0058] S2.3: Assign weights based on the importance and relevance of the content. Collect data on relevant knowledge items based on the evaluation criteria, including information on indicators such as relevance, accuracy, timeliness, importance and frequency of use. Clean, organize and analyze the collected data to ensure the accuracy and consistency of the data. Select an appropriate weight calculation method to calculate the weight of each knowledge item, and then complete the weight assignment of each knowledge item.

[0059] The knowledge relevance vector library is constructed based on the constructed database as follows:

[0060]

[0061] x i ,x j is a single sample in the knowledge base;

[0062] n is the total number of knowledge base samples;

[0063] x ij For x i With x j of relevance.

[0064] Two different sample data in the database are associated, and context retrieval is quickly implemented using the vector library. The vector library converts the knowledge in the database into vector form, allowing LLM to understand and process the information more easily. The vector representation makes it easier for LLM to capture the semantics and contextual information of text or data, thereby improving LLM's ability to understand complex language phenomena. The vector library of correlation supports LLM to perform fast similarity search, allowing LLM to quickly find the most relevant knowledge items to the current query in a large amount of data, which helps LLM to generate text, answer questions or make inferences based on a broader and more accurate knowledge base. By utilizing the knowledge correlation in the vector library, LLM can generate more coherent, accurate and context-rich text, which helps to improve the use of LLM in different fields such as dialogue systems, content creation, and question and answer.

[0065] S3: Contextual retrieval: Based on user needs, contexts related to user questions are retrieved through question word embedding and knowledge vector library word embedding slices. Contexts related to questions are retrieved through word embedding slices based on the similarity with the question. This is used to improve the accuracy and relevance of the model when understanding and generating text. By combining contextual information in external knowledge bases or internal databases, it can more comprehensively understand user questions and generate more accurate and useful answers.

[0066] S4: Weight and reorder the context retrieval by weight: Weight the context retrieved in step S3 according to the knowledge base weight in step S2, and reorder the retrieved context to obtain the target knowledge context. The weighting can evaluate the importance of the retrieval results based on different factors, such as the matching degree of keywords, the source authority of the document, the timeliness of the document, etc. By assigning different weights to these factors, the relevance of the retrieval results to the user query can be more accurately measured, thereby improving the quality of the retrieval results. The reordering is based on the weighted results, and the retrieval results with higher weights are placed in front. When browsing the retrieval results, users can find the most relevant and valuable information to their needs more quickly, according to user needs.

[0067] Step S4 includes the following sub-steps:

[0068] S4.1: Read the input context;

[0069] S4.2: Generate multiple candidate results, each representing a different possible output, perform data processing and feature extraction on the contextual data, and generate multiple candidate results to provide more choices and increase the chance of finding the most accurate result. Although a single candidate result is not precise enough, the set of multiple candidate results is likely to contain options that are highly matched with user needs, which helps to reduce misjudgments and missed judgments and improve the overall accuracy of the system. In addition, user needs are often diverse and personalized, and generating multiple candidate results can meet the different needs of different users.

[0070] S4.3: Extracting the features of relevance and confidence in candidate results helps improve the accuracy and reliability of the results;

[0071] S4.4: Assign a weight to each candidate result based on the relevance and confidence of the candidate result and the consistency of the context according to a preset rule or model;

[0072] S4.5: Weight the candidate results using the assigned weights, calculate the weighted scores, and sort the candidate results according to the weighted scores, select the result with the highest score as the final output, and obtain the re-sorted target knowledge context.

[0073] The reordering in step S4 is performed in the following manner:

[0074] Integrate the knowledge retrieval similarity in step S3 with the knowledge relevance vector in step S2. When the matching target ID is m,

[0075]

[0076] simSo i Retrieve similarity for original knowledge;

[0077] simS wi Retrieve similarity for the integrated knowledge;

[0078] f pos (x) = 2 × sigmoid (x) - 1

[0079]

[0080] score_max = 1;

[0081] score_min = 0;

[0082] alpha=[0,10].

[0083] By calculating the similarity between the query vector and each vector in the knowledge base, the similarity between the two is quantitatively evaluated. Based on the similarity calculation, the vectors related to the query are further integrated to form a more focused and relevant result set. By considering the intrinsic connections and weights between different vectors, the results finally presented to the user are more accurate and meet the needs. During the integration process, the system will adjust the weights according to the importance of different features and the specific needs of the user. Through reasonable weight adjustment, the system can more accurately reflect the user's real needs and preferences and achieve more accurate result feedback.

[0084] S5: Generate question results: Take the target knowledge context and user questions reordered in step S4 as inputs to the LLM, and generate answers to the questions based on user prompts.

[0085] The implementation principle of this embodiment:

[0086] After completing the construction of the target knowledge base, single samples in the knowledge base are annotated manually, and the relevance annotation of single samples is completed to form a knowledge relevance vector library. When the user uses the sensor, the user's sensor is matched with the target sensor. When the user asks a question, the system converts the user's question into text through voice recognition, and retrieves the context related to the question through question word embedding and knowledge vector library word embedding slices. The retrieved context is weighted and reordered according to the target knowledge base weight. According to the knowledge relevance vector library, knowledge base samples that are connected between different samples are established, and the vector library is used to realize rapid retrieval of context. The knowledge retrieval similarity and knowledge relevance vector are integrated to obtain the target knowledge context with knowledge retrieval similarity that better matches the user's question, and then through LLM, more accurate answers are generated according to user questions, and responses that are more in line with the user are provided.

[0087] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with this technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the scope of protection of the present invention. For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is limited by the attached claims rather than the above description, and it is intended to include all changes within the meaning and scope of the equivalent elements of the claims in the present invention.

Claims

1. A method for improving the accuracy of LLM question answering based on target weight, characterized in that: The following steps are involved: S1: Matching explanation target: Matching the user sensor and the target sensor so that the target sensor receives the signal of the user sensor; S2: Build the target knowledge base and generate knowledge base weights: Define the type of knowledge base according to the target user group and needs, and build a database. Generate a set of knowledge base weights based on the matching user IDs; S3: Context retrieval: According to the user's needs, the context related to the user's question is retrieved through the question word embedding and the knowledge vector library word embedding slices; S4: weighting and reordering the context retrieval by weight: weighting the context retrieved in step S3 according to the knowledge base weight in step S2, and reordering the retrieved context to obtain the target knowledge context; S5: Generate question results: Take the target knowledge context and user questions reordered in step S4 as inputs to the LLM, and generate answers to the questions based on user prompts.

2. A method for improving the accuracy of LLM question answers based on target weights according to claim 1, characterized in that: Generating the knowledge base weight in step S2 includes the following sub-steps: S2.1: Clarify the weight assessment criteria; S2.2: Method for calculating selection weights; S2.3: Assign weights based on the importance and relevance of the content.

3. A method for improving the accuracy of answering LLM questions based on target weights according to claim 2, characterized in that: In step S2, a knowledge relevance vector library is constructed according to the database, as shown below: x i ,x j is a single sample in the knowledge base; n is the total number of knowledge base samples; x ij For x i With x j of relevance.

4. The method for improving the accuracy of answering LLM questions based on target weight according to claim 1, characterized in that: Step S4 includes the following sub-steps: S4.1: Read the input context; S4.2: Generate multiple candidate results, each representing a different possible output; S4.3: Extract the features of relevance and confidence in the candidate results; S4.4: Assign a weight to each candidate result based on the relevance and confidence of the candidate result and the consistency of the context according to a preset rule or model; S4.5: Weight the candidate results using the assigned weights, calculate the weighted scores, and sort the candidate results according to the weighted scores, select the result with the highest score as the final output, and obtain the re-sorted target knowledge context.

5. The method for improving the accuracy of answering LLM questions based on target weight according to claim 3, characterized in that: The reordering in step S4 is performed as follows: Integrate the knowledge retrieval similarity in step S3 with the knowledge relevance vector in step S2. When the matching target ID is m, simSo i Retrieve similarity for original knowledge; simS wi Retrieve similarity for the integrated knowledge; f pos (x)=2×sigmoid(x)-1 score_max = 1; score_min = 0; alpha=[0,10].

Images