Question and answer optimization method and device, equipment and storage medium

Abstract

The invention discloses a question and answer optimization method and device, equipment and a storage medium, and relates to the technical field of natural language processing, and the method comprises the steps: generating an initial answer based on a user question, carrying out the multi-dimensional scoring of the initial answer, determining a target quality scoring threshold value through a machine learning algorithm and a user feedback mechanism, and obtaining a target quality scoring result; if the scoring result is smaller than the target quality scoring threshold value, determining a target retrieval enhancement generation strategy based on the scoring result and the question type of the user question; the target retrieval enhancement generation strategy comprises vector retrieval, knowledge graph query, multi-hop reasoning and information retrieval; and optimizing the initial answer by using a target retrieval enhancement generation strategy, performing multi-dimensional scoring on the optimized answer to obtain a new scoring result, and skipping to the step of judging whether the scoring result is smaller than the target quality scoring threshold or not until the scoring result is not smaller than the target quality scoring threshold or meets an iteration termination condition. And obtaining a target answer. The defects of an existing question answering system for complex questions are overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and particularly relates to a question - answering optimization method, device, equipment and storage medium. Background Art

[0002] Most existing question - answering systems rely on pre - constructed knowledge bases, rule engines or specific retrieval mechanisms to answer questions raised by users. Such systems perform well in certain application scenarios, especially when the questions focus on specific fields, known information or common question - answering patterns. However, when faced with complex questions, cross - domain questions or dynamic scenarios with rapid information updates, the limitations of traditional question - answering systems become apparent. The answers generated by them often fail to accurately match the user's needs, and may even give outdated or inaccurate answers due to the limitations of the knowledge base content. In addition, such systems usually have weak quality control after answer generation and lack an effective self - optimization mechanism.

[0003] As can be seen from the above, how to address the deficiencies of existing question - answering systems when facing complex questions and pay attention to user needs is an urgent problem to be solved currently. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a question - answering optimization method, device, equipment and storage medium, which can address the deficiencies of existing question - answering systems when facing complex questions and pay attention to user needs. The specific solutions are as follows:

[0005] In a first aspect, the present application provides a question - answering optimization method, including:

[0006] Generating an initial answer based on the user's question and using a large - language model, then performing multi - dimensional scoring on the initial answer through the current question - asking scenario to obtain a scoring result, and using a machine - learning algorithm and a user feedback mechanism to determine a target quality scoring threshold, and judging whether the scoring result is less than the target quality scoring threshold;

[0007] If the scoring result is less than the target quality scoring threshold, then analyze the scoring result, and then determine a corresponding target retrieval - enhanced generation strategy based on the analysis result and the question type corresponding to the user's question; the target retrieval - enhanced generation strategy includes vector retrieval, knowledge - graph query, multi - hop reasoning and information retrieval;

[0008] Using the target retrieval - enhanced generation strategy to optimize the initial answer to obtain an optimized answer, performing multi - dimensional scoring on the optimized answer to obtain a new scoring result, and jumping to the step of judging whether the scoring result is less than the target quality scoring threshold until the scoring result is not less than the target quality scoring threshold or meets the iteration termination condition to obtain a target answer.

[0009] Optionally, generate an initial answer based on the user's question using a large language model, then perform multi-dimensional scoring on the initial answer through the current question scenario to obtain a scoring result, and use machine learning algorithms and user feedback mechanisms to determine a target quality scoring threshold, and judge whether the scoring result is less than the target quality scoring threshold, including:

[0010] Obtain the input user question and the context information corresponding to the user question, and splice the user question and the context information to obtain an input sequence;

[0011] Input the input sequence into the large language model to generate an initial answer, and perform relevance scoring, accuracy scoring, logical scoring, and information integrity scoring on the initial answer respectively to obtain an initial scoring vector;

[0012] Adjust the scoring weights corresponding to the initial scoring vector through the current question scenario to obtain adjusted scoring weights, and determine the comprehensive score of the initial answer based on the initial scoring vector and the adjusted scoring weights;

[0013] Determine the mean and standard deviation of each dimension score corresponding to the test question-and-answer data based on the test question-and-answer data, and determine the initial quality scoring threshold through the mean and the standard deviation;

[0014] Update the initial quality scoring threshold based on machine learning algorithms and user feedback mechanisms to obtain a target quality scoring threshold, and judge whether the comprehensive score is less than the target quality scoring threshold.

[0015] Optionally, if the scoring result is less than the target quality scoring threshold, analyze the scoring result, and then determine a corresponding target retrieval enhanced generation strategy based on the analysis result and the question type corresponding to the user question, including:

[0016] If the comprehensive score is less than the target quality scoring threshold, analyze the initial answer and the comprehensive score to obtain an analysis result;

[0017] Determine a preset relevance scoring threshold based on the question type corresponding to the user question, and judge whether the relevance score in the comprehensive score is less than the preset relevance scoring threshold based on the analysis result;

[0018] If the relevance score in the comprehensive score is less than the preset relevance scoring threshold, determine that the target retrieval enhanced generation strategy is vector retrieval;

[0019] Correspondingly, optimizing the initial answer by using the target retrieval enhancement generation strategy to obtain an optimized answer includes:

[0020] Using word embedding technology to convert the user question and each document in the preset document library into word vectors respectively to obtain a question vector and a document vector;

[0021] Using the vector space model to calculate the similarity between the question vector and each document vector in the preset document library, determining the document corresponding to the document vector with the highest similarity score as the target document, and optimizing the initial answer based on the target document to obtain an optimized answer.

[0022] Optionally, if the scoring result is less than the target quality scoring threshold, analyzing the scoring result, and then determining the corresponding target retrieval enhancement generation strategy based on the analysis result and the question type corresponding to the user question, including:

[0023] If the comprehensive score is less than the target quality scoring threshold, analyzing the initial answer and the comprehensive score to obtain an analysis result;

[0024] Determining a preset accuracy scoring threshold based on the question type corresponding to the user question, and judging whether the accuracy score in the comprehensive score is less than the preset accuracy scoring threshold based on the analysis result;

[0025] If the accuracy score in the comprehensive score is less than the preset accuracy scoring threshold, determining the target retrieval enhancement generation strategy as knowledge graph query;

[0026] Correspondingly, optimizing the initial answer by using the target retrieval enhancement generation strategy to obtain an optimized answer includes:

[0027] Constructing a target query statement based on the user question and the graph query language, querying corresponding relevant information from the preset knowledge graph by using the target query statement, and screening and integrating the relevant information to obtain first target information;

[0028] Using the first target information to correct the initial answer to obtain an optimized answer.

[0029] Optionally, if the scoring result is less than the target quality scoring threshold, analyzing the scoring result, and then determining the corresponding target retrieval enhancement generation strategy based on the analysis result and the question type corresponding to the user question, including:

[0030] If the comprehensive score is less than the target quality scoring threshold, analyzing the initial answer and the comprehensive score to obtain an analysis result;

[0031] Determine a preset logical scoring threshold based on the problem type corresponding to the user problem, and judge whether the logical score in the comprehensive score is less than the preset logical scoring threshold based on the analysis result;

[0032] If the logical score in the comprehensive score is less than the preset logical scoring threshold, determine that the target retrieval enhancement generation strategy is multi-hop reasoning;

[0033] Correspondingly, the use of the target retrieval enhancement generation strategy to optimize the initial answer to obtain an optimized answer includes:

[0034] Decompose the user problem to obtain several sub-problems, and obtain second target information corresponding to the sub-problems from a preset knowledge graph based on the sub-problems;

[0035] Establish the logical relationship between the sub-problems, and identify the initial answer based on natural language processing technology and the logical relationship to obtain the logical break position;

[0036] Use the second target information to gradually reason about the user problem to obtain intermediate conclusions;

[0037] Determine the corresponding logical chain based on the intermediate conclusion and in combination with the logical relationship, and use the logical chain, the logical break position and the user problem to determine the optimized answer.

[0038] Optionally, if the scoring result is less than the target quality scoring threshold, analyze the scoring result, and then determine the corresponding target retrieval enhancement generation strategy based on the analysis result and the problem type corresponding to the user problem, including:

[0039] If the comprehensive score is less than the target quality scoring threshold, analyze the initial answer and the comprehensive score to obtain an analysis result;

[0040] Determine a preset information integrity scoring threshold based on the problem type corresponding to the user problem, and judge whether the information integrity score in the comprehensive score is less than the preset information integrity scoring threshold based on the analysis result;

[0041] If the information integrity score in the comprehensive score is less than the preset information integrity scoring threshold, determine that the target retrieval enhancement generation strategy is information retrieval;

[0042] Correspondingly, the use of the target retrieval enhancement generation strategy to optimize the initial answer to obtain an optimized answer includes:

[0043] Determine the target topic and target keywords corresponding to the target problem, and retrieve corresponding relevant records from a preset structured database based on the target topic and the target keywords;

[0044] Extract the relevant records to obtain the third target information, and use the third target information to optimize the initial answer to obtain an optimized answer.

[0045] Optionally, after obtaining a new scoring result by performing multi-dimensional scoring on the optimized answer and jumping to the step of determining whether the scoring result is less than the target quality scoring threshold, until the scoring result is not less than the target quality scoring threshold or meets the iteration termination condition to obtain the target answer, the following steps are further included:

[0046] Determine the system prediction score based on the new scoring result, and obtain the user feedback score by using the score of the user terminal for the target answer;

[0047] Determine whether the difference between the system prediction score and the user feedback score meets a preset difference condition;

[0048] If the difference between the system prediction score and the user feedback score meets the preset difference condition, determine whether the system prediction score is greater than the user feedback score;

[0049] If the system prediction score is greater than the user feedback score, reduce the respective scoring weights corresponding to the multi-dimensional scoring;

[0050] If the system prediction score is not greater than the user feedback score, increase the respective scoring weights corresponding to the multi-dimensional scoring.

[0051] In a second aspect, the present application provides a question and answer optimization device, including:

[0052] An initial answer scoring module, configured to generate an initial answer based on a user question by using a large language model, then perform multi-dimensional scoring on the initial answer through the current question scenario to obtain a scoring result, and determine a target quality scoring threshold by using a machine learning algorithm and a user feedback mechanism, and determine whether the scoring result is less than the target quality scoring threshold;

[0053] A scoring result analysis module, configured to, if the scoring result is less than the target quality scoring threshold, analyze the scoring result, and then determine a corresponding target retrieval enhancement generation strategy based on the analysis result and the question type corresponding to the user question; the target retrieval enhancement generation strategy includes vector retrieval, knowledge graph query, multi-hop reasoning, and information retrieval;

[0054] An initial answer optimization module, which is used to optimize the initial answer by using the target retrieval enhancement generation strategy to obtain an optimized answer, perform multi-dimensional scoring on the optimized answer to obtain a new scoring result, and jump to the step of determining whether the scoring result is less than the target quality scoring threshold until the scoring result is not less than the target quality scoring threshold or meets the iteration termination condition to obtain a target answer.

[0055] In a third aspect, the present application provides an electronic device, including:

[0056] A memory, which is used to store a computer program;

[0057] A processor, which is used to execute the computer program to implement the foregoing question-and-answer optimization method.

[0058] In a fourth aspect, the present application provides a computer-readable storage medium, which is used to store a computer program, wherein the computer program, when executed by a processor, implements the foregoing question-and-answer optimization method.

[0059] Based on the user's question, the present application uses a large language model to generate an initial answer, then performs multi-dimensional scoring on the initial answer through the current question scenario to obtain a scoring result, determines the target quality scoring threshold by using a machine learning algorithm and a user feedback mechanism, and determines whether the scoring result is less than the target quality scoring threshold; if the scoring result is less than the target quality scoring threshold, analyze the scoring result, and then determine the corresponding target retrieval enhancement generation strategy based on the analysis result and the question type corresponding to the user's question; the target retrieval enhancement generation strategy includes vector retrieval, knowledge graph query, multi-hop reasoning, and information retrieval; use the target retrieval enhancement generation strategy to optimize the initial answer to obtain an optimized answer, perform multi-dimensional scoring on the optimized answer to obtain a new scoring result, and jump to the step of determining whether the scoring result is less than the target quality scoring threshold until the scoring result is not less than the target quality scoring threshold or meets the iteration termination condition to obtain a target answer.

[0060] As can be seen from the above, the present application performs multi-dimensional scoring on the generated initial answer to obtain a scoring result, and determines the target quality scoring threshold based on a machine learning algorithm and a user feedback mechanism. If the scoring result is less than the target quality scoring threshold, a corresponding target retrieval enhancement generation strategy is determined based on the scoring result and the question type of the user question, so as to optimize the initial answer using the target retrieval enhancement generation strategy, and perform multi-dimensional scoring on the optimized answer again to obtain a new scoring result, and then jump to the step of determining whether the scoring result is less than the target quality scoring threshold, until the scoring result is not less than the target quality scoring threshold or meets the iteration termination condition, so as to obtain the target answer. In this way, by continuously adjusting the scoring criteria and the dynamic selection of the retrieval enhancement generation strategy through user feedback, the quality of the question and answer and the system performance are gradually optimized, so as to better meet the user's needs and improve the intelligence level of the question and answer system. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings.

[0062] Figure 1 It is a flowchart of a question and answer optimization method disclosed in the present application;

[0063] Figure 2 It is a flowchart of a specific question and answer optimization method disclosed in the present application;

[0064] Figure 3 It is a schematic structural diagram of a question and answer optimization device disclosed in the present application;

[0065] Figure 4 It is a structural diagram of an electronic device disclosed in the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0066] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0067] Currently, existing question-and-answer systems mostly rely on pre-built knowledge bases, rule engines, or retrieval mechanisms, and perform well in specific domains and common question-and-answer patterns. However, in the face of complex, cross-domain problems or scenarios with rapid information updates, their limitations become apparent. It is difficult for the answers to accurately match the user's needs and they may be outdated or inaccurate. At the same time, these systems have weak quality control after answer generation and lack self-optimization mechanisms. Therefore, in a dynamic and changing application environment, the performance of traditional question-and-answer systems needs to be improved. For this reason, this application provides a question-and-answer optimization method, which continuously adjusts the scoring criteria and the dynamic selection of retrieval enhancement generation strategies through user feedback, gradually optimizing the question-and-answer quality and system performance, so as to better meet the user's needs and improve the intelligence level of the question-and-answer system.

[0068] See Figure 1 As shown, an embodiment of the present invention discloses a question-and-answer optimization method, including:

[0069] Step S11: Generate an initial answer based on the user's question and using a large language model, and then perform multi-dimensional scoring on the initial answer through the current question scenario to obtain a scoring result, and use a machine learning algorithm and a user feedback mechanism to determine a target quality scoring threshold, and judge whether the scoring result is less than the target quality scoring threshold.

[0070] In this embodiment, the user's question input by the user terminal and the context information corresponding to the user's question are obtained, and the user's question and the context information are concatenated to obtain an input sequence for inputting into the large language model. Based on the input sequence and using the generation module of the large language model, an initial answer is generated; the large language model can be a GPT (Generative Pre-trained Transformer, that is, a generative pre-trained transformer) series or a BERT (Bidirectional Encoder Representations from Transformers, that is, a pre-trained language model), and the corresponding generation formula is:

[0071] ;

[0072] Among them, is the initial answer, is the user's question, is the context information. After obtaining the initial answer, the initial answer is respectively scored for relevance, accuracy, logic, and information integrity to obtain an initial scoring vector, and the formula corresponding to the initial scoring vector is:

[0073] ;

[0074] Among them, is the initial scoring vector, is the relevance score, is the accuracy score, is the logic score, is the custom dimension score. to are the scoring weights for each item. After obtaining the initial scoring vector, according to the different current question scenarios, it is necessary to dynamically adjust the scoring weights corresponding to the initial scoring vector to obtain the adjusted scoring weights. It is worth mentioning that in addition to performing relevance scoring, accuracy scoring, logic scoring, and information integrity scoring on the initial answer, evidence support scoring or other dimension scoring that helps optimize the initial answer can also be performed according to the actual situation.

[0075] In a specific implementation manner, if the user's question is "how to improve English writing skills", and if the content of the initial answer is the method of improving English listening, it indicates that the relevance of the initial answer is relatively low. If the full score is 10 points, the relevance score corresponding to the initial answer is 1 point; by performing relevance scoring, accuracy scoring, logic scoring, and information integrity scoring on the initial answer, the initial scoring vector may be [1, 5, 6, 7]. In another specific implementation manner, if the user's question is in the professional academic field, then it is necessary to increase the scoring weights corresponding to accuracy and logic respectively. At this time, the scoring weight corresponding to accuracy scoring may be increased from the initial 0.25 to 0.4, and the scoring weight corresponding to logic scoring is increased from 0.25 to 0.3, while the scoring weights corresponding to relevance and information integrity scoring are correspondingly reduced. After adjusting the weights in this way, the high requirements for answer accuracy and logic in the academic scenario can be more prominent.

[0076] It can be understood that after obtaining the adjusted scoring weights, based on the initial scoring vector and the adjusted scoring weights, determine the comprehensive score of the initial answer, and then perform multi-dimensional scoring on the filtered test question-and-answer data, and determine the mean and standard deviation of the multi-dimensional scoring results, so as to determine the initial quality scoring threshold using the mean and the standard deviation. Then, collect high-quality historical question-and-answer data, and use a machine learning model for learning and optimization. At the same time, collect user feedback on the quality of historical question-and-answer data from the user side, such as likes, comments, etc., and input the user feedback into the machine learning model for optimization to obtain the target machine learning model. Based on the initial quality scoring threshold and using the target machine learning model, determine the target quality scoring threshold, and judge whether the comprehensive score is less than the target quality scoring threshold.

[0077] Specifically, an initial answer is generated based on the user's question and using a large language model, and then the initial answer is multi-dimensionally scored according to the current question scenario to obtain a scoring result. A target quality scoring threshold is determined using a machine learning algorithm and a user feedback mechanism, and it is judged whether the scoring result is less than the target quality scoring threshold, including: obtaining the input user question and the context information corresponding to the user question, and splicing the user question and the context information to obtain an input sequence; inputting the input sequence into the large language model to generate an initial answer, and respectively performing a relevance score, an accuracy score, a logicality score, and an information integrity score on the initial answer to obtain an initial scoring vector; adjusting the respective scoring weights corresponding to the initial scoring vector according to the current question scenario to obtain adjusted scoring weights, and determining the comprehensive score of the initial answer based on the initial scoring vector and the adjusted scoring weights; determining the mean and standard deviation of the respective dimension scores corresponding to the test question-and-answer data based on the test question-and-answer data, and determining an initial quality scoring threshold through the mean and the standard deviation; updating the initial quality scoring threshold using a machine learning algorithm and a user feedback mechanism to obtain a target quality scoring threshold, and judging whether the comprehensive score is less than the target quality scoring threshold.

[0078] Step S12, if the scoring result is less than the target quality scoring threshold, analyze the scoring result, and then determine a corresponding target retrieval enhancement generation strategy based on the analysis result and the question type corresponding to the user question; the target retrieval enhancement generation strategy includes vector retrieval, knowledge graph query, multi-hop reasoning, and information retrieval.

[0079] In this embodiment, if the comprehensive score is less than the target quality scoring threshold, analyze the initial answer and the comprehensive score to obtain an analysis result, then determine corresponding preset relevance scoring thresholds, preset accuracy scoring thresholds, preset logicality scoring thresholds, and preset information integrity scoring thresholds according to the question type corresponding to the user question, then judge whether each score in the comprehensive score is less than the corresponding preset scoring threshold through the analysis result, and determine a target retrieval enhancement generation strategy based on the judgment result.

[0080] In the first specific implementation manner, if the comprehensive score is less than the target quality score threshold, the initial answer and the comprehensive score are analyzed to obtain an analysis result, and then a preset relevance score threshold is determined for the corresponding question type based on the user question. For example, if the question type corresponding to the user question belongs to professional academic research or precise information retrieval, the determined preset relevance score threshold is relatively high. It is judged whether the relevance score in the comprehensive score is less than the preset relevance score threshold through the analysis result. If the relevance score in the comprehensive score is less than the preset relevance score threshold, it is determined that the target retrieval enhancement generation strategy is vector retrieval. Specifically, if the score result is less than the target quality score threshold, the score result is analyzed, and then the corresponding target retrieval enhancement generation strategy is determined based on the analysis result and the question type corresponding to the user question, including: if the comprehensive score is less than the target quality score threshold, the initial answer and the comprehensive score are analyzed to obtain an analysis result; a preset relevance score threshold is determined based on the question type corresponding to the user question, and it is judged whether the relevance score in the comprehensive score is less than the preset relevance score threshold based on the analysis result; if the relevance score in the comprehensive score is less than the preset relevance score threshold, it is determined that the target retrieval enhancement generation strategy is vector retrieval.

[0081] In the second specific implementation manner, if the comprehensive score is less than the target quality score threshold, the initial answer and the comprehensive score are analyzed to obtain an analysis result, and then a preset accuracy score threshold is determined for the corresponding question type based on the user question. For example, if the question type corresponding to the user question belongs to the financial or financial field, the determined preset accuracy score threshold is relatively high. It is judged whether the accuracy score in the comprehensive score is less than the preset accuracy score threshold through the analysis result. If the accuracy score in the comprehensive score is less than the preset accuracy score threshold, it is determined that the target retrieval enhancement generation strategy is knowledge graph query. Specifically, if the score result is less than the target quality score threshold, the score result is analyzed, and then the corresponding target retrieval enhancement generation strategy is determined based on the analysis result and the question type corresponding to the user question, including: if the comprehensive score is less than the target quality score threshold, the initial answer and the comprehensive score are analyzed to obtain an analysis result; a preset accuracy score threshold is determined based on the question type corresponding to the user question, and it is judged whether the accuracy score in the comprehensive score is less than the preset accuracy score threshold based on the analysis result; if the accuracy score in the comprehensive score is less than the preset accuracy score threshold, it is determined that the target retrieval enhancement generation strategy is knowledge graph query.

[0082] In the third specific implementation manner, if the comprehensive score is less than the target quality score threshold, the initial answer and the comprehensive score are analyzed to obtain an analysis result, and then a preset logical score threshold is determined for the corresponding question type based on the user question. For example, if the question type corresponding to the user question belongs to the fields of debate and speech or project planning, the determined preset logical score threshold is relatively high. It is judged whether the logical score in the comprehensive score is less than the preset logical score threshold through the analysis result. If the logical score in the comprehensive score is less than the preset logical score threshold, it is determined that the target retrieval enhancement generation strategy is multi-hop reasoning. Specifically, if the score result is less than the target quality score threshold, the score result is analyzed, and then the corresponding target retrieval enhancement generation strategy is determined based on the analysis result and the question type corresponding to the user question, including: if the comprehensive score is less than the target quality score threshold, the initial answer and the comprehensive score are analyzed to obtain an analysis result; a preset logical score threshold is determined based on the question type corresponding to the user question, and it is judged whether the logical score in the comprehensive score is less than the preset logical score threshold based on the analysis result; if the logical score in the comprehensive score is less than the preset logical score threshold, it is determined that the target retrieval enhancement generation strategy is multi-hop reasoning.

[0083] In the fourth specific implementation manner, if the comprehensive score is less than the target quality score threshold, the initial answer and the comprehensive score are analyzed to obtain an analysis result, and then a preset information integrity score threshold is determined for the corresponding question type based on the user question. For example, if the question type corresponding to the user question belongs to the fields of medical diagnosis or case investigation, the determined preset information integrity score threshold is relatively high. It is judged whether the information integrity score in the comprehensive score is less than the preset information integrity score threshold through the analysis result. If the information integrity score in the comprehensive score is less than the preset information integrity score threshold, it is determined that the target retrieval enhancement generation strategy is information retrieval. Specifically, if the score result is less than the target quality score threshold, the score result is analyzed, and then the corresponding target retrieval enhancement generation strategy is determined based on the analysis result and the question type corresponding to the user question, including: if the comprehensive score is less than the target quality score threshold, the initial answer and the comprehensive score are analyzed to obtain an analysis result; a preset information integrity score threshold is determined based on the question type corresponding to the user question, and it is judged whether the information integrity score in the comprehensive score is less than the preset information integrity score threshold based on the analysis result; if the information integrity score in the comprehensive score is less than the preset information integrity score threshold, it is determined that the target retrieval enhancement generation strategy is information retrieval.

[0084] Step S13: Optimize the initial answer using the target retrieval enhanced generation strategy to obtain an optimized answer, perform multi-dimensional scoring on the optimized answer to obtain a new scoring result, and jump to the step of determining whether the scoring result is less than the target quality scoring threshold until the scoring result is not less than the target quality scoring threshold or meets the iteration termination condition to obtain the target answer.

[0085] In this embodiment, after determining the target retrieval enhanced generation strategy, use the target retrieval enhanced generation strategy to optimize the initial answer, perform multi-dimensional scoring on the optimized answer to obtain a new scoring result, and jump to the step of determining whether the scoring result is less than the target quality scoring threshold until the scoring result is not less than the target quality scoring threshold or meets the iteration termination condition to obtain the target answer; the iteration termination condition may be that the number of iterations reaches the target iteration number threshold; where the iteration formula is:

[0086] ;

[0087] where, is the optimized answer, is the target retrieval enhanced generation strategy, is the target iteration number threshold.

[0088] In the first specific implementation manner, if the target retrieval enhanced generation strategy is vector retrieval, use the word embedding technology to perform word segmentation processing on the user question and each document in the preset document library respectively to obtain a question vector and a document vector, use the vector space model to calculate the similarities between the question vector and each of the document vectors in the preset document library, determine the document corresponding to the document vector with the highest similarity score as the target document, and optimize the initial answer by extracting the key information and knowledge of the target document to obtain the optimized answer. If the user question involves an image problem, the feature vector of the image and the document vector can be fused and then the similarity calculation is performed, and then the initial answer is optimized. Specifically, the using the target retrieval enhanced generation strategy to optimize the initial answer to obtain the optimized answer includes: using the word embedding technology to convert the user question and each document in the preset document library into word vectors respectively to obtain a question vector and a document vector; using the vector space model to calculate the similarity between the question vector and each of the document vectors in the preset document library, determine the document corresponding to the document vector with the highest similarity score as the target document, and optimize the initial answer based on the target document to obtain the optimized answer.

[0089] In the second specific implementation manner, if the target retrieval enhanced generation strategy is a knowledge graph query, a target query statement is constructed based on the user question and the syntax rules of the graph query language, so as to query corresponding relevant information from a preset knowledge graph. Since the information returned by the knowledge graph may contain some redundant or irrelevant content, it is necessary to screen and integrate the relevant information to obtain first target information, and then use the first target information to correct the initial answer to obtain an optimized answer. Specifically, the using the target retrieval enhanced generation strategy to optimize the initial answer to obtain an optimized answer includes: constructing a target query statement based on the user question and the graph query language, so as to query corresponding relevant information from a preset knowledge graph, and screening and integrating the relevant information to obtain first target information; using the first target information to correct the initial answer to obtain an optimized answer.

[0090] In the third specific implementation manner, if the target retrieval enhanced generation strategy is multi-hop reasoning, the user question is decomposed to obtain a number of simple sub-questions, and second target information corresponding to the sub-questions is obtained from a preset knowledge graph based on the sub-questions; the second target information includes multi-modal information such as text, images, and audio; then, by analyzing the internal relationships of the sub-questions, the logical relationships between the sub-questions are established, and based on natural language processing technology and the logical relationships, the initial answer is identified and analyzed, and the places with logical problems in the initial answer are determined as logical break positions. The second target information is used to gradually reason about the user question to obtain intermediate conclusions, a logical chain is formed based on the intermediate conclusions and in combination with the logical relationships, the logical break positions in the initial answer are filled using the logical chain, and the optimized answer is determined in combination with the user question. Specifically, the using the target retrieval enhanced generation strategy to optimize the initial answer to obtain an optimized answer includes: decomposing the user question to obtain a number of sub-questions, and obtaining second target information corresponding to the sub-questions from a preset knowledge graph based on the sub-questions; establishing the logical relationships between the sub-questions, and identifying the initial answer based on natural language processing technology and the logical relationships to obtain logical break positions; using the second target information to gradually reason about the user question to obtain intermediate conclusions; determining corresponding logical chains based on the intermediate conclusions and in combination with the logical relationships, and using the logical chains, the logical break positions, and the user question to determine the optimized answer.

[0091] In the fourth specific implementation manner, if the target retrieval enhanced generation strategy is information retrieval, determine the target topic and target keywords corresponding to the target problem, retrieve the corresponding relevant records from the preset structured database based on the target topic and the target keywords, extract the relevant records to obtain the third target information directly related to the target problem, and use the third target information to optimize the initial answer to obtain the optimized answer; in addition to the preset structured database, consider integrating other data sources, such as using unstructured text data, image data, social media data, etc. for retrieval. Specifically, using the target retrieval enhanced generation strategy to optimize the initial answer to obtain the optimized answer includes: determining the target topic and target keywords corresponding to the target problem, and retrieving the corresponding relevant records from the preset structured database based on the target topic and the target keywords; extracting the relevant records to obtain the third target information, and using the third target information to optimize the initial answer to obtain the optimized answer. It is worth mentioning that if there are multiple cases with low scores in the scoring results, a hybrid strategy can be used to combine multiple retrieval enhanced generation strategies to optimize the initial answer.

[0092] It can be understood that after obtaining the target answer, the user terminal can be used to provide feedback on the target answer, so as to adjust the multi-dimensional scoring and the target retrieval enhanced generation strategy based on the feedback content. Determine the new scoring result as the system prediction score, obtain the user feedback score using the user terminal's score for the target answer, and determine whether the difference between the system prediction score and the user feedback score meets the preset difference condition. If the difference between the system prediction score and the user feedback score meets the preset difference condition, then adjust the respective scoring weights corresponding to the multi-dimensional scoring through the following formula. The adjustment formula is:

[0093] ;

[0094] Among them, is the adjusted scoring weight, is the learning rate, is the user feedback score, Predict a score for the system. Specifically, perform multi-dimensional scoring on the optimized answer to obtain a new scoring result, and jump to the step of determining whether the scoring result is less than the target quality scoring threshold until the scoring result is not less than the target quality scoring threshold or meets the iteration termination condition to obtain the target answer. After that, it further includes: determining the system prediction score based on the new scoring result, and obtaining the user feedback score by using the score of the user terminal for the target answer; determining whether the difference between the system prediction score and the user feedback score meets the preset difference condition; if the difference between the system prediction score and the user feedback score meets the preset difference condition, then determining whether the system prediction score is greater than the user feedback score; if the system prediction score is greater than the user feedback score, then reducing the respective scoring weights corresponding to the multi-dimensional scoring; if the system prediction score is not greater than the user feedback score, then increasing the respective scoring weights corresponding to the multi-dimensional scoring.

[0095] As can be seen from the above, this application performs multi-dimensional scoring on the generated initial answer to obtain a scoring result, and determines the target quality scoring threshold based on the machine learning algorithm and the user feedback mechanism. If the scoring result is less than the target quality scoring threshold, then determine the corresponding target retrieval enhancement generation strategy based on the scoring result and the question type of the user question, so as to optimize the initial answer by using the target retrieval enhancement generation strategy, and perform multi-dimensional scoring on the optimized answer again to obtain a new scoring result, and then jump to the step of determining whether the scoring result is less than the target quality scoring threshold until the scoring result is not less than the target quality scoring threshold or meets the iteration termination condition to obtain the target answer. In this way, by continuously adjusting the scoring criteria and the dynamic selection of the retrieval enhancement generation strategy through user feedback, the Q&A quality and system performance are gradually optimized, so as to better meet the user's needs and improve the intelligent level of the Q&A system.

[0096] As can be known from the above embodiments, this application realizes the optimization of the initial answer based on the user feedback mechanism and the dynamic selection of the target retrieval enhancement generation strategy. Therefore, the process of realizing the optimization of the initial answer based on the user feedback mechanism and the dynamic selection of the target retrieval enhancement generation strategy is described.

[0097] See Figure 2 As shown, an embodiment of the present invention discloses a specific Q&A optimization method, including:

[0098] In this embodiment, the user's question and the corresponding context information are obtained, and the user's question and the context information are input into a large language model to generate an initial answer. The initial answer is multi-dimensionally scored based on the technical field corresponding to the current question scenario to obtain a comprehensive score. Then, by screening the collected test Q&A data, the target Q&A data obtained by screening is multi-dimensionally scored, and the mean and standard deviation of the multi-dimensionally scored results are determined based on the obtained multi-dimensionally scored results, so as to use the mean and the standard deviation to determine an initial quality score threshold. Based on high-quality historical Q&A data and combined with the user feedback of the user terminal on the quality of the historical Q&A data, the initial machine learning model is learned and optimized to obtain a target machine learning model. Based on the initial quality score threshold and using the target machine learning model, a target quality score threshold is determined, and it is judged whether the comprehensive score is less than the target quality score threshold.

[0099] If the comprehensive score is less than the target quality score threshold, the initial answer and the comprehensive score are analyzed to obtain an analysis result. Then, corresponding preset relevance score thresholds, preset accuracy score thresholds, preset logic score thresholds, and preset information integrity score thresholds are determined for the corresponding question types using the user's question. Next, it is judged whether the relevance score, accuracy score, logic score, and information integrity score in the comprehensive score are less than the corresponding preset relevance score threshold, preset accuracy score threshold, preset logic score threshold, and preset information integrity score threshold through the analysis result. If the relevance score in the comprehensive score is less than the preset relevance score threshold, the target retrieval enhancement generation strategy is determined to be vector retrieval; if the accuracy score in the comprehensive score is less than the preset accuracy score threshold, the target retrieval enhancement generation strategy is determined to be knowledge graph query; if the logic score in the comprehensive score is less than the preset logic score threshold, the target retrieval enhancement generation strategy is determined to be multi-hop reasoning; if the information integrity score in the comprehensive score is less than the preset information integrity score threshold, the target retrieval enhancement generation strategy is determined to be information retrieval.

[0100] Next, use the target retrieval enhancement generation strategy to optimize the initial answer to obtain an optimized answer, perform multi-dimensional scoring on the optimized answer to obtain a new scoring result, and jump to the step of determining whether the scoring result is less than the target quality scoring threshold until the scoring result is not less than the target quality scoring threshold or reaches the target iteration count threshold to obtain a target answer, output the target answer so that the client can provide feedback on and score the target answer to obtain a user feedback score, and determine the new scoring result as the system prediction score, and then determine whether the difference between the system prediction score and the user feedback score meets a preset difference condition; if the difference between the system prediction score and the user feedback score meets the preset difference condition, it indicates that there is a large difference between the system prediction score and the user feedback score, and it may be that the client is not satisfied with the system prediction score. Therefore, it is necessary to determine whether the system prediction score is greater than the user feedback score; if the system prediction score is greater than the user feedback score, reduce the respective scoring weights corresponding to the multi-dimensional scoring; if the system prediction score is not greater than the user feedback score, increase the respective scoring weights corresponding to the multi-dimensional scoring to dynamically adjust the scoring criteria based on user feedback.

[0101] As can be seen from the above, in this embodiment, by performing multi-dimensional scoring on the initial answer and selecting a dynamic target retrieval enhancement generation strategy based on the scoring result to specifically optimize the initial answer, and after obtaining the optimized answer, performing iterative scoring optimization on the optimized answer, the final obtained target answer can meet the answering requirements of complex questions. After the target answer is output, by continuously adjusting the scoring criteria and the selection of the target retrieval enhancement generation strategy through the feedback of the client, it can better meet the user's needs, thereby improving the quality and comprehensiveness of the optimized question and answer.

[0102] Correspondingly, as shown in Figure 3 the present application also provides a question and answer optimization device, including:

[0103] An initial answer scoring module 11, configured to generate an initial answer based on a user question using a large language model, then perform multi-dimensional scoring on the initial answer through the current question scenario to obtain a scoring result, and use a machine learning algorithm and a user feedback mechanism to determine a target quality scoring threshold, and determine whether the scoring result is less than the target quality scoring threshold;

[0104] A scoring result analysis module 12, configured to, if the scoring result is less than the target quality scoring threshold, analyze the scoring result, and then determine a corresponding target retrieval enhancement generation strategy based on the analysis result and the question type corresponding to the user question; the target retrieval enhancement generation strategy includes vector retrieval, knowledge graph query, multi-hop reasoning, and information retrieval;

[0105] An initial answer optimization module 13, configured to optimize the initial answer by using the target retrieval enhancement generation strategy to obtain an optimized answer, perform multi-dimensional scoring on the optimized answer to obtain a new scoring result, and jump to the step of determining whether the scoring result is less than the target quality scoring threshold until the scoring result is not less than the target quality scoring threshold or meets the iteration termination condition to obtain a target answer.

[0106] As can be seen from the above, in this application, multi-dimensional scoring is performed on the generated initial answer to obtain a scoring result, and the target quality scoring threshold is determined based on the machine learning algorithm and the user feedback mechanism. If the scoring result is less than the target quality scoring threshold, the corresponding target retrieval enhancement generation strategy is determined based on the scoring result and the question type of the user question, so as to optimize the initial answer by using the target retrieval enhancement generation strategy, and perform multi-dimensional scoring on the optimized answer again to obtain a new scoring result, and then jump to the step of determining whether the scoring result is less than the target quality scoring threshold until the scoring result is not less than the target quality scoring threshold or meets the iteration termination condition to obtain a target answer. In this way, by continuously adjusting the scoring criteria and the dynamic selection of the retrieval enhancement generation strategy through user feedback, the Q&A quality and system performance are gradually optimized, so as to better meet the user's needs and improve the intelligence level of the Q&A system.

[0107] In some specific embodiments, the initial answer scoring module 11 may specifically include:

[0108] An information splicing unit, configured to obtain the input user question and the context information corresponding to the user question, and splice the user question and the context information to obtain an input sequence;

[0109] An initial answer generation unit, configured to input the input sequence into the large language model to generate an initial answer, and perform relevance scoring, accuracy scoring, logical scoring, and information integrity scoring on the initial answer respectively to obtain an initial scoring vector;

[0110] A scoring weight adjustment unit, configured to adjust the respective scoring weights corresponding to the initial scoring vector according to the current question scenario to obtain adjusted scoring weights, and determine the comprehensive score of the initial answer based on the initial scoring vector and the adjusted scoring weights;

[0111] An initial threshold determination unit, configured to determine the mean and standard deviation of each dimension score corresponding to the test Q&A data based on the test Q&A data, and determine the initial quality scoring threshold through the mean and the standard deviation;

[0112] A target threshold determination unit, configured to update the initial quality score threshold based on a machine learning algorithm and a user feedback mechanism to obtain a target quality score threshold, and determine whether the comprehensive score is less than the target quality score threshold.

[0113] In some specific embodiments, the scoring result analysis module 12 may specifically include:

[0114] A first initial answer analysis unit, configured to analyze the initial answer and the comprehensive score to obtain an analysis result if the comprehensive score is less than the target quality score threshold;

[0115] A relevance score threshold determination unit, configured to determine a preset relevance score threshold based on the question type corresponding to the user question, and determine whether the relevance score in the comprehensive score is less than the preset relevance score threshold based on the analysis result;

[0116] A first target strategy determination unit, configured to determine the target retrieval enhancement generation strategy as vector retrieval if the relevance score in the comprehensive score is less than the preset relevance score threshold;

[0117] Correspondingly, the initial answer optimization module 13 may specifically include:

[0118] A document conversion unit, configured to convert the user question and each document in the preset document library into word vectors respectively by using word embedding technology to obtain a question vector and a document vector;

[0119] A target document determination unit, configured to calculate the similarity between the question vector and each document vector in the preset document library by using a vector space model, determine the document corresponding to the document vector with the highest similarity score as the target document, and optimize the initial answer based on the target document to obtain an optimized answer.

[0120] In some specific embodiments, the scoring result analysis module 12 may specifically include:

[0121] A second initial answer analysis unit, configured to analyze the initial answer and the comprehensive score to obtain an analysis result if the comprehensive score is less than the target quality score threshold;

[0122] An accuracy score threshold determination unit, configured to determine a preset accuracy score threshold based on the question type corresponding to the user question, and determine whether the accuracy score in the comprehensive score is less than the preset accuracy score threshold based on the analysis result;

[0123] A second target strategy determination unit, configured to determine that the target retrieval enhancement generation strategy is a knowledge graph query if the accuracy score in the comprehensive score is less than the preset accuracy score threshold;

[0124] Correspondingly, the initial answer optimization module 13 may specifically include:

[0125] A first target information determination unit, configured to construct a target query statement based on the user question and a graph query language, query corresponding relevant information from a preset knowledge graph using the target query statement, and screen and integrate the relevant information to obtain first target information;

[0126] An initial answer correction unit, configured to correct the initial answer using the first target information to obtain an optimized answer.

[0127] In some specific embodiments, the score result analysis module 12 may specifically include:

[0128] A third initial answer analysis unit, configured to analyze the initial answer and the comprehensive score to obtain an analysis result if the comprehensive score is less than the target quality score threshold;

[0129] A logical score threshold determination unit, configured to determine a preset logical score threshold based on the question type corresponding to the user question, and determine whether the logical score in the comprehensive score is less than the preset logical score threshold based on the analysis result;

[0130] A third target strategy determination unit, configured to determine that the target retrieval enhancement generation strategy is multi-hop reasoning if the logical score in the comprehensive score is less than the preset logical score threshold;

[0131] Correspondingly, the initial answer optimization module 13 may specifically include:

[0132] A second target information determination unit, configured to decompose the user question to obtain a plurality of sub-questions, and obtain second target information corresponding to the sub-questions from a preset knowledge graph based on the sub-questions;

[0133] A logical relationship establishment unit, configured to establish a logical relationship between the sub-questions, and identify the initial answer based on natural language processing technology and the logical relationship to obtain a logical break position;

[0134] An intermediate conclusion determination unit, configured to gradually reason about the user question using the second target information to obtain each intermediate conclusion;

[0135] A logical chain determination unit, configured to determine a corresponding logical chain based on the intermediate conclusion and in combination with the logical relationship, and determine an optimized answer by using the logical chain, the logical break position, and the user question.

[0136] In some specific embodiments, the scoring result analysis module 12 may specifically include:

[0137] A fourth initial answer analysis unit, configured to analyze the initial answer and the comprehensive score if the comprehensive score is less than the target quality score threshold, so as to obtain an analysis result;

[0138] An information integrity score threshold determination unit, configured to determine a preset information integrity score threshold based on the question type corresponding to the user question, and determine whether the information integrity score in the comprehensive score is less than the preset information integrity score threshold based on the analysis result;

[0139] A fourth target strategy determination unit, configured to determine that the target retrieval enhancement generation strategy is information retrieval if the information integrity score in the comprehensive score is less than the preset information integrity score threshold;

[0140] Correspondingly, the initial answer optimization module 13 may specifically include:

[0141] A relevant record retrieval unit, configured to determine a target theme and target keywords corresponding to the target question, and retrieve corresponding relevant records from a preset structured database based on the target theme and the target keywords;

[0142] A third target information determination unit, configured to extract the relevant records to obtain third target information, and optimize the initial answer by using the third target information to obtain an optimized answer.

[0143] In some specific embodiments, the Q&A optimization device may further specifically include:

[0144] A user feedback score determination unit, configured to determine a system prediction score based on a new scoring result, and obtain a user feedback score by using the score given by the user side for the target answer;

[0145] A difference judgment unit, configured to judge whether the difference between the system prediction score and the user feedback score meets a preset difference condition;

[0146] A score judgment unit, configured to judge whether the system prediction score is greater than the user feedback score if the difference between the system prediction score and the user feedback score meets the preset difference condition;

[0147] A scoring weight reduction unit, configured to reduce the scoring weights corresponding to the multi-dimensional scoring if the system predicted score is greater than the user feedback score;

[0148] A scoring weight increase unit, configured to increase the scoring weights corresponding to the multi-dimensional scoring if the system predicted score is not greater than the user feedback score.

[0149] Furthermore, an embodiment of the present application also discloses an electronic device. Figure 4 It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment. The content in the figure should not be considered as any limitation on the scope of use of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the question-and-answer optimization method disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0150] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of the present application, and specific limitations are not imposed on it here; the input / output interface 25 is used to obtain external input data or output data to the outside, and its specific interface type can be selected according to specific application needs, and specific limitations are not made here.

[0151] In addition, as a carrier for resource storage, the memory 22 may be a read-only memory, a random access memory, a magnetic disk, or an optical disc, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be short-term storage or permanent storage.

[0152] Among them, the operating system 221 is used to manage and control each hardware device on the electronic device 20 and the computer program 222, and it may be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the question-and-answer optimization method executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program that can be used to complete other specific tasks.

[0153] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the question-and-answer optimization method disclosed above is implemented. For the specific steps of this method, reference may be made to the corresponding content disclosed in the foregoing embodiments, and details will not be repeated herein.

[0154] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference may be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and reference may be made to the description in the method part for relevant parts.

[0155] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0156] The steps of the methods or algorithms described in combination with the embodiments disclosed in this article can be directly implemented by hardware, software modules executed by a processor, or a combination of the two. The software modules can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field.

[0157] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising a..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0158] The above has introduced the technical solution provided by this application in detail. Specific examples are used in this article to elaborate on the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. A question-answering optimization method, characterized in that: include: Generate an initial answer based on the user's question using a large language model, then perform a multi-dimensional scoring on the initial answer based on the current question scenario to obtain a scoring result, and use a machine learning algorithm and a user feedback mechanism to determine a target quality scoring threshold to determine whether the scoring result is less than the target quality scoring threshold; If the scoring result is less than the target quality scoring threshold, the scoring result is analyzed, and then a corresponding target retrieval enhancement generation strategy is determined based on the analysis result and the question type corresponding to the user question; The target retrieval enhancement generation strategy includes vector retrieval, knowledge graph query, multi-hop reasoning and information retrieval; The target retrieval enhancement generation strategy is used to optimize the initial answer to obtain an optimized answer, and the optimized answer is scored in multiple dimensions to obtain a new scoring result, and the process jumps to the step of determining whether the scoring result is less than the target quality scoring threshold, until the scoring result is not less than the target quality scoring threshold or the iteration termination condition is met to obtain the target answer.

2. The question-answering optimization method according to claim 1, characterized in that: The initial answer is generated based on the user question and using the large language model, and then the initial answer is scored in multiple dimensions according to the current question scenario to obtain a scoring result, and a target quality scoring threshold is determined by using a machine learning algorithm and a user feedback mechanism, and it is judged whether the scoring result is less than the target quality scoring threshold, including: Obtaining an input user question and context information corresponding to the user question, and concatenating the user question and the context information to obtain an input sequence; Inputting the input sequence into the large language model to generate an initial answer, and respectively performing relevance scoring, accuracy scoring, logic scoring, and information completeness scoring on the initial answer to obtain an initial scoring vector; Adjusting each scoring weight corresponding to the initial scoring vector according to the current question scenario to obtain an adjusted scoring weight, and determining a comprehensive score of the initial answer based on the initial scoring vector and the adjusted scoring weight; Determine, based on the test question and answer data, a mean and a standard deviation of the scores of each dimension corresponding to the test question and answer data, and determine an initial quality score threshold by the mean and the standard deviation; The initial quality score threshold is updated based on the use of a machine learning algorithm and a user feedback mechanism to obtain a target quality score threshold, and it is determined whether the comprehensive score is less than the target quality score threshold.

3. The question-answering optimization method according to claim 2, characterized in that: If the scoring result is less than the target quality scoring threshold, the scoring result is analyzed, and then a corresponding target retrieval enhancement generation strategy is determined based on the analysis result and the question type corresponding to the user question, including: If the comprehensive score is less than the target quality score threshold, analyzing the initial answer and the comprehensive score to obtain an analysis result; Determining a preset relevance score threshold based on the question type corresponding to the user question, and judging whether the relevance score in the comprehensive score is less than the preset relevance score threshold based on the analysis result; If the relevance score in the comprehensive score is less than the preset relevance score threshold, determining that the target search enhancement generation strategy is vector search; Accordingly, the method of optimizing the initial answer by using the target retrieval enhancement generation strategy to obtain an optimized answer includes: The user question and each document in the preset document library are converted into word vectors respectively by using word embedding technology to obtain a question vector and a document vector; The vector space model is used to calculate the similarity between the question vector and each document vector in the preset document library, the document corresponding to the document vector with the highest similarity score is determined as the target document, and the initial answer is optimized based on the target document to obtain an optimized answer.

4. The question-answering optimization method according to claim 2, characterized in that: If the scoring result is less than the target quality scoring threshold, the scoring result is analyzed, and then a corresponding target retrieval enhancement generation strategy is determined based on the analysis result and the question type corresponding to the user question, including: If the comprehensive score is less than the target quality score threshold, analyzing the initial answer and the comprehensive score to obtain an analysis result; Determining a preset accuracy score threshold based on the question type corresponding to the user question, and judging whether the accuracy score in the comprehensive score is less than the preset accuracy score threshold based on the analysis result; If the accuracy score in the comprehensive score is less than the preset accuracy score threshold, determining that the target retrieval enhancement generation strategy is a knowledge graph query; Accordingly, the method of optimizing the initial answer by using the target retrieval enhancement generation strategy to obtain an optimized answer includes: Constructing a target query statement based on the user question and the graph query language, using the target query statement to query corresponding relevant information from a preset knowledge graph, and screening and integrating the relevant information to obtain first target information; The initial answer is modified using the first target information to obtain an optimized answer.

5. The question-answering optimization method according to claim 2, characterized in that: If the scoring result is less than the target quality scoring threshold, the scoring result is analyzed, and then a corresponding target retrieval enhancement generation strategy is determined based on the analysis result and the question type corresponding to the user question, including: If the comprehensive score is less than the target quality score threshold, analyzing the initial answer and the comprehensive score to obtain an analysis result; Determine a preset logic score threshold based on the question type corresponding to the user question, and determine whether the logic score in the comprehensive score is less than the preset logic score threshold based on the analysis result; If the logic score in the comprehensive score is less than the preset logic score threshold, determining that the target retrieval enhancement generation strategy is multi-hop reasoning; Accordingly, the method of optimizing the initial answer by using the target retrieval enhancement generation strategy to obtain an optimized answer includes: Decomposing the user question to obtain a plurality of sub-questions, and acquiring second target information corresponding to the sub-questions from a preset knowledge graph based on the sub-questions; Establishing a logical relationship between the sub-questions, and identifying the initial answer based on natural language processing technology and the logical relationship to obtain a logical break position; Using the second target information to perform step-by-step reasoning on the user question to obtain various intermediate conclusions; Based on the intermediate conclusion and in combination with the logical relationship, a corresponding logic chain is determined, and an optimized answer is determined using the logic chain, the logical break position and the user question.

6. The question-answering optimization method according to claim 2, characterized in that: If the scoring result is less than the target quality scoring threshold, the scoring result is analyzed, and then a corresponding target retrieval enhancement generation strategy is determined based on the analysis result and the question type corresponding to the user question, including: If the comprehensive score is less than the target quality score threshold, analyzing the initial answer and the comprehensive score to obtain an analysis result; Determining a preset information integrity score threshold based on the question type corresponding to the user question, and judging whether the information integrity score in the comprehensive score is less than the preset information integrity score threshold based on the analysis result; If the information integrity score in the comprehensive score is less than the preset information integrity score threshold, determining that the target retrieval enhancement generation strategy is information retrieval; Accordingly, the method of optimizing the initial answer by using the target retrieval enhancement generation strategy to obtain an optimized answer includes: Determine a target topic and a target keyword corresponding to the target question, and retrieve corresponding related records from a preset structured database based on the target topic and the target keyword; The relevant records are extracted to obtain third target information, and the initial answer is optimized using the third target information to obtain an optimized answer.

7. The question-answer optimization method according to any one of claims 1 to 6, characterized in that: The step of performing multi-dimensional scoring on the optimized answer to obtain a new scoring result, and jumping to the step of determining whether the scoring result is less than the target quality scoring threshold, until the scoring result is not less than the target quality scoring threshold or the iteration termination condition is met to obtain the target answer, further includes: Determine the system prediction score based on the new scoring result, and obtain the user feedback score using the user's score for the target answer; Determine whether the difference between the system predicted score and the user feedback score meets a preset difference condition; If the difference between the system predicted score and the user feedback score meets a preset difference condition, determining whether the system predicted score is greater than the user feedback score; If the system predicted score is greater than the user feedback score, then the score weights corresponding to the multi-dimensional score are reduced; If the system predicted score is not greater than the user feedback score, the score weights corresponding to the multi-dimensional score are increased.

8. A question-answering optimization device, characterized in that: include: An initial answer scoring module is used to generate an initial answer based on the user question and using a large language model, and then perform multi-dimensional scoring on the initial answer according to the current question scenario to obtain a scoring result, and use a machine learning algorithm and a user feedback mechanism to determine a target quality scoring threshold, and determine whether the scoring result is less than the target quality scoring threshold; A scoring result analysis module, configured to analyze the scoring result if the scoring result is less than the target quality scoring threshold, and then determine a corresponding target retrieval enhancement generation strategy based on the analysis result and the question type corresponding to the user question; The target retrieval enhancement generation strategy includes vector retrieval, knowledge graph query, multi-hop reasoning and information retrieval; The initial answer optimization module is used to optimize the initial answer using the target retrieval enhancement generation strategy to obtain an optimized answer, and to perform multi-dimensional scoring on the optimized answer to obtain a new scoring result, and jump to the step of determining whether the scoring result is less than the target quality scoring threshold, until the scoring result is not less than the target quality scoring threshold or the iteration termination condition is met to obtain the target answer.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the question-answering optimization method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the question-answering optimization method according to any one of claims 1 to 7 is implemented.

Images