Intelligent question and answer method and terminal based on retrieval-generation model

Abstract

The invention discloses an intelligent question and answer method and terminal based on a retrieval-generation model. The method comprises the steps that a first target answer matched with a target question sent by a user is retrieved in a question and answer pair knowledge base; if the first target answer is not retrieved, generating a second target answer matched with the target question through a retrieval-generation model, and returning the second target answer to the user; obtaining feedback data of the user about the second target answer, and calling an evaluation index model according to the target question and the feedback data to analyze and process the second target answer to obtain a quality score; and if the quality score of the second target answer is greater than a preset target score, updating the target question and the second target answer to the question and answer pair knowledge base. According to the method, computing resources consumed by the RAG question-answering system can be reduced, the response speed is increased, and the reliability of generating answers by questions and answers is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic question answering, and particularly to an intelligent question answering method and a terminal based on a retrieval-generation model. Background Art

[0002] With the rapid development of artificial intelligence technology, intelligent response technology has been widely applied in many fields such as intelligent customer service and chatbots. As the core of the intelligent response system, the knowledge base contains a large number of static question-answer pairs. When a user asks a question, the system retrieves and matches the most relevant entry from the knowledge base and then provides a preset answer. The construction and maintenance of the knowledge base directly determine the coverage breadth and accuracy of the intelligent response system. The quantity and quality of the question-answer pairs have a crucial impact on the performance of the system. The wider the coverage, the higher the probability that the system can provide accurate answers to users, thereby enhancing the user experience. However, question-answer pairs usually rely on manual addition, and this method has many limitations, including high labor costs, poor performance in dealing with long-tail problems (i.e., uncommon problems), a relatively long knowledge base update cycle, and insufficient depth of natural language understanding.

[0003] With the rapid development of large language models (LLMs), RAG (Retrieval-Augmented Generation) has been widely applied to enterprise private domain knowledge question answering scenarios. RAG combines information retrieval with generative models, enabling dynamic retrieval in a wide range of data sources and generating fluent and highly context-relevant answers. Different from traditional methods, RAG not only relies on static question-answer pairs in the knowledge base but can also retrieve information in real time from a vast amount of documents and integrate multiple sources when generating answers. This makes it show higher flexibility and depth in dealing with complex problems not covered in the knowledge base. Instead of manually adding each specific question-answer pair, RAG can generate the most relevant answers based on the latest documents, policies, or data, thus reducing the complexity and latency of knowledge base maintenance.

[0004] However, compared with rule-based question answering systems, since RAG needs to perform both information retrieval and generation processes simultaneously, this will bring relatively high computational resource consumption. Especially when a large number of user requests need to be responded to quickly, more powerful hardware support may be required. And involving multiple-step processing flows of retrieval and generation, RAG may have response latency in some scenarios. Especially when facing high-concurrency requests, the response time may not be as fast as that of the predefined question-answer pair system. In addition, the generative model of RAG sometimes produces content that is not completely accurate or trustworthy, especially when the retrieved information is incomplete or the model misinterprets the question.

[0005] In summary, RAG still faces challenges in terms of computational resource consumption, response speed, and reliability of generated content. Summary of the Invention

[0006] The technical problem to be solved by the present invention is: to provide an intelligent question-answering method and terminal based on a retrieval-generation model, which can effectively reduce the computing resources consumed by the RAG question-answering system, improve the response speed, and improve the reliability of the answers generated by the question-answering.

[0007] To solve the above technical problem, a technical solution adopted by the present invention is: An intelligent question-answering method based on a retrieval-generation model, comprising: Retrieving a first target answer matching the target question sent by the user in the question-answer pair knowledge base; If the first target answer is not retrieved, generating a second target answer matching the target question through a retrieval-generation model, and returning the second target answer to the user; Obtaining feedback data of the user on the second target answer, and analyzing and processing the second target answer according to the target question and the feedback data by using an evaluation index model to obtain a quality score; If the quality score of the second target answer is greater than a preset target score, updating the target question and the second target answer to the question-answer pair knowledge base.

[0008] To solve the above technical problem, another technical solution adopted by the present invention is: An intelligent question-answering terminal based on a retrieval-generation model, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, and when the processor executes the computer program, each step in the above-mentioned intelligent question-answering method based on a retrieval-generation model is implemented.

[0009] The beneficial effects of the present invention are as follows: When receiving the target question of the user, first retrieve the first target answer that matches the target question sent by the user in the Q&A pair knowledge base. If retrieved, it is directly returned to the user, without having to call the retrieval-generation model for each question of the user, improving the response speed of the intelligent Q&A system and reducing the consumption of computing resources. If no matching answer is retrieved in the Q&A pair knowledge base, then generate the second target answer that matches the target question through the retrieval-generation model and return it to the user, ensuring that the system can handle complex and long-tail questions. For the second target answer generated by the retrieval-generation model, call the evaluation index model based on the user's feedback data and the target question for quality scoring, avoiding the model generating content that does not match the target question, improving the accuracy and reliability of the response, and thus enhancing the user experience. In addition, update the second target answer with a higher quality score and the target question to the Q&A pair knowledge base, realizing the automatic update and maintenance of the knowledge base, and effectively improving the coverage breadth of the Q&A pair knowledge base. Through the fusion retrieval of the Q&A pair knowledge base and the retrieval-generation model, not only reduces the computing resources consumed by the retrieval-generation model, improves the response speed, but also ensures the accuracy of the answer. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a flowchart of an intelligent Q&A method based on a retrieval-generation model provided by an embodiment of the present invention; Figure 2 It is a schematic architecture diagram of an intelligent Q&A method based on a retrieval-generation model provided by an embodiment of the present invention; Figure 3 It is a schematic diagram of dimension evaluation of an evaluation index model provided by an embodiment of the present invention; Figure 4 It is a schematic structural diagram of an intelligent Q&A terminal based on a retrieval-generation model provided by an embodiment of the present invention; Reference Signs Description: 100, an intelligent Q&A terminal based on a retrieval-generation model; 101, a memory; 102, a processor. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0011] To describe in detail the technical content, achieved objectives and effects of the present invention, the following is described in conjunction with the embodiments and with reference to the accompanying drawings.

[0012] An intelligent Q&A method based on a retrieval-generation model provided by an embodiment of the present invention includes: Retrieve the first target answer that matches the target question sent by the user in the Q&A pair knowledge base; If the first target answer is not retrieved, then generate the second target answer that matches the target question through the retrieval-generation model and return the second target answer to the user; Obtain the feedback data of the user regarding the second target answer, and call the evaluation index model according to the target question and the feedback data to analyze and process the second target answer to obtain a quality score; If the quality score of the second target answer is greater than the preset target score, update the target question and the second target answer to the Q&A pair knowledge base.

[0013] As can be seen from the above description, the beneficial effects of the present invention are as follows: when receiving the target question of the user, first retrieve the first target answer matching the target question sent by the user in the Q&A pair knowledge base. If retrieved, it is directly returned to the user, without having to call the retrieval-generation model for each question of the user, improving the response speed of the intelligent Q&A system and reducing the consumption of computing resources. If no matching answer is retrieved in the Q&A pair knowledge base, then generate the second target answer matching the target question through the retrieval-generation model and return it to the user, ensuring that the system can handle complex and long-tail questions. For the second target answer generated by the retrieval-generation model, call the evaluation index model for quality scoring based on the user's feedback data and the target question, avoiding the generation of content that does not match the target question by the model, improving the accuracy and reliability of the response, and thus enhancing the user experience. In addition, update the second target answer with a higher quality score and the target question to the Q&A pair knowledge base, realizing the automatic update and maintenance of the knowledge base, and effectively improving the coverage breadth of the Q&A pair knowledge base. Through the fusion retrieval of the Q&A pair knowledge base and the retrieval-generation model, not only reduces the computing resources consumed by the retrieval-generation model, improves the response speed, but also ensures the accuracy of the answer.

[0014] Further, the retrieval-generation model includes an external knowledge source library and a large language model; Generating the second target answer matching the target question through the retrieval-generation model includes: Retrieve retrieval information in the external knowledge source library according to the target question; Perform reordering processing on the retrieval information that meets the preset matching conditions to obtain target information; Call the large language model to generate the second target answer according to the target information.

[0015] As can be seen from the above description, by performing information retrieval through the external knowledge source library and generating answers in combination with the large language model, not only enhances the flexibility and scalability of the Q&A system, enabling it to handle more extensive and complex questions, but also improves the accuracy and relevance of the retrieval results through reordering processing, further enhancing the response quality of the system.

[0016] Furthermore, the evaluation dimensions in the evaluation index model include a semantic dimension, an answer dimension, and a user dimension; Calling the evaluation index model according to the target question and the feedback data to analyze and process the second target answer to obtain a quality score includes: Performing semantic analysis on the second target answer according to the target information to obtain a semantic dimension score; Performing multi-dimensional feature analysis on the second target answer according to the target question to obtain an answer dimension score; Performing sentiment analysis on the second target answer according to the feedback data to obtain a user dimension score; Determining the quality score according to the semantic dimension score, the answer dimension score, and the user dimension score.

[0017] As can be seen from the above description, the semantic dimension, the answer dimension, and the user dimension in the evaluation index model provide a comprehensive quality assessment for the generated second target answer, can more accurately measure the quality of the answer, and ensure that only high-quality answers will be updated to the Q&A pair knowledge base. This not only improves the accuracy and reliability of the knowledge base, but also reduces the accumulation of invalid or low-quality Q&A pairs, thereby improving the overall performance of the Q&A system.

[0018] Furthermore, performing semantic analysis on the second target answer according to the target information to obtain a semantic dimension score includes: Semantically vectorizing the target information and the second target answer respectively to calculate the cosine similarity between the target information and the second target answer to obtain a semantic similarity; Extracting entity sets from the target information and the second target answer respectively to calculate the overlap degree between the target information and the second target answer; Extracting keywords from the target information and the second target answer respectively to calculate the key information coverage ratio between the target information and the second target answer; Determining the semantic faithfulness between the target information and the second target answer according to the semantic similarity, the overlap degree, and the key information coverage ratio; Performing calculation processing on the semantic similarity and the semantic faithfulness based on a preset semantic weight to obtain a semantic dimension score.

[0019] As described above, the semantic dimension score is calculated through semantic analysis, the degree of overlap of entity sets, and the keyword coverage ratio. The result of semantic analysis can determine the similarity between the target information and the generated answer. The level of the degree of overlap of entity sets can reflect the entity matching degree between the target information and the generated answer. The size of the keyword coverage ratio can reflect whether the generated answer completely covers the key content in the target information. The combination of the three can more accurately evaluate the semantic relevance between the generated answer and the retrieved information. Through in-depth semantic analysis, it helps to screen out answers that are more matching with the original question intention, avoid the model generating content that does not conform to the retrieved information, and reduce the occurrence of the "hallucination" problem.

[0020] Further, performing multi-dimensional feature analysis on the second target answer according to the target question to obtain an answer dimension score includes: Calculating the answer relevance between the target question and the second target answer through a text matching algorithm based on word frequency; Calculating the coverage ratio of the number of topics that the second target answer appears in the number of topics that the target question appears through a latent topic model to obtain answer integrity; Performing calculation processing on the answer relevance and the answer integrity based on a preset answer weight to obtain an answer dimension score.

[0021] As described above, the answer dimension score comprehensively considers answer relevance and integrity, and evaluates the quality of the answer through a text matching algorithm based on word frequency and a latent topic model, ensuring that the generated answer is not only closely related to the original question, but also can comprehensively cover the topics involved in the question, thereby improving the comprehensiveness and depth of the response.

[0022] Further, the feedback data includes the interaction log and behavior data of the user after receiving the second target answer; Performing sentiment analysis on the second target answer according to the feedback data to obtain a user dimension score includes: Extracting the sentiment tendency type of the user from the interaction log through a preset sentiment analysis model, and generating a user sentiment polarity based on the sentiment tendency type; Collecting the satisfaction score of the user from the interaction log, and performing normalization processing on the behavior data; Generating a user satisfaction based on the user sentiment polarity, the satisfaction score, and the normalized behavior data; Performing calculation processing on the user sentiment polarity and the user satisfaction based on a preset user weight to obtain a user dimension score.

[0023] As described above, the user sentiment polarity is an indicator that describes the intensity of the sentiment tendency expressed in text data. For example, a positive sentiment polarity indicates a positive sentiment, a negative sentiment polarity indicates a negative sentiment, and a sentiment polarity of zero indicates a neutral sentiment. By directly extracting the sentiment tendency type and satisfaction score from the user interaction log without the user actively scoring, the evaluation result is more credible. At the same time, by combining the user's behavior data for standardization processing, the satisfaction of the user with the generated answer can be more accurately evaluated. The present invention provides feedback data for the improvement and optimization of the system by recording the interaction between the user and the system, which helps to continuously improve the accuracy of the response and the user satisfaction.

[0024] Furthermore, it further includes: If the quality score of the second target answer is not greater than the preset target score, store the second target answer as the historical answer to the target question, and adjust the prompt word of the target question according to the semantic dimension score, the answer dimension score, and the user dimension score; Based on the adjusted target question, return to execute the step of generating a second target answer that matches the target question through the retrieval-generation model.

[0025] As described above, if the quality score of the second target answer is not high, store it as the historical answer and adjust the prompt word of the target question according to the scoring result. Targeted adjustments are made to the target question for the deficiencies in different dimension scores, so as to continuously guide the retrieval-generation model to generate new answers that better meet the dimension scores, improve the quality of the generated answers, and achieve automatic iteration.

[0026] Furthermore, determining the quality score according to the semantic dimension score, the answer dimension score, and the user dimension score includes: If it is detected that the target question has the historical answer, compare the historical answer with the second target answer to obtain the answer duplication degree; Calculate and process according to the answer duplication degree and the user dimension score to obtain the penalty factor; Determine the comprehensive dimension score according to the semantic dimension score, the answer dimension score, and the user dimension score, and determine the quality score according to the penalty factor and the comprehensive dimension score.

[0027] As described above, by detecting the duplication degree between the historical answer and the second target answer and calculating the penalty factor accordingly, it is possible to avoid generating duplicate or meaningless answers, which helps to reduce the accumulation of invalid Q&A pairs. At the same time, the penalty factor takes into account the user dimension score to evaluate whether the optimized generated answer meets the user's expectations, avoid deviating from the user's question, and improve the accuracy of the response. Combining the comprehensive dimension score and the penalty factor to determine the final quality score makes the evaluation result more comprehensive and reasonable.

[0028] Further, before calling the evaluation index model to analyze and process the second target answer based on the target question and the feedback data to obtain a quality score, the following steps are also included: Call the question security verification model to detect whether there are sensitive words in the target question. If so, optimize the sensitive words in the target question to obtain a standardized target question.

[0029] As can be seen from the above description, before performing the quality score, calling the question security verification model to detect whether there are sensitive words in the target question and performing optimization processing can ensure the compliance of the question-and-answer pair, effectively avoid risks and disputes caused by sensitive word problems, and improve the credibility and reliability of the system.

[0030] Another embodiment of the present invention provides an intelligent question-and-answer terminal based on a retrieval-generation model, including a memory, a processor, and a computer program stored on the memory and running on the processor. When the processor executes the computer program, each step in the above-mentioned intelligent question-and-answer method based on a retrieval-generation model is implemented.

[0031] As can be seen from the above description, the beneficial effects of the present invention are as follows: when receiving a user's target question, first retrieve the first target answer matching the target question sent by the user in the question-and-answer pair knowledge base. If retrieved, it is directly returned to the user, without having to call the retrieval-generation model for each user question, improving the response speed of the intelligent question-and-answer system and reducing the consumption of computing resources. If no matching answer is retrieved in the question-and-answer pair knowledge base, then generate a second target answer matching the target question through the retrieval-generation model and return it to the user, ensuring that the system can handle complex and long-tail questions. For the second target answer generated by the retrieval-generation model, call the evaluation index model to perform a quality score based on the user's feedback data and the target question, avoiding the model generating content that does not match the target question, improving the accuracy and reliability of the response, and thus enhancing the user experience. In addition, update the second target answer with a higher quality score and the target question to the question-and-answer pair knowledge base, realizing the automatic update and maintenance of the knowledge base, and effectively improving the coverage breadth of the question-and-answer pair knowledge base. Through the fusion retrieval of the question-and-answer pair knowledge base and the retrieval-generation model, not only the computing resources consumed by the retrieval-generation model are reduced, the response speed is increased, but also the accuracy of the answer is guaranteed.

[0032] The above-mentioned intelligent question-and-answer method and terminal based on a retrieval-generation model of the present invention can be applied to scenarios with a large number of question-and-answer requests, can effectively reduce the computing resources consumed by the RAG question-and-answer system, improve the response speed, and improve the reliability of the answers generated by the question-and-answer. The following is illustrated through specific embodiments: Please refer toFigures 1 to 2 , Embodiment 1 of the present invention is as follows: An intelligent question-and-answer method based on a retrieval-generation model, comprising: S10. Retrieve a first target answer that matches the target question sent by the user in the question-and-answer pair knowledge base. If the first target answer is retrieved in the question-and-answer pair knowledge base, directly return the first target answer to the user without performing the following steps S20 to S50.

[0033] In some embodiments, the specific manner of retrieving the first target answer in the question-and-answer pair knowledge base is the same as the specific manner of retrieving target information in the external knowledge source library. Among them, when retrieving the first target answer in the question-and-answer pair knowledge base, the most relevant content after re-ranking processing is used as the first target answer.

[0034] S20. If the first target answer is not retrieved, generate a second target answer that matches the target question through the retrieval-generation model and return the second target answer to the user.

[0035] Among them, the retrieval-generation model includes an external knowledge source library and a large language model.

[0036] Specifically, step S20 includes: S201. Retrieve retrieval information in the external knowledge source library according to the target question.

[0037] In an alternative implementation, when calling the retrieval-generation model to generate an answer, in addition to retrieving in the external knowledge source library, it is also possible to synchronously retrieve in the question-and-answer pair knowledge base, thereby improving the accuracy and coverage of the response.

[0038] In some embodiments, the retrieval methods in step S201 include full-text retrieval and vector retrieval. Full-text retrieval is a keyword-based retrieval method that indexes all the words in the knowledge source library to find matching content in the index according to the keywords input by the user. Vector retrieval converts the target question input by the user and the text content into vector forms and finds the most relevant documents to the query by calculating the similarity between the vectors. In an alternative implementation, full-text retrieval is implemented based on the BM25 algorithm, and vector retrieval is implemented based on the Faiss vector library.

[0039] S202. Re-rank the retrieval information that meets the preset matching conditions to obtain target information.

[0040] In some embodiments, the matching conditions corresponding to different retrieval methods are different. When using the full-text retrieval method, the retrieved information is recalled by setting whether the relevance score reaches a specified score as the preset matching condition. When using the vector retrieval method, the retrieved information is recalled by setting whether the vector similarity reaches a specified value as the preset matching condition.

[0041] In some embodiments, since the matching conditions of the target information retrieved by different retrieval methods are different, the evaluation criteria of different retrieval methods are unified through re-ranking processing. The re-ranking processing in step S202 is specifically as follows: the retrieved information that meets the preset matching condition is re-ranked through the reciprocal rank fusion algorithm (RRF), and the top N relevant contents in the ranking are used as the target information. The reciprocal rank fusion algorithm (RRF) is a method for combining multiple result sets with different relevance metrics into a single result set.

[0042] S203. Call the large language model according to the target information to generate the second target answer.

[0043] In some embodiments, if no retrieved information that meets the preset matching condition is retrieved from the external knowledge source library according to the target question, the preset abnormal status information is returned to the user to end the question and answer.

[0044] It should be noted that the question and answer pair knowledge base is a structured database composed of question and answer pair texts to quickly respond to common and high-frequency standardized questions. The external knowledge source library is an unstructured database composed of data such as documents, pictures, audio and video materials, and supports retrieval and answer generation through RAG technology to handle complex and long-tail questions. As Figure 2 shown, since the structured database mainly relies on simple indexing and keyword matching and has low requirements for hardware resources, a low-computing-power hardware configuration can be used for the question and answer pair knowledge base. Unstructured data needs to process complex unstructured data and perform semantic retrieval, which requires high hardware resources. Therefore, a high-computing-power hardware configuration is required for the external knowledge source library. In this way, the hardware configuration cost of the intelligent question and answer system can be effectively reduced.

[0045] Specifically, before step S30, it further includes: S25. Call the question security verification model to detect whether there are sensitive words in the target question. If so, optimize the sensitive words in the target question to obtain a standardized target question.

[0046] S30. Obtain the feedback data of the user on the second target answer, and call the evaluation index model according to the target question and the feedback data to analyze and process the second target answer to obtain a quality score.

[0047] Among them, the evaluation dimensions in the evaluation index model include the semantic dimension, the answer dimension, and the user dimension.

[0048] S40. If the quality score of the second target answer is greater than the preset target score, update the target question and the second target answer to the Q&A pair knowledge base.

[0049] In some embodiments, for the Q&A records generated in some interaction scenarios, such as the chat records between the customer service and the user, when the intelligent Q&A system is idle, the large language model will be automatically called to extract Q&A pairs from the Q&A records and update them to the Q&A pair knowledge base.

[0050] S50. If the quality score of the second target answer is not greater than the preset target score, store the second target answer as the historical answer of the target question, adjust the prompt words of the target question according to the semantic dimension score, the answer dimension score, and the user dimension score, and return to execute step S20 based on the adjusted target question.

[0051] In some embodiments, if the quality score of the second target answer is still less than or equal to the target score after the number of times of generating the second target answer reaches the preset number of rounds, select the second target answer with the highest quality score in all rounds and save it to the Q&A pair knowledge base. At the same time, send a message to the administrator information terminal to notify the administrator to conduct a manual review of the target answer, so as to ensure the accuracy and reliability of the final answer.

[0052] Please refer to Figure 3 , Embodiment 2 of the present invention is: An intelligent Q&A method based on a retrieval-generation model, which is different from Embodiment 1 in that: the specific implementation manner of step S30 is limited, including: S301. Perform semantic analysis on the second target answer according to the target information to obtain the semantic dimension score, specifically: S3011. Respectively perform semantic vectorization on the target information and the second target answer to calculate the cosine similarity between the target information and the second target answer, and obtain the semantic similarity.

[0053] S3012. Extract entity sets from the target information and the second target answer respectively to calculate the overlap degree between the target information and the second target answer.

[0054] In some embodiments, step S3012 is specifically as follows: Entities are extracted from the target information and the target answer respectively through a pre-trained named entity recognition model. The entities extracted usually include person names, locations, organizations, dates, and times, etc. The entity set extracted from the target information is E1, and the entity set extracted from the target answer is E2. By calculating the ratio of the size of the intersection of the two entity sets to the size of the union, the Jaccard similarity is obtained. The Jaccard similarity is the degree of overlap between the target information and the target answer. The higher the value of the Jaccard similarity, the higher the entity matching degree between the target answer and the target information.

[0055] S3013. Extract keywords from the target information and the second target answer respectively to calculate the key information coverage ratio between the target information and the second target answer.

[0056] In some embodiments, step S3013 is specifically as follows: First, the target information and the target answer are respectively segmented through the Chinese word segmentation library jieba of Python, and then keywords are extracted from the target information and the target answer respectively through the TF-IDF algorithm. The keyword set extracted from the target information is W1, and the keyword set extracted from the target answer is W2. The TF-IDF algorithm is a statistical method used to evaluate the importance of a word for a document in a document collection. Calculate the ratio of the size of the intersection of the two keyword sets to the size of the keyword set W1 to obtain the key information coverage ratio between the target information and the target answer.

[0057] S3014. Determine the semantic faithfulness between the target information and the second target answer according to the semantic similarity, the degree of overlap, and the key information coverage ratio.

[0058] In some embodiments, step S3014 is specifically as follows: Semantic faithfulness = α × semantic similarity + β × degree of overlap + γ × key information coverage ratio. Where α, β, and γ represent different weight parameters.

[0059] S3015. Perform calculation processing on the semantic similarity and the semantic faithfulness based on a preset semantic weight to obtain a semantic dimension score.

[0060] In some embodiments, step S3015 is specifically as follows: Semantic dimension score = w y1 × semantic similarity + w y2 × semantic faithfulness. Where w y1 and w y2 represent weight parameters of different semantic evaluation indicators.

[0061] It can be seen from this that based on semantic similarity, it can be ensured that the content generated by the large language model is faithful to the retrieved original document, avoiding the deviation of the generated content from the facts; based on semantic fidelity, it can be determined whether the content generated by the large language model accurately reflects the information of the original document, thus ensuring the consistency between the retrieved information and the generated information.

[0062] S302. Perform multi-dimensional feature analysis on the second target answer according to the target question to obtain the answer dimension score, specifically: S3021. Calculate the answer relevance between the target question and the second target answer through a text matching algorithm based on word frequency.

[0063] In some embodiments, step S3021 is specifically: first, perform word segmentation on the target information and the target answer respectively through the Chinese word segmentation library jieba in Python, then calculate the relevance score through the BM25 algorithm, and finally standardize the relevance score to obtain the answer relevance. The BM25 algorithm is a text matching algorithm based on word frequency, which determines the relevance between the contents of two documents by the occurrence frequency of a certain word.

[0064] S3022. Calculate the coverage ratio of the number of topics in which the second target answer appears in the number of topics in which the target question appears through a latent topic model to obtain the answer integrity.

[0065] In some embodiments, step S3022 is specifically: first, perform word segmentation on the target question and the target answer respectively through the Chinese word segmentation library jieba in Python, then train the LDA model on the target question to obtain the topic distribution of the question. The LDA model regards each document as a mixture composed of multiple topics, and each topic is a probability distribution composed of a group of keywords. Map the word segmentation result of the target answer to the LDA topic space to obtain the topic distribution of the answer, and calculate the coverage ratio of the topic distribution of the answer in the topic distribution of the question to obtain the answer integrity.

[0066] S3023. Perform calculation processing on the answer relevance and the answer integrity based on the preset answer weights to obtain the answer dimension score.

[0067] In some embodiments, step S3023 is specifically: Answer dimension score = w d1 × Answer relevance + w d2 × Answer integrity. Wherein, w d1 and w d2 represent the weight parameters of different answer evaluation indicators.

[0068] It can be seen from this that based on the answer relevance, it is possible to evaluate whether the generated answer focuses on the core theme of the user's question and avoid deviation in the generated answer; based on the answer integrity, it is possible to detect whether the generated answer comprehensively covers the key information of the user's question and avoid omissions in the generated answer, thus ensuring the matching of the generated answer with the user's question.

[0069] S303. Perform sentiment analysis on the second target answer according to the feedback data to obtain a user dimension score, where the feedback data includes the interaction log and behavior data of the user after receiving the second target answer, specifically: S3031. Extract the user's sentiment tendency type from the interaction log through a preset sentiment analysis model, and generate a user sentiment polarity based on the sentiment tendency type.

[0070] S3032. Collect the user's satisfaction score from the interaction log and perform standardization processing on the behavior data.

[0071] S3033. Generate a user satisfaction based on the user sentiment polarity, satisfaction score, and standardized behavior data.

[0072] In some embodiments, the behavior data includes the interaction duration between the user and the system and the executed interaction operations. Step S3033 is specifically: user satisfaction = m1 × user sentiment polarity + m2 × satisfaction score + m3 × interaction duration + m4 × interaction operation. Where m1, m2, m3, and m4 represent different weight parameters.

[0073] S3034. Perform calculation processing on the user sentiment polarity and user satisfaction based on a preset user weight to obtain a user dimension score.

[0074] In some embodiments, step S3034 is specifically: user dimension score = w u1 × user sentiment polarity + w u2 × user satisfaction. Where w u1 and w u2 represent the weight parameters of different user evaluation indicators.

[0075] It can be seen from this that based on the user sentiment polarity, it is possible to evaluate whether the generated answer matches the user's emotional expectation, and based on the user satisfaction, it is possible to evaluate whether the generated answer meets the user's expected needs, so that the generated answer can be consistent with the user's expectations and needs both emotionally and contentually, thereby enhancing the overall user experience.

[0076] In some embodiments, the above semantic similarity, semantic faithfulness, answer relevance, answer integrity, user sentiment polarity, and user satisfaction are all evaluation indicators included in the evaluation indicator model. Each evaluation indicator is configured with a corresponding weight parameter, and the weight parameter can be set according to requirements to adjust the performance of the generated answer on different indicators.

[0077] S304. Determine the quality score according to the semantic dimension score, answer dimension score, and user dimension score. Specifically: S3041. If a historical answer to the target question is detected, compare the historical answer with the second target answer to obtain the answer duplication degree.

[0078] In some embodiments, the Jaccard Similarity method is used to determine the answer duplication degree. The Jaccard Similarity method is a statistical method for comparing the similarity between two sets.

[0079] S3042. Perform calculation processing based on the answer duplication degree and user dimension score to obtain a penalty factor.

[0080] In some embodiments, step S3042 is specifically: Penalty factor = α × answer duplication degree + β × (1 - user sentiment polarity) + γ × (1 - user satisfaction). Among them, if no historical answer to the target question is detected, the answer duplication degree is set to 0. It can be seen from this that if the generated answer has too high a duplication degree with the historical answer, the penalty factor increases; if the user sentiment polarity is negative, the penalty factor increases; if the user satisfaction is poor, the penalty factor increases.

[0081] S3043. Determine the comprehensive dimension score according to the semantic dimension score, answer dimension score, and user dimension score, and determine the quality score according to the penalty factor and the comprehensive dimension score.

[0082] In some embodiments, step S3043 is specifically: Comprehensive dimension score = semantic dimension score + answer dimension score + user dimension score, Quality score = comprehensive dimension score × (1 - penalty factor).

[0083] Please refer to Figure 4 , Embodiment 3 of the present invention is: An intelligent question-answering terminal 100 based on a retrieval-generation model, including a memory 101, a processor 102, and a computer program stored on the memory 101 and running on the processor 102. When the processor 102 executes the computer program, it implements each step in an intelligent question-answering method based on a retrieval-generation model as described in Embodiment 1 and Embodiment 2.

[0084] In summary, the present invention provides an intelligent question-answering method and a terminal based on a retrieval-generation model. When processing a user's question, it first uses the structured question-answer data in the question-answer knowledge base to quickly process common questions, ensuring low-latency response and high reliability. At the same time, it uses the flexibility and scalability of the retrieval-generation model to generate a specified answer to deeply answer complex questions. Based on the dual-retrieval architecture combining the question-answer knowledge base and the retrieval-generation model, it not only improves the response efficiency and accuracy of the answer, but also optimizes the use of computing resources, thus realizing an efficient and flexible intelligent question-answering mechanism. In addition, after the question-answer pairs generated by the retrieval-generation model are evaluated by a preset evaluation index model, the question-answer pairs that meet the target score will be updated to the question-answer knowledge base to update and expand the question-answer data in real time. Multiple evaluation dimensions are set in the evaluation index model. By adjusting the relevant weights of the evaluation dimensions, the emphasis direction of the question-answer pairs generated by the retrieval-generation model in different dimensions can be adjusted, enabling the question-answer knowledge base to be dynamically adjusted and expanded according to the changes in business requirements. At the same time, it avoids the generation of content that does not match the target question by the model, improves the accuracy and reliability of the answer, and thus enhances the user experience.

[0085] The above are only the embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. All equivalent transformations made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in the relevant technical fields, are similarly included in the patent protection scope of the present invention.

Claims

1. An intelligent question-answering method based on a retrieval-generation model, characterized in that, Including: Retrieving a first target answer that matches the target question sent by the user in the Q&A pair knowledge base; If the first target answer is not retrieved, generating a second target answer that matches the target question through a retrieval-generation model and returning the second target answer to the user; Obtaining feedback data of the user on the second target answer, and analyzing and processing the second target answer according to the target question and the feedback data by using an evaluation index model to obtain a quality score; If the quality score of the second target answer is greater than a preset target score, updating the target question and the second target answer to the Q&A pair knowledge base.

2. The method according to claim 1, wherein The retrieval-generation model includes an external knowledge source library and a large language model; Generating a second target answer that matches the target question through a retrieval-generation model includes: Retrieving retrieval information in the external knowledge source library according to the target question; Performing re-ranking processing on the retrieval information that meets the preset matching conditions to obtain target information; Invoking the large language model according to the target information to generate a second target answer.

3. The method according to claim 2, wherein The evaluation dimensions in the evaluation index model include a semantic dimension, an answer dimension, and a user dimension; Analyzing and processing the second target answer according to the target question and the feedback data by using an evaluation index model to obtain a quality score includes: Performing semantic analysis on the second target answer according to the target information to obtain a semantic dimension score; Performing multi-dimensional feature analysis on the second target answer according to the target question to obtain an answer dimension score; Performing sentiment analysis on the second target answer according to the feedback data to obtain a user dimension score; Determining a quality score according to the semantic dimension score, the answer dimension score, and the user dimension score.

4. The method according to claim 3, characterized in that, Performing semantic analysis on the second target answer according to the target information to obtain a semantic dimension score includes: Performing semantic vectorization on the target information and the second target answer respectively to calculate the cosine similarity between the target information and the second target answer, and obtaining a semantic similarity; Extracting entity sets from the target information and the second target answer respectively to calculate the overlapping degree between the target information and the second target answer; Extracting keywords from the target information and the second target answer respectively to calculate the key information coverage ratio between the target information and the second target answer; Determining the semantic faithfulness between the target information and the second target answer according to the semantic similarity, the overlapping degree, and the key information coverage ratio; Performing calculation processing on the semantic similarity and the semantic faithfulness based on a preset semantic weight to obtain a semantic dimension score.

5. The method according to claim 3, characterized in that, Performing multi-dimensional feature analysis on the second target answer according to the target question to obtain an answer dimension score includes: Calculating the answer relevance between the target question and the second target answer through a text matching algorithm based on word frequency; Calculating the coverage ratio of the number of topics that appear in the second target answer in the number of topics that appear in the target question through a latent topic model to obtain answer integrity; The answer dimension score is obtained by performing calculation processing on the answer relevance and the answer integrity based on a preset answer weight.

6. The method according to claim 3, wherein The feedback data includes the interaction log and behavior data of the user after receiving the second target answer; Performing sentiment analysis on the second target answer according to the feedback data to obtain the user dimension score includes: Extracting the emotional tendency type of the user from the interaction log through a preset sentiment analysis model, and generating a user emotional polarity based on the emotional tendency type; Collecting the satisfaction score of the user from the interaction log, and performing standardization processing on the behavior data; Generating a user satisfaction based on the user emotional polarity, the satisfaction score, and the standardized behavior data; The user dimension score is obtained by performing calculation processing on the user emotional polarity and the user satisfaction based on a preset user weight.

7. The method according to claim 3, wherein It also includes: If the quality score of the second target answer is not greater than a preset target score, storing the second target answer as the historical answer of the target question, and adjusting the prompt word of the target question according to the semantic dimension score, the answer dimension score, and the user dimension score; Based on the adjusted target question, return to execute the step of generating a second target answer that matches the target question through a retrieval-generation model.

8. The method according to claim 7, wherein Determining the quality score according to the semantic dimension score, the answer dimension score, and the user dimension score includes: If it is detected that the target question has the historical answer, comparing the historical answer with the second target answer to obtain an answer duplication degree; Performing calculation processing according to the answer duplication degree and the user dimension score to obtain a penalty factor; Determining a comprehensive dimension score according to the semantic dimension score, the answer dimension score, and the user dimension score, and determining the quality score according to the penalty factor and the comprehensive dimension score.

9. The method according to claim 1, wherein Before analyzing and processing the second target answer according to the target question and the feedback data by calling an evaluation index model to obtain a quality score, it also includes: Calling a question security verification model to detect whether the target question contains sensitive words, and if so, optimizing the sensitive words in the target question to obtain a standardized target question.

10. An intelligent question-answering terminal based on a retrieval-generation model, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements each step in the intelligent question-answering method based on a retrieval-generation model according to any one of claims 1-9.

Images