Intelligent knowledge question-answering system and method based on large language model

Abstract

The invention discloses an intelligent knowledge question-answering system and method based on a large language model, and belongs to the technical field of language processing and artificial intelligence. The method specifically comprises the following steps: S1, question input: a questioner inputs a question needing intelligent knowledge questioning and answering; s2, outputting questions and answers: outputting the questions and answers on the basis of a large language model according to questions which are input by a questioner and need to be subjected to intelligent knowledge questioning and answering; and S3, collecting user behavior data: collecting various behavior data after the user asks questions and outputs answers to the questions after the questions are completed. And judging whether the answer to the question is given or not according to a comparison result, when the value of the accuracy index about the output answer to the question is greater than a preset threshold value of the accuracy index about the output answer to the question, indicating that the answer to the question is good, and otherwise, indicating that the answer to the question is poor.

Description

Technical Field

[0001] The present invention relates to the technical fields of language processing and artificial intelligence, and particularly to an intelligent knowledge Q&A system and method based on a large language model. Background Art

[0002] In the era of information explosion, we receive a vast amount of information through the Internet every day, whether from the knowledge accumulation in the personal work process, information transmission on social media, knowledge sharing within the team, or other channels. At the same time, our ability to store, transmit, and obtain knowledge information has become unprecedentedly powerful. Traditional Q&A systems mainly rely on keyword matching or rule design. However, this method has certain limitations in dealing with complex and natural language questions. In recent years, with the continuous development of deep learning and large language models, using vectors and large language models to answer questions has become a new direction. Large language models can not only understand many human questions and instructions and smoothly conduct multi-round conversations, but also show the ability to solve various general problems in more and more fields. However, most of the existing intelligent knowledge Q&A systems lack the tracking and analysis of Q&A answers and cannot update the knowledge base in a timely manner to improve the accuracy of the answers given by the intelligent knowledge Q&A system. Summary of the Invention

[0003] Object of the Invention: The object of the present invention is to provide an intelligent knowledge Q&A system and method based on a large language model, which can solve the problem that most of the existing intelligent knowledge Q&A systems lack the tracking and analysis of Q&A answers and cannot update the knowledge base in a timely manner to improve the accuracy of the answers given by the intelligent knowledge Q&A system.

[0004] Technical Solution: To solve the above technical problems, according to one aspect of the present invention, more specifically, it is an intelligent knowledge Q&A method based on a large language model, which specifically includes the following steps:

[0005] S1. Input of the question: The questioner inputs the question that needs to be answered by intelligent knowledge.

[0006] S2. Output of the question answer: Based on the question input by the questioner that needs to be answered by intelligent knowledge, the question answer is output based on the large language model.

[0007] S3. Collection of user behavior data: Collect various behavior data of the user after asking the question and after the question answer is output after the question is completed.

[0008] S4. User Behavior Data Analysis: Analyze and process the relevant behavior data regarding copying answers, the relevant data on feedback for question answers, and the relevant behavior data on repeated questions to obtain the copying behavior index, user feedback index, and repeated question index respectively. Finally, obtain the accuracy index for the output question answers through the copying behavior index, user feedback index, and repeated question index;

[0009] S5. Threshold Setting and Real-Time Comparison: Set the threshold for the accuracy index of the output question answers and perform real-time comparison between the accuracy index of the output question answers and the pre-set threshold for the accuracy index of the output question answers;

[0010] S6. Comparison Result Feedback and Instruction Execution: Provide real-time feedback on the comparison result, determine the quality of the given question answers based on the comparison result, and issue instructions for updating the knowledge base of the large language model according to the comparison result.

[0011] Furthermore, in step S4, when analyzing the relevant behavior data regarding copying answers, the copying behavior index is obtained by acquiring the number of times each user's answer is adopted, the total number of answers of each user, the length of the text copied for each answer, the length of the text modified after copying for each answer, and the original text length of the answer given for each answer, and analyzing the above data:

[0012] ;

[0013] Among them, is the copying behavior index, is the number of times each user's answer is adopted, is the total number of answers of each user, is the total number of users, is the length of the text copied for each answer, is the length of the text modified after copying for each answer, is the original text length of the answer given for each answer, is the total number of answers.

[0014] Furthermore, in step S4, when analyzing the relevant behavior data regarding feedback on question answers, the user feedback index is obtained by acquiring the number of likes of each user, the number of dislikes of each user, and the authoritative weight of the user, and analyzing and processing the above data:

[0015] ;

[0016] Among them, is the user feedback index, is the number of likes of each user, The number of times each user steps on a post is the user's authoritative weight , and its value is determined according to the user's identity: 0.5 for ordinary users and 1.5 for expert users in the question field.

[0017] Furthermore, in step S4, when analyzing the relevant behavior data regarding repeated questions, the number of times each user repeats a question for each question they raise, the total number of questions each user raises, and the time interval between two adjacent repeated questions of the same question each user raises will be analyzed, and the above data will be used to obtain the repeated question index:

[0018] ;

[0019] Among them, is the repeated question index, is the number of times each user repeats a question for each question they raise, is the number of questions raised by the user, is the time interval between two adjacent repeated questions of the same question each user raises, is the number of times each user repeats the same question they raise, is the sensitivity coefficient of the time interval, and its value is 0.3.

[0020] Furthermore, in step S4, when performing data analysis, the accuracy index of the output question answer will be obtained through comprehensive analysis of the copy behavior index, user feedback index, repeated question index, the number of times each user deletes an answer, and the number of times each user reports an answer:

[0021] ;

[0022] Among them, is the accuracy index of the output question answer, is the number of times each user deletes an answer, is the number of times each user reports an answer.

[0023] Furthermore, in step S5, the threshold of the accuracy index of the output question answer can be set and modified at any time according to the needs in actual use.

[0024] Further, in step S6, when determining the quality of the question answer based on the comparison result, the larger the value of the accuracy index for the output question answer is than the threshold value of the accuracy index for the output question answer set in advance, the more the given question answer conforms to the question raised by the questioner. On the contrary, it means that the given question answer less conforms to the question raised by the questioner. At the same time, when the value of the accuracy index for the output question answer is less than or equal to the threshold value of the accuracy index for the output question answer set in advance, an instruction for updating the knowledge base of the large language model is issued.

[0025] According to another aspect of the present invention, there is provided an intelligent knowledge Q&A system based on a large language model, which is used to implement the above-mentioned intelligent knowledge Q&A method based on a large language model, including: a question input module, a question answer output module, a user data collection module, a threshold setting module, a core processing module, a comparison result receiving and instruction issuing module, and an instruction execution module;

[0026] The question input module: is used for the questioner to input the question that needs to be answered by intelligent knowledge.

[0027] The question answer output module: is used to output and display the question answer based on the large language model according to the question that needs to be answered by intelligent knowledge input by the questioner.

[0028] The user data collection module: is used to collect various behavior data of the user after asking questions and after the question answer is output after the question is asked.

[0029] The threshold setting module: is used to set the threshold value of the accuracy index for the output question answer.

[0030] The core processing module: is used to analyze and process the relevant behavior data regarding the copied answer, the relevant data regarding the feedback on the question answer, and the relevant behavior data regarding repeated questions, respectively obtain the copy behavior index, the user feedback index, and the repeated question index, and finally obtain the accuracy index for the output question answer through the copy behavior index, the user feedback index, and the repeated question index, and compare the accuracy index for the output question answer with the threshold value of the accuracy index for the output question answer set in advance in real time;

[0031] The comparison result receiving and instruction issuing module: is used to receive the result of the real-time comparison by the core processing module of the accuracy index for the output question answer with the threshold value of the accuracy index for the output question answer set in advance, and issue an instruction for updating the knowledge base of the large language model when the value of the accuracy index for the output question answer is less than or equal to the threshold value of the accuracy index for the output question answer set in advance.

[0032] Instruction Execution Module: It is used to receive the instruction for updating the knowledge base of the large language model issued by the Comparison Result Receiving and Instruction Issuing Module, and execute the received instruction.

[0033] Beneficial Effects: By obtaining the number of times each user's answer is adopted, the total number of answers of each user, the text length of each answer copied, the text length of each answer modified after being copied, the original text length of the answer given for each answer, and analyzing the above data to obtain the replication behavior index, by obtaining the number of likes of each user, the number of dislikes of each user, and the authoritative weight of the user, and analyzing and processing the above data to obtain the user feedback index, by obtaining the number of times each user repeats a question for each question they ask, the total number of questions of each user, and the time interval between two adjacent repetitions of the same question asked by each user, and analyzing the above data to obtain the repeated question index, and by comprehensively analyzing the replication behavior index, the user feedback index, the repeated question index, the number of times each answer is deleted by each user, and the number of times each answer is reported by each user to obtain the accuracy index of the output question answer, comparing the accuracy index of the output question answer with the threshold of the accuracy index of the output question answer preset in real time, and determining the quality of the question answer given through the comparison result. When the value of the accuracy index of the output question answer is greater than the threshold of the accuracy index of the output question answer preset, it means that the question answer given is more in line with the question raised by the questioner. On the contrary, it means that the question answer given is less in line with the question raised by the questioner; at the same time, when the value of the accuracy index of the output question answer is less than or equal to the threshold of the accuracy index of the output question answer preset, an instruction for updating the knowledge base of the large language model is issued. Description of the Drawings

[0034] Figure 1 is a schematic diagram of the method flow;

[0035] Figure 2 is a schematic diagram of the system principle. Detailed Implementation Manner

[0036] To make the technical solution of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0037] Embodiment 1

[0038] First step, question input: The questioner inputs the question that needs to be answered intelligently through the question input module.

[0039] Second step, question answer output: Through the question answer output module, based on the question input by the questioner that needs to be answered intelligently, the question answer is output and displayed based on the large language model.

[0040] Step 3: Collect user behavior data: Use the user data collection module to collect various behavior data of users after asking questions and after the questions are answered and the answers are output.

[0041] Step 4: Analyze user behavior data: Use the core processing unit to analyze and process the relevant behavior data of users regarding copying answers, the relevant data on feedback on question answers, and the relevant behavior data of repeated questions, respectively obtain the copying behavior index, user feedback index, and repeated question index, and finally obtain the accuracy index of the output question answers through the copying behavior index, user feedback index, and repeated question index.

[0042] When analyzing the relevant behavior data regarding copying answers, the number of times each user's answer is adopted, the total number of answers of each user, the length of the text copied for each answer, the length of the text modified after copying for each answer, and the original text length of the answer given for each answer will be obtained, and the copying behavior index will be obtained by analyzing the above data:

[0043] ;

[0044] Among them, is the copying behavior index, is the number of times each user's answer is adopted, is the total number of answers of each user, is the total number of users, is the length of the text copied for each answer, is the length of the text modified after copying for each answer, is the original text length of the answer given for each answer, is the total number of answers.

[0045] When analyzing the relevant behavior data regarding feedback on question answers, the number of likes of each user, the number of dislikes of each user, and the authoritative weight of the user will be obtained, and the user feedback index will be obtained by analyzing and processing the above data:

[0046] ;

[0047] Among them, is the user feedback index, is the number of likes of each user, is the number of dislikes of each user, and is the authoritative weight of the user , and its value is determined according to the user's identity: 0.5 for ordinary users and 1.5 for expert users in the question field.

[0048] When analyzing the relevant behavioral data on repeated questions, the number of times each user repeats each question they ask, the total number of questions each user asks, the time interval between two adjacent repeated questions of the same question asked by each user, and the above data are analyzed to obtain the repeated question index:

[0049] ;

[0050] Among them, is the repeated question index, is the number of times each user repeats each question they ask, is the number of questions asked by the user, is the time interval between two adjacent repeated questions of the same question asked by each user, is the number of times the user repeats the same question they ask, is the sensitivity coefficient of the time interval, and its value is 0.3.

[0051] When conducting data analysis, the accuracy index of the output question answer is obtained through comprehensive analysis of the copy behavior index, user feedback index, repeated question index, the number of times each user deletes an answer, and the number of times each user reports an answer:

[0052] ;

[0053] Among them, is the accuracy index of the output question answer, is the number of times each user deletes an answer, is the number of times each user reports an answer.

[0054] Step 5, setting and real-time comparison of the threshold: Set the threshold of the accuracy index of the output question answer through the threshold setting module, and perform real-time comparison between the accuracy index of the output question answer and the pre-set threshold of the accuracy index of the output question answer. At the same time, the threshold of the accuracy index of the output question answer can be set and modified at any time according to the needs in actual use.

[0055] Step 6, Comparison Result Feedback and Instruction Execution: The result of the real-time comparison by the core processing module between the accuracy index of the output question answer and the threshold of the accuracy index of the output question answer preset is received through the comparison result receiving and instruction issuing module, and when the value of the accuracy index of the output question answer is less than or equal to the threshold of the accuracy index of the output question answer preset, an instruction to update the large language model knowledge base is issued. Then, the instruction to update the large language model knowledge base issued by the comparison result receiving and instruction issuing module is received through the instruction execution module, and the received instruction to update the large language model knowledge base is executed.

[0056] Meanwhile, when judging the quality of the question answer through the comparison result, when the value of the accuracy index of the output question answer is greater than the threshold of the accuracy index of the output question answer preset, it means that the provided question answer is more in line with the question raised by the questioner; conversely, it means that the provided question answer is less in line with the question raised by the questioner. At the same time, when the value of the accuracy index of the output question answer is less than or equal to the threshold of the accuracy index of the output question answer preset, an instruction to update the large language model knowledge base is issued.

[0057] Embodiment 2

[0058] When obtaining the number of times each user's answer is adopted, the total number of answers of each user, the length of the text copied for each answer, the length of the text modified after copying for each answer, the original text length of the answer given for each answer, and analyzing the above data to obtain the replication behavior index, =1953, =2160, =35786, =2431 =43520, then there is:

[0059] .

[0060] Through the above calculation, it can be obtained that the replication behavior index =1.59. Among them, the larger the replication behavior index, the more in line with the question raised by the questioner the provided question answer is.

[0061] When obtaining the number of likes of each user, the number of dislikes of each user, and the authoritative weight of the user, and analyzing and processing the above data to obtain the user feedback index, =1237, =2160, then there is:

[0062] .

[0063] From the above calculations, it can be obtained that the user feedback index = . Among them, the larger the user feedback index, the more in line with the question raised by the questioner the answer to the question is.

[0064] When obtaining the repeated question index by repeating the number of times each question is repeated by each user for each question they raise, the total number of questions asked by each user, and the time interval between two adjacent repeated questions of the same question raised by each user, and analyzing the above data, = 396, = 2160, then there is:

[0065] .

[0066] From the above calculations, it can be obtained that the repeated question index = 0.18. Among them, the smaller the repeated question index, the more in line with the question raised by the questioner the answer to the question is.

[0067] When obtaining the accuracy index of the output question answer through comprehensive analysis of the copy behavior index, user feedback index, repeated question index, the number of times each answer is deleted by each user, and the number of times each answer is reported by each user, = 1.59, = , = 0.18, = 267, = 185, = 2160, then there is:

[0068] .

[0069] From the above calculations, it can be obtained that the accuracy index of the output question answer = 1.77. Among them, the larger the value of the accuracy index of the output question answer , the more in line with the question raised by the questioner the answer to the question is. At the same time, when the preset threshold of the accuracy index of the output question answer is 1.5, at this time 1.77 > 1.5, then no instruction to update the knowledge base of the large language model is issued; when the preset threshold of the accuracy index of the output question answer is 2, at this time 1.77 < 2, then an instruction to update the knowledge base of the large language model is issued.

[0070] The above-described embodiments merely represent several implementation manners of the present invention. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the present invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all fall within the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the appended claims.

Claims

1. An intelligent knowledge Q&A method based on large language models, characterized in that, Specifically, it includes the following steps: Step S1, question input: The questioner inputs a question that requires intelligent knowledge Q&A. Step S2, question answer output: Based on the question input by the questioner that requires intelligent knowledge Q&A, the question answer is output based on the large language model. Step S3, collecting user behavior data: Collecting various behavior data of the user after asking questions and after the question answer is output after the question is asked. Step S4, user behavior data analysis: Analyzing and processing the relevant behavior data regarding copying answers, the relevant data regarding feedback on the question answer, and the relevant behavior data regarding repeated questions, respectively obtaining the copy behavior index, user feedback index, and repeated question index. Finally, the accuracy index regarding the output question answer is obtained through the copy behavior index, user feedback index, and repeated question index. Step S5, setting and real-time comparison of the threshold: Setting the threshold of the accuracy index regarding the output question answer, and performing real-time comparison between the accuracy index regarding the output question answer and the pre-set threshold of the accuracy index regarding the output question answer. Step S6, feedback of comparison result and instruction execution: Providing real-time feedback on the comparison result, determining the quality of the question answer given through the comparison result, and issuing an instruction to update the knowledge base of the large language model according to the comparison result.

2. The intelligent knowledge Q&A method based on a large language model according to claim 1, wherein: In step S4, when analyzing the relevant behavior data regarding copying answers, the copy behavior index is obtained by acquiring the number of times each user's answer is adopted, the total number of answers of each user, the text length of each answer copied, the text length of each answer modified after being copied, and the original text length of the answer given for each answer, and analyzing the above data. ； Among them, is the replication behavior index, is the number of times each user's answer is adopted, is the total number of answers of each user, is the total number of users, is the length of the text copied for each answer, is the length of the text modified after each answer is copied, is the original length of the text giving the answer for each answer, is the total number of answers.

3. The intelligent knowledge Q&A method based on a large language model according to claim 2, characterized in that: In step S4, when analyzing the relevant behavior data regarding feedback on the question answer, the user feedback index is obtained by acquiring the number of likes of each user, the number of dislikes of each user, and the authoritative weight of the user, and analyzing and processing the above data. ； Among them, is the user feedback index, is the number of likes for each user, is the number of dislikes for each user, and is the user's authority weight , and its value is determined according to the user's identity: 0.5 for ordinary users and 1.5 for expert users in the question field.

4. An intelligent knowledge Q&A method based on a large language model according to claim 3, characterized in that: In step S4, when analyzing the relevant behavior data regarding repeated questions, the repeated question index is obtained by analyzing the number of times each user repeats each question they ask, the total number of questions asked by each user, and the time interval between two adjacent repeated questions of the same question asked by each user, and analyzing the above data. ； Among them, is the repeated question index, is the number of times each user repeats asking each question they raise, is the number of questions raised by the user, is the time interval between two adjacent repeated questions of the same question raised by each user, is the number of times the user repeats asking the same question, is the sensitivity coefficient of the time interval, and its value is 0.

3.

5. An intelligent knowledge Q&A method based on a large language model according to claim 1, characterized in that: In step S4, when performing data analysis, the accuracy index regarding the output question answer is obtained by comprehensively analyzing the copy behavior index, user feedback index, repeated question index, the number of times each answer is deleted by each user, and the number of times each answer is reported by each user. ； Among them, is the accuracy index for the output of the question answer, is the number of times the answer is deleted by each user, is the number of times the answer is reported by each user.

6. The intelligent knowledge Q&A method based on the large language model according to claim 1, characterized in that: In step S5, the threshold of the accuracy index regarding the output question answer can be set and modified at any time according to the needs in actual use.

7. The intelligent knowledge Q&A method based on a large language model according to claim 1, wherein: In step S6, when determining the quality of the question answer based on the comparison result, the greater the value of the precision index for the output question answer is than the threshold of the precision index for the output question answer set in advance, the more the given question answer conforms to the question raised by the questioner. Conversely, it means that the given question answer conforms less to the question raised by the questioner. At the same time, when the value of the precision index for the output question answer is less than or equal to the threshold of the precision index for the output question answer set in advance, an instruction to update the knowledge base of the large language model is issued.

8. An intelligent knowledge Q&A system based on a large language model, characterized in that, The intelligent knowledge Q&A system is used to implement an intelligent knowledge Q&A method based on a large language model according to any one of claims 1-7, including: a question input module, a question answer output module, a user data collection module, a threshold setting module, a core processing module, a comparison result receiving and instruction issuing module, and an instruction execution module; The question input module: is used for the questioner to input the question that needs to be answered intelligently. The question answer output module: is used to output and display the question answer based on the large language model according to the question for intelligent knowledge Q&A input by the questioner. The user data collection module: is used to collect various behavioral data of the user during the question asking and after the question answer is output after the question asking is completed. The threshold setting module: is used to set the threshold of the precision index for the output question answer. The core processing module: is used to analyze and process the relevant behavioral data on copying answers, the relevant data on question answer feedback, and the relevant behavioral data on repeated questions, respectively obtain the copying behavior index, the user feedback index, and the repeated question index, and finally obtain the precision index for the output question answer through the copying behavior index, the user feedback index, and the repeated question index, and compare the precision index for the output question answer with the threshold of the precision index for the output question answer set in advance in real time. The comparison result receiving and instruction issuing module: is used to receive the result of the real-time comparison by the core processing module of the precision index for the output question answer with the threshold of the precision index for the output question answer set in advance, and issue an instruction to update the knowledge base of the large language model when the value of the precision index for the output question answer is less than or equal to the threshold of the precision index for the output question answer set in advance. The instruction execution module: is used to receive the instruction to update the knowledge base of the large language model issued by the comparison result receiving and instruction issuing module and execute the received instruction.

Images