An intelligent knowledge question answering system and method based on large language model

Abstract

The present invention discloses an intelligent knowledge question-answering system and method based on a large language model, which belongs to the field of language processing and artificial intelligence technology. The method specifically includes the following steps: S1, question input: the questioner inputs the question that needs to be intelligently asked and answered; S2, question answer output: according to the question that needs to be intelligently asked and answered input by the questioner, the answer to the question is output based on the large language model; S3, user behavior data collection: various behavior data of the user before asking the question and after the question is completed and the answer to the question is output are collected. The quality of the answer to the question is determined by comparing the results. When the value of the accuracy index of the output answer to the question is greater than the pre-set threshold value of the accuracy index of the output answer to the question, it means that the answer to the question is good. Otherwise, it means that the answer to the question is poor.

Description

Technical Field

[0001] The present invention relates to the field of language processing and artificial intelligence technology, and in particular to an intelligent knowledge question-answering system and method based on a large language model. Background Art

[0002] In an era of information explosion, we receive massive amounts of information daily through the internet, whether accumulated through personal work, shared through social media, shared within teams, or through other channels. Simultaneously, our ability to store, transmit, and access knowledge and information has become more powerful than ever. Traditional question-answering systems primarily rely on keyword matching or rule-based design, but this approach has limitations when dealing with complex and natural language-based questions. In recent years, with the continuous development of deep learning and large language models, the use of vectors and large language models for question answering has become a new direction. Large language models can not only understand many human questions and commands, allowing for smooth multi-turn conversations, but are also demonstrating the ability to solve a wide range of common problems in a growing number of fields. However, most existing intelligent knowledge question-answering systems lack tracking and analysis of answers, and are unable to timely update the knowledge base to improve the accuracy of the answers provided by the intelligent knowledge question-answering systems. Summary of the Invention

[0003] Purpose of the invention: The purpose of the present invention is to provide an intelligent knowledge question-answering system and method based on a large language model; it can solve the problem that most existing intelligent knowledge question-answering systems lack tracking and analysis of question and answer answers, and cannot update the knowledge base in a timely manner to improve the accuracy of the answers given by the intelligent knowledge question-answering system.

[0004] Technical Solution: To solve the above technical problems, according to one aspect of the present invention, more specifically, an intelligent knowledge question answering method based on a large language model, specifically comprising the following steps:

[0005] S1. Question input: The questioner inputs the question that needs to be answered through intelligent knowledge quiz;

[0006] S2. Question answer output: Based on the questioner's input, intelligent knowledge question answering is performed and the answer is output based on the large language model;

[0007] S3. Collecting user behavior data: Collecting various behavior data of users before asking questions and after asking questions and outputting answers;

[0008] S4. User behavior data analysis: Analyze and process the behavior data related to copying answers, feedback on question answers, and repeated questioning, and derive the copying behavior index, user feedback index, and repeated questioning index, respectively. Finally, the accuracy index of the output question answer is derived from the copying behavior index, user feedback index, and repeated questioning index.

[0009] S5. Threshold setting and real-time comparison: setting a threshold for the accuracy index of the output question answer, and performing real-time comparison between the accuracy index of the output question answer and the pre-set threshold for the accuracy index of the output question answer;

[0010] S6. Comparison result feedback and instruction execution: Provide real-time feedback on the comparison results, determine the quality of the answers to the questions based on the comparison results, and issue instructions to update the large language model knowledge base based on the comparison results.

[0011] Furthermore, in step S4, when analyzing the behavior data related to copying answers, the number of times each user's answer was adopted, the total number of answers given by each user, the length of the text of each answer copied, the length of the text modified after copying each answer, and the length of the original text of each answer are obtained, and the copying behavior index is obtained by analyzing the above data:

[0012] ;

[0013] in, is the replication behavior index, The number of times each user's answer is adopted, The total number of answers for each user, is the total number of users, The length of the text copied for each answer, The length of the text copied and modified for each answer, For each answer, give the original text length of the answer, is the total number of responses.

[0014] Furthermore, in step S4, when analyzing the behavioral data related to the feedback on the answer to the question, the number of likes and dislikes of each user, and the authority weight of the user are obtained, and the user feedback index is obtained by analyzing and processing the above data:

[0015] ;

[0016] in, is the user feedback index, The number of likes for each user, The number of dislikes for each user is the user's authority weight , its value is determined according to the user identity: 0.5 for ordinary users and 1.5 for expert users in the questioning field.

[0017] Furthermore, in step S4, when analyzing the behavioral data related to repeated questioning, the number of times each user repeatedly asks each question asked to him / her, the total number of questions asked by each user, and the interval between two consecutive repeated questions asked to him / her by each user are analyzed to obtain a repeated questioning index:

[0018] ;

[0019] in, is the repeated question index, The number of times each user repeats the question for each question they ask, The number of questions asked for this user, The time interval between two consecutive repetitions of the same question asked by each user. The number of times the user has asked the same question again. is the sensitivity coefficient of the interval length, and its value is 0.3.

[0020] Furthermore, in step S4, when performing data analysis, the accuracy index of the output answer to the question is obtained by comprehensively analyzing the copy behavior index, user feedback index, repeated question index, the number of times the answer is deleted by each user, and the number of times the answer is reported by each user:

[0021] ;

[0022] in, is the accuracy index of the answer to the output question, To answer the number of times it was deleted by each user, The number of times the answer was reported by each user.

[0023] Furthermore, in step S5, the threshold value of the accuracy index of the output answer to the question can be set and modified at any time according to the needs in actual use.

[0024] Furthermore, in step S6, when determining the quality of the answer to the question by comparing the results, when the value of the precision index of the output answer to the question is greater than the preset threshold value of the precision index of the output answer to the question, it means that the answer to the question is more consistent with the question raised by the questioner; conversely, it means that the answer to the question is less consistent with the question raised by the questioner; at the same time, when the value of the precision index of the output answer to the question is less than or equal to the preset threshold value of the precision index of the output answer to the question, an instruction to update the large language model knowledge base is issued.

[0025] According to another aspect of the present invention, there is provided an intelligent knowledge question-answering system based on a large language model, which is used to implement the intelligent knowledge question-answering method based on a large language model described above, comprising: a question input module, a question answer output module, a user data collection module, a threshold setting module, a core processing module, a comparison result receiving and instruction issuing module, and an instruction execution module;

[0026] Question input module: used for the questioner to input questions that require intelligent knowledge quiz;

[0027] Question and answer output module: used to answer questions based on the questioner's input, and output and display the answers based on the large language model;

[0028] User data collection module: used to collect various behavioral data of users when they ask questions and after they finish asking questions and output answers;

[0029] Threshold setting module: used to set the threshold of the accuracy index of the output question answer;

[0030] Core processing module: used to analyze and process the relevant behavioral data on copying answers, the relevant data on feedback on question answers, and the relevant behavioral data on repeated questions, and respectively derive the copying behavior index, user feedback index, and repeated question index. Finally, the accuracy index of the output question answer is derived from the copying behavior index, user feedback index, and repeated question index, and the accuracy index of the output question answer is compared with a pre-set threshold value of the accuracy index of the output question answer in real time;

[0031] Comparison result receiving and instruction issuing module: used to receive the results of the core processing module's real-time comparison of the accuracy index of the output question answer with a pre-set threshold value of the accuracy index of the output question answer, and issue an instruction to update the large language model knowledge base when the value of the accuracy index of the output question answer is less than or equal to the pre-set threshold value of the accuracy index of the output question answer.

[0032] Instruction execution module: used to receive instructions for updating the large language model knowledge base issued by the comparison result receiving and instruction issuing module, and execute the received instructions.

[0033] Beneficial effects: by obtaining the number of times each user's answer is adopted, the total number of answers of each user, the length of the text of each answer copied, the length of the text modified after each answer is copied, and the original text length of each answer, and analyzing the above data to obtain the copying behavior index, by obtaining the number of likes of each user, the number of dislikes of each user, the authority weight of the user, and analyzing and processing the above data to obtain the user feedback index, by analyzing the number of times each user repeatedly asks the same question to each question asked by him, the total number of questions asked by each user, the interval between two consecutive repeated questions asked by each user, and analyzing the above data to obtain the repeated questioning index, and by analyzing the copying behavior index, user feedback index, repeated questioning index, the number of answers adopted, the total number of answers answered by each user, the length of time ... The number of deletions by each user and the number of reports of the answer by each user are comprehensively analyzed to obtain the precision index of the output answer to the question, and the precision index of the output answer to the question is compared with the pre-set threshold of the precision index of the output answer to the question in real time. The quality of the answer to the question is judged by the comparison result. When the value of the precision index of the output answer to the question is greater than the pre-set threshold of the precision index of the output answer to the question, it means that the answer to the question is more consistent with the question raised by the questioner, and vice versa. At the same time, when the value of the precision index of the output answer to the question is less than or equal to the pre-set threshold of the precision index of the output answer to the question, an instruction to update the large language model knowledge base is issued. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a schematic diagram of the method flow chart;

[0035] Figure 2 It is a schematic diagram of the system principle. DETAILED DESCRIPTION

[0036] In order to make the technical solution of the present invention clearer, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. Example

[0037] The first step is question input: the questioner inputs questions that require memory intelligent knowledge questions and answers through the question input module.

[0038] The second step is question answer output: the question answer output module is used to conduct intelligent knowledge question answering according to the needs of the questioner's input, and the answer to the question is output and displayed based on the large language model.

[0039] The third step is to collect user behavior data: the user data collection module is used to collect various behavior data of users before they ask questions and after they complete the questions and output the answers.

[0040] The fourth step is user behavior data analysis: the core processing unit analyzes and processes the user's behavior data on copying answers, feedback on question answers, and repeated questioning, and derives the copying behavior index, user feedback index, and repeated questioning index respectively. Finally, the accuracy index of the output question answer is derived through the copying behavior index, user feedback index, and repeated questioning index.

[0041] When analyzing the behavior data related to copying answers, we obtain the number of times each user's answer is adopted, the total number of answers each user has given, the length of the text of each answer that is copied, the length of the text that is modified after each answer is copied, and the length of the original text of each answer, and analyze the above data to obtain the copying behavior index:

[0042] ;

[0043] in, is the replication behavior index, The number of times each user's answer is adopted, The total number of answers for each user, is the total number of users, The length of the text copied for each answer, The length of the text copied and modified for each answer, For each answer, give the original text length of the answer, is the total number of responses.

[0044] When analyzing behavioral data related to feedback on question answers, we obtain the number of likes and dislikes for each user, and the user's authority weight, and analyze and process the above data to obtain the user feedback index:

[0045] ;

[0046] in, is the user feedback index, The number of likes for each user, The number of dislikes for each user is the user's authority weight , its value is determined according to the user identity: 0.5 for ordinary users and 1.5 for expert users in the questioning field.

[0047] When analyzing the behavioral data related to repeated questions, we analyze the number of times each user repeatedly asks each question, the total number of questions each user asks, and the interval between two consecutive repeated questions asked by each user. We then calculate the repeated question index:

[0048] ;

[0049] in, is the repeated question index, The number of times each user repeats the question for each question they ask, The number of questions asked for this user, The time interval between two consecutive repetitions of the same question asked by each user. The number of times the user has asked the same question again. is the sensitivity coefficient of the interval length, and its value is 0.3.

[0050] When conducting data analysis, we will comprehensively analyze the copying behavior index, user feedback index, repeated question index, the number of times the answer is deleted by each user, and the number of times the answer is reported by each user to obtain the accuracy index of the output answer:

[0051] ;

[0052] in, is the accuracy index of the answer to the output question, To answer the number of times it was deleted by each user, The number of times the answer was reported by each user.

[0053] Step 5. Threshold setting and real-time comparison: Set the threshold of the accuracy index of the output answer to the question through the threshold setting module, and compare the accuracy index of the output answer to the question with the pre-set threshold of the accuracy index of the output answer to the question in real time. At the same time, the threshold of the accuracy index of the output answer to the question can be set and modified at any time according to the needs of actual use.

[0054] Step 6, comparison result feedback and instruction execution: The comparison result receiving and instruction issuing module receives the result of the real-time comparison between the accuracy index of the output answer to the question and the pre-set threshold value of the accuracy index of the output answer to the question by the core processing module, and when the value of the accuracy index of the output answer to the question is less than or equal to the pre-set threshold value of the accuracy index of the output answer to the question, issues an instruction to update the large language model knowledge base, and then receives the instruction to update the large language model knowledge base issued by the comparison result receiving and instruction issuing module through the instruction execution module, and executes the received instruction to update the large language model knowledge base.

[0055] At the same time, when judging the quality of the answer to the question through the comparison results, when the value of the precision index of the output answer to the question is greater than the pre-set threshold value of the precision index of the output answer to the question, it means that the answer to the question is more consistent with the question raised by the questioner, and vice versa, it means that the answer to the question is less consistent with the question raised by the questioner; at the same time, when the value of the precision index of the output answer to the question is less than or equal to the pre-set threshold value of the precision index of the output answer to the question, an instruction to update the large language model knowledge base is issued. Example

[0056] By obtaining the number of times each user's answer was adopted, the total number of answers of each user, the length of the text copied for each answer, the length of the text modified after each answer was copied, and the length of the original text of each answer, and analyzing the above data to obtain the copying behavior index, =1953, =2160, =35786, =2431, =43520, then:

[0057] ;

[0058] Through the above calculation, it can be concluded that the copy behavior index =1.59. The larger the copying behavior index, the more the answer to the question is consistent with the question asked by the questioner.

[0059] By obtaining the number of likes, dislikes, and authority weight of each user, and analyzing and processing the above data to obtain the user feedback index, =1237, =2160, then:

[0060] ;

[0061] Through the above calculation, we can get the user feedback index = Among them, the larger the user feedback index is, the more the answer to the question is in line with the question raised by the questioner.

[0062] The repeated questioning index is obtained by analyzing the number of times each user repeatedly asks each question, the total number of questions asked by each user, and the interval between two consecutive repeated questions asked by each user. =396, =2160, then:

[0063]

[0064] Through the above calculation, it can be concluded that the repeated question index =0.18. The smaller the repetition index, the more the answer to the question is consistent with the question asked by the questioner.

[0065] When the accuracy index of the output answer is obtained by comprehensively analyzing the copy behavior index, user feedback index, repeated question index, the number of times the answer is deleted by each user, and the number of times the answer is reported by each user, =1.59, = , =0.18, =267, =185, =2160, then:

[0066] ;

[0067] Through the above calculation, we can get the accuracy index of the output answer to the question =1.77. Among them, the accuracy index of the output answer to the question The larger the value, the more consistent the answer to the question is with the questioner. At the same time, when the preset threshold value of the accuracy index for outputting the answer to the question is 1.5, if 1.77>1.5, the instruction to update the large language model knowledge base will not be issued; when the preset threshold value of the accuracy index for outputting the answer to the question is 2, if 1.77<2, the instruction to update the large language model knowledge base will be issued.

[0068] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. An intelligent knowledge question answering method based on a large language model, characterized in that: The specific steps include: S1. Question input: The questioner inputs the question that needs to be answered through intelligent knowledge quiz; S2. Question answer output: Based on the questioner's input, intelligent knowledge question answering is performed and the answer is output based on the large language model; S3. Collecting user behavior data: Collecting various behavior data of users before asking questions and after asking questions and outputting answers; S4. User behavior data analysis: Analyze and process the behavior data related to copying answers, feedback on question answers, and repeated questioning, and derive the copying behavior index, user feedback index, and repeated questioning index, respectively. Finally, the accuracy index of the output question answer is derived from the copying behavior index, user feedback index, and repeated questioning index. S5. Threshold setting and real-time comparison: setting a threshold for the accuracy index of the output question answer, and performing real-time comparison between the accuracy index of the output question answer and the pre-set threshold for the accuracy index of the output question answer; S6. Comparison result feedback and instruction execution: Provide real-time feedback on the comparison results, determine the quality of the answer to the question based on the comparison results, and issue instructions to update the large language model knowledge base based on the comparison results; In step S4, when performing data analysis, the accuracy index of the output answer to the question is obtained by comprehensively analyzing the copy behavior index, user feedback index, repeated question index, the number of times the answer is deleted by each user, and the number of times the answer is reported by each user: ;in, is the accuracy index of the answer to the output question, is the replication behavior index, is the user feedback index, is the repeated question index, To answer the number of times it was deleted by each user, The number of times the answer was reported by each user.

2. The intelligent knowledge question answering method based on a large language model according to claim 1, characterized in that: In step S4, when analyzing the behavior data related to copying answers, the number of times each user's answer is adopted, the total number of answers given by each user, the length of the text of each answer copied, the length of the text modified after copying each answer, and the length of the original text of each answer are obtained, and the copying behavior index is obtained by analyzing the above data: ； in, is the replication behavior index, The number of times each user's answer is adopted, The total number of answers for each user, is the total number of users, The length of the text copied for each answer, The length of the text copied and modified for each answer, For each answer, give the original text length of the answer, is the total number of responses.

3. The intelligent knowledge question answering method based on a large language model according to claim 2, characterized in that: In step S4, when analyzing the behavioral data related to the feedback on the answer to the question, the number of likes and dislikes of each user, and the authority weight of the user are obtained, and the user feedback index is obtained by analyzing and processing the above data: ； in, is the user feedback index, The number of likes for each user, The number of dislikes for each user is the user's authority weight , its value is determined according to the user identity: 0.5 for ordinary users and 1.5 for expert users in the questioning field.

4. The intelligent knowledge question answering method based on a large language model according to claim 3, characterized in that: In step S4, when analyzing the behavioral data related to repeated questioning, the number of times each user repeatedly asks each question asked to them, the total number of questions asked by each user, and the interval between two consecutive repeated questions asked to them by each user are analyzed to obtain a repeated question index: ； in, is the repeated question index, The number of times each user repeats the question for each question they ask, The number of questions asked for this user, The time interval between two consecutive repetitions of the same question asked by each user. The number of times the user has asked the same question again. is the sensitivity coefficient of the interval length, and its value is 0.

3.

5. The intelligent knowledge question answering method based on a large language model according to claim 1, characterized in that: In step S5, the threshold value of the accuracy index of the output answer to the question can be set and modified at any time according to the needs in actual use.

6. The intelligent knowledge question answering method based on a large language model according to claim 1, characterized in that: In step S6, when determining the quality of the answer to the question by comparing the results, when the value of the precision index of the output answer to the question is greater than the preset threshold value of the precision index of the output answer to the question, it means that the answer to the question is more consistent with the question raised by the questioner; conversely, it means that the answer to the question is less consistent with the question raised by the questioner; at the same time, when the value of the precision index of the output answer to the question is less than or equal to the preset threshold value of the precision index of the output answer to the question, an instruction to update the large language model knowledge base is issued.

7. An intelligent knowledge question-answering system based on a large language model, characterized in that: The system is used to implement the intelligent knowledge question-answering method based on a large language model as described in any one of claims 1 to 6, comprising: a question input module, a question answer output module, a user data collection module, a threshold setting module, a core processing module, a comparison result receiving and instruction issuing module, and an instruction execution module; Question input module: used for the questioner to input questions that require intelligent knowledge quiz; Question and answer output module: used to answer questions based on the questioner's input, and output and display the answers based on the large language model; User data collection module: used to collect various behavioral data of users when they ask questions and after they finish asking questions and output answers; Threshold setting module: used to set the threshold of the accuracy index of the output question answer; Core processing module: used to analyze and process the relevant behavioral data on copying answers, the relevant data on feedback on question answers, and the relevant behavioral data on repeated questions, and respectively derive the copying behavior index, user feedback index, and repeated question index. Finally, the accuracy index of the output question answer is derived from the copying behavior index, user feedback index, and repeated question index, and the accuracy index of the output question answer is compared with a pre-set threshold value of the accuracy index of the output question answer in real time; Comparison result receiving and instruction issuing module: used to receive the result of the real-time comparison between the accuracy index of the output answer and the preset threshold value of the accuracy index of the output answer, and issue an instruction to update the large language model knowledge base when the accuracy index of the output answer is less than or equal to the preset threshold value of the accuracy index of the output answer; Instruction execution module: used to receive instructions for updating the large language model knowledge base issued by the comparison result receiving and instruction issuing module, and execute the received instructions.

Images