Intelligent question and answer method and system based on large model

Abstract

The application provides a large model-based intelligent question and answer method and system, which mainly proceeds from two aspects: 1, through accurate question classification and matching of different answering strategies, the answering efficiency and quality of the system are effectively improved. 2, in the "guess you want to ask" question recommendation link, the relevance and logicality of the dialogue context are fully considered, and an intention attenuation coefficient is introduced to calculate the weight of the recommended question, so as to improve the accuracy of the recommendation result. For standard answering type questions, the system searches and matches effective, concise and accurate answers in the database; for non-standard answering type questions, the system generates detailed and comprehensive answer content through the RAG method. In the recommendation process of the "guess you want to ask" question, the relevance and logicality of the current question and the historical dialogue of the user are comprehensively considered, and the current intention of the user is described through the intention attenuation coefficient, and accordingly the weight of the recommended question is adjusted, and then the rationality and accuracy of the recommendation result are improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to an intelligent question-answering method and system based on a large model. Background Technology

[0002] In today's science and technology innovation service field, intelligent question-answering systems play an important role. Currently, most intelligent question-answering systems developed in this field are general-purpose tools, mainly relying on large language models (LLM) to process and answer various questions raised by users.

[0003] However, as practical applications continue to expand and deepen, the limitations of existing intelligent question-answering systems are becoming increasingly apparent. On the one hand, users have different requirements for answers to different types of questions. However, existing tools lack effective question type recognition capabilities, which makes it difficult for the generated answers to meet the specific needs of users.

[0004] On the other hand, existing intelligent question-answering tools, in their "You Might Want to Ask" recommendation function, only focus on the relevance of user questions to historical dialogue data, completely ignoring the logic inherent in the question-and-answer context. In reality, users often follow certain logical habits when solving problems. For example, when discussing topics related to technology transfer, they usually first understand "What is technology transfer?" and then further ask questions with a logical progression, such as "How to conduct technology transfer?" and "What are the stages involved in technology transfer?" Because existing tools cannot comprehensively consider this logic, the recommended questions are disconnected from the user's actual needs, resulting in unreliable recommendation results.

[0005] Furthermore, existing tools completely fail to consider the crucial factor that user intent changes as the conversation progresses when recommending "you might want to ask" questions. In actual conversations, users' focus and needs constantly evolve, and existing tools cannot capture and adapt to these changes in real time. This results in recommendations that fail to match the user's latest intent, significantly reducing recommendation accuracy.

[0006] A patent search revealed an invention patent with publication number CN109145099 A, which discloses an artificial intelligence-based question-answering method and apparatus. The method includes: acquiring a target question sent by a user and extracting a first feature vector of the target question; querying a preset question-answering information database to obtain a pre-stored second feature vector corresponding to each question set; calculating the matching degree between the second feature vector of each question set and the first feature vector of the target question according to a preset algorithm; comparing all matching degrees with a preset threshold, obtaining the maximum matching degree greater than the threshold, and determining the target question set corresponding to the maximum matching degree; querying the question-answering information database to obtain pre-stored response text fragments corresponding to the target question set, and feeding back the response text fragments to the user. This patent obtains feature vectors for the target question and a preset question set by extracting keywords and word frequencies, ignoring the semantic information hidden in the text; it can only match the response content from pre-stored candidate text fragments. If the user's question cannot be matched with a question set that meets the threshold requirements in the preset question-and-answer information database, the user may not be able to obtain the response content; it processes each question independently in the process of outputting the response content, without considering the influence of historical dialogues; the research content is limited to how to provide feedback and answers to users, and the level of intelligence is insufficient.

[0007] In summary, given the problems of the existing technologies, researching a large-model-based intelligent question-answering method and system has become a critical task that urgently needs to be addressed. Summary of the Invention

[0008] In view of the shortcomings of the existing technology, the purpose of this invention is to provide an intelligent question answering method and system based on a large model.

[0009] The present invention provides an intelligent question-answering method based on a large model, comprising the following steps:

[0010] Step S1: Obtain the user's current problem;

[0011] Step S2: Determine the relevance and logic between the current question and the historical dialogues, and obtain the historical dialogues that are relevant to the current question and the historical dialogues that are logically consistent with the current question.

[0012] Step S3: Determine the type of the current question. If the current question is a standard answer question, match the most similar historical question in the standard question-and-answer database and output the answer corresponding to the most similar historical question as the latest answer. If the current question is a non-standard answer question, use the RAG method to generate the latest answer based on the science and technology service domain knowledge base and historical dialogues that are relevant to the current question.

[0013] Step S4: Based on the latest dialogue and historical dialogues that are relevant to the current question, match multiple relevant questions to be recommended using a relevance model; and based on the latest dialogue and historical dialogues that are logically consistent with the current question, match multiple logical questions to be recommended using a logical consistency model.

[0014] Step S5: Identify entities among multiple relevant and logically related questions to be recommended; calculate the weight of the entities and the weight of the questions to be recommended based on the concept of intent decay; reorder the questions to be recommended according to the calculated weights; and recommend the top 3 questions to the user.

[0015] Step S6: Update the historical question and answer database.

[0016] Preferably, in step S1, the user inputs a question through a dialog interface, and the system obtains the content of the question and records it as the current question Q. a , indicating the a-th question asked by the user in the chat interface.

[0017] Preferably, step S2 includes the following sub-steps:

[0018] Step S2.1: If a is 1, determine the current problem Q. a For the first question entered by the user, there is no history of dialogue, so proceed directly to steps S3 to S6;

[0019] Step S2.2: If a is not 1, then use the relevance model and the logical model to determine the current problem Q. a To determine the relevance and logical consistency of historical dialogue content (HC) with the current question (Q), we can examine its relevance and logic. a Maintaining relevance to historical dialogues, and to current issues Q. a Maintain a logical historical dialogue.

[0020] Preferably, step S2.2 includes the following sub-steps:

[0021] Step S2.2.1, Correlation Judgment: First, determine Q. a Compared with the previous round of dialogue HC a-1 The correlation score R(HC) a-1 Q a If R(HC) a-1 Q a If the value is less than the set threshold, then the current problem Q is considered to be... a Irrelevant to the content of the historical dialogue, stop judging Q. a Relevance to other historical dialogues; if R(HC) a-1 Q a If Q is greater than the threshold, then a comprehensive judgment is made. a Compared with the first two rounds of dialogue (HC) a-1 ,HCa-2 The correlation score R(HC) a-1,a-2 Q a If R(HC) a-1,a-2 Q a If the value is less than the threshold, then the current problem Q is determined. a This is irrelevant to the content prior to round a-1 of the dialogue, and the judgment of Q is stopped. a Relevance to other historical dialogues; if R(HC) a-1,a-2 Q a If Q is greater than the threshold, then continue to judge Q. a Compared with the first three rounds of dialogue (HC) a-1 ,HC a-2 ,HC a-3 The correlation score R(HC) a-1,a-2,a-3 Q a And so on, continuing to judge Q. a The relevance between the content and other historical dialogues is assessed until the relevance score falls below a threshold, at which point the assessment stops.

[0022] Step S2.2.2, Logical Judgment: First, judge Q. a Compared with the previous round of dialogue HC a-1 Logical score L(HC) a-1 Q a If L(HC) a-1 Q a If the value is less than a preset threshold, the current question is determined to be irrelevant to the content of the historical dialogue, and the process of judging Q is stopped. a The logic of dialogue with other histories; if L(HC) a-1 Q a If Q is greater than the threshold, then a comprehensive judgment is made. a Compared with the first two rounds of dialogue (HC) a-1 ,HC a-2 Logical score L(HC) a-1,a-2 Q a If L(HC) a-1,a-2 Q a If the value is less than the threshold, then the current question is determined to be irrelevant to the context before round a-1 of the dialogue, and the evaluation of Q is stopped. a The logic of dialogue with other histories; if L(HC) a-1,a-2 Q a If Q is greater than the threshold, then continue to judge Q. a Compared with the first three rounds of dialogue (HC) a-1 ,HC a-2 ,HC a-3 The logicality of L(HC) a-1,a-2,a-3 Q a And so on, continuing to judge Q. aThe logic between the dialogue and other historical content is assessed until the logic score falls below a threshold, at which point the assessment stops.

[0023] Preferably, step S3 includes the following sub-steps:

[0024] Step S3.1, if the current problem Q a If the question belongs to the standard answer category, then it will be matched with the current question Q in the predefined standard question and answer database. a The database identifies similar historical questions and outputs the answers corresponding to the most similar historical questions. The standard question and answer database stores standard answer-type questions and answers, and all data has been checked and verified by professional business personnel to ensure the validity and accuracy of the answers.

[0025] Step S3.2, if the current problem Q a For questions that fall under the non-standard answer category, the RAG method is used to generate the latest answer. Step S3.2 includes the following sub-steps:

[0026] Step S3.2.1: Call the custom-defined science and technology innovation service domain knowledge base. This knowledge base stores text fragments of policy documents and science and technology innovation service cases that have been split and cleaned. Then, combine these text fragments with the current question Q. a Convert the text into embedded vectors, calculate the similarity between the two, and return the top 30 text fragments with the highest similarity.

[0027] Step S3.2.2: Use BGE Reranker to reorder the TOP30 text fragments and retain the TOP10 text fragments of the reordering result;

[0028] Step S3.2.3: If, after step S2, it is determined that there is no problem Q that is related to the current problem Q... a If the relevant historical dialogue content (HC) is used, the top 10 text fragments will be submitted to the LLM to generate the latest response; otherwise, both the HC and the top 10 text fragments will be submitted to the LLM to generate the latest response.

[0029] Preferably, step S4 includes the following sub-steps:

[0030] Step S4.1: Match relevant historical questions from the historical question-and-answer database using a relevance model, and retain the top 10 as relevant questions to be recommended: If no relevant questions exist for the current question Q... a To save relevant historical dialogue content (HC), simply use Q. a And the top 10 most relevant questions to be recommended based on the latest input relevance model; if there is a question Q that is relevant to the current question. a To maintain relevance to historical dialogue content (HC), then HC and Q will be included. aThe latest corresponding answer is simultaneously submitted to the relevance model to match the top 10 most relevant questions to be recommended.

[0031] Step S4.2: Match historical questions that conform to the dialogue logic from the historical question-answering database using a logical model, and retain the top 10 as logically recommended questions: If no matching question exists for the current question Q... a To maintain the logical flow of historical dialogue content (HC), simply use Q. a And the corresponding latest answer input logical model matches the TOP 10 logical questions to be recommended; if there is a matching with the current question Q a To maintain the logical consistency of historical dialogue content (HC), then HC and Q... a The latest corresponding answer is simultaneously submitted to the top 10 logically related questions to be recommended by the logical model.

[0032] Preferably, step S5 includes the following sub-steps:

[0033] Step S5.1: Identify all entities E from 10 relevant questions to be recommended and 10 logical questions to be recommended;

[0034] Step S5.2: Based on the calculation result of step S2, obtain the result related to the current problem Q. a Maintain relevant historical dialogues and calculate the value of a single entity e according to the following formula. i The number of times n(e) appears in the latest conversation and relevant historical conversations i The formula includes the decay effect of user intent changing over time:

[0035]

[0036] Where t (t = 1, 2, ..., n) represents the occurrence of entity e. i In a certain dialogue turn, n represents the latest dialogue turn, and the set T contains entity e. i All dialogue rounds that occur, Represents entity e i The number of times counted in dialogue round t, i.e., the number of times entity e is counted. i It appears once. Increment the count result by one, p_decay(e i ,t) is the intentional attenuation coefficient, representing the entity e i Its effectiveness in conveying user intent diminishes as the conversation lengthens, when e i The earlier a word appears in the conversation, the weaker its effect on conveying the current user's intent. p_decay(e i The formula for calculating t is as follows:

[0037] p_decay(ei ,t)=2 -β(n-t)

[0038] Where β>0, the larger β is, the faster the degree of intention decay decreases over time.

[0039] Step S5.3: Calculate the entity weights using the TF-IDF method;

[0040] Step S5.4: Calculate the relevance weight w of the question to be recommended. r If the relevance recommendation question contains only one entity, then the entity's weight is used as the relevance recommendation question weight w. r If the relevance of the recommendation question involves multiple entities, then the average weight of these multiple entities is taken as the weight w of the relevance of the recommendation question. r ;

[0041] Step S5.5: Calculate the logical weight w of the question to be recommended. l The process involves obtaining 10 logically structured questions to be recommended, identifying the entities they contain, calculating entity weights based on the logically structured historical dialogue content, and finally obtaining the weight w of the 10 logically structured questions to be recommended. l .

[0042] Preferably, step S5.3 includes the following sub-steps:

[0043] Step S5.3.1, calculate the value of a single entity e. i The TF value is calculated using the following formula:

[0044]

[0045] Where, ∑ i n(e i This represents the total number of times all entities appear in the relevant history of conversations;

[0046] Step S5.3.2, calculate the value of a single entity e. i The IDF value is calculated using the following formula:

[0047]

[0048] The total number of dialogue rounds is the sum of the rounds of the latest dialogue and the relevant historical dialogues, including entity e. i The number of dialogue rounds is also counted within the scope of the latest dialogue and relevant historical dialogues;

[0049] Step S5.3.3: Multiply TF and IDF to obtain entity e. i weight w(e) i The calculation formula is as follows:

[0050] w(ei ) = TF(e i )×IDF(e i )

[0051] Preferably, step S6 includes the following sub-steps:

[0052] Step S6.1: If the user selects a question from the top 3 recommended questions, then the selected question is taken as the (a+1)th question Q. a+1 If the user does not select any of the top 3 recommended questions, then the question manually entered by the user will be used as the (a+1)th question Q. a+1 ;

[0053] Step S6.2, Q a+1 Execute steps S1-S5 as input;

[0054] Step S6.3, if Q a+1 If the user manually enters the value, then Q will be... a+1 The latest corresponding answer will be updated to the historical Q&A database.

[0055] This invention also provides an intelligent question-answering system based on a large model, comprising:

[0056] Module M1 retrieves the user's current problem;

[0057] Module M2 determines the relevance and logical consistency between the current question and historical dialogues, and retrieves historical dialogues that are relevant to the current question and those that are logically consistent with the current question.

[0058] Module M3 determines the type of the current question. If the current question is a standard answer question, it matches the most similar historical question in the standard question-and-answer database and outputs the answer corresponding to the most similar historical question as the latest answer. If the current question is a non-standard answer question, it uses the RAG method to generate the latest answer based on the science and technology innovation service domain knowledge base and historical dialogues that are relevant to the current question.

[0059] Module M4 matches multiple relevant questions to be recommended based on the latest dialogue and historical dialogues that are relevant to the current question through a relevance model, and matches multiple logical questions to be recommended based on the latest dialogue and historical dialogues that are logically relevant to the current question through a logicality model.

[0060] Module M5 identifies entities among multiple relevant and logically related questions to be recommended. It calculates the weights of entities and questions to be recommended based on the concept of intent decay. It then re-ranks the questions to be recommended according to the calculated weights and recommends the top 3 questions to the user.

[0061] Module M6 updates the historical question and answer database.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] 1. This invention collects a large amount of dialogue data and trains a question classifier, which can effectively distinguish question types and adopt corresponding answer strategies for different types of questions, thereby improving system operating efficiency and answer quality.

[0064] 2. This invention improves the recommendation quality of "Guess What You Want to Ask" questions in intelligent question-answering systems. In practical application scenarios, it fully considers users' habits and expectations, comprehensively integrating the relevance and logic of the question-answering context when making question recommendations. Through a deep understanding of the dialogue topic and a grasp of logical inertia, the recommendation results closely revolve around the user's focus and conform to the actual dialogue logic.

[0065] 3. This invention innovatively introduces the concept of intent decay to assign weight to the questions to be recommended, accurately depicting the dynamic changes in user intent as the conversation progresses, adjusting the recommendation strategy in real time, ensuring that the questions recommended by the system highly match the user's current real needs, and significantly improving the user interaction experience.

[0066] 4. This invention first obtains text vectors through embedding vectors, enabling more accurate capture of semantic relationships within the text. Second, it uses the RAG method to generate answers, leveraging external knowledge bases and large language model technology to generate more flexible response content for users, eliminating the need for pre-built information databases and effectively expanding the response boundaries of the question-and-answer system. Furthermore, when generating answers, the influence of historical dialogues is considered, along with their relevance and logic, to ensure the relevance and accuracy of the response content. Finally, in addition to researching how to improve answer quality, this invention also explores how to recommend "guess what you might want to ask" questions, thus improving the functional design of the question-and-answer system and enhancing its intelligent user experience. Attached Figure Description

[0067] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0068] Figure 1 This is a flowchart of an intelligent question-answering method based on a large model, as described in an embodiment of the present invention.

[0069] Figure 2 This is a flowchart of step S5 in an embodiment of the present invention. Detailed Implementation

[0070] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0071] This invention aims to improve the performance of intelligent question-answering systems, primarily through two aspects: First, by accurately classifying questions and matching different answer strategies, the system's answering efficiency and quality are effectively improved. Second, in the "Guess What You Want to Ask" question recommendation stage, the relevance and logic of the dialogue context are fully considered, and an intent decay coefficient is introduced to calculate the weight of the questions to be recommended, thereby improving the accuracy of the recommendation results. Specifically, in terms of question classification and answering, this invention trains a dedicated question classifier to classify user questions. For standard answer questions, the system searches the database and matches effective, concise, and accurate answers, thereby reducing the occurrence of "illusion" answers. For non-standard answer questions, the system generates detailed and comprehensive answer content through the RAG (Retrieval Augmentation) method, thereby improving the system's operating efficiency and answer quality. In the "Guess What You Want to Ask" question recommendation process, this invention comprehensively considers the relevance and logic between the user's current question and historical dialogues, and simultaneously uses an intent decay coefficient to characterize the user's current intent, adjusting the weight of the questions to be recommended accordingly, thereby improving the rationality and accuracy of the recommendation results.

[0072] Example 1:

[0073] Figure 1 This is a flowchart of an intelligent question-answering method based on a large model, as described in an embodiment of the present invention.

[0074] like Figure 1 As shown, this embodiment provides an intelligent question-answering method based on a large model, including the following steps:

[0075] Step S1: Obtain the user's current problem;

[0076] Specifically, the user enters a question through a dialog interface, and the system retrieves the content of the question and records it as the current question Q. a , indicating the a-th question asked by the user in the chat interface.

[0077] Step S2: Determine the relevance and logic between the current question and the historical dialogues, and obtain the historical dialogues that are relevant to the current question and the historical dialogues that are logically consistent with the current question.

[0078] Specifically, step S2 includes the following sub-steps:

[0079] Step S2.1: If a is 1, determine the current problem Q.a For the first question entered by the user, there is no history of dialogue, so there is no need to judge the relevance and logic of the context. Steps S3 to S6 are executed directly.

[0080] Step S2.2: If a is not 1, then use the relevance model and the logical model to determine the current problem Q. a To determine the relevance and logical consistency of historical dialogue content (HC) with the current question (Q), we can examine its relevance and logic. a Maintaining relevance to historical dialogues, and to current issues Q. a Maintain a logical historical dialogue.

[0081] Step S2.2: Sequentially retrieve all dialogue content from the preceding text and make judgments.

[0082] Further, step S2.2 includes the following sub-steps:

[0083] Step S2.2.1, Correlation Judgment: First, determine Q. a Compared with the previous round of dialogue HC a-1 The correlation score R(HC) a-1 Q a Before making a judgment, LLM is used to analyze the previous round of dialogue HC. a-1 The content is summarized and generalized to reduce the overlap between historical dialogue content and Q&A. a The difference in text length between R(HC) a-1 Q a If the value is less than the set threshold, then the current problem Q is considered to be... a Irrelevant to the content of the historical dialogue, stop judging Q. a Relevance to other historical dialogues; if R(HC) a-1 Q a If Q is greater than the threshold, then a comprehensive judgment is made. a Compared with the first two rounds of dialogue (HC) a-1 ,HC a-2 The correlation score R(HC) a-1,a-2 Q a Similarly, LLM is used before judgment (HC). a-1 ,HC a-2 Summarize and generalize the content of R(HC) to shorten the text length. a-1,a-2 Q a If the value is less than the threshold, then the current problem Q is determined. a This is irrelevant to the content prior to round a-1 of the dialogue, and the judgment of Q is stopped. a Relevance to other historical dialogues; if R(HC) a-1,a-2 Q a If Q is greater than the threshold, then continue to judge Q. a Compared with the first three rounds of dialogue (HC) a-1,HC a-2 ,HC a-3 The correlation score R(HC) a-1,a-2,a-3 Q a Similarly, LLM is used for (HC) a-1 ,HC a-2 ,HC a-3 Summarize and generalize the content of Q, and then, by analogy, continue to judge Q. a The relevance between the content and other historical dialogues is assessed until the relevance score falls below a threshold, at which point the assessment stops.

[0084] Step S2.2.2, the logical judgment process is similar to the relevance judgment process: first, judge Q. a Compared with the previous round of dialogue HC a-1 Logical score L(HC) a-1 Q a Before making a judgment, LLM was used to evaluate HC. a-1 Summarize and generalize the content, reducing the amount of historical dialogue content and questions. a The difference in text length between them. If L(HC) a-1 Q a If the value is less than a preset threshold, the current question is determined to be irrelevant to the content of the historical dialogue, and the process of judging Q is stopped. a The logic of dialogue with other histories; if L(HC) a-1 Q a If Q is greater than the threshold, then a comprehensive judgment is made. a Compared with the first two rounds of dialogue (HC) a-1 ,HC a-2 Logical score L(HC) a-1,a-2 Q a Similarly, LLM is used before judgment (HC). a-1 ,HC a-2 Summarize and generalize the content of L(HC) to shorten the text length. a-1,a-2 Q a If the value is less than the threshold, then the current question is determined to be irrelevant to the context before round a-1 of the dialogue, and the evaluation of Q is stopped. a The logic of dialogue with other histories; if L(HC) a-1,a-2 Q a If Q is greater than the threshold, then continue to judge Q. a Compared with the first three rounds of dialogue (HC) a-1 ,HC a-2 ,HC a-3 The logicality of L(HC) a-1,a-2,a-3 Q a Similarly, LLM is used for (HC) a-1 ,HC a-2 ,HC a-3Summarize and generalize the content of Q, and then continue to judge Q by analogy. a The logic between the dialogue and other historical content is assessed until the logic score falls below a threshold, at which point the assessment stops.

[0085] Specifically, the relevance model in step S2.2 is trained based on the BERT model. The historical dialogue data used for training is shown in Table 1. The first column represents the total number of valid dialog boxes activated by all users, and the second column represents all dialogue sequences in each dialog box where the user interacts with the system. i A i Let represent the dialogue in the i-th round. During training, the dialogue sequence is preprocessed and manually labeled. Taking the dialogue sequence (Q1A1, Q2A2, Q3A3) as an example, the first step is to organize all dialogues into the format R(C,Q), where C includes all dialogue combinations except for the dialogue containing question Q; the second step is to use LLM to summarize the key content of C, shortening the text length difference between C and question Q, and then using the summarized C and question Q as input data for model training; the third step is to manually label the relevance between C and question Q, with relevance values ​​between [0, 1], where 0 indicates no relevance and 1 indicates complete consistency, and the relevance labels are used as output data for model training. The organized training data is shown in Table 2.

[0086] Table 1 Historical Dialogue Data

[0087] NO. Dialogue sequence 1 <![CDATA[Q1A1,Q2A2,Q1A3]]> 2 <![CDATA[Q1A1,Q2Q2]]> …… ……

[0088] Table 2. Training data format for the correlation model.

[0089]

[0090] Specifically, the logic model in step S2.2 is trained based on the BERT model, using the dialogue data in Table 1. The dialogue sequences are preprocessed and manually labeled. Taking the dialogue sequence (Q1A1, Q2A2, Q3A3) as an example, the first step is to organize all question-answer pairs into the format L(C,Q), where C includes all dialogue combinations except for the dialogue containing question Q. The second step is to use LLM to summarize the key content of C, reducing the text length difference between C and question Q. Then, the summarized C and question Q are used as input data for model training. The third step is to manually label the logic between C and question Q, with values ​​between [0, 1], where 0 indicates no logic and 1 indicates perfect logic. The logic labels are used as output data for training. The processed training data is shown in Table 3.

[0091] Table 3. Logical Model Training Data Format

[0092]

[0093] Step S3: Determine the type of the current question. If the current question is a standard answer question, match the most similar historical question in the standard question-and-answer database and output the answer corresponding to the most similar historical question as the latest answer. If the current question is a non-standard answer question, use the RAG method to generate the latest answer based on the science and technology innovation service domain knowledge base and historical dialogues that are relevant to the current question.

[0094] Specifically, a problem classifier is used to determine the current problem Q. a Whether a question falls under the category of standard or non-standard answers depends on its type. For standard-answer questions, such as those related to tax incentives or project applications, the system matches effective, concise, and accurate answers from the database to reduce the likelihood of misleading responses. For non-standard-answer questions, such as those related to service resource inquiries or technology transfer scheme design, the system generates detailed and comprehensive answers using the RAG method.

[0095] The question classifier is trained based on the BERT model. The training data is a set of business questions that have been manually labeled. Each question is labeled by the user as belonging to the "standard answer" or "non-standard answer" category.

[0096] Specifically, step S3 includes the following sub-steps:

[0097] Step S3.1, if the current problem Q a If the question belongs to the standard answer category, then it will be matched with the current question Q in the predefined standard question and answer database. a The database identifies similar historical questions and outputs the answers corresponding to the most similar historical questions. The standard question-and-answer database stores standard answer pairs of questions, all of which have been checked and verified by professional business personnel to ensure the validity and accuracy of the answers.

[0098] Specifically, in step S3.1, the process of matching standard answer type questions with historical questions includes the following steps:

[0099] Step S3.1.1, present the user's current problem W a And historical issues Q h Convert the data into embedded vectors, and use the cosine similarity formula to calculate the similarity between the user question and each historical question, obtaining s. cos (Q a Q h );

[0100] Step S3.1.2: Perform full-text search using the BM25 algorithm to obtain the similarity s between the user's question and each historical question. bm (Q a Q h );

[0101] Step S3.1.3: The similarities obtained in the first two steps are weighted and summed to obtain the final similarity between the user's question and each historical question, retaining the top 10 historical questions. The weighted summation formula is S(Q...). a Q h ) = w cos ×s cos (Q a Q h )+w bm ×s bm (Q a Q h ), w cos s cos (Q a Q h The weight of w bm s bm (Q a Q h The weight of );

[0102] Step S3.1.4: Use BGE Reranker to reorder the top 10 historical questions, retain the top 1 historical question, and return its corresponding answer to the user.

[0103] Step S3.2, if the current problem Q a For questions that fall under the non-standard answer category, the RAG method is used to generate the latest answer. Step S3.2 includes the following sub-steps:

[0104] Step S3.2.1: Call the custom-defined science and technology innovation service domain knowledge base. This knowledge base stores text fragments of policy documents and science and technology innovation service cases that have been split and cleaned. Then, combine these text fragments with the current question Q. a Convert the text into embedded vectors, calculate the similarity between the two, and return the top 30 text fragments with the highest similarity.

[0105] Step S3.2.2: Use BGE Reranker to reorder the TOP30 text fragments and retain the TOP10 text fragments of the reordering result;

[0106] Step S3.2.3: If, after step S2, it is determined that there is no problem Q that is related to the current problem Q... a If the relevant historical dialogue content (HC) is used, the top 10 text fragments will be submitted to the LLM to generate the latest response; otherwise, both the HC and the top 10 text fragments will be submitted to the LLM to generate the latest response.

[0107] Step S4: Based on the latest dialogue and historical dialogues that are relevant to the current question, match 10 relevant questions to be recommended using a relevance model; and based on the latest dialogue and historical dialogues that are logically consistent with the current question, match 10 logical questions to be recommended using a logical consistency model.

[0108] Specifically, based on the latest dialogue and historical dialogue content that maintains relevance and logic, 10 questions to be recommended are matched from the historical question-and-answer database, which stores all user questions recorded by the system.

[0109] Step S4 includes the following sub-steps:

[0110] Step S4.1: Match relevant historical questions from the historical question-and-answer database using a relevance model, and retain the top 10 as relevant questions to be recommended: If no relevant questions exist for the current question Q... a To save relevant historical dialogue content (HC), simply use Q. a And the top 10 most relevant questions to be recommended based on the latest input relevance model; if there is a question Q that is relevant to the current question. a To maintain relevance to historical dialogue content (HC), then HC and Q will be included. a The latest corresponding answer is simultaneously submitted to the relevance model to match the top 10 most relevant questions for recommendation.

[0111] Step S4.2: Match historical questions that conform to the dialogue logic from the historical question-answering database using a logical model, and retain the top 10 as logically recommended questions: If no matching question exists for the current question Q... a To maintain the logical flow of historical dialogue content (HC), simply use Q. a And the corresponding latest answer input logical model matches the TOP 10 logical questions to be recommended; if there is a matching with the current question Q a To maintain the logical consistency of historical dialogue content (HC), then HC and Q... a The latest corresponding answer is simultaneously submitted to the top 10 logically related questions to be recommended by the logical model.

[0112] Step S5: Identify entities among 10 relevant questions to be recommended and 10 logical questions to be recommended. Calculate the weight of the entities and the weight of the questions to be recommended based on the concept of intent decay. Re-rank the questions to be recommended according to the calculated weights and recommend the top 3 questions to the user.

[0113] Specifically, the 20 questions to be recommended obtained in step S4 are weighted and summed to obtain a comprehensive score for each question. These scores are then reordered, and the top 3 are selected as the final "Guess You May Ask" recommended questions and returned to the user. The weighted summation formula is T(Q) = w r×R(Q)+w l ×L(Q), T(Q) represents the final score of the question Q to be recommended, w r R(Q) represents the weight of the recommendation result of the relevance model, and R(Q) represents the relevance score of the question Q to be recommended. l L(Q) represents the weight of the logical model recommendation result, and L(Q) represents the logical score of the question Q to be recommended.

[0114] The reordering process of the "Guess You Want to Ask" recommended questions in step S5 is as follows: Figure 2 As shown, the weight w of the question to be recommended is calculated using the following method. r and w l Since the calculation method is the same, we will use w here. r The calculation process will be explained using an example:

[0115] Step S5 includes the following sub-steps:

[0116] Step S5.1: Identify all entities E from 10 relevant questions to be recommended and 10 logical questions to be recommended;

[0117] Step S5.2: Based on the calculation result of step S2, obtain the result related to the current problem Q. a Maintain relevant historical dialogues and calculate the value of a single entity e according to the following formula. i The number of times n(e) appears in the latest conversation and relevant historical conversations i The formula includes the decay effect of user intent changing over time:

[0118]

[0119] Where t (t = 1, 2, ..., n) represents the occurrence of entity e. i In a certain dialogue turn, n represents the latest dialogue turn, and the set T contains entity e. i All dialogue rounds that occur, Represents entity e i The number of times counted in dialogue round t, i.e., the number of times entity e is counted. i It appears once. Increment the count result by one, p_decay(e i ,t) is the intentional attenuation coefficient, representing the entity e i Its effectiveness in conveying user intent diminishes as the conversation lengthens, when e i The earlier a word appears in the conversation, the weaker its effect on conveying the current user's intent. p_decay(e i The formula for calculating t is as follows:

[0120] p_decay(e i ,t)=2-β(n-t)

[0121] Where β>0, the larger β is, the faster the degree of intention decay decreases over time.

[0122] Step S5.3: Calculate the entity weights using the TF-IDF method.

[0123] Specifically, step S5.3 includes the following sub-steps:

[0124] Step S5.3.1, calculate the value of a single entity e. i The TF value is calculated using the following formula:

[0125]

[0126] Where, ∑ i n(e i This represents the total number of times all entities appear in the relevant history of conversations;

[0127] Step S5.3.2, calculate the value of a single entity e. i The IDF value is calculated using the following formula:

[0128]

[0129] The total number of dialogue rounds is the sum of the rounds of the latest dialogue and the relevant historical dialogues, including entity e. i The number of dialogue rounds is also counted within the scope of the latest dialogue and relevant historical dialogues.

[0130] Step S5.3.3: Multiply TF and IDF to obtain entity e. i weight w(e) i The calculation formula is as follows:

[0131] w(e i ) = TF(e i )×IDF(e i )

[0132] Step S5.4: Calculate the relevance weight w of the question to be recommended. r If the relevance recommendation question contains only one entity, then the entity's weight is used as the relevance recommendation question weight w. r If the relevance of the recommendation question involves multiple entities, then the average weight of these multiple entities is taken as the weight w of the relevance of the recommendation question. r ;

[0133] Step S5.5: Calculate the logical weight w of the question to be recommended. lThe approach is the same as above: obtain 10 logically related questions to be recommended, identify the entities they contain, calculate entity weights based on the logically related historical dialogue content, and finally obtain the weights w of the 10 logically related questions to be recommended. l .

[0134] Step S6: Update the historical question and answer database.

[0135] Specifically, step S6 includes the following sub-steps:

[0136] Step S6.1: If the user selects a question from the top 3 recommended questions, then the selected question is taken as the (a+1)th question Q. a+1 If the user does not select any of the top 3 recommended questions, then the question manually entered by the user will be used as the (a+1)th question Q. a+1 ;

[0137] Step S6.2, Q a+1 Execute steps S1-S5 as input;

[0138] Step S6.3, if Q a+1 If the user manually enters the value, then Q will be... a+1 The latest corresponding answer will be updated to the historical Q&A database.

[0139] Example 2:

[0140] The present invention also provides a large-model-based intelligent question answering system. The large-model-based intelligent question answering system can be implemented by executing the process steps of the large-model-based intelligent question answering method. That is, those skilled in the art can understand the large-model-based intelligent question answering method as a preferred implementation of the large-model-based intelligent question answering system.

[0141] Specifically, this large-model-based intelligent question-answering system includes:

[0142] Module M1 retrieves the user's current problem;

[0143] Module M2 determines the relevance and logical consistency between the current question and historical dialogues, and retrieves historical dialogues that are relevant to the current question and those that are logically consistent with the current question.

[0144] Module M3 determines the type of the current question. If the current question is a standard answer question, it matches the most similar historical question in the standard question-and-answer database and outputs the answer corresponding to the most similar historical question as the latest answer. If the current question is a non-standard answer question, it uses the RAG method to generate the latest answer based on the science and technology innovation service domain knowledge base and historical dialogues that are relevant to the current question.

[0145] Module M4 matches multiple relevant questions to be recommended based on the latest dialogue and historical dialogues that are relevant to the current question through a relevance model, and matches multiple logical questions to be recommended based on the latest dialogue and historical dialogues that are logically relevant to the current question through a logicality model.

[0146] Module M5 identifies entities among multiple relevant and logically related questions to be recommended. It calculates the weights of entities and questions to be recommended based on the concept of intent decay. It then re-ranks the questions to be recommended according to the calculated weights and recommends the top 3 questions to the user.

[0147] Module M6 updates the historical question and answer database.

[0148] Specifically, module M2 includes the following sub-modules:

[0149] Module M2.1, if a is 1, determine the current problem Q. a For the first question entered by the user, there is no history of dialogue, so there is no need to judge the relevance and logic of the context. Modules M3 to M6 are executed directly.

[0150] Module M2.2: If a is not 1, then the relevance model and the logical model are used to determine the current problem Q. a To determine the relevance and logical consistency of historical dialogue content (HC) with the current question (Q), we can examine its relevance and logic. a Maintaining relevance to historical dialogues, and to current issues Q. a Maintain a logical historical dialogue.

[0151] Module M2.2 sequentially retrieves all the preceding dialogue content for judgment.

[0152] Furthermore, module M2.2 includes the following sub-modules:

[0153] Module M2.2.1, Correlation Judgment: First, determine Q. a Compared with the previous round of dialogue HC a-1 The correlation score R(HC) a-1 Q a Before making a judgment, LLM is used to analyze the previous round of dialogue HC. a-1 The content is summarized and generalized to reduce the overlap between historical dialogue content and Q&A. a The difference in text length between R(HC) a-1 Q a If the value is less than the set threshold, then the current problem Q is considered to be... a Irrelevant to the content of the historical dialogue, stop judging Q. a Relevance to other historical dialogues; if R(HC) a-1 Q a If Q is greater than the threshold, then a comprehensive judgment is made.a Compared with the first two rounds of dialogue (HC) a-1 ,HC a-2 The correlation score R(HC) a-1,a-2 Q a Similarly, LLM is used before judgment (HC). a-1 ,HC a-2 Summarize and generalize the content of R(HC) to shorten the text length. a-1,a-2 Q a If the value is less than the threshold, then the current problem Q is determined. a This is irrelevant to the content prior to round a-1 of the dialogue, and the judgment of Q is stopped. a Relevance to other historical dialogues; if R(HC) a-1,a-2 Q a If Q is greater than the threshold, then continue to judge Q. a Compared with the first three rounds of dialogue (HC) a-1 ,HC a-2 ,HC a-3 The correlation score R(HC) a-1,a-2,a-3 Q a Similarly, LLM is used for (HC) a-1 ,HC a-2 ,HC a-3 Summarize and generalize the content of Q, and then, by analogy, continue to judge Q. a The relevance between the content and other historical dialogues is assessed until the relevance score falls below a threshold, at which point the assessment stops.

[0154] Module M2.2.2, the logical judgment process is similar to the relevance judgment process: first, judge Q. a Compared with the previous round of dialogue HC a-1 Logical score L(HC) a-1 Q a Before making a judgment, LLM was used to evaluate HC. a-1 Summarize and generalize the content, reducing the amount of historical dialogue content and questions. a The difference in text length between them. If L(HC) a-1 Q a If the value is less than a preset threshold, the current question is determined to be irrelevant to the content of the historical dialogue, and the process of judging Q is stopped. a The logic of dialogue with other histories; if L(HC) a-1 Q a If Q is greater than the threshold, then a comprehensive judgment is made. a Compared with the first two rounds of dialogue (HC) a-1 ,HC a-2 Logical score L(HC) a-1,a-2 Q a Similarly, LLM is used before judgment (HC). a-1 ,HCa-2 Summarize and generalize the content of L(HC) to shorten the text length. a-1,a-2 Q a If the value is less than the threshold, then the current question is determined to be irrelevant to the context before round a-1 of the dialogue, and the evaluation of Q is stopped. a The logic of dialogue with other histories; if L(HC) a-1,a-2 Q a If Q is greater than the threshold, then continue to judge Q. a Compared with the first three rounds of dialogue (HC) a-1 ,HC a-2 ,HC a-3 The logicality of L(HC) a-1,a-2,a-3 Q a Similarly, LLM is used for (HC) a-1 ,HC a-2 ,HC a-3 Summarize and generalize the content of Q, and then continue to judge Q by analogy. a The logic between the dialogue and other historical content is assessed until the logic score falls below a threshold, at which point the assessment stops.

[0155] Specifically, the relevance model in module M2.2 is trained based on the BERT model. The historical dialogue data used for training is shown in Table 1. The first column represents the total number of valid dialog boxes activated by all users, and the second column represents all dialogue sequences in each dialog box where the user interacts with the system. i A i Let represent the dialogue in the i-th round. During training, the dialogue sequence is preprocessed and manually labeled. Taking the dialogue sequence (Q1A1, Q2A2, Q3A3) as an example, the first step is to organize all dialogues into the format R(C,Q), where C includes all dialogue combinations except for the dialogue containing question Q; the second step is to use LLM to summarize the key content of C, shortening the text length difference between C and question Q, and then using the summarized C and question Q as input data for model training; the third step is to manually label the relevance between C and question Q, with relevance values ​​between [0, 1], where 0 indicates no relevance and 1 indicates complete consistency, and the relevance labels are used as output data for model training. The organized training data is shown in Table 2.

[0156] Table 1 Historical Dialogue Data

[0157] NO. Dialogue sequence 1 <![CDATA[Q1A1,Q2A2,Q1A3]]> 2 <![CDATA[Q1A1,Q2A2]]> …… ……

[0158] Table 2. Training data format for the correlation model.

[0159]

[0160] Specifically, the logic model in module M2.2 is trained based on the BERT model, using the dialogue data in Table 1. The dialogue sequences are preprocessed and manually labeled. Taking the dialogue sequence (Q1A1, Q2A2, Q3A3) as an example, the first step is to organize all question-answer pairs into the format L(C,Q), where C includes all dialogue combinations except for the dialogue containing question Q. The second step is to use LLM to summarize the key content of C, reducing the text length difference between C and question Q. Then, the summarized C and question Q are used as input data for model training. The third step is to manually label the logic between C and question Q, with values ​​between [0, 1], where 0 indicates no logic and 1 indicates perfect logic. The logic labels are used as output data for training. The processed training data is shown in Table 3.

[0161] Table 3. Logical Model Training Data Format

[0162]

[0163] Specifically, module M3 includes the following sub-modules:

[0164] Module M3.1, if the current problem Q a If the question belongs to the standard answer category, then it will be matched with the current question Q in the predefined standard question and answer database. a The database identifies similar historical questions and outputs the answers corresponding to the most similar historical questions. The standard question-and-answer database stores standard answer pairs of questions, all of which have been checked and verified by professional business personnel to ensure the validity and accuracy of the answers.

[0165] Specifically, in module M3.1, the process of matching standard answer type questions with historical questions includes the following modules:

[0166] Module M3.1.1 will display the user's current question Q. a And historical issues Q h Convert the data into embedded vectors, and use the cosine similarity formula to calculate the similarity between the user question and each historical question, obtaining s. cos (Q a Q h );

[0167] Module M3.1.2 uses the BM25 algorithm for full-text search to obtain the similarity s between the user's question and each historical question. bm (Q a Q h );

[0168] Module M3.1.3 performs a weighted summation of the similarities obtained in the first two steps to obtain the final similarity between the user's question and each historical question, and retains the top 10 historical questions. The weighted summation formula is S(Qa Q h ) = w cos ×s cos (W a Q h )+w bm ×s bm (Q a Q h ), w cos s cos (Q a Q h The weight of w bm s bm (Q a Q h The weight of );

[0169] Module M3.1.4 uses BGE Reranker to reorder the top 10 historical questions, retains the top historical question, and returns its corresponding answer to the user.

[0170] Module M3.2, if the current problem Q a For questions that are not in the standard answer category, the RAG method is used to generate the latest answer. Module M3.2 includes the following sub-modules:

[0171] Module M3.2.1 calls a custom knowledge base for science and technology innovation services. This knowledge base stores text fragments of policy documents and science and technology innovation service cases that have been split and cleaned. The text fragments are then linked to the current question Q. a Convert the text into embedded vectors, calculate the similarity between the two, and return the top 30 text fragments with the highest similarity.

[0172] Module M3.2.2 uses BGE Reranker to reorder the top 30 text fragments and retain the top 10 text fragments of the reordering result;

[0173] Module M3.2.3, if module M2 determines that there is no problem Q that matches the current question. a If the relevant historical dialogue content (HC) is used, the top 10 text fragments will be submitted to the LLM to generate the latest response; otherwise, both the HC and the top 10 text fragments will be submitted to the LLM to generate the latest response.

[0174] Module M4 matches 10 relevant questions to be recommended based on the latest dialogue and historical dialogues that are relevant to the current question, using a relevance model; and matches 10 logical questions to be recommended based on the latest dialogue and historical dialogues that are logically relevant to the current question, using a logicality model.

[0175] Specifically, based on the latest dialogue and historical dialogue content that maintains relevance and logic, 10 questions to be recommended are matched from the historical question-and-answer database, which stores all user questions recorded by the system.

[0176] Module M4 includes the following sub-modules:

[0177] Module M4.1 uses a relevance model to match relevant historical questions from the historical question-and-answer database, retaining the top 10 as relevant questions to be recommended: if no relevant questions exist for the current question Q... a To save relevant historical dialogue content (HC), simply use Q. a And the top 10 most relevant questions to be recommended based on the latest input relevance model; if there is a question Q that is relevant to the current question. a To maintain relevance to historical dialogue content (HC), then HC and Q will be included. a The latest corresponding answer is simultaneously submitted to the relevance model to match the top 10 most relevant questions for recommendation.

[0178] Module M4.2 uses a logical model to match historical questions from the historical question-and-answer database that conform to the dialogue logic, retaining the top 10 as logically recommended questions: if no such question exists for the current question Q... a To maintain the logical flow of historical dialogue content (HC), simply use Q. a And the corresponding latest answer input logical model matches the TOP 10 logical questions to be recommended; if there is a matching with the current question Q a To maintain the logical consistency of historical dialogue content (HC), then HC and Q... a The latest corresponding answer is simultaneously submitted to the top 10 logically related questions to be recommended by the logical model.

[0179] Module M5 identifies entities among 10 relevant and 10 logical questions to be recommended. It calculates the weights of the entities and the questions to be recommended based on the concept of intent decay. It then re-ranks the questions to be recommended according to the calculated weights and recommends the top 3 questions to the user.

[0180] Specifically, the 20 questions to be recommended obtained from module M4 are weighted and summed to obtain a comprehensive score for each question. This score is then reordered, and the top 3 are selected as the final "You Might Want to Ask" recommended questions and returned to the user. The weighted summation formula is T(Q) = w r ×R(Q)+w l ×L(Q), T(Q) represents the final score of the question Q to be recommended, w r R(Q) represents the weight of the recommendation result of the relevance model, and R(Q) represents the relevance score of the question Q to be recommended. lL(Q) represents the weight of the logical model recommendation result, and L(Q) represents the logical score of the question Q to be recommended.

[0181] The reordering process of the "Guess What You May Ask" recommendation questions in module M5 is as follows: Figure 2 As shown, the weight w of the question to be recommended is calculated using the following method. r and w l Since the calculation method is the same, we will use w here. r The calculation process will be explained using an example:

[0182] Module M5 includes the following sub-modules:

[0183] Module M5.1 identifies all entities E from 10 relevant questions to be recommended and 10 logical questions to be recommended;

[0184] Module M5.2, based on the calculation results of module M2, obtains the result related to the current problem Q. a Maintain relevant historical dialogues and calculate the value of a single entity e according to the following formula. i The number of times n(e) appears in the latest conversation and relevant historical conversations i The formula includes the decay effect of user intent changing over time:

[0185]

[0186] Where t (t = 1, 2, ..., n) represents the occurrence of entity e. i In a certain dialogue turn, n represents the latest dialogue turn, and the set T contains entity e. i All dialogue rounds that occur, Represents entity e i The number of times counted in dialogue round t, i.e., the number of times entity e is counted. i It appears once. Increment the count result by one, p_decay(e i ,t) is the intentional attenuation coefficient, representing the entity e i Its effectiveness in conveying user intent diminishes as the conversation lengthens, when e i The earlier a word appears in the conversation, the weaker its effect on conveying the current user's intent. p_decay(e i The formula for calculating t is as follows:

[0187] p_decay(e i ,t)=2 -β(n-t)

[0188] Where β>0, the larger β is, the faster the degree of intention decay decreases over time.

[0189] Module M5.3 uses the TF-IDF method to calculate the weights of entities.

[0190] Specifically, module M5.3 includes the following sub-modules:

[0191] Module M5.3.1, calculates a single entity e i The TF value is calculated using the following formula:

[0192]

[0193] Where, ∑ i n(e i This represents the total number of times all entities appear in the relevant history of conversations;

[0194] Module M5.3.2, calculates a single entity e i The IDF value is calculated using the following formula:

[0195]

[0196] The total number of dialogue rounds is the sum of the rounds of the latest dialogue and the relevant historical dialogues, including entity e. i The number of dialogue rounds is also counted within the scope of the latest dialogue and relevant historical dialogues.

[0197] Module M5.3.3 multiplies TF and IDF to obtain entity e. i weight w(e) i The calculation formula is as follows:

[0198] w(e i ) = TF(e i )×IDF(e i )

[0199] Module M5.4 calculates the relevance weight w of the question to be recommended. r If the relevance recommendation question contains only one entity, then the entity's weight is used as the relevance recommendation question weight w. r If the relevance of the recommendation question involves multiple entities, then the average weight of these multiple entities is taken as the weight w of the relevance of the recommendation question. r ;

[0200] Module M5.5 calculates the logical weight w of the question to be recommended. l The approach is the same as above: obtain 10 logically related questions to be recommended, identify the entities they contain, calculate entity weights based on the logically related historical dialogue content, and finally obtain the weights w of the 10 logically related questions to be recommended. l .

[0201] Module M6 updates the historical question and answer database.

[0202] Specifically, module M6 includes the following sub-modules:

[0203] In module M6.1, if a user selects one question from the top 3 recommended questions, then the selected question will be designated as the (a+1)th question Q. a+1 If the user does not select any of the top 3 recommended questions, then the question manually entered by the user will be used as the (a+1)th question Q. a+1 ;

[0204] Module M6.2 will Q a+1 As input, execute modules M1-M5;

[0205] Module M6.3, if Q a+1 If the user manually enters the value, then Q will be... a+1 The latest corresponding answer will be updated to the historical Q&A database.

[0206] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0207] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A smart question-answering method based on a large model, characterized in that, Includes the following steps: Step S1: Obtain the user's current problem; In step S1, the user inputs a question through a dialog interface, and the system obtains the content of the question and records it. , indicating the a-th question raised by the user in the dialog interface; Step S2: Determine the relevance and logical consistency between the current question and the historical dialogues, and obtain the historical dialogues that are relevant to the current question and the historical dialogues that are logically consistent with the current question; Step S2 includes the following sub-steps: Step S2.1: If a is 1, determine the current problem. For the first question entered by the user, there is no history of dialogue, so proceed directly to steps S3 to S6; Step S2.2: If a is not 1, then use the relevance model and the logical model to determine the current problem respectively. To understand the relevance and logic of historical dialogue content (HC) and to grasp its relevance to the current issue. Maintaining relevant historical dialogue, as well as its relevance to current issues. Maintaining a logical historical dialogue; Step S3: Determine the type of the current question. If the current question belongs to the standard answer type, match the most similar historical question in the standard question and answer database and output the answer corresponding to the most similar historical question as the latest answer. If the current question is a non-standard answer question, the RAG method is used to generate the latest answer based on the knowledge base of science and technology innovation services and the historical dialogues that are relevant to the current question; Step S4: Based on the latest dialogue and the historical dialogues that are relevant to the current question, match multiple relevant questions to be recommended using a relevance model; and based on the latest dialogue and the historical dialogues that are logically consistent with the current question, match multiple logical questions to be recommended using a logical consistency model. Step S5: Identify entities among the multiple relevant and logically related questions to be recommended, calculate the weight of the entity and the weight of the question to be recommended based on the concept of intent decay, reorder the questions to be recommended according to the calculated weights, and recommend the top 3 questions to the user. Step S6: Update the historical question and answer database.

2. The intelligent question-answering method based on a large model according to claim 1, characterized in that, Step S2.2 includes the following sub-steps: Step S2.2.1, Correlation Judgment: First, determine the correlation. Compared with the previous round of dialogue correlation score ,if If the value is less than the set threshold, then the current problem is identified. The judgment is stopped as it is unrelated to the aforementioned historical dialogue content. Relevance to other historical dialogues; if If it exceeds the threshold, then a comprehensive judgment is made. Compared to the first two rounds of dialogue correlation ,if If the value is less than the threshold, then the current problem is determined. This is irrelevant to the content prior to round a-1 of the dialogue, and the judgment stops. Relevance to other historical dialogues; if If it exceeds the threshold, continue the judgment. Compared to the first three rounds of dialogue correlation And so on, continuing to judge. The correlation between the content and other historical dialogues is evaluated until the correlation score is less than the threshold, at which point the evaluation stops. Step S2.2.2, Logical Judgment: First, determine... Compared with the previous round of dialogue Logical score ,if If the value is less than a preset threshold, the current question is determined to be irrelevant to the historical dialogue content, and the judgment is stopped. The logic of dialogue with other histories; if If it exceeds the threshold, then a comprehensive judgment is made. Compared to the first two rounds of dialogue logic ,if If the value is less than the threshold, the current question is determined to be irrelevant to the context preceding the (a-1)th round of dialogue, and the evaluation process is stopped. The logic of dialogue with other histories; if If it exceeds the threshold, continue the judgment. Compared to the first three rounds of dialogue logic And so on, continuing the judgment. The logical consistency between the dialogue and other historical content is assessed until the logical consistency score falls below the threshold, at which point the assessment stops.

3. The intelligent question-answering method based on a large model according to claim 2, characterized in that, Step S3 includes the following sub-steps: Step S3.1, if the current problem If the question belongs to the standard answer category, then it will be matched with the current question in the predefined standard question and answer database. The database identifies similar historical questions and outputs the answer corresponding to the historical question with the highest similarity. The standard question and answer database stores standard answer-type question and answer data that have all been checked and verified by professional business personnel to ensure the validity and accuracy of the answer content. Step S3.2, if the current problem For questions that fall under the non-standard answer category, the RAG method is used to generate the latest answer. Step S3.2 includes the following sub-steps: Step S3.2.1: Invoke the custom-defined science and technology innovation service domain knowledge base. This knowledge base stores text fragments of policy documents and science and technology innovation service cases that have been split and cleaned. Then, combine these text fragments with the current question. Convert the text into embedded vectors, calculate the similarity between the two, and return the top 30 text fragments with the highest similarity. Step S3.2.2: Use BGE Reranker to reorder the TOP30 text fragments and retain the TOP10 text fragments of the reordering result; Step S3.2.3: If, as determined in step S2, there is no problem related to the current issue... If the relevant historical dialogue content (HC) is obtained, the TOP10 text fragments will be submitted to LLM to generate the latest response; otherwise, both the HC and the TOP10 text fragments will be submitted to LLM to generate the latest response.

4. The intelligent question-answering method based on a large model according to claim 3, characterized in that, Step S4 includes the following sub-steps: Step S4.1: Match relevant historical questions from the historical question-and-answer database using the relevance model, and retain the top 10 as relevant questions to be recommended: if no relevant questions exist for the current question... If you want to save relevant historical dialogue content (HC), then use it directly. And the top 10 most relevant questions to be recommended based on the latest answer input and the relevance model matching it; If there is a problem related to the current issue To maintain relevance, historical dialogue content (HC) will be included. The latest corresponding answer is simultaneously submitted to the relevance model to match the top 10 most relevant questions to be recommended. Step S4.2: Match historical questions that conform to the dialogue logic from the historical question-and-answer database using the logical model, and retain the top 10 as logically recommended questions: If no matching question exists for the current question... To maintain the logical flow of historical dialogue content (HC), simply use... The latest answer input is used to match the top 10 logically related questions to be recommended by the logical model; If there is a problem related to the current issue To maintain the logical consistency of historical dialogue content (HC), then HC, The latest corresponding answer is simultaneously submitted to the top 10 logically related questions to be recommended by the logical model.

5. The intelligent question-answering method based on a large model according to claim 4, characterized in that, Step S5 includes the following sub-steps: Step S5.1: Identify all entities E from 10 relevant questions to be recommended and 10 logical questions to be recommended; Step S5.2: Based on the calculation result of step S2, obtain the result related to the current problem. Maintain relevant historical dialogues and calculate individual entities according to the following formula. Number of times it appears in the latest conversation and relevant historical conversations The formula includes the decay effect of user intent changing over time: in, Indicates the presence of an entity A certain dialogue turn, where n represents the latest dialogue turn, and the set Contains entities All dialogue rounds that occur, Representing entities The number of times counted in dialogue round t, i.e., the number of entities It appears once. Add one to the count result. It is the intentional attenuation coefficient, representing the entity Its effectiveness in conveying user intent diminishes as the conversation lengthens. The earlier a word appears in the conversation, the weaker its effect on conveying the current user's intent. The calculation formula is as follows: in, , The larger the value, the faster the degree of intention decay decreases over time; Step S5.3: Calculate the weight of the entity using the TF-IDF method; Step S5.4: Calculate the weight of the relevant question to be recommended. If the relevance recommendation question contains only one entity, then the weight of that entity is used as the weight of the relevance recommendation question. If the relevance recommendation question contains multiple entities, then the average weight of the multiple entities is taken as the weight of the relevance recommendation question. ; Step S5.5: Calculate the weight of the logically related question to be recommended. The process involves obtaining 10 logically structured questions to be recommended, identifying the entities they contain, calculating entity weights based on the logically structured historical dialogue content, and finally obtaining the weights of the 10 logically structured questions to be recommended. .

6. The intelligent question-answering method based on a large model according to claim 5, characterized in that, Step S5.3 includes the following sub-steps: Step S5.3.1, calculate a single entity The TF value is calculated using the following formula: in, This represents the total number of times all entities appear in the relevant historical dialogues; Step S5.3.2, calculate a single entity The IDF value is calculated using the following formula: The total number of dialogue rounds is the sum of the rounds of the latest dialogue and the relevant historical dialogues. Also included in the statistics within the scope of the latest dialogues and relevant historical dialogues; Step S5.3.3: Multiply TF and IDF to obtain the entity. weight The calculation formula is as follows: 。 7. The intelligent question-answering method based on a large model according to claim 6, characterized in that, Step S6 includes the following sub-steps: Step S6.1: If the user selects a question from the TOP3 recommended questions, then the selected question is taken as the (a+1)th question. If the user does not select any of the top 3 recommended questions, then the question manually entered by the user will be used as the (a+1)th question. ; Step S6.2, will Execute steps S1-S5 as input; Step S6.3, if If the user manually enters the information, then... The latest corresponding answer will be updated to the historical question and answer database.

8. An intelligent question-answering system based on a large model, characterized in that, include: Module M1 retrieves the user's current problem; In module M1, the user inputs a question through a dialog interface, and the system retrieves and records the content of the question. , indicating the a-th question raised by the user in the dialog interface; Module M2 determines the relevance and logical consistency between the current question and the historical dialogues, and obtains the historical dialogues that are relevant to the current question and the historical dialogues that are logically consistent with the current question. The module M2 includes the following sub-modules: Module M2.1, if a is 1, determine the current problem. For the first question entered by the user, there is no history of dialogue, so directly execute modules M3 to M6; Module M2.2: If a is not 1, then the relevance model and the logical model are used to determine the current problem respectively. To understand the relevance and logic of historical dialogue content (HC) and to grasp its relevance to the current issue. Maintaining relevant historical dialogue, as well as its relevance to current issues. Maintaining a logical historical dialogue; Module M3 determines the type of the current question. If the current question belongs to the standard answer type, it matches the most similar historical question in the standard question and answer database and outputs the answer corresponding to the most similar historical question as the latest answer. If the current question is a non-standard answer question, the RAG method is used to generate the latest answer based on the knowledge base of science and technology innovation services and the historical dialogues that are relevant to the current question; Module M4, based on the latest dialogue and the historical dialogues that are relevant to the current question, matches multiple relevant questions to be recommended through a relevance model, and based on the latest dialogue and the historical dialogues that are logically consistent with the current question, matches multiple logical questions to be recommended through a logicality model. Module M5 identifies entities among the multiple relevant and logically related questions to be recommended, calculates the weights of the entities and the questions to be recommended based on the concept of intent decay, reorders the questions to be recommended according to the calculated weights, and recommends the top 3 questions to the user. Module M6 updates the historical question and answer database.

Images