Intelligent question and answer method and device and storage medium

Abstract

The embodiment of the invention provides an intelligent question and answer method and device and a storage medium, and the method comprises the steps: judging the belonging field of a question input by a user, the belonging field comprising a general knowledge field and a professional knowledge field; determining a language model adopted for analyzing the question input by the user according to the field to which the question belongs; and analyzing the user input question according to the language model, and generating a reply to the user input question. The field to which the question input by the user belongs is judged, and the questions in the professional knowledge field and the general knowledge field are analyzed by adopting different models, so that the models can adopt targeted processing strategies according to the field to which the question input by the user belongs, the continuity of multiple rounds of dialogues is ensured, and the user experience is improved. And the solving capability of the model for complex problems is improved.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and in particular relates to an intelligent question-answering method, device and storage medium. Background Technology

[0002] With the rapid development of artificial intelligence and natural language processing technologies, intelligent question-and-answer systems have been widely used in various industries, especially in public service sectors such as railways and aviation. These systems can provide users with information consultation and business processing guidance 24 hours a day, greatly improving service efficiency and reducing the pressure on human customer service representatives.

[0003] In the railway industry, intelligent question-answering systems have become a key infrastructure for optimizing service experience and ensuring operational efficiency. In principle, intelligent question-answering systems generally rely on Large Language Models (LLMs), which parse user-input questions and generate corresponding responses. In practical applications, user-input questions exhibit a significant binary characteristic. One type is general knowledge questions, which are typically high-frequency, broad in scope, and have relatively fixed answers, mainly revolving around daily travel information such as train schedules and ticketing services; these questions are often asked by ordinary passengers. The other type is professional knowledge questions, which are logically complex, require knowledge base support, and involve core knowledge areas such as scheduling procedures, signaling technology, and safety standards; these questions are usually asked by railway internal technical personnel.

[0004] In related technologies, intelligent question-answering systems typically employ a uniform approach to handle these two distinctly different types of questions. Specifically, both types of questions are indiscriminately input into a large language model, allowing the model to parse and generate responses automatically. However, the lack of differentiated processing strategies for specialized knowledge questions and general knowledge questions leads to mutual interference between the two types. This causes the model to misjudge the user's true intent (whether it's a specialized knowledge question or a general knowledge question), triggering a series of chain reactions such as interruptions in multiple rounds of dialogue and failures to solve complex problems, significantly impacting the user experience.

[0005] It should be noted that the information disclosed in the background section of this application is intended only to enhance the understanding of the general background of this application, and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0006] This application provides an intelligent question-answering method, device, and storage medium to address issues in related technologies where the lack of differentiated processing strategies for general and specialized knowledge questions leads to misjudgments of the user's true intent by the model, resulting in interruptions in multiple rounds of dialogue and failures in solving complex problems.

[0007] Firstly, embodiments of this application provide an intelligent question-answering method, including: Determine the domain to which the user input question belongs, where the domain includes general knowledge domains and professional knowledge domains; Based on the domain, determine the language model used to parse the user input question; The user input question is parsed based on the language model, and a response to the user input question is generated.

[0008] In one possible implementation, determining the language model used to parse the user input question based on the domain includes: If the user input question belongs to a general knowledge domain, then the first language model is used to parse the user input question; If the domain of the user input question is a professional knowledge domain, then a second language model is used to perform credibility matching on the knowledge subgraph related to the user input question to generate a credible knowledge subgraph. The user input question is parsed using the first language model based on the trusted knowledge subgraph; The first language model has more parameters than the second language model.

[0009] One possible implementation also includes: If the domain to which the user input question belongs is a general knowledge domain, then the user input question and the first language model's response to the user input question are stored as a first historical record; If the user input question belongs to a professional knowledge domain, then the user input question and the first language model's response to the user input question are stored as a second historical record. Wherein, the first historical record is used as the context employed by the first language model when parsing user input questions in the general knowledge domain; the second historical record is used as the context employed by the first language model when parsing user input questions in the professional knowledge domain.

[0010] In one possible implementation, the step of using a second language model to perform credibility matching on the knowledge subgraph related to the user input question to generate a credible knowledge subgraph includes: The comprehensive evaluation result of the knowledge subgraph is determined based on its topic relevance, contextual coherence, and / or semantic consistency. The credible knowledge subgraph is determined based on the comparison between the comprehensive evaluation result of the knowledge subgraph and the first preset evaluation result. Wherein, the topic relevance of the knowledge subgraph is the topic relevance between the knowledge subgraph and the user input question, the contextual coherence of the knowledge subgraph is the contextual coherence between the knowledge subgraph and the historical records, and the semantic consistency of the knowledge subgraph is the semantic consistency between the generation tendency of the first language model for the user input question and the knowledge subgraph.

[0011] In one possible implementation, after determining the credible knowledge subgraph based on the comparison between the comprehensive evaluation result of the knowledge subgraph and the first preset evaluation result, the method further includes: If there are factual conflicts among the trusted knowledge subgraphs, the comprehensive evaluation result of the trusted knowledge subgraphs shall be determined based on the authority of the data source and / or the timeliness of the update of the trusted knowledge subgraphs. Based on the comparison between the comprehensive evaluation result of the trusted knowledge subgraph and the second preset evaluation result, the preferred trusted knowledge subgraph is determined.

[0012] In one possible implementation, determining the comprehensive evaluation result of the knowledge subgraph based on its topic relevance, contextual coherence, and / or semantic consistency includes: Based on the topic relevance of the knowledge subgraph, a first evaluation result of the knowledge subgraph is determined, wherein the topic relevance is positively correlated with the first evaluation result; Based on the contextual coherence of the knowledge subgraph, a second evaluation result of the knowledge subgraph is determined, wherein the contextual coherence is positively correlated with the second evaluation result; Based on the semantic consistency of the knowledge subgraph, a third evaluation result of the knowledge subgraph is determined, wherein the semantic consistency is positively correlated with the third evaluation result; The comprehensive evaluation result of the knowledge subgraph is determined based on the weighted values ​​of the first evaluation result, the second evaluation result, and the third evaluation result.

[0013] In one possible implementation, determining the comprehensive evaluation result of the trusted knowledge subgraph based on the authority of its data source and / or the timeliness of its updates includes: Based on the authority of the data source of the trusted knowledge subgraph, a first evaluation result of the trusted knowledge subgraph is determined, and the first evaluation result of the trusted knowledge subgraph is positively correlated with the authority of the data source; Based on the update timeliness of the trusted knowledge subgraph, a second evaluation result of the trusted knowledge subgraph is determined, and the second evaluation result of the trusted knowledge subgraph is positively correlated with the update timeliness of the trusted knowledge subgraph. The comprehensive evaluation result of the trusted knowledge subgraph is determined by weighting the first evaluation result and the second evaluation result of the trusted knowledge subgraph.

[0014] In one possible implementation, determining the reliable knowledge subgraph based on a comparison between the comprehensive evaluation result of the knowledge subgraph and the first preset evaluation result includes: The knowledge subgraph whose comprehensive evaluation result is better than the first preset evaluation result is determined as the credible knowledge subgraph; In one possible implementation, determining the preferred trusted knowledge subgraph based on a comparison between the comprehensive evaluation result of the trusted knowledge subgraph and the second preset evaluation result includes: The trusted knowledge subgraph whose comprehensive evaluation result is better than the second preset evaluation result is determined as the preferred trusted knowledge subgraph.

[0015] Secondly, embodiments of this application provide an electronic device, including: processor; Memory; And a computer program, wherein the computer program is stored in the memory, and when the computer program is executed by the processor, causes the electronic device to perform the method described in any one of the first aspects.

[0016] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of the first aspects.

[0017] In this embodiment, by determining the domain of the user's input question and using different models to analyze questions in the professional knowledge domain and the general knowledge domain, the model can adopt targeted processing strategies according to the domain of the user's input question, ensuring the continuity of multi-turn dialogue and improving the model's ability to solve complex problems. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating an intelligent question-answering method provided in an embodiment of this application; Figure 2 A flowchart illustrating another intelligent question-answering method provided in an embodiment of this application; Figure 3 A flowchart illustrating a method for credibility matching of a knowledge subgraph provided in this application embodiment; Figure 4 A schematic flowchart illustrating a preferred trusted knowledge subgraph determination method provided in this application embodiment; Figure 5 A flowchart illustrating a method for separately storing historical records, provided in an embodiment of this application; Figure 6 A structural diagram of an intelligent question-answering system provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0021] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0022] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0023] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0024] In the railway industry, intelligent question-answering systems have become a key infrastructure for optimizing service experience and ensuring operational efficiency. In principle, intelligent question-answering systems generally rely on Large Language Models (LLMs), which parse user input questions and generate corresponding responses. In practical applications, by pre-training the LLM, it gains the ability to recognize user input questions and provide matching responses, thus enabling rapid responses to user input.

[0025] In railway Q&A scenarios, user-inputted questions exhibit a significant binary characteristic. Specifically, one category consists of general knowledge questions. Examples include: "Please help me check how to change this ticket," "What's the difference between seat D and seat F on a high-speed train?", "Which stations does train G3 stop at?", "Can I get a ticket on the waitlist?", and "What is the refund fee?". These questions are characterized by high frequency, low complexity, and relatively fixed answers. They mainly revolve around information closely related to the daily travel of ordinary passengers, such as train schedule inquiries, ticket refund and change procedures, seat type distinctions, waitlist rules, station facilities, and regulations regarding carry-on items. The questioners for this type of question are primarily ordinary passengers whose core need is to quickly obtain accurate, concise, and practical information, without requiring complex logical deductions or professional knowledge.

[0026] Another category consists of questions requiring specialized knowledge. For example, questions like "What should be done when the CTCS-3 level train control system loses track occupancy?", "What industry standards should be followed in determining the priority of dispatching orders during train arrival and departure operations?", "What detection technology and repair plan should be adopted when uneven settlement occurs on high-speed railway ballastless tracks?", "What are the common methods for troubleshooting logical errors in railway signal interlocking systems?", "What are the specific regulations regarding the frequency and responsibility division of track inspections by engineering sections in the 'Railway Safety Production Responsibility System Management Measures'?", and "How are the feeder protection settings of traction substations adjusted under AT power supply? What is the logical sequence of protection actions when a permanent grounding fault occurs in the contact network?" demonstrate that these questions are logically complex, require knowledge base support, and often involve specialized knowledge areas such as dispatching procedures, signaling technology, and safety standards. The answers are not common sense but require accurate knowledge base retrieval by the system. The questioners are typically professional technical personnel within the railway system.

[0027] In related technologies, a unified processing flow is typically used for both specialized knowledge questions and general knowledge questions. Specifically, inputting both types of questions into a large language model and letting the model parse and generate responses automatically will obviously lead to many problems.

[0028] On the one hand, because large language models lack the ability to classify the intent of user input questions (distinguishing between professional knowledge questions and general knowledge questions), they are easily misled by common words in the questions, and may misjudge professional knowledge questions as general knowledge questions, or respond to basic needs with professional terms.

[0029] On the other hand, the coherence of multi-turn dialogues relies on the analysis of the historical context of the dialogue by large language types. However, under a unified processing flow, the cross-input of the two types of questions will disrupt the coherence of the context. When users switch question types in the dialogue (such as asking about ticketing first and then about technical issues), the model is prone to losing its core focus due to the mixing of contextual information, failing to form a logical loop, and ultimately causing the dialogue to be forced to stop.

[0030] Furthermore, for complex professional knowledge questions, it is often necessary to rely on knowledge bases for reference. However, since large language models lack the ability to classify the intent of user input questions, they do not actively search the knowledge base for professional knowledge questions. As a result, when faced with professional questions that require logical deduction and multi-dimensional information integration, large language models often output superficial and vague responses, failing to effectively solve the actual work problems of professionals.

[0031] To address the aforementioned issues, this application provides an intelligent question-answering method. By determining the domain of the user-input question and employing different models for parsing questions in specialized knowledge domains and general knowledge domains, the model can adopt targeted processing strategies based on the domain of the user-input question, ensuring the continuity of multi-turn dialogues and improving the model's ability to solve complex problems.

[0032] See Figure 1 The above is a flowchart illustrating an intelligent question-answering method provided in the application embodiment, as shown below. Figure 1 As shown, the method specifically includes the following steps.

[0033] Step S101: Determine the domain of the user's input question.

[0034] As mentioned above, directly inputting user input questions into the model without differentiation will obviously lead to various problems. Therefore, in this embodiment of the application, it is first necessary to determine the domain to which the user input question belongs. The domain includes general knowledge domains and professional knowledge domains, so that the system can adopt different processing strategies for different domains in the subsequent step S102.

[0035] To determine the domain of user input questions, this application employs an intent classification model based on the BERT (Bidirectional Encoder Representation from Transformers) architecture. This model discriminates the intent of the user input question, thereby determining whether it belongs to a general knowledge domain or a specialized knowledge domain. For example, when a user inputs the question "How do I cancel my ticket?", the intent classification model accurately determines that the question belongs to the general knowledge domain; however, when the user inputs the question "When a single point of failure occurs in the railway GSM-R network, how does its redundancy backup mechanism switch? What impact will the switching process have on train operations?", the intent classification model accurately determines that the question belongs to the specialized knowledge domain.

[0036] In practical applications, the aforementioned intent classification model first needs to be fine-tuned and trained on railway-related corpora to ensure that it has strong domain discrimination capabilities and can accurately capture keywords and semantic patterns in user input questions, thereby achieving high-precision judgment of the domain to which the user input questions belong.

[0037] Of course, those skilled in the art can adjust the method for determining the domain of user input questions according to their needs, such as changing the architecture of the intent classification model. This application does not impose specific limitations on this.

[0038] Step S102: Determine the language model to be used to parse user input questions based on the domain.

[0039] In this embodiment, different language models are used to parse the user input question according to its domain, thereby formulating targeted processing strategies for different domains of user input questions. It is understood that the language model should have the ability to parse user input questions within its domain, thus ensuring correct parsing and accurate and reasonable responses to user input questions. Here, "parsing" includes key processing steps such as keyword extraction, semantic understanding, and logical relationship analysis of the user input question.

[0040] In practical applications, for questions in specialized knowledge domains, a specialized knowledge model can be used. This model can be pre-trained with various specialized knowledge within the railway field, ensuring it possesses sufficient knowledge in that domain and can provide accurate responses to specialized knowledge questions. Furthermore, a knowledge base retrieval module can be added to the specialized knowledge model, enabling it to retrieve knowledge not covered during training, thereby improving its practicality. For questions in general knowledge domains, existing large language models can be used. For example, the ChatGPT interface can be directly called; that is, general knowledge domain questions can be directly processed using ChatGPT. Since ChatGPT has been trained on a large general knowledge base, it can obviously provide accurate responses to general knowledge questions in the railway field.

[0041] At the same time, because questions on specialized domain knowledge and questions on general domain knowledge are handled separately, the two types of questions will not overlap, thus ensuring the continuity of multi-round dialogues.

[0042] Of course, those skilled in the art can adjust the selection of the model according to actual needs, such as using multiple language models. This application does not impose specific restrictions on this.

[0043] Step S103: Parse the user input question according to the language model and generate a response to the user input question.

[0044] Based on the language model determined by the domain of the user input question, the user input question is parsed, and a response is generated. As mentioned above, this language model should have the ability to parse user input questions within its domain, so it can obviously generate reasonable and accurate answers to user input questions, thereby effectively meeting the user's inquiry needs.

[0045] It should be noted that the above problem analysis takes the railway field as an example. Those skilled in the art can adjust the application field of the intelligent question answering method according to their needs, and this application does not impose specific restrictions on it.

[0046] In this embodiment, by determining the domain of the user's input question and using different models to analyze questions in the professional knowledge domain and the general knowledge domain, the model can adopt targeted processing strategies according to the domain of the user's input question, ensuring the continuity of multi-turn dialogue and improving the model's ability to solve complex problems.

[0047] To enable intelligent question-answering systems to provide more accurate answers to user-input questions, multiple models can be used collaboratively to handle specialized knowledge questions, while a single model can be used to handle general knowledge questions.

[0048] See Figure 2 This is a flowchart illustrating another intelligent question-answering method provided in an embodiment of this application, as shown below. Figure 2 As shown, in Figure 1 Based on the aforementioned embodiment, step S102 specifically includes the following steps.

[0049] Step S201: If the domain of the user input question is a general knowledge domain, then the first language model is used to parse the user input question.

[0050] In the embodiments of this application, the first language model is a large language model, that is, a model that is pre-trained on a wide range of general corpora and has strong common sense question answering and natural language understanding capabilities (such as ChatGPT). It is obvious that using the first language model to parse user input questions in the general knowledge domain is sufficient to meet the user's question answering needs. That is, the first language model can generate reasonable and accurate responses to questions in the general knowledge domain.

[0051] Step S202: If the domain of the user input question is a professional knowledge domain, then the second language model is used to perform credibility matching on the knowledge subgraph related to the user input question to generate a credible knowledge subgraph.

[0052] In this embodiment, the second language model is a semantic matching model. It is understood that the first language model has a larger number of parameters than the second language model.

[0053] In practical applications, if the user input question is a professional knowledge question, it is often necessary to retrieve the knowledge subgraph of the user input question and parse the user input question based on the knowledge subgraph. This can alleviate the problem of illusion (i.e., generating content that is inconsistent with the facts, fictitious, or misleading) when the model answers questions in the professional knowledge domain.

[0054] In related technologies, Retrieval-Augmented Generation (RAG) is commonly used to retrieve knowledge subgraphs from user-input questions. This involves dynamically retrieving contextual information relevant to the current question from structured knowledge bases, document collections, or railway regulations databases, and integrating this external knowledge into the generation process to enhance the factual basis and interpretability of the model's output. However, the parameterized knowledge implicitly stored within the large language model through pre-training may be inconsistent with the explicit knowledge obtained from external retrieval in terms of timeliness, expression, or factual content. In some cases, the information retrieved by RAG may even conflict with the internal knowledge of the large language model, causing the model to become self-contradictory and generate incorrect responses.

[0055] Therefore, in this embodiment, a second language model is used to perform credibility matching on the knowledge subgraph related to the user input question, generating a credible knowledge subgraph. It can be understood that by using a semantic matching model to filter out knowledge subgraphs consistent with the knowledge within the large language model (i.e., credible knowledge subgraphs), interference from erroneous information is suppressed, thereby ensuring that subsequent user input questions are correctly parsed based on the credible knowledge subgraph, and accurate and reasonable responses are generated.

[0056] Step S203: Use a first language model to parse the user input question based on the trusted knowledge subgraph.

[0057] In this embodiment, for user input questions in the professional knowledge domain, the first language model is still used as the executor for parsing the questions. Since the first language model already has the ability to parse questions in the general knowledge domain, it only lacks relevant knowledge in the professional knowledge domain. Therefore, using the first language model to parse user input questions based on the reliable knowledge subgraph generated by the second language model can avoid the manpower and resources required to train another professional knowledge domain model, while also ensuring the correct parsing of professional knowledge domain questions and generating reasonable and accurate responses.

[0058] Of course, those skilled in the art can adjust the specific selection of the first language model and the second language model according to their needs, and the embodiments of this application do not impose specific restrictions on this.

[0059] In this embodiment, the first language model and the second language model work together to parse user input questions. This not only retains the strong generalization ability of the large language model in general scenarios, but also enhances the accuracy, controllability and interpretability of the intelligent question answering method in professional fields through the semantic matching model. This effectively avoids problems such as the lack of professional knowledge that may occur when a single model is used in cross-domain tasks.

[0060] In practical applications, in order to ensure that the second language model can generate reliable and trustworthy knowledge subgraphs, an effective method for trustworthiness matching of knowledge subgraphs is needed.

[0061] See Figure 3 This is a schematic flowchart of a method for credibility matching of a knowledge subgraph provided in an embodiment of this application, as shown below. Figure 3 As shown, Figure 2 In the illustrated embodiment, step S202 specifically includes the following steps.

[0062] Step S301: Determine the comprehensive evaluation result of the knowledge subgraph based on its topic relevance, contextual coherence, and / or semantic consistency.

[0063] Specifically, the topic relevance of the knowledge subgraph refers to its relevance to the user input question. By judging the topic relevance of the knowledge subgraph to the user input question, we can determine whether the knowledge subgraph is highly relevant to the user input question. Based on this, we can filter out knowledge subgraphs that are not relevant or have low relevance to the user input question, thus preventing irrelevant knowledge subgraphs from affecting the model's parsing of the user input question. The contextual coherence of the knowledge subgraph refers to its contextual coherence with the historical records. By judging whether the knowledge subgraph is coherent with the historical records, we can obtain knowledge subgraphs with high historical coherence and filter out knowledge subgraphs with low coherence, ensuring that the knowledge subgraphs will not cause topic jumps when the model responds to the user input question. The semantic consistency of the knowledge subgraph refers to the semantic consistency between the first language model's generation tendency for the user input question and the knowledge subgraph. By judging semantic consistency, we can identify potential knowledge conflicts and filter out knowledge subgraphs with low semantic consistency, while retaining knowledge subgraphs with high semantic consistency, ensuring that the knowledge subgraphs will not conflict with the model's internal knowledge and avoiding contextual breaks and jumps.

[0064] Based on the above three indicators, knowledge subgraphs can be evaluated in a multi-dimensional and comprehensive manner, generating comprehensive evaluation results and providing a reliable data foundation for determining credible knowledge subgraphs based on the comprehensive evaluation results.

[0065] In practical applications, in order to accurately analyze the impact of the above evaluation indicators on the comprehensive evaluation results, it is often necessary to first evaluate the evaluation results of the topic relevance, contextual coherence and / or semantic consistency of the knowledge subgraph separately; at the same time, in order to facilitate quantification, the evaluation results are usually expressed in the form of a score.

[0066] Therefore, in one possible implementation, the first evaluation result of the knowledge subgraph is determined based on the topic relevance of the knowledge subgraph; the second evaluation result is determined based on the contextual coherence of the knowledge subgraph; the third evaluation result is determined based on the semantic consistency of the knowledge subgraph; and the comprehensive evaluation result of the knowledge subgraph is determined based on the weighted value of the first evaluation result, the second evaluation result, and the third evaluation result.

[0067] Among them, the topic relevance of the knowledge subgraph is positively correlated with the first evaluation result, the contextual coherence of the knowledge subgraph is positively correlated with the second evaluation result, and the semantic consistency of the knowledge subgraph is positively correlated with the third evaluation result.

[0068] It is understandable that by determining the first, second, and third evaluation results of a knowledge subgraph, the knowledge subgraph can be evaluated from multiple dimensions. At the same time, the weights of different evaluation results can be adjusted according to actual needs to achieve differentiated screening strategies and improve the flexibility of determining credible knowledge subgraphs.

[0069] Step S302: Determine the credible knowledge subgraph based on the comparison between the comprehensive evaluation result of the knowledge subgraph and the first preset evaluation result.

[0070] In practical applications, it is usually necessary to analyze the comprehensive evaluation results of knowledge subgraphs, determine the quality of the comprehensive evaluation results, and select reliable knowledge subgraphs for subsequent parsing of user input questions. Therefore, reliable knowledge subgraphs are determined based on the comparison between the comprehensive evaluation results of the knowledge subgraphs and the first preset evaluation results. It can be understood that the first preset evaluation results are the reference knowledge subgraph evaluation results used in this application embodiment, used to judge the quality of the comprehensive evaluation results of the knowledge subgraphs.

[0071] In one possible implementation, a knowledge subgraph whose overall evaluation result is better than a first preset evaluation result is determined as a trustworthy knowledge subgraph. It can be understood that if the overall evaluation result of a knowledge subgraph is better than the first preset evaluation result, it indicates that the overall evaluation result of the knowledge subgraph is good and can be used as a trustworthy knowledge subgraph.

[0072] Of course, those skilled in the art can adjust the logic of determining the credible knowledge subgraph based on the comparison results according to their needs. For example, a knowledge subgraph whose comprehensive evaluation result is close to (the difference is less than a certain value) the first preset evaluation result can be determined as a credible knowledge subgraph. This application does not impose specific restrictions on this.

[0073] Of course, those skilled in the art can also adjust the method of determining the trusted knowledge subgraph according to their needs, such as determining the trusted knowledge subgraph together with the evaluation results of topic relevance, contextual coherence and / or semantic consistency indicators and the comprehensive evaluation results. This application embodiment does not impose specific limitations on this.

[0074] In practical applications, after determining the trusted knowledge subgraphs, the first language model parses the user input question based on these subgraphs. However, because conflict verification is not performed between trusted knowledge subgraphs, issues such as factual conflicts may arise. In this case, if conflicting trusted knowledge subgraphs are... Figure 1 Even with the same input to the first language model, problems still arise in the first language model's parsing of user input questions. Therefore, further processing of the trusted knowledge subgraph is needed to eliminate conflicts and other issues between trusted knowledge subgraphs.

[0075] See Figure 4 This is a schematic flowchart illustrating a method for determining a preferred trusted knowledge subgraph provided in an embodiment of this application. Figure 4 As shown, the method specifically includes the following steps.

[0076] Step S401: If there are factual conflicts between trusted knowledge subgraphs, the comprehensive evaluation result of the trusted knowledge subgraphs is determined based on the authority of the data source and / or the timeliness of the update of the trusted knowledge subgraphs.

[0077] In this embodiment, the comprehensive evaluation result of a trusted knowledge subgraph is determined based on the authority of its data source and / or the timeliness of its update. The authority of the data source refers to the proportion of data in the trusted knowledge subgraph originating from official documents. This indicates the trustworthiness of the subgraph; a higher proportion originating from official documents signifies greater trustworthiness. By assessing the authority of the source, trusted knowledge subgraphs can be effectively further filtered. The timeliness of the update refers to the update timestamp of the trusted knowledge subgraph. A timestamp closer to the actual time indicates higher timeliness. By assessing the timeliness of the update, trusted knowledge subgraphs with lower timeliness can be filtered out, while those with higher timeliness are retained, thus ensuring that the trusted knowledge subgraph is relatively recent.

[0078] It is understandable that the comprehensive evaluation result of the credible knowledge subgraph can be determined based on either the authority of the data source or the timeliness of the update, or both.

[0079] In practical applications, the determination of whether a factual conflict exists can be accomplished by a trained conflict detection module, which can be trained using labeled data in the railway field to enable conflict detection.

[0080] In practical applications, in order to accurately analyze the impact of the above evaluation indicators on the comprehensive evaluation results, it is often necessary to evaluate the source authority and update timeliness of the credible knowledge subgraph separately; at the same time, in order to facilitate quantification, a scoring method is usually used to represent the evaluation results.

[0081] Therefore, in one possible implementation, the first evaluation result of the trusted knowledge subgraph is determined based on the authority of its data source; the second evaluation result is determined based on the timeliness of its updates; and the comprehensive evaluation result is determined based on the weighted value of the first and second evaluation results.

[0082] Among them, the first evaluation result of the credible knowledge subgraph is positively correlated with the authority of the data source, and the second evaluation result of the credible knowledge subgraph is positively correlated with the timeliness of the update of the credible knowledge subgraph.

[0083] It is understandable that by determining the first and second evaluation results of the credible knowledge subgraphs, the credible knowledge subgraphs can be evaluated from multiple dimensions. At the same time, the weights of different evaluation results can be adjusted according to actual needs to achieve differentiated screening strategies and improve the flexibility of determining the best credible knowledge subgraphs.

[0084] Step S402: Based on the comparison between the comprehensive evaluation result of the trusted knowledge subgraph and the second preset evaluation result, determine the preferred trusted knowledge subgraph.

[0085] In practical applications, it is typically necessary to analyze the comprehensive evaluation results of the trusted knowledge subgraphs to determine their quality and select preferred trusted knowledge subgraphs for subsequent parsing of user input questions. Therefore, the preferred trusted knowledge subgraph is determined based on a comparison between the comprehensive evaluation results of the trusted knowledge subgraphs and a second preset evaluation result. It can be understood that the second preset evaluation result is the reference evaluation result used in this application embodiment to judge the quality of the comprehensive evaluation results of the trusted knowledge subgraphs.

[0086] In one possible implementation, a trusted knowledge subgraph whose overall evaluation result is better than a second preset evaluation result is determined as a preferred trusted knowledge subgraph. It can be understood that if the overall evaluation result of a trusted knowledge subgraph is better than the second preset evaluation result, it indicates that the overall evaluation result of the trusted knowledge subgraph is better and can be used as a preferred trusted knowledge subgraph.

[0087] Of course, those skilled in the art can adjust the logic of determining the preferred credible knowledge subgraph based on the comparison results according to their needs. For example, a credible knowledge subgraph whose comprehensive evaluation result is close to (the difference is less than a certain value) the second preset evaluation result can be determined as the preferred credible knowledge subgraph. This application does not impose specific limitations on this.

[0088] Of course, those skilled in the art can adjust the method of determining the preferred credible knowledge subgraph according to their needs. For example, the preferred credible knowledge subgraph can be determined by combining the evaluation results of the timeliness of updates and / or the evaluation results of the authority of the source with the comprehensive evaluation results of the credible knowledge subgraph. This application does not impose specific limitations on this.

[0089] In this embodiment, by further filtering conflicting trusted knowledge subgraphs, a preferred trusted knowledge subgraph is determined, further improving the credibility of the knowledge subgraphs used by the first language model. This effectively enhances the accuracy of the first language model in parsing user input questions and generating responses. Simultaneously, it significantly improves the consistency, accuracy, and credibility of the intelligent question-answering method in complex and dynamic knowledge environments, enabling its application in professional knowledge question-answering scenarios with extremely high requirements for answer reliability, such as scheduling and command, equipment maintenance, and emergency response.

[0090] In practical applications, before using a second language model for credibility matching of knowledge subgraphs, the second language model needs to be trained to ensure reliable credibility matching. In this embodiment, a training dataset composed of positive and negative samples is used to train the second language model. Specifically, positive samples are correctly matched problem-knowledge subgraph pairs, and negative samples are incorrectly matched problem-knowledge subgraph pairs (with topic bias, logical contradictions, or version conflicts). During training, the credibility matching task of knowledge subgraphs can be regarded as a binary classification problem, i.e., one class is correct matching and the other is incorrect matching. Training is performed by minimizing intra-class correlation and maximizing inter-class correlation, so that the model can accurately identify credible knowledge subgraphs. At the same time, the structure of the second language model can adopt a Transformer architecture with dual towers or cross-coding structure to ensure that the model's complexity is sufficient to complete the credibility matching task. Of course, those skilled in the art can adjust the structure and training method of the second language model according to their needs, and this embodiment does not impose specific limitations on this.

[0091] In practical applications, first-language models require historical records as context to parse user input questions. However, in multi-turn switching scenarios, the system struggles to effectively distinguish the current dialogue's historical records (i.e., it's uncertain whether the question is about specialized knowledge or general knowledge), which reduces the reliability and applicability of the question-and-answer format. Therefore, it's necessary to store the historical records of questions and answers in specialized knowledge domains and those in general knowledge domains separately.

[0092] See Figure 5 This is a flowchart illustrating a method for separately storing historical records provided in an embodiment of this application. Figure 5 As shown, the method specifically includes the following steps.

[0093] Step S501: If the domain of the user input question is a general knowledge domain, then store the user input question and the first language model's response to the user input question as the first historical record.

[0094] The first historical record is used as the context when the first language model parses user input questions in the general knowledge domain. In other words, the first language model can directly use the first historical record as the context when parsing questions in the general knowledge domain.

[0095] Step S502: If the domain of the user input question is a professional knowledge domain, then store the user input question and the first language model's response to the user input question as a second historical record.

[0096] The second historical record is used as the context for the first language model to parse user input questions in the domain of professional knowledge. In other words, the first language model can directly use the second historical record as the context when parsing questions in the domain of professional knowledge.

[0097] In practical applications, during subsequent rounds of dialogue, the system dynamically invokes dialogue history matching the current question type as context input to the first language model, achieving accurate context tracking and on-demand provision. Simultaneously, it prevents content from outside the current question's domain from contaminating the context, thus eliminating cross-domain information interference. This allows the first language model to directly access historical records matching the domain of the user's input question, reducing the ineffective allocation of the first language model's attention resources and lowering the risk of misjudgments or illusions caused by contextual noise. By storing and managing question-and-answer pairs for general knowledge domains and those for specialized knowledge domains separately, the system ensures that the first language model can obtain relevant context immediately when processing specialized domain questions, avoiding interference from historical records in specialized domains and improving the professionalism and relevance of the responses.

[0098] See Figure 6 This is a structural diagram of an intelligent question-answering system provided in an embodiment of this application.

[0099] like Figure 6 As shown, the intelligent question answering system 600 includes an intent classification module 601, a question response module 602, a knowledge base retrieval module 603, a semantic matching module 604, a conflict detection module 605, and a history storage module 606.

[0100] Specifically, the intent classification module 601 includes an intent classification model, which is used to determine the domain to which the user input question belongs. The domain includes professional knowledge domains and general knowledge domains. If the domain is a professional knowledge domain, the question is input into the knowledge base retrieval module 603; if the domain is a general knowledge domain, the question is input into the question response module 602.

[0101] The question response module 602 includes a large language model. By calling the context stored in the history storage module 606 that matches the user input question, it parses the user input question and generates a response. If the user input question is a general knowledge domain question, it is parsed directly; if the user input question is a professional knowledge domain question, it is parsed based on the preferred reliable knowledge subgraph output by the conflict detection module 605.

[0102] The knowledge base retrieval module 603 is used to retrieve knowledge subgraphs related to the user's input question and input the knowledge subgraphs into the semantic matching module 604.

[0103] The semantic matching module 604 includes a semantic matching model, which is used to perform credibility matching on the knowledge subgraph, generate a credible knowledge subgraph, and input it into the conflict detection module.

[0104] The conflict detection module 605 includes a conflict detection model for detecting conflicts in the trusted knowledge subgraph. If a conflict exists, a preferred trusted knowledge subgraph is generated and input into the question response module 602.

[0105] The history storage module 606 is used to store user-input questions and responses from the question-response module. If the user-input question is a general knowledge domain question, the question and the response from the question-response module are stored as a first history record; if the user-input question is a specialized knowledge domain question, the question and the response from the question-response module are stored as a second history record.

[0106] To illustrate the technical effects achieved in the embodiments of this application, an experimental environment was constructed in the railway intelligent question-and-answer scenario. A test set was built using real railway regulations documents, technical manuals, customer service dialogue logs and other data, covering two types of tasks: general knowledge domain question-and-answer (such as ticketing and passenger service) and professional knowledge domain question-and-answer (such as train operation regulations and equipment fault handling), with a total of 1200 test samples (600 from the general knowledge domain and 600 from the professional knowledge domain).

[0107] The experiment compared four methods: Method Baseline A included only the question-answering module; Method Baseline B included only the question-answering module and the knowledge base retrieval module; Method Baseline C included only the question-answering module, the knowledge base retrieval module, and the intent classification module; and finally, the method provided in this application embodiment included all the modules in the above-mentioned intelligent question-answering system.

[0108] The experiment employed four evaluation metrics: accuracy, illusion rate, context relevance, and response time. Accuracy was defined as the proportion of model responses that matched the standard answer (human scoring); illusion rate was the proportion of generated content containing factual errors or fabricated information; context relevance was the effectiveness of the model's use of historical information in multi-turn dialogues; and response time was the average time from question input to response generation.

[0109] The table below shows the experimental results of four different methods across 1200 test samples.

[0110] As can be seen from the table, although the method of this application embodiment increases the response time by introducing a semantic matching module and a conflict detection module, it has the highest accuracy, the lowest illusion rate and the highest context relevance, which is significantly better than other methods.

[0111] Furthermore, a further analysis of the question-and-answer questions in the professional knowledge domain was conducted. The table below shows the accuracy and illusion rate of different methods in 600 test samples in the professional knowledge domain.

[0112] As can be seen, the embodiments of this application significantly improve the accuracy of answering questions in the field of professional knowledge, and the rate of hallucination is significantly reduced.

[0113] Furthermore, the contextual relevance of different methods was analyzed. The table below shows the contextual relevance of four different methods in different dialogue rounds.

[0114] As can be seen, the embodiments of this application avoid information interference by storing historical records separately, and effectively ensure context relevance in multi-turn dialogue scenarios.

[0115] Finally, the scenarios involving conflicts are analyzed, and the table below shows the performance of different methods in conflict scenarios.

[0116] As can be seen, the embodiments of this application, through the semantic matching module and the dynamic conflict detection module, can effectively identify knowledge subgraphs with stronger source authority and more timely updates, significantly improving the reliability of intelligent question answering methods in knowledge update scenarios.

[0117] Corresponding to the above embodiments, this application also provides an electronic device.

[0118] See Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device 700 may include a processor 701, a memory 702, and a communication unit 703. These components communicate via one or more buses. Those skilled in the art will understand that the electronic device structure shown in the figures does not constitute a limitation on the embodiments of this application. It may be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0119] The communication unit 703 is used to establish a communication channel, thereby enabling the electronic device to communicate with other devices.

[0120] The processor 701 serves as the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It executes software programs and / or modules stored in the memory 702, and calls data stored in the memory to perform various functions and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 701 may consist only of a central processing unit (CPU). In this embodiment, the CPU may have a single processing core or include multiple processing cores.

[0121] Memory 702 is used to store the execution instructions of processor 701. Memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0122] When the execution instructions in memory 702 are executed by processor 701, the electronic device 700 is able to perform some or all of the steps in the above method embodiments.

[0123] Corresponding to the above embodiments, this application also provides a computer-readable storage medium, wherein the computer-readable storage medium may store a program, wherein when the program runs, it can control the device where the computer-readable storage medium is located to execute some or all of the steps in the above method embodiments. In specific implementation, the computer-readable storage medium may be a magnetic disk, an optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0124] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, or the existence of B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects have an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0125] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0126] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0127] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0128] The above description is merely a specific embodiment of this application. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application. The protection scope of this application should be determined by the protection scope of the claims.

Claims

1. An intelligent question-answering method, characterized in that, include: Determine the domain to which the user input question belongs, where the domain includes general knowledge domains and professional knowledge domains; Based on the domain, determine the language model used to parse the user input question; The user input question is parsed based on the language model, and a response to the user input question is generated.

2. The method according to claim 1, characterized in that, The step of determining the language model used to parse the user input question based on the domain includes: If the user input question belongs to a general knowledge domain, then the first language model is used to parse the user input question; If the domain of the user input question is a professional knowledge domain, then a second language model is used to perform credibility matching on the knowledge subgraph related to the user input question to generate a credible knowledge subgraph. The user input question is parsed using the first language model based on the trusted knowledge subgraph; The first language model has more parameters than the second language model.

3. The method according to claim 2, characterized in that, Also includes: If the domain to which the user input question belongs is a general knowledge domain, then the user input question and the first language model's response to the user input question are stored as a first historical record; If the user input question belongs to a professional knowledge domain, then the user input question and the first language model's response to the user input question are stored as a second historical record. Wherein, the first historical record is used as the context employed by the first language model when parsing user input questions in the general knowledge domain; the second historical record is used as the context employed by the first language model when parsing user input questions in the professional knowledge domain.

4. The method according to claim 2, characterized in that, The step of using a second language model to perform credibility matching on the knowledge subgraph related to the user input question and generating a credible knowledge subgraph includes: The comprehensive evaluation result of the knowledge subgraph is determined based on its topic relevance, contextual coherence, and / or semantic consistency. The credible knowledge subgraph is determined based on the comparison between the comprehensive evaluation result of the knowledge subgraph and the first preset evaluation result. Wherein, the topic relevance of the knowledge subgraph is the topic relevance between the knowledge subgraph and the user input question, the contextual coherence of the knowledge subgraph is the contextual coherence between the knowledge subgraph and the historical records, and the semantic consistency of the knowledge subgraph is the semantic consistency between the generation tendency of the first language model for the user input question and the knowledge subgraph.

5. The method according to claim 4, characterized in that, After determining the credible knowledge subgraph based on the comparison between the comprehensive evaluation result of the knowledge subgraph and the first preset evaluation result, the method further includes: If there are factual conflicts among the trusted knowledge subgraphs, the comprehensive evaluation result of the trusted knowledge subgraphs shall be determined based on the authority of the data source and / or the timeliness of the update of the trusted knowledge subgraphs. Based on the comparison between the comprehensive evaluation result of the trusted knowledge subgraph and the second preset evaluation result, the preferred trusted knowledge subgraph is determined.

6. The method according to claim 4, characterized in that, The step of determining the comprehensive evaluation result of the knowledge subgraph based on its topic relevance, contextual coherence, and / or semantic consistency includes: Based on the topic relevance of the knowledge subgraph, a first evaluation result of the knowledge subgraph is determined, wherein the topic relevance is positively correlated with the first evaluation result; Based on the contextual coherence of the knowledge subgraph, a second evaluation result of the knowledge subgraph is determined, wherein the contextual coherence is positively correlated with the second evaluation result; Based on the semantic consistency of the knowledge subgraph, a third evaluation result of the knowledge subgraph is determined, wherein the semantic consistency is positively correlated with the third evaluation result; The comprehensive evaluation result of the knowledge subgraph is determined based on the weighted values ​​of the first evaluation result, the second evaluation result, and the third evaluation result.

7. The method according to claim 5, characterized in that, The step of determining the comprehensive evaluation result of the trusted knowledge subgraph based on the authority of its data source and / or the timeliness of its updates includes: Based on the authority of the data source of the trusted knowledge subgraph, a first evaluation result of the trusted knowledge subgraph is determined, and the first evaluation result of the trusted knowledge subgraph is positively correlated with the authority of the data source; Based on the update timeliness of the trusted knowledge subgraph, a second evaluation result of the trusted knowledge subgraph is determined, and the second evaluation result of the trusted knowledge subgraph is positively correlated with the update timeliness of the trusted knowledge subgraph. The comprehensive evaluation result of the trusted knowledge subgraph is determined by weighting the first evaluation result and the second evaluation result of the trusted knowledge subgraph.

8. The method according to claim 4, characterized in that, The step of determining the credible knowledge subgraph based on the comparison between the comprehensive evaluation result of the knowledge subgraph and the first preset evaluation result includes: The knowledge subgraph whose comprehensive evaluation result is better than the first preset evaluation result is determined as the credible knowledge subgraph.

9. The method according to claim 5, characterized in that, The step of determining the preferred reliable knowledge subgraph based on the comparison between the comprehensive evaluation result of the reliable knowledge subgraph and the second preset evaluation result includes: The trusted knowledge subgraph whose comprehensive evaluation result is better than the second preset evaluation result is determined as the preferred trusted knowledge subgraph.

10. An electronic device, characterized in that, include: processor; Memory; And a computer program, wherein the computer program is stored in the memory, and when the computer program is executed by the processor, causes the electronic device to perform the method of any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 9.

Images