Intelligent question and answer method, related device, equipment and storage medium

Abstract

The invention discloses an intelligent question-answering method, a related device, equipment and a storage medium, and the intelligent question-answering method comprises the steps: obtaining a plurality of historical data of a question-answering system in a target scene; wherein the historical data at least comprises input questions and output answers of the question and answer system; performing analysis based on the plurality of historical data to obtain a plurality of task types related to the target scene and the difficulty level of each task type; based on the plurality of task types and the difficulty level of each task type, constructing a plurality of question and answer models; wherein any question and answer model is suitable for at least one task type; and in response to a new input question in the target scene, selecting the question and answer model as a target model for answering the new input question. According to the scheme, the intelligent question and answer model operation efficiency can be improved, the model computing power consumption can be reduced, different requirements of different tasks on the model capability are considered, and the intelligent question and answer efficiency and effect can be balanced.

Description

Technical Field

[0001] This application relates to the technical field of natural language processing, and in particular to an intelligent question-answering method and related devices, equipment, and storage media. Background Art

[0002] Since the generative large model, intelligent question-answering technology has been gradually applied to many scenarios such as medical care and education, greatly meeting the self-service question-asking needs of users.

[0003] Currently, intelligent question-answering technology usually relies on a single model to process all types of interaction requests, with low operating efficiency, high computing power consumption, and it is also difficult to take into account the different requirements for model capabilities of different tasks. In view of this, how to improve the operating efficiency of the intelligent question-answering model, reduce the computing power consumption of the model, and at the same time take into account the different requirements for model capabilities of different tasks has become an urgent problem to be solved. Summary of the Invention

[0004] The main technical problem to be solved by this application is to provide an intelligent question-answering method and related devices, equipment, and storage media, which can improve the operating efficiency of the intelligent question-answering model, reduce the computing power consumption of the model, and at the same time take into account the different requirements for model capabilities of different tasks.

[0005] To solve the above technical problem, in the first aspect of this application, an intelligent question-answering method is provided, including: obtaining a number of historical data of the question-answering system in the target scenario; where the historical data at least includes the input questions and output answers of the question-answering system; analyzing based on the number of historical data to obtain a number of task types involved in the target scenario and the difficulty levels of each task type; constructing multiple question-answering models based on the number of task types and the difficulty levels of each task type; where any one of the question-answering models is applicable to at least one task type; in response to a new input question in the target scenario, selecting a question-answering model as the target model for answering the new input question.

[0006] To solve the above technical problem, in the second aspect of this application, an intelligent question-answering device is provided, including: a data acquisition module, a data analysis module, a model construction module, and a model selection module. The data acquisition module is used to obtain a number of historical data of the question-answering system in the target scenario; where the historical data at least includes the input questions and output answers of the question-answering system; the data analysis module is used to analyze based on the number of historical data to obtain a number of task types involved in the target scenario and the difficulty levels of each task type; the model construction module is used to construct multiple question-answering models based on the number of task types and the difficulty levels of each task type; where any one of the question-answering models is applicable to at least one task type; the model selection module is used to select a question-answering model as the target model for answering the new input question in response to a new input question in the target scenario.

[0007] To solve the above technical problems, a third aspect of the present application provides an electronic device, which at least includes a memory and a processor coupled to each other. At least program instructions are stored in the memory, and the processor is configured to execute the program instructions to implement the intelligent question-answering method in the above first aspect.

[0008] To solve the above technical problems, a fourth aspect of the present application provides a computer-readable storage medium storing program instructions that can be run by a processor, and the program instructions are used to implement the intelligent question-answering method in the above first aspect.

[0009] In the above solution, a number of historical data of the question-answering system in the target scenario are obtained, and the historical data at least include the input questions and output answers of the question-answering system. Based on the analysis of the number of historical data, a number of task types involved in the target scenario and the difficulty levels of each task type are obtained. Thus, based on the number of task types and the difficulty levels of each task type, multiple question-answering models are constructed, and any question-answering model is applicable to at least one task type. Furthermore, in response to a new input question in the target scenario, a question-answering model is selected as the target model for answering the new input question. Therefore, it is possible to analyze a number of task types involved in the target scenario based on the historical data of the target scenario, and in combination with the difficulty levels of each task type, differently construct different question-answering models to adaptively select a question-answering model to answer a new input question. Compared with uniformly using a single model to answer different questions, it is possible to improve the model operation efficiency of intelligent question answering, reduce the model computing power consumption, and at the same time take into account the different requirements of different tasks for the model capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a schematic flowchart of an embodiment of the intelligent question-answering method of the present application;

[0011] Figure 2 is a schematic diagram of the process of an embodiment of the intelligent question-answering method of the present application;

[0012] Figure 3 is a schematic framework diagram of an embodiment of the intelligent question-answering device of the present application;

[0013] Figure 4 is a schematic framework diagram of an embodiment of the electronic device of the present application;

[0014] Figure 5 is a schematic framework diagram of an embodiment of the computer-readable storage medium of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0015] The following will describe the solutions of the embodiments of the present application in detail with reference to the accompanying drawings of the specification.

[0016] In the following description, specific details such as specific system architectures, interfaces, and technologies are presented for the purpose of illustration rather than limitation, in order to provide a thorough understanding of the present application.

[0017] The terms "system" and "network" are often used interchangeably in this document. The term "and / or" in this document merely describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. Additionally, the fragment " / " in this document generally indicates that the associated objects before and after are in an "or" relationship. Furthermore, "plurality" in this document means two or more than two.

[0018] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of an embodiment of the intelligent question-and-answer method of the present application.

[0019] Specifically, it may include the following steps:

[0020] Step S11: Obtain a number of historical data of the question-and-answer system in the target scenario.

[0021] In the embodiments of the present disclosure, the historical data at least includes the input questions and output answers of the question-and-answer system. It should be noted that the question-and-answer system is an existing system used to answer input questions in the target scenario. The question-and-answer system may include, but is not limited to, large language models, etc. The system architecture of the question-and-answer system is not limited herein. Additionally, the large language model may include, but is not limited to, open-source large models such as Llama, Bloom, etc., or the large language model may also be obtained by fine-tuning the parameters of the open-source large model based on a specific corpus, or the large language model may also be a custom large model. The specific source of the large language model is not limited herein.

[0022] In one implementation scenario, taking the education field as an example, the target scenario may include, but is not limited to: assisting teaching, etc., which is not limited herein; or, taking the medical field as an example, the target scenario may include, but is not limited to: intelligent medical consultation, outpatient guidance, etc., which is not limited herein. The above examples are merely several possible examples of the target scenario when taking the education and medical fields as examples, and the target scenario is not limited herein, nor will they be enumerated one by one.

[0023] In an implementation scenario, the historical data can include not only the input questions and output answers, but also further include the evaluation feedback of the output answers. As a possible example, after receiving an input question, the Q&A system can provide an intelligent answer to obtain the output answer of the Q&A system to the input question. At the same time, it can also receive the feedback instructions from the system user for the output answer to parse the feedback instructions and obtain the evaluation feedback of the system user on the output answer. Alternatively, the input question and its output answer can be input into an evaluation model to evaluate the output answer of the input question by the evaluation model, so that the output content of the evaluation model can be obtained as the evaluation feedback of the output answer. It should be noted that the evaluation model can be obtained by fine-tuning the parameters of the large language model based on a specific corpus. The specific corpus can specifically include sample questions in the target scenario, sample answers to the sample questions, and sample evaluations of the sample answers by experienced personnel such as experts in the target scenario, so as to train accordingly and enable the evaluation model to learn the evaluation criteria of experienced personnel on question answers in the target scenario. The specific training process can refer to the technical details of parameter fine-tuning and will not be elaborated here.

[0024] Step S12: Analyze based on a number of historical data to obtain a number of task types involved in the target scenario and the difficulty levels of each task type.

[0025] In an implementation scenario, before analyzing the historical data, the historical data can also be preprocessed first. The preprocessing can include, but is not limited to: removing invalid data, removing redundant data, etc., so as to ensure the accuracy and integrity of the data as much as possible.

[0026] In an implementation scenario, as a possible implementation example, the input questions in a number of historical data can be classified first to obtain a number of task types involved in the target scenario, and then the output answers of the input questions under the task types by the Q&A system can be analyzed to obtain the difficulty levels of the task types. In the above manner, the input questions are classified first to obtain a number of task types involved in the target scenario, and then the output answers of the input questions under each task type are analyzed to obtain the difficulty levels of the task types, which can gradually analyze the task types involved in the target scenario and their difficulty levels.

[0027] In a specific implementation scenario, still taking the target scenario of assisting teaching as an example, by analyzing the questioning method, the final focus, etc. of the input question, it can be analyzed that the task types involved in the target scenario of assisting teaching include but are not limited to the following: confirmation of problem-solving ideas, confirmation of problem-solving answers, inquiry about specific details of known information, confirmation of known information, knowledge point query, generalization of problem-solving ideas for similar knowledge points, regression summary of knowledge points, discrimination of similar knowledge points, mathematical calculation, graphic rendering for auxiliary understanding, need for step-by-step guidance due to unclear requirements, inquiry about complex problem-solving ideas, extended questions, etc. The specific content of the task types is not limited here. When the target scenario is other situations, the possible content of the task types can be inferred by analogy, and no further examples will be given here.

[0028] In a specific implementation scenario, as mentioned above, the historical data can also include the evaluation feedback of the output answers. After obtaining several task types involved in the target scenario, the response ability of the question-and-answer system for each task type can be statistically obtained based on the evaluation feedback of each output answer under the task type. Exemplarily, the evaluation feedback can include but is not limited to evaluation contents such as "helpful" and "unhelpful". The response ability can specifically represent the coverage rate of the effective answers of the question-and-answer system for the task type. For any task type, the proportion of the number of output answers with evaluation feedback such as "helpful" indicating effective answers in all output answers under this task type can be statistically calculated as the response ability of the question-and-answer system for this task type. On this basis, the difficulty level of the task type can be obtained based on the response ability of the question-and-answer system for the task type. It should be noted that there is a correlation between the response ability and the difficulty level. For example, the stronger the response ability of the question-and-answer system for the task type (such as the larger the coverage rate of the effective answers of the question-and-answer system for the task type), the lower the difficulty level represented by the difficulty degree. On the contrary, the weaker the response ability of the question-and-answer system for the task type (such as the smaller the coverage rate of the effective answers of the question-and-answer system for the task type), the higher the difficulty level represented by the difficulty degree. In the above manner, since the historical data also includes the evaluation feedback of the output answers, by statistically obtaining the response ability of the question-and-answer system for each task type based on the evaluation feedback of each output answer under the task type, and obtaining the difficulty level of the task type based on the response ability of the question-and-answer system for the task type, the difficulty levels of each task type can be determined through mathematical statistics, which helps to reduce the complexity of determining the difficulty level as much as possible.

[0029] In another implementation scenario, different from the foregoing implementation manner, as another possible implementation example, in order to improve the analysis efficiency of task types, a first prompt instruction may be constructed based on input questions in a number of historical data, and the first prompt instruction is used to instruct a large language model to analyze the input questions in the number of historical data to extract a number of task types involved in the target scenario. Then, the first prompt instruction is input into the large language model, and the output content of the large language model is obtained to obtain a number of task types involved in the target scenario. In addition, a second prompt instruction may be constructed based on the evaluation feedback of each output answer under the task type by the question-and-answer system, and the second prompt instruction is used to instruct the large language model to analyze the evaluation feedback of each output answer under the same task type to determine the difficulty level of the corresponding task type. Then, the second prompt instruction is input into the large language model, and the output content of the large language model is obtained to obtain the difficulty level of the task type.

[0030] Step S13: Construct a plurality of question-and-answer models based on a number of task types and the difficulty level of each task type.

[0031] In the embodiments of the present disclosure, any question-and-answer model is applicable to at least one task type. That is to say, for any question-and-answer model, it may only be applicable to one task type, or it may be applicable to multiple task types (for example, it can be applicable to two task types, three task types, etc.), which is not limited here. As a possible implementation example, in the case where any question-and-answer model is only applicable to one task type, taking the auxiliary teaching scenario as an example, if the task type set in the auxiliary teaching scenario includes the following tasks: confirmation of problem-solving ideas, confirmation of problem-solving answers, inquiry about specific details of known information, confirmation of specific details of known information, knowledge point query, generalization of problem-solving ideas for similar knowledge points, summary of knowledge point regression, discrimination of similar knowledge points, mathematical calculation, graphic rendering for auxiliary understanding, need for step-by-step guidance due to unclear requirements, inquiry about complex problem-solving ideas, and extended questions, then for any of the above task types, a question-and-answer model applicable to it can be designed separately in advance. For example, a certain question-and-answer model is applicable to the task type "confirmation of problem-solving ideas" (that is, this question-and-answer model specifically has the model ability to analyze and distinguish problem-solving ideas for the input test questions), and a certain question-and-answer model is applicable to the task type "confirmation of problem-solving answers" (that is, this question-and-answer model specifically has the model ability to think and distinguish problem-solving answers for the input test questions), and so on, which will not be elaborated here. That is to say, in the case where any question-and-answer model is only applicable to one task type, the question-and-answer models applicable to different task types may be different in terms of model architecture, model parameters, etc. Or, in the case where any question-and-answer model is applicable to more than one task type, the model capabilities relied on by different task types can be analyzed, and then for multiple task types that rely on the same model ability, a question-and-answer model that is uniformly applicable can be designed in advance. At this time, for any question-and-answer model, it can be applicable to more than one task type.Taking the above-mentioned situations as examples where the set of task types in the auxiliary teaching scenario includes the above cases, for the task types of "confirming the problem-solving idea", "confirming the problem-solving answer", "asking for specific details of known information", and "confirming specific details of known information", they mainly rely on semantic understanding ability. Therefore, a unified applicable Q&A model can be designed in advance for the above task types (that is, this Q&A model specifically has strong semantic understanding ability, so that it can be particularly applicable to the above task types that require semantic understanding ability). For the task types of "knowledge point query", "summarizing the problem-solving ideas of similar knowledge points", "regressing and summarizing knowledge points", and "distinguishing between similar knowledge points", they mainly rely on the ability to build a knowledge system by leveraging external knowledge. Therefore, a unified applicable Q&A model can also be designed in advance for the above task types (that is, this Q&A model specifically has strong ability to build a knowledge system by leveraging external knowledge, so that it can be particularly applicable to the above task types that require the ability to build a knowledge system by leveraging external knowledge). And for the task types of "mathematical calculation" and "graph rendering for auxiliary understanding", they mainly rely on the ability to assist reasoning by leveraging external plugin capabilities. Therefore, a unified applicable Q&A model can also be designed in advance for the above task types (that is, this Q&A model specifically has strong ability to assist reasoning by leveraging external plugin capabilities, so that it can be particularly applicable to the above task types that require the ability to assist reasoning by leveraging external plugin capabilities). And for the task types of "unclear requirements need to be gradually guided", "asking for complex problem-solving ideas", and "expansion questions", they mainly rely on complex logical reasoning ability. Therefore, a unified applicable Q&A model can also be designed in advance for the above task types (that is, this Q&A model specifically has strong complex logical reasoning ability, so that it can be particularly applicable to the above task types that require complex logical reasoning ability). For specific details, please refer to the relevant descriptions below and will not be elaborated here. That is to say, when any Q&A model is applicable to more than one task type, the model capabilities of each Q&A model can be different. Specifically, it can be manifested that Q&A models with different model capabilities are different in terms of model architecture, model parameters, etc. Of course, the above examples are only several possible examples in the auxiliary teaching scenario where any Q&A model is applicable to at least one task type, and other possible situations will not be listed one by one here.

[0032] In an implementation scenario, as a possible implementation example, the model capabilities required for several task types can be analyzed first, so that the model architectures required for different model capabilities can be determined. Based on the difficulty levels of the task types for which the model capabilities are required, the parameter scales of the model architectures required for the model capabilities can be determined. Furthermore, multiple question-and-answer models can be constructed based on the model architectures and parameter scales required for different model capabilities respectively. By the above method, different model capabilities are analyzed through several task types, the model architectures and their parameter scales are determined accordingly, and then multiple question-and-answer models are constructed based on this, which can improve the rationality of constructing multiple question-and-answer models in the target scenario as much as possible and help improve the adaptability to diverse requirements in the target scenario.

[0033] In a specific implementation scenario, still taking the target scenario as teaching assistance as an example, in response to any one of the task types of confirming the problem-solving idea, confirming the problem-solving answer, asking for specific details of known information, and confirming specific details of known information, the model capability can be determined as semantic understanding capability. In response to any one of the task types of knowledge point query, summarizing the problem-solving ideas of similar knowledge points, summarizing the regression of knowledge points, and differentiating similar knowledge points, the model capability can be determined as the knowledge system construction capability with the help of external knowledge. In response to any one of the task types of mathematical calculation and using external plug-in capabilities to assist reasoning in graphic rendering and understanding, the model capability can be determined as the complex logical reasoning capability. Of course, the above examples are only several possible examples of analyzing the model capabilities required for task types when the target scenario is teaching assistance, and other possible situations will not be listed one by one here.

[0034] In a specific implementation scenario, still taking the target scenario as auxiliary teaching as an example, when the model ability is semantic understanding ability, the Q&A model required for the model ability is the largest model of the first order of magnitude. When the model ability is the knowledge system construction ability with the help of external knowledge, the Q&A model required for the model ability is the second largest model of the second order of magnitude and based on retrieval-augmented generation. When the model ability is to assist reasoning with the help of external plugin capabilities, the Q&A model required for the model ability is the third largest model of the third order of magnitude and based on plugin assistance. When the model ability is complex logical reasoning ability, the Q&A model required for the model ability is the fourth largest model of the fourth order of magnitude. As a possible implementation example, the second order of magnitude and the third order of magnitude are both larger than the first order of magnitude, and the fourth order of magnitude is larger than either of the second order of magnitude and the third order of magnitude. Of course, the order of magnitude of the above Q&A model may also be other situations, which are not limited here. It should be noted that the plugin may include but is not limited to APIs (such as software interfaces for weather queries, etc.), knowledge bases (such as legal provisions, rules and regulations, industry standards, etc.), etc., and the specific content of the plugin is not limited here. For the sake of convenience of description, the largest model can be denoted as M base , the second largest model is denoted as M rag , the third largest model is denoted as M plugin , and the fourth largest model is denoted as M large . That is to say, the largest model M base has the smallest parameter scale, the fourth largest model denoted as M large has the largest parameter scale, and the second largest model denoted as M rag and the third largest model denoted as M plugin have a medium parameter scale, and the parameter scales of the two can be similar. Of course, the above examples are only several possible examples of multiple Q&A models in the case where the target scenario is teaching assistance, and other possible situations are not exemplified one by one here.

[0035] In a specific implementation scenario, after obtaining multiple Q&A models, each Q&A model can also be trained based on a number of historical data to improve the model performance of the Q&A model for intelligent Q&A in the target scenario. It should be noted that for the specific process of model training, the technical details of training methods such as supervised training, unsupervised training, and semi-supervised training can be referred to, which will not be elaborated here.

[0036] In another implementation scenario, different from the foregoing implementation, as another possible implementation example, after analyzing the model capabilities required for several task types respectively, a third prompt instruction can be constructed based on each model capability and the difficulty level of the task types for which the model capability is required. The third prompt instruction is used to instruct the large language model to determine the model architecture and its parameter scale required for the model capability. Then, input the third prompt instruction into the large language model and obtain the output content of the large language model, and the model architecture required for different model capabilities and the parameter scale of the model architecture required for the model capability can be obtained.

[0037] Step S14: In response to a new input question in the target scenario, select a question-and-answer model as the target model for answering the new input question.

[0038] In one implementation scenario, as a possible implementation, the task type of the new input question can be analyzed, so that based on the task types adapted to the model capabilities respectively possessed by each question-and-answer model, a question-and-answer model can be selected as the target model for answering the new input question. Exemplarily, still taking the target scenario as teaching assistance, if the task type of the new input question is "confirmation of problem-solving ideas", then since the model capability "semantic understanding ability" of the first large model M base includes "confirmation of problem-solving ideas" in the task types it adapts to, the first large model M base can be selected as the target model for answering the new input question. Of course, the above example is only a possible example of selecting a question-and-answer model when the target scenario is teaching assistance, and other possible situations are not exemplified one by one here.

[0039] In another implementation scenario, different from the foregoing implementation, as another possible implementation, it is also possible to make a prediction based on the new input question to obtain a routing label, and the routing label includes the probability values of each question-and-answer model being selected to answer the new input question respectively. Then, select the question-and-answer model with the maximum probability value as the target model for answering the new input question. Exemplarily, still taking the target scenario as teaching assistance, if the routing label of the new input question is [0.7 0.1 0.1 0.1], then since the first large model M base has the maximum probability value, the first large model M base can be selected as the target model for answering the new input question. Of course, the above example is only a possible example of selecting a question-and-answer model when the target scenario is teaching assistance, and other possible situations are not exemplified one by one here.

[0040] In yet another implementation scenario, different from the foregoing implementation, as yet another possible implementation, please refer to Figure 2 , Figure 2 is a schematic diagram of the process of an embodiment of the intelligent question-and-answer method of the present application. AsFigure 2 As shown, after making a prediction based on a new input question and obtaining a routing label, multiple Q&A models can be sorted in ascending order of parameter scale. As a possible implementation example, there may be two or more Q&A models with the same parameter scale. In the case of the same parameter scale, they can be sorted in descending order of inference efficiency. Exemplarily, still taking the target scenario of assisting teaching as an example, after sorting multiple Q&A models according to the above criteria, they are: the largest model M base , the second largest model M rag , the third largest model M plugin , the fourth largest model M large . Of course, the above example is only a possible example of sorting multiple Q&A models when the target scenario is teaching assistance, and other possible situations will not be exemplified one by one here. After sorting multiple Q&A models, it is possible to further detect whether each Q&A model is selected as the target model in sequence among the sorted multiple Q&A models until the target model is determined. As a possible implementation example, as mentioned above, the routing label may include the probability values of multiple Q&A models being respectively selected to answer the new input question. Then, candidate models can be selected in sequence among the sorted multiple Q&A models. Still taking the target scenario of assisting teaching as an example, the largest model M base , the second largest model M rag , the third largest model M plugin , and the fourth largest model M large can be selected as candidate models in sequence. On this basis, it can be detected whether the probability value of the candidate model being selected to answer the new input question is not lower than the probability threshold of the candidate model. In response to the detection being not lower, the candidate model can be selected as the target model, and in response to the detection being lower, the step of selecting candidate models in sequence among the sorted multiple Q&A models can be returned until the target model is obtained. Still taking the target scenario of assisting teaching as an example, it can first be determined whether the probability value p base of the largest model M base being selected to answer the new input question is not lower than the probability threshold t base of the largest model M base . If so, the largest model M base can be selected as the target model for answering the new input question. Otherwise, it can continue to determine whether the probability value p rag of the second largest model M rag being selected to answer the new input question is not lower than the probability threshold t rag of the second largest model M rag . If so, the second largest model M rag can be selected as the target model for answering the new input question. Otherwise, it can continue to determine whether the probability value p pluginThe probability value p of being selected to answer the new input question plugin is not lower than that of the third largest model M plugin 's probability threshold t plugin , if so, the third largest model M can be selected plugin as the target model for answering the new input question, otherwise, the fourth largest model M can be continuously judged large The probability value p of being selected to answer the new input question large is not lower than that of the fourth largest model M large 's probability threshold t large , if so, the fourth largest model M can be selected large as the target model for answering the new input question. Or, as another possible implementation example, for each Q&A model, it can first be detected based on the routing label whether the probability value of selecting this Q&A model to answer the new input question is not lower than the probability threshold of this Q&A model. If so, this Q&A model is selected as a candidate model. After that, based on the sorting results of multiple Q&A models, the sequence positions of each candidate model can be determined, and the candidate model ranked first is selected as the target model. Of course, the above examples are only a possible example of selecting a Q&A model when the target scenario is teaching assistance, and other possible situations are not exemplified one by one here. In the above manner, a prediction is made based on the new input question to obtain a routing label, and the routing label includes the probability values of multiple Q&A models being respectively selected to answer the new input question. Then, the multiple Q&A models are sorted in ascending order of parameter scale, and in the case of the same parameter scale, they are sorted in descending order of inference efficiency. Among the multiple Q&A models after sorting, candidate models are selected in turn, so as to detect whether the probability value of the candidate model being selected to answer the new input question is not lower than the probability threshold of the candidate model. Furthermore, in response to the detection being not lower, the candidate model is selected as the target model, and in response to the detection being lower, the step of selecting candidate models in turn among the multiple Q&A models after sorting is returned until the target model is selected. Therefore, through threshold streaming judgment, it is possible to preferentially select a Q&A model with a smaller parameter scale and secondarily select a Q&A model with a higher inference efficiency as the target model for answering the new input question on the premise of satisfying the answer to the new input question.

[0041] In an implementation scenario, the target model can be selected by the routing model from multiple question-and-answer models for a new input question (e.g., in the foregoing implementation example, the routing label can be predicted by the routing model). Then, after constructing multiple question-and-answer models and before selecting a question-and-answer model, answer sampling can be performed on the sample questions based on the multiple question-and-answer models respectively to obtain the sample answers of the multiple question-and-answer models for the sample questions at each sampling. And based on the acceptability of each sample answer at each sampling, the detection results indicating whether the answer hits can be obtained for the multiple question-and-answer models at the corresponding sampling. Thus, statistics can be performed on the detection results of the multiple question-and-answer models at each sampling to obtain the sample routing label of the sample question. Furthermore, based on the sample question labeled with the sample routing label, the routing model can be trained until the training end condition is met. In the above manner, by collecting the sample routing label of the sample question through answer sampling and training the routing model accordingly, the routing model can be forced to learn the suitability of each question-and-answer model for different input questions.

[0042] In a specific implementation scenario, the routing model can include, but is not limited to, convolutional neural network, recurrent neural network, Transformer, etc. As a possible example, the routing model can be constructed with BERT (Bidirectional Encoder Representation from Transformers) plus a four-classification layer. Here, the network structure of the routing model is not limited.

[0043] In a specific implementation scenario, for the sake of convenience of description, the i-th sample question can be denoted as x i , taking the example that each question-and-answer model performs n times of answer sampling. For any question-and-answer model, n sample answers can be sampled:

[0044]

[0045] In the above formula (1), represents the set of sample answers obtained by the question-and-answer model M x for answering and sampling the i-th sample question, where represents the first sample answer therein, represents the second sample answer therein, represents the n-th sample answer therein.

[0046] In a specific implementation scenario, the acceptability of the sample answer can be obtained through manual annotation or evaluation by a general large model. As a possible example, the acceptability can include, but is not limited to: unacceptable (Acceptance n)、Acceptable m )、Excellent p )。Of course, the above examples are only a few possible examples of the acceptable degree, and other possible situations will not be listed one by one here. In addition, for the sake of convenience of description, the acceptable degree of the aforementioned sample answers can be expressed as:

[0047]

[0048] In the above formula (2), represents the acceptable degree of the set of sample answers obtained by sampling the answers to the i-th sample question by the question-answering model M x , where represents the acceptable degree of the first sample answer among them, represents the acceptable degree of the second sample answer among them, represents the acceptable degree of the n-th sample answer among them.

[0049] In a specific implementation scenario, as mentioned above, the acceptable degree is any one of unacceptable, acceptable, and excellent, and the question-answering model whose detection result represents a hit in the answer during each sampling is used as the hit model. Then, for the sample answers of each answer sampling of multiple question-answering models, first, among the question-answering models with an acceptable degree of the sample answers above acceptable, preferably select the question-answering model with the smallest parameter scale as the hit model. Then, in the case where the selection fails due to the same parameter scale, secondly, select the question-answering model with a higher acceptable degree as the hit model. And in the case where the secondary selection fails due to the same acceptable degree, thirdly, select the question-answering model with a higher inference efficiency as the hit model. Still taking the target scenario of assisting teaching as an example, 4 sample answers can be obtained for each answer sampling (that is, the sample answers of the aforementioned first largest model to the fourth largest model for the same sample question). As a possible example, if the acceptable degree of the sample answer of the first largest model is above acceptable (e.g., Acceptable m , Excellent p ), then it can be directly determined that the answer of the first largest model hits in this answer sampling, while the answers of the second largest model to the fourth largest model do not hit. As another possible example, if the acceptable degrees of the sample answers of both the second largest model and the third largest model are above acceptable, then since the parameter scales of the two are similar and the selection fails, and since the acceptable degree of the second largest model is Acceptable m , and the acceptable degree of the third largest model is Excellent p, it can be determined that the answer of the third-largest model hits in this answer sampling, but the answers of the first-largest model, the second-largest model, and the fourth-largest model do not hit. As another possible example, if the acceptability of the sample answers of both the second-largest model and the third-largest model is Acceptable m , then due to the similar parameter scales and the same acceptability of both, the secondary selection fails. Since the inference efficiency of the second-largest model is higher, it can be determined that the answer of the second-largest model hits in this answer sampling, but the answers of the first-largest model, the third-largest model, and the fourth-largest model do not hit. Of course, the above examples are only possible examples of determining whether the answer hits according to the acceptability in the actual application process. Other possible criteria are not limited here, nor will other possible criteria be exemplified one by one. In the above manner, the acceptability can be any one of unacceptable, acceptable, and excellent, and each time during sampling, the Q&A model whose detection result represents an answer hit is used as the hit model. Among the Q&A models with an acceptability of more than acceptable for the sample answers, the Q&A model with the smallest parameter scale is preferably used as the hit model. In the case of a failed preference due to the same parameter scale, the Q&A model with a higher acceptability is secondarily selected as the hit model. And in the case of a failed secondary selection due to the same acceptability, the Q&A model with a higher inference efficiency is again selected as the hit model. It can determine whether the answer hits according to the priority order of parameter scale, acceptability, and inference efficiency on the premise of ensuring the acceptability of the answer. In this way, forming the sample routing label can force the routing model to also perform model routing according to the above criteria.

[0050] In a specific implementation scenario, after obtaining the detection results of each answer sampling, statistics can be performed accordingly to obtain the sample routing label of the sample question. Specifically, for each Q&A model, the proportion of the number of times its answer hits in n answer samplings can be statistically calculated as its element value in the sample routing label. Still taking the target scenario of assisting teaching as an example, if the number of sampling times n is 5, then as a possible example, the detection result of n answer samplings for the i-th sample question can be expressed as:

[0051]

[0052] In the above formula (3), Des i represents the detection result of 5 answer samplings for the i-th sample question, where represents that the first-largest model M base hits the answer at the first answer sampling, represents that the first-largest model M base hits the answer at the second answer sampling, represents that the second-largest model M rag hits the answer at the third answer sampling, Denote the second largest model M during the 4th answer sampling rag Answer hits, Denote the fourth largest model M during the 5th answer sampling large Answer hits. Based on this, frequency statistics can be performed to obtain the sample routing label:

[0053]

[0054] In the above formula (4), Prob i Denote the sample routing label of the i-th sample question, where Denote the sample probability value (i.e., hit rate) of the first largest routing model answering the i-th sample question, Denote the sample probability value (i.e., hit rate) of the second largest routing model answering the i-th sample question, Denote the sample probability value (i.e., hit rate) of the third largest routing model answering the i-th sample question, Denote the sample probability value (i.e., hit rate) of the fourth largest routing model answering the i-th sample question. Still taking the above example, the following sample routing label (i.e., the fitting target of the routing model) can be statistically obtained:

[0055] Prob i =[0.4, 0.4, 0.0, 0.2]……(5)

[0056] Of course, the above example is only a possible example of the hit results of each answer sampling when the target scenario is auxiliary teaching. Other possible situations are not listed one by one here.

[0057] In a specific implementation scenario, after obtaining the sample routing label of the sample question, the routing model can be trained based on the sample question marked with the sample routing label. Specifically, the sample question can be routed based on the routing model to obtain the predicted routing labels of multiple Q&A models, and label smoothing and normalization are performed based on the sample routing label to obtain the target routing label. And the predicted routing label includes the predicted probability values of multiple Q&A models being selected to answer the sample question respectively (specifically, refer to the relevant description of the sample routing label above, which will not be elaborated here). Based on this, the network parameters of the routing model can be adjusted based on the distribution difference between the predicted routing label and the target routing label. Exemplarily, the distribution difference between the predicted routing label and the target routing label can be measured based on loss functions such as KL divergence:

[0058]

[0059] In the above formula (6), x i Denote the i-th sample question, f θ (xi ) represents the predicted routing label obtained by the routing model for routing the sample question, Prob i represents the sample routing label of the i-th sample question, LabelSmoothing represents label smoothing, and softmax represents normalization. It should be noted that through label smoothing, it is possible to avoid as much as possible that the sample routing label is overly biased towards a certain Q&A model, soften the absolute situation of 0 or 1 on a certain Q&A model, so as to force the routing model to learn a smooth decision boundary and show better robustness. In addition, the training end condition can include that the number of training times is not less than a preset number (such as 1000, etc.), the training loss continues to be not higher than a preset threshold, etc., and the training end condition is not limited here. Of course, after the routing model training is completed, the routing strategy can also be dynamically adjusted according to the real-time feedback and performance monitoring of the system, such as adjusting the model selection threshold, optimizing the model combination, etc., to improve the overall performance of the system and user satisfaction. At the same time, new interaction data can be continuously collected for updating and optimizing the routing model. In the target scenario of assisted teaching, the auxiliary learning model can also be updated and optimized together with the routing model to ensure that the system can continuously adapt to the changes of the student group and the development of needs.

[0060] For the above solution, a number of historical data of the Q&A system in the target scenario are obtained, and the historical data at least includes the input questions and output answers of the Q&A system. Based on the analysis of the number of historical data, a number of task types involved in the target scenario and the difficulty levels of each task type are obtained. Then, based on the number of task types and the difficulty levels of each task type, multiple Q&A models are constructed, and any Q&A model is applicable to at least one task type. Furthermore, in response to a new input question in the target scenario, a Q&A model is selected as the target model for answering the new input question. Therefore, it is possible to analyze a number of task types involved in the target scenario based on the historical data of the target scenario, and combine the difficulty levels of each task type to differentially construct different Q&A models to adaptively select Q&A models to answer new input questions. Compared with uniformly using a single model to answer different questions, it can improve the model operation efficiency of intelligent Q&A, reduce the model computing power consumption, and at the same time take into account the different task requirements for the model capabilities.

[0061] Please refer to Figure 3 , Figure 3It is a schematic framework diagram of an embodiment of the intelligent question-answering device of the present application. The intelligent question-answering device 30 includes: a data acquisition module 31, a data analysis module 32, a model construction module 33, and a model selection module 34. The data acquisition module 31 is used to acquire a number of historical data of the question-answering system in the target scenario; wherein, the historical data at least includes the input questions and output answers of the question-answering system. The data analysis module 32 is used to analyze based on a number of historical data to obtain a number of task types involved in the target scenario and the difficulty levels of each task type. The model construction module 33 is used to construct a number of question-answering models based on a number of task types and the difficulty levels of each task type; wherein, any question-answering model is applicable to at least one task type. The model selection module 34 is used to select a question-answering model as the target model for answering the new input question in response to the new input question in the target scenario.

[0062] In the above solution, the intelligent question-answering device 30 acquires a number of historical data of the question-answering system in the target scenario, and the historical data at least includes the input questions and output answers of the question-answering system. It analyzes based on a number of historical data to obtain a number of task types involved in the target scenario and the difficulty levels of each task type. Thus, based on a number of task types and the difficulty levels of each task type, it constructs a number of question-answering models, and any question-answering model is applicable to at least one task type. Furthermore, in response to the new input question in the target scenario, it selects a question-answering model as the target model for answering the new input question. Therefore, it can analyze a number of task types involved in the target scenario from the historical data of the target scenario, and combine the difficulty levels of each task type to differentially construct different question-answering models to adaptively select a question-answering model to answer the new input question. Compared with uniformly using a single model to answer different questions, it can improve the model operation efficiency of intelligent question-answering, reduce the model computing power consumption, and at the same time take into account the different requirements of different tasks for the model capabilities.

[0063] In some disclosed embodiments, the data analysis module 32 includes a task classification sub-module, which is used to classify the input questions in a number of historical data to obtain a number of task types involved in the target scenario. The data analysis module 32 includes a difficulty analysis sub-module, which is used to analyze the output answers of the input questions under the task type by the question-answering system to obtain the difficulty level of the task type.

[0064] In some disclosed embodiments, the historical data further includes the evaluation feedback of the output answers. The difficulty analysis sub-module includes a feedback statistics unit, which is used to statistically analyze the evaluation feedback of each output answer under the task type by the question-answering system to obtain the response ability of the question-answering system to the task type. The difficulty analysis sub-module includes a degree determination unit, which is used to obtain the difficulty level of the task type based on the response ability of the question-answering system to the task type.

[0065] In some disclosed embodiments, the model construction module 33 includes a capability analysis sub-module for analyzing the model capabilities required for several task types respectively; the model construction module 33 includes an architecture determination sub-module for determining the model architectures required for different model capabilities respectively; the model construction module 33 includes a scale determination sub-module for determining the parameter scale of the model architecture required for the model capabilities based on the difficulty level of the task types for which the model capabilities are required; the model construction module 33 includes a model construction sub-module for constructing multiple question-and-answer models based on the model architectures required for different model capabilities respectively and their parameter scales.

[0066] In some disclosed embodiments, the target scenario is auxiliary teaching. The capability analysis sub-module includes a first response unit for determining that the model capability is semantic understanding capability in response to the task type being any one of confirmation of problem-solving ideas, confirmation of problem-solving answers, inquiry about specific details of known information, and confirmation of specific details of known information; and / or, the capability analysis sub-module includes a second response unit for determining that the model capability is the knowledge system construction capability with the help of external knowledge in response to the task type being any one of knowledge point query, generalization of problem-solving ideas for similar knowledge points, regression summary of knowledge points, and discrimination of similar knowledge points; and / or, the capability analysis sub-module includes a third response unit for determining that the model capability is the auxiliary reasoning with the help of external plug-in capabilities in response to the task type being any one of mathematical calculation and graphic rendering for auxiliary understanding; and / or, the capability analysis sub-module includes a fourth response unit for determining that the model capability is complex logical reasoning capability in response to the task type being any one of unclear requirements that need to be gradually guided, inquiry about complex problem-solving ideas, and extended questions.

[0067] In some disclosed embodiments, the target scenario is auxiliary teaching. In the case where the model capability is semantic understanding capability, the question-and-answer model required for the model capability is the first large model of the first parameter order of magnitude, and / or, in the case where the model capability is the knowledge system construction capability with the help of external knowledge, the question-and-answer model required for the model capability is the second large model of the second parameter order of magnitude and based on retrieval-augmented generation, and / or, in the case where the model capability is the auxiliary reasoning with the help of external plug-in capabilities, the question-and-answer model required for the model capability is the third large model of the third parameter order of magnitude and based on plug-in assistance, and / or, in the case where the model capability is complex logical reasoning capability, the question-and-answer model required for the model capability is the fourth large model of the fourth parameter order of magnitude.

[0068] In some disclosed embodiments, the target model is selected by a routing model from multiple question-and-answer models for a new input question. The intelligent question-and-answer device 30 includes an answer sampling module, which is used to perform answer sampling on sample questions based on multiple question-and-answer models respectively, and obtain sample answers of multiple question-and-answer models for the sample questions at each sampling; the intelligent question-and-answer device 30 includes a hit detection module, which is used to obtain detection results indicating whether the answers are hit by multiple question-and-answer models at the corresponding sampling based on the acceptability of each sample answer at each sampling; the intelligent question-and-answer device 30 includes a hit statistics module, which is used to perform statistics on the detection results of multiple question-and-answer models at each sampling respectively, and obtain sample routing labels of the sample questions; the intelligent question-and-answer device 30 includes a model training module, which is used to train the routing model based on the sample questions labeled with sample routing labels until the training end condition is met.

[0069] In some disclosed embodiments, the acceptability is any one of unacceptable, acceptable, and excellent. And the question-and-answer model whose detection result indicates that the answer is hit at each sampling is used as the hit model. The hit detection module includes a preferred sub-module, which is used to preferentially select the question-and-answer model with the smallest parameter scale as the hit model among the question-and-answer models whose acceptability of the sample answer is above acceptable; the hit detection module includes a secondary selection sub-module, which is used to secondarily select the question-and-answer model with a higher acceptability as the hit model in the case where the selection fails due to the same parameter scale; the hit detection module includes a re-secondary selection sub-module, which is used to re-secondary select the question-and-answer model with a higher inference efficiency as the hit model in the case where the secondary selection fails due to the same acceptability.

[0070] In some disclosed embodiments, the model training module includes a model routing sub-module, which is used to route the sample questions based on the routing model to obtain predicted routing labels of multiple question-and-answer models; the model training module includes a label processing sub-module, which is used to perform label smoothing and normalization based on the sample routing labels to obtain target routing labels; among them, the predicted routing labels include predicted probability values that multiple question-and-answer models are respectively selected to answer the sample questions; the model training module includes a parameter adjustment sub-module, which is used to adjust the network parameters of the routing model based on the distribution difference between the predicted routing labels and the target routing labels.

[0071] In some disclosed embodiments, the model selection module 34 includes a label prediction sub-module, which is used to make a prediction based on the new input question to obtain a routing label; the model selection module 34 includes a model sorting sub-module, which is used to sort multiple question-and-answer models in ascending order of parameter scale; the model selection module 34 includes a model selection sub-module, which is used to sequentially select candidate models among the multiple question-and-answer models after sorting, and detect whether each question-and-answer model is selected as the target model based on the routing label until the target model is selected and determined.

[0072] In some disclosed embodiments, the routing label includes probability values of multiple question-and-answer models respectively being selected to answer a new input question. The model selection sub-module is specifically configured to, among the multiple question-and-answer models after sorting, sequentially select candidate models, detect whether the probability value of a candidate model being selected to answer the new input question is not lower than the probability threshold of the candidate model, in response to the detection being not lower, select the candidate model as the target model, and in response to the detection being lower, return to the step of sequentially selecting candidate models among the multiple question-and-answer models after sorting until the target model is selected.

[0073] Please refer to Figure 4 , Figure 4 FIG. is a schematic framework diagram of an embodiment of the electronic device of the present application. The electronic device 40 at least includes a memory 41 and a processor 42 that are coupled to each other. At least program instructions are stored in the memory 41, and the processor 42 is configured to execute the program instructions to implement the steps in any of the above-mentioned embodiments of the intelligent question-and-answer method. Specifically, reference can be made to the foregoing disclosed embodiments, which will not be elaborated herein. As a possible example, the electronic device 40 may include, but is not limited to, a smart phone, a tablet computer, a server, etc. The specific type of the electronic device 40 is not limited herein.

[0074] Specifically, the processor 42 is configured to control itself and the memory 41 to implement the steps in any of the above-mentioned embodiments of the intelligent question-and-answer method. The processor 42 may also be referred to as a CPU (Central Processing Unit). The processor 42 may be an integrated circuit chip with signal processing capabilities. The processor 42 may also be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Additionally, the processor 42 may be implemented jointly by integrated circuit chips.

[0075] In the above solution, the electronic device 40 obtains a number of historical data of the question-and-answer system in the target scenario, and the historical data at least includes the input questions and output answers of the question-and-answer system. Based on the analysis of the number of historical data, a number of task types involved in the target scenario and the difficulty levels of each task type are obtained. Thus, based on the number of task types and the difficulty levels of each task type, multiple question-and-answer models are constructed, and any question-and-answer model is applicable to at least one task type. Furthermore, in response to a new input question in the target scenario, a question-and-answer model is selected as the target model for answering the new input question. Therefore, it is possible to analyze a number of task types involved in the target scenario from the historical data of the target scenario, and combine the difficulty levels of each task type to differentially construct different question-and-answer models to adaptively select a question-and-answer model to answer the new input question. Compared with uniformly using a single model to answer different questions, it is possible to improve the model operation efficiency of intelligent question answering, reduce the model computing power consumption, and at the same time take into account the different task requirements for the model capabilities.

[0076] Please refer to Figure 5 , Figure 5 FIG. is a schematic framework diagram of an embodiment of the computer-readable storage medium 50 of the present application. The computer-readable storage medium 50 stores program instructions 51 that can be run by a processor, and the program instructions 51 are used to implement the steps in any of the above-mentioned intelligent question-and-answer method embodiments.

[0077] In the above solution, the computer-readable storage medium 50 obtains a number of historical data of the question-and-answer system in the target scenario, and the historical data at least includes the input questions and output answers of the question-and-answer system. Based on the analysis of the number of historical data, a number of task types involved in the target scenario and the difficulty levels of each task type are obtained. Thus, based on the number of task types and the difficulty levels of each task type, multiple question-and-answer models are constructed, and any question-and-answer model is applicable to at least one task type. Furthermore, in response to a new input question in the target scenario, a question-and-answer model is selected as the target model for answering the new input question. Therefore, it is possible to analyze a number of task types involved in the target scenario from the historical data of the target scenario, and combine the difficulty levels of each task type to differentially construct different question-and-answer models to adaptively select a question-and-answer model to answer the new input question. Compared with uniformly using a single model to answer different questions, it is possible to improve the model operation efficiency of intelligent question answering, reduce the model computing power consumption, and at the same time take into account the different task requirements for the model capabilities.

[0078] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0079] The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments. For their similarities, reference can be made to each other. For the sake of brevity, they will not be elaborated herein again.

[0080] In several embodiments provided in the present application, it should be understood that the disclosed methods and apparatuses can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the apparatus or unit can be in electrical, mechanical or other forms.

[0081] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0082] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0083] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0084] If the technical solution of this application involves personal information, before the product applying the technical solution of this application processes personal information, it has clearly informed the personal information processing rules and obtained the independent consent of the individual. If the technical solution of this application involves sensitive personal information, before the product applying the technical solution of this application processes sensitive personal information, it has obtained the individual's separate consent and at the same time meets the requirements of "express consent". For example, at personal information collection devices such as cameras, a clear and prominent sign is set up to inform that the personal information collection scope has been entered and personal information will be collected. If an individual voluntarily enters the collection scope, it is regarded as consenting to the collection of their personal information; or on the device for personal information processing, when the personal information processing rules are informed by obvious signs / information, personal authorization is obtained through pop-up messages or by asking the individual to upload their personal information by themselves, etc.; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

Claims

1. An intelligent question-answering method, characterized in that, Including: Obtain a number of historical data of the question-and-answer system in the target scenario; wherein, the historical data at least includes the input questions and output answers of the question-and-answer system; Analyze based on the number of historical data to obtain a number of task types involved in the target scenario and the difficulty levels of each task type; Based on the number of task types and the difficulty levels of each task type, construct multiple question-and-answer models; wherein, any one of the question-and-answer models is applicable to at least one of the task types; In response to a new input question in the target scenario, select the question-and-answer model as the target model for answering the new input question.

2. The method according to claim 1, characterized in that The analyzing based on the number of historical data to obtain a number of task types involved in the target scenario and the difficulty levels of each task type includes: Classify the input questions in the number of historical data to obtain a number of task types involved in the target scenario; Analyze the output answers of the input questions under the task type by the question-and-answer system to obtain the difficulty level of the task type.

3. The method according to claim 2, wherein The historical data further includes evaluation feedback of the output answers. The analyzing the output answers of the input questions under the task type by the question-and-answer system to obtain the difficulty level of the task type includes: Statistically analyze the evaluation feedback of each output answer under the task type by the question-and-answer system to obtain the response ability of the question-and-answer system for the task type; Based on the response ability of the question-and-answer system for the task type, obtain the difficulty level of the task type.

4. The method according to claim 1, wherein The constructing multiple question-and-answer models based on the number of task types and the difficulty levels of each task type includes: Analyze the model capabilities required for each of the number of task types; Determine the model architectures required for different model capabilities, and based on the difficulty levels of the task types for which the model capabilities are required, determine the parameter scales of the model architectures required for the model capabilities; Based on the model architectures required for different model capabilities and their parameter scales, construct the multiple question-and-answer models.

5. The method according to claim 4, wherein When the target scenario is auxiliary teaching, the analyzing the model capabilities required for each of the number of task types includes: In response to the task type being any one of confirmation of problem-solving ideas, confirmation of problem-solving answers, asking for specific details of known information, and confirmation of specific details of known information, determine that the model capability is semantic understanding ability; And / or, in response to the task type being any one of knowledge point query, generalization of problem-solving ideas for similar knowledge points, regression summary of knowledge points, and discrimination of similar knowledge points, determine that the model capability is the ability to construct a knowledge system with the help of external knowledge; And / or, in response to the task type being any one of mathematical calculation and graphic rendering for auxiliary understanding, determine that the model capability is the ability to assist reasoning with the help of external plugin capabilities; And / or, in response to the task type being any one of unclear requirements requiring step-by-step guidance, asking for complex problem-solving ideas, and extended questions, determine that the model capability is complex logical reasoning ability.

6. The method according to claim 4, wherein The target scenario is auxiliary teaching. When the model ability is semantic understanding ability, the Q&A model required for the model ability is the first large model with the first parameter order of magnitude, and / or when the model ability is the knowledge system construction ability with the help of external knowledge, the Q&A model required for the model ability is the second large model with the second parameter order of magnitude and based on retrieval augmented generation, and / or when the model ability is the auxiliary reasoning with the help of external plugin ability, the Q&A model required for the model ability is the third large model with the third parameter order of magnitude and based on plugin assistance, and / or when the model ability is complex logical reasoning ability, the Q&A model required for the model ability is the fourth large model with the fourth parameter order of magnitude.

7. The method according to claim 1, characterized in that The target model is selected from the multiple Q&A models by the routing model for the new input question. The training method of the routing model further includes: Performing answer sampling on the sample questions respectively by the multiple Q&A models to obtain the sample answers of the multiple Q&A models to the sample questions at each sampling; Based on the acceptability of each sample answer at each sampling, obtaining the detection results indicating whether the answers are hit by the multiple Q&A models at the corresponding sampling; Based on the detection results of the multiple Q&A models at each sampling respectively, performing statistics to obtain the sample routing labels of the sample questions; Training the routing model based on the sample questions marked with the sample routing labels until the training end condition is met.

8. The method according to claim 7, characterized in that The acceptability is any one of unacceptable, acceptable, and excellent. And the Q&A model whose detection result indicates that the answer is hit at each sampling is used as the hit model. The obtaining the detection results indicating whether the answers are hit by the multiple Q&A models at the corresponding sampling based on the acceptability of each sample answer at each sampling includes: Among the Q&A models with the acceptability of the sample answer above the acceptable level, preferably selecting the Q&A model with the smallest parameter scale as the hit model; In the case where the selection fails due to the same parameter scale, secondly selecting the Q&A model with a higher acceptability as the hit model; In the case where the secondary selection fails due to the same acceptability, thirdly selecting the Q&A model with a higher reasoning efficiency as the hit model.

9. The method according to claim 7, characterized in that The training the routing model based on the sample questions marked with the sample routing labels includes: Routing the sample questions by the routing model to obtain the predicted routing labels of the multiple Q&A models, and performing label smoothing and normalization based on the sample routing labels to obtain the target routing labels; wherein, the predicted routing labels include the predicted probability values of the multiple Q&A models being respectively selected to answer the sample questions; Adjusting the network parameters of the routing model based on the distribution difference between the predicted routing labels and the target routing labels.

10. The method according to claim 1, characterized in that, The selecting the Q&A model as the target model for answering the new input question includes: Performing prediction based on the new input question to obtain the routing label; Sort the multiple question-and-answer models in ascending order of parameter scale; Among the multiple question-and-answer models after sorting, sequentially detect whether each of the question-and-answer models is selected as the target model based on the routing tag until the target model is determined.

11. The method according to claim 10, wherein The routing tag includes the probability values of the multiple question-and-answer models being respectively selected to answer the new input question. The sequentially detecting whether each of the question-and-answer models is selected as the target model based on the routing tag until the target model is determined includes: Among the multiple question-and-answer models after sorting, sequentially select candidate models; Detect whether the probability value of the candidate model being selected to answer the new input question is not lower than the probability threshold of the candidate model; In response to the detection being not lower, select the candidate model as the target model; In response to the detection being lower, return to the step of sequentially selecting candidate models among the multiple question-and-answer models after sorting until the target model is obtained.

12. An intelligent question-answering device, characterized in that, Include: A data acquisition module for acquiring a number of historical data of the question-and-answer system in a target scenario; wherein the historical data at least includes the input questions and output answers of the question-and-answer system; A data analysis module for analyzing based on the number of historical data to obtain a number of task types involved in the target scenario and the difficulty level of each task type; A model construction module for constructing multiple question-and-answer models based on the number of task types and the difficulty level of each task type; wherein any one of the question-and-answer models is applicable to at least one of the task types; A model selection module for, in response to a new input question in the target scenario, selecting the question-and-answer model as the target model for answering the new input question.

13. An electronic device, characterized in that, At least includes a memory and a processor coupled to each other. At least program instructions are stored in the memory, and the processor is configured to execute the program instructions to implement the intelligent question-and-answer method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, Stores program instructions that can be run by a processor, and the program instructions are used to implement the intelligent question-and-answer method according to any one of claims 1 to 11.

Images