Problem solving method, device, computer equipment and storage medium

Abstract

The present application relates to a method, device, computer equipment and storage medium for problem processing. The method comprises: obtaining a question query statement; searching for candidate questions matching the question query statement from a preset question library; respectively forming a question combination with the question query statement and each candidate question, inputting the combination into a similarity prediction model, and outputting the similarity between the candidate questions in the question combination and the question query statement; the similarity prediction model is obtained by adjusting the model parameters of the basic language model by using sample question combinations and corresponding similarity labels from multiple data sources as training sets through multi-task iterative training; selecting the candidate question with the highest similarity as the target question matching the question query statement. The use of this method can improve the processing accuracy.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a problem processing method, apparatus, computer equipment and storage medium. Background Art

[0002] With the rapid development of science and technology, natural language processing technology has also been developing continuously. Natural language processing technology is usually used to process sentences. For example, natural language processing technology is used to judge sentence similarity.

[0003] In the traditional method, the sample data is directly used to train the initial model framework to obtain a model for similarity recognition. However, due to the limited amount of labeled data, the prediction accuracy of the model is relatively low, which leads to a relatively low accuracy rate in sentence processing. Summary of the invention

[0004] Based on this, it is necessary to provide a problem processing method, device, computer equipment and storage medium that can improve the accuracy of sentence processing in response to the above technical problems.

[0005] A method for solving a problem, the method comprising:

[0006] Get the query statement;

[0007] Searching for candidate questions matching the question query statement from a preset question library;

[0008] The question query statement and each of the candidate questions are respectively combined into a question combination, which is input into a similarity prediction model, and the similarity between the candidate questions in the question combination and the question query statement is output; the similarity prediction model is obtained by adjusting the model parameters of the basic language model through multi-task iterative training using sample question combinations and corresponding similarity labels from multiple data sources as training sets;

[0009] The candidate question with the highest similarity is selected as the target question matching the question query statement.

[0010] A problem handling device, comprising:

[0011] A query module is used to obtain a question query statement; and search for candidate questions matching the question query statement from a preset question library;

[0012] A similarity prediction module is used to form a question combination with the question query statement and each of the candidate questions, input the combination into a similarity prediction model, and output the similarity between the candidate questions in the question combination and the question query statement; the similarity prediction model is obtained by adjusting the model parameters of the basic language model through multi-task iterative training using sample question combinations and corresponding similarity labels from multiple data sources as training sets;

[0013] The matching module is used to select the candidate question with the highest similarity as the target question that matches the question query statement.

[0014] In one embodiment, the apparatus comprises:

[0015] The model training module is used to obtain a training set; the training set includes sample question combinations from multiple data sources and similarity labels corresponding to the sample question combinations; the training set is used to iteratively perform multi-task training on the basic language model to obtain a similarity prediction model.

[0016] In one embodiment, the model training module is also used to iteratively perform multi-task training on each of the at least two basic language models using the training set to obtain a prediction model corresponding to the basic language model; different basic language models have different processing granularities; and a similarity prediction model is obtained based on the prediction models corresponding to each basic language model.

[0017] In one embodiment, the model training module is also used to use the training set to iteratively perform multi-task training on the first basic language model to obtain a prediction model corresponding to the first basic language model; the first basic language model is a basic language model with characters as the processing granularity; the training set is used to iteratively perform multi-task training on the second basic language model to obtain a prediction model corresponding to the second basic language model; the second basic language model is a basic language model with words as the processing granularity.

[0018] In one embodiment, the model training module is also used to obtain preset hyperparameters corresponding to the basic language model; the preset hyperparameters include an initial learning rate; the initial learning rate is used as the current learning rate, and according to the current learning rate, the training set is used to perform a current round of multi-task training on the basic language model; after the current round of training is completed, the learning rate is reduced, and the reduced learning rate is used as the current learning rate, the next round is used as the current round, and the current round of multi-task training of the basic language model is returned to until the iteration stop condition is reached to obtain a prediction model corresponding to the basic language model.

[0019] In one embodiment, the model training module is also used to input the training sets corresponding to multiple data sources into the basic language model of the current round for multi-task training to obtain the output vector corresponding to the sample problem combination in the training set; input the output vector corresponding to the sample problem combination of the same data source into the fully connected layer corresponding to the data source for full connection processing; determine the sub-loss function corresponding to each of the data sources based on the result of the full connection processing; determine the final loss function based on each of the sub-loss functions; and adjust the model parameters of the basic language model based on the loss function and the current learning rate.

[0020] In one embodiment, the loss function is obtained by weighted summing up the sub-loss functions according to corresponding weights; the model training module is also used to determine the data source corresponding to the sub-loss function in the loss function, and for each data source corresponding to the sub-loss function, determine the fully connected layer corresponding to the data source, and update the parameters of the fully connected layer according to the sub-loss function.

[0021] In one embodiment, the model training module is also used to use the training set to iteratively perform multi-task training on the basic language model to obtain a corresponding prediction model for each round; select a target prediction model from the prediction models; the prediction accuracy of the target prediction model is higher than that of the non-target prediction model in the prediction model; predict the unlabeled sample question combination according to the target prediction model, and determine the sample label of the unlabeled sample question combination according to the prediction result; use the unlabeled sample question combination and the sample label as sample data to iteratively train the initial similarity prediction model to obtain a similarity prediction model.

[0022] In one embodiment, the model training module is also used to verify the prediction accuracy of the prediction model corresponding to each round according to a preset verification set; and screen a preset number of prediction models from the prediction models obtained after each round of multi-task training in order of prediction accuracy from high to low to obtain a target prediction model.

[0023] In one embodiment, there are multiple target prediction models; the model training module is also used to obtain unlabeled sample question combinations; the unlabeled sample question combinations are input into each of the target prediction models respectively, and the prediction results corresponding to the unlabeled sample question combinations are output; the prediction results corresponding to the same unlabeled sample question combination are averaged, and the average value is used as the sample label of the unlabeled sample question combination.

[0024] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps in the problem handling method described in each embodiment of the present application.

[0025] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the steps in the problem handling method described in each embodiment of the present application.

[0026] The above-mentioned problem processing method, device, computer equipment and storage medium first search for candidate questions matching the problem query statement from a preset question library, then respectively combine the problem query statement with each of the candidate questions to form a problem combination, input the combination into a similarity prediction model, and output the similarity between the candidate questions and the problem query statement in the problem combination. Since the similarity prediction model is obtained by adjusting the model parameters of the basic language model through multi-task iterative training using sample problem combinations and corresponding similarity labels from multiple data sources as training sets, the similarity prediction model obtained after the adjustment inherits the strong generalization and strong transfer capabilities of the basic language model, and can therefore accurately predict the similarity between the candidate questions and the problem query statement, thereby accurately translating and matching the problem query statement expressed in natural language to the existing target question, thereby improving the accuracy of problem processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 An application environment diagram of a problem handling method in an embodiment;

[0028] Figure 2 A schematic diagram of a problem solving method in one embodiment;

[0029] Figure 3 A schematic diagram of a target problem matched in one embodiment;

[0030] Figure 4 A schematic diagram of an interface for crawling similar questions in an embodiment;

[0031] Figure 5 A schematic diagram of a public data source in an embodiment;

[0032] Figure 6 Schematic diagram of the principle of multi-task training of a basic language model in one embodiment;

[0033] Figure 7 A schematic diagram of obtaining an output vector in one embodiment;

[0034] Figure 8 A schematic diagram of a prediction effect in an embodiment;

[0035] Fig. 9 A schematic diagram of prediction effect in another embodiment;

[0036] Fig.10 A performance comparison analysis diagram in one embodiment;

[0037] Fig.11 is a structural block diagram of a problem handling device in an embodiment;

[0038] Fig.12 is a structural block diagram of a problem handling device in another embodiment;

[0039] Fig.13 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0041] The problem solving method provided in this application can be applied to Figure 1 In the application environment shown, the terminal 102 and the server 104 communicate via a network. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, smart speakers, and portable wearable devices, and the server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0042] The user can input a question query statement through the terminal 102, and the terminal 102 can send the question query statement to the server 104. The server 104 can search for candidate questions that match the question query statement from the preset question library. The server 104 pre-combines sample questions from multiple data sources and corresponding similarity labels as a training set, adjusts the model parameters of the basic language model through multi-task iterative training, and trains to obtain a similarity prediction model. The server 104 can respectively combine the question query statement with each candidate question to form a question combination, input it into the similarity prediction model, and output the similarity between the candidate question and the question query statement in the question combination. The server 104 can select the candidate question with the highest similarity as the target question that matches the question query statement. Among them, the question combination can be a question list, that is, the questions are presented in the form of a list.

[0043] It is understandable that the server 104 may return the target question to the terminal 102. The server 104 may also directly query the corresponding answer from the preset answer library with the target question.

[0044] It should be noted that when the terminal has the ability to execute the problem handling method in the embodiment of the present application, it can also be executed by the terminal itself, and is not limited to being executed by the server.

[0045] It can be understood that the problem handling method in each embodiment of the present application is equivalent to using artificial intelligence technology to automatically translate the question query statement expressed in natural language into the target question.

[0046] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0047] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0048] It can be understood that the problem handling method in each embodiment of the present application is equivalent to using natural language processing technology and machine learning technology. For example, translating a question query statement expressed in natural language into a target question is equivalent to using natural language processing technology. For another example, the process of model training is equivalent to using machine learning technology.

[0049] Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

[0050] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0051] In one embodiment, Figure 2 As shown, a method for solving the problem is provided, which is applied to Figure 1 The method includes the following steps:

[0052] S202, obtaining a question query statement.

[0053] Among them, the question query statement is a question expressed in natural language.

[0054] Specifically, the user may input a question query statement in the terminal by voice or text, and the terminal may send the question query statement to the server.

[0055] It is understandable that the server may directly obtain the stored question query statement locally, or may obtain the question query statement sent by the terminal in real time.

[0056] To make it easier to understand question query statements, let's take an example. For example, "What is the name of the largest volcano in the world?" is a question expressed in natural language, which belongs to a question query statement. Another example is "Why is the sky blue in the world?" is also a question query statement expressed in natural language.

[0057] In one embodiment, an application supporting voice recognition can be installed and run in the terminal. Among them, the user can use natural language to speak the question by voice, and the terminal can use the application to perform voice recognition and generate a question query statement. It can be understood that an application supporting voice recognition is not limited to only voice recognition, and the application can also have other functions. The application supporting voice recognition can be at least one of an audio and video playback application, a sound processing application in a smart speaker, and a human-computer interaction application (i.e., an application that can realize human-computer dialogue).

[0058] In one embodiment, the terminal may provide an input interface, and the user may input a desired query statement in the input interface, that is, input the query statement in text form.

[0059] In other embodiments, the user may also input a question query statement to the terminal by means of gestures or writing in the air. It is understood that in this case, the terminal may detect the user's gestures or writing paths in the air to obtain the input question query statement.

[0060] S204: searching for candidate questions matching the question query statement from a preset question library.

[0061] The preset question library is a preset question library. The preset question library includes a collection of known questions. It can be understood that the questions in the preset question library are more standardized than the question query statements input by the user. The question query statements are more colloquial than the questions in the preset question library.

[0062] Specifically, the server may perform at least one of word segmentation and part-of-speech judgment on the question query statement to generate a word vector corresponding to the question query statement. The server may perform a similarity comparison between the word vector and each question in a preset question library to obtain a similarity score, and then select a question with a preset similarity score in the front from the preset question library as a candidate question matching the question query statement.

[0063] It can be understood that there may be at least one candidate question, which is equivalent to searching for a set of candidate questions matching the question query statement from a preset question library.

[0064] S206, respectively combining the question query statement and each candidate question into a question combination, inputting the combination into a similarity prediction model, and outputting the similarity between the candidate question and the question query statement in the question combination.

[0065] The similarity prediction model is obtained by adjusting the model parameters of the basic language model through multi-task iterative training by combining sample questions from multiple data sources and the corresponding similarity labels as the training set.

[0066] The basic language model is a pre-trained language model. A pre-trained language model is a model that has been pre-trained with a large number of data sets (this type of model is often large and requires a lot of memory resources for training). It can be understood that you can use the training set to fine-tune the pre-trained language model according to your own model requirements and obtain a model that achieves your own tasks.

[0067] In one embodiment, the basic language model can be a BERT model (i.e., a BERT-base model). The BERT model is trained based on Chinese Wikipedia data and is a general domain pre-trained language model. Since the question query statement itself may be more colloquial, in order to make the model more compatible with the question query statement, it is necessary to add a training set to the BERT model to continue training for domain adaptation. The training set includes sample question combinations from multiple data sources and corresponding similarity labels.

[0068] Domain Adaptation, a related field of machine learning and transfer learning, is used to learn models from a source data distribution that perform well on a different (but related) target data distribution.

[0069] It is understandable that the basic language model can also be other models. For example, the WWM-BERT model is a Chinese BERT pre-training model based on Whole Word Masking released by Harbin Institute of Technology, which is an improved model of the BERT model.

[0070] Data source refers to the data source that provides sample question combinations. It can be understood that different data sources are used to implement different training tasks in multi-task iterative training. Among them, question combination refers to a combination of a pair of questions. For example, the question query statement is "Which is the highest mountain in Guangdong Province?" There are 3 candidate questions queried from the preset question library, as shown in Table 1:

[0071] Which mountain is the highest in Shenzhen? Which is the highest mountain in the world? Where is the highest mountain in Guangdong?

[0072] (Table 1)

[0073] Then, by combining the question query statement with each candidate question to form a question combination, we can get 3 question combinations, namely, question combination 1: "Which is the highest mountain in Guangdong Province?; Which mountain is the highest in Shenzhen?", question combination 2: "Which is the highest mountain in Guangdong Province?; Which mountain is the highest in the world?", question combination 3: "Which is the highest mountain in Guangdong Province?; Where is the highest mountain in Guangdong?".

[0074] The server inputs the question combination into the pre-trained similarity prediction model and outputs the similarity corresponding to the question combination. That is, the similarity between the candidate question and the question query statement in the question combination. Take question combination 1: "Which is the highest mountain in Guangdong Province?; Which mountain is the highest in Shenzhen?" as an example, input it into the similarity prediction model, and the output is the similarity between the question query statement "Which is the highest mountain in Guangdong Province?" and the candidate question "Which mountain is the highest in Shenzhen?"

[0075] S208, selecting the candidate question with the highest similarity as the target question that matches the question query statement.

[0076] Specifically, the terminal may sort the candidate questions in descending order of similarity, and select the candidate question with the highest similarity as the target question matching the question query statement according to the sorting result.

[0077] It is understandable that in step S204, the candidate questions found may have an initial ranking. The terminal may re-rank the candidate questions in the order of high to low similarity in the manner of step S208.

[0078] For ease of understanding, Table 2 is used as an example for illustration.

[0079] Where is the highest mountain in Guangdong? 0.9 Which mountain is the highest in Shenzhen? 0.4 Which is the highest mountain in the world? 0.3

[0080] (Table 2)

[0081] It can be understood that the numbers after Table 2 are the similarities between each candidate question and the question query statement. It can be seen from Table 2 that, compared with the initial sorting in Table 1, the candidate questions have been re-sorted in order of similarity from high to low. Among them, "Where is the highest mountain in Guangdong?" has the highest similarity with the question query statement "Which is the highest mountain in Guangdong Province?", so the candidate question can be selected from the preset question library as the target question for matching the question query statement. That is, using the method in the embodiment of the present application, the question query statement expressed in natural language "Which is the highest mountain in Guangdong Province?" can be translated into "Where is the highest mountain in Guangdong?" This target question exists in the preset question library.

[0082] Figure 3 FIG. 1 is a schematic diagram of a target problem matched in an embodiment. Figure 3 , the left side is the "question query statement", and the right side is the "target question" matched by translation. For example, the user wants to know what traditional Chinese crafts are there, but when entering the question, he only said "Chinese crafts" in a more colloquial way. Then, through the problem processing method in the embodiment of the present application, it can be translated into the question "What traditional crafts are there in China" that already exists in the question library.

[0083] It is understandable that the server can directly return the target question to the terminal for output display. The server can also search for the corresponding answer from the preset answer library according to the target question.

[0084] In one embodiment, the method further includes: determining a preset answer library; the answers in the preset answer library have a corresponding relationship with the questions in the preset question library; and searching for an answer corresponding to the target question from the preset answer library according to the corresponding relationship.

[0085] The preset answer library is a preset answer library. The preset answer library includes a collection of answers. It can be understood that the answers in the preset answer library are answers to known questions. Therefore, the answers in the preset answer library have a corresponding relationship with the questions in the preset question library.

[0086] The server may search for an answer corresponding to the target question from the preset answer library according to the correspondence between the answers in the preset answer library and the questions in the preset question library.

[0087] For example, if the query sentence "Which is the highest mountain in Guangdong Province?" matches the target question "Which mountain in Guangdong Province is the highest?" in the preset question library, the answer corresponding to "Which mountain in Guangdong Province is the highest?" can be found in the preset answer library. It can be understood that by accurately finding the target question of the query sentence, the accuracy of question-answering processing can be improved, that is, the accuracy of the answer can be improved.

[0088] The above-mentioned problem processing method first searches for candidate questions that match the problem query statement from a preset problem library, then respectively combines the problem query statement with each of the candidate questions into a problem combination, inputs the combination into a similarity prediction model, and outputs the similarity between the candidate questions and the problem query statement in the problem combination. Since the similarity prediction model is obtained by adjusting the model parameters of the basic language model through multi-task iterative training using sample problem combinations and corresponding similarity labels from multiple data sources as training sets, the similarity prediction model obtained after the adjustment inherits the strong generalization and strong transfer capabilities of the basic language model, and can therefore accurately predict the similarity between the candidate questions and the problem query statement, thereby accurately translating and matching the problem query statement expressed in natural language to the existing target question, thereby improving the accuracy of problem processing.

[0089] In one embodiment, the training steps of the similarity prediction model include: obtaining a training set; the training set includes sample question combinations from multiple data sources and similarity labels corresponding to the sample question combinations; using the training set, iteratively performing multi-task training on the basic language model to obtain a similarity prediction model.

[0090] In one embodiment, different data sources belong to different domains.

[0091] In one embodiment, the data source may include a private data source and a public data source. Among them, the private data source refers to the data source prepared by the model training party itself. The public data source is a data source provided by a third party. For example, the data source provided by some competition organizers.

[0092] In one embodiment, similar questions can be crawled from the web page by a crawler, and similar questions can be deduplicated. Then, similar questions with a high frequency of occurrence (i.e., a frequency of occurrence greater than or equal to a preset frequency threshold) can be combined into a sample question combination, and similarity tags can be manually added to the sample question combination (for example, if question a is similar to question b, a similarity tag of "1" is added) to obtain a private data source.

[0093] In one embodiment, the initial private data source can also be expanded using the principle of equivalence and similarity to obtain the final private data source. The principle of equivalence and similarity states that problems with common similar problems are equivalent to having a similar relationship. For example, if problem a is similar to problem b, and problem b is similar to problem c, then problem a is similar to problem c.

[0094] Figure 4 A schematic diagram of an interface for crawling similar questions in an embodiment. Figure 4 Take the Baidu Knows page as an example. In this page, there are 8 questions similar to "A typhoon is coming, what should we do?" Then, these similar questions can be combined with "A typhoon is coming, what should we do?" to form a sample question combination. In addition, since these similar questions are similar to "A typhoon is coming, what should we do?", these similar questions also have similar relationships and can also form a sample question combination.

[0095] In one embodiment, the public data source may include a question semantic matching dataset constructed by a top international conference on natural language processing (e.g., LCQMC, a question semantic matching dataset constructed by Harbin Institute of Technology at the top international conference on natural language processing COLING2018). It can be understood that the public data source contains 240,000 pairs of Chinese sentences with colloquial descriptions (i.e., sample question combinations), and the similarity label is 1 or 0, 1 for semantically similar sentences, and 0 for semantically dissimilar sentences.

[0096] In one embodiment, the public data source may include a data set provided by a natural language processing intelligent customer service competition. For example, ATEC, a data set provided by the 2018 ATEC Ant Financial NLP Intelligent Customer Service Competition. It can be understood that the public data source contains 480,000 pairs of sentences in vertical fields (i.e., sample question combinations), mainly in the financial field, with similarity labels of 1 or 0, 1 for semantically similar sentences, 0 for semantically dissimilar sentences, and a label accuracy rate of 96%.

[0097] In one embodiment, the public data source may include a data set provided by a question matching competition. For example, CCKS, a data set provided by the 2018 WeBank Customer Question Matching Competition. It can be understood that the public data source contains 100,000 pairs of sentences in vertical fields (i.e., sample question combinations). All corpora come from the original intelligent customer service logs in the banking field, and have been screened and manually annotated for intent matching. The similarity label is 1 or 0, 1 for semantically similar sentences, and 0 for semantically dissimilar sentences.

[0098] Figure 5 FIG. 1 is a schematic diagram of a public data source in an embodiment. Figure 5 , showing three public data sources. It should be noted that, for each public data source, the two sentences shown in the same row are a set of sample question combinations, and the "1" or "0" marked in the same row is the similarity label corresponding to the sample question combination.

[0099] Among them, (a) is an example of Chinese sentence pairs (i.e., sample question combinations) and corresponding similarity labels in the public data source LCQMC. For example, "Do you think she is pretty?" and "Do you think variety show A is good?" belong to a sentence pair (i.e., a set of sample question combinations), and the "0" marked at the end means that the two questions are not similar.

[0100] Figure 5 (b) in the figure is an example of Chinese sentence pairs (i.e. sample question combinations) and corresponding similarity labels in the public data source ATEC. For example, "Who has pictures of Sword Art Online?" and "Who has pictures of all the swords in Sword Art Online?" belong to a sentence pair (i.e. a set of sample question combinations), and the "1" marked at the end indicates that the two questions are similar.

[0101] Figure 5 (c) in the figure is an example of Chinese sentence pairs (i.e. sample question combinations) and corresponding similarity labels in the public data source CCKS. For example, "Does WeChat consumption count?" and "How much money do I have to pay back?" belong to a sentence pair (i.e. a set of sample question combinations), and the "0" marked at the end means that the two questions are not similar.

[0102] Specifically, the server can select sample data from multiple different data sources according to preset rules or randomly, and form a training set based on the sample data selected from the multiple data sources. Then, the server can iteratively perform multi-task training on the basic language model based on the training set to fine-tune the model parameters of the basic language model to obtain a similarity prediction model. It can be understood that a set of sample data includes a sample question combination and a corresponding similarity label.

[0103] It should be noted that there may be one or more basic language models. When there are multiple basic language models, the training set may be used to iteratively perform multi-task training on each basic language model to obtain a similarity prediction model.

[0104] In the above embodiment, a combination of sample questions from multiple data sources is used as a training set to iteratively perform multi-task training on the basic language model, which can achieve domain adaptation and improve the generalization and migration capabilities of the similarity prediction model, thereby improving the accuracy of problem processing.

[0105] In one embodiment, a training set is used to iteratively perform multi-task training on a basic language model to obtain a similarity prediction model, including: for each basic language model of at least two basic language models, a training set is used to iteratively perform multi-task training on the basic language model to obtain a prediction model corresponding to the basic language model; different basic language models have different processing granularities; and a similarity prediction model is obtained according to the prediction models corresponding to each basic language model.

[0106] The processing granularity of the basic language model refers to the processing granularity of the basic language model on the text. It can be understood that the processing granularity can include at least one of character granularity, word granularity and sentence granularity. Character granularity means that the basic language model processes a single character. Word granularity means that the basic language model processes each word after word segmentation. Sentence granularity means that the basic language model processes sentences as units.

[0107] Specifically, the server may use a training set for each basic language model to iteratively perform multi-task training on the basic language model to obtain a prediction model corresponding to the basic language model. It can be understood that the prediction model corresponding to the basic language model is the prediction model obtained by training the basic language model. The server may combine the prediction models obtained by training the basic language models according to corresponding weights to obtain a similarity prediction model.

[0108] In one embodiment, for each of the at least two basic language models, using a training set, iteratively performing multi-task training on the basic language model to obtain a prediction model corresponding to the basic language model includes: using the training set, iteratively performing multi-task training on a first basic language model to obtain a prediction model corresponding to the first basic language model; the first basic language model is a basic language model with characters as processing granularity; using the training set, iteratively performing multi-task training on a second basic language model to obtain a prediction model corresponding to the second basic language model; the second basic language model is a basic language model with words as processing granularity.

[0109] In one embodiment, the first basic language model may be a BERT model. It is understood that the BERT model is a basic language model with characters as the processing granularity.

[0110] In one embodiment, the second basic language model may be a WWM-BERT model. It is understandable that, due to the BERT model, Chinese is segmented based on characters, without considering Chinese word segmentation in traditional NLP (Natural Language Processing), while the WWM-BERT model released by Harbin Institute of Technology is a Chinese BERT pre-trained model based on Whole Word Masking, that is, it is a basic language model based on words as the processing granularity.

[0111] It can be understood that after the multi-task training of the first basic language model is iteratively completed, a prediction model corresponding to the first basic language model can be obtained, and after the multi-task training of the second basic language model is iteratively completed, a prediction model corresponding to the second basic language model can be obtained. The server can combine the prediction model corresponding to the first basic language model and the prediction model corresponding to the second basic language model to obtain a final similarity prediction model.

[0112] In one embodiment, for the final model lightweighting of this solution, in order to further speed up the reasoning speed of BERT, the mixed precision method can be used as a reference. The parameters of the general model are saved and calculated using FP32 (single precision), so in order to increase the reasoning speed, a single FP32 register can be used to cache two FP16 (half-precision) variables and perform parallel calculations. In addition, the FP32 variables in the model can be replaced with FP16 through the multiplication and addition operations of mixed precision matrices, which can effectively compress the model size and improve the reasoning speed of the model, while having little effect on the model itself.

[0113] In the above embodiment, the basic language models of different processing granularities are iteratively trained with multiple tasks, and a similarity prediction model is obtained based on the prediction models obtained by training the basic language models. The accuracy of the similarity prediction model can be improved through the complementarity between the basic language models of different processing granularities.

[0114] In one embodiment, a training set is used to iteratively perform multi-task training on a basic language model to obtain a prediction model corresponding to the basic language model, including: obtaining preset hyperparameters corresponding to the basic language model; the hyperparameters include an initial learning rate; using the initial learning rate as the current learning rate, and using the training set to perform a current round of multi-task training on the basic language model according to the current learning rate; reducing the learning rate after the current round of training, and using the reduced learning rate as the current learning rate, using the next round as the current round, returning to perform the current round of multi-task training on the basic language model until an iteration stop condition is reached to obtain a prediction model corresponding to the basic language model.

[0115] Specifically, the server may select a set of preset hyperparameters. The preset hyperparameters include an initial learning rate. The server may use a standard fine-tuning method, starting from the initial learning rate, using it as the current learning rate, using the training set, performing the current round of multi-task training on the basic language model, reducing the learning rate after the current round of training, and using the reduced learning rate as the current learning rate, using the next round as the current round, returning to the current round of multi-task training on the basic language model, until the iteration stop condition is reached, and obtaining the prediction model corresponding to the basic language model.

[0116] In one embodiment, the initial learning rate is set from 1e-5 to 5e-5.

[0117] In one embodiment, the iteration stopping condition may be that the trained prediction model converges, or reaches a preset number of iterations, or the learning rate drops to 0 after one iteration.

[0118] In one embodiment, according to the current learning rate, using the training set, the current round of multi-task training for the basic language model includes: inputting the training sets corresponding to multiple data sources into the current round of the basic language model for multi-task training to obtain the output vector corresponding to the sample problem combination in the training set; inputting the output vector corresponding to the sample problem combination of the same data source into the fully connected layer corresponding to the data source for full connection processing; determining the sub-loss function corresponding to each data source according to the result of the full connection processing; determining the final loss function according to each sub-loss function; and adjusting the model parameters of the basic language model according to the loss function and the current learning rate.

[0119] It should be noted that the basic language model of the current round is the basic language model trained in the current round, not the initial basic language model before training.

[0120] Specifically, in each round of training, the server can input the training set taken from multiple data sources into the basic language model of the current round for multi-task training. Since the training set includes multiple sample question combinations, in each round of training, the basic language model trained in the current round can output a vector for each sample question combination, that is, the output vector corresponding to each sample question combination in the training set. The server can input the output vector corresponding to the sample question combination of the same data source into the fully connected layer corresponding to the data source for full connection processing. The server can determine the sub-loss function corresponding to each data source based on the result of the full connection processing. The server can perform weighted averaging of each sub-loss function according to the corresponding weights to obtain the final loss function. The server can adjust the model parameters of the basic language model according to the loss function and the current learning rate.

[0121] In one embodiment, each sub-loss function can be obtained through logistic regression and cross entropy, that is, (ylog(p)+(1-y)log(1-p)).

[0122] In one embodiment, the loss function is obtained by weighted summing up the sub-loss functions according to the corresponding weights. The method also includes: determining the data source corresponding to the sub-loss function in the loss function, determining the fully connected layer corresponding to the data source for each sub-loss function, and updating the parameters of the fully connected layer according to the sub-loss function. That is, when calculating the sub-loss function, for a sample problem combination from a certain data source, in addition to updating the shared BERT model parameters according to the sub-loss function during back propagation, the parameters of the fully connected layer corresponding to the data source can also be updated. This further improves the accuracy of model adjustment.

[0123] Figure 6 Schematic diagram of the principle of multi-task training of a basic language model in one embodiment. Figure 6 The following is an example of a basic language model, the BERT model. Figure 6 , sample data is selected from different data sources (data sources 1 to 4, PAIRBANK, LCQMC, ATEC and CCKS respectively) to obtain a training set. Then, the training set is input into the basic language model, and the basic language model is trained on multiple tasks. The basic language model can be a BERT model or a WWM-BERT model. The basic language model outputs the corresponding output vector for each sample question combination. The output vector can be a 768-dimensional vector. It can be understood that Figure 6For simplicity, only the output vector for one sample problem combination is shown. The output vector corresponding to the sample problem combination of the same data source is input into the fully connected layer corresponding to the data source for full connection processing. For example, the output vector corresponding to the sample problem combination selected from data source 1 is uniformly input into the fully connected layer of data source 1 for full connection processing. Then, based on the result of the full connection processing, the prediction result (i.e., logit) is determined, and based on the difference between the prediction result and the similarity label, the sub-loss function corresponding to each data source is determined, that is, Figure 6 The sub-loss functions 1 to 4 shown in FIG. It can be understood that the final loss function can be determined according to each sub-loss function; and the model parameters of the basic language model can be adjusted according to the loss function and the current learning rate.

[0124] In one embodiment, the training sets corresponding to multiple data sources are input into the basic language model of the current round for multi-task training, and the step of obtaining the output vector corresponding to the sample question combination in the training set includes: inputting the sample question combination in the training set into the basic language model of the current round, in the basic language model, for each sample question in the sample question combination, encoding each character or each word segment in the sample question to obtain a word vector, performing semantic decoding based on the word vector, and obtaining the output vector corresponding to the sample question combination. It can be understood that the output vector is the semantic vector corresponding to the sample question combination. The computer device can connect the output vector to a fully connected layer and a logistic regression layer. That is, the output vector corresponding to the sample question combination of the same data source can be input into the fully connected layer corresponding to the data source and subsequent processing.

[0125] Figure 7 FIG. 4 is a schematic diagram of obtaining an output vector in an embodiment. Figure 7 The BERT model is used as an example. Figure 7 The two questions in the sample question set are question 1 and question 2 (i.e., two sentences). The characters in question 1 and question 2 are input into the BERT model for encoding, and the word vectors E1~E1 are obtained respectively. N , and word vector E ’ 1~E ’ M Then decode these word vectors to get the corresponding vectors T1~T N , and T ’ 1~T ’ M According to the decoded vector, the semantic vector C is obtained, which is the output vector of the BERT model. Further, the output vector C can be connected to a classification layer (ClassLabel) for conversion, and then connected to softmax for classification processing. It can be understood that Figure 6Accessing the fully connected layer and calculating the loss function in belong to the steps of classification processing.

[0126] In the above embodiment, sample data from multiple data sources are used as training sets to calculate sub-loss functions of multiple tasks, and then the loss function is determined comprehensively to adjust the model parameters, thereby improving the generalization and migration capabilities of the model, thereby improving the accuracy of problem processing.

[0127] In one embodiment, a training set is used to iteratively perform multi-task training on a basic language model to obtain a similarity prediction model, including: using a training set to iteratively perform multi-task training on a basic language model to obtain a prediction model corresponding to each round; selecting a target prediction model from the prediction model; the prediction accuracy of the target prediction model is higher than that of a non-target prediction model in the prediction model; predicting an unlabeled sample question combination according to the target prediction model, and determining a sample label of the unlabeled sample question combination according to the prediction result; using the unlabeled sample question combination and the sample label as sample data, iteratively training the initial similarity prediction model to obtain a similarity prediction model.

[0128] The prediction accuracy refers to the accuracy of similarity prediction for question combinations. Unlabeled sample question combinations refer to sample question combinations without sample labels. The initial similarity prediction model is a model framework for similarity matching judgment of different texts in a combination.

[0129] The initial similarity prediction model is a lightweight model that is smaller than the basic language model.

[0130] In one embodiment, the initial similarity prediction model may be an ESIM model framework. The ESIM model (Enhanced Sequential Inference Model) is a text matching model.

[0131] It can be understood that in the process of iteratively performing multi-task training on the basic language model using the training set, a corresponding prediction model will be obtained in each round (that is, there will be a trained basic language model in each round).

[0132] The server may select a target prediction model from the prediction models; the prediction accuracy of the target prediction model is higher than that of the non-target prediction model in the prediction model.

[0133] In one embodiment, selecting a target prediction model from the prediction models includes: for each round of corresponding prediction models, verifying the prediction accuracy of the prediction model according to a preset verification set; and screening a preset number of prediction models from the prediction models obtained after each round of multi-task training in order of prediction accuracy from high to low to obtain a target prediction model.

[0134] Specifically, the server can input the verification sample problem combination in the preset verification set into the prediction model, output the prediction result, and determine the prediction accuracy according to the difference between the prediction result and the sample label corresponding to the verification sample problem combination. Then, in descending order of prediction accuracy, the prediction models obtained after each round of multi-task training are screened by a preset number of prediction models to obtain the target prediction model.

[0135] Furthermore, the server can predict the unlabeled sample question combination according to the target prediction model, obtain the prediction result (i.e., logits), and determine the sample label of the unlabeled sample question combination according to the prediction result. Then, the server can substitute the unlabeled sample question combination and the corresponding sample label as sample data into the initial similarity prediction model, perform iterative training, and finally obtain the similarity prediction model.

[0136] It can be understood that in the process of iteratively training the unlabeled sample question combination and the corresponding sample label as sample data and substituting it into the initial similarity prediction model, the model obtained in each round of training can output the predicted similarity corresponding to the sample question combination, and determine the loss function based on the difference between the predicted similarity and the sample label. The similarity prediction model is trained by finding the minimum value of the loss function.

[0137] In one embodiment, the initial similarity prediction model may be trained using a training step size of batch size=512, a learning rate of 0.001, and 1000 steps per epoch.

[0138] In one embodiment, the formula of the loss function of the similarity prediction model is as follows:

[0139]

[0140] in, is the similarity predicted by the similarity prediction model of the current round for the i-th sample problem combination; Yi is the similarity predicted by the target prediction model for the i-th sample problem combination; n is the number of sample problem combinations. MSE (mean-square error) is the mean square error. That is, the loss function can be determined based on the mean square error between the similarity predicted by the target prediction model and the similarity predicted by the similarity prediction model.

[0141] It should be noted that the initial similarity prediction model is iteratively trained using the sample question combination and the sample label as sample data, and the final model can be used to judge the similarity between two questions in the question combination, and thus, a similarity prediction model can be finally obtained.

[0142] In one embodiment, there are multiple target prediction models; according to the target prediction models, predicting the similarity corresponding to the unlabeled sample question combination includes: obtaining the unlabeled sample question combination; inputting the unlabeled sample question combination into each target prediction model respectively, and outputting each prediction result corresponding to the unlabeled sample question combination; averaging each prediction result corresponding to the same unlabeled sample question combination, and using the average value as the sample label of the unlabeled sample question combination.

[0143] For example, from the prediction models obtained in the process of iterative multi-task training of the basic language model, three prediction models with good prediction effects are selected as target prediction models. Then, these three target prediction models can be used to make predictions on the unlabeled sample question combination. For the unlabeled sample question combination 1, each target prediction model will output the prediction results for the unlabeled sample question combination 1, and then the prediction results they output (i.e., the three prediction results) are averaged, and the obtained score is used as the sample label of the unlabeled sample question combination 1.

[0144] It is understandable that the prediction results can reflect the similarity between the questions in the sample question combination, but it is not a similarity with a value in the interval [0,1]. Therefore, using it as a sample label reduces the limitation and is no longer limited to manually labeled labels such as 0 and 1, making the trained similarity prediction model more accurate.

[0145] In addition, in the embodiment of the present application, the prediction result based on the target prediction model is used as sample data to train the similarity prediction model, which is equivalent to using the knowledge distillation method, so that the final similarity prediction model inherits the accuracy of the basic language model due to the training using a large number of data sets, and is more lightweight than the network structure and parameter amount of the basic language model, thereby saving system resource consumption and improving performance. It is equivalent to using so there is a problem of large system resource consumption and high performance requirements. In fact, the similarity prediction model trained by the knowledge distillation method in this embodiment is verified by the verification set, and F1_score = 81.0%, which is a very high prediction accuracy. Moreover, the similarity prediction model is lightweight, which can reduce system resource consumption, reduce performance requirements, and improve response efficiency while ensuring prediction accuracy.

[0146] Reference Figures 8 to 9 , the experimental results are now used to prove the effect that the method in the embodiment of the present application can produce.

[0147] On the test set of our pairbank (a data source), we tested and verified five models, namely BERT_ENSEMBLE (fine-tuned on multiple datasets based on the BERT-base model and the WWM-BERT model using multi-task learning) (its corresponding precision-recall curve is shown by 802, and the corresponding F1 score curve is shown by 902), BERT_SINGLE_MODEL (fine-tuned on the pairbank dataset based on the BERT model) (its corresponding precision-recall curve is shown by 804, and the corresponding F1 score curve is shown by 904), ESIM_DISTILLED (i.e., the ESIM model based on knowledge distillation training) (its corresponding precision-recall curve is shown by 806, and the corresponding F1 score curve is shown by 906), ESIM_MULTITASK (ESIM model + multi-task learning) (its corresponding precision-recall curve is shown by 808, and the corresponding F1 score curve is shown by 908), and the traditional ESIM model (its corresponding precision-recall curve is shown by 810, and the corresponding F1 score curve is shown by 910). It can be understood that BERT_ENSEMBLE is a model trained based on at least two basic language models using training sets from multiple data sources. ESIM_DISTILLED is an ESIM model trained using knowledge distillation. The remaining three models are reference groups that do not use the method in the embodiments of the present application.

[0148] from Figure 8 The PR (Precision Recall) precision recall rate indicator, and Fig. 9 From the F1 score (a weighted average of the model's precision and recall), we can see that BERT_ENSEMBLE has the best PR and FI values. Although ESIM_DISTILLED is slightly worse than BERT_ENSEMBLE, it is much better than the traditional ESIM model and ESIM_MULTITASK model. Therefore, ESIM_DISTILLED also has a high evaluation in terms of PR and F1 score indicators.

[0149] Fig.10 It is a performance comparison analysis diagram in one embodiment. Fig.10 The curve shown in A is the latency curve corresponding to the ESIM model obtained based on knowledge distillation training. The BERT models with 2, 4, and 12 network structures are used as control groups, and the corresponding latency curves are B to D respectively. Fig.10It can be seen that the 12-layer BERT model represented by the latency curve D has a latency of more than 50ms when the concurrency is 1. The ESIM model obtained by knowledge distillation training represented by curve A has the best latency performance when the concurrency is <= 4. Therefore, the ESIM model has a shorter latency.

[0150] comprehensive Figures 8 to 10 It can be seen that the similarity prediction model obtained by fine-tuning the BERT-base model and the WWM-BERT model on multiple data sets using multi-task learning has very high prediction accuracy and recall. The ESIM model obtained based on knowledge distillation training has high prediction accuracy and recall, while having low performance requirements, low system resource consumption, fast processing speed, and short latency.

[0151] In addition, compared with the traditional ESIM model trained directly with data sets, the ESIM model trained based on knowledge distillation inherits the generalization and migration capabilities of the basic language model, so it has higher generalization and migration capabilities. Under a certain accuracy rate, it can recall more questions and cover more statements. Moreover, for questions in new fields, there is no need to train the model separately, and migration can obtain better question sentence understanding capabilities.

[0152] It should be understood that, although the steps in each flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in each flowchart may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0153] In one embodiment, Fig.11 As shown, a problem processing device 1100 is provided. The device can adopt a software module or a hardware module, or a combination of the two to become a part of a computer device. The device specifically includes: a query module 1102, a similarity prediction module 1104 and a matching module 1106; wherein:

[0154] The query module 1102 is used to obtain a question query statement; and search for candidate questions matching the question query statement from a preset question library;

[0155] The similarity prediction module 1104 is used to form a question combination with each candidate question, input the combination into a similarity prediction model, and output the similarity between the candidate question and the question query statement in the question combination; the similarity prediction model is obtained by adjusting the model parameters of the basic language model through multi-task iterative training using sample question combinations and corresponding similarity labels from multiple data sources as training sets;

[0156] The matching module 1106 is used to select the candidate question with the highest similarity as the target question that matches the question query statement.

[0157] In one embodiment, the apparatus further comprises:

[0158] The answer search module 1108 is used to determine a preset answer library; the answers in the preset answer library have a corresponding relationship with the questions in the preset question library; according to the corresponding relationship, the answer corresponding to the target question is searched from the preset answer library.

[0159] like Fig.12 As shown, in one embodiment, the device includes: a model training module 1103 and an answer search module 1108; wherein:

[0160] The model training module 1103 is used to obtain a training set; the training set includes sample question combinations from multiple data sources and similarity labels corresponding to the sample question combinations; the training set is used to iteratively perform multi-task training on the basic language model to obtain a similarity prediction model.

[0161] In one embodiment, the model training module 1103 is also used to iteratively perform multi-task training on each basic language model of at least two basic language models using a training set to obtain a prediction model corresponding to the basic language model; different basic language models have different processing granularities; and a similarity prediction model is obtained based on the prediction models corresponding to each basic language model.

[0162] In one embodiment, the model training module 1103 is also used to use the training set to iteratively perform multi-task training on the first basic language model to obtain a prediction model corresponding to the first basic language model; the first basic language model is a basic language model with characters as the processing granularity; the training set is used to iteratively perform multi-task training on the second basic language model to obtain a prediction model corresponding to the second basic language model; the second basic language model is a basic language model with words as the processing granularity.

[0163] In one embodiment, the model training module 1103 is also used to obtain preset hyperparameters corresponding to the basic language model; the preset hyperparameters include an initial learning rate; the initial learning rate is used as the current learning rate, and according to the current learning rate, the training set is used to perform the current round of multi-task training on the basic language model; after the current round of training is completed, the learning rate is reduced, and the reduced learning rate is used as the current learning rate, the next round is used as the current round, and the current round of multi-task training of the basic language model is returned until the iteration stop condition is reached to obtain a prediction model corresponding to the basic language model.

[0164] In one embodiment, the model training module 1103 is also used to input the training sets corresponding to multiple data sources into the basic language model of the current round for multi-task training to obtain the output vector corresponding to the sample problem combination in the training set; input the output vector corresponding to the sample problem combination of the same data source into the fully connected layer corresponding to the data source for full connection processing; determine the sub-loss function corresponding to each data source based on the result of the full connection processing; determine the final loss function based on each sub-loss function; and adjust the model parameters of the basic language model based on the loss function and the current learning rate.

[0165] In one embodiment, the loss function is obtained by weighted summing up the sub-loss functions according to the corresponding weights; the model training module 1103 is also used to determine the data source corresponding to the sub-loss function in the loss function, and for each data source corresponding to the sub-loss function, determine the fully connected layer corresponding to the data source, and update the parameters of the fully connected layer according to the sub-loss function.

[0166] In one embodiment, the model training module 1103 is also used to use the training set to iteratively perform multi-task training on the basic language model to obtain a corresponding prediction model for each round; select a target prediction model from the prediction model; the prediction accuracy of the target prediction model is higher than that of the non-target prediction model in the prediction model; predict the unlabeled sample question combination according to the target prediction model, and determine the sample label of the unlabeled sample question combination according to the prediction result; use the unlabeled sample question combination and the sample label as sample data to iteratively train the initial similarity prediction model to obtain a similarity prediction model.

[0167] In one embodiment, the model training module 1103 is also used to verify the prediction accuracy of the prediction model corresponding to each round according to a preset verification set; and screen a preset number of prediction models from the prediction models obtained after each round of multi-task training in order of prediction accuracy from high to low to obtain a target prediction model.

[0168] In one embodiment, there are multiple target prediction models; the model training module 1103 is also used to obtain unlabeled sample question combinations; the unlabeled sample question combinations are input into each target prediction model respectively, and the prediction results corresponding to the unlabeled sample question combinations are output; the prediction results corresponding to the same unlabeled sample question combination are averaged, and the average value is used as the sample label of the unlabeled sample question combination.

[0169] For the specific definition of the problem handling device, please refer to the definition of the problem handling method above, which will not be repeated here. Each module in the above-mentioned problem handling device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0170] In one embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Fig.13 As shown. The computer device includes a processor, a memory and a communication interface connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a problem handling method is implemented.

[0171] Those skilled in the art will understand that Fig.13 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0172] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0173] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0174] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (Ran Document Object Model Access Memory, RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (Static Ran Document Object Model Access Memory, SRAM) or dynamic random access memory (Dynamic Ran Document Object Model Access Memory, DRAM), etc.

[0175] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0176] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A problem solving method, characterized in that: The method comprises: Get the query statement; Searching for candidate questions matching the question query statement from a preset question library; The question query statement and each of the candidate questions are respectively combined into a question combination, which is input into a similarity prediction model, and the similarity between the candidate questions in the question combination and the question query statement is output; the similarity prediction model is obtained by adjusting the model parameters of the basic language model through multi-task iterative training using sample question combinations and corresponding similarity labels from multiple data sources as training sets; Selecting the candidate question with the highest similarity as the target question matching the question query statement; The training steps of the similarity prediction model include: The training set is used to iteratively perform multi-task training on the basic language model to obtain a similarity prediction model, including: In the current round of multi-task training of the basic language model, the training sets corresponding to the multiple data sources are input into the current round of the basic language model for multi-task training to obtain output vectors corresponding to the sample question combinations in the training set; The output vector corresponding to the sample problem combination of the same data source is input into the fully connected layer corresponding to the data source for full connection processing; Determine the sub-loss function corresponding to each of the data sources according to the result of the full connection processing; Determine a final loss function according to each of the sub-loss functions; According to the loss function and the current learning rate, the model parameters of the basic language model are adjusted.

2. The method according to claim 1, characterized in that The method further comprises: Determine a preset answer library; the answers in the preset answer library correspond to the questions in the preset question library; From the preset answer library, find the answer corresponding to the target question.

3. The method according to claim 1, characterized in that The method of using the training set to iteratively perform multi-task training on the basic language model to obtain a similarity prediction model includes: For each of the at least two basic language models, the training set is used to iteratively perform multi-task training on the basic language model to obtain a prediction model corresponding to the basic language model; different basic language models have different processing granularities; A similarity prediction model is obtained according to the prediction models corresponding to the basic language models.

4. The method according to claim 3, characterized in that The step of iteratively performing multi-task training on each of the at least two basic language models using the training set to obtain a prediction model corresponding to the basic language model comprises: Using the training set, iteratively performing multi-task training on a first basic language model to obtain a prediction model corresponding to the first basic language model; the first basic language model is a basic language model with characters as processing granularity; The training set is used to iteratively perform multi-task training on the second basic language model to obtain a prediction model corresponding to the second basic language model; the second basic language model is a basic language model with words as the processing granularity.

5. The method according to claim 3, characterized in that: The adopting the training set to iteratively perform multi-task training on the basic language model to obtain a prediction model corresponding to the basic language model comprises: Obtaining preset hyperparameters corresponding to the basic language model; the preset hyperparameters include an initial learning rate; Using the initial learning rate as the current learning rate, and performing a current round of multi-task training on the basic language model using the training set according to the current learning rate; After the current round of training is completed, the learning rate is reduced, and the reduced learning rate is used as the current learning rate, the next round is used as the current round, and the multi-task training of the current round on the basic language model is returned until the iteration stop condition is reached to obtain a prediction model corresponding to the basic language model.

6. The method according to claim 1, characterized in that The loss function is obtained by weighted summing up the sub-loss functions according to their corresponding weights; The method further comprises: Determine a data source corresponding to a sub-loss function in the loss function; For each data source corresponding to the sub-loss function, a fully connected layer corresponding to the data source is determined, and the parameters of the fully connected layer are updated according to the sub-loss function.

7. The method according to claim 1, characterized in that The method of using the training set to iteratively perform multi-task training on the basic language model to obtain a similarity prediction model includes: Using the training set, iteratively perform multi-task training on the basic language model to obtain a prediction model corresponding to each round; Selecting a target prediction model from the prediction models; the prediction accuracy of the target prediction model is higher than that of a non-target prediction model in the prediction model; Predicting the unlabeled sample question combination according to the target prediction model, and determining the sample label of the unlabeled sample question combination according to the prediction result; The unlabeled sample question combination and the sample label are used as sample data, and the initial similarity prediction model is iteratively trained to obtain a similarity prediction model.

8. The method according to claim 7, characterized in that The selecting a target prediction model from the prediction models comprises: For each round of corresponding prediction models, verify the prediction accuracy of the prediction model according to a preset verification set; In the order of prediction accuracy from high to low, a preset number of prediction models are screened from the prediction models obtained after each round of multi-task training to obtain the target prediction model.

9. The method according to claim 7, characterized in that: There are multiple target prediction models; predicting the unlabeled sample question combination according to the target prediction model, and determining the sample label of the unlabeled sample question combination according to the prediction result includes: Obtain unlabeled sample question sets; Inputting the unlabeled sample question combinations into the target prediction models respectively, and outputting the prediction results corresponding to the unlabeled sample question combinations; The prediction results corresponding to the same unlabeled sample question combination are averaged, and the average value is used as the sample label of the unlabeled sample question combination.

10. A problem handling device, characterized in that: The device comprises: A query module is used to obtain a question query statement; and search for candidate questions matching the question query statement from a preset question library; A similarity prediction module is used to form a question combination with the question query statement and each of the candidate questions, input the combination into a similarity prediction model, and output the similarity between the candidate questions in the question combination and the question query statement; the similarity prediction model is obtained by adjusting the model parameters of the basic language model through multi-task iterative training using sample question combinations and corresponding similarity labels from multiple data sources as training sets; A matching module is used to select the candidate question with the highest similarity as the target question matching the question query statement; The model training module is used to use the training set to iteratively perform multi-task training on the basic language model to obtain a similarity prediction model, including: in the current round of multi-task training of the basic language model, the training sets corresponding to multiple data sources are input into the basic language model of the current round for multi-task training to obtain the output vector corresponding to the sample problem combination in the training set; the output vector corresponding to the sample problem combination of the same data source is input into the fully connected layer corresponding to the data source to perform full connection processing; according to the result of the full connection processing, the sub-loss function corresponding to each of the data sources is determined; according to each of the sub-loss functions, the final loss function is determined; according to the loss function and the current learning rate, the model parameters of the basic language model are adjusted.

11. The device according to claim 10, characterized in that The device also includes: The answer search module is used to determine a preset answer library; the answers in the preset answer library have a corresponding relationship with the questions in the preset question library; according to the corresponding relationship, the answer corresponding to the target question is searched from the preset answer library.

12. The device according to claim 10, characterized in that The model training module is also used to iteratively perform multi-task training on each of the at least two basic language models using the training set to obtain a prediction model corresponding to the basic language model; different basic language models have different processing granularities; and a similarity prediction model is obtained based on the prediction models corresponding to each basic language model.

13. The device according to claim 12, characterized in that The model training module is also used to use the training set to iteratively perform multi-task training on the first basic language model to obtain a prediction model corresponding to the first basic language model; the first basic language model is a basic language model with characters as the processing granularity; the training set is used to iteratively perform multi-task training on the second basic language model to obtain a prediction model corresponding to the second basic language model; the second basic language model is a basic language model with words as the processing granularity.

14. The device according to claim 12, characterized in that The model training module is also used to obtain preset hyperparameters corresponding to the basic language model; the preset hyperparameters include an initial learning rate; the initial learning rate is used as the current learning rate, and according to the current learning rate, the training set is used to perform a current round of multi-task training on the basic language model; after the current round of training is completed, the learning rate is reduced, and the reduced learning rate is used as the current learning rate, the next round is used as the current round, and the current round of multi-task training of the basic language model is returned to until the iteration stop condition is reached to obtain a prediction model corresponding to the basic language model.

15. The device according to claim 10, characterized in that The loss function is obtained by weighted summing up the sub-loss functions according to their corresponding weights; The model training module is also used to determine the data source corresponding to the sub-loss function in the loss function; for each data source corresponding to the sub-loss function, determine the fully connected layer corresponding to the data source, and update the parameters of the fully connected layer according to the sub-loss function.

16. The device according to claim 10, characterized in that The model training module is also used to use the training set to iteratively perform multi-task training on the basic language model to obtain a corresponding prediction model for each round; select a target prediction model from the prediction models; the prediction accuracy of the target prediction model is higher than that of the non-target prediction model in the prediction model; predict the unlabeled sample question combination according to the target prediction model, and determine the sample label of the unlabeled sample question combination according to the prediction result; use the unlabeled sample question combination and the sample label as sample data to iteratively train the initial similarity prediction model to obtain a similarity prediction model.

17. The device according to claim 16, characterized in that The model training module is also used to verify the prediction accuracy of the prediction model corresponding to each round according to a preset verification set; and screen a preset number of prediction models from the prediction models obtained after each round of multi-task training in order of prediction accuracy from high to low to obtain a target prediction model.

18. The device according to claim 16, characterized in that There are multiple target prediction models; the model training module is also used to obtain unlabeled sample question combinations; the unlabeled sample question combinations are input into each of the target prediction models, and the prediction results corresponding to the unlabeled sample question combinations are output; The prediction results corresponding to the same unlabeled sample question combination are averaged, and the average value is used as the sample label of the unlabeled sample question combination.

19. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images