Training method, classification method, terminal and storage medium

Abstract

The embodiment of the application relates to the technical field of digital medical treatment, and specifically provides a text classification model training method, a classification method, a terminal and a storage medium. The method comprises the following steps: inputting a training sample into a text coding network to obtain a first text vector corresponding to the training sample; inputting the first text vector into a context coding network to obtain a second text vector corresponding to the first text vector; inputting the second text vector into a first classification network and a second classification network respectively to obtain a first category probability distribution of the training sample classified by the first classification network and a second category probability distribution of the training sample classified by the second classification network, constructing a first loss function according to the first category probability distribution and constructing a second loss function according to the second category probability distribution, and determining a fusion loss function according to the first loss function and the second loss function; and iteratively updating a text classification model based on the training sample and the fusion loss function to obtain a target text classification model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital medical treatment, and in particular to a text classification model training method, a text classification method, a terminal and a storage medium. BACKGROUND

[0002] With the rapid development of computer science and technology, there are now many artificial intelligence means applied to the medical service industry, thereby allowing patients to enjoy safe, convenient and high-quality diagnosis and treatment services. Medical inquiry systems have emerged as the times require. Dialog interaction technology is used to solve patients' medical and health consultation needs, such as disease knowledge, medical process, symptom judgment, etc. The consultation efficiency is stronger than that of the hospital's manual reply, that is, the patient can be provided with medical and health information services without consuming the doctor's energy.

[0003] The medical inquiry system is a special question and answer system, and the patient interacts with it with a clear purpose, that is, to provide their own information to obtain relevant information about the disease, and the dialog process conforms to the traditional doctor-patient inquiry process. Therefore, the intent in the medical inquiry system has both commonality and domain characteristics compared with the general question and answer system. Effective identification of the patient's specific intent can reduce the number of redundant rounds of question and answer and quickly and accurately assist the inquiry system in making diagnosis and decision.

[0004] The intent category recognition method used in the existing medical inquiry system usually uses text classification technology to recognize single-round dialog, and regards a single intent label as a category. However, the number of intent categories has a great influence on the recognition accuracy, the more the number is, the more difficult the recognition is, and the intent category expansion is not easy. Moreover, for the medical inquiry system, the general multi-classification method cannot effectively mine the internal relationship between intents, which further affects the effect of intent recognition and is not conducive to the overall optimization of the system. SUMMARY

[0005] The main purpose of the embodiments of the present application is to provide a text classification model training method, a text classification method, a terminal and a storage medium, which aims to improve the accuracy of intent text classification in medical inquiry, and thereby improve the diagnosis and decision-making effect of the medical inquiry system and provide better services for patients.

[0006] In a first aspect, the embodiments of the present application provide a text classification model training method, comprising:

[0007] Obtaining a training sample, wherein the training sample comprises text data and a category label.

[0008] Inputting the training sample into a text encoding network of a text classification model to be trained to obtain a first text vector corresponding to the training sample.

[0009] inputting the first text vector into a context encoding network of the text classification model to obtain a second text vector corresponding to the first text vector, wherein the context encoding network is a long short-term memory network, and a vector representation accuracy of the target object in the training sample by the second text vector is greater than that by the first text vector.

[0010] inputting the second text vector into a first classification network of the text classification model to obtain a first category probability distribution of the training sample classified by the first classification network, and inputting the second text vector into a second classification network of the text classification model to obtain a second category probability distribution of the training sample classified by the second classification network.

[0011] constructing a first loss function according to the first category probability distribution and a second loss function according to the second category probability distribution, and determining a fusion loss function according to the first loss function and the second loss function.

[0012] iteratively updating the text classification model based on the training sample and the fusion loss function to obtain a target text classification model.

[0013] In a second aspect, the embodiments of the present application further provide a text classification method of a text classification model, comprising:

[0014] inputting text data into the text classification model to obtain a classification result of the first classification network and a classification result of the second classification network, wherein the text classification model is obtained according to any one of the training steps of the text classification model provided in the present application.

[0015] In a third aspect, the embodiments of the present application further provide a terminal device, which comprises a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the training method steps and the text classification method steps of any one of the text classification models provided in the present application are realized.

[0016] In a fourth aspect, the embodiments of the present application further provide a storage medium for computer readable storage, characterized in that the storage medium stores one or more programs, and the one or more programs are executable by one or more processors to realize the training method steps and the text classification method steps of any one of the text classification models provided in the present application.

[0017] The embodiment of the application provides a text classification model training method, a classification method, a terminal and a storage medium, wherein the text classification training method is characterized in that: the training sample is input into a text encoding network and a context encoding network of a text classification model to be trained to perform text vector representation, and then a text vector corresponding to the training sample is obtained; the text vector is input into a first classification network and a second classification network of the text classification model respectively, a first category probability distribution of the training sample classified by the first classification network and a second category probability distribution of the training sample classified by the second classification network are obtained; then a first loss function is constructed according to the first category probability distribution, a second loss function is constructed according to the second category probability distribution, and a fusion loss function is determined according to the first loss function and the second loss function; the text classification model is iteratively updated based on the training sample and the fusion loss function, and a target text classification model is obtained. Therefore, the internal correlation of the text classification result can be mined by using the multi-classification network, the accuracy of patient question and answer text recognition in the medical inquiry system is improved, the redundant rounds of patient questions and answers in the medical inquiry system are effectively reduced, the efficiency of the questions and answers in the medical inquiry system is improved, the diagnosis and decision of the inquiry system are quickly and accurately assisted, and better inquiry services are provided for patients. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0019] Figure 1 A flowchart of a text classification model training method provided by the embodiment of the application is shown in the figure.

[0020] Figure 2 A structure schematic block diagram of a text classification model corresponding to the embodiment of the application is shown in the figure.

[0021] Figure 3 A specific implementation structure diagram of a text classification model provided by the embodiment of the application is shown in the figure.

[0022] Figure 4 A specific implementation structure diagram of a text classification model provided by the embodiment of the application is shown in the figure. Figure 1 A specific implementation structure diagram of a text classification model provided by the embodiment of the application is shown in the figure.

[0023] Figure 5 A structure schematic block diagram of a terminal device provided by the embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0024] With reference to the drawings, the technical solutions in the embodiments of the present application will be clearly and completely described below, obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work belong to the scope of protection of the present application.

[0025] The flowcharts shown in the drawings are only illustrative, not necessarily including all the contents and operations / steps, and not necessarily executed in the described order. For example, some operations / steps can also be decomposed, combined or partially merged, so the actual execution order can be changed according to the actual situation.

[0026] It should be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and the appended claims of the present application, unless otherwise clear from the context, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0027] The embodiments of the present application provide a text classification model training method, a classification method, a terminal and a storage medium.

[0028] The text classification training method performs text vector representation by inputting the training sample into the text encoding network and the context encoding network of the text classification model to be trained, and then obtains the text vector corresponding to the training sample; the text vector is input into the first classification network and the second classification network of the text classification model respectively, to obtain the first category probability distribution classified by the first classification network and the second category probability distribution classified by the second classification network; then the first loss function is constructed according to the first category probability distribution and the second loss function is constructed according to the second category probability distribution, and the fusion loss function is determined according to the first loss function and the second loss function; the text classification model is iteratively updated based on the training sample and the fusion loss function, to obtain a target text classification model. Thus, the internal correlation of the text classification result can be mined by using the multi-classification network, the accuracy of the question and answer text recognition in the medical inquiry system is improved, the redundant rounds of the question and answer in the medical inquiry system are effectively reduced, the efficiency of the question and answer in the medical inquiry system is improved, and the diagnosis and decision of the inquiry system are quickly and accurately assisted, so that the diagnosis and treatment services for patients are better provided.

[0029] Some embodiments of the present application will be described in detail below with reference to the drawings. In the case of no conflict, the embodiments and features in the embodiments described below can be combined with each other.

[0030] Please refer to Figure 1 , Figure 1A flowchart of a training method of a text classification model is provided for the embodiments of the present application.

[0031] As shown in Figure 2 The training method of the text classification model includes steps S1 to S6. Figure 4 A structural schematic diagram corresponding to the text classification model is provided.

[0032] Step S1: Obtain training samples, which include text data and category labels.

[0033] Exemplarily, in a medical inquiry system, intent recognition is used to classify user input dialogues to determine the purpose type expressed by the user during the dialogue with the system, thereby assisting the system to feed back different processing logic and response rhetoric to support the final dialogue decision of the system. When the user input dialogues in the medical inquiry system are accurately classified, the user's needs can be accurately responded to, helping the user to solve problems. Otherwise, the user needs to constantly input information to obtain a satisfactory result, and with the increase in the number of times, the user will increase the questioning of the medical inquiry system and reduce the use of the medical inquiry system.

[0034] In order to identify the intent of the user input text in the medical inquiry system, the corresponding training data is obtained before the text model training, wherein the training data includes text data and category labels to ensure subsequent model training. The existing text intent recognition method of the question and answer system usually adopts text classification technology for single-round dialogue recognition, takes a single intent label as a category, and then performs text multi-classification recognition. With the increase in the number of text intent classification categories, the influence on the recognition effect precision also increases. The more the number of text intent classification categories, the more difficult the recognition is. Moreover, the general multi-classification method cannot effectively mine the internal relations between the text intent classification categories, further affecting the effect of text intent classification, and is not conducive to the overall optimization of the system.

[0035] Exemplarily, the medical inquiry system is a special question and answer system, and the patient interacts with it with a clear purpose, i.e., providing its own information to obtain relevant information about the disease, and the dialogue process conforms to the traditional doctor-patient inquiry process. Therefore, the intent in the inquiry system has both commonality and domain characteristics compared with the general question and answer system. Effectively identifying the specific intent of the patient can reduce the number of redundant rounds of question and answer, and quickly and accurately assist the inquiry system to make diagnosis decisions. Therefore, the category labels of the text data of the training samples can be split to obtain two sets of category labels.

[0036] For example, a single text intent classification category is split into two text intent classification sub-categories: a general text classification category A and a domain text classification category B. The general text classification category A refers to the basic dialogue behaviors of the user such as asking, stating, etc., and the domain text classification category B refers to the characteristic domain of the user dialogue such as symptoms, medical history, diagnosis and treatment history, drugs, etc. The combination of the two can form a complete dialogue text classification category, such as stating symptoms, asking medical history, etc.

[0037] Optionally, after obtaining the training data, the training data needs to be preprocessed. The preprocessing process is very important and may be directly related to the quality of the final result. The training samples can be cleaned to remove duplicate samples and incomplete samples and remove useless information in the text. For example, the training samples may contain English characters and irregular punctuation marks, which will bring noise to the data and increase the difficulty of classification. Therefore, regular expressions can be used to remove these characters and symbols, so that only Chinese characters, numbers and full-width punctuation marks are retained in the text.

[0038] Optionally, after obtaining the training data, the number of texts between each label in the training data may have a sample distribution imbalance problem. Up-sampling can be used to enhance the sample quantity of the corpus with less sample quantity, or down-sampling can be used to reduce the sample quantity of the corpus with more sample quantity.

[0039] Optionally, after obtaining the training data, the training data needs to be split to obtain a training set and a test set. The split ratio of the training data can be 8:2 or 9:1, or the training data can be split to obtain a training set, a test set and a validation set. The split ratio of the training data can be 8:1:1, and the specific data division form is not limited here and can be divided according to actual needs.

[0040] Step S2: inputting the training sample into a text encoding network of the text classification model to be trained to obtain a first text vector corresponding to the training sample.

[0041] For example, the language and characters used by humans for communication are usually not directly recognizable and processable by computers, so the training sample needs to be input into the text encoding network of the text classification model to be trained, and the training sample is numerically or vectorized, and then converted into a form recognizable by the computer. The essence of text representation is to model the text features. Good text representation can fully reflect the meaning expressed by the word and the semantic information contained in the text, greatly improving the learning effect of the subsequent algorithm.

[0042] Exemplarily, the design of the text encoding network can be divided into two categories: traditional text representation and distributed representation method. The traditional text representation method can be one-hot encoding or bag-of-words model, and the distributed representation method is mainly to construct a text encoding network according to the context representation of semantics, to seek the internal relationship between words, and the common deep learning language representation model is BERT (Bidirectional Encoder Representation from Transformers), TextCNN (Text Convolutional Neural Networks), and FastText.

[0043] For example, the text encoding network of the text classification model to be trained can utilize the BERT training representation model, input the training sample into the BERT model, process the training sample, fine-tune the trained deep learning model by using the input training sample, and take the model output of the last layer of the BERT model as the first text vector corresponding to the training sample.

[0044] In some embodiments, inputting the training sample into the text encoding network of the text classification model to be trained to obtain the first text vector corresponding to the training sample comprises: inputting the training sample into the text encoding layer of the text classification model to be trained to obtain the first vector expression and the second vector expression of the target object in the training sample; and performing pooling processing on the first vector expression and the second vector expression by using a pooling algorithm to obtain the first text vector corresponding to the training sample.

[0045] Exemplarily, the training sample is input into the text encoding network of the text classification model to be trained, and the training sample is converted into a form that can be recognized by a computer. When the training sample is processed by the text encoding network, it will pass through a multi-layer stacked language model coding structure to obtain the representation ability of the semantic relationship of the words in the training sample. When the training sample passes through the multi-layer stacked language model coding structure, the output of the last layer and any other layer in the model can be taken as the first vector expression and the second vector expression of the target object in the training sample, and the first vector expression and the second vector expression are processed by pooling to obtain the text vector result.

[0046] For example, the text encoding network adopts a BERT model, where BERT is a language model structure stacked with multiple layers of Transformer-based attention mechanism layers, and BERT can be 12 layers or 24 layers, without limitation. The first layer of the BERT model is taken as the first vector representation of the target object in the training sample, and the last layer of the BERT model is taken as the second vector representation of the target object in the training sample, and then the first vector representation and the second vector representation are subjected to mean pooling processing to obtain the text vector result, that is, the average value of the first vector representation and the second vector representation is taken as the text vector result.

[0047] For example, when the training text is "I have a headache, what department should I call", the text is processed by the text encoding network such as the 12-layer BERT model, and the text representation of "I" in the first layer of the BERT model is [1, 2, 4, 0, 0], and the text representation of "I" in the last layer of the BERT model is [3, 4, 6, 2, 8], that is, the first vector representation of "I" is [1, 2, 4, 0, 0], and the second vector representation is [3, 4, 6, 2, 8], and the text vector result after mean pooling is [(1+3) / 2, (2+4) / 2, (4+6) / 2, (0+2) / 2, (0+8) / 2], that is, the text vector result is [2, 3, 5, 1, 4].

[0048] Step S3, input the first text vector into the context encoding network of the text classification model to obtain a second text vector corresponding to the first text vector, wherein the context encoding network is a long short-term memory network, and the vector representation accuracy of the second text vector for the target object in the training sample is greater than that of the first text vector.

[0049] For example, the transformer is used in the process of text encoding using BERT, and the transformer is based on self-attention, that is, the position information is weakened in the process of text encoding, and only the position encoding is used to tell the model the position information of the input, but the position information of the text in the sequence labeling task is necessary, and even the direction information is necessary, so the context encoding network is needed to learn the dependency relationship on the text sequence.

[0050] For example, sentence A "I like this doctor because he does not charge randomly" and sentence B "I do not like this doctor because he charges randomly". The words in sentence A and sentence B are the same, but the order of the words is different, and the meaning expressed is very different, so the context encoding network is needed to learn the dependency relationship on the text sequence.

[0051] The representation of a sentence can be obtained by word vectors, which can be combined into the representation of a sentence on the basis of the representation of a word. An addition method is used, i.e. the representations of all words are added, or an average method is used. However, these methods have a big problem that the order of words in a sentence is not considered. The LSTM model can better capture the long-distance dependency. Because LSTM can learn to remember which information and forget which information through the training process.

[0052] Exemplarily, the context encoding network is a long short-term memory network, the core concept of which is a cell state and a "gate" structure. The cell state is equivalent to the path of information transmission, which enables information to be passed down in the sequence, and can be regarded as the "memory" of the network. In theory, the cell state can pass the relevant information in the sequence processing process all the way down. Therefore, even the information of an earlier time step can be carried into the cell of a later time step, which overcomes the influence of short-term memory. The addition and removal of information are realized through the "gate" structure, which learns to save or forget which information in the training process, and thus learns the dependency relationship on the text sequence, obtains the second text vector corresponding to the first text vector obtained by the text encoding network, and the vector representation accuracy of the target object in the training sample by the second text vector is greater than that by the first text vector.

[0053] Optionally, the position information of the text in the sequence labeling task is necessary, and even the direction information is necessary. The long short-term memory network can learn the sequence information from front to back in the sequence, but there is a problem in modeling the sentence by using the long short-term memory network: it cannot encode information from back to front. Therefore, a bidirectional long short-term memory network can be used to better capture the bidirectional semantic dependency.

[0054] In step S4, the second text vector is input into the first classification network of the text classification model to obtain the first category probability distribution of the training sample classified by the first classification network, and the second text vector is input into the second classification network of the text classification model to obtain the second category probability distribution of the training sample classified by the second classification network.

[0055] Exemplarily, the second text vector is sent to the first classification network and the second classification network respectively to obtain the first category probability distribution classified by the first classification network and the second category probability distribution classified by the second classification network respectively corresponding to the second text vector, wherein the classification category of the first classification network can be a general text classification category, and the classification category of the second classification network can be a domain text classification category.

[0056] For example, the first category of network classification represents the general text classification category, which refers to the user's basic dialogue behavior such as asking questions and making statements, while the second category of network classification represents the domain text classification category, which refers to the characteristic domain to which the user's dialogue belongs, such as symptoms, medical history, medical history, and drugs.

[0057] In some implementations, the first classification network includes a first fully connected layer and a first activation layer. Inputting a second text vector into the first classification network of the text classification model to obtain a first category probability distribution of the training samples classified by the first classification network includes: inputting the second text vector into the first fully connected layer, using the first fully connected layer to extract features from the input second text vector to obtain a first feature vector; inputting the first feature vector into the first activation layer, using the first activation layer to activate the first feature vector to obtain a first category probability distribution, wherein the first activation layer is a softmax classification layer.

[0058] For example, the training samples are quantified or vectorized into a form that the computer can recognize, that is, the training samples are converted into second text vectors. The second text vectors are input into the first fully connected layer to synthesize the features of the second text vectors to obtain the first feature vector. The output of the first fully connected layer is then passed to the first activation layer. The activation function of the first activation layer is used to activate the first feature vector to obtain the probability distribution of the first category.

[0059] For example, when the training text is transformed into the second text vector of the target text, denoted as xi, where i represents the position of the target text in the sentence, such as x3 representing the second text vector of the 3rd character in the training text, then the second text vector representation of the training text is x = x1, x2, ..., x... n By inputting x into the first fully connected layer, we can obtain the first feature vector z. j =w j *x+b j =w j1 *x1+w j2 *x2+...+w jn *x n +b j Where j represents the j-th class in the first category, w j This represents the weight of the feature in the j-th class, i.e., the importance of each feature dimension and its influence on the final class. Then, the output of the first fully connected layer, the first feature vector z... j The first feature vector z is passed to the first activation layer and its softmax activation function is used to process the first feature vector z. jThe activation processing is performed, and then the score on each category is output, and the output is the probability corresponding to each category, and the probabilities of all categories are between 0 and 1, and the sum is 1. For example, the probability distribution of the first category is as follows: inquiry: 0.7, statement: 0.3, when the first classification network classifies the category, which represents the general text classification category, indicating that the basic dialogue behavior of the user is inquiry or statement.

[0060] In some embodiments, the second classification network includes a second full connection layer and a second activation layer. The second text vector is input into the second classification network of the text classification model, and the second category probability distribution of the training sample classified by the second classification network is obtained, including: inputting the second text vector into the second full connection layer, using the second full connection layer to extract features of the input second text vector to obtain a second feature vector; inputting the second feature vector into the second activation layer, using the second activation layer to activate the second feature vector to obtain the second category probability distribution, wherein the second activation layer is a softmax classification layer.

[0061] Exemplarily, the training sample is converted into a form that can be recognized by a computer through numerical or vectorization, i.e., the training sample is converted into a second text vector, the features of the second text vector are integrated by inputting the second text vector into the second full connection layer to obtain a second feature vector, and then the output second feature vector of the second full connection layer is transmitted to the second activation layer, and the activation function of the second activation layer is used to activate the second feature vector to obtain the probability distribution of the second category.

[0062] For example, when the second text vector of the training text converted into the target character is xi, where i represents the position of the target character in the sentence, and x3 represents the second text vector of the third character in the training text, the second text vector representation of the training text is x=x1, x2,..., x n The second feature vector z j can be obtained by inputting x into the second full connection layer. j *x+b j = w j1 *x1+w j2 *x2+...+w jn *x n +b j where j represents the jth category in the first category, w jThe weight of the jth feature, i.e. the importance of each dimension of the feature, the influence degree on the final category. The second feature vector zj of the output of the second full connection layer is transmitted to the second activation layer, and the activation function softmax of the second activation layer is used to activate the second feature vector zj, and then the score of each category is output. The output is the probability corresponding to each category, and the probability values of all categories are between 0 and 1, and the sum is 1. For example, the second classification network classifies the category of domain text classification, which indicates that the user's dialogue belongs to the feature domain such as symptoms, medical history, diagnosis and treatment history, and medicine. The probability distribution of the second category can be symptoms: 0.3, medical history: 0.2, diagnosis and treatment history: 0.4, and medicine: 0.1.

[0063] Step S5, constructing a first loss function according to the first category probability distribution and a second loss function according to the second category probability distribution, and determining a fusion loss function according to the first loss function and the second loss function.

[0064] Exemplarily, in the model training process, the model parameters need to be continuously optimized according to the loss function and the preset value, and then the target model parameters are obtained. In the classification model training process, the class label corresponding to the maximum prediction probability of the training sample is required to be the same as the real class label, and the probability value is required to be larger in the case of correct prediction. The first loss function constructed by the first category probability distribution and the second loss function constructed by the second category probability distribution are fused to obtain a fusion loss function, so that the first classification network classification and the second classification network classification are jointly modeled to improve the mining ability of the model to the label association.

[0065] In some embodiments, constructing a first loss function according to a first category probability distribution comprises: obtaining a maximum value of the first category probability distribution of the training sample according to the first category probability distribution; obtaining a transformed result of the maximum value of the first category probability distribution by using a log function; and obtaining the first loss function by summing the transformed result of the maximum value of the first category probability distribution and taking a negative value.

[0066] Exemplarily, when the first category probability distribution of the training sample is obtained, the maximum probability in the first category probability distribution is the prediction category result corresponding to the training sample, and the more the prediction categories corresponding to the training sample are the same as the real category label, the better the effect of the model. The first loss function can be shown in formula 1.

[0067]

[0068] wherein, M is the number of categories, p im is the prediction probability of sample i belonging to category m.

[0069] For example, if the correct answer is "question" and the output of the first classification network corresponding to the correct answer is 0.6, the error is -log 0.6 = 0.51; if the output corresponding to "question" is 0.1, the error is -log 0.1 = 2.30. That is, the value of the error is determined by the output result corresponding to the correct answer label, and the more correct answer labels, the smaller the error, that is, the smaller the loss function.

[0070] In some embodiments, the second loss function is constructed according to the second category probability distribution, including: obtaining the maximum value of the second category probability distribution of the training sample according to the second category probability distribution; obtaining the transformed result of the maximum value of the second category probability distribution by using the log function according to the maximum value of the second category probability distribution; and obtaining the second loss function by summing the transformed result of the maximum value of the second category probability distribution of the training sample and taking the negative value.

[0071] For example, when the second category probability distribution of the training sample is obtained, the maximum value of the probability in the second category probability distribution is the prediction category result corresponding to the training sample, and the more the prediction category corresponding to the training sample is the same as the real category label, the better the effect of the model. The second loss function can be shown in formula 2.

[0072]

[0073] wherein, M is the number of categories, p im is the predicted probability that sample i belongs to category m.

[0074] For example, if the correct answer is "question" and the output of the first classification network corresponding to the correct answer is 0.6, the error is -log 0.6 = 0.51; if the output corresponding to "question" is 0.1, the error is -log 0.1 = 2.30. That is, the value of the error is determined by the output result corresponding to the correct answer label, and the more correct answer labels, the smaller the error, that is, the smaller the loss function.

[0075] Please refer to Figure 3 In some embodiments, the fusion loss function is determined according to the first loss function and the second loss function, including steps S501 to S503.

[0076] In step S501, the gradient of the first loss function is obtained according to the first loss function, and the first modulus result corresponding to the gradient of the first loss function is obtained according to the gradient of the first loss function.

[0077] Step S502, according to the first modulus length result, the reciprocal is obtained, the first weight coefficient corresponding to the first loss function is obtained; according to the second loss function, the gradient of the second loss function is obtained, and the second modulus length result corresponding to the gradient of the second loss function is obtained according to the gradient of the second loss function.

[0078] Step S503, according to the second modulus length result, the reciprocal is obtained, the second weight coefficient corresponding to the second loss function is obtained; according to the first loss function and the second loss function, the weight is added according to the first weight coefficient corresponding to the first loss function and the second weight coefficient corresponding to the second loss function, and the fusion loss function is obtained.

[0079] Exemplarily, the first loss function classified by the first classification network and the second loss function classified by the second classification network are jointly modeled by giving different weights, so as to improve the mining ability of the model to the label association. The model fusion loss function is represented as formula 3.

[0080]

[0081] Wherein, L i is the loss function of each classification task, and a i is the weight coefficient of each classification task.

[0082] Exemplarily, the first weight coefficient corresponding to the first loss function is obtained by calculating the first modulus length result corresponding to the gradient of the first loss function according to the gradient of the first loss function, and then calculating the reciprocal of the first modulus length, and then obtaining the first weight coefficient corresponding to the first loss function. The second weight coefficient corresponding to the second loss function is obtained by calculating the second modulus length result corresponding to the gradient of the second loss function according to the gradient of the second loss function, and then calculating the reciprocal of the second modulus length, and then obtaining the second weight coefficient corresponding to the second loss function. The task weight a i is dynamically updated, and the value is the current time t, L i (t) gradient modulus, as shown in formula 4.

[0083]

[0084] sg[·] represents the stop gradient operator, which is to set the derivative function to 0 without changing the original function value, so as to achieve the effect of real-time adjusting the gradient proportion with dynamic weight.

[0085] Step S6, based on the training sample and the fusion loss function, the text classification model is iteratively updated to obtain the target text classification model.

[0086] Exemplarily, the text classification model is iteratively updated according to the training samples, and when the value of the fusion loss function meets the preset condition, the model training is stopped, and thus a target text classification model meeting the current preset condition is obtained. Alternatively, when the value of the fusion loss function does not meet the preset condition, but the value of the fusion loss function in the training round reaches a minimum value as the target text classification model, wherein the specific structure of the text classification model can be as shown in Figure 5 .

[0087] For example, by training the training data, the training data is divided into training samples, test samples and validation samples, and when the value of the fusion loss function of the validation samples reaches the set threshold or the training round reaches the preset value, the model training is stopped, and when the accuracy of the test samples meets the actual demand, the model training is completed and is used in practice, and when the accuracy of the test samples does not meet the actual demand, the model needs to be analyzed, and the model training is re-performed or the model training is performed after increasing the number of collected samples or increasing the training round.

[0088] The application also provides a text classification method, which comprises: inputting text data into a text classification model to obtain a classification result of a first classification network and a classification result of a second classification network, wherein the text classification model is obtained according to any one of the training methods of the text classification model. For detailed description of related contents, please refer to the training method of the text classification model described above, which will not be repeated here.

[0089] Exemplarily, a target text classification model is obtained by training the training data, the text input by a user in a medical inquiry system is sent to the target text classification model, the classification result of the first classification network and the classification result of the second classification network are obtained, and the classification result of the first classification network and the classification result of the second classification network are spliced to obtain a text classification result.

[0090] For example, an inquiry generally has a universal process logic of “asking symptoms-distinguishing diseases-prescribing”, and there is an association relationship between the intent labels. The method splits a single intent label into two intent sub-labels: a general intent A and a domain intent B. The first classification network is used to identify the general intent A, and the second classification network is used to identify the domain intent B. The general intent refers to the basic dialogue behavior of the user such as inquiry and statement, and the domain intent refers to the characteristic field of the user dialogue such as symptoms, medical history, diagnosis and treatment history, and drugs. When the classification and identification result of the first classification network is statement and the classification and identification result of the second classification network is symptoms, the two can be combined to form a complete dialogue intent label statement symptoms.

[0091] Please refer to Figure 5 , Figure 5 The structure of the terminal device provided in the embodiment of the application is shown in the schematic block diagram.

[0092] As shown in Figure 5 The terminal device 300 includes a processor 301 and a memory 302, which are connected through a bus 303, such as an I2C (Inter-integrated Circuit) bus.

[0093] Specifically, the processor 301 is configured to provide computing and control capabilities to support the operation of the entire server. The processor 301 can be a central processing unit (CPU), and the processor 301 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0094] Specifically, the memory 302 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a U disk, or a mobile hard disk, etc.

[0095] Those skilled in the art can understand that ​ The structure shown in the figure is only a block diagram of part of the structure related to the embodiment of the present application, and does not constitute a limitation on the terminal device to which the embodiment of the present application is applied. The specific terminal device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0096] The processor 301 is configured to run a computer program stored in the memory, and implement the training method of the text classification model provided in any embodiment of the present application when the computer program is executed.

[0097] In some embodiments, the processor 301 is configured to run a computer program stored in the memory, applied to a terminal device, and implement the following steps when the computer program is executed:

[0098] Obtain a training sample, the training sample including text data and a category label;

[0099] Input the training sample into a text encoding network of a text classification model to be trained to obtain a first text vector corresponding to the training sample;

[0100] inputting the first text vector into a context encoding network of the text classification model to obtain a second text vector corresponding to the first text vector, wherein the context encoding network is a long short-term memory network, and a vector representation accuracy of the target object in the training sample by the second text vector is greater than that by the first text vector;

[0101] inputting the second text vector into a first classification network of the text classification model to obtain a first category probability distribution of the training sample classified by the first classification network, and inputting the second text vector into a second classification network of the text classification model to obtain a second category probability distribution of the training sample classified by the second classification network;

[0102] constructing a first loss function according to the first category probability distribution and a second loss function according to the second category probability distribution, and determining a fusion loss function according to the first loss function and the second loss function;

[0103] iteratively updating the text classification model based on the training sample and the fusion loss function to obtain a target text classification model.

[0104] In some embodiments, the processor 301, in the process of inputting the training sample into a text encoding network of the text classification model to be trained to obtain a first text vector corresponding to the training sample, performs:

[0105] inputting the training sample into a text encoding layer of the text classification model to be trained to obtain a first vector representation and a second vector representation of a target object in the training sample;

[0106] performing pooling processing on the first vector representation and the second vector representation by using a pooling algorithm to obtain the first text vector corresponding to the training sample.

[0107] In some embodiments, the first classification network includes a first full connection layer and a first activation layer, and the processor 301, in the process of inputting the second text vector into the first classification network of the text classification model to obtain the first category probability distribution of the training sample classified by the first classification network, performs:

[0108] inputting the second text vector into the first full connection layer to perform feature extraction on the input second text vector by using the first full connection layer to obtain a first feature vector;

[0109] inputting the first feature vector into the first activation layer to perform activation processing on the first feature vector by using the first activation layer to obtain the first category probability distribution, wherein the first activation layer is a softmax classification layer.

[0110] In some embodiments, the second classification network comprises a second fully connected layer and a second activation layer, and the processor 301 performs the following in the process of inputting the second text vector into the second classification network of the text classification model to obtain the second category probability distribution of the training sample classified by the second classification network:

[0111] inputting the second text vector into the second fully connected layer, and performing feature extraction on the input second text vector by using the second fully connected layer to obtain a second feature vector;

[0112] inputting the second feature vector into the second activation layer, and performing activation processing on the second feature vector by using the second activation layer to obtain the second category probability distribution, wherein the second activation layer is a softmax classification layer.

[0113] In some embodiments, the processor 301 performs the following in the process of constructing a first loss function according to the first category probability distribution:

[0114] obtaining a first category probability distribution maximum value of the training sample according to the first category probability distribution;

[0115] obtaining a transformed result of the first category probability distribution maximum value by using a log function to transform the first category probability distribution maximum value;

[0116] summing the transformed result of the first category probability distribution maximum value of the training sample and taking a negative value to obtain the first loss function.

[0117] In some embodiments, the processor 301 performs the following in the process of constructing a second loss function according to the second category probability distribution:

[0118] obtaining a second category probability distribution maximum value of the training sample according to the second category probability distribution;

[0119] obtaining a transformed result of the second category probability distribution maximum value by using a log function to transform the second category probability distribution maximum value;

[0120] summing the transformed result of the second category probability distribution maximum value of the training sample and taking a negative value to obtain the second loss function.

[0121] In some embodiments, the processor 301 performs the following in the process of determining a fusion loss function according to the first loss function and the second loss function:

[0122] According to the first loss function, a gradient of the first loss function is obtained, and a first modulus length result corresponding to the gradient of the first loss function is obtained according to the gradient of the first loss function;

[0123] According to the first modulus length result, an inverse is obtained, and a first weight coefficient corresponding to the first loss function is obtained;

[0124] According to the second loss function, a gradient of the second loss function is obtained, and a second modulus length result corresponding to the gradient of the second loss function is obtained according to the gradient of the second loss function;

[0125] According to the second modulus length result, an inverse is obtained, and a second weight coefficient corresponding to the second loss function is obtained;

[0126] According to the first loss function and the second loss function, a weight sum is performed according to the first weight coefficient corresponding to the first loss function and the second weight coefficient corresponding to the second loss function, and a fusion loss function is obtained.

[0127] It should be noted that, for the convenience and brevity of description, the specific working process of the terminal device described above can refer to the corresponding process in the training method of the text classification model described above, and will not be repeated here.

[0128] The embodiment of the present application also provides a storage medium for computer readable storage, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any one of the training methods of the text classification model provided by the embodiments of the present application.

[0129] The storage medium can be an internal storage unit of the terminal device, such as a terminal device memory. The storage medium can also be an external storage device of the terminal device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0130] Those skilled in the art can understand that all or some of the steps in the methods disclosed above and the functional modules / units in the devices can be implemented by software, firmware, hardware, or a combination thereof. In hardware embodiments, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component can have multiple functions, or one function or step can be performed by several physical components in cooperation. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on computer-readable media, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As is well known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and can include any information delivery media.

[0131] It should be understood that the term "and / or" as used herein refers to any combination of associated listed items, and all possible combinations, and includes these combinations. It should be noted that the terms "comprising", "including", or any other variant thereof, are intended to cover non-exclusive inclusion, so that processes, methods, articles, or systems including a series of elements not only include those elements, but also include other elements not explicitly listed, or other elements inherent to such processes, methods, articles, or systems. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or system including the element.

[0132] The above sequence numbers of the embodiments of the present application are only for description, and do not represent advantages or disadvantages of the embodiments. The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for training a text classification model, characterized in that, The training method comprises: obtaining a training sample, the training sample comprising text data and a category label; inputting the training sample into a text encoding network of a text classification model to be trained to obtain a first text vector corresponding to the training sample; inputting the first text vector into a context encoding network of the text classification model to obtain a second text vector corresponding to the first text vector, wherein the context encoding network is a long short-term memory network, and the second text vector has a higher vector representation accuracy of a target object in the training sample than the first text vector; inputting the second text vector into a first classification network of the text classification model to obtain a first category probability distribution of the training sample classified by the first classification network, and inputting the second text vector into a second classification network of the text classification model to obtain a second category probability distribution of the training sample classified by the second classification network; the classification category of the first classification network is general text, and the general text comprises basic dialogue behaviors of a user; the classification category of the second classification network is domain text, and the domain text comprises a characteristic domain to which a user dialogue belongs; constructing a first loss function according to the first category probability distribution and a second loss function according to the second category probability distribution, and determining a fusion loss function according to the first loss function and the second loss function; iteratively updating the text classification model based on the training sample and the fusion loss function to obtain a target text classification model.

2. The method of claim 1, wherein, The inputting of the training sample into the text encoding network of the text classification model to be trained to obtain the first text vector corresponding to the training sample comprises: inputting the training sample into the text encoding network of the text classification model to be trained to obtain a first vector representation and a second vector representation of a target object in the training sample; performing pooling processing on the first vector representation and the second vector representation by using a pooling algorithm to obtain the first text vector corresponding to the training sample.

3. The method of claim 1, wherein, The first classification network comprises a first full connection layer and a first activation layer, and the inputting of the second text vector into the first classification network of the text classification model to obtain the first category probability distribution of the training sample classified by the first classification network comprises: inputting the second text vector into the first full connection layer, performing feature extraction on the input second text vector by using the first full connection layer to obtain a first feature vector; inputting the first feature vector into the first activation layer, performing activation processing on the first feature vector by using the first activation layer to obtain the first category probability distribution, wherein the first activation layer is a softmax classification layer.

4. The method of claim 3, wherein, The second classification network comprises a second full connection layer and a second activation layer, and the inputting of the second text vector into the second classification network of the text classification model to obtain the second category probability distribution of the training sample classified by the second classification network comprises: inputting the second text vector into the second full connection layer, performing feature extraction on the input second text vector by using the second full connection layer to obtain a second feature vector; inputting the second feature vector into the second activation layer, performing activation processing on the second feature vector by using the second activation layer to obtain the second category probability distribution, wherein the second activation layer is a softmax classification layer.

5. The method of claim 1, wherein, The first loss function is constructed according to the first category probability distribution, including: According to the first category probability distribution, the maximum value of the first category probability distribution of the training sample is obtained; According to the maximum value of the first category probability distribution, the conversion result of the maximum value of the first category probability distribution is obtained by using the log function; The conversion result of the maximum value of the first category probability distribution of the training sample is summed and taken as a negative value, and then the first loss function is obtained.

6. The method of claim 5, wherein, The second loss function is constructed according to the second category probability distribution, including: According to the second category probability distribution, the maximum value of the second category probability distribution of the training sample is obtained; According to the maximum value of the second category probability distribution, the conversion result of the maximum value of the second category probability distribution is obtained by using the log function; The conversion result of the maximum value of the second category probability distribution of the training sample is summed and taken as a negative value, and then the second loss function is obtained.

7. The method of claim 6, wherein, According to the first loss function and the second loss function, a fusion loss function is determined, including: According to the first loss function, the gradient of the first loss function is obtained, and the first module length result corresponding to the gradient of the first loss function is obtained according to the gradient of the first loss function; According to the first module length result, the reciprocal is obtained, and the first weight coefficient corresponding to the first loss function is obtained; According to the second loss function, the gradient of the second loss function is obtained, and the second module length result corresponding to the gradient of the second loss function is obtained according to the gradient of the second loss function; According to the second module length result, the reciprocal is obtained, and the second weight coefficient corresponding to the second loss function is obtained; According to the first loss function and the second loss function, the weight sum is performed according to the first weight coefficient corresponding to the first loss function and the second weight coefficient corresponding to the second loss function, and the fusion loss function is obtained.

8. A method of text classification characterized by, The method comprises: inputting text data into a text classification model to obtain the classification result of the first classification network and the classification result of the second classification network, wherein the text classification model is obtained according to the training method of the text classification model in any one of claims 1-7.

9. A terminal device, comprising: The terminal device comprises a processor and a memory; The memory is used to store a computer program; The processor is used to execute the computer program and realize the training method of the text classification model and the text classification method in claim 8 when the computer program is executed. The terminal device comprises a processor and a memory; The memory is used to store a computer program; The processor is used to execute the computer program and realize the training method of the text classification model and the text classification method in claim 8 when the computer program is executed.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium, when executed by the one or more processors, causes the one or more processors to perform the steps of the method for training a text classification model as claimed in any one of claims 1 to 7 and the steps of the method for text classification as claimed in claim 8.

Images