Knowledge distillation-based oral language scene chief complaint symptom detection method and device, medium and computer equipment

Abstract

The invention provides a knowledge distillation-based oral language scene chief complaint symptom detection method and device, a medium and computer equipment, and the method comprises the steps: carrying out the data labeling of collected oral language chief complaint data, and obtaining a plurality of student data sets and a plurality of teacher data sets; training a student network by using the student data group, and training a teacher network by using the teacher data group; performing knowledge distillation on the trained student network by using the trained teacher network to obtain a chief complaint symptom detection network; inputting a to-be-identified oral chief complaint statement into the chief complaint symptom detection network for identification to obtain a corresponding symptom prediction result; by adopting the method provided by the invention, through a knowledge distillation technology, the detection precision of a student network on the chief complaint symptom in an oral language scene is effectively improved; knowledge of the trained teacher network is migrated to the student network, the student network learns more complex mode information from the teacher network, and the ability of the student network to classify oral chief complaint data is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of symptom recognition, and more specifically, to a method, device, medium and computer equipment for detecting main complaint symptoms in oral scenarios based on knowledge distillation. Background Art

[0002] In the existing technology, with the rapid development of large language models, the importance of generative AI in various fields has become increasingly prominent. However, in the actual application of the TCM consultation model for consultation, due to the high degree of colloquialism in the user input text, the main symptoms captured by the large model are inaccurate. At the same time, the large number of symptoms in the disease symptom library also brings great difficulties to symptom positioning. Summary of the invention

[0003] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a method, device, medium and computer equipment for detecting spoken scenario chief complaint symptoms based on knowledge distillation, so as to overcome the shortcomings of the prior art that spoken chief complaint symptoms are difficult to accurately identify.

[0004] The above technical objectives of the present invention are achieved through the following technical solutions: A method for detecting symptoms in oral scenarios based on knowledge distillation, comprising: The collected oral complaint data are annotated to obtain a number of student data groups and a number of teacher data groups; wherein the student data groups all include: oral complaint data and symptoms; and the teacher data groups all include: oral fluency data and symptoms; The student network is trained using the student data set, and the teacher network is trained using the teacher data set; knowledge distillation is performed on the trained student network using the trained teacher network to obtain a chief complaint symptom detection network; wherein both the student network and the teacher network are multi-label classification models; The colloquial chief complaint sentence to be identified is input into the chief complaint symptom detection network for identification to obtain the corresponding symptom prediction result.

[0005] In one embodiment, the collected spoken complaint data is labeled to obtain a number of student data groups and a number of teacher data groups, specifically including: Cleaning the collected oral complaint data, annotating the cleaned oral complaint data, and obtaining oral fluent data and symptoms corresponding to the oral complaint data; Recording the spoken complaint data, the spoken fluency data and the symptoms as a triple data set; Extract the spoken complaint data and symptoms from each of the three data sets and record them as student data sets; The oral fluency data and symptoms are extracted from each of the three data sets and recorded as a teacher data set.

[0006] In one embodiment, the training of the student network using the student data set specifically includes: Convert the spoken complaint data in the student data set into spoken sentence vectors, convert the symptoms in the student data set into symptom vectors using One-hot encoding, and use the symptom vectors as labels corresponding to the spoken sentence vectors; Initialize the weight data of the student network, input each transformed student data group as a training sample into the student network, pass through each hidden layer in turn, and obtain the result of the output layer; wherein each hidden layer is a fully connected layer, and each hidden layer is nonlinearly mapped through a ReLU activation function; the output layer is a fully connected layer, and the output layer uses a Sigmoid activation function, and the result output by the output layer is the probability of each symptom; Calculate the loss function between the student network prediction output and the true symptom label, specifically including:

[0007] in, Indicates The samples have The true value of the label; Represents the student network prediction sample Belong to The probability of a label, its value range is [0,1]; is the number of samples used to train the student network, indicates the number of symptom categories; The model optimizer is used to optimize the model weights of the student network according to the loss function to obtain a trained student network.

[0008] In one embodiment, converting the spoken complaint data in the student data set into a spoken sentence vector specifically includes: The Encoder in the BERT model is used to encode each of the spoken complaint data to generate a corresponding spoken sentence vector.

[0009] In one embodiment, the encoder in the BERT model is used to encode each of the spoken complaint data to generate a corresponding spoken sentence vector, specifically including: The spoken complaint data is segmented according to the word granularity, and each character after segmentation is converted into a one-hot vector , and obtain the word embedding matrix corresponding to each of the spoken complaint data ,include:

[0010] in, Indicates One-hot vector corresponding to the character ; Using the query weight matrix in the BERT model , key weight matrix and the value weight matrix , calculate the word embedding matrix The query vector , key vector and the value vector ,include:

[0011]

[0012]

[0013] The query vector is respectively analyzed using each attention mechanism in the BERT model. , key vector and the value vector Calculate and generate the corresponding attention vector ,include:

[0014]

[0015]

[0016] in, Indicates The attention vector calculated by the attention mechanism; Indicates The query weight matrix in the attention mechanism; Indicates The key weight matrix in the attention mechanism; Indicates The value weight matrix in the attention mechanism; express The dimension of the vector; The attention vectors generated by the attention mechanisms are concatenated and compressed to obtain spoken sentence vectors corresponding to the spoken complaint data, including:

[0017] in, represents the concatenation function; is the additional weight matrix.

[0018] In one embodiment, the training of the teacher network using the teacher data set specifically includes: The oral fluent data in the teacher data set are converted into fluent sentence vectors, the symptoms in the teacher data set are converted into symptom vectors using One-hot encoding, and the symptom vectors are used as labels corresponding to the fluent sentence vectors; Initialize the weight data of the teacher network, input each transformed teacher data group as a training sample into the teacher network, pass through each hidden layer in turn, and obtain the result of the output layer; wherein each hidden layer is a fully connected layer, and each hidden layer is nonlinearly mapped through a ReLU activation function; the output layer is a fully connected layer, and the output layer uses a Sigmoid activation function, and the result output by the output layer is the probability of each symptom; Calculate the loss function between the teacher network prediction output and the true symptom label, specifically including:

[0019] in, Indicates The samples have The true value of the label; Represents the teacher network prediction sample Belong to The probability of a label, its value range is [0,1]; is the number of samples used to train the teacher network, indicates the number of symptom categories; A model optimizer is used to optimize the model weights of the teacher network according to the loss function to obtain a trained teacher network.

[0020] In one embodiment, converting the spoken fluent data in the teacher data set into a fluent sentence vector specifically includes: The Encoder in the BERT model is used to encode each of the fluent spoken data to generate a corresponding fluent sentence vector.

[0021] In one embodiment, the method of using the trained teacher network to perform knowledge distillation on the trained student network to obtain a main complaint symptom detection network specifically includes: Input the colloquial complaint data into the trained teacher network to obtain the first prediction result output by the teacher network , the first prediction result The teacher network predicts that the spoken complaint data meets the first The predicted probability of each symptom; The colloquial complaint data is input into the trained student network to obtain a second prediction result output by the student network. , the second prediction result The student network predicts that the spoken complaint data meets the first The predicted probability of each symptom; Using all first prediction results And the second prediction result Calculate the divergence loss between the student network and the teacher network ; Using all first prediction results And the second prediction result Calculate the total distillation loss of the student network ; Using the divergence loss , the loss function between the student network prediction output and the true symptom label and the total distillation loss of the student network , calculate the total loss function of the student network ; The gradient in the loss function is used to update the parameters of the student network to obtain the chief complaint symptom detection network after knowledge distillation.

[0022] In one embodiment, the spoken complaint data is input into the trained teacher network to obtain a first prediction result output by the teacher network. , specifically including:

[0023] in, Represents the predicted value output by the teacher network without activation function processing; Represents the probability after being processed by the Sigmoid activation function.

[0024] In one embodiment, the spoken complaint data is input into the trained student network to obtain a second prediction result output by the student network. , specifically including:

[0025] in, Represents the predicted value output by the student network without activation function processing; Represents the probability distribution after being processed by the Sigmoid activation function.

[0026] In one embodiment, it is characterized in that the use of all first prediction results And the second prediction result Calculate the divergence loss between the student network and the teacher network , specifically including:

[0027] in, Indicates the number of symptom categories.

[0028] In one embodiment, the use of all first prediction results And the second prediction result Calculate the total distillation loss of the student network , specifically including:

[0029] in, represents the overall prediction distribution of the student network for the input data; represents the overall prediction distribution of the teacher network for the input data; Indicates true category label; Represents the control factor used to adjust the classification loss and distillation losses Importance in the training process; represents the classification loss of the student network; represents the distillation loss of the student network.

[0030] In one embodiment, the use of the divergence loss , the loss function between the student network prediction output and the true symptom label and the total distillation loss of the student network , calculate the total loss function of the student network , specifically including: .

[0031] A device for detecting symptoms of oral complaints based on knowledge distillation, comprising: A labeling unit is used to label the collected oral complaint data to obtain a plurality of student data groups and a plurality of teacher data groups; wherein the student data groups all include: oral complaint data and symptoms; and the teacher data groups all include: oral fluency data and symptoms; A training unit, used to train a student network using the student data set, and to train a teacher network using the teacher data set; and to perform knowledge distillation on the trained student network using the trained teacher network to obtain a main complaint symptom detection network; The prediction unit is used to input the colloquial chief complaint sentence to be identified into the chief complaint symptom detection network for identification, and obtain the corresponding symptom prediction result.

[0032] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0033] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0034] In summary, the present invention has the following beneficial effects: a method for detecting chief complaint symptoms in oral scenarios based on knowledge distillation, comprising: data annotation of collected oral chief complaint data to obtain several student data groups and several teacher data groups; wherein the student data groups all include: spoken chief complaint data and symptoms; the teacher data groups all include: spoken smooth data and symptoms; using the student data group to train the student network, and using the teacher data group to train the teacher network; using the trained teacher network to perform knowledge distillation on the trained student network to obtain a chief complaint symptom detection network; inputting the spoken chief complaint sentence to be identified into the chief complaint symptom detection network for identification to obtain the corresponding symptom prediction result; adopting the method of the present invention, through the knowledge distillation technology, using the training process of the teacher network and the student network, effectively improves the detection accuracy of the chief complaint symptoms in the oral scenario; through knowledge distillation, the knowledge of the trained teacher network is transferred to the student network, and the student network learns more complex and deep pattern information from the teacher network, thereby improving its understanding and classification ability of the spoken chief complaint data. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of a method for detecting symptoms of oral scenarios based on knowledge distillation of the present invention; Figure 2 is a structural diagram of a device for detecting symptoms of oral scenarios based on knowledge distillation in an embodiment of the present invention; Figure 3 is an internal structure diagram of a computer device in an embodiment of the present invention; Figure 4 Schematic diagram of the knowledge distillation principle in an embodiment of the present invention.

[0036] In the figure: 1. Labeling unit; 2. Training unit; 3. Prediction unit. DETAILED DESCRIPTION

[0037] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings. Several embodiments of the present invention are shown in the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein.

[0038] The following at least one (item) involved in the present application indicates one (item) or more (items). More than one (item) refers to two (items) or more than two (items). "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In addition, it should be understood that although the terms first, second, etc. may be used to describe each object in the present application, these objects should not be limited to these terms. These terms are only used to distinguish each object from each other.

[0039] The terms "including" and "having" and any variations thereof mentioned in the following description of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices. It should be noted that in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any method or design described in this application as "exemplary" or "for example" should not be interpreted as being more preferred or advantageous than other methods or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way. Example 1 In order to solve the above problems, the present invention provides a method for detecting symptoms in oral scenarios based on knowledge distillation. Figure 1 As shown, including: The collected oral complaint data are annotated to obtain a number of student data groups and a number of teacher data groups; wherein the student data groups all include: oral complaint data and symptoms; and the teacher data groups all include: oral fluency data and symptoms; The student network is trained using the student data set, and the teacher network is trained using the teacher data set; knowledge distillation is performed on the trained student network using the trained teacher network to obtain a chief complaint symptom detection network; wherein both the student network and the teacher network are multi-label classification models; The colloquial chief complaint sentence to be identified is input into the chief complaint symptom detection network for identification to obtain the corresponding symptom prediction result.

[0040] In the actual medical diagnosis process, the spoken chief complaint data contains various spoken expressions, especially when the patient is sick, which will seriously affect the patient's ability to express himself, or the patient's companion will affect the judgment of the actual condition due to retelling. The patient's explanation will also contain a large number of exclamations, repetitive words, dialects, blank text, etc. For example: "Oh, my head hurts so much today, especially this morning, it's so uncomfortable!" "My child has been very listless these days, always saying that his head hurts." In order to improve the efficiency and accuracy of medical consultation, this application proposes a chief complaint symptom detection algorithm, which improves the accuracy of symptom detection through a BERT-based knowledge distillation training method, assists the large model of traditional Chinese medicine consultation to correctly capture the chief complaint symptom information, and accurately locates it in the symptom library. First, we need to establish a knowledge base of TCM disease symptoms based on the 11th revision of the International Classification of Diseases (ICD-11), which contains 222 TCM disease symptoms. Based on these 222 TCM disease symptoms, we need to annotate the collected oral complaint data, that is, to match the collected oral complaint data with the symptoms they want to express one by one. And while annotating, we divide the data into teacher data group and student data group. Among them, the teacher data group is oral smooth data and symptoms, and the student data group is oral complaint data and symptoms. Among them, the spoken complaint data is the most original colloquial expression content collected, and the oral smooth data in the teacher data group is the content obtained after smoothing the spoken complaint data. For example, the original spoken complaint data is: "Oh, I have a very, very painful head today, especially this morning, and I am so uncomfortable"; after oral smoothing, the oral smooth sentence "I have a very painful head this morning" is obtained. The main function is to remove stop words and interjections, and add meaningless interjections such as "Alas, ah, er, de, ah, ba" to the stop word library and filter them, thereby reducing the feature dimension of the spoken complaint text. Then invite professional Chinese medicine doctors to annotate the text after smoothing. Under the condition of ensuring the complete expression of the complaint information, each oral smooth text is classified into symptoms according to the disease symptom library, so that each spoken complaint data can be associated with the corresponding symptoms. Therefore, the spoken complaint data, oral smooth data and symptoms can be associated one by one. Form a training data set, then use the teacher data set to train the teacher network, and use the student data set to train the student network. The structure of the teacher network is more complex, and the training results are more accurate. The structure of the student network is simpler and lighter, which is convenient for deployment on terminal devices with low computing power. Through the joint training of the teacher network and the student network, the model can learn from the data from different perspectives, thereby improving its generalization ability.After the two models are trained, the complex model knowledge of the teacher network is transferred to the student network using knowledge distillation technology, so that the student network can obtain prediction performance close to that of the teacher network while maintaining relatively low complexity, reducing the use of computing resources and improving the reasoning speed. In the final recognition process, the student network that has learned the knowledge of the teacher network is used to predict the oral complaint data to be predicted, which can improve the reasoning speed and maintain a high reasoning accuracy while using fewer computing resources.

[0041] In one embodiment, the data annotation of the collected spoken complaint data to obtain a plurality of student data groups and a plurality of teacher data groups specifically includes: data cleaning of the collected spoken complaint data, annotation of the cleaned spoken complaint data to obtain oral fluency data and symptoms corresponding to the spoken complaint data one by one; recording the spoken complaint data, the oral fluency data and the symptoms as a triple data group; extracting the spoken complaint data and the symptoms from each of the triple data groups to record as a student data group; extracting the oral fluency data and the symptoms from each of the triple data groups to record as a teacher data group.

[0042] In actual use, data cleaning is used to remove meaningless words in the spoken complaint data, mainly by filtering them with stop words, thereby reducing the feature dimensions of the spoken complaint text. By constructing a three-dimensional data set (spoken complaint data, oral fluency data, symptoms), multi-dimensional information input is provided for subsequent training. By separating the same data into student data sets and teacher data sets, which are used for training different networks respectively, the utilization rate of training data is effectively improved and data redundancy is avoided. At the same time, the teacher network can provide a more standard reference to help the student network better learn symptom information under irregular expressions.

[0043] In one embodiment, the student network is trained using the student data set, specifically including: converting the spoken complaint data in the student data set into spoken sentence vectors, converting the symptoms in the student data set into symptom vectors using One-hot encoding, and using the symptom vectors as labels corresponding to the spoken sentence vectors; initializing the weight data of the student network, inputting each converted student data set into the student network as a training sample, passing through each hidden layer in turn, and obtaining the result of the output layer; wherein each hidden layer is a fully connected layer, and each hidden layer is nonlinearly mapped through a ReLU activation function; the output layer is a fully connected layer, and the output layer uses a Sigmoid activation function, and the result output by the output layer is the probability of each symptom; Calculate the loss function between the student network prediction output and the true symptom label, specifically including:

[0044] in, Indicates The samples have The true value of the label; Represents the student network prediction sample Belong to The probability of a label, its value range is [0,1]; is the number of samples used to train the student network, represents the number of symptom categories; using a model optimizer to optimize the model weights of the student network according to the loss function to obtain a trained student network.

[0045] In actual use, symptoms are converted into symptom vectors using One-hot encoding, so that each symptom corresponds to a unique vector. This representation method is simple and intuitive, which helps the student network to clearly distinguish different categories of symptoms in classification problems and improves the efficiency of label expression during training. Each hidden layer of the student network is a fully connected layer, which means that each neuron is connected to all neurons in the previous layer. This structure is suitable for various data inputs and can capture the global characteristics of the input data, improving the network's ability to express complex data patterns. The Sigmoid activation function is used in the output layer to limit the output value between 0 and 1, so that the probability value corresponding to the symptom can be directly output, which is convenient for the model to classify different symptom categories. The process of the student network, through the combination of fully connected layers, activation functions, and loss functions, enables the student network to have a strong generalization ability when learning complex patterns of spoken complaint data, which helps to improve the performance of the model on unseen data in real scenarios.

[0046] In one embodiment, converting the spoken complaint data in the student data set into a spoken sentence vector specifically includes: The Encoder in the BERT model is used to encode each of the spoken complaint data to generate a corresponding spoken sentence vector.

[0047] In one embodiment, the encoder in the BERT model is used to encode each of the spoken complaint data to generate a corresponding spoken sentence vector, specifically including: The spoken complaint data is segmented according to the word granularity, and each character after segmentation is converted into a one-hot vector , and obtain the word embedding matrix corresponding to each of the spoken complaint data ,include:

[0048] in, Indicates One-hot vector corresponding to the character ; Using the query weight matrix in the BERT model , key weight matrix and the value weight matrix , calculate the word embedding matrix The query vector , key vector and the value vector ,include:

[0049]

[0050]

[0051] The query vector is respectively analyzed using each attention mechanism in the BERT model. , key vector and the value vector Calculate and generate the corresponding attention vector ,include:

[0052]

[0053]

[0054] in, Indicates The attention vector calculated by the attention mechanism; Indicates The query weight matrix in the attention mechanism; Indicates The key weight matrix in the attention mechanism; Indicates The value weight matrix in the attention mechanism; express The dimension of the vector; The attention vectors generated by the attention mechanisms are concatenated and compressed to obtain spoken sentence vectors corresponding to the spoken complaint data, including:

[0055] in, represents the concatenation function; is the additional weight matrix.

[0056] Specifically, this embodiment uses the BERT model to convert the colloquial chief complaint data into corresponding colloquial smooth data, effectively enhancing the model's semantic understanding ability in the colloquial chief complaint data. Among them, by performing word-granular character segmentation operations on the colloquial chief complaint data, the model can capture the subtle semantic differences of each character in the sentence; by calculating the query, key, and value vectors through the Attention mechanism, the model can capture the long-distance dependencies in the sentence, thereby improving the understanding of the contextual relationship in complex sentences; for the colloquial chief complaint data, the self-attention mechanism of the BERT model can help the model to effectively understand the semantics and generate accurate sentence vectors in the case of non-standard expressions, thereby improving the model's fault tolerance for colloquial expressions and avoiding recognition errors due to differences in grammar or structure.

[0057] In one embodiment, the teacher network is trained using the teacher data group, specifically including: converting the spoken fluent data in the teacher data group into a fluent sentence vector, converting the symptoms in the teacher data group into a symptom vector using One-hot encoding, and using the symptom vector as a label corresponding to the fluent sentence vector; initializing the weight data of the teacher network, inputting each converted teacher data group into the teacher network as a training sample, passing through each hidden layer in turn, and obtaining the result of the output layer; wherein each hidden layer is a fully connected layer, and each hidden layer is nonlinearly mapped through a ReLU activation function; the output layer is a fully connected layer, and the output layer uses a Sigmoid activation function, and the result output by the output layer is the probability of each symptom; Calculate the loss function between the teacher network prediction output and the true symptom label, specifically including:

[0058] in, Indicates The samples have The true value of the label; Represents the teacher network prediction sample Belong to The probability of a label, its value range is [0,1]; is the number of samples used to train the teacher network, represents the number of symptom categories; using a model optimizer to optimize the model weights of the teacher network according to the loss function to obtain a trained teacher network.

[0059] In one embodiment, converting the spoken fluent data in the teacher data set into a fluent sentence vector specifically includes: The Encoder in the BERT model is used to encode each of the fluent spoken data to generate a corresponding fluent sentence vector.

[0060] The colloquial smooth data in the teacher data set is converted into smooth sentence vectors, ensuring the standardization of the input data and reducing the noise that may exist in colloquial expressions. Each hidden layer of the teacher network is a fully connected layer, ensuring that each neuron is connected to all neurons in the previous layer, and has strong expressive power; the model optimizer is used to optimize the weights of the teacher network according to the loss function, ensuring that the adjustment direction of the model weights is correct at each iteration, thereby accelerating the convergence of the model and reducing training time; the training of the teacher network enables it to extract highly accurate prediction results, which lays the foundation for the subsequent knowledge distillation process. By transferring the knowledge of the teacher network to the student network, the student network can perform close to the performance of the teacher network in complex tasks while maintaining low network complexity.

[0061] In one embodiment, the method of using the trained teacher network to perform knowledge distillation on the trained student network to obtain a chief complaint symptom detection network specifically includes: inputting the colloquial chief complaint data into the trained teacher network to obtain a first prediction result output by the teacher network , the first prediction result The teacher network predicts that the spoken complaint data meets the first The predicted probability of a symptom; inputting the colloquial complaint data into the trained student network to obtain a second prediction result output by the student network , the second prediction result The student network predicts that the spoken complaint data meets the first The predicted probability of each symptom; using all the first prediction results And the second prediction result Calculate the divergence loss between the student network and the teacher network ; Using all first prediction results And the second prediction result Calculate the total distillation loss of the student network ; Using the divergence loss , the loss function between the student network prediction output and the true symptom label and the total distillation loss of the student network , calculate the total loss function of the student network ; Use the gradient in the loss function to update the parameters of the student network to obtain the chief complaint symptom detection network after knowledge distillation.

[0062] Specifically, if Figure 4As shown in the figure, in actual use, after both the teacher network and the student network are trained, it is necessary to use knowledge distillation technology to enable the student network to learn the characteristics of the teacher network, so that the student network can make the model's reasoning accuracy closer to the teacher network without increasing the complexity of the network. First, through sentence-level knowledge distillation, that is, the colloquial text classification model (student model) and the smooth text classification model (teacher model) are compared and learned, so that the colloquial model learns the ability of the smooth text model and optimizes the model performance. Then through word-level knowledge distillation: through the distillation between the teacher network (smooth text classification model) and the student network (colloquial complaint classification model), the student network colloquial text classification model can more accurately handle complex and changeable colloquial texts. The process of knowledge distillation not only enables the student network to better fit the training data, but also effectively improves its generalization ability on unseen data. By learning refined prediction results from the teacher network, the student network can better handle complex and non-standard input data. By calculating the divergence loss (kl_loss) and distillation loss (kd_loss), combined with the original loss function (Loss1), the learning of the student network can be more comprehensively optimized. This multi-loss optimization strategy enables the student network to not only perform well in matching the labels of the training data, but also make more accurate predictions by referring to the prediction results of the teacher network.

[0063] In one embodiment, the spoken complaint data is input into the trained teacher network to obtain a first prediction result output by the teacher network. , specifically including:

[0064] in, Represents the predicted value output by the teacher network without activation function processing; Represents the probability after being processed by the Sigmoid activation function.

[0065] In one embodiment, the spoken complaint data is input into the trained student network to obtain a second prediction result output by the student network. , specifically including:

[0066] in, Represents the predicted value output by the student network without activation function processing; Represents the probability distribution after being processed by the Sigmoid activation function.

[0067] In one embodiment, the use of all first prediction results And the second prediction result Calculate the divergence loss between the student network and the teacher network , specifically including:

[0068] in, Indicates the number of symptom categories.

[0069] By calculating the divergence loss (KL divergence) between the student network and the teacher network, this step can transfer the high-quality predicted probability distribution of the teacher network to the student network, helping the student network to output a smoother and more reasonable probability distribution; through the KL divergence loss, the student network can learn the teacher network's understanding of the complex patterns of the data, especially the subtle differences between different categories.

[0070] In one embodiment, the use of all first prediction results And the second prediction result Calculate the total distillation loss of the student network , specifically including:

[0071] in, represents the overall prediction distribution of the student network for the input data; represents the overall prediction distribution of the teacher network for the input data; Indicates true category label; Represents the control factor used to adjust the classification loss and distillation losses Importance in the training process; represents the classification loss of the student network; represents the distillation loss of the student network.

[0072] By introducing the control factor , this step can be achieved in the classification loss and distillation losses Flexible adjustment between them. The learning ratio of the student network to the true label and the teacher network's predicted distribution is controlled to ensure that the student network can fully absorb the high-quality knowledge of the teacher network while learning the true label. By combining classification loss and distillation loss, this step can improve the generalization ability of the student network. Classification loss The student network is able to accurately fit the training data, while the distillation loss It helps the student network learn more hidden features and complex patterns from the teacher network, so that it can perform better on unseen data. By integrating the classification loss and distillation loss into a total distillation loss ,This step provides a unified optimization objective for the student network.

[0073] In one embodiment, the use of the divergence loss , the loss function between the student network prediction output and the true symptom label and the total distillation loss of the student network , calculate the total loss function of the student network , specifically including: .

[0074] Specifically, by using the divergence loss , classification loss of the student network and total distillation loss Integrated into the total loss function , this step can consider multiple learning objectives at the same time. Combined with multiple loss terms, the student network can learn more generalization characteristics during training. Divergence loss, classification loss, and distillation loss focus on different learning objectives. Integrating these losses can balance the model's weight between learning true labels and transferring teacher network knowledge, so that the student network will not be overly dependent on a certain goal during the learning process, ensuring comprehensive learning.

[0075] In one embodiment, it also includes: constructing a detection vocabulary library, which includes symptoms and colloquial matching words listed according to each symptom; after obtaining the colloquial chief complaint data, determining whether there is a colloquial matching word that is the same as the colloquial chief complaint data, and if so, taking the symptom corresponding to the colloquial matching word as the symptom prediction result.

[0076] Specifically, in this application, in order to reduce the amount of calculation of the device and avoid the problem of wasting computing resources due to the device running too many model calculation data, this application also includes the following steps: construct a detection word library, the detection word library stores 222 symptoms collected and summarized in advance, lists the spoken chief complaint expression according to each symptom, and then summarizes the corresponding preposition and postposition words according to the spoken chief complaint expression. When the spoken chief complaint data is collected, the spoken chief complaint data is first matched with the preposition-postposition word phrase in the database. The matching process is mainly to detect whether there are prepositions and postpositions in the spoken chief complaint data, and there may be other contents between the prepositions and postpositions. As long as there are prepositions and postpositions in the spoken chief complaint data, and the preposition is before the postposition, it can be determined that the corresponding symptoms are matched, and the symptoms corresponding to the phrase are used as prediction results. In the case that the preposition-postposition phrase cannot be matched, the symptom prediction model constructed in the previous part is called to predict and identify the symptoms to be expressed in the spoken chief complaint data to obtain the final recognition result.

[0077]

[0078] As shown in Table 1 above, for the labeling and matching of gray hair symptoms, first list the descriptive keywords of gray hair symptoms, including "gray hair", "white hair", "hair color change", "white hair", "growing gray hair", etc.; based on the previous keywords, summarize and list the colloquial expressions in daily language, and then summarize and extract the preposition-postposition phrases from the colloquial expressions, where the preposition and postposition can be arbitrarily combined for symptom matching. In order to reduce the computing power consumption caused by the calculation model, before obtaining a colloquial complaint data, first use the matching rules to try to match in the detection vocabulary to determine whether there is a preposition-postposition phrase that can be matched. If so, the matched symptoms are used as the prediction results. If not, the subsequent model is used to extract the complaint symptoms.

[0079] Embodiment 2 See also Figure 2 , a device for detecting symptoms of oral scene complaints based on knowledge distillation, the device for detecting symptoms of oral scene complaints based on knowledge distillation comprising: The labeling unit 1 is used to label the collected oral complaint data to obtain a plurality of student data groups and a plurality of teacher data groups; wherein the student data groups all include: oral complaint data and symptoms; and the teacher data groups all include: oral fluency data and symptoms; A training unit 2 is used to train the student network using the student data set and train the teacher network using the teacher data set; and perform knowledge distillation on the trained student network using the trained teacher network to obtain a main complaint symptom detection network; The prediction unit 3 is used to input the colloquial chief complaint sentence to be identified into the chief complaint symptom detection network for identification, and obtain the corresponding symptom prediction result.

[0080] For the specific limitations of the oral scenario chief complaint symptom detection device based on knowledge distillation, please refer to the limitations of the oral scenario chief complaint symptom detection method based on knowledge distillation above, which will not be repeated here. Each module in the above-mentioned oral scenario chief complaint symptom detection device based on knowledge distillation can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0081] Those skilled in the art will understand that Figure 2The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation of the scheme of the present application. A specific oral scenario chief complaint symptom detection device based on knowledge distillation may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0082] Embodiment 3 A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for detecting symptoms in oral scenarios based on knowledge distillation as described in Example 1 is implemented.

[0083] Embodiment 4 In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the computer program is executed by the processor, a method for detecting symptoms of oral scenarios based on knowledge distillation is implemented.

[0084] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0085] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented: S1. Annotate the collected oral complaint data to obtain several student data groups and several teacher data groups; wherein the student data groups all include: oral complaint data and symptoms; and the teacher data groups all include: oral fluency data and symptoms; S2, using the student data set to train the student network, and using the teacher data set to train the teacher network; using the trained teacher network to perform knowledge distillation on the trained student network to obtain a main complaint symptom detection network; S3. Input the colloquial chief complaint sentence to be identified into the chief complaint symptom detection network for identification, and obtain the corresponding symptom prediction result.

[0086] In one embodiment, the collected spoken complaint data is labeled to obtain a number of student data groups and a number of teacher data groups, specifically including: Cleaning the collected oral complaint data, annotating the cleaned oral complaint data, and obtaining oral fluent data and symptoms corresponding to the oral complaint data; Recording the spoken complaint data, the spoken fluency data and the symptoms as a triple data set; Extract the spoken complaint data and symptoms from each of the three data sets and record them as student data sets; The oral fluency data and symptoms are extracted from each of the three data sets and recorded as a teacher data set.

[0087] In one embodiment, it also includes: converting each of the spoken complaint data into a corresponding spoken sentence vector; converting each of the spoken smooth data into a corresponding smooth sentence vector.

[0088] In one embodiment, the converting each of the spoken complaint data into a corresponding spoken sentence vector; converting each of the spoken smooth data into a corresponding smooth sentence vector specifically includes: Using the Encoder in the BERT model to encode each of the spoken complaint data to generate a corresponding spoken sentence vector; The Encoder in the BERT model is used to encode each of the fluent spoken data to generate a corresponding fluent sentence vector.

[0089] In one embodiment, the encoder in the BERT model is used to encode each of the spoken complaint data to generate a corresponding spoken sentence vector, specifically including: The spoken complaint data is segmented according to the word granularity, and each character after segmentation is converted into a one-hot vector , and obtain the word embedding matrix corresponding to each of the spoken complaint data ,include:

[0090] in, Indicates One-hot vector corresponding to the character ; Using the query weight matrix in the BERT model , key weight matrix and the value weight matrix , calculate the word embedding matrix The query vector , key vector and the value vector ,include:

[0091]

[0092]

[0093] The query vector is respectively analyzed using each attention mechanism in the BERT model. , key vector and the value vector Calculate and generate the corresponding attention vector ,include:

[0094]

[0095]

[0096] in, Indicates The attention vector calculated by the attention mechanism; Indicates The query weight matrix in the attention mechanism; Indicates The key weight matrix in the attention mechanism; Indicates The value weight matrix in the attention mechanism; express The dimension of the vector; The attention vectors generated by the attention mechanisms are concatenated and compressed to obtain spoken sentence vectors corresponding to the spoken complaint data, including:

[0097] in, represents the concatenation function; is the additional weight matrix.

[0098] In one embodiment, the training of the student network using the student data set specifically includes: Convert the spoken complaint data in the student data set into spoken sentence vectors, convert the symptoms in the student data set into symptom vectors using One-hot encoding, and use the symptom vectors as labels corresponding to the spoken sentence vectors; Initialize the weight data of the student network, input each transformed student data group as a training sample into the student network, pass through each hidden layer in turn, and obtain the result of the output layer; wherein each hidden layer is a fully connected layer, and each hidden layer is nonlinearly mapped through a ReLU activation function; the output layer is a fully connected layer, and the output layer uses a Sigmoid activation function, and the result output by the output layer is the probability of each symptom; Calculate the loss function between the student network prediction output and the true symptom label, specifically including:

[0099] in, Indicates The samples have The true value of the label; Represents the student network prediction sample Belong to The probability of a label, its value range is [0,1]; is the number of samples used to train the student network, indicates the number of symptom categories; The model optimizer is used to optimize the model weights of the student network according to the loss function to obtain a trained student network.

[0100] In one embodiment, converting the spoken complaint data in the student data set into a spoken sentence vector specifically includes: The Encoder in the BERT model is used to encode each of the spoken complaint data to generate a corresponding spoken sentence vector.

[0101] In one embodiment, the encoder in the BERT model is used to encode each of the spoken complaint data to generate a corresponding spoken sentence vector, specifically including: The spoken complaint data is segmented according to the word granularity, and each character after segmentation is converted into a one-hot vector , and obtain the word embedding matrix corresponding to each of the spoken complaint data ,include:

[0102] in, Indicates One-hot vector corresponding to the character ; Using the query weight matrix in the BERT model , key weight matrix and the value weight matrix , calculate the word embedding matrix The query vector , key vector and the value vector ,include:

[0103]

[0104]

[0105] The query vector is respectively analyzed using each attention mechanism in the BERT model. , key vector and the value vector Calculate and generate the corresponding attention vector ,include:

[0106]

[0107]

[0108] in, Indicates The attention vector calculated by the attention mechanism; Indicates The query weight matrix in the attention mechanism; Indicates The key weight matrix in the attention mechanism; Indicates The value weight matrix in the attention mechanism; express The dimension of the vector; The attention vectors generated by the attention mechanisms are concatenated and compressed to obtain spoken sentence vectors corresponding to the spoken complaint data, including:

[0109] in, represents the concatenation function; is the additional weight matrix.

[0110] In one embodiment, the training of the teacher network using the teacher data set specifically includes: The oral fluent data in the teacher data set are converted into fluent sentence vectors, the symptoms in the teacher data set are converted into symptom vectors using One-hot encoding, and the symptom vectors are used as labels corresponding to the fluent sentence vectors; Initialize the weight data of the teacher network, input each transformed teacher data group as a training sample into the teacher network, pass through each hidden layer in turn, and obtain the result of the output layer; wherein each hidden layer is a fully connected layer, and each hidden layer is nonlinearly mapped through a ReLU activation function; the output layer is a fully connected layer, and the output layer uses a Sigmoid activation function, and the result output by the output layer is the probability of each symptom; Calculate the loss function between the teacher network prediction output and the true symptom label, specifically including:

[0111] in, Indicates The samples have The true value of the label; Represents the teacher network prediction sample Belong to The probability of a label, its value range is [0,1]; is the number of samples used to train the teacher network, indicates the number of symptom categories; A model optimizer is used to optimize the model weights of the teacher network according to the loss function to obtain a trained teacher network.

[0112] In one embodiment, converting the spoken fluent data in the teacher data set into a fluent sentence vector specifically includes: The Encoder in the BERT model is used to encode each of the fluent spoken data to generate a corresponding fluent sentence vector.

[0113] In one embodiment, the method of using the trained teacher network to perform knowledge distillation on the trained student network to obtain a main complaint symptom detection network specifically includes: Input the colloquial complaint data into the trained teacher network to obtain the first prediction result output by the teacher network , the first prediction result The teacher network predicts that the spoken complaint data meets the first The predicted probability of each symptom; The colloquial complaint data is input into the trained student network to obtain a second prediction result output by the student network. , the second prediction result The student network predicts that the spoken complaint data meets the first The predicted probability of each symptom; Using all first prediction results And the second prediction result Calculate the divergence loss between the student network and the teacher network ; Using all first prediction results And the second prediction result Calculate the total distillation loss of the student network ; Using the divergence loss , the loss function between the student network prediction output and the true symptom label and the total distillation loss of the student network , calculate the total loss function of the student network ; The gradient in the loss function is used to update the parameters of the student network to obtain the chief complaint symptom detection network after knowledge distillation.

[0114] In one embodiment, the spoken complaint data is input into the trained teacher network to obtain a first prediction result output by the teacher network. , specifically including:

[0115] in, Represents the predicted value output by the teacher network without activation function processing; Represents the probability after being processed by the Sigmoid activation function.

[0116] In one embodiment, the spoken complaint data is input into the trained student network to obtain a second prediction result output by the student network. , specifically including:

[0117] in, Represents the predicted value output by the student network without activation function processing; Represents the probability distribution after being processed by the Sigmoid activation function.

[0118] In one embodiment, it is characterized in that the use of all first prediction results And the second prediction result Calculate the divergence loss between the student network and the teacher network , specifically including:

[0119] in, Indicates the number of symptom categories.

[0120] In one embodiment, the use of all first prediction results And the second prediction result Calculate the total distillation loss of the student network , specifically including:

[0121] in, represents the overall prediction distribution of the student network for the input data; represents the overall prediction distribution of the teacher network for the input data; Indicates true category label; Represents the control factor used to adjust the classification loss Distillation loss Importance in the training process; represents the classification loss of the student network; represents the distillation loss of the student network.

[0122] In one embodiment, the use of the divergence loss , the loss function between the student network prediction output and the true symptom label and the total distillation loss of the student network , calculate the total loss function of the student network , specifically including: .

[0123] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0124] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for detecting symptoms in oral scenarios based on knowledge distillation, characterized in that: include: Annotate the collected oral complaint data to obtain several student data groups and several teacher data groups; The student data set includes: oral complaint data and symptoms; the teacher data set includes: oral fluency data and symptoms; The student network is trained using the student data set, and the teacher network is trained using the teacher data set; knowledge distillation is performed on the trained student network using the trained teacher network to obtain a chief complaint symptom detection network; wherein both the student network and the teacher network are multi-label classification models; The colloquial chief complaint sentence to be identified is input into the chief complaint symptom detection network for identification to obtain the corresponding symptom prediction result.

2. According to claim 1, a method for detecting symptoms in oral scenarios based on knowledge distillation is characterized in that: The collected oral complaint data are labeled to obtain a number of student data groups and a number of teacher data groups, specifically including: Cleaning the collected oral complaint data, annotating the cleaned oral complaint data, and obtaining oral fluent data and symptoms corresponding to the oral complaint data; Recording the spoken complaint data, the spoken fluency data and the symptoms as a triple data set; Extracting spoken complaint data and symptoms from each of the three data sets to record as student data sets; The oral fluency data and symptoms are extracted from each of the three data sets and recorded as a teacher data set.

3. According to claim 1, a method for detecting symptoms in oral scenarios based on knowledge distillation is characterized in that: The training of the student network using the student data set specifically includes: Convert the spoken complaint data in the student data set into spoken sentence vectors, convert the symptoms in the student data set into symptom vectors using One-hot encoding, and use the symptom vectors as labels corresponding to the spoken sentence vectors; Initialize the weight data of the student network, input each transformed student data group as a training sample into the student network, pass through each hidden layer in turn, and obtain the result of the output layer; wherein each hidden layer is a fully connected layer, and each hidden layer is nonlinearly mapped through a ReLU activation function; the output layer is a fully connected layer, and the output layer uses a Sigmoid activation function, and the result output by the output layer is the probability of each symptom; Calculate the loss function between the student network prediction output and the true symptom label, specifically including: in, Indicates The samples have The true value of the label; Represents the student network prediction sample Belong to The probability of a label, its value range is [0,1]; is the number of samples used to train the student network, indicates the number of symptom categories; The model optimizer is used to optimize the model weights of the student network according to the loss function to obtain a trained student network.

4. The method for detecting symptoms in oral scenarios based on knowledge distillation according to claim 1, characterized in that: The training of the teacher network using the teacher data set specifically includes: The oral fluent data in the teacher data set are converted into fluent sentence vectors, the symptoms in the teacher data set are converted into symptom vectors using One-hot encoding, and the symptom vectors are used as labels corresponding to the fluent sentence vectors; Initialize the weight data of the teacher network, input each transformed teacher data group as a training sample into the teacher network, pass through each hidden layer in turn, and obtain the result of the output layer; wherein each hidden layer is a fully connected layer, and each hidden layer is nonlinearly mapped through a ReLU activation function; the output layer is a fully connected layer, and the output layer uses a Sigmoid activation function, and the result output by the output layer is the probability of each symptom; Calculate the loss function between the teacher network prediction output and the true symptom label, specifically including: in, Indicates The samples have The true value of the label; Represents the teacher network prediction sample Belong to The probability of a label, its value range is [0,1]; is the number of samples used to train the teacher network, indicates the number of symptom categories; A model optimizer is used to optimize the model weights of the teacher network according to the loss function to obtain a trained teacher network.

5. The method for detecting symptoms in oral scenarios based on knowledge distillation according to claim 1, characterized in that: The method uses the trained teacher network to perform knowledge distillation on the trained student network to obtain a main complaint symptom detection network, specifically including: Input the colloquial complaint data into the trained teacher network to obtain the first prediction result output by the teacher network , the first prediction result The teacher network predicts that the spoken complaint data meets the first The predicted probability of each symptom; The colloquial complaint data is input into the trained student network to obtain a second prediction result output by the student network. , the second prediction result The student network predicts that the spoken complaint data meets the first The predicted probability of each symptom; Using all first prediction results And the second prediction result Calculate the divergence loss between the student network and the teacher network ; Using all first prediction results And the second prediction result Calculate the total distillation loss of the student network ; Using the divergence loss , the loss function between the student network prediction output and the true symptom label and the total distillation loss of the student network , calculate the total loss function of the student network ; The gradient in the loss function is used to update the parameters of the student network to obtain the chief complaint symptom detection network after knowledge distillation.

6. The method for detecting symptoms in oral scenarios based on knowledge distillation according to claim 5, characterized in that: The use of all first prediction results And the second prediction result Calculate the divergence loss between the student network and the teacher network , including: in, indicates the number of symptom categories; The use of all first prediction results And the second prediction result Calculate the total distillation loss of the student network , including: in, represents the overall prediction distribution of the student network for the input data; represents the overall prediction distribution of the teacher network for the input data; Indicates true category label; Represents the control factor used to adjust the classification loss and distillation losses Importance in the training process; represents the classification loss of the student network; represents the distillation loss of the student network.

7. The method for detecting symptoms in oral scenarios based on knowledge distillation according to claim 6, characterized in that: The use of the divergence loss , the loss function between the student network prediction output and the true symptom label and the total distillation loss of the student network , calculate the total loss function of the student network , including: .

8. A device for detecting symptoms in oral scenarios based on knowledge distillation, characterized in that: The oral scene main complaint symptom detection device based on knowledge distillation includes: A labeling unit is used to label the collected oral complaint data to obtain a plurality of student data groups and a plurality of teacher data groups; wherein the student data groups all include: oral complaint data and symptoms; and the teacher data groups all include: oral fluency data and symptoms; A training unit, used to train a student network using the student data set, and to train a teacher network using the teacher data set; and to perform knowledge distillation on the trained student network using the trained teacher network to obtain a main complaint symptom detection network; The prediction unit is used to input the colloquial chief complaint sentence to be identified into the chief complaint symptom detection network for identification, and obtain the corresponding symptom prediction result.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the method for detecting symptoms in oral scenarios based on knowledge distillation as described in any one of claims 1 to 7.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the method for detecting symptoms in oral scenarios based on knowledge distillation as described in any one of claims 1 to 7.

Images