Method and system for analyzing medical evaluation through knowledge distillation adversarial network

Abstract

The invention relates to a method and system for analyzing medical evaluation through a knowledge distillation adversarial network, and the method comprises the steps: inputting medical evaluation data after emotion labeling into an improved generative adversarial network obtained through the combination of the generative adversarial network and an MASH module of a transformer model, so as to generate a training sample; a teacher model is initialized through the training sample, and knowledge learned in the initialized teacher model is extracted through knowledge distillation to a first student model obtained by combining a GRU model, a BERT model and a first convolutional layer and a second student model obtained by combining an LSTM network, the BERT model and a second convolutional layer; and training the first student model and the second student model by taking the minimum extraction loss as a target, and obtaining an analysis result of the medical evaluation data based on the trained first student model and second student model. Therefore, under the condition that the requirements for a large amount of high-quality data and a large amount of time are reduced, analysis of the medical evaluation data is achieved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and system for analyzing medical evaluation through an adversarial network of knowledge distillation. Background Art

[0002] Medical evaluations of medical institutions, such as treatment quality, facility evaluation, and patient experience, help patients make wise choices when choosing medical institutions, and also help medical institutions improve their own medical services. AI-driven systems can quickly collect medical evaluations, then use NLP to analyze, extract the attitudes of medical evaluations, and classify treatment quality, facility evaluation, and patient experience based on attitudes. Among them, sentiment classification is an important problem in NLP, which attempts to classify text into sentiment levels, which is crucial for evaluating medical evaluations in various situations. However, sentiment classification currently relies heavily on huge language models, requires a large amount of high-quality data and takes a lot of time. It is difficult to implement under the constraints of the real world, which affects the development of medical services. Summary of the invention

[0003] The technical problem to be solved by the present invention is: the present invention provides a method and system for analyzing medical evaluation through an adversarial network of knowledge distillation, which realizes the analysis of medical evaluation data while reducing the demand for a large amount of high-quality data and a large amount of time.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is: In a first aspect, the present invention provides a method for analyzing medical evaluations through an adversarial network based on knowledge distillation, comprising: Acquire medical evaluation data, and perform emotion annotation on the medical evaluation data to obtain emotion-annotated medical evaluation data, wherein the emotion annotation includes positive emotion, negative emotion, and mixed emotion; The generative adversarial network is combined with the MASH module of the transformer model to obtain an improved generative adversarial network, the medical evaluation data with emotion annotation is input into the improved generative adversarial network to generate training samples, and the training samples are input into the teacher model for initialization to obtain an initialized teacher model; The GRU model, the first BERT model and the first convolutional layer are combined to obtain the first student model, and the LSTM network, the second BERT model and the second convolutional layer are combined to obtain the second student model. The knowledge learned in the initialized teacher model is refined into the first student model and the second student model through knowledge distillation, and the first student model and the second student model are trained with the goal of minimizing the refinement loss to obtain the trained first student model and the trained second student model. The analysis results of the medical evaluation data are obtained based on the trained first student model and the trained second student model.

[0005] The beneficial effects of the present invention are as follows: training samples are generated by generative adversarial networks without relying on a large amount of high-quality data in the real world, and the generative adversarial networks are combined with the MASH module of the transformer model, so that the improved generative adversarial network can more comprehensively and accurately capture the dependencies existing in the input emotion-labeled medical evaluation data, so as to improve the authenticity of the generated training samples and improve the quality of the training samples. The knowledge learned in the teacher model is refined into the first student model and the second student model through knowledge distillation, and the first student model and the second student model are trained with the minimum extraction loss as the goal. The first student model and the second student model integrate the advantages of the GRU model, the BERT model and the LSTM network, and improve the accuracy and robustness of the trained first student model and the second student model. In addition, the compression and acceleration technology of knowledge distillation is used to enable the trained first student model and the second student model to realize the analysis of medical evaluation while reducing the consumption of computing resources and the reasoning time, thereby improving the analysis efficiency of medical evaluation and promoting the development of medical services.

[0006] Optionally, the obtaining of medical evaluation data includes: Build a web search engine using Python's Selenium module and Chromium driver; Generate a URL corresponding to a first page for each medical institution in a preset list of medical institutions by using the network search engine, and automatically navigate to the corresponding first page based on the URL using the Selenium module to obtain medical evaluation data, wherein the first page includes a map page, a social network page, and a medical institution website page; Parsing the medical evaluation data by using Beautiful Soup to obtain parsed medical evaluation data, wherein the parsed medical evaluation data includes a medical evaluation text, a medical score, and a timestamp; The parsed medical evaluation data is format-converted to obtain format-converted medical evaluation data, wherein the format conversion includes CSV format conversion and JSON format conversion.

[0007] According to the above description, a URL corresponding to the first page is generated for each medical institution through a web search engine. Based on the URL, the Selenium module of the web search engine can automatically perform navigation to obtain medical evaluation data without manual acquisition, thereby improving the efficiency of medical data acquisition. The first page includes a map page, a social network page, and a medical institution website page, thereby improving the integrity and comprehensiveness of the acquired medical evaluation data. The acquired medical evaluation data will be parsed and format converted to ensure the availability of the final medical evaluation data.

[0008] Optionally, the improved generative adversarial network includes an improved generator and an improved discriminator, and the improved generative adversarial network is obtained by combining the generative adversarial network with the MASH module of the transformer model, including: Construct an improved generator, combining the generator of the generative adversarial network with the MHSA module of the transformer model, introducing the MHSA module as the initial layer to apply linear projection to modify the dimensional input tensor to adapt to the actual number of attention heads, and the MHSA module will calculate the proportional dot product on all the attention heads to generate the attention weights applied to the corresponding values ​​of the tensor, and all the attention weights will be processed in series and linearly to obtain the final attention output; Combining the MHSA module with an additional layer of spectral normalization linear transformation, Leaky ReLU activation function and Dropout filtering regularization, and adding an attention layer in the end block to obtain an improved generator; An improved discriminator is constructed by combining each linear layer in the discriminator of the generative adversarial network with an additional layer of spectral normalization linear transformation to obtain a combined linear layer, and adding residual connections between each combined linear layer. An arbitrary combined linear layer is selected and combined with a Softmax activation function as the output layer to obtain an improved discriminator.

[0009] According to the above description, the improved generative adversarial network not only improves the generator but also improves the discriminator. When improving the generator, the MHSA module is introduced as the initial layer, and the MHSA module is used to generate attention weights to improve the generator's potential to capture subtle dependencies in input noise. The MHSA module is combined with an additional layer of spectral normalization linear transformation, Leaky ReLU activation function and Dropout filtering regularization to improve the generator's ability to capture subtle features and gradients within the latent space, thereby improving the authenticity, consistency and diversity of the generated training samples. When improving the discriminator, the linear layer is combined with an additional layer of spectral normalization linear transformation to stabilize the training process while improving convergence. Residual connections are added between the combined linear layers to help the gradient move in the network and reduce the problem of gradient disappearance in the generative adversarial network. The combined linear layer is combined with the Softmax activation function as the output layer to improve the discriminator's ability to distinguish between real medical evaluation data and generated medical evaluation data.

[0010] Optionally, inputting the emotion-labeled medical evaluation data into the improved generative adversarial network to generate training samples comprises: The medical evaluation data after emotion annotation is preprocessed to obtain preprocessed medical evaluation data, and the preprocessed medical evaluation data is input into an improved generative adversarial network to generate training samples, wherein the preprocessing includes: removing invalid text, word form restoration and text vectorization.

[0011] According to the above description, the medical evaluation data input into the improved generative adversarial network is preprocessed including removing invalid text, word restoration and text vectorization. Removing invalid text can reduce the noise in the medical evaluation data and improve the quality of the medical evaluation data. Word form restoration improves the standardization of the medical evaluation data. Text vectorization makes the medical evaluation data more suitable for the improved generative adversarial network and improves the availability of the medical evaluation data.

[0012] Optionally, the step of refining the knowledge learned from the initialized teacher model to the first student model and the second student model respectively through knowledge distillation, training the first student model and the second student model with a goal of minimizing the refining loss, and obtaining the trained first student model and the trained second student model includes: Introducing a first maximum pooling layer into the first student model, and using the first maximum pooling layer and the first convolutional layer as a first feature extractor, and capturing a first feature state of the knowledge through the first feature extractor; Introducing a second maximum pooling layer into the second student model, and using the second maximum pooling layer and the second convolutional layer as a second feature extractor, and capturing a second feature state of the knowledge through the second feature extractor; Based on the first feature state and the second feature state, the first student model and the second student model are trained with the minimum refinement loss as the goal to obtain a trained first student model and a trained second student model.

[0013] According to the above description, the first maximum pooling layer and the second maximum pooling layer are respectively introduced into the first student model and the second student model, and they are used together with the corresponding convolutional layers as feature extractors, thereby improving the ability of the first student model and the second student model to capture local patterns and features, that is, ensuring the accuracy of the captured first feature states and second feature states, thereby improving the analysis capabilities of the first student model and the second student model trained based on the first feature states and the second feature states.

[0014] Optionally, the training the first student model and the second student model based on the first feature state and the second feature state with a minimum extraction loss as a goal to obtain the trained first student model and the trained second student model includes: The first student model is embedded by the first BERT model output t , according to the embedding e t , the first feature state and the recursive update equation of the GRU model calculate the first current hidden state of the knowledge, and the recursive update equation is:

[0015] Among them, h t represents the first current hidden state at position t, h t-1 represents the first characteristic state at position t-1, e t represents the vector generated by the first BERT model after processing the input text at position t; The first update gate is calculated by the first update gate mechanism formula, wherein the first update gate mechanism formula is:

[0016] Among them, z t represents the first update gate at position t, represents the sigmoid activation function, W z represents the weight matrix associated with the first update gate, b z represents the bias vector associated with the first update gate, h t-1 represents the first characteristic state at position t-1, et represents the vector generated by the first BERT model after processing the input text at position t; When the first update gate is lower than the update threshold, the first current hidden state is retained, otherwise, the reset gate is calculated by the reset gate mechanism formula, and the reset gate mechanism formula is:

[0017] Among them, r t represents the reset gate at position t, W r represents the weight matrix associated with the reset gate, b r represents the bias vector associated with the reset gate, h t-1 represents the first characteristic state at position t-1, e t represents the vector generated by the first BERT model after processing the input text at position t; The candidate gate is calculated according to the reset gate and the candidate gate mechanism formula, and the candidate gate mechanism formula is:

[0018] in, represents the candidate gate at position t, tanh represents the hyperbolic tangent function, W h represents the weight matrix associated with the candidate gate, b h represents the bias vector associated with the candidate gate, r t represents the reset gate at position t, h t-1 represents the first characteristic state at position t-1, e t represents the vector generated by the first BERT model after processing the input text at position t; The first current hidden state of the knowledge is recalculated according to the candidate gate, the reset gate and the first hidden formula to obtain the latest first current hidden state, wherein the first hidden formula is:

[0019] Among them, H t represents the latest first current hidden state at position t, z t represents the first update gate at position t, h t-1 represents the first characteristic state at position t-1, represents the candidate gate at position t; The second student model outputs embeddings through the second BERT model , according to the embedding , the input gate of the knowledge is calculated according to the second feature state and the input gate mechanism formula of the LSTM network, the forget gate of the knowledge is calculated according to the second feature state and the forget mechanism formula of the LSTM, and the second update gate of the knowledge is calculated according to the second feature state and the second update gate mechanism formula of the LSTM, and the input gate mechanism formula is:

[0020] in, represents the input gate at position t, represents the sigmoid activation function, W i represents the weight matrix associated with the input gate, b i represents the bias vector associated with the input gate, represents the second characteristic state at position t-1, Represents the vector generated by the second BERT model after processing the input text at position t

[0021] The forgetting mechanism formula is:

[0022] in, represents the forget gate at position t, represents the sigmoid activation function, W f represents the weight matrix associated with the forget gate, b f represents the bias vector associated with the forget gate, represents the second characteristic state at position t-1, Represents the vector generated by the second BERT model after processing the input text at position t

[0023] The second update gate mechanism formula is:

[0024] in, represents the second update gate at position t, tanh represents the hyperbolic tangent function, W g represents the weight matrix associated with the second update gate, b g represents the bias vector associated with the second update gate, represents the second characteristic state at position t-1, Represents the vector generated by the second BERT model after processing the input text at position t

[0025] The candidate storage unit is calculated according to the second update gate, the input gate and the candidate storage unit formula, and the candidate storage unit formula is:

[0026] in, represents the candidate storage unit at position t, represents the second update gate at position t, represents the input gate at position t; The latest storage unit is calculated according to the candidate storage unit, the forget gate and the storage update formula, and the storage update formula is:

[0027] in, represents the latest storage unit at position t, represents the forget gate at position t, represents the candidate storage unit at position t, represents the latest storage unit at position t-1; The output gate of the knowledge is calculated according to the second feature state and the output gate mechanism formula of LSTM, and the output gate mechanism formula is:

[0028] in, represents the output gate at position t, represents the sigmoid activation function, W o represents the weight matrix associated with the output gate, b o represents the bias vector associated with the output gate, represents the second characteristic state at position t-1, Represents the vector generated by the second BERT model after processing the input text at position t

[0029] The second current hidden state of the knowledge is calculated according to the output gate, the latest storage unit and the second hidden formula, where the second hidden formula is:

[0030] in, represents the second current hidden state at position t, represents the output gate at position t, represents the latest storage unit at position t, tanh represents the hyperbolic tangent function; The first current hidden state and the second current hidden state are input into a dense layer for classification to obtain a predicted probability distribution for each category, a refinement loss is calculated according to the predicted probability distribution, and the first student model and the second student model are trained with the minimum refinement loss as the goal to obtain a trained first student model and a trained second student model.

[0031] According to the above description, the first student model calculates the first current hidden state of knowledge using the input embedding e of the first BERT model and the recursive update equation of the GRU model, thereby improving the ability of the first student model to capture the potential time dependency of the input knowledge, and according to the first update gate, decides the balance between retaining the previous first current hidden state and incorporating the candidate gate to recalculate the first current hidden state, so that the first student model can adaptively update the first current hidden state representation according to the input embedding e and the previous context, thereby improving the analysis ability of the first student model, and the second student model updates the storage unit according to the forgetting gate of the forgetting mechanism to obtain the latest storage unit, and the output gate can regulate the information flow from the latest storage unit to the second current hidden state. The second current hidden state is calculated based on the output gate activation and the latest storage unit, which improves the second student model's ability to capture the context and sequential information contributed by LSTM in the second BERT model, thereby improving the analysis ability of the second student model.

[0032] Optionally, inputting the first current hidden state and the second current hidden state into a dense layer for classification includes: The first current hidden state and the second current hidden state are input into a dense layer for classification to obtain a prediction score for each category, and a Softmax activation function is used to convert the prediction score into a prediction probability distribution.

[0033] According to the above description, the linear transformation of the dense layer can convert the first current hidden state and the second current hidden state into the same dimension as the number of categories, so as to ensure the accuracy of the predicted score of each category obtained, and use the Softmax activation function to convert the predicted score into a predicted probability distribution to ensure the accuracy of the predicted probability distribution obtained.

[0034] Optionally, the calculating the refinement loss according to the predicted probability distribution, training the first student model and the second student model with the minimum refinement loss as the goal, and obtaining the trained first student model and the trained second student model includes: The sparse classification cross entropy is used to calculate the refinement loss of each training sample according to the predicted probability distribution, and the total refinement loss is calculated according to the refinement loss and the total refinement loss formula. The total refinement loss formula is:

[0035] in, represents the total refining loss, represents the extraction loss of the βth training sample, and N represents the total number of training samples; The first student model and the second student model are trained using the Adam algorithm with the goal of minimizing the total refinement loss to obtain a trained first student model and a trained second student model.

[0036] According to the above description, the use of sparse classification cross entropy to calculate the refinement loss can effectively deal with the category imbalance problem, optimize the prediction probability distribution of the student model, take the average refinement loss of all training samples as the total refinement loss, and train with the minimum total refinement loss as the goal to ensure the rationality of the trained first student model and the trained second student model. When training with a small total refinement loss as the goal, the Adam algorithm is used, so that the advantages of momentum and adaptive learning rate can be combined during training, and the training process can converge quickly and avoid falling into the local optimum, thereby improving the analytical ability of the trained first student model and the trained second student model.

[0037] In a second aspect, the present invention provides a system for analyzing medical evaluations through an adversarial network using knowledge distillation, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for analyzing medical evaluations through an adversarial network using knowledge distillation described in the first aspect is implemented.

[0038] Among them, the technical effect corresponding to the system for analyzing medical evaluations through an adversarial network of knowledge distillation provided in the second aspect refers to the relevant description of the method for analyzing medical evaluations through an adversarial network of knowledge distillation provided in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flowchart of a method for analyzing medical evaluations through an adversarial network using knowledge distillation provided in this embodiment; Figure 2 A schematic diagram of the overall process of a method for analyzing medical evaluation through an adversarial network using knowledge distillation provided in this embodiment; Figure 3 A schematic diagram of the process of training the first student model and the second student model involved in this embodiment; Figure 4 A schematic diagram of the structure of a system for analyzing medical evaluations through an adversarial network using knowledge distillation provided in this embodiment.

[0040] Description of Reference Numerals 1. A system for analyzing medical evaluations through adversarial networks using knowledge distillation; 2. Processor; 3. Memory. DETAILED DESCRIPTION

[0041] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0042] Embodiment 1 Please refer to Figures 1 to 3 The present invention provides a method for analyzing medical evaluation through an adversarial network using knowledge distillation, comprising the steps of: S1. Acquire medical evaluation data, and perform emotion annotation on the medical evaluation data to obtain emotion-annotated medical evaluation data, wherein the emotion annotation includes positive emotion, negative emotion, and mixed emotion; In this embodiment, if Figure 2 As shown, the acquired medical evaluation data are subjected to emotion annotation including positive emotion, negative emotion and mixed emotion. Mixed emotion refers to medical evaluation data carrying both positive emotion and negative emotion, and medical evaluation data with neutral attitude. After emotion annotation, the number of same emotion annotations is counted, that is, the number of positive emotion data, the number of negative emotion data and the number of mixed emotion data are obtained. When any number of emotion data is lower than the emotion threshold, an oversampling strategy is used to increase the number of emotion data corresponding to the value lower than the emotion threshold. For example, if the emotion threshold is 50 and the number of mixed emotion data is 35, an oversampling strategy is used to increase the number of mixed emotion data so that it is not lower than the emotion threshold. The specific emotion threshold is set according to the actual situation.

[0043] At this time, the acquisition of medical evaluation data in step S1 includes: S11. Build a web search engine using Python's Selenium module and Chromium driver. S12, generating a URL corresponding to a first page for each medical institution in a preset list of medical institutions through the network search engine, and automatically navigating to the corresponding first page to obtain medical evaluation data based on the URL using the Selenium module, wherein the first page includes a map page, a social network page, and a medical institution website page; S13, parsing the medical evaluation data by using Beautiful Soup to obtain parsed medical evaluation data, where the parsed medical evaluation data includes a medical evaluation text, a medical score, and a timestamp; S14. Perform format conversion on the parsed medical evaluation data to obtain format-converted medical evaluation data, wherein the format conversion includes CSV format conversion and JSON format conversion.

[0044] In this embodiment, if Figure 2 As shown, a web searcher is constructed through Python's Selenium module and Chromium driver, so that a URL corresponding to the first page of each medical institution in the preset list of medical institutions is generated through the web searcher. Based on the URL, the Selenium module of the web searcher can automatically navigate to the corresponding first page to obtain medical evaluation data therefrom. The first page includes a map page, a social network page and a medical institution website page, that is, the obtained medical evaluation data comes from the map page, the social network page and the medical institution website page, wherein the preset medical institution list can be a pre-built and stored medical institution list or a currently customized medical institution list. Beautiful Soup is a parsing library based on HTML DOM. Therefore, the obtained medical evaluation data is parsed through BeautifulSoup to obtain medical evaluation data including medical evaluation text, medical score and timestamp, and the medical evaluation data is converted into CSV format and JSON format.

[0045] S2. Combining the generative adversarial network with the MASH module of the transformer model to obtain an improved generative adversarial network, inputting the emotion-labeled medical evaluation data into the improved generative adversarial network to generate training samples, and inputting the training samples into the teacher model for initialization to obtain an initialized teacher model; In this embodiment, if Figure 2As shown in the figure, the generative adversarial network is improved by combining the generative adversarial network with the MASH module of the transformer model. The MASH module refers to the Multi-Head Attention module to obtain an improved generative adversarial network, so that the emotion-labeled medical evaluation data obtained in step S1 is input into the improved generative adversarial network to generate training samples, and the training samples are input into the teacher model for initialization. At this time, the teacher model is a large and powerful neural network model, such as: DistilBERT (knowledge distillation BERT model), XLNet (Exploring the Limits of Pre-trained Language Models, natural language processing model), ERNIE3.0 (Enhanced Representation through kNowledge IntEgration 3.0, Wenxin 3.0), ALBERT (lightweight BERT model), m-BERT (Multilingual BERT, dual-language training model), Indic-BERT (Indian language optimized BERT model variant) and XLM-RoBERTa (Cross-Lingual Language Model RoBERTa, multi-language pre-training model), so as to obtain the initialized teacher model.

[0046] At this time, the improved generative adversarial network in step S2 includes an improved generator and an improved discriminator, and the improved generative adversarial network is combined with the MASH module of the transformer model to obtain the improved generative adversarial network including: S21. Construct an improved generator, combining the generator of the generative adversarial network with the MHSA module of the transformer model, introducing the MHSA module as the initial layer to apply linear projection to modify the dimensional input tensor to adapt to the actual number of attention heads, and the MHSA module will calculate the proportional dot product on all the attention heads to generate attention weights applied to the corresponding values ​​of the tensor, and all the attention weights will be processed in series and linearly to obtain the final attention output; S22, combining the MHSA module with an additional layer of spectral normalization linear transformation, Leaky ReLU activation function and Dropout filtering regularization, and adding an attention layer in the end block to obtain an improved generator; In this embodiment, if Figure 2As shown, the improved generative adversarial network includes an improved generator and an improved discriminator, wherein the improved generator is obtained by combining the original generator of the generative adversarial network with the MHSA module of the transformer model, and introducing the MHSA module as the initial layer, which can continuously focus on the potential of subtle dependencies in the input noise. The MHSA module applies linear projection to modify the dimensional input tensor to adapt to the actual number of attention heads. Specifically, the MHSA module applies linear projection Q, K, V to modify B The input tensor X of dimension, where Q represents the Query vector, K represents the Key vector, V represents the Value vector, B represents the Batch Size, i.e., the number of samples in a batch of input data, L represents the Length or Sequence Length, i.e., the length of the sequence data, and C represents the Channels or Features, i.e., the number of features for each time step or each word. The input tensor X represents a shape representation method, and the MHSA module calculates the proportional dot product on all the attention heads to generate the attention weights applied to the corresponding values ​​of the tensor, i.e., the attention weights of V(X). All the attention weights are processed in series and linearly to obtain the final attention output, which can be represented by Y. The MHSA module is combined with an additional layer of spectral normalization linear transformation, a Leaky ReLU activation function, and a Dropout filtering regularization. At this time, the additional layer of spectral normalization linear transformation selects a spectral normalization linear transformation with a shape of (512,256), and the Leaky The ReLU activation function actually introduces a small negative slope to the negative input area to limit the complete suppression of information, thereby improving the generator's ability to capture subtle features and gradients within the latent space, and adding an attention layer to the ending block to obtain an improved generator. That is, the improved generator adds two stages: 1 is the initial layer: MHSA module, and 2 is the ending block: attention layer.

[0047] S23. Construct an improved discriminator, combine each linear layer in the discriminator of the generative adversarial network with an additional layer of spectral normalization linear transformation to obtain a combined linear layer, add residual connections between each combined linear layer, arbitrarily select a combined linear layer and a Softmax activation function to combine as the output layer to obtain an improved discriminator.

[0048] In this embodiment, if Figure 2As shown in the figure, the original discriminator of the generative adversarial network is composed of multiple layers, such as convolutional layers, fully connected layers and linear layers. Each layer is followed by a Leaky ReLU activation function and Dropout filtering regularization. The improved discriminator combines each linear layer with an additional layer of spectral normalization linear transformation on the basis of the original discriminator, and adds residual connections between each combined linear layer except the last layer, and selects any combined linear layer and a Softmax activation function to combine as the output layer. The output layer mainly generates logits, that is, the numerical value of the probability distribution between the actual category and the false category, so as to obtain the improved discriminator, in which the Leaky ReLU activation function after each layer is a Leaky ReLU activation function with a slope of 0.2.

[0049] At this time, the step S2 of inputting the emotion-labeled medical evaluation data into the improved generative adversarial network to generate training samples includes: S24. Preprocessing the emotion-labeled medical evaluation data to obtain preprocessed medical evaluation data, and inputting the preprocessed medical evaluation data into an improved generative adversarial network to generate training samples, wherein the preprocessing includes: removing invalid text, word form restoration, and text vectorization.

[0050] In this embodiment, the medical evaluation data after emotion annotation is preprocessed including removing invalid text, word form restoration and text vectorization to obtain preprocessed medical evaluation data, wherein removing invalid text is to delete all forms of emoticons, punctuation marks and repeated data, word form restoration is to normalize the text by retaining the original form of word units, and text vectorization is to convert the medical evaluation data into numerical representation.

[0051] S3. Combine the GRU model, the first BERT model and the first convolutional layer to obtain the first student model, combine the LSTM network, the second BERT model and the second convolutional layer to obtain the second student model, and use knowledge distillation to refine the knowledge learned in the initialized teacher model to the first student model and the second student model respectively, train the first student model and the second student model with the goal of minimizing the refinement loss to obtain the trained first student model and the trained second student model, and obtain the analysis results of the medical evaluation data based on the trained first student model and the trained second student model.

[0052] In this embodiment, if Figure 2As shown, unlike the traditional method of directly using a small, lightweight neural network as a student model, the present application combines two improved student models, wherein the first student model is formed by combining a GRU model, a first BERT model and a first convolutional layer, and the second student model is formed by combining an LSTM network, a second BERT model and a second convolutional layer. The knowledge learned in the initialized teacher model is respectively refined into the first student model and the second student model through knowledge distillation, and the first student model and the second student model are trained with the goal of minimizing the refinement loss, and the analysis results of the medical evaluation data are obtained based on the trained first student model and the trained second student model.

[0053] At this time, in step S3, the knowledge learned in the initialized teacher model is respectively refined into the first student model and the second student model through knowledge distillation, and the first student model and the second student model are trained with the minimum extraction loss as the goal, and the trained first student model and the trained second student model are obtained, including: S31, introducing a first maximum pooling layer into the first student model, and using the first maximum pooling layer and the first convolutional layer as a first feature extractor, and capturing a first feature state of the knowledge through the first feature extractor; S32, introducing a second maximum pooling layer into the second student model, and using the second maximum pooling layer and the second convolutional layer as a second feature extractor, and capturing a second feature state of the knowledge through the second feature extractor; S33. Based on the first feature state and the second feature state, the first student model and the second student model are trained with the minimum refining loss as the goal to obtain a trained first student model and a trained second student model.

[0054] In this embodiment, if Figure 2 As shown, the first maximum pooling layer and the second maximum pooling layer are introduced into the first student model and the second student model respectively, and the first maximum pooling layer and the first convolution layer are used together as the first feature extractor of the first student model, and the second maximum pooling layer and the second convolution layer are used together as the second feature extractor of the second student model. The first feature state of the knowledge extracted from the teacher model is captured by the first feature extractor, and the second feature state of the knowledge extracted from the teacher model is captured by the second feature extractor. When the first feature state is captured by the first convolution layer or the second feature state is captured by the second convolution layer, the captured first feature state or the second feature state can be expressed by a convolution feature formula, and the convolution feature formula is as follows:

[0055] Among them, Cαt represents the feature state output by the convolutional layer for the αth filter at position t, w αj represents the weight matrix associated with the αth filter, represents the value of the refined knowledge at the αth filter j-1 position, k represents the number of all positions, θ α represents the bias vector associated with the αth filter; The first maximum pooling layer and the second maximum pooling layer are used to downsample the first feature state and the second feature state. Specifically, the first maximum pooling layer and the second maximum pooling layer select the maximum activation in each window with a specified pool size of 2, thereby reducing the dimensions of the first feature state and the second feature state while retaining important information. The first feature state and the second feature state after the first maximum pooling layer and the second maximum pooling layer can be expressed by the pooling feature formula, which is:

[0056] in, represents the feature state output by the maximum pooling layer for the αth filter at position t, C α2t represents the feature state output by the αth filter at position t in a window with a pooling size of 2, C α2t+1 Represents the feature state output by the αth filter at position t+1 in a window with a pooling size of 2; Based on the finally obtained first characteristic state and second characteristic state, the first student model and the second student model are trained with the minimum refinement loss as the goal to obtain the trained first student model and the trained second student model.

[0057] In this embodiment, step S33 includes training the first student model and training the second student model, wherein the specific steps of training the first student model are as follows: The first student model is embedded by the first BERT model output t , according to the embedding e t , the first feature state and the recursive update equation of the GRU model calculate the first current hidden state of the knowledge, and the recursive update equation is:

[0058] Among them, h t represents the first current hidden state at position t, h t-1 represents the first characteristic state at position t-1, e t represents the vector generated by the first BERT model after processing the input text at position t; The first update gate is calculated by the first update gate mechanism formula, wherein the first update gate mechanism formula is:

[0059] Among them, z t represents the first update gate at position t, represents the sigmoid activation function, W z represents the weight matrix associated with the first update gate, b z represents the bias vector associated with the first update gate, h t-1 represents the first characteristic state at position t-1, e t represents the vector generated by the first BERT model after processing the input text at position t; When the first update gate is lower than the update threshold, the first current hidden state is retained, otherwise, the reset gate is calculated by the reset gate mechanism formula, and the reset gate mechanism formula is:

[0060] Among them, r t represents the reset gate at position t, W r represents the weight matrix associated with the reset gate, b r represents the bias vector associated with the reset gate, h t-1 represents the first characteristic state at position t-1, e t represents the vector generated by the first BERT model after processing the input text at position t; The candidate gate is calculated according to the reset gate and the candidate gate mechanism formula, and the candidate gate mechanism formula is:

[0061] in, represents the candidate gate at position t, tanh represents the hyperbolic tangent function, W h represents the weight matrix associated with the candidate gate, b h represents the bias vector associated with the candidate gate, r t represents the reset gate at position t, h t-1 represents the first characteristic state at position t-1, e t represents the vector generated by the first BERT model after processing the input text at position t; The first current hidden state of the knowledge is recalculated according to the candidate gate, the reset gate and the first hidden formula to obtain the latest first current hidden state, wherein the first hidden formula is:

[0062] Among them, H trepresents the latest first current hidden state at position t, z t represents the first update gate at position t, h t-1 represents the first characteristic state at position t-1, represents the candidate gate at position t; In this embodiment, if Figure 3 As shown, when the first student model is trained to calculate the first current hidden state, not only the first BERT model output embedding e is used t , and also utilizes the recursive update equation of the GRU model, and introduces a series of mathematical operations of the GRU model in the process of integrating the GRU model into the first BERT model, such as: the first update gate mechanism formula, the reset gate mechanism formula, the candidate gate mechanism formula and the first hidden formula, and determines whether to retain the previous first current hidden state or recalculate the latest first current hidden state according to the calculated first update gate, so that the first student model can embed e according to the input t and the previous context to adaptively update the hidden state representation.

[0063] Among them, the specific steps for training the second student model are as follows: The second student model outputs embeddings through the second BERT model , according to the embedding , the input gate of the knowledge is calculated according to the second feature state and the input gate mechanism formula of the LSTM network, the forget gate of the knowledge is calculated according to the second feature state and the forget mechanism formula of the LSTM, and the second update gate of the knowledge is calculated according to the second feature state and the second update gate mechanism formula of the LSTM, and the input gate mechanism formula is:

[0064] in, represents the input gate at position t, represents the sigmoid activation function, W i represents the weight matrix associated with the input gate, b i represents the bias vector associated with the input gate, represents the second characteristic state at position t-1, Represents the vector generated by the second BERT model after processing the input text at position t

[0065] The forgetting mechanism formula is:

[0066] in, represents the forget gate at position t, represents the sigmoid activation function, W f represents the weight matrix associated with the forget gate, b f represents the bias vector associated with the forget gate, represents the second characteristic state at position t-1, Represents the vector generated by the second BERT model after processing the input text at position t

[0067] The second update gate mechanism formula is:

[0068] in, represents the second update gate at position t, tanh represents the hyperbolic tangent function, W g represents the weight matrix associated with the second update gate, b g represents the bias vector associated with the second update gate, represents the second characteristic state at position t-1, Represents the vector generated by the second BERT model after processing the input text at position t

[0069] The candidate storage unit is calculated according to the second update gate, the input gate and the candidate storage unit formula, and the candidate storage unit formula is:

[0070] in, represents the candidate storage unit at position t, represents the second update gate at position t, represents the input gate at position t; The latest storage unit is calculated according to the candidate storage unit, the forget gate and the storage update formula, and the storage update formula is:

[0071] in, represents the latest storage unit at position t, represents the forget gate at position t, represents the candidate storage unit at position t, represents the latest storage unit at position t-1; The output gate of the knowledge is calculated according to the second feature state and the output gate mechanism formula of LSTM, and the output gate mechanism formula is:

[0072] in, represents the output gate at position t, represents the sigmoid activation function, W o represents the weight matrix associated with the output gate, b o represents the bias vector associated with the output gate, represents the second characteristic state at position t-1, Represents the vector generated by the second BERT model after processing the input text at position t

[0073] The second current hidden state of the knowledge is calculated according to the output gate, the latest storage unit and the second hidden formula, where the second hidden formula is:

[0074] in, represents the second current hidden state at position t, represents the output gate at position t, represents the latest storage unit at position t, tanh represents the hyperbolic tangent function; In this embodiment, if Figure 3 As shown, when the second student model is trained to calculate the second current hidden state, not only the output embedding of the second BERT model is used , and also utilized the forgetting update mechanism of the LSTM network. In the process of integrating the LSTM network into the second BERT model, a series of mathematical operations of the LSTM network were introduced, such as: input gate mechanism formula, forgetting mechanism formula, second update gate mechanism formula, candidate storage unit formula, storage update formula, output gate mechanism formula and second hidden formula. The output gate can regulate the information flow from the storage unit to the second hidden state.

[0075] The first current hidden state and the second current hidden state are input into a dense layer for classification to obtain a predicted probability distribution for each category, a refinement loss is calculated according to the predicted probability distribution, and the first student model and the second student model are trained with the minimum refinement loss as the goal to obtain a trained first student model and a trained second student model.

[0076] At this time, the inputting the first current hidden state and the second current hidden state into a dense layer for classification includes: The first current hidden state and the second current hidden state are input into a dense layer for classification to obtain a prediction score for each category, and a Softmax activation function is used to convert the prediction score into a prediction probability distribution.

[0077] In this embodiment, if Figure 3As shown, the first current hidden state and the second current hidden state are input into the dense layer for classification to obtain the predicted probability distribution of each category. The category at this time refers to the category corresponding to the emotion labeling in step S1. The dense layer can map the first current hidden state and the second current hidden state to the category space of the emotion classification task, thereby obtaining the predicted score of each category, and the Softmax activation function is used to convert the obtained predicted score into the predicted probability distribution.

[0078] At this time, the refining loss is calculated according to the predicted probability distribution, and the first student model and the second student model are trained with the minimum refining loss as the goal, and the trained first student model and the trained second student model are obtained, including: The sparse classification cross entropy is used to calculate the refinement loss of each training sample according to the predicted probability distribution, and the total refinement loss is calculated according to the refinement loss and the total refinement loss formula. The total refinement loss formula is:

[0079] in, represents the total refining loss, represents the extraction loss of the βth training sample, and N represents the total number of training samples; The first student model and the second student model are trained using the Adam algorithm with the goal of minimizing the total refinement loss to obtain a trained first student model and a trained second student model.

[0080] In this embodiment, if Figure 3 As shown, sparse classification cross entropy is used to calculate the refinement loss of each training sample according to the predicted probability distribution, and the total refinement loss is calculated according to the total refinement loss formula. The Adam algorithm is used to train the first student model and the second student model with the minimum total refinement loss as the goal to obtain the trained first student model and the trained second student model.

[0081] Embodiment 2 Please refer to Figure 4 The present invention provides a system 1 for analyzing medical evaluations through an adversarial network of knowledge distillation, comprising a memory 3, a processor 2, and a computer program stored in the memory 3 and executable on the processor 2, wherein the processor 2 implements the steps in Example 1 when executing the computer program.

[0082] Since the system / device described in the above embodiments of the present invention is a system / device used to implement the method of the above embodiments of the present invention, a person skilled in the art can understand the specific structure and deformation of the system / device based on the method described in the above embodiments of the present invention, and thus will not be described in detail here. All systems / devices used in the method of the above embodiments of the present invention belong to the scope of protection of the present invention.

[0083] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0084] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions.

[0085] It should be noted that in the claims, any reference numerals placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In the claims enumerating several means, several of these means may be embodied by the same hardware. The use of the words first, second, third, etc., is for convenience of expression only and does not indicate any order. These words may be understood as part of the component name.

[0086] In addition, it should be noted that, in the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0087] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments after knowing the basic creative concept. Therefore, the claims should be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0088] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention should also include these modifications and variations.

Claims

1. A method for analyzing medical evaluation through adversarial networks using knowledge distillation, characterized in that: include: Acquire medical evaluation data, and perform emotion annotation on the medical evaluation data to obtain emotion-annotated medical evaluation data, wherein the emotion annotation includes positive emotion, negative emotion, and mixed emotion; The generative adversarial network is combined with the MASH module of the transformer model to obtain an improved generative adversarial network, the medical evaluation data with emotion annotation is input into the improved generative adversarial network to generate training samples, and the training samples are input into the teacher model for initialization to obtain an initialized teacher model; The GRU model, the BERT model and the first convolutional layer are combined to obtain the first student model, and the LSTM network, the BERT model and the second convolutional layer are combined to obtain the second student model. The knowledge learned in the initialized teacher model is refined into the first student model and the second student model through knowledge distillation, and the first student model and the second student model are trained with the minimum refinement loss as the goal to obtain the trained first student model and the trained second student model. The analysis results of the medical evaluation data are obtained based on the trained first student model and the trained second student model.

2. The method for analyzing medical evaluation through adversarial networks using knowledge distillation as claimed in claim 1, characterized in that: The obtaining of medical evaluation data includes: Build a web search engine using Python's Selenium module and Chromium driver; Generate a URL corresponding to a first page for each medical institution in a preset list of medical institutions by using the network search engine, and automatically navigate to the corresponding first page based on the URL using the Selenium module to obtain medical evaluation data, wherein the first page includes a map page, a social network page, and a medical institution website page; Parsing the medical evaluation data by using Beautiful Soup to obtain parsed medical evaluation data, wherein the parsed medical evaluation data includes a medical evaluation text, a medical score, and a timestamp; The parsed medical evaluation data is format-converted to obtain format-converted medical evaluation data, wherein the format conversion includes CSV format conversion and JSON format conversion.

3. The method for analyzing medical evaluation through adversarial networks using knowledge distillation as claimed in claim 1, characterized in that: The improved generative adversarial network includes an improved generator and an improved discriminator. The improved generative adversarial network is obtained by combining the generative adversarial network with the MASH module of the transformer model, including: Construct an improved generator, combining the generator of the generative adversarial network with the MHSA module of the transformer model, introducing the MHSA module as the initial layer to apply linear projection to modify the dimensional input tensor to adapt to the actual number of attention heads, and the MHSA module will calculate the proportional dot product on all the attention heads to generate the attention weights applied to the corresponding values ​​of the tensor, and all the attention weights will be processed in series and linearly to obtain the final attention output; Combining the MHSA module with an additional layer of spectral normalization linear transformation, Leaky ReLU activation function and Dropout filtering regularization, and adding an attention layer in the end block to obtain an improved generator; An improved discriminator is constructed by combining each linear layer in the discriminator of the generative adversarial network with an additional layer of spectral normalization linear transformation to obtain a combined linear layer, and adding residual connections between each combined linear layer. An arbitrary combined linear layer is selected and combined with a Softmax activation function as the output layer to obtain an improved discriminator.

4. The method for analyzing medical evaluation through adversarial networks using knowledge distillation as claimed in claim 1, characterized in that: The step of inputting the emotion-labeled medical evaluation data into the improved generative adversarial network to generate training samples includes: The medical evaluation data after emotion annotation is preprocessed to obtain preprocessed medical evaluation data, and the preprocessed medical evaluation data is input into an improved generative adversarial network to generate training samples, wherein the preprocessing includes: removing invalid text, word form restoration and text vectorization.

5. The method for analyzing medical evaluation through adversarial networks using knowledge distillation as claimed in claim 1, characterized in that: The method of extracting the knowledge learned from the initialized teacher model into the first student model and the second student model respectively through knowledge distillation, training the first student model and the second student model with the minimum extraction loss as the goal, and obtaining the trained first student model and the trained second student model includes: Introducing a first maximum pooling layer into the first student model, and using the first maximum pooling layer and the first convolutional layer as a first feature extractor, and capturing a first feature state of the knowledge through the first feature extractor; Introducing a second maximum pooling layer into the second student model, and using the second maximum pooling layer and the second convolutional layer as a second feature extractor, and capturing a second feature state of the knowledge through the second feature extractor; Based on the first feature state and the second feature state, the first student model and the second student model are trained with the minimum refinement loss as the goal to obtain a trained first student model and a trained second student model.

6. The method for analyzing medical evaluation through adversarial networks using knowledge distillation as claimed in claim 5, characterized in that: The step of training the first student model and the second student model based on the first feature state and the second feature state with a minimum extraction loss as a goal to obtain the trained first student model and the trained second student model comprises: The first student model is embedded by the first BERT model output t , according to the embedding e t , the first feature state and the recursive update equation of the GRU model calculate the first current hidden state of the knowledge, and the recursive update equation is: Among them, h t represents the first current hidden state at position t, h t-1 represents the first characteristic state at position t-1, e t represents the vector generated by the first BERT model after processing the input text at position t; The first update gate is calculated by the first update gate mechanism formula, wherein the first update gate mechanism formula is: Among them, z t represents the first update gate at position t, represents the sigmoid activation function, W z represents the weight matrix associated with the first update gate, b z represents the bias vector associated with the first update gate, h t-1 represents the first characteristic state at position t-1, e t represents the vector generated by the first BERT model after processing the input text at position t; When the first update gate is lower than the update threshold, the first current hidden state is retained, otherwise, the reset gate is calculated by the reset gate mechanism formula, and the reset gate mechanism formula is: Among them, r t represents the reset gate at position t, W r represents the weight matrix associated with the reset gate, b r represents the bias vector associated with the reset gate, h t-1 represents the first characteristic state at position t-1, e t represents the vector generated by the first BERT model after processing the input text at position t; The candidate gate is calculated according to the reset gate and the candidate gate mechanism formula, and the candidate gate mechanism formula is: in, represents the candidate gate at position t, tanh represents the hyperbolic tangent function, W h represents the weight matrix associated with the candidate gate, b h represents the bias vector associated with the candidate gate, r t represents the reset gate at position t, h t-1 represents the first characteristic state at position t-1, e t represents the vector generated by the first BERT model after processing the input text at position t; The first current hidden state of the knowledge is recalculated according to the candidate gate, the reset gate and the first hidden formula to obtain the latest first current hidden state, wherein the first hidden formula is: = Among them, H t represents the latest first current hidden state at position t, z t represents the first update gate at position t, h t-1 represents the first characteristic state at position t-1, represents the candidate gate at position t; The second student model outputs embeddings through the second BERT model , according to the embedding , the input gate of the knowledge is calculated according to the second feature state and the input gate mechanism formula of the LSTM network, the forget gate of the knowledge is calculated according to the second feature state and the forget mechanism formula of the LSTM, and the second update gate of the knowledge is calculated according to the second feature state and the second update gate mechanism formula of the LSTM, and the input gate mechanism formula is: in, represents the input gate at position t, represents the sigmoid activation function, W i represents the weight matrix associated with the input gate, b i represents the bias vector associated with the input gate, represents the second characteristic state at position t-1, Represents the vector generated by the second BERT model after processing the input text at position t The forgetting mechanism formula is: in, represents the forget gate at position t, represents the sigmoid activation function, W f represents the weight matrix associated with the forget gate, b f represents the bias vector associated with the forget gate, represents the second characteristic state at position t-1, Represents the vector generated by the second BERT model after processing the input text at position t The second update gate mechanism formula is: in, represents the second update gate at position t, tanh represents the hyperbolic tangent function, W g represents the weight matrix associated with the second update gate, b g represents the bias vector associated with the second update gate, represents the second characteristic state at position t-1, Represents the vector generated by the second BERT model after processing the input text at position t The candidate storage unit is calculated according to the second update gate, the input gate and the candidate storage unit formula, and the candidate storage unit formula is: in, represents the candidate storage unit at position t, represents the second update gate at position t, represents the input gate at position t; The latest storage unit is calculated according to the candidate storage unit, the forget gate and the storage update formula, and the storage update formula is: in, represents the latest storage unit at position t, represents the forget gate at position t, represents the candidate storage unit at position t, represents the latest storage unit at position t-1; The output gate of the knowledge is calculated according to the second feature state and the output gate mechanism formula of LSTM, and the output gate mechanism formula is: in, represents the output gate at position t, represents the sigmoid activation function, W o represents the weight matrix associated with the output gate, b o represents the bias vector associated with the output gate, represents the second characteristic state at position t-1, Represents the vector generated by the second BERT model after processing the input text at position t The second current hidden state of the knowledge is calculated according to the output gate, the latest storage unit and the second hidden formula, where the second hidden formula is: in, represents the second current hidden state at position t, represents the output gate at position t, represents the latest storage unit at position t, tanh represents the hyperbolic tangent function; The first current hidden state and the second current hidden state are input into a dense layer for classification to obtain a predicted probability distribution for each category, a refinement loss is calculated according to the predicted probability distribution, and the first student model and the second student model are trained with the minimum refinement loss as the goal to obtain a trained first student model and a trained second student model.

7. The method for analyzing medical evaluation through adversarial networks using knowledge distillation as claimed in claim 6, characterized in that: The inputting the first current hidden state and the second current hidden state into a dense layer for classification comprises: The first current hidden state and the second current hidden state are input into a dense layer for classification to obtain a prediction score for each category, and a Softmax activation function is used to convert the prediction score into a prediction probability distribution.

8. The method for analyzing medical evaluation through adversarial networks using knowledge distillation as claimed in claim 6, characterized in that: The step of calculating the refinement loss according to the predicted probability distribution, training the first student model and the second student model with the minimum refinement loss as the goal, and obtaining the trained first student model and the trained second student model comprises: The sparse classification cross entropy is used to calculate the refinement loss of each training sample according to the predicted probability distribution, and the total refinement loss is calculated according to the refinement loss and the total refinement loss formula. The total refinement loss formula is: in, represents the total refining loss, represents the refinement loss of the βth training sample, and N represents the total number of training samples; The first student model and the second student model are trained using the Adam algorithm with the goal of minimizing the total refinement loss to obtain a trained first student model and a trained second student model.

9. A system for analyzing medical evaluations through adversarial networks using knowledge distillation, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

Images