A Prediction Method and System for Tuberculous Meningoencephalitis Based on Multi-Scale Knowledge Distillation

Abstract

This invention discloses a method and system for predicting tuberculous meningoencephalitis (TBMeningoencephalitis) based on multi-scale knowledge distillation. The method involves: splitting TBMeningoencephalitis images into multi-scale feature maps; inputting these feature maps into a feature encoder to obtain single-dimensional feature vectors for each scale; pre-training a teacher model to obtain its weights; obtaining multi-scale teacher prediction results based on these weights; calculating a first loss by performing cross-entropy loss on the multi-scale teacher and student prediction results; calculating a second loss by performing cross-entropy loss on the ground truth labels and multi-scale student prediction results; summing the first and second losses to obtain the total loss; concatenating the single-dimensional feature vectors of each scale feature map to obtain a concatenated vector; and obtaining the TBMeningoencephalitis prediction result based on the concatenated vector and the total loss. This invention improves the accuracy of TBMeningoencephalitis prediction.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and smart healthcare, and in particular to a method and system for predicting tuberculous meningoencephalitis based on multi-scale knowledge distillation. Background Technology

[0002] In the imaging diagnosis of tuberculous meningoencephalitis, radiologists need to observe all MRI images from various cross-sections of the head and simultaneously locate the lesion at corresponding positions in different cross-sections to make a diagnosis. This diagnostic method is not only inefficient but also has a relatively high error rate. Therefore, how to help doctors achieve efficient and accurate imaging diagnosis is an urgent problem to be solved.

[0003] In current deep learning image classification models, most models utilize the feature maps of the last layer of the network structure for classification prediction. Because different feature maps learn varying degrees of information from the original image—feature maps closer to the top retain more detailed information, while those closer to the bottom retain only the overall semantic information—detailed information is lost in the lower-level feature maps, leading to inaccurate classification. Summary of the Invention

[0004] This invention aims to at least solve one of the technical problems existing in the prior art. To this end, this invention proposes a method and system for predicting tuberculous meningoencephalitis based on multi-scale knowledge distillation, which can improve the accuracy of tuberculous meningoencephalitis prediction.

[0005] In a first aspect, embodiments of the present invention provide a method for predicting tuberculous meningoencephalitis based on multi-scale knowledge distillation, the method comprising:

[0006] The input tuberculous meningoencephalitis image is split into multiple scales to obtain a multi-scale feature map;

[0007] A feature encoder is constructed based on the Transformer structure, and the multi-scale feature map is input into the feature encoder to obtain a one-dimensional feature vector for each scale feature map;

[0008] The teacher model was pre-trained using a dataset of images of tuberculous meningoencephalitis to obtain the weights of the teacher model, and multi-scale teacher prediction results were obtained based on the weights of the teacher model.

[0009] The first loss is obtained by performing cross-entropy loss calculation on the multi-scale teacher prediction results and the multi-scale student prediction results obtained through the student model, and the second loss is obtained by performing cross-entropy loss calculation on the true labels and the multi-scale student prediction results obtained through the student model.

[0010] Add the first loss and the second loss to obtain the total loss;

[0011] The single-dimensional feature vectors of each scale feature map are concatenated to obtain a concatenated vector;

[0012] Based on the splicing vector and the total loss, the prediction result of tuberculous meningoencephalitis is obtained through the student model.

[0013] Compared with the prior art, the first aspect of the present invention has the following beneficial effects:

[0014] This method performs multi-scale segmentation on input tuberculous meningoencephalitis images to obtain multi-scale feature maps. A feature encoder is constructed based on a Transformer structure, and the multi-scale feature maps are input into the encoder to obtain single-dimensional feature vectors for each scale. This extracts multi-scale feature information, improving the accuracy of subsequent classification predictions by the student model. The teacher model is pre-trained using a tuberculous meningoencephalitis image dataset to obtain its weights. Based on these weights, multi-scale teacher predictions are obtained. Cross-entropy loss is calculated between the multi-scale teacher predictions and the multi-scale student predictions obtained through the student model to obtain the first loss. Finally, the true labels and the multi-scale predictions obtained through the student model are compared. The cross-entropy loss is calculated based on the student's prediction results to obtain a second loss. The first and second losses are then added together to obtain the total loss. By transferring the knowledge learned from the complex model (i.e., the teacher model) to the lightweight model (i.e., the student model), multi-scale loss calculations are performed on the lightweight model to overcome the impact of insufficient data in the lightweight model. The single-dimensional feature vectors of each scale feature map are concatenated to obtain a concatenated vector. Based on the concatenated vector and the total loss, the prediction result of tuberculous meningoencephalitis is obtained through the student model. By concatenating the single-dimensional feature vectors of each scale feature map, the top-level detailed information and the bottom-level abstract semantic information are combined. The prediction using the concatenated vector and the total loss achieves good prediction results. Therefore, this method can assist doctors in diagnosing tuberculous meningoencephalitis, improve the accuracy of diagnosis, and reduce misdiagnosis.

[0015] According to some embodiments of the present invention, using a ResNet50 network as a student model, the step of performing multi-scale decomposition on the input tuberculous meningoencephalitis image to obtain multi-scale feature maps includes:

[0016] The tuberculous meningoencephalitis image is input into the student model for multi-scale decomposition;

[0017] The feature maps of the second layer (layer2), the third layer (layer3), and the fourth layer (layer4) of the student model are extracted to obtain multi-scale feature maps.

[0018] According to some embodiments of the present invention, the step of inputting the multi-scale feature map into the feature encoder to obtain a one-dimensional feature vector for each scale feature map includes:

[0019] The multi-scale feature map is converted into multiple feature vectors through the convolutional and flattening layers in the feature encoder.

[0020] An attention mechanism is used to calculate the correlation between each feature vector and other feature vectors;

[0021] Based on the correlation, the unidimensional feature vector of each scale feature map is obtained by globally averaging the feature vectors corresponding to each scale feature map.

[0022] According to some embodiments of the present invention, the attention mechanism is obtained in the following manner:

[0023] MultiHead(Q,K,V)=concat(head1(q1,k1,v1),…,head h (q h ,k h ,v h ))W o

[0024] head i (q i ,k i ,v i ) = Attention(q i ,k i ,v i )

[0025]

[0026] Where Q, K, and V represent a set of learnable weight matrices, q i ,k i ,v i W represents the corresponding components of the Q, K, and V weight matrices. o This represents a learnable parameter. This represents the scaling factor.

[0027] According to some embodiments of the present invention, a Swin-Transformer network model built with an attention mechanism is used as the teacher model. The teacher model is pre-trained using a tuberculous meningoencephalitis image dataset to obtain the weights of the teacher model, and multi-scale teacher prediction results are obtained based on the weights of the teacher model, including:

[0028] The teacher model was pre-trained using a dataset of images of tuberculous meningoencephalitis to obtain the weights of the teacher model;

[0029] Input a tuberculous meningoencephalitis image into the teacher model and obtain the feature vectors output by the second layer (layer2), the third layer (layer3), and the fourth layer (layer4) of the teacher model;

[0030] The feature vectors output from the second layer (layer2), the third layer (layer3), and the fourth layer (layer4) of the teacher model are classified and predicted using the weights of the teacher model to obtain multi-scale teacher prediction results.

[0031] According to some embodiments of the present invention, the step of calculating a first loss by performing cross-entropy loss calculation on the multi-scale teacher prediction results and the multi-scale student prediction results obtained through the student model, and calculating a second loss by performing cross-entropy loss calculation on the true labels and the multi-scale student prediction results obtained through the student model, includes:

[0032] The first loss is obtained by calculating the cross-entropy loss of the multi-scale teacher prediction results corresponding to the second layer (layer2), the third layer (layer3), and the fourth layer (layer4) in the teacher model and the multi-scale student prediction results corresponding to the second layer (layer2), the third layer (layer3), and the fourth layer (layer4) in the student model.

[0033] The cross-entropy loss is calculated by performing cross-entropy loss on the real labels and the multi-scale student prediction results corresponding to the second layer (layer2), third layer (layer3), and fourth layer (layer4) of the student model to obtain the second loss.

[0034] According to some embodiments of the present invention, the total loss is calculated in the following manner:

[0035] ζ(x,W)=α*CE(y,σ(Z s ;T=1))+β*CE(σ(Z t ;T=τ),σ(Z s ,T=τ))

[0036]

[0037] Where x represents the input, W represents the parameters of the student model, y represents the true label, CE represents the cross-entropy loss function, σ represents the softmax temperature activation function, and Z represents the input. s Z represents the output of the student model neuron. t This represents the output of the teacher model neuron, where α and β represent two weighting parameters, T represents the temperature coefficient, τ represents the specific value, and Z represents the output of the teacher model neuron. i This represents a neuron.

[0038] Secondly, embodiments of the present invention also provide a tuberculous meningoencephalitis prediction system based on multi-scale knowledge distillation, the tuberculous meningoencephalitis prediction system based on multi-scale knowledge distillation comprising:

[0039] The multi-scale feature map acquisition unit is used to perform multi-scale decomposition on the input tuberculous meningoencephalitis image to obtain multi-scale feature maps.

[0040] A single-dimensional feature vector acquisition unit is used to construct a feature encoder based on the Transformer structure and input the multi-scale feature map into the feature encoder to obtain a single-dimensional feature vector for each scale feature map.

[0041] The teacher model pre-training unit is used to pre-train the teacher model using a tuberculous meningoencephalitis image dataset to obtain an offline distilled teacher model, and to obtain multi-scale teacher prediction results based on the offline distilled teacher model.

[0042] The cross-entropy loss calculation unit is used to calculate the cross-entropy loss of the multi-scale teacher prediction results and the multi-scale student prediction results obtained through the student model to obtain the first loss, and to calculate the cross-entropy loss of the true labels and the multi-scale student prediction results obtained through the student model to obtain the second loss.

[0043] The total loss calculation unit is used to add the first loss and the second loss to obtain the total loss;

[0044] The splicing vector acquisition unit is used to splice the single-dimensional feature vectors of each scale feature map to obtain a spliced ​​vector;

[0045] The prediction result acquisition unit is used to obtain the prediction result of tuberculous meningoencephalitis through the student model based on the splicing vector and the total loss.

[0046] Thirdly, embodiments of the present invention also provide a tuberculous meningoencephalitis prediction device based on multi-scale knowledge distillation, comprising at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, the instructions being executed by the at least one control processor to enable the at least one control processor to perform a tuberculous meningoencephalitis prediction method based on multi-scale knowledge distillation as described above.

[0047] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the tuberculous meningoencephalitis prediction method based on multi-scale knowledge distillation as described above.

[0048] It is understood that the beneficial effects of the second to fourth aspects compared with the related technologies are the same as the beneficial effects of the first aspect compared with the related technologies. Please refer to the relevant description in the first aspect above, which will not be repeated here. Attached Figure Description

[0049] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0050] Figure 1 This is a flowchart of a method for predicting tuberculous meningoencephalitis based on multi-scale knowledge distillation, according to an embodiment of the present invention.

[0051] Figure 2 This is a schematic diagram of the student model part structure according to an embodiment of the present invention;

[0052] Figure 3 This is a schematic diagram of a feature encoder structure according to an embodiment of the present invention;

[0053] Figure 4 This is a schematic diagram illustrating the calculation of cross-entropy loss according to an embodiment of the present invention;

[0054] Figure 5 This is a schematic diagram of the overall student model structure according to an embodiment of the present invention;

[0055] Figure 6 This is a structural diagram of a tuberculous meningoencephalitis prediction system based on multi-scale knowledge distillation, according to an embodiment of the present invention. Detailed Implementation

[0056] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0057] In the description of this invention, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance, or implicitly indicating the number of technical features indicated, or implicitly indicating the order of the technical features indicated.

[0058] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the drawings and are only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0059] In the description of this invention, it should be noted that, unless otherwise explicitly defined, terms such as "setting," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0060] In the imaging diagnosis of tuberculous meningoencephalitis, radiologists need to observe all MRI images from various cross-sections of the head and simultaneously locate the lesion at corresponding positions in different cross-sections to make a diagnosis. This diagnostic method is not only inefficient but also has a relatively high error rate. Therefore, how to help doctors achieve efficient and accurate imaging diagnosis is an urgent problem to be solved.

[0061] In current deep learning image classification models, most models utilize the feature maps of the last layer of the network structure for classification prediction. Because different feature maps learn varying degrees of information from the original image—feature maps closer to the top retain more detailed information, while those closer to the bottom retain only the overall semantic information—detailed information is lost in the lower-level feature maps, leading to inaccurate classification.

[0062] To address the aforementioned issues, this invention performs multi-scale segmentation on the input tuberculous meningoencephalitis images to obtain multi-scale feature maps. A feature encoder is constructed based on a Transformer structure, and the multi-scale feature maps are input into the encoder to obtain single-dimensional feature vectors for each scale, extracting multi-scale feature information to improve the accuracy of subsequent classification predictions by the student model. The teacher model is pre-trained using a tuberculous meningoencephalitis image dataset to obtain its weights. Based on these weights, multi-scale teacher prediction results are obtained. Cross-entropy loss is calculated between the multi-scale teacher prediction results and the multi-scale student prediction results obtained through the student model to obtain the first loss. Finally, the true labels and the results obtained through the student model are compared. The multi-scale student prediction results are used to calculate cross-entropy loss, obtaining a second loss. The first and second losses are then added together to obtain the total loss. By transferring the knowledge learned by the complex model (i.e., the teacher model) to the lightweight model (i.e., the student model), multi-scale loss calculation is performed on the lightweight model, overcoming the impact of insufficient data in the lightweight model. The single-dimensional feature vectors of each scale feature map are concatenated to obtain a concatenated vector. Based on the concatenated vector and the total loss, the student model obtains the prediction result for tuberculous meningoencephalitis. By concatenating the single-dimensional feature vectors of each scale feature map, the top-level detailed information and the bottom-level abstract semantic information are combined. Using the concatenated vector and the total loss for prediction achieves good prediction results. Therefore, this invention can assist doctors in diagnosing tuberculous meningoencephalitis, improve the accuracy of tuberculous meningoencephalitis diagnosis, and reduce misdiagnosis.

[0063] Reference Figure 1 This invention provides a method for predicting tuberculous meningoencephalitis based on multi-scale knowledge distillation. This method includes, but is not limited to, steps S100 to S700, wherein:

[0064] Step S100: Perform multi-scale segmentation on the input tuberculous meningoencephalitis image to obtain multi-scale feature maps;

[0065] Step S200: Construct a feature encoder based on the Transformer structure, and input the multi-scale feature maps into the feature encoder to obtain the one-dimensional feature vector of each scale feature map;

[0066] Step S300: The teacher model is pre-trained using the tuberculous meningoencephalitis image dataset to obtain the weights of the teacher model, and the multi-scale teacher prediction results are obtained based on the weights of the teacher model.

[0067] Step S400: Calculate the cross-entropy loss between the multi-scale teacher prediction results and the multi-scale student prediction results obtained through the student model to obtain the first loss; and calculate the cross-entropy loss between the true labels and the multi-scale student prediction results obtained through the student model to obtain the second loss.

[0068] Step S500: Add the first loss and the second loss to obtain the total loss;

[0069] Step S600: Concatenate the single-dimensional feature vectors of each scale feature map to obtain the concatenated vector;

[0070] Step S700: Based on the splicing vector and the total loss, obtain the prediction results of tuberculous meningoencephalitis through the student model.

[0071] In steps S100 to S700 of some embodiments, to improve the accuracy of the student model's classification prediction, this embodiment performs multi-scale decomposition on the input tuberculous meningoencephalitis image to obtain multi-scale feature maps; constructs a feature encoder based on the Transformer structure, and inputs the multi-scale feature maps into the feature encoder to obtain single-dimensional feature vectors for each scale feature map; to overcome the impact of insufficient data on the lightweight model, this embodiment uses the tuberculous meningoencephalitis image dataset to pre-train the teacher model, obtains the weights of the teacher model, and obtains multi-scale teacher prediction results based on the weights of the teacher model; calculates the cross-entropy loss between the multi-scale teacher prediction results and the multi-scale student prediction results obtained through the student model to obtain a first loss; calculates the cross-entropy loss between the true label and the multi-scale student prediction results obtained through the student model to obtain a second loss; and adds the first loss and the second loss to obtain the total loss; to obtain better prediction results, this embodiment concatenates the single-dimensional feature vectors of each scale feature map to obtain a concatenated vector; and obtains the tuberculous meningoencephalitis prediction result through the student model based on the concatenated vector and the total loss.

[0072] In some embodiments, the ResNet50 network is used as a student model to perform multi-scale decomposition on the input tuberculous meningoencephalitis image to obtain multi-scale feature maps, including:

[0073] Tuberculous meningoencephalitis images were input into the student model for multi-scale decomposition;

[0074] Extract feature maps from the second layer (layer2), the third layer (layer3), and the fourth layer (layer4) of the student model to obtain multi-scale feature maps.

[0075] In this embodiment, feature information at multiple scales is extracted to improve the accuracy of classification prediction for the subsequent student model.

[0076] In some embodiments, multi-scale feature maps are input into a feature encoder to obtain a one-dimensional feature vector for each scale feature map, including:

[0077] The multi-scale feature map is converted into multiple feature vectors through convolutional layers and flattening layers in the feature encoder;

[0078] An attention mechanism is used to calculate the correlation between each feature vector and other feature vectors;

[0079] Based on the correlation, the unidimensional feature vector of each scale feature map is obtained by globally averaging the feature vectors corresponding to each scale feature map.

[0080] In some embodiments, the attention mechanism is obtained in the following manner:

[0081] MultiHead(Q,K,V)=concat(head1(q1,k1,v1),…,head h (q h ,k h ,v h ))W o

[0082] head i (q i ,k i ,v i ) = Attention(q i ,k i ,v i )

[0083]

[0084] Where Q, K, and V represent a set of learnable weight matrices, q i ,k i ,v i W represents the corresponding components of the Q, K, and V weight matrices. o This represents a learnable parameter. This represents the scaling factor.

[0085] In some embodiments, a Swin-Transformer network model built with an attention mechanism is used as the teacher model. The teacher model is pre-trained using a tuberculous meningoencephalitis image dataset to obtain the weights of the teacher model. Based on these weights, multi-scale teacher prediction results are obtained, including:

[0086] The teacher model was pre-trained using a dataset of images of tuberculous meningoencephalitis to obtain the weights of the teacher model;

[0087] Input tuberculous meningoencephalitis images into the teacher model and obtain the feature vectors output by the second layer (layer2), the third layer (layer3), and the fourth layer (layer4) of the teacher model;

[0088] The feature vectors output from the second layer (layer 2), the third layer (layer 3), and the fourth layer (layer 4) of the teacher model are used for classification and prediction through the weights of the teacher model to obtain multi-scale teacher prediction results.

[0089] In some embodiments, a first loss is obtained by calculating the cross-entropy loss between the multi-scale teacher prediction results and the multi-scale student prediction results obtained through the student model, and a second loss is obtained by calculating the cross-entropy loss between the true labels and the multi-scale student prediction results obtained through the student model, including:

[0090] The first loss is obtained by calculating the cross-entropy loss between the multi-scale teacher prediction results corresponding to the second layer (layer2), the third layer (layer3), and the fourth layer (layer4) in the teacher model and the multi-scale student prediction results corresponding to the second layer (layer2), the third layer (layer3), and the fourth layer (layer4) in the student model.

[0091] The cross-entropy loss is calculated by combining the real labels with the multi-scale student prediction results corresponding to the second layer (layer 2), the third layer (layer 3), and the fourth layer (layer 4) in the student model to obtain the second loss.

[0092] In this embodiment, cross-entropy loss is applied to the prediction results obtained by the teacher model and the prediction results obtained by the student model. This allows the knowledge learned by the complex model (i.e., the teacher model) to be transferred to the lightweight model (i.e., the student model). Multi-scale loss calculation is performed on the lightweight model to overcome the impact of insufficient data in the lightweight model.

[0093] In some embodiments, the total loss is calculated as follows:

[0094] ζ(x,W)=α*CE(y,σ(Z s ;T=1))+β*CE(σ(Z t ;T=τ),σ(Z s ,T=τ))

[0095]

[0096] Where x represents the input, W represents the parameters of the student model, y represents the true label, CE represents the cross-entropy loss function, σ represents the softmax temperature activation function, and Z represents the input. s Z represents the output of the student model neuron. t This represents the output of the teacher model neuron, where α and β represent two weighting parameters, T represents the temperature coefficient, τ represents the specific value, and Z represents the output of the teacher model neuron. i This represents a neuron.

[0097] To facilitate understanding by those skilled in the art, a set of preferred embodiments is provided below:

[0098] 1. Select a student model and extract multi-scale feature vectors.

[0099] Using the ResNet50 network architecture as the student model, the input tuberculous meningoencephalitis image is decomposed to separate feature maps of different scales. A global attention mechanism is then applied to these feature maps to condense the information, resulting in multi-scale, one-dimensional feature vectors. The specific steps include:

[0100] Step 1, refer to Figure 2 The ResNet50 network is used to decompose the input tuberculous meningoencephalitis image into multi-scale feature maps, and the last layer feature maps of layer 2, layer 3 and layer 4 in the network are extracted. The shapes of the corresponding feature maps of layer 2, layer 3 and layer 4 are 28*28*512, 14*14*1024 and 7*7*2048, respectively.

[0101] Step 2: Construct a feature encoder using a Transformer architecture and perform global attention processing on the feature maps. The structure diagram of the feature encoder is as follows: Figure 3 As shown, the formula for calculating multi-head self-attention is as follows:

[0102] MultiHead(Q,K,V)=concat(head1(q1,k1,v1),…,head h (q h ,k h ,v h ))W o

[0103] head i (q i ,k i ,v i ) = Attention(q i ,k i ,v i )

[0104]

[0105] Step 3: Since Transformer is mainly used in the field of natural language processing, the input data is required to be in the form of word vectors. Therefore, in this embodiment, the shapes of the feature maps corresponding to layer 2, layer 3 and layer 4 are processed to 784*C1, 196*C2 and 49*C3 respectively, and a single pixel and its channel are used as feature vectors, where C1, C2 and C3 are hyperparameters of the feature encoder;

[0106] Step 4: Input the multi-scale feature maps into the feature encoder. The feature encoder converts the multi-scale feature maps into multiple feature vectors through convolutional layers and flattening layers. Then, it uses an attention mechanism to calculate the correlation between the current feature vector and other feature vectors. Finally, based on the correlation, it obtains the single-dimensional feature vector of the current scale feature map by global averaging. Figure 2 As shown in A, B, and C.

[0107] 2. Select a teacher model and pre-train it to obtain an offline teacher distillation model.

[0108] Using the Swin-Transformer network model as the teacher model, it was trained on the current tuberculous meningoencephalitis dataset. The best training results were saved for offline distillation training of the student models. The specific steps included:

[0109] Step 1: Select the Swin-Transformer network model, which is built entirely on the attention mechanism, as the teacher model;

[0110] Step 2: Pre-train the teacher model on the tuberculous meningoencephalitis dataset to obtain the offline distillation teacher model (i.e., the weights of the teacher model);

[0111] Step 2: Adjust the training process of the teacher model using the obtained offline distillation teacher model, and perform classification prediction on the feature vectors output by layer 2, layer 3 and layer 4 in the teacher model to obtain multi-scale teacher prediction results.

[0112] 3. Multiscale distillation training for student models.

[0113] The cross-entropy loss is calculated by combining the multi-scale teacher predictions obtained from the teacher model and the true labels (i.e., the labels inherent in the dataset) corresponding to the current tuberculous meningoencephalitis dataset with the multi-scale student predictions obtained from the student model. These losses are then added together in a certain proportion to obtain the total loss. The specific steps include:

[0114] Step 1: Pass the current training data through the forward propagation network of the teacher model to obtain the predicted probability values ​​of the categories in the multi-scale features;

[0115] Step 2: Perform cross-entropy loss on the probability values ​​predicted by the student model, the predicted probability values ​​obtained by the teacher model, and the true labels.

[0116] Step 3: Sum all the losses according to a certain proportion to obtain the total loss of the student model. The formula for calculating the total loss is as follows:

[0117] ζ(x,W)=α*CE(y,σ(Z s ;T=1))+β*CE(σ(Z t ;T=τ),σ(Z s ,T=τ))

[0118]

[0119] The specific process is as follows: Figure 4As shown, the Soft loss is the cross-entropy loss between the multi-scale student predictions obtained by the student model and the multi-scale teacher predictions obtained by the teacher model, and the Hard loss is the cross-entropy loss between the multi-scale student predictions obtained by the student model and the true labels.

[0120] 4. Concatenate the multi-scale feature vectors of the student model to perform classification prediction.

[0121] Specifically, the single-dimensional feature vectors A, B, and C corresponding to the multi-scale feature maps of the student model are concatenated to form the final feature vector used for prediction. The specific process is as follows: Figure 5 As shown, the student model obtains the prediction results of tuberculous meningoencephalitis based on the splicing vector and the total loss.

[0122] In this embodiment, the knowledge learned by the complex model (teacher model) is transferred to the lightweight model (student model), overcoming the impact of insufficient data in the lightweight model, and assisting doctors in making efficient and accurate diagnoses of tuberculous meningoencephalitis.

[0123] Reference Figure 6 This invention also provides a tuberculous meningoencephalitis prediction system based on multi-scale knowledge distillation. This system includes a multi-scale feature map acquisition unit 100, a single-dimensional feature vector acquisition unit 200, a teacher model pre-training unit 300, a cross-entropy loss calculation unit 400, a total loss calculation unit 500, a spliced ​​vector acquisition unit 600, and a prediction result acquisition unit 700, wherein:

[0124] The multi-scale feature map acquisition unit 100 is used to perform multi-scale decomposition on the input tuberculous meningoencephalitis image to obtain a multi-scale feature map.

[0125] The single-dimensional feature vector acquisition unit 200 is used to construct a feature encoder based on the Transformer structure and input multi-scale feature maps into the feature encoder to obtain single-dimensional feature vectors for each scale feature map.

[0126] Teacher model pre-training unit 300 is used to pre-train the teacher model using a tuberculous meningoencephalitis image dataset to obtain an offline distilled teacher model, and to obtain multi-scale teacher prediction results based on the offline distilled teacher model.

[0127] The cross-entropy loss calculation unit 400 is used to calculate the cross-entropy loss of the multi-scale teacher prediction results and the multi-scale student prediction results obtained through the student model to obtain the first loss, and to calculate the cross-entropy loss of the true label and the multi-scale student prediction results obtained through the student model to obtain the second loss.

[0128] The total loss calculation unit 500 is used to add the first loss and the second loss to obtain the total loss;

[0129] The splicing vector acquisition unit 600 is used to splice the single-dimensional feature vectors of feature maps at each scale to obtain a spliced ​​vector;

[0130] The prediction result acquisition unit 700 is used to obtain the prediction result of tuberculous meningoencephalitis through the student model based on the splicing vector and the total loss.

[0131] It should be noted that since the tuberculous meningoencephalitis prediction system based on multi-scale knowledge distillation in this embodiment is based on the same inventive concept as the tuberculous meningoencephalitis prediction method based on multi-scale knowledge distillation described above, the corresponding content in the method embodiment is also applicable to this system embodiment, and will not be described in detail here.

[0132] This invention also provides a tuberculous meningoencephalitis prediction device based on multi-scale knowledge distillation, comprising: at least one control processor and a memory for communicatively connecting to the at least one control processor.

[0133] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0134] The non-transient software program and instructions required to implement the tuberculous meningoencephalitis prediction method based on multi-scale knowledge distillation in the above embodiments are stored in memory. When executed by a processor, the tuberculous meningoencephalitis prediction method based on multi-scale knowledge distillation in the above embodiments is executed, for example, the method described above is executed. Figure 1 The method steps S100 to S700.

[0135] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0136] This invention also provides a computer-readable storage medium storing computer-executable instructions. These instructions are executed by one or more control processors, causing the processors to perform a method for predicting tuberculous meningoencephalitis based on multi-scale knowledge distillation as described in the above-described method embodiments. For example, they can execute the methods described above. Figure 1 The functions of steps S100 to S700 in the method.

[0137] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0138] The above is a detailed description of the preferred embodiments of this application. However, the embodiments of this application are not limited to the above-described implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the embodiments of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of the embodiments of this application.

Claims

1. A method for predicting tuberculous meningoencephalitis based on multi-scale knowledge distillation, characterized in that, The tuberculous meningoencephalitis prediction method based on multi-scale knowledge distillation includes: The input tuberculous meningoencephalitis image is split into multiple scales to obtain a multi-scale feature map, including: The tuberculous meningoencephalitis images were input into the student model for multi-scale decomposition. Extract the feature maps of the second layer (layer2), the third layer (layer3), and the fourth layer (layer4) of the student model to obtain multi-scale feature maps; A feature encoder is constructed based on the Transformer structure, and the multi-scale feature map is input into the feature encoder to obtain a one-dimensional feature vector for each scale feature map; A teacher model was pre-trained using a dataset of images of tuberculous meningoencephalitis to obtain the weights of the teacher model. Based on these weights, multi-scale teacher prediction results were obtained, including: The teacher model was pre-trained using a dataset of images of tuberculous meningoencephalitis to obtain the weights of the teacher model; Input a tuberculous meningoencephalitis image into the teacher model and obtain the feature vectors output by the second layer (layer2), the third layer (layer3), and the fourth layer (layer4) of the teacher model; The feature vectors output from the second layer (layer2), the third layer (layer3), and the fourth layer (layer4) of the teacher model are classified and predicted using the weights of the teacher model to obtain multi-scale teacher prediction results. The first loss is obtained by performing cross-entropy loss calculation on the multi-scale teacher prediction results and the multi-scale student prediction results obtained through the student model. The second loss is obtained by performing cross-entropy loss calculation on the true labels and the multi-scale student prediction results obtained through the student model, including: The first loss is obtained by calculating the cross-entropy loss of the multi-scale teacher prediction results corresponding to the second layer (layer2), the third layer (layer3), and the fourth layer (layer4) in the teacher model and the multi-scale student prediction results corresponding to the second layer (layer2), the third layer (layer3), and the fourth layer (layer4) in the student model. The cross-entropy loss is calculated by performing cross-entropy loss on the real labels and the multi-scale student prediction results corresponding to the second layer (layer2), third layer (layer3), and fourth layer (layer4) of the student model to obtain the second loss. Add the first loss and the second loss to obtain the total loss; The single-dimensional feature vectors of each scale feature map are concatenated to obtain a concatenated vector; Based on the splicing vector and the total loss, the prediction result of tuberculous meningoencephalitis is obtained through the student model.

2. The method for predicting tuberculous meningoencephalitis based on multi-scale knowledge distillation according to claim 1, characterized in that, The step of inputting the multi-scale feature map into the feature encoder to obtain a one-dimensional feature vector for each scale feature map includes: The multi-scale feature map is converted into multiple feature vectors through the convolutional and flattening layers in the feature encoder. An attention mechanism is used to calculate the correlation between each feature vector and other feature vectors; Based on the correlation, the unidimensional feature vector of each scale feature map is obtained by globally averaging the feature vectors corresponding to each scale feature map.

3. The method for predicting tuberculous meningoencephalitis based on multi-scale knowledge distillation according to claim 2, characterized in that, The attention mechanism is obtained in the following way: Where Q, K, and V represent a set of learnable weight matrices. This represents the corresponding components of the Q, K, and V weight matrices. This represents a learnable parameter. This represents the scaling factor.

4. The method for predicting tuberculous meningoencephalitis based on multi-scale knowledge distillation according to claim 1, characterized in that, The total loss is calculated as follows: in, Indicates input, The parameters representing the student model, Indicates the true label, Represents the cross-entropy loss function. This represents the softmax temperature activation function. This represents the output of the student model's neurons. This represents the output of the teacher model's neurons. and This represents two weight parameters. Indicates the temperature coefficient. Indicates a specific value, This represents a neuron.

5. A tuberculous meningoencephalitis prediction system based on multi-scale knowledge distillation, characterized in that, The tuberculous meningoencephalitis prediction system based on multi-scale knowledge distillation includes: The multi-scale feature map acquisition unit is used to perform multi-scale segmentation on the input tuberculous meningoencephalitis image to obtain multi-scale feature maps, including: The tuberculous meningoencephalitis image is input into the student model for multi-scale decomposition; Extract the feature maps of the second layer (layer2), the third layer (layer3), and the fourth layer (layer4) of the student model to obtain multi-scale feature maps; A single-dimensional feature vector acquisition unit is used to construct a feature encoder based on the Transformer structure and input the multi-scale feature map into the feature encoder to obtain a single-dimensional feature vector for each scale feature map. The teacher model pre-training unit is used to pre-train the teacher model using a tuberculous meningoencephalitis image dataset to obtain an offline distilled teacher model, and to obtain multi-scale teacher prediction results based on the offline distilled teacher model, including: The teacher model was pre-trained using a dataset of images of tuberculous meningoencephalitis to obtain the weights of the teacher model; Input a tuberculous meningoencephalitis image into the teacher model and obtain the feature vectors output by the second layer (layer2), the third layer (layer3), and the fourth layer (layer4) of the teacher model; The feature vectors output from the second layer (layer2), the third layer (layer3), and the fourth layer (layer4) of the teacher model are classified and predicted using the weights of the teacher model to obtain multi-scale teacher prediction results. The cross-entropy loss calculation unit is used to calculate the cross-entropy loss of the multi-scale teacher prediction results and the multi-scale student prediction results obtained through the student model to obtain a first loss, and to calculate the cross-entropy loss of the true labels and the multi-scale student prediction results obtained through the student model to obtain a second loss, including: The first loss is obtained by calculating the cross-entropy loss of the multi-scale teacher prediction results corresponding to the second layer (layer2), the third layer (layer3), and the fourth layer (layer4) in the teacher model and the multi-scale student prediction results corresponding to the second layer (layer2), the third layer (layer3), and the fourth layer (layer4) in the student model. The cross-entropy loss is calculated by performing cross-entropy loss on the real labels and the multi-scale student prediction results corresponding to the second layer (layer2), third layer (layer3), and fourth layer (layer4) of the student model to obtain the second loss. The total loss calculation unit is used to add the first loss and the second loss to obtain the total loss; The splicing vector acquisition unit is used to splice the single-dimensional feature vectors of each scale feature map to obtain a spliced ​​vector; The prediction result acquisition unit is used to obtain the prediction result of tuberculous meningoencephalitis through the student model based on the splicing vector and the total loss.

6. A device for predicting tuberculous meningoencephalitis based on multi-scale knowledge distillation, characterized in that, The method includes at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor to enable the at least one control processor to perform the tuberculous meningoencephalitis prediction method based on multi-scale knowledge distillation as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the tuberculous meningoencephalitis prediction method based on multi-scale knowledge distillation as described in any one of claims 1 to 4.

Images