Imaging Prediction of Tumor Treatment Responsiveness Classification Method, Device, Equipment and Medium

Abstract

The present invention provides a method, apparatus, device and medium for classifying the treatment reactivity of tumors by image prediction, which relates to the technical field of image recognition. The method includes: obtaining a basic network model, introducing a deep supervision strategy and a multi-layer perceptron structure into the basic network model to obtain an initial network model; determining a teacher-student network model according to the initial network model; obtaining an image data set, which includes a first data subset and a second data subset. The first data subset includes a plurality of first data, and the second data subset includes a plurality of second data, and each second data corresponds to a first data; training the teacher-student network model based on the first data subset and the second data subset to obtain a target model; obtaining an image to be classified, and inputting the image to be classified into the target model to obtain a corresponding classification result. This application improves the prediction accuracy of pathological complete remission.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular, to a method, device, equipment and medium for classifying the treatment reactivity of tumors by image prediction. Background Art

[0002] To determine whether the treatment effect of a tumor after neoadjuvant therapy reaches the pathological complete response rate (PCR), the prior art mainly predicts and classifies the treatment reactivity through a neoadjuvant therapy effect prediction model based on deep learning. However, the existing prediction models mainly rely on pre-treatment imaging data and do not combine clinical data and biochemical indicators, resulting in poor interpretability and unable to achieve good prediction effects. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to overcome the deficiencies in the prior art and provide a method, device, equipment and medium for classifying the treatment reactivity of tumors by image prediction. The present invention provides the following technical solutions:

[0004] In a first aspect, the present invention provides a method for classifying the treatment reactivity of tumors by image prediction, the method comprising:

[0005] Obtain a basic network model, introduce a deep supervision strategy and a multi-layer perceptron structure into the basic network model to obtain an initial network model;

[0006] Determine a teacher-student network model according to the initial network model;

[0007] Obtain an image data set, the image data set comprising: a first data subset and a second data subset, the first data subset comprising a plurality of first data, the second data subset comprising a plurality of second data, each of the second data corresponding to one of the first data;

[0008] Train the teacher-student network model based on the first data subset and the second data subset to obtain a target model;

[0009] Obtain an image to be classified, and input the image to be classified into the target model to obtain a corresponding classification result.

[0010] In an embodiment, the basic network model comprises: N residual modules and a fully connected layer, and introducing the deep supervision strategy into the basic network model comprises:

[0011] Add an output layer to each of the residual modules;

[0012] Determine the outputs of the output layers corresponding to the first to the (N - 1)th residual modules as intermediate outputs respectively, and determine the output of the output layer corresponding to the Nth residual module as the final output;

[0013] Process each of the intermediate outputs and the final output using the softmax function to obtain corresponding soft labels;

[0014] Based on each of the soft labels, determine the final loss value according to the relative entropy divergence loss function;

[0015] Determine the final loss function according to the final loss value;

[0016] Update the basic network model according to the final loss function.

[0017] In one embodiment, introduce a multi - layer perceptron structure into the basic network model, including: introducing the multi - layer perceptron structure before the fully - connected layer, and the multi - layer perceptron structure is used to fuse at least one preset feature.

[0018] In one embodiment, the teacher - student network model includes: a teacher network model and a student network model. Training the teacher - student network model based on the first data subset and the second data subset to obtain a target model includes: dividing the first data subset into a first training set and a first test set according to a preset ratio; training the teacher network model based on the first training set and the first test set to obtain a target teacher network model; training the student network model based on the second data subset and the target teacher network model to obtain the target model.

[0019] In one embodiment, training the student network model based on the second data subset and the target teacher network model to obtain the target model includes: dividing the second data subset into a second training set and a second test set according to a preset ratio; training the student network model based on the second training set and the second test set, and during the training process, updating the student network model based on the output of the target teacher network model, and determining the updated student network model as the target model.

[0020] In one embodiment, obtaining the image dataset includes:

[0021] Obtain multiple first images, and perform image processing on each of the first images according to a preset processing method to obtain corresponding second images;

[0022] Adopt the largest connected component method to crop each of the second images according to the corresponding preset mask image to obtain corresponding third images;

[0023] Normalize each of the third images to obtain corresponding fourth images;

[0024] For each of the fourth images, perform random transformation processing to obtain multiple fifth images, and perform labeled classification on each of the fifth images;

[0025] Each of the fifth images and the classification labels respectively corresponding to each of the fifth images constitute the image dataset.

[0026] In a second aspect, the present invention provides an image prediction tumor treatment reactivity classification device, the device comprising:

[0027] A first determination module, configured to obtain a basic network model, introduce a deep supervision strategy and a multi-layer perceptron structure into the basic network model to obtain an initial network model;

[0028] A second determination module, configured to determine a teacher-student network model according to the initial network model;

[0029] A data acquisition module, configured to acquire an image dataset, the image dataset including: a first data subset and a second data subset, the first data subset including a plurality of first data, the second data subset including a plurality of second data, and each of the second data respectively corresponding to one of the first data;

[0030] A model training module, configured to train the teacher-student network model based on the first data subset and the second data subset to obtain a target model;

[0031] A classification module, configured to acquire an image to be classified, and input the image to be classified into the target model to obtain a corresponding classification result.

[0032] In a third aspect, the present invention provides an electronic device, comprising a memory and a processor, the memory storing a computer program, and the computer program, when running on the processor, executes the image prediction tumor treatment reactivity classification method described in the first aspect.

[0033] In a fourth aspect, the present invention provides a computer-readable storage medium, the computer-readable storage medium storing a computer program, and the computer program, when executed by a processor, executes the image prediction tumor treatment reactivity classification method described in the first aspect.

[0034] The image prediction tumor treatment reactivity classification method, device, equipment and medium provided by the present invention obtain a basic network model, introduce a deep supervision strategy and a multi-layer perceptron structure into the basic network model to obtain an initial network model; determine a teacher-student network model according to the initial network model; obtain an image data set, where the image data set includes: a first data subset and a second data subset, the first data subset includes a plurality of first data, the second data subset includes a plurality of second data, and each of the second data corresponds to one of the first data; train the teacher-student network model based on the first data subset and the second data subset to obtain a target model; obtain an image to be classified, and input the image to be classified into the target model to obtain a corresponding classification result. Through the reconstructed target model, the present application classifies and predicts the pathological complete remission situation, improving the prediction accuracy of the pathological complete remission situation.

[0035] In order to make the above objects, features and advantages of the present invention more obvious and understandable, the following specifically enumerates preferred embodiments and, in conjunction with the accompanying drawings, makes a detailed description as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0037] Figure 1 FIG. shows a schematic flowchart of an image prediction tumor treatment reactivity classification method provided by an embodiment of the present invention;

[0038] Figure 2 FIG. shows another schematic flowchart of an image prediction tumor treatment reactivity classification method provided by an embodiment of the present invention;

[0039] Figure 3 FIG. shows still another schematic flowchart of an image prediction tumor treatment reactivity classification method provided by an embodiment of the present invention;

[0040] Figure 4 FIG. shows a schematic diagram of the first and second images before and after cropping provided by an embodiment of the present application;

[0041] Figure 5 FIG. shows a schematic training diagram of a teacher-student network model provided by an embodiment of the present application;

[0042] Figure 6 FIG. shows a schematic structural diagram of an image prediction tumor treatment reactivity classification device provided by an embodiment of the present application;

[0043] Figure 7 Shows a schematic structural diagram of an electronic device provided by an embodiment of the present application.

[0044] Description of main component symbols:

[0045] 600 - Image prediction tumor treatment responsiveness classification device; 610 - First determination module; 620 - Second determination module; 630 - Data acquisition module; 640 - Model training module; 650 - Classification module; 700 - Electronic device; 701 - Transceiver; 702 - Processor; 703 - Memory. Detailed implementation manners

[0046] The embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and should not be construed as a limitation of the present invention.

[0047] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "a plurality" means two or more unless otherwise specifically defined.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used in the description of the template herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0049] Embodiment 1

[0050] To predict whether a patient who receives neoadjuvant therapy can achieve a pathological complete response rate (PCR), existing technologies mostly use radiomics-based feature extraction for prediction. The disadvantages are as follows: (1) It depends on manual feature extraction and cannot capture complex biological information; (2) When dealing with high-dimensional data, overfitting is likely to occur; (3) The labor cost is high and human bias is easily introduced. To address the above deficiencies, an existing technology proposes a neoadjuvant therapy effect prediction model based on deep learning. Through the neural network of deep learning, it automatically learns image features and processes image data. However, the existing deep learning-based neoadjuvant therapy effect prediction models mainly rely on pre-treatment image data, without combining clinical data and biochemical indicators. The interpretability of the models is poor and good prediction effects cannot be achieved. For this, please refer to Figure 1 , this application proposes a method for classifying the responsiveness of imaging prediction of tumor treatment, including steps S110 to S150.

[0051] Step S110: Obtain a basic network model, introduce a deep supervision strategy (DSS, Deep Supervision Strategy) and a multi-layer perceptron structure (MLP, Multi-Layer Perceptron) into the basic network model to obtain an initial network model.

[0052] In this embodiment, the basic network model is: 3D ResNet-18. Using 3D ResNet-18 as the basic network, introducing a deep supervision strategy and a multi-layer perceptron structure, and determining the improved basic network model as the initial network model.

[0053] It can be understood that introducing a multi-level deep supervision strategy enables the model to learn from different stages of the network, thereby learning richer image features and enhancing the feature capture ability of the model; introducing a multi-layer perceptron structure can integrate the model output with multiple clinical features, such as age, gender, BMI, tumor location, CEA, and CA125, etc., to further improve the prediction effect of the model.

[0054] In an implementation manner, the basic network model includes: N residual modules and a fully connected layer. Please refer to Figure 2 , the introduction of the deep supervision strategy into the basic network model includes: steps S111 to S116.

[0055] Specifically, the basic network model includes multiple residual modules, which are connected in sequence. By adding an output layer to each residual module in the basic network model, the model generates multiple intermediate outputs and a final output. The final loss function of the model is re-determined based on each output, and further, the network parameters of the model are optimized and updated according to the final loss function.

[0056] S111, add an output layer to each of the residual modules.

[0057] Specifically, an output layer is added at the output end of each residual module, so that the model generates multiple intermediate outputs and a final output. The loss function of the model is adjusted according to each intermediate output and the final output to improve the prediction ability of the model.

[0058] S112, determine the outputs of the output layers corresponding to the 1st to the (N - 1)th residual modules as intermediate outputs respectively, and determine the output of the output layer corresponding to the Nth residual module as the final output.

[0059] For example, the basic network model includes four residual modules connected in sequence. An output layer is added at the output end of each residual module respectively. The output results of the output layers corresponding to the first residual module, the second residual module, and the third residual module are determined as intermediate outputs, and the output result of the output layer corresponding to the fourth residual module is determined as the final output.

[0060] S113, process each of the intermediate outputs and the final output using the normalized exponential function to obtain corresponding soft labels.

[0061] Specifically, after each intermediate output and the final output pass through the normalized exponential function (softmax) respectively, corresponding multiple soft labels are obtained.

[0062] S114, based on each of the soft labels, determine the final loss value according to the relative entropy divergence loss function.

[0063] Specifically, use the relative entropy (KL, Kullback - Leibler) loss function to calculate the loss value between the soft label corresponding to each intermediate output and the soft label corresponding to the final output, and add the calculated loss values to the initial loss function of the model to obtain the final loss function. It can be understood that the initial loss function is a preset basic objective function.

[0064] S115, determine the final loss function according to the final loss value.

[0065] For example, if the initial loss function of the model is a, and the loss value between the soft label corresponding to the intermediate output and the soft label corresponding to the final output is b, then the final output loss function is: a + b × alpha. It can be understood that alpha is a preset constant used to adjust the proportion of the intermediate output in the loss function, and its specific value can be set according to actual needs and is not limited here.

[0066] S116. Update the basic network model according to the final loss function.

[0067] Specifically, perform backpropagation according to the final loss function, calculate the gradient, and use an optimization algorithm to update the weights of the basic network model to minimize the value of the final loss function.

[0068] Step S120. Determine the teacher-student network model according to the initial network model.

[0069] In this embodiment, the initial network model is embedded into a preset knowledge distillation framework to obtain the teacher-student network model. It can be understood that the teacher-student network model includes: a teacher network model and a student network model.

[0070] In one implementation manner, a multi-layer perceptron structure is introduced into the basic network model, including: introducing the multi-layer perceptron structure before the fully connected layer, and the multi-layer perceptron structure is used to fuse at least one preset feature.

[0071] In this embodiment, the multi-layer perceptron structure is used to integrate the CT image and at least one preset feature, where the preset features include: clinical features, specifically including: age, gender, body mass index (BMI), tumor location, carcinoembryonic antigen (CEA), and / or cancer antigen 125 (CA125). These features are fused before the fully connected layer of the model to further improve the prediction accuracy of the model. Among them, the multi-layer perceptron structure includes: a linear transformation layer and a ReLU activation function.

[0072] Step S130. Obtain an image data set, where the image data set includes: a first data subset and a second data subset. The first data subset includes a plurality of first data, and the second data subset includes a plurality of second data, and each of the second data corresponds to one of the first data.

[0073] In this embodiment, the first data includes computed tomography (CT) images after treatment, hereinafter referred to as CT images. The first data also includes classification labels corresponding to each CT image. The second data includes CT images before treatment and classification labels corresponding to each CT image. It can be understood that for the same treatment object, the CT image before treatment corresponds to the CT image after treatment.

[0074] In one embodiment, please refer to Figure 3 , the obtaining of the image data set includes steps S131 to S135.

[0075] Step S131: Obtain multiple first images, and perform image processing on each of the first images according to a preset processing method to obtain corresponding second images.

[0076] It should be noted that the first images are CT images. Before performing image processing on the first images, the format of the first images needs to be converted to a preset format first. For example, the preset format is the NRRD format. If the obtained first images are in the DICOM format, then the first images need to be converted from the DICOM format to the NRRD format first, and then image processing is performed on the first images after format adjustment.

[0077] In one embodiment, the performing image processing on each of the first images according to a preset processing method to obtain corresponding second images includes: adjusting each of the first images to a preset window width, preset window level, preset brightness, and preset pixel value respectively, and then performing resampling to obtain the second images corresponding to each of the first images.

[0078] It can be understood that the window width controls the contrast of the gray level in the image display. A larger window width will result in a lower contrast of the image, while a smaller window width will result in an increased contrast of the image. The window level determines the position of the center of the gray level of the image. The image brightness and the preset pixel value also affect the clarity of the image. Therefore, in this embodiment, in order to avoid the adverse effects of the image clarity on the model prediction, it is necessary to adjust the window width, window level, brightness, and pixel value of each first image.

[0079] It should be noted that the preset window frame, preset window level, preset brightness, and preset pixel value can be adjusted according to reference experience values, specifically according to actual needs. For example, the window width is adjusted to 350 so that the image can best display the esophagus part; the window level is adjusted to 0 so that the center of the gray level of the image is located at the center of the entire gray scale range.

[0080] Step S132: Using the largest connected component method, crop each of the second images according to the corresponding preset mask image to obtain corresponding third images.

[0081] In this embodiment, the preset mask image is usually a binary image of the same size as the CT image. Different second images correspond to different preset mask images. According to the corresponding preset mask image, the second image is cropped using the largest connected component method to obtain the corresponding third image, where the third image contains the region of interest (ROI), for example: the tumor location.

[0082] It should be noted that the largest connected component method refers to constructing a connected component containing non-zero pixel values based on the non-zero pixel values in the preset mask image. Compared with the traditional cropping method, it can better remove noise and automatically extract the largest region of interest.

[0083] Further, before cropping the second image according to the preset mask image, a cropping distance can be preset, for example, 5 pixels outward from the non-zero pixel values in the preset mask image, to ensure the region of interest, such as the tissue around the tumor. Specifically, reference can be made to Figure 4 , Figure 4 which shows a schematic diagram of the second image before and after cropping provided by the embodiment of the present application.

[0084] Step S133, standardize each of the third images to obtain the corresponding fourth images.

[0085] It can be understood that images obtained by different scanners and scanning protocols may produce images with different directions and arrangements, that is, the directions and arrangements of the cropped third images are different. To avoid the adverse effects of this difference on the prediction results, each of the third images is respectively converted to the standard RAS+ direction, and the image intensity values are standardized by subtracting the mean and dividing by the standard deviation, so as to reduce the influence brought by these changes.

[0086] Step S134, for each of the fourth images, perform random transformation processing to obtain multiple fifth images, and perform label classification on each of the fifth images.

[0087] In this embodiment, the random transformation processing includes but is not limited to: RandomFlip, RandomAffine, RandomElasticDeformation, and RandomSwap of the image slices. It can be understood that by performing random transformation processing on the fourth image to generate more training data, that is, the fifth images, it can help the model generalize better.

[0088] Step S135, each of the fifth images and the classification labels respectively corresponding to each of the fifth images constitute the image dataset.

[0089] It can be understood that the classification structure labels include: achieving a pathological complete remission rate and not achieving a pathological complete remission rate.

[0090] Step S140, training the teacher-student network model based on the first data subset and the second data subset to obtain a target model.

[0091] It should be noted that the teacher-student network model includes a teacher network model and a student network model. The first data subset includes multiple post-treatment CT images and classification labels respectively corresponding to each post-treatment CT image. The second data subset includes multiple pre-treatment CT images and classification labels respectively corresponding to each pre-treatment CT image.

[0092] In this embodiment, the first data subset is divided into a first training set and a first test set according to a preset ratio, and the second data subset is divided into a second training set and a second test set. The teacher network model is trained based on the first training set and the first test set to obtain a target teacher network model. The student network model is trained based on the second training set and the second test set, and during the training process, the network parameters of the student network model are adjusted based on the output of the target teacher network model. Specifically, reference can be made to Figure 5 , Figure 5 which shows a training schematic diagram of the teacher-student network model provided by an embodiment of the present application.

[0093] In an implementation manner, the teacher-student network model includes: a teacher network model and a student network model. Training the teacher-student network model based on the first data subset and the second data subset to obtain a target model includes: dividing the first data subset into a first training set and a first test set according to a preset ratio; training the teacher network model based on the first training set and the first test set to obtain a target teacher network model. Training the student network model based on the second data subset and the target teacher network model to obtain the target model.

[0094] In this embodiment, before training the student network model, it is necessary to first train the teacher network model according to the first data subset to obtain a target teacher network model. Specifically, the first data subset is divided into a first training set and a first test set according to a preset ratio, such as 8:2, and the teacher network model is trained based on the first training set and the first test set to obtain a target teacher network model.

[0095] In one embodiment, training the student network model based on the second data subset and the target teacher network model to obtain the target model includes: dividing the second data subset into a second training set and a second test set according to a preset ratio; training the student network model based on the second training set and the second test set, and during the training process, updating the student network model based on the output of the target teacher network model, and determining the updated student network model as the target model.

[0096] In this embodiment, during the process of inputting the data of the second training set and the second test set into the student network model to train the student network model, the data is also simultaneously input into the target teacher network model. During the training process of the student network model, the parameters are also adjusted according to the output of the target teacher network model. Finally, the trained student network model is determined as the target model.

[0097] It can be understood that the target teacher network model learns the features of the post-treatment CT images, so as to correct the student network model that makes predictions only based on the pre-treatment CT images, and thus the trained target model can also achieve good prediction results only through the pre-treatment CT images.

[0098] Step S150: Obtain the image to be classified, and input the image to be classified into the target model to obtain the corresponding classification result.

[0099] In this embodiment, the image to be classified is a CT image. After inputting the image to be classified into the target model, the corresponding prediction probability value is obtained, and then the sigmoid function is used to implement binary classification to obtain the corresponding classification result, that is, reaching the pathological complete remission rate or not reaching the pathological complete remission rate.

[0100] It should be noted that before inputting the image to be classified into the target model, the image to be classified also needs to be preprocessed, specifically including: adjusting the image to be classified to a preset format, adjusting the window width of the image to be classified to a preset window width, adjusting the window level to a preset window level, adjusting the brightness to a preset brightness, and adjusting the pixel values to preset pixel values.

[0101] The image prediction tumor treatment reactivity classification method provided by the embodiments of this application obtains a basic network model, introduces a deep supervision strategy and a multi-layer perceptron structure into the basic network model to obtain an initial network model; determines a teacher-student network model according to the initial network model; obtains an image data set, where the image data set includes: a first data subset and a second data subset, the first data subset includes multiple first data, the second data subset includes multiple second data, and each of the second data corresponds to one of the first data; trains the teacher-student network model based on the first data subset and the second data subset to obtain a target model; obtains an image to be classified, and inputs the image to be classified into the target model to obtain a corresponding classification result. This application classifies and predicts the pathological complete remission situation through the reconstructed target model, improving the prediction accuracy of the pathological complete remission situation.

[0102] Embodiment 2

[0103] In addition, please refer to Figure 6 , the embodiments of the present invention also provide an image prediction tumor treatment reactivity classification device 600, and the device includes:

[0104] A first determination module 610, configured to obtain a basic network model, introduce a deep supervision strategy and a multi-layer perceptron structure into the basic network model to obtain an initial network model;

[0105] A second determination module 620, configured to determine a teacher-student network model according to the initial network model;

[0106] A data acquisition module 630, configured to obtain an image data set, where the image data set includes: a first data subset and a second data subset, the first data subset includes multiple first data, the second data subset includes multiple second data, and each of the second data corresponds to one of the first data;

[0107] A model training module 640, configured to train the teacher-student network model based on the first data subset and the second data subset to obtain a target model;

[0108] A classification module 650, configured to obtain an image to be classified, and input the image to be classified into the target model to obtain a corresponding classification result.

[0109] The image prediction tumor treatment reactivity classification device 600 provided by the embodiments of this application is used to implement the image prediction tumor treatment reactivity method described in Embodiment 1 above. To avoid repetition, it will not be elaborated here.

[0110] The imaging prediction tumor treatment responsiveness classification device provided by the embodiment of the present application obtains a basic network model through a first determination module, introduces a deep supervision strategy and a multi-layer perceptron structure into the basic network model to obtain an initial network model; a second determination module determines a teacher-student network model according to the initial network model; a data acquisition module acquires an imaging data set, and the imaging data set includes: a first data subset and a second data subset, the first data subset includes a plurality of first data, the second data subset includes a plurality of second data, and each of the second data corresponds to one of the first data; a model training module trains the teacher-student network model based on the first data subset and the second data subset to obtain a target model; a classification module acquires an imaging to be classified and inputs the imaging to be classified into the target model to obtain a corresponding classification result. The present application classifies and predicts the pathological complete remission situation through the reconstructed target model, improving the prediction accuracy of the pathological complete remission situation.

[0111] Embodiment 3

[0112] In addition, an embodiment of the present invention provides an electronic device, including a memory and a processor, and the memory stores a computer program, and when the computer program runs on the processor, it executes the imaging prediction tumor treatment responsiveness classification method described in Embodiment 1.

[0113] Specifically, please refer to Figure 7 , the electronic device 700 includes: a transceiver 701, a bus interface and a processor 702. The processor 702 is used to obtain a basic network model, introduce a deep supervision strategy and a multi-layer perceptron structure into the basic network model to obtain an initial network model; determine a teacher-student network model according to the initial network model; acquire an imaging data set, and the imaging data set includes: a first data subset and a second data subset, the first data subset includes a plurality of first data, the second data subset includes a plurality of second data, and each of the second data corresponds to one of the first data; train the teacher-student network model based on the first data subset and the second data subset to obtain a target model; acquire an imaging to be classified and input the imaging to be classified into the target model to obtain a corresponding classification result.

[0114] In the embodiment of the present invention, the electronic device 700 further includes: a memory 703. In Figure 7In this case, the bus architecture may include any number of interconnected buses and bridges, specifically, various circuits of one or more processors represented by processor 702 and a memory represented by memory 703 are linked together. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, etc., which are well known in the art, and thus will not be further described herein. The bus interface provides an interface. The transceiver 701 may be a plurality of components, that is, including a transmitter and a receiver, and provides a unit for communicating with various other devices on a transmission medium. The processor 702 is responsible for managing the bus architecture and general processing, and the memory 703 may store data used by the processor 702 when performing operations.

[0115] The electronic device 700 provided by an embodiment of the present invention may execute the imaging prediction tumor treatment reactivity classification method provided in the above-mentioned method embodiment 1. To avoid repetition, it will not be elaborated herein.

[0116] Embodiment 4

[0117] In addition, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the imaging prediction tumor treatment reactivity classification method described in Embodiment 1.

[0118] In this embodiment, the computer-readable storage medium may be a read-only memory (ROM for short), a random access memory (RAM for short), a magnetic disk, or an optical disc, etc.

[0119] The computer-readable storage medium provided in this embodiment can implement the imaging prediction tumor treatment reactivity classification method provided in Embodiment 1. To avoid repetition, it will not be elaborated herein.

[0120] In all the examples shown and described herein, any specific value should be construed as merely exemplary, rather than as a limitation. Therefore, other examples of the exemplary embodiments may have different values.

[0121] It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0122] The above-described embodiments merely represent several implementation manners of the present invention. Their descriptions are relatively specific and detailed, but should not be construed as a limitation on the scope of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention.

Claims

1. A classification method for predicting tumor treatment responsiveness using images, characterized in that: The method comprises: Acquire a basic network model, the basic network model includes: N residual modules and a fully connected layer, introduce a deep supervision strategy and a multi-layer perceptron structure into the basic network model, and obtain an initial network model; introduce a multi-layer perceptron structure into the basic network model, including: introducing the multi-layer perceptron structure before the fully connected layer, and the multi-layer perceptron structure is used to fuse at least one preset feature; Determine a teacher-student network model according to the initial network model, wherein the teacher-student network model includes: a teacher network model and a student network model; Acquire an image data set, the image data set comprising: a first data subset and a second data subset, the first data subset comprising a plurality of first data, the second data subset comprising a plurality of second data, each second data respectively corresponding to one first data, the first data subset comprising a plurality of post-treatment CT images and a classification label respectively corresponding to each post-treatment CT image, the second data subset comprising a plurality of pre-treatment CT images and a classification label respectively corresponding to each pre-treatment CT image; Training the teacher-student network model based on the first data subset and the second data subset to obtain a target model; Acquire an image to be classified, and input the image to be classified into the target model to obtain a corresponding classification result; The training of the teacher-student network model based on the first data subset and the second data subset to obtain a target model includes: dividing the first data subset into a first training set and a first test set according to a preset ratio; training the teacher network model based on the first training set and the first test set to obtain a target teacher network model; training the student network model based on the second data subset and the target teacher network model to obtain the target model; The method of training the student network model based on the second data subset and the target teacher network model to obtain the target model includes: dividing the second data subset into a second training set and a second test set according to a preset ratio; training the student network model based on the second training set and the second test set, and updating the student network model based on the output of the target teacher network model during the training process, and determining the updated student network model as the target model.

2. The image-based classification method for predicting tumor treatment responsiveness according to claim 1, characterized in that: The introducing a deep supervision strategy into the basic network model comprises: Adding an output layer to each of the residual modules; Determine the outputs of the output layer corresponding to the 1st to N-1th residual modules as intermediate outputs respectively, and determine the output of the output layer corresponding to the Nth residual module as the final output; Using a normalized exponential function to process each of the intermediate outputs and the final output respectively to obtain a corresponding soft label; Based on each of the soft labels, determining a final loss value according to a relative entropy divergence loss function; Determine a final loss function according to the final loss value; The basic network model is updated according to the final loss function.

3. The image-based classification method for predicting tumor treatment responsiveness according to claim 1, characterized in that: The obtaining of the image data set comprises: Acquire a plurality of first images, and perform image processing on each of the first images according to a preset processing method to obtain a corresponding second image; Using the maximum connected domain method, the second images of each institute are respectively cropped according to the corresponding preset mask image to obtain the corresponding third image; Performing standardization processing on each of the third images respectively to obtain a corresponding fourth image; For each of the fourth images, random transformation processing is performed to obtain a plurality of fifth images, and each of the fifth images is marked and classified; Each of the fifth images and the classification labels respectively corresponding to the fifth images constitute the image data set.

4. The image-based classification method for predicting tumor treatment responsiveness according to claim 3, characterized in that: The performing image processing on each of the first images according to a preset processing method to obtain a corresponding second image includes: Each of the first images is adjusted to a preset window width, a preset window level, a preset brightness and a preset pixel value, and then resampled to obtain the second image corresponding to each of the first images.

5. An image-based classification device for predicting tumor treatment responsiveness, characterized in that: The device comprises: The first determination module is used to obtain a basic network model, wherein the basic network model includes: N residual modules and a fully connected layer, and a deep supervision strategy and a multi-layer perceptron structure are introduced into the basic network model to obtain an initial network model; the multi-layer perceptron structure is introduced into the basic network model, including: introducing the multi-layer perceptron structure before the fully connected layer, and the multi-layer perceptron structure is used to fuse at least one preset feature; A second determining module is used to determine a teacher-student network model according to the initial network model, wherein the teacher-student network model includes: a teacher network model and a student network model; A data acquisition module, used to acquire an image data set, the image data set comprising: a first data subset and a second data subset, the first data subset comprising a plurality of first data, the second data subset comprising a plurality of second data, each second data respectively corresponding to one first data, the first data subset comprising a plurality of post-treatment CT images and classification labels respectively corresponding to each post-treatment CT image, the second data subset comprising a plurality of pre-treatment CT images and classification labels respectively corresponding to each pre-treatment CT image; A model training module, used for training the teacher-student network model based on the first data subset and the second data subset to obtain a target model; A classification module is used to obtain an image to be classified, and input the image to be classified into the target model to obtain a corresponding classification result; The training of the teacher-student network model based on the first data subset and the second data subset to obtain a target model includes: dividing the first data subset into a first training set and a first test set according to a preset ratio; training the teacher network model based on the first training set and the first test set to obtain a target teacher network model; training the student network model based on the second data subset and the target teacher network model to obtain the target model; The method of training the student network model based on the second data subset and the target teacher network model to obtain the target model includes: dividing the second data subset into a second training set and a second test set according to a preset ratio; training the student network model based on the second training set and the second test set, and updating the student network model based on the output of the target teacher network model during the training process, and determining the updated student network model as the target model.

6. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is run on the processor, the image prediction and tumor treatment responsiveness classification method according to any one of claims 1 to 4 is executed.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image prediction and tumor treatment responsiveness classification method according to any one of claims 1 to 4 is implemented.

Images