Method for realizing brain image characteristic curative effect prediction processing based on characteristic similarity measurement and partial annotation data

Abstract

The invention relates to a method for realizing brain image characteristic curative effect prediction processing based on characteristic similarity measurement and partial annotation data. The method comprises the following steps: inputting brain image data; performing non-uniform deviation correction, denoising and gray normalization on the calibrated and uncalibrated brain images, and registering the brain images to a standard space; inputting the calibrated brain image and the uncalibrated brain image into a deep neural network for model classification and training processing so as to extract feature codes containing original image information; and respectively acquiring a supervised loss function and an unsupervised loss function aiming at a supervised training part and an unsupervised training part of the model through full connection layer feature coding of the deep neural network, so as to acquire a treatment response condition prediction value of a corresponding patient. The invention further relates to a corresponding device, a processor and a storage medium thereof. By adopting the method, the device, the processor and the storage medium of the invention, accurate curative effect prediction can be realized according to the brain image of the patient.

Description

Technical Field

[0001] The present invention relates to the field of semi-supervised learning, in particular to the field of medical imaging, and specifically refers to a method, device, processor and computer-readable storage medium thereof for realizing brain imaging feature efficacy prediction processing based on feature similarity measurement and partial annotated data. Background Art

[0002] The prediction of the efficacy of mental illness is very important for developing individualized treatment plans for patients and achieving accurate diagnosis and treatment. By using existing deep learning methods (such as convolutional neural networks, etc.), taking the patient's brain images as input data, and taking the efficacy indicators quantitatively described based on the patient's treatment records (including the response before and after treatment, etc.) as the model prediction target, and conducting supervised training, an individualized efficacy prediction model for patients can be obtained.

[0003] The training of deep learning models based on supervised training relies on large-scale, high-quality, calibrated data. In clinical practice, due to the cost of data recording and acquisition, high-quality, quantitative patient treatment record information paired with brain imaging data is difficult to obtain, which limits the training effect of supervised deep learning models and thus limits the performance of efficacy prediction models. On the other hand, it is relatively easy to obtain large quantities of unlabeled brain imaging data. Therefore, we can combine some high-quality calibrated data with larger-scale uncalibrated brain imaging data to design a model training strategy and further improve the performance of the efficacy prediction model. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a method, device, processor and computer-readable storage medium for predicting the therapeutic effect of brain image features based on feature similarity measurement and partial annotation data. It is mainly used to solve the following technical problems:

[0005] 1. Achieve accurate prediction of therapeutic efficacy based on the patient's brain images;

[0006] 2. Using a large amount of uncalibrated data to assist model training solves the problem of difficulty in obtaining high-quality calibrated data in efficacy prediction, and further improves the efficacy prediction performance of the model;

[0007] 3. In order to solve the problem that uncalibrated data is difficult to use, a new feature similarity measurement strategy is proposed. The data enhancement + maximum feature similarity training method for uncalibrated data is adopted to improve the feature extraction ability of the backbone model during training, thereby assisting the training of the efficacy prediction model.

[0008] In order to achieve the above-mentioned purpose, the method, device, processor and computer-readable storage medium thereof for implementing brain image feature therapeutic effect prediction processing based on feature similarity measurement and partial annotated data of the present invention are as follows:

[0009] The method for predicting the therapeutic effect of brain image features based on feature similarity measurement and partial annotated data has the following main features:

[0010] (1) Input brain image data;

[0011] (2) Both the calibrated and uncalibrated brain images were subjected to non-uniform bias correction, denoising, grayscale normalization, and registration to the standard space;

[0012] (3) Both the calibrated and uncalibrated brain images are input into a deep neural network for model classification and training, thereby extracting feature codes containing the original image information;

[0013] (4) By encoding the features of the fully connected layer of the deep neural network, a supervised loss function and an unsupervised loss function are obtained for the supervised training part and the unsupervised training part of the model, respectively, so as to obtain the predicted value of the treatment response of the corresponding patient.

[0014] Preferably, the following processing is specifically performed for the supervised training part of the model:

[0015] The calibrated data is input into the deep neural network, and the quantitative prediction value of the deep neural network for the treatment response is obtained from the output layer. The supervised loss function L is obtained by calculating the mean square error between the prediction value and the actual clinical quantitative treatment response record corresponding to the currently input calibrated data. s :

[0016]

[0017] Among them, m is the number of calibrated samples participating in the training; y i A true quantitative record of treatment response for clinical practice; is the predicted value of the neural network for the treatment response.

[0018] Preferably, the unsupervised training part of the model includes the following processing steps:

[0019] (1) For each uncalibrated patient brain image data, the traditional data enhancement method and another data enhancement method based on variational autoencoder and generative adversarial network are used to perform data enhancement processing;

[0020] (2) inputting the two brain image data obtained by two different data enhancement methods into the deep neural network respectively, and obtaining the feature codes of the two brain image data from the previous layer of the output layer of the neural network respectively;

[0021] (3) By calculating the difference between the feature encodings of the two brain image data, the unsupervised loss function L is designed. u , thereby optimizing the deep neural network.

[0022] Preferably, the unsupervised loss function L u Designed in the following way:

[0023] Note that each training batch contains n unlabeled data, the feature encoding dimension obtained by the previous layer of the deep neural network output layer is d, and the brain image feature encoding combinations obtained by two different data enhancement methods in the current batch are the data sets The specific process includes the following:

[0024] (a) First, calculate the Euclidean distance matrices A and B of X and Z respectively;

[0025] (b) Referring to the definition of matrix covariance, calculate the covariance of two Euclidean distance matrices A and B;

[0026] (c) Referring to the definition of Pearson correlation coefficient, the covariance of the obtained Euclidean distance matrices A and B is normalized;

[0027] (d) In order to minimize the difference between the brain image feature encodings obtained by two different data enhancement methods, the feature similarity between the two is increased as much as possible. Therefore, the definition is:

[0028] L u =-Corr

[0029] Among them, Corr is the covariance obtained by normalizing the Euclidean distance matrix A and B by variance.

[0030] Preferably, the model obtains the total model training loss function L through the supervised loss function and the unsupervised loss function:

[0031] L=L s +λL u

[0032] Among them, λ is the weight parameter of unsupervised training.

[0033] Preferably, the processing step (a) is as follows:

[0034] For the Euclidean distance matrix A, each element ai,j For two samples x in X i and x j The Euclidean distance between them; for the Euclidean distance matrix B, each element b i,j For two samples z in X i and z j The Euclidean distance between:

[0035] a i,j =||x i -x j || 2

[0036] b i,j =||z i -z j || 2

[0037] For the element A with coordinate position (i, j) in the Euclidean distance matrix A, i,j and the element B with coordinate position (i, j) in the Euclidean distance matrix B i,j Centralization is performed to remove the data offset within the Euclidean distance matrix.

[0038] Preferably, the processing step (b) is specifically to calculate the covariance between the two Euclidean distance matrices a and B in the following manner:

[0039]

[0040] If the two data sets X and Z are independent of each other, then Cov = 0; if the two data sets X and Z have a positive dependency, then Cov>0.

[0041] Preferably, the processing step (c) is specifically as follows:

[0042] Calculate the variance Vat(A) of the Euclidean distance matrix A of the data set X:

[0043]

[0044] Calculate the variance Var(B) of the Euclidean distance matrix B of the data set Z:

[0045]

[0046] Referring to the definition of Pearson's correlation coefficient, the feature similarity measure is defined as the covariance normalized by the variance:

[0047]

[0048] At this time, the value of the feature similarity measure is between -1 and 1. If the two data sets X and Z are independent of each other, Corr = 0.

[0049] The device for predicting and processing the therapeutic effect of brain image features based on feature similarity measurement and partial annotated data has the following main features: the device comprises:

[0050] a processor configured to execute computer executable instructions;

[0051] A memory stores one or more computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the method for predicting the therapeutic effect of brain imaging features based on feature similarity measurement and partial annotation data are implemented.

[0052] The processor for implementing brain image feature efficacy prediction processing based on feature similarity measurement and partially annotated data has the main feature that the processor is configured to execute computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the above-mentioned method for implementing brain image feature efficacy prediction processing based on feature similarity measurement and partially annotated data are implemented.

[0053] The main feature of the computer-readable storage medium is that a computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the above-mentioned method for predicting the therapeutic effect of brain imaging features based on feature similarity measurement and partial annotation data.

[0054] The method of predicting the efficacy of brain image features based on feature similarity measurement and partially labeled data was adopted. 2470 unlabeled brain images and 344 labeled data of admission / discharge CGI-SI (Clinical Global Evaluation Scale-Illness Severity) changes were recorded from Shanghai Mental Health Center. A ten-fold cross-validation experimental strategy was adopted to achieve an average prediction MAE of 4.2, which was higher than the supervised learning training effect using only labeled data (average MAE = 6.8) and the use of MSE as L u (average MAE = 5.7), using Pearson correlation coefficient as L u (Average MAE = 5.1). BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 The present invention is a processing flow chart of a method for implementing brain image feature efficacy prediction processing based on feature similarity measurement and partial annotated data. DETAILED DESCRIPTION

[0056] In order to more clearly describe the technical content of the present invention, further description is given below in conjunction with specific embodiments.

[0057] Before describing in detail embodiments according to the present invention, it should be noted that, hereinafter, the terms "comprises", "includes" or any other variations are intended to cover non-exclusive inclusion, whereby a process, method, article or apparatus comprising a series of elements includes not only these elements, but also other elements not explicitly listed or inherent to such process, method, article or apparatus.

[0058] See also Figure 1 As shown, the method for predicting the therapeutic effect of brain image features based on feature similarity measurement and partial annotated data includes the following steps:

[0059] (1) Input brain image data;

[0060] (2) Both the calibrated and uncalibrated brain images were subjected to non-uniform bias correction, denoising, grayscale normalization, and registration to the standard space;

[0061] (3) Both the calibrated and uncalibrated brain images are input into a deep neural network for model classification and training, thereby extracting feature codes containing the original image information;

[0062] (4) By encoding the features of the fully connected layer of the deep neural network, a supervised loss function and an unsupervised loss function are obtained for the supervised training part and the unsupervised training part of the model, respectively, so as to obtain the predicted value of the treatment response of the corresponding patient.

[0063] As a preferred embodiment of the present invention, the following processing is specifically performed for the supervised training part of the model:

[0064] The calibrated data is input into the deep neural network, and the quantitative prediction value of the deep neural network for the treatment response is obtained from the output layer. The supervised loss function L is obtained by calculating the mean square error between the prediction value and the actual clinical quantitative treatment response record corresponding to the currently input calibrated data. s :

[0065]

[0066] Among them, m is the number of calibrated samples participating in the training; y i A true quantitative record of treatment response for clinical practice; is the predicted value of the neural network for the treatment response.

[0067] As a preferred embodiment of the present invention, the unsupervised training part of the model includes the following processing steps:

[0068] (1) For each uncalibrated patient brain image data, the traditional data enhancement method and another data enhancement method based on variational autoencoder and generative adversarial network are used to perform data enhancement processing;

[0069] (2) inputting the two brain image data obtained by two different data enhancement methods into the deep neural network respectively, and obtaining the feature codes of the two brain image data from the previous layer of the output layer of the neural network respectively;

[0070] (3) By calculating the difference between the feature encodings of the two brain image data, the unsupervised loss function L is designed. u , thereby optimizing the deep neural network.

[0071] As a preferred embodiment of the present invention, the unsupervised loss function L u Designed in the following way:

[0072] Note that each training batch contains n unlabeled data, the feature encoding dimension obtained by the previous layer of the deep neural network output layer is d, and the brain image feature encoding combinations obtained by two different data enhancement methods in the current batch are the data sets The specific process includes the following:

[0073] (a) First, calculate the Euclidean distance matrices A and B of X and Z respectively;

[0074] (b) Referring to the definition of matrix covariance, calculate the covariance of two Euclidean distance matrices A and B;

[0075] (c) Referring to the definition of Pearson correlation coefficient, the covariance of the obtained Euclidean distance matrices A and B is normalized;

[0076] (d) In order to minimize the difference between the brain image feature encodings obtained by two different data enhancement methods, the feature similarity between the two is increased as much as possible. Therefore, the definition is:

[0077] L u =-Corr

[0078] Among them, Corr is the covariance obtained by normalizing the Euclidean distance matrix A and B by variance.

[0079] Preferably, the model obtains the total model training loss function L through the supervised loss function and the unsupervised loss function:

[0080] L=L s +λL u

[0081] Among them, λ is the weight parameter of unsupervised training.

[0082] As a preferred embodiment of the present invention, the processing process (a) is specifically as follows:

[0083] For the Euclidean distance matrix A, each element a i,j For two samples x in X i and x j The Euclidean distance between them; for the Euclidean distance matrix B, each element b i,j For two samples z in X i and z j The Euclidean distance between:

[0084] a i,j =||x i -x j || 2

[0085] b i,j =||z i -z j || 2

[0086] For the element A with coordinate position (i, j) in the Euclidean distance matrix A, i,j and the element B with coordinate position (i, j) in the Euclidean distance matrix B i,j Centralization is performed to remove the data offset within the Euclidean distance matrix.

[0087] As a preferred embodiment of the present invention, the processing process (b) is specifically to calculate the covariance between the two Euclidean distance matrices A and B in the following manner:

[0088]

[0089] If the two data sets X and Z are independent of each other, then Cov = 0; if the two data sets X and Z have a positive dependency, then Cov>0.

[0090] As a preferred embodiment of the present invention, the processing step (c) is specifically as follows:

[0091] Calculate the variance Var(A) of the Euclidean distance matrix A of the data set X:

[0092]

[0093] Calculate the variance Var(B) of the Euclidean distance matrix B of the data set Z:

[0094]

[0095] Referring to the definition of Pearson's correlation coefficient, the feature similarity measure is defined as the covariance normalized by the variance:

[0096]

[0097] At this time, the value of the feature similarity measure is between -1 and 1. If the two data sets X and Z are independent of each other, Corr = 0.

[0098] In practical applications, this technical solution adopts a semi-supervised training strategy, which combines supervised training and unsupervised training.

[0099] Considering the actual clinical application scenario, the available data include brain imaging data that matches the quantitative efficacy records and brain imaging data without corresponding efficacy records. Brain imaging data with matching efficacy records are used as supervised training data, and brain imaging data without corresponding records are used as unsupervised training data.

[0100] Specifically, Figure 1 As shown, the core of the model used for classification is a deep neural network (including but not limited to convolutional neural networks such as ResNet), which: 1) accepts brain images as input; 2) the previous layer of the output layer is a multi-layer fully connected neural network, so the feature code containing the original image information can be extracted from this layer; 3) the output layer is a quantitative treatment response regression prediction value (such as the change value of the PANSS score corresponding to schizophrenia).

[0101] In the supervised training part of the model, the calibrated data is input into the neural network, and the quantitative prediction value of the neural network for the treatment response is obtained from the output layer. By calculating the mean square error between the predicted value and the actual clinical quantitative treatment response record corresponding to the current input calibrated data, the supervised loss function L can be obtained. s :

[0102]

[0103] Among them, m is the number of calibrated samples participating in the training; y i A true quantitative record of treatment response for clinical practice; is the predicted value of the neural network for the treatment response.

[0104] In the unsupervised training part of the model, the purpose of using uncalibrated data is to enhance the feature extraction ability of the backbone model (i.e., the deep neural network in the figure) without the help of supervised information. Brain images are complex three-dimensional images, which not only contain image features related to mental illness and efficacy, but also include noise, differences caused by scanners, differences in age, gender, and other heterogeneity between individual patients. This technical solution uses uncalibrated data to enable the deep neural network to exclude interference caused by other influences and focus on the extraction of efficacy-related features. Specifically, for each uncalibrated patient data, data enhancement is first performed by two different data enhancement methods, including traditional noise addition, shear transformation, etc., as well as data enhancement methods based on variational autoencoders, generative adversarial networks, etc. The two brain image data obtained by the two different data enhancement methods are respectively input into the deep neural network, and the feature encoding of the two brain image data can be obtained from the previous layer of the output layer of the network. Ideally, if the deep neural network focuses entirely on the extraction of efficacy-related features, the feature encoding of the brain image data calculated by the network model should only contain efficacy-related features, and should not contain other noise, age, gender, and other effects. Therefore, the feature encoding of brain image data obtained by two different data enhancement methods should be exactly the same, although the image data itself is different due to data enhancement. Based on this theory, the unsupervised loss function L can be designed by calculating the difference between the feature encodings of brain image data obtained by two different data enhancement methods. u , in order to optimize the backbone network (deep neural network).

[0105] Intuitively, L u It can be defined using methods such as mean square error and Pearson correlation. In practical applications, the feature coding obtained by the neural network obeys a certain distribution, and the importance of each dimension of the feature coding is different (because it involves the weighting of the fully connected layer). Using mean square error directly applied to the feature coding level will lead to poor model convergence effect; using Pearson correlation based on linear correlation cannot guide the training of the model when the two feature codes are nonlinearly correlated. Therefore, the present invention designs a feature similarity measurement loss function that can describe linear and nonlinear correlations. Note that each training batch (batch, that is, the training data that is input into the network model at one time and simultaneously during the deep neural network training process) contains n unlabeled data, the feature coding dimension obtained by the previous layer of the deep neural network output layer is d, and the brain image feature coding combinations obtained by two different data enhancement methods in the current batch are respectively First, calculate the Euclidean distance matrices A and B of X and Z respectively:

[0106] For the Euclidean distance matrix A, each element ai,j are two samples x in X i and x j The Euclidean distance (two-norm) between them; for the Euclidean distance matrix B, each element b i,j are two samples z in X i and z j The Euclidean distance (two-norm) between:

[0107] a i,j =||x i -x j || 2

[0108] b i,j =||z i -z j || 2

[0109] The establishment of the above Euclidean distance matrix can characterize the relative relationship between samples in X and Z, so as to obtain more complex linear and nonlinear dependencies between X and Z in subsequent calculations. Since the calculation of the above Euclidean distance matrices A and B depends on the bi-norm, the calculated values ​​are the absolute distances between samples rather than the relative distances. Therefore, the Euclidean distance matrices A and B need to be centered:

[0110] Taking the Euclidean distance matrix A as an example, first calculate the mean of each row of matrix A:

[0111]

[0112] Then calculate the mean of each column of matrix A:

[0113]

[0114] Next we calculate the mean of the elements of the entire matrix:

[0115]

[0116] Finally, using the above calculation results, the element A with coordinate position (i, j) in matrix A is i,j Centralization:

[0117]

[0118] For the element B with coordinate position (i,j) in matrix B i,j The steps for centralization are the same as above.

[0119] Through centralization, the data offset within the Euclidean distance matrix is ​​removed. Each element in the matrix represents the relative distance between samples, which is more conducive to the subsequent calculation of the dependency relationship between samples in the two data sets X and Z.

[0120] Referring to the definition of matrix covariance, calculate the covariance of two Euclidean distance matrices A and B:

[0121]

[0122] The covariance Cov of the Euclidean distance matrix A and B reflects the dependency of the two data sets X and Z. If the two data sets X and Z are independent of each other, Cov = 0; if there is a positive dependency between the two data sets X and Z, Cov>0.

[0123] The covariance Cov calculated above has a value range of [-∞,+∞]. As a measure of similarity (similar to the Pearson correlation coefficient), the feature similarity measure should be dimensionless and should be a normalized value, so Cov needs to be normalized. First, calculate the variance of the Euclidean distance matrix A of the data set X:

[0124]

[0125] The calculation of the variance Var(B) of the Euclidean distance matrix B of the data set Z itself is the same as above.

[0126] Referring to the definition of Pearson's correlation coefficient, the feature similarity measure is defined as the covariance normalized by the variance:

[0127]

[0128] At this time, the value of the feature similarity measure is between -1 and 1. If the two data sets X and Z are independent of each other, Corr = 0.

[0129] As mentioned above, the difference between the feature encodings of brain image data obtained by two different data enhancement methods should be as small as possible, that is, the similarity should be as high as possible. Therefore, it is defined as:

[0130] L u =-Corr

[0131] The model adjusts the parameters of the backbone neural network model based on supervised training and unsupervised training. The total model training loss function L is:

[0132] L=L s +λL u

[0133] Among them, λ is the weight parameter of unsupervised training, which determines the contribution of unsupervised training to model parameter training. According to experiments, λ can be taken as 0.5 first, or adjusted according to the actual training effect.

[0134] When the backbone neural network model converges after training, the model parameters are fixed, and new patient brain imaging data is directly input into the backbone neural network to output individualized and quantitative efficacy prediction results.

[0135] The device for predicting and processing the therapeutic effect of brain image features based on feature similarity measurement and partial annotated data, wherein the device comprises:

[0136] a processor configured to execute computer executable instructions;

[0137] A memory stores one or more computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the method for predicting the therapeutic effect of brain imaging features based on feature similarity measurement and partial annotation data are implemented.

[0138] The processor for implementing brain image feature efficacy prediction processing based on feature similarity measurement and partial annotated data, wherein the processor is configured to execute computer executable instructions, and when the computer executable instructions are executed by the processor, the steps of the method for implementing brain image feature efficacy prediction processing based on feature similarity measurement and partial annotated data are implemented.

[0139] The computer-readable storage medium stores a computer program thereon, and the computer program can be executed by a processor to implement the steps of the above-mentioned method for predicting the therapeutic effect of brain imaging features based on feature similarity measurement and partial annotation data.

[0140] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0141] It should be understood that each part of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution device.

[0142] A person of ordinary skill in the art may understand that all or part of the steps of the method for implementing the above-mentioned embodiment may be completed by instructing the relevant hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one of the steps of the method embodiment or a combination thereof.

[0143] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0144] In the description of this specification, the description with reference to the terms "an embodiment", "some embodiments", "example", "specific example", or "embodiment" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0145] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

[0146] The method of predicting the efficacy of brain image features based on feature similarity measurement and partially labeled data was adopted. 2470 unlabeled brain images and 344 labeled data of admission / discharge CGI-SI (Clinical Global Evaluation Scale-Illness Severity) changes were recorded from Shanghai Mental Health Center. A ten-fold cross-validation experimental strategy was adopted to achieve an average prediction MAE of 4.2, which was higher than the supervised learning training effect using only labeled data (average MAE = 6.8) and the use of MSE as L u (average MAE = 5.7), using Pearson correlation coefficient as L u (Average MAE = 5.1).

[0147] In this specification, the present invention has been described with reference to specific embodiments thereof. However, it is apparent that various modifications and variations may be made without departing from the spirit and scope of the present invention. Therefore, the specification and drawings should be regarded as illustrative rather than restrictive.

Claims

1. A method for predicting the therapeutic effect of brain imaging features based on feature similarity measurement and partial annotated data, characterized in that: The method comprises the following steps: (1) Input brain image data; (2) Both the calibrated and uncalibrated brain images were subjected to non-uniform bias correction, denoising, grayscale normalization, and registration to the standard space; (3) Both the calibrated and uncalibrated brain images are input into a deep neural network for model classification and training, thereby extracting feature codes containing the original image information; (4) By encoding the features of the fully connected layer of the deep neural network, a supervised loss function and an unsupervised loss function are obtained for the supervised training part and the unsupervised training part of the model, respectively, so as to obtain the predicted value of the treatment response of the corresponding patient.

2. The method for predicting the therapeutic effect of brain imaging features based on feature similarity measurement and partial annotated data according to claim 1, characterized in that: The supervised training part of the model is processed as follows: The calibrated data is input into the deep neural network, and the quantitative prediction value of the deep neural network for the treatment response is obtained from the output layer. The supervised loss function L is obtained by calculating the mean square error between the prediction value and the actual clinical quantitative treatment response record corresponding to the currently input calibrated data. s : Among them, m is the number of calibrated samples participating in training; y i To provide a true quantitative record of treatment response in clinical practice; is the predicted value of the neural network for the treatment response.

3. The method for predicting the therapeutic effect of brain imaging features based on feature similarity measurement and partial annotated data according to claim 2, characterized in that: The unsupervised training part of the model includes the following processes: (1) For each uncalibrated patient brain image data, the traditional data enhancement method and another data enhancement method based on variational autoencoder and generative adversarial network are used to perform data enhancement processing; (2) inputting the two brain image data obtained by two different data enhancement methods into the deep neural network respectively, and obtaining the feature codes of the two brain image data from the previous layer of the output layer of the neural network respectively; (3) By calculating the difference between the feature encodings of the two brain image data, the unsupervised loss function L is designed. u , thereby optimizing the deep neural network.

4. The method for predicting the therapeutic effect of brain imaging features based on feature similarity measurement and partial annotated data according to claim 3, characterized in that: The unsupervised loss function L u Designed in the following way: Note that each training batch contains n unlabeled data, the feature encoding dimension obtained by the previous layer of the deep neural network output layer is d, and the brain image feature encoding combinations obtained by two different data enhancement methods in the current batch are the data sets The specific process includes the following: (a) First, calculate the Euclidean distance matrices A and B of X and Z respectively; (b) Referring to the definition of matrix covariance, calculate the covariance of two Euclidean distance matrices A and B; (c) referring to the definition of Pearson correlation coefficient, the covariance of the obtained Euclidean distance matrices A and B is normalized; (d) In order to minimize the difference between the brain image feature encodings obtained by two different data enhancement methods, the feature similarity between the two is increased as much as possible. Therefore, the definition is: L u =-Corr Among them, Corr is the covariance obtained by normalizing the Euclidean distance matrix A and B by variance.

5. The method for predicting the therapeutic effect of brain imaging features based on feature similarity measurement and partial annotated data according to claim 4, characterized in that: The model obtains the total model training loss function L through the supervised loss function and the unsupervised loss function: L=L s +λL u Among them, λ is the weight parameter of unsupervised training.

6. The method for predicting the therapeutic effect of brain imaging features based on feature similarity measurement and partial annotated data according to claim 4, characterized in that: The specific processing process (a) is as follows: For the Euclidean distance matrix A, each element a i，j For two samples x in X i and x j The Euclidean distance between them; for the Euclidean distance matrix B, each element b i，j For two samples z in X i and z j The Euclidean distance between: a i，j =||x i -x j || 2 b i，j =||z i -z j || 2 For the element A with coordinate position (i, j) in the Euclidean distance matrix A, i，j and the element B with coordinate position (i, j) in the Euclidean distance matrix B i，j Centralization is performed to remove the data offset within the Euclidean distance matrix.

7. The method for predicting the therapeutic effect of brain image features based on feature similarity measurement and partial annotated data according to claim 6, characterized in that: Process (b) is to calculate the covariance between two Euclidean distance matrices A and B in the following manner: If the two data sets X and Z are independent of each other, then Cov = 0; if the two data sets X and Z have a positive dependency, then Cov>0.

8. The method for predicting the therapeutic effect of brain imaging features based on feature similarity measurement and partial annotated data according to claim 7, characterized in that: The specific processing step (c) is as follows: Calculate the variance Var(A) of the Euclidean distance matrix A of the data set X: Calculate the variance Var(B) of the Euclidean distance matrix B of the data set Z: Referring to the definition of Pearson's correlation coefficient, the feature similarity measure is defined as the covariance normalized by the variance: At this time, the value of the feature similarity measure is between -1 and 1. If the two data sets X and Z are independent of each other, Corr = 0.

9. A device for predicting the therapeutic effect of brain image features based on feature similarity measurement and partial annotated data, characterized in that: The device comprises: a processor configured to execute computer executable instructions; A memory storing one or more computer executable instructions, wherein when the computer executable instructions are executed by the processor, the steps of the method for predicting the therapeutic effect of brain imaging features based on feature similarity measurement and partial annotated data as described in any one of claims 1 to 8 are implemented.

10. A processor for predicting the therapeutic effect of brain image features based on feature similarity measurement and partial annotated data, characterized in that: The processor is configured to execute computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the method for predicting the therapeutic effect of brain imaging features based on feature similarity measurement and partially labeled data as described in any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the method for predicting the therapeutic effect of brain imaging features based on feature similarity measurement and partial annotation data as described in any one of claims 1 to 8.

Images