Prediction model construction method based on medical images, prediction method and device

Abstract

The embodiment of the present application discloses a prediction model construction method based on medical images, a prediction method and a device, and the prediction model construction method based on medical images includes: obtaining training sample annotation results, the training sample annotation results are obtained by binary classification annotation of the first image to be predicted of multiple sample users; performing image registration on the template image to be registered and the first image to be predicted to obtain the segmentation result of the first image to be predicted; the first image to be predicted, the training sample annotation results and the segmentation result of the first image to be predicted are input into a preset deep neural network based on multi-objective learning for training to construct the prediction model based on medical images. Using the prediction model based on medical images provided by the embodiment of the present application to predict Alzheimer's disease is conducive to improving the prediction accuracy of Alzheimer's disease.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a prediction model construction method based on medical images, a prediction method and a device. Background Art

[0002] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been widely used in various fields, such as common smart homes, virtual assistants, driverless cars, and smart medical care. In the medical field, Alzheimer's disease (AD) is a neurodegenerative disease that progresses slowly and worsens over time. Its diagnosis mostly relies on medical imaging such as magnetic resonance imaging (MRI) or positron emission computed tomography (PET). At present, the prediction of Alzheimer's disease based on medical images can be to divide the brain into different regions according to its anatomical structure, extract artificially designed features from each region, and then train the classifier to predict it, or directly use Figure 1 The 3D-CNN (3D Convolutional Neural Networks) model shown in the figure performs multiple convolution operations on the input MRI image (using a 3*3*3 convolution kernel and a 2*2*2 maximum pooling), connects the output results of the convolution operation, and then processes the MRI image through a fully connected layer and a classifier softmax to output the predicted result (the probability of having Alzheimer's disease P AD , normal probability P NC and the probability of mild cognitive impairment P MCI ). However, the former reduces the dimension of input features and loses some feature information, and the 3D-CNN prediction method ignores prior information. Both have low prediction accuracy for Alzheimer's disease. Summary of the invention

[0003] To solve the above problems, the present application provides a prediction model construction method based on medical images, a prediction method and a device. The prediction model based on medical images constructed in the present application is conducive to improving the prediction accuracy of Alzheimer's disease.

[0004] A first aspect of an embodiment of the present application provides a method for constructing a prediction model based on medical images, the method comprising:

[0005] Obtaining training sample annotation results, where the training sample annotation results are obtained by performing binary classification annotation on first to-be-predicted images of multiple sample users;

[0006] Performing image registration on the template image to be registered and the first image to be predicted to obtain a first segmentation result of the first image to be predicted;

[0007] The first image to be predicted, the training sample annotation result and the first segmentation result of the first image to be predicted are input into a preset deep neural network based on multi-objective learning for training to construct the prediction model based on medical images.

[0008] A second aspect of an embodiment of the present application provides a prediction method based on medical images, the method comprising:

[0009] Acquire an original medical image, and perform alignment and normalization processing on the original medical image to obtain a second image to be predicted;

[0010] Inputting the second image to be predicted into a pre-built feature extraction subnetwork of a prediction model based on medical images to perform a downsampling operation, and extracting feature information of the second image to be predicted;

[0011] Performing an upsampling operation on the second image to be predicted through the image segmentation subnetwork of the prediction model based on medical images, and outputting an image segmentation result of the second image to be predicted;

[0012] The feature information of the second image to be predicted is classified by the classifier of the prediction model based on medical images to obtain a prediction result of the second image to be predicted.

[0013] A third aspect of an embodiment of the present application provides a device for constructing a prediction model based on medical images, the device comprising:

[0014] A sample labeling module, used to obtain a training sample labeling result, wherein the training sample labeling result is obtained by performing binary classification labeling on first to-be-predicted images of multiple sample users;

[0015] An image registration module, used to perform image registration on the template image to be registered and the first image to be predicted to obtain a first segmentation result of the first image to be predicted;

[0016] A model building module is used to input the first image to be predicted, the training sample annotation result and the first segmentation result of the first image to be predicted into a preset deep neural network based on multi-objective learning for training, so as to build the prediction model based on medical images.

[0017] A fourth aspect of the embodiments of the present application provides a prediction device based on medical images, the device comprising:

[0018] An image acquisition module, used for acquiring an original medical image, and performing alignment and normalization processing on the original medical image to obtain a second image to be predicted;

[0019] A feature extraction module, used for inputting the second image to be predicted into a pre-built feature extraction sub-network of a prediction model based on medical images to perform a downsampling operation, so as to extract feature information of the second image to be predicted;

[0020] An image segmentation module, used for performing an upsampling operation on the second image to be predicted through the image segmentation subnetwork of the prediction model based on medical images, and outputting an image segmentation result of the second image to be predicted;

[0021] The result prediction module is used to classify the feature information of the second image to be predicted through the classifier of the prediction model based on medical images to obtain the prediction result of the second image to be predicted.

[0022] A fifth aspect of an embodiment of the present application provides an electronic device, the electronic device comprising an input device and an output device, and also comprising a processor adapted to implement one or more instructions; and

[0023] A computer storage medium storing one or more instructions, wherein the one or more instructions are suitable for being loaded by the processor and executing the steps in the method described in the first aspect or the second aspect.

[0024] A sixth aspect of an embodiment of the present application provides a computer storage medium, which stores one or more instructions, and the one or more instructions are suitable for being loaded by a processor and executing the steps in the method described in the first aspect or the second aspect above.

[0025] The embodiment of the present application obtains the training sample annotation results, which are obtained by performing binary classification annotation on the first images to be predicted of multiple sample users; this enables the deep neural network based on multi-objective learning to quickly learn classification when used for Alzheimer's disease prediction, that is, suffering from or will suffer from Alzheimer's disease in the near future, or not suffering from Alzheimer's disease. The template image to be registered is image-registered with the first image to be predicted to obtain the first segmentation result of the first image to be predicted; in this way, the first image to be predicted is segmented using the image registration method to obtain a standard segmentation image with higher segmentation accuracy. The first image to be predicted, the training sample annotation results, and the first segmentation result of the first image to be predicted are input into a preset deep neural network based on multi-objective learning for training to construct the prediction model based on medical images; in this way, the use of easy-to-learn binary classification annotation results and standard segmentation images with higher segmentation accuracy to train the deep neural network based on multi-objective learning can improve the prediction accuracy of the constructed prediction model based on medical images. It can be seen that the prediction model based on medical images, which is constructed by performing binary classification labeling on the first image to be predicted and using image registration to perform image segmentation, is beneficial to improving the prediction accuracy of Alzheimer's disease when used for the prediction of Alzheimer's disease. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 A schematic diagram of the structure of a 3D-CNN provided in an embodiment of the present application;

[0028] Figure 2a An application architecture diagram provided for an embodiment of the present application;

[0029] Figure 2b An example diagram of a construction framework of a medical image-based prediction model provided in an embodiment of the present application;

[0030] Figure 3 A flowchart of a method for constructing a prediction model based on medical images provided in an embodiment of the present application;

[0031] Figure 4 An example diagram of image registration provided in an embodiment of the present application;

[0032] Figure 5An example diagram of solving a spatial transformation relationship T provided in an embodiment of the present application;

[0033] Figure 6 An example diagram of an image segmentation result provided in an embodiment of the present application;

[0034] Figure 7 A flowchart of another method for building a prediction model based on medical images provided in an embodiment of the present application;

[0035] Figure 8 A schematic diagram of the structure of a deep neural network based on multi-objective learning provided in an embodiment of the present application;

[0036] Fig. 9 An example diagram of a separable convolution provided in an embodiment of the present application;

[0037] Fig.10 A schematic diagram of a flow chart of a prediction method based on medical images provided in an embodiment of the present application;

[0038] Fig.11 A schematic diagram of the structure of a device for building a prediction model based on medical images provided in an embodiment of the present application;

[0039] Fig.12 A schematic diagram of the structure of a prediction device based on medical images provided in an embodiment of the present application;

[0040] Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0042] The terms "including" and "having" and any variations thereof that appear in the specification, claims and drawings of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices. In addition, the terms "first", "second" and "third" are used to distinguish different objects, rather than to describe a specific order.

[0043] The embodiments of the present application provide a medical image-based prediction model construction scheme and a medical image-based prediction scheme to predict medical images using a medical image-based prediction model constructed by the construction scheme. Taking Alzheimer's disease as an example, the medical image-based prediction model provided by the present application has higher accuracy in predicting that a user has or will suffer from Alzheimer's disease in the near future. Of course, the medical image-based prediction model is not limited to Alzheimer's disease. As long as the predicted classification results are correlated with certain parts of the image, it can be used for image segmentation and classification prediction, such as Parkinson's disease, frontotemporal dementia and other brain diseases. Therefore, it can be seen that the scheme proposed in the present application has strong clinical significance in the medical field. The medical image-based prediction model construction scheme and the medical image-based prediction scheme can be applied to electronic devices, which include but are not limited to smart phones, desktop computers, tablet computers, servers, etc. Specifically, the electronic device can obtain medical images from medical imaging equipment to build a prediction model or directly make predictions, or can obtain medical images from open source data to build a prediction model or directly make predictions. For example: when building the medical image-based prediction model, the medical research laboratory can obtain relevant data from ADNI (Alzheimer's Disease Neuroimaging Initiative), and the electronic device will perform model building related operations; for another example: when examining patients, the hospital can collect medical images of patients through medical imaging equipment, and the electronic device will perform examination related operations through the constructed medical image-based prediction model, and so on.

[0044] First, the following examples are used in conjunction with the relevant drawings to introduce the network system architecture to which the solution of the embodiment of the present application may be applied. Figure 2a , Figure 2a An application architecture diagram provided for an embodiment of the present application, such as Figure 2aAs shown, it includes an electronic device, a medical imaging device and a database, wherein the electronic device and the medical imaging device and the database communicate through a wired or wireless network connection. The electronic device can be an electronic device in a medical research laboratory or an electronic device in a hospital examination room. As long as it is an electronic device capable of model building and image prediction, it can be applied to the embodiments of the present application; the medical imaging device can be an image acquisition device commonly used in medicine, for example, a magnetic resonance imaging device, a positron emission tomography device, an electronic computer tomography (Computed Tomography, CT) device, etc. The medical imaging device usually sends the collected medical image to the electronic device in real time, or sends the collected medical image only when receiving an upload instruction from the electronic device, or sends the collected medical image to the database for storage. The medical imaging device is not strictly limited to providing only the second image to be predicted. The database in this application can be an internal database (including local and cloud databases) or an external open source database. Similarly, the database is not strictly limited to providing only the first image to be predicted, and it can also provide the second image to be predicted. Studies have shown that although Alzheimer's disease can cause atrophy of the entire brain, the hippocampus is the first area to be damaged. In one embodiment, when predicting Alzheimer's disease, after the electronic device obtains the patient's brain area image captured by the medical imaging device, the pre-built medical image-based prediction model can segment the hippocampus area with high accuracy and output the classification prediction result.

[0045] In addition, the embodiment of the present application adopts Figure 2b The model building framework shown builds a prediction model based on medical images. The network structure first manually annotates and segments the sample images, takes the corresponding labels as the annotation results, and uses the annotation results, sample images and image segmentation results to train a deep neural network based on multi-objective learning to obtain a prediction model based on medical images.

[0046] The medical image-based prediction model construction scheme provided in the embodiment of the present application is more practical in the inspection or prediction scenarios of Alzheimer's disease, frontotemporal dementia, etc. compared with the current schemes of extracting artificially designed features, retraining classifiers for prediction, and using 3D-CNN models for prediction.

[0047] based on Figure 2a The application architecture shown and Figure 2b The overall network structure shown in FIG. 1 is a medical image-based prediction model construction method. The medical image-based prediction model construction method can be executed by an electronic device. Specifically, Alzheimer's disease is used as an example. Figure 3 , the method for constructing a prediction model based on medical images may include the following steps:

[0048] S31, obtaining training sample annotation results, wherein the training sample annotation results are obtained by performing binary classification annotation on first to-be-predicted images of multiple sample users.

[0049] In a specific embodiment of the present application, the sample user refers to the object to which the first image to be predicted belongs, for example: the experimental object to which the brain image data in the ADNI data set belongs, the examination object to which the brain image data stored in the hospital examination room belongs, etc., including the object suffering from or about to suffer from Alzheimer's disease in the near future, or the object not suffering from and will not suffer from Alzheimer's disease in the near future, and the first image to be predicted refers to the brain region image of the sample user collected by the medical imaging device. Binary classification labeling is to label the first image to be predicted that suffers from or will suffer from Alzheimer's disease in the near future with a specific label, and to label the first image to be predicted that does not suffer from and will not suffer from Alzheimer's disease in the near future with another specific label. For example, it is common to label the first image to be predicted that suffers from or will suffer from Alzheimer's disease in the near future with 1, and label the first image to be predicted that does not suffer from and will not suffer from Alzheimer's disease in the near future with 0. The sample labeling result refers to the above-mentioned labeling label of the first image to be predicted. The use of binary classification labeling makes the output of the constructed prediction model based on medical images only a pre-selected label or the probability of the label.

[0050] S32: Perform image registration on the template image to be registered and the first image to be predicted to obtain a first segmentation result of the first image to be predicted.

[0051] In a specific embodiment of the present application, the template image to be registered is a pre-selected standard image, and each area to be segmented in the brain is marked with a segmentation label. For example, if the first image to be predicted needs to be segmented into three areas, the pixels in each area will be assigned one of the three values ​​1, 2, and 3, and the value assigned to the pixel is the segmentation label of each area. The first segmentation result of the first image to be predicted refers to the segmentation image of each brain area obtained by the image registration method. Due to different acquisition conditions, the first image to be predicted may only reflect the characteristics of some areas in the brain, such as Figure 4 As shown in the figure, the rectangular frame can represent the entire brain area, and the irregular shape in the frame can represent several preset areas that need to be segmented from the brain area. To accurately segment several preset areas from the first image to be predicted, it is necessary to perform spatial transformation on the first image to be predicted and align the first image to be predicted with the template image to be registered. This is image registration. Aligning the first image to be predicted with the sample image to be registered is to define a similarity measure and find a spatial transformation relationship T (T is a matrix) so that the similarity of the two images after the spatial transformation is maximized, which can be expressed as:

[0052] S(T)=S(l1(x),l0(T(x)))

[0053] Among them, S represents the similarity measure, T represents the spatial transformation relationship, l1 represents the first image to be predicted, l0 represents the standard image to be registered, x represents the pixel point in the image, and the registration process is the process of seeking the optimal spatial transformation.

[0054] In a possible implementation, performing image registration on the template image to be registered and the first image to be predicted includes:

[0055] Extracting feature information from the template image to be registered and the first image to be predicted to form a feature space;

[0056] Determine a spatial transformation relationship T using an optimization algorithm according to the extracted feature space, so that the similarity between the template image to be registered and the first image to be predicted after the transformation reaches a preset value;

[0057] The segmentation labels on the template image to be registered are transformed to the first image to be predicted.

[0058] Specifically, the method for extracting feature information of the first image to be predicted and the template image to be registered is not limited here. For example, a U-shaped convolutional neural network can be used for extraction, or a residual network can be used for extraction, etc. The space where all feature information of the template image to be registered and the first image to be predicted exists is the feature space. An optimization algorithm is used in the feature space to determine a spatial transformation relationship T so that the similarity between the template image to be registered and the first image to be predicted reaches a maximum value after the transformation. The process of determining the spatial transformation relationship T can be as follows: Figure 5 As shown, firstly, an initial T is obtained, and then the first image to be predicted l1 is spatially transformed to obtain the transformed image. The optimization algorithm is used to evaluate the similarity between the transformed image and the template image to be registered. The formula is as follows:

[0059]

[0060] Among them, Ω is a set of all pixels in the feature space. The T that can minimize the value of the above formula is the optimal T. Otherwise, a new T is solved and the initial T is updated. Through the spatial transformation of the optimal T, the segmentation labels of each area on the template image to be registered can be transformed to the first image to be predicted, thereby obtaining the first segmentation result of the first image to be predicted. Figure 6 As shown, the first image to be predicted on the left is segmented by image registration method, and the first segmentation result obtained is the image shown on the right.

[0061] S33, inputting the first image to be predicted, the training sample annotation result and the first segmentation result of the first image to be predicted into a preset deep neural network based on multi-objective learning for training, so as to construct the prediction model based on medical images.

[0062] In a specific embodiment of the present application, after obtaining the first segmentation results of all the first images to be predicted in step S32, the first images to be predicted, the training sample annotation results and the first segmentation results of the first images to be predicted are input into a preset deep neural network based on multi-objective learning for training, and the backpropagation method is used to optimize the network parameters until the network parameters are fixed, thereby obtaining a constructed prediction model based on medical images.

[0063] It can be seen that the embodiment of the present application obtains the training sample annotation results, and the training sample annotation results are obtained by performing binary classification annotation on the first images to be predicted of multiple sample users; in this way, the deep neural network based on multi-objective learning can quickly learn classification when used for Alzheimer's disease prediction, that is, suffering from or will suffer from Alzheimer's disease in the near future, or not suffering from Alzheimer's disease. The template image to be registered is image-registered with the first image to be predicted to obtain the first segmentation result of the first image to be predicted; in this way, the first image to be predicted is segmented by the image registration method to obtain a standard segmentation image with higher segmentation accuracy. The first image to be predicted, the training sample annotation results, and the first segmentation result of the first image to be predicted are input into the preset deep neural network based on multi-objective learning for training to construct the prediction model based on medical images; in this way, the use of easy-to-learn binary classification annotation results and standard segmentation images with higher segmentation accuracy to train the deep neural network based on multi-objective learning can improve the prediction accuracy of the constructed prediction model based on medical images. It can be seen that the prediction model based on medical images, which is constructed by performing binary classification labeling on the first image to be predicted and using image registration to perform image segmentation, is beneficial to improving the prediction accuracy of Alzheimer's disease when used for the prediction of Alzheimer's disease.

[0064] See also Figure 7 , Figure 7 A flowchart of another method for constructing a prediction model based on medical images provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, steps S71-S76 are included:

[0065] S71, obtaining a training sample labeling result, wherein the training sample labeling result is obtained by performing binary classification labeling on first to-be-predicted images of multiple sample users;

[0066] S72, performing image registration on the template image to be registered and the first image to be predicted to obtain a first segmentation result of the first image to be predicted;

[0067] Among them, steps S71-S72 are Figure 3 The relevant description has been made in the illustrated embodiment, and will not be repeated here to avoid repetition.

[0068] S73, inputting the first image to be predicted into a feature extraction subnetwork of a deep neural network based on multi-objective learning to perform a downsampling operation, extracting feature information of the first image to be predicted, and inputting the extracted feature information into a classifier of the deep neural network based on multi-objective learning for classification processing, and outputting a classification result;

[0069] S74, inputting the extracted feature information into the image segmentation subnetwork of the multi-objective learning based deep neural network to perform an upsampling operation, and outputting a second segmentation result of the first image to be predicted;

[0070] In the specific embodiment of the present application, Figure 8 As shown, a preset deep neural network based on multi-objective learning is used to construct a prediction model based on medical images, wherein the deep neural network based on multi-objective learning includes three parts: a feature extraction subnetwork, an image segmentation subnetwork and a classifier. The feature extraction subnetwork is used to extract features of the input to be segmented, the image segmentation subnetwork performs image segmentation based on the feature information extracted by the feature extraction subnetwork, and the classifier performs classification prediction based on the feature information extracted by the feature extraction subnetwork.

[0071] In a possible implementation, inputting the first image to be predicted into the feature extraction subnetwork to perform a downsampling operation to extract feature information of the first image to be predicted includes:

[0072] Performing convolution processing on the first image to be predicted through a convolution block of the feature extraction subnetwork;

[0073] Performing maximum pooling processing on the feature map obtained through the convolution processing through a maximum pooling layer of the feature extraction subnetwork;

[0074] After alternating processing of M convolution blocks and N maximum pooling layers, feature information of the first image to be predicted is extracted, where M is equal to N+1.

[0075] like Figure 8As shown in , the feature extraction subnetwork consists of 5 convolution blocks. Each convolution block except the last one is connected to a max pooling layer, that is, 4 max pooling layers. The first image to be predicted is first convolved by a convolution block to obtain a feature map, and then passed through a max pooling layer for max pooling to reduce the length, width and height of the feature map to 1 / 2 of the original, so that the next convolution block can extract features within a larger receptive field. For the feature map whose length, width and height are reduced to 1 / 2 of the original, convolution processing is performed again through a convolution block to obtain another feature map, and then the length, width and height are reduced to 1 / 2 of the original through maximum pooling processing, until the last convolution block is convolution processed to extract the high-dimensional feature information of the first image to be predicted. The number of channels extracted by each convolution block is twice that of the previous convolution block. For example, if the number of channels of the feature map extracted by the first convolution block is 64, the following ones are 128, 256, 512, and 1024 respectively.

[0076] The classifier of the deep neural network based on multi-objective learning is composed of two fully connected layers, and its input is the high-dimensional feature information extracted by the feature extraction subnetwork. Before inputting the classifier, the high-dimensional feature information is flattened. For example, suppose the length, width, height and number of channels of the high-dimensional feature information are L, W, H, and C respectively. After flattening, each eigenvalue remains unchanged, but the position changes, and a one-dimensional vector with a length of L×W×H×C is obtained. The one-dimensional vector is integrated into a value through the convolution processing of the fully connected layer of the classifier and output, that is, the classification result of the first image to be predicted is obtained.

[0077] In a possible implementation, inputting the extracted feature information into the image segmentation subnetwork for upsampling, and outputting a second segmentation result of the first image to be predicted includes:

[0078] Performing up-convolution processing on the feature information extracted by the feature extraction sub-network through an up-convolution layer;

[0079] Merging the feature map obtained by up-convolution with the feature map of the same size extracted by the convolution block of the feature extraction subnetwork;

[0080] Performing convolution processing on the merged feature map through a convolution block of the image segmentation subnetwork;

[0081] After N upper convolution layers, N merging, and alternating processing of N convolution blocks, a second segmentation result of the first image to be predicted is output.

[0082] like Figure 8As shown in , the image segmentation subnetwork is composed of 4 convolution blocks, and its 4 convolution blocks correspond to the first 4 convolution blocks of the feature extraction subnetwork one by one. The feature maps processed by the corresponding two convolution blocks have the same size. The input of the image segmentation subnetwork is the high-dimensional feature information obtained by the feature extraction subnetwork. The high-dimensional feature information is first expanded to twice its original length, width and height through an up-convolution layer. For the feature map obtained after the size expansion, the maximum pooling operation of the maximum pooling layer in the feature extraction stage will lose the information of some pixels. In order to compensate for this part of the lost information, the feature map of the same size extracted by the corresponding convolution block of the feature extraction subnetwork is merged with the feature map obtained after the size expansion, and then the merged feature map is convolved through a convolution block. After 4 times of up-convolution-merging-convolution, the last convolution block outputs a segmented image of the same size as the first image to be predicted, that is, the second segmentation result of the first image to be predicted. Similarly, the number of channels of the features extracted by each convolution block is reduced to 1 / 2 of the previous one.

[0083] It should be noted that the convolution blocks in the above-mentioned feature extraction subnetwork and image segmentation subnetwork are in the form of superposition of 4 convolution layers. The convolution kernel size of the 1st and 3rd layers is 1×1×1, and the convolution kernel size of the 2nd and 4th layers is 3×3×3. Each convolution layer uses ReLU (rectified linear unit) as the activation function, and the maximum pooling and up-convolution windows are 2×2 with a step size of 2. In order to improve the operational efficiency of the deep neural network based on multi-objective learning, the convolution blocks in the embodiments of the present application use separable convolution in the 2nd and 4th layers when performing convolution processing, splitting a convolution operation into two steps. For example, Fig. 9 As shown in the figure, for a 12×12×3 feature map, firstly, three convolution kernels of size 5×5×1 are used to extract features on the cross section to obtain a 8×8×1 feature map. The three channels are superimposed to obtain a 8×8×3 feature map. Then, for the 8×8×3 feature map, a 1×1×3 convolution kernel is used to perform convolution to obtain a 8×8×1 feature map. This reduces the original 5×5×3=75 parameters to 5×5+3=28 parameters. If the feature map is 12×12×3, then the feature map is 8×8×3. The embodiment of the present application first uses a 3×3×1 convolution kernel for convolution to obtain a 10×10×3 feature map, and then convolves the 10×10×3 feature map with a 1×1×3 convolution kernel to obtain a 10×10×1 feature map, reducing the original 3×3×3=27 parameters to 3×3+3=12 parameters, and reducing the parameters of each convolution block by more than 50%, greatly improving the computational efficiency of the deep neural network based on multi-objective learning.

[0084] S75, constructing a loss function of the deep neural network based on multi-objective learning;

[0085] In a specific embodiment of the present application, the first image to be predicted is a brain region image of the sample user, and the loss function of the deep neural network based on multi-objective learning is constructed as follows:

[0086] A first loss function is constructed for the classification result of the brain region image of the sample user, and the first loss function is defined as:

[0087] L ce =-(ylog(p)+(1-y)log(1-p))

[0088] Among them, L ce represents the value of the first loss function, y represents the sample label, the sample label of the sample user who suffers from or will suffer from Alzheimer's disease within a preset time is 1, the sample label of the sample user who does not suffer from and will not suffer from Alzheimer's disease within a preset time is 0, and p represents the probability that the classification result of the brain region image is a positive sample;

[0089] A second loss function is constructed for the second segmentation result of the brain region image of the sample user, and the second loss function is defined as:

[0090]

[0091] Among them, T dice represents the value of the second loss function, Ω represents the brain region in the brain region image, l represents the hippocampus subregion or other subregions in the brain region, S1(l) represents the first segmentation result of the brain region image, S2(l) represents the second segmentation result of the brain region image, ω l represents the weight of each sub-region in the brain region;

[0092] The loss function of the deep neural network based on multi-objective learning is obtained according to the first loss function and the second loss function:

[0093]

[0094] Wherein, L represents the value of the loss function of the deep neural network based on multi-objective learning, and is the preset coefficient.

[0095] Among them, the first preset situation can be understood as having or will have Alzheimer's disease in the near future, and the second preset situation can be understood as not having and will not have Alzheimer's disease in the near future. Since the hippocampus subregion is the first subregion to be damaged among the subregions of the brain, the hippocampus subregion is the key region for image segmentation. In the second loss function, we set the weight of the hippocampus subregion to 5 and the weights of other subregions to 1. In addition, in the embodiment of the present application, Set to 1, Set to 2 to increase the impact of classification results on network training and ensure that the network takes classification accuracy as the main goal. The weighted sum of the first loss function and the second loss function is the loss function of the entire deep neural network based on multi-objective learning.

[0096] S76, according to the constructed loss function, using the Adam-based gradient descent method to optimize the network parameters of the deep neural network based on multi-objective learning until the output meets the expected classification result and the second segmentation result.

[0097] In a specific embodiment of the present application, the parameters of the network are optimized by the gradient descent method based on Adam in the back-propagation algorithm. The initial learning rate is determined to be 0.001, and is reduced to one-fifth after every 20 epochs. One epoch is to complete a forward propagation operation and a back-propagation operation for all the first images to be predicted. The batch size is determined to be 5, that is, 5 first images to be predicted are used for training in each iteration, and each iteration is a network parameter weight update. Each network parameter weight update requires 5 first images to be predicted to perform a forward operation to obtain the value of the entire network loss function, that is, repeat the process of feature information extraction, classification processing and image segmentation. After 100 epochs of training, the expected classification result and the second segmentation result are obtained. The network parameter weights of the entire network are fixed to obtain a constructed prediction model based on medical images.

[0098] The embodiment of the present application obtains the training sample annotation results, which are obtained by performing binary classification annotation on the first images to be predicted of multiple sample users; this enables the deep neural network based on multi-objective learning to quickly learn classification when used for Alzheimer's disease prediction, that is, suffering from or will suffer from Alzheimer's disease in the near future, or not suffering from Alzheimer's disease. The template image to be registered is image-registered with the first image to be predicted to obtain the first segmentation result of the first image to be predicted; in this way, the first image to be predicted is segmented using the image registration method to obtain a standard segmentation image with higher segmentation accuracy. The first image to be predicted, the training sample annotation results, and the first segmentation result of the first image to be predicted are input into a preset deep neural network based on multi-objective learning for training to construct the prediction model based on medical images; in this way, the use of easy-to-learn binary classification annotation results and standard segmentation images with higher segmentation accuracy to train the deep neural network based on multi-objective learning can improve the prediction accuracy of the constructed prediction model based on medical images. It can be seen that the prediction model based on medical images, which is constructed by performing binary classification labeling on the first image to be predicted and using image registration to perform image segmentation, is beneficial to improving the prediction accuracy of Alzheimer's disease when used for the prediction of Alzheimer's disease.

[0099] See also Fig.10 , Fig.10 A prediction method based on medical images is provided in an embodiment of the present application. The prediction method based on medical images can be used Figure 3 or Figure 7 The medical image-based prediction model constructed in the embodiment is implemented, and the method includes steps S1001-S1004:

[0100] S1001, acquiring an original medical image, and aligning and normalizing the original medical image to obtain a second image to be predicted;

[0101] In a specific embodiment of the present application, the original medical image is an image of the brain area of ​​the object captured by a medical imaging device. The object can be an experimental subject in a medical laboratory or a patient in a hospital, etc. After the original medical image is acquired, it is first aligned and normalized to obtain a second image to be predicted with better effect. The second image to be predicted refers to an image that is about to be classified and predicted using a prediction model based on medical images.

[0102] S1002, inputting the second image to be predicted into a pre-built feature extraction subnetwork of a prediction model based on medical images to perform a downsampling operation, and extracting feature information of the second image to be predicted;

[0103] In the specific embodiment of the present application, based on Figure 8The network structure shown also performs convolution processing on the second image to be predicted through a convolution block of the feature extraction subnetwork, and then performs maximum pooling processing on the feature map obtained through a maximum pooling layer of the feature extraction subnetwork, reducing the length, width, and height of the obtained feature map to 1 / 2 of the original. After alternating processing of 5 convolution blocks and 4 maximum pooling layers, the feature information of the second image to be predicted is extracted.

[0104] S1003, performing an upsampling operation on the second image to be predicted through the image segmentation subnetwork of the prediction model based on medical images, and outputting an image segmentation result of the second image to be predicted;

[0105] In the specific embodiment of the present application, based on Figure 8 The network structure shown in the figure also first performs up-convolution processing on the feature information of the second image to be predicted extracted by the feature extraction subnetwork through an up-convolution layer, and then merges the feature map obtained by the up-convolution processing with the feature map of the same size extracted by the convolution block of the feature extraction subnetwork, and then convolution processing is performed on the merged feature map through a convolution block of the image segmentation subnetwork. After 4 up-convolution layers, 4 mergers and alternating processing of 4 convolution blocks, a segmented image of the same size as the second image to be predicted is output as the image segmentation result of the second image to be predicted.

[0106] S1004: Classify the feature information of the second image to be predicted by the classifier of the prediction model based on medical images to obtain a prediction result of the second image to be predicted.

[0107] In a specific embodiment of the present application, the extracted feature information of the second image to be predicted is flattened to obtain a one-dimensional feature vector as the input of the classifier. After being processed by two fully connected layers of the classifier, the one-dimensional vector is integrated into a value for output. The value can be the probability p(a) of label a suffering from or about to suffer from Alzheimer's disease or the probability p(b) of label b in other cases. The output value is the prediction result. It should be noted that the feature information extraction and image segmentation in this embodiment also adopt the form of separable convolution.

[0108] The embodiment of the present application obtains the original medical image, aligns and normalizes the original medical image, and obtains the second image to be predicted; the second image to be predicted is input into the pre-constructed feature extraction subnetwork of the prediction model based on medical images for downsampling operation, and the feature information of the second image to be predicted is extracted; the second image to be predicted is upsampled by the image segmentation subnetwork based on the prediction model of medical images, and the image segmentation result of the second image to be predicted is output; the feature information of the second image to be predicted is classified by the classifier of the prediction model based on medical images, and the prediction result of the second image to be predicted is obtained. Since the above-mentioned prediction model based on medical images adopts binary classification annotation to obtain training samples and image registration method to perform accurate image segmentation in the training stage, and the network parameters of the deep neural network based on multi-objective learning are optimized after 100 epochs of training, therefore, using the prediction model based on medical images to predict Alzheimer's disease is beneficial to improve the prediction accuracy of Alzheimer's disease.

[0109] Based on the description of the above-mentioned medical image-based prediction model construction method embodiment, the present application embodiment also provides a medical image-based prediction model construction device, which can be a computer program (including program code) running in a terminal. The medical image-based prediction model construction device can execute Figure 3 or Figure 7 See the method shown in Fig.11 , the device comprises:

[0110] A sample labeling module 1101 is used to obtain a training sample labeling result, where the training sample labeling result is obtained by performing binary classification labeling on first to-be-predicted images of multiple sample users;

[0111] An image registration module 1102 is used to perform image registration on the template image to be registered and the first image to be predicted to obtain a first segmentation result of the first image to be predicted;

[0112] The model building module 1103 is used to input the first image to be predicted, the training sample annotation result and the first segmentation result of the first image to be predicted into a preset deep neural network based on multi-objective learning for training, so as to build the prediction model based on medical images.

[0113] In a possible implementation manner, the image registration module 1102 is specifically configured to:

[0114] Extracting feature information from the template image to be registered and the first image to be predicted to form a feature space;

[0115] Determine a spatial transformation relationship T using an optimization algorithm according to the extracted feature space, so that the similarity between the template image to be registered and the first image to be predicted after the transformation reaches a preset value;

[0116] The segmentation labels on the template image to be registered are transformed to the first image to be predicted.

[0117] In a possible implementation, the multi-objective learning-based deep neural network includes a feature extraction subnetwork, an image segmentation subnetwork, and a classifier; the model construction module 1103 inputs the first image to be predicted, the training sample annotation result, and the first segmentation result of the first image to be predicted into a preset multi-objective learning-based deep neural network for training, specifically for:

[0118] Inputting the first image to be predicted into the feature extraction subnetwork for downsampling operation, extracting feature information of the first image to be predicted, and inputting the extracted feature information into the classifier for classification processing, and outputting a classification result;

[0119] Inputting the extracted feature information into the image segmentation subnetwork for upsampling operation, and outputting a second segmentation result of the first image to be predicted;

[0120] Constructing a loss function of the deep neural network based on multi-objective learning;

[0121] According to the constructed loss function, the Adam-based gradient descent method is used to optimize the network parameters of the deep neural network based on multi-objective learning until the output meets the expected classification result and the second segmentation result.

[0122] In a possible implementation manner, the model building module 1103 is specifically used to: input the first image to be predicted into the feature extraction subnetwork for downsampling operation to extract feature information of the first image to be predicted;

[0123] Performing convolution processing on the first image to be predicted through a convolution block of the feature extraction subnetwork;

[0124] Performing maximum pooling processing on the feature map obtained through the convolution processing through a maximum pooling layer of the feature extraction subnetwork;

[0125] After alternating processing of M convolution blocks and N maximum pooling layers, feature information of the first image to be predicted is extracted, where M is equal to N+1.

[0126] In a possible implementation manner, the model building module 1103 is specifically used to input the extracted feature information into the image segmentation subnetwork for upsampling operation and output the second segmentation result of the first image to be predicted:

[0127] Performing up-convolution processing on the feature information extracted by the feature extraction sub-network through an up-convolution layer;

[0128] Merging the feature map obtained by up-convolution with the feature map of the same size extracted by the convolution block of the feature extraction subnetwork;

[0129] Performing convolution processing on the merged feature map through a convolution block of the image segmentation subnetwork;

[0130] After N upper convolution layers, N merging, and alternating processing of N convolution blocks, a second segmentation result of the first image to be predicted is output.

[0131] In a possible implementation manner, the first image to be predicted is a brain region image of the sample user, and the model construction module 1103 is specifically used to construct the loss function of the deep neural network based on multi-objective learning:

[0132] A first loss function is constructed for the classification result of the brain region image of the sample user, and the first loss function is defined as:

[0133] L ce =-(ylog(p)+(1-y)log(1-p))

[0134] Among them, L ce represents the value of the first loss function, y represents the sample label,

[0135] The sample label of the sample user who suffers from or will suffer from Alzheimer's disease within a preset time is 1, and the sample label of the sample user who does not suffer from and will not suffer from Alzheimer's disease within a preset time is 0, and p represents the probability that the classification result of the first brain region image is a positive sample;

[0136] A second loss function is constructed for the second segmentation result of the brain region image of the sample user, and the second loss function is defined as:

[0137]

[0138] Among them, L dicerepresents the value of the second loss function, Ω represents the brain region in the brain region image, l represents the hippocampus subregion or other subregions in the brain region, S1(l) represents the first segmentation result of the brain region image, S2(l) represents the second segmentation result of the brain region image, ω l represents the weight of each sub-region in the brain region;

[0139] The loss function of the deep neural network based on multi-objective learning is obtained according to the first loss function and the second loss function:

[0140]

[0141] Wherein, L represents the value of the loss function of the deep neural network based on multi-objective learning, and is the preset coefficient.

[0142] Based on the description of the above-mentioned prediction method embodiment based on medical images, the present application embodiment also provides a prediction device based on medical images, see Fig.12 , the device comprises:

[0143] The image acquisition module 1201 is used to acquire an original medical image, and perform alignment and normalization processing on the original medical image to obtain a second image to be predicted;

[0144] A feature extraction module 1202 is used to input the second image to be predicted into a pre-built feature extraction sub-network of a prediction model based on medical images to perform a downsampling operation, so as to extract feature information of the second image to be predicted;

[0145] An image segmentation module 1203 is used to perform an upsampling operation on the second image to be predicted through the image segmentation subnetwork of the prediction model based on medical images, and output an image segmentation result of the second image to be predicted;

[0146] The result prediction module 1204 is used to classify the feature information of the second image to be predicted by using the classifier of the medical image-based prediction model to obtain a prediction result of the second image to be predicted.

[0147] According to one embodiment of the present application, Fig.11 The device for constructing a predictive model based on medical images and Fig.12The various units in the medical image-based prediction device shown can be individually or completely combined into one or several other units to form, or one (some) of the units can be further divided into multiple functionally smaller units to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present invention, the medical image-based prediction model construction device and the medical image-based prediction device can also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0148] According to another embodiment of the present application, the program can be executed by running on a general computing device such as a computer including a central processing unit (CPU), a random access memory medium (RAM), a read-only memory medium (ROM), and other processing elements and storage elements. Figure 3 , Figure 7 or Fig.10 A computer program (including program code) for each step involved in the corresponding method shown in Fig.11 or Fig.12 The apparatus and device shown in the figure are used to implement the above method of the embodiment of the present invention. The computer program can be recorded on a computer-readable recording medium, for example, and loaded into the above computing device through the computer-readable recording medium and run therein.

[0149] Based on the description of the above method embodiment and device embodiment, the present invention also provides an electronic device. Fig.13 The electronic device at least includes a processor 1301, an input device 1302, an output device 1303, and a computer storage medium 1304. The processor 1301, the input device 1302, the output device 1303, and the computer storage medium 1304 in the electronic device may be connected via a bus or other means.

[0150] The computer storage medium 1304 may be stored in the memory of the electronic device, and the computer storage medium 1304 is used to store a computer program, and the computer program includes program instructions, and the processor 1301 is used to execute the program instructions stored in the computer storage medium 1304. The processor 1301 (or CPU (Central Processing Unit)) is the computing core and control core of the electronic device, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function.

[0151] In one embodiment, the processor 1301 of the electronic device provided in the embodiment of the present application can be used to perform a series of prediction model construction processes based on medical images, including:

[0152] Obtaining training sample annotation results, where the training sample annotation results are obtained by performing binary classification annotation on first to-be-predicted images of multiple sample users;

[0153] Performing image registration on the template image to be registered and the first image to be predicted to obtain a first segmentation result of the first image to be predicted;

[0154] The first image to be predicted, the training sample annotation result and the first segmentation result of the first image to be predicted are input into a preset deep neural network based on multi-objective learning for training to construct the prediction model based on medical images.

[0155] In one embodiment, the processor 1301 performs the image registration of the template image to be registered and the first image to be predicted, including:

[0156] Extracting feature information from the template image to be registered and the first image to be predicted to form a feature space;

[0157] Determine a spatial transformation relationship T using an optimization algorithm according to the extracted feature space, so that the similarity between the template image to be registered and the first image to be predicted after the transformation reaches a preset value;

[0158] The segmentation labels on the template image to be registered are transformed to the first image to be predicted.

[0159] In one embodiment, the multi-objective learning-based deep neural network includes a feature extraction subnetwork, an image segmentation subnetwork, and a classifier; the processor 1301 executes the step of inputting the first image to be predicted, the training sample annotation result, and the first segmentation result of the first image to be predicted into a preset multi-objective learning-based deep neural network for training, including:

[0160] Inputting the first image to be predicted into the feature extraction subnetwork for downsampling operation, extracting feature information of the first image to be predicted, and inputting the extracted feature information into the classifier for classification processing, and outputting a classification result;

[0161] Inputting the extracted feature information into the image segmentation subnetwork for upsampling operation, and outputting a second segmentation result of the first image to be predicted;

[0162] Constructing a loss function of the deep neural network based on multi-objective learning;

[0163] According to the constructed loss function, the Adam-based gradient descent method is used to optimize the network parameters of the deep neural network based on multi-objective learning until the output meets the expected classification result and the second segmentation result.

[0164] In one embodiment, the processor 1301 performs the downsampling operation of inputting the first image to be predicted into the feature extraction subnetwork to extract feature information of the first image to be predicted, including:

[0165] Performing convolution processing on the first image to be predicted through a convolution block of the feature extraction subnetwork;

[0166] Performing maximum pooling processing on the feature map obtained through the convolution processing through a maximum pooling layer of the feature extraction subnetwork;

[0167] After alternating processing of M convolution blocks and N maximum pooling layers, feature information of the first image to be predicted is extracted, where M is equal to N+1.

[0168] In one embodiment, the processor 1301 performs the upsampling operation of inputting the extracted feature information into the image segmentation subnetwork and outputting the second segmentation result of the first image to be predicted, including:

[0169] Performing up-convolution processing on the feature information extracted by the feature extraction sub-network through an up-convolution layer;

[0170] Merging the feature map obtained by up-convolution with the feature map of the same size extracted by the convolution block of the feature extraction subnetwork;

[0171] Performing convolution processing on the merged feature map through a convolution block of the image segmentation subnetwork;

[0172] After N upper convolution layers, N merging, and alternating processing of N convolution blocks, a second segmentation result of the first image to be predicted is output, including:

[0173] Performing up-convolution processing on the feature information extracted by the feature extraction sub-network through an up-convolution layer;

[0174] Merging the feature map obtained by up-convolution with the feature map of the same size extracted by the convolution block of the feature extraction subnetwork;

[0175] Performing convolution processing on the merged feature map through a convolution block of the image segmentation subnetwork;

[0176] After N upper convolution layers, N merging, and alternating processing of N convolution blocks, a second segmentation result of the first image to be predicted is output.

[0177] In one embodiment, the first image to be predicted is a brain region image of the sample user, and the processor 1301 executes the loss function of constructing the deep neural network based on multi-objective learning, including:

[0178] A first loss function is constructed for the classification result of the brain region image of the sample user, and the first loss function is defined as:

[0179] L ce =-(ylog(p)+(1-y)log(1-p))

[0180] Among them, L ce represents the value of the first loss function, y represents the sample label, the sample label of the sample user who suffers from or will suffer from Alzheimer's disease within a preset time is 1, the sample label of the sample user who does not suffer from and will not suffer from Alzheimer's disease within a preset time is 0, and p represents the probability that the classification result of the first brain region image is a positive sample;

[0181] A second loss function is constructed for the second segmentation result of the brain region image of the sample user, and the second loss function is defined as:

[0182]

[0183] Among them, T dice represents the value of the second loss function, Ω represents the brain region in the brain region image, l represents the hippocampus subregion or other subregions in the brain region, S1(l) represents the first segmentation result of the brain region image, S2(l) represents the second segmentation result of the brain region image, ω l represents the weight of each sub-region in the brain region;

[0184] The loss function of the deep neural network based on multi-objective learning is obtained according to the first loss function and the second loss function:

[0185]

[0186] Wherein, L represents the value of the loss function of the deep neural network based on multi-objective learning, and is the preset coefficient.

[0187] In one embodiment, the processor 1301 of the electronic device provided in the embodiment of the present application can be used to perform a series of prediction processes based on medical images, including:

[0188] Acquire an original medical image, and perform alignment and normalization processing on the original medical image to obtain a second image to be predicted;

[0189] Inputting the second image to be predicted into a pre-built feature extraction subnetwork of a prediction model based on medical images to perform a downsampling operation, and extracting feature information of the second image to be predicted;

[0190] Performing an upsampling operation on the second image to be predicted through the image segmentation subnetwork of the prediction model based on medical images, and outputting an image segmentation result of the second image to be predicted;

[0191] The feature information of the second image to be predicted is classified by the classifier of the prediction model based on medical images to obtain a prediction result of the second image to be predicted.

[0192] Exemplarily, the electronic device may be a computer, a laptop, a tablet computer, a PDA, a server, etc. The electronic device may include but is not limited to a processor 1301, an input device 1302, an output device 1303, and a computer storage medium 1304. Those skilled in the art will appreciate that the schematic diagram is merely an example of an electronic device and does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or different components.

[0193] It should be noted that since the processor 1301 of the electronic device implements the above-mentioned medical image-based prediction model construction method and the steps in the medical image-based prediction method when executing the computer program, the above-mentioned medical image-based prediction model construction method and the medical image-based prediction method embodiments are all applicable to the electronic device and can achieve the same or similar beneficial effects.

[0194] The embodiment of the present application also provides a computer storage medium (Memory), which is a memory device in an electronic device for storing programs and data. It is understandable that the computer storage medium here can include both the built-in storage medium in the terminal and the extended storage medium supported by the terminal. The computer storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor 1301 are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage; optionally, it can also be at least one computer storage medium located away from the aforementioned processor 1301. In one embodiment, the processor 1301 can load and execute one or more instructions stored in the computer storage medium to implement the above-mentioned corresponding steps of the prediction model construction method based on medical images and the prediction method based on medical images.

[0195] It should be noted that since the computer program of the computer storage medium implements the steps in the above-mentioned medical image-based prediction model construction method and the medical image-based prediction method when executed by the processor, all embodiments or implementations of the above-mentioned medical image-based prediction model construction method and the medical image-based prediction method are applicable to the computer-readable storage medium and can achieve the same or similar beneficial effects.

[0196] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for constructing a prediction model based on medical images, characterized in that: The method comprises: Obtaining training sample annotation results, where the training sample annotation results are obtained by performing binary classification annotation on first to-be-predicted images of multiple sample users; Performing image registration on the template image to be registered and the first image to be predicted to obtain a first segmentation result of the first image to be predicted; the template image to be registered includes a plurality of regions to be segmented, each of the regions to be segmented is marked with a segmentation label; Inputting the first image to be predicted, the training sample annotation result, and the first segmentation result of the first image to be predicted into a preset deep neural network based on multi-objective learning for training, so as to construct the prediction model based on medical images; The step of performing image registration on the template image to be registered and the first image to be predicted includes: extracting feature information from the template image to be registered and the first image to be predicted to form a feature space; and determining a spatial transformation relationship using an optimization algorithm based on the extracted feature space. T , so that the similarity between the template image to be registered and the first image to be predicted after the transformation reaches a preset value; transforming the segmentation label on the template image to be registered to the first image to be predicted; Among them, the deep neural network based on multi-objective learning includes a feature extraction subnetwork, an image segmentation subnetwork and a classifier; the first image to be predicted, the training sample annotation result and the first segmentation result of the first image to be predicted are input into a preset deep neural network based on multi-objective learning for training, including: inputting the first image to be predicted into the feature extraction subnetwork for downsampling operation, extracting feature information of the first image to be predicted, and inputting the extracted feature information into the classifier for classification processing, and outputting the classification result; inputting the extracted feature information into the image segmentation subnetwork for upsampling operation, and outputting the second segmentation result of the first image to be predicted; constructing the loss function of the deep neural network based on multi-objective learning; according to the constructed loss function, using the Adam-based gradient descent method to optimize the network parameters of the deep neural network based on multi-objective learning until the output meets the expected classification result and the second segmentation result.

2. The method according to claim 1, characterized in that The step of inputting the first image to be predicted into the feature extraction subnetwork for downsampling to extract feature information of the first image to be predicted includes: Performing convolution processing on the first image to be predicted through a convolution block of the feature extraction subnetwork; Performing maximum pooling processing on the feature map obtained through the convolution processing through a maximum pooling layer of the feature extraction subnetwork; After alternating processing of M convolution blocks and N maximum pooling layers, feature information of the first image to be predicted is extracted, where M is equal to N+1.

3. The method according to claim 1, characterized in that The step of inputting the extracted feature information into the image segmentation subnetwork for upsampling operation and outputting a second segmentation result of the first image to be predicted comprises: Performing up-convolution processing on the feature information extracted by the feature extraction sub-network through an up-convolution layer; Merging the feature map obtained by up-convolution with the feature map of the same size extracted by the convolution block of the feature extraction subnetwork; Performing convolution processing on the merged feature map through a convolution block of the image segmentation subnetwork; After N upper convolution layers, N merging, and alternating processing of N convolution blocks, a second segmentation result of the first image to be predicted is output.

4. The method according to any one of claims 1 to 3, characterized in that: The first image to be predicted is a brain region image of the sample user, and the loss function of the deep neural network based on multi-objective learning is constructed, including: A first loss function is constructed for the classification result of the brain region image of the sample user, and the first loss function is defined as: in, represents the value of the first loss function, represents a sample label, the sample label of the sample user who suffers from or will suffer from Alzheimer's disease within a preset time is 1, and the sample label of the sample user who does not suffer from and will not suffer from Alzheimer's disease within a preset time is 0, represents the probability that the classification result of the brain region image is a positive sample; A second loss function is constructed for the second segmentation result of the brain region image of the sample user, and the second loss function is defined as: in, represents the value of the second loss function, represents a brain region in the brain region image, represents a hippocampal subregion or other subregion of a brain region, represents a first segmentation result of the brain region image, represents a second segmentation result of the brain region image, represents the weight of each sub-region in the brain region; The loss function of the deep neural network based on multi-objective learning is obtained according to the first loss function and the second loss function: in, represents the value of the loss function of the deep neural network based on multi-objective learning, and is the preset coefficient.

5. A prediction method based on medical images using a prediction model based on medical images constructed by the method described in any one of claims 1 to 4, characterized in that: The method comprises: Acquire an original medical image, and perform alignment and normalization processing on the original medical image to obtain a second image to be predicted; Inputting the second image to be predicted into a pre-built feature extraction subnetwork of a prediction model based on medical images to perform a downsampling operation, and extracting feature information of the second image to be predicted; Performing an upsampling operation on the feature information of the second image to be predicted through the image segmentation subnetwork based on the prediction model of medical images, and outputting an image segmentation result of the second image to be predicted; The feature information of the second image to be predicted is classified by the classifier of the prediction model based on medical images to obtain a prediction result of the second image to be predicted.

6. A device for constructing a prediction model based on medical images, characterized in that: The device comprises: A sample labeling module, used to obtain a training sample labeling result, wherein the training sample labeling result is obtained by performing binary classification labeling on first to-be-predicted images of multiple sample users; An image registration module, used for performing image registration on the template image to be registered and the first image to be predicted to obtain a first segmentation result of the first image to be predicted; the template image to be registered includes a plurality of regions to be segmented, each of which is marked with a segmentation label; A model building module, used for inputting the first image to be predicted, the training sample annotation result and the first segmentation result of the first image to be predicted into a preset deep neural network based on multi-objective learning for training, so as to build the prediction model based on medical images; Among them, the image registration module is specifically used for performing image registration on the template image to be registered and the first image to be predicted: extracting feature information from the template image to be registered and the first image to be predicted to form a feature space; determining a spatial transformation relationship T using an optimization algorithm based on the extracted feature space, so that the similarity between the template image to be registered and the first image to be predicted after the transformation reaches a preset value; transforming the segmentation label on the template image to be registered to the first image to be predicted; Among them, the deep neural network based on multi-objective learning includes a feature extraction subnetwork, an image segmentation subnetwork and a classifier; the model construction module inputs the first image to be predicted, the training sample annotation result and the first segmentation result of the first image to be predicted into a preset deep neural network based on multi-objective learning for training, and is specifically used to: input the first image to be predicted into the feature extraction subnetwork for downsampling operation, extract feature information of the first image to be predicted, and input the extracted feature information into the classifier for classification processing, and output the classification result; input the extracted feature information into the image segmentation subnetwork for upsampling operation, and output the second segmentation result of the first image to be predicted; construct the loss function of the deep neural network based on multi-objective learning; according to the constructed loss function, use the Adam-based gradient descent method to optimize the network parameters of the deep neural network based on multi-objective learning until the output meets the expected classification result and the second segmentation result.

7. A prediction device based on medical images, characterized in that: The device comprises: An image acquisition module, used for acquiring an original medical image, and performing alignment and normalization processing on the original medical image to obtain a second image to be predicted; A feature extraction module, used for inputting the second image to be predicted into a feature extraction subnetwork of a pre-constructed prediction model based on medical images to perform a downsampling operation, and extracting feature information of the second image to be predicted; wherein the prediction model based on medical images is constructed using the method described in any one of claims 1 to 4; An image segmentation module, configured to perform an upsampling operation on the feature information of the second image to be predicted through the image segmentation subnetwork of the prediction model based on medical images, and output an image segmentation result of the second image to be predicted; The result prediction module is used to classify the feature information of the second image to be predicted through the classifier of the prediction model based on medical images to obtain the prediction result of the second image to be predicted.

8. An electronic device, comprising an input device and an output device, characterized in that: Also includes: a processor adapted to implement one or more instructions; as well as, A computer storage medium storing one or more instructions, wherein the one or more instructions are suitable for being loaded by the processor and executing the method according to any one of claims 1 to 5.

9. A computer storage medium, characterized in that The computer storage medium stores one or more instructions, and the one or more instructions are suitable for being loaded by a processor and executing the method according to any one of claims 1-5.

Images