Tri-unet brain age prediction method based on different scale features of MRI
By combining deep and shallow features using the Tri-UNet model, the problem of insufficient utilization of multi-scale features in existing technologies is solved, achieving accuracy and visualization of brain age prediction and improving prediction accuracy.
Patent Information
- Application Number
- CN202210665816.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-13
- Publication Date
- 2026-01-20
- Estimated Expiration
- 2042-06-13
AI Technical Summary
Existing brain age prediction methods based on UNet networks fail to fully utilize multi-scale features, resulting in low prediction accuracy and an inability to visualize brain aging.
The Tri-UNet model is adopted, which combines deep and shallow features. Through the multi-layer encoding and decoding structure and residual structure of Tri-UNet, the feature information fusion is enhanced. The brain age is predicted using a three-branch residual block and an age prediction network (ResSame).
It improves the accuracy and visualization of brain age prediction, and evaluates model performance using metrics such as MAE, RMSE, r, and R2.
Smart Images
Figure CN115187513B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the field of medical image classification, and particularly relates to a Tri-UNet brain age prediction method based on different scale features of magnetic resonance imaging (MRI). BACKGROUND
[0002] In the current research, the biological age predicted by machine learning or deep learning methods and neuroimaging data is called brain predicted age. First, a calculation model is constructed by using neuroimaging data of a healthy population, which is regarded as the aging trajectory of a healthy brain, and the severity of brain aging of an individual is judged by the deviation degree of the individual from the aging trajectory, so as to judge the severity of cognitive aging or the risk of suffering from age-related brain diseases.
[0003] The research on brain age prediction aims to help individuals measure the degree of brain aging and screen the risk of suffering from diseases, and help individuals suffering from age-related diseases to further diagnose brain atrophy lesions caused by diseases. Therefore, an accurate brain age prediction method needs to be constructed to obtain more refined brain aging conditions. Some works attempt to predict brain age at the voxel level to realize the visualization of brain aging conditions. However, in the method for predicting brain age by using U-Net, the regional information between different feature layers is not fully combined, resulting in low prediction accuracy.
[0004] The mainstream method for predicting brain age at present is a deep learning method based on ResNet, and there are few methods for predicting brain age at the voxel level based on UNet network. The residual structure in ResNet can convert end-to-end feature mapping into residual mapping, so as to better learn the feature information of the front and back layers of the network and improve the prediction accuracy. The multi-layer encoding and decoding structure of U-Net can combine deep features and shallow features, so as to better learn the context semantic information of the input features, improve the prediction accuracy, and also learn refined features.
[0005] In summary, the difficulties of brain age prediction are as follows:
[0006] 1. The method for predicting brain age by using UNet network does not fully utilize multi-scale features;
[0007] 2. The whole brain age prediction cannot realize the visualization of brain aging conditions.
[0008] The limitations of the existing brain age prediction method based on UNet are as follows:
[0009] 1. The information fusion between different feature layers is not sufficient;
[0010] 2. The characteristics of different regions are ignored in whole-brain age prediction, which may represent different information, and the prediction accuracy is low. SUMMARY
[0011] In view of the above defects or improvement needs of the prior art, the present application provides a Tri-UNet brain age prediction method based on MRI different scale features, which can fully combine deep features and shallow features, better learn the context semantic information of input features, and thus improve the brain age prediction accuracy.
[0012] The present application is achieved by the following technical measures.
[0013] The present application provides a Tri-UNet brain age prediction method based on MRI different scale features, which comprises the following steps:
[0014] (1) Pre-process the image: use the full-automatic processing flow recon-all of the medical image processing software FreeSurfer to pre-process the original image, including skull stripping, image correction, image registration, image segmentation, spatial normalization, and spatial smoothing processing.
[0015] (2) Crop the image: due to the limitation of server GPU memory, crop the whole brain image into a smaller image block, and then use the cropped image block as the model input. If the server GPU memory is sufficient, this step can be omitted, and the whole brain image can be used as the model input, and the prediction performance will be better.
[0016] (3) Image alignment: map the age label value into an image block of the same size as the cropped image in step (2) to form an age brain image block label.
[0017] (4) Model training: put the cropped image block in step (2) and the age brain image block label obtained in step (3) into the Tri-UNet model for full-supervised training.
[0018] (5) Evaluate the results: use the model trained in step (4) to predict the age of the test data, and calculate the relevant evaluation indicators for the prediction results.
[0019] In the above technical solution, step (1) of pre-processing the image specifically comprises:
[0020] (1-1) Skull stripping: the original image of medical image data contains some non-brain tissues, such as skull, blood vessels, muscles, cerebellum, etc. In order to avoid affecting the subsequent processing flow, the accuracy of brain tissue segmentation, and the final experimental results, the non-brain structure in the whole brain image is stripped;
[0021] (1-2) Image correction: including anterior commissure (AC)-posterior commissure (PC) correction, resampling image size to 256x256x256, and correcting non-uniform tissue intensity using a multi-comparison algorithm;
[0022] (1-3) Image registration: for quantitative analysis of several different images, it is necessary to strictly align several images, that is, image registration. The whole brain image is transformed by one or a series of spatial transformations so that it is consistent with the corresponding points on the medical image standard template fsaverage image. This consistency refers to the same anatomical point on the human body having the same spatial position on the two matching images;
[0023] (1-4) Image segmentation: when processing MRI data, sometimes only the state of a certain specific region is concerned, which requires extracting the tissue of the target site according to the anatomical structure of the brain. After determining the brain region with high correlation with age prediction, the required brain region is segmented as the input image of the network model, and then separate and joint analysis is performed;
[0024] (1-5) Spatial normalization: registering the image to the FreeSurfer standard brain template space fsaverage to unify the coordinate space of all images;
[0025] (1-6) Spatial smoothing processing: spatial smoothing processing is to suppress the noise of the image, improve the signal-to-noise ratio, and reduce the residual anatomical or functional inconsistencies between images. Generally, a Gaussian kernel function is used for smoothing.
[0026] In the above technical solution, in step (2), the (256, 256, 256) size brain image preprocessed in step (1) is cropped into a (32, 32, 32) size brain image block. Specifically, in the training process, a (32, 32, 32) size brain image block is randomly cropped at different positions of the brain image, and in the testing process, the (256, 256, 256) size brain image preprocessed in step (1) is sequentially cropped into a (32, 32, 32) size brain image block according to the principle of step length of 8.
[0027] In the above technical solution, in step (3), the age label needs to be converted from a number to an age brain image, so that the age label of each voxel can be obtained. Specifically, first, a matrix A is generated according to whether the pixel value in the image block cropped in step (2) is 0, that is, the pixel value in the image block is 0, the element value in the matrix A is 0, and the pixel value in the image block is not 0, the element value in the matrix A is 1, then the matrix A is multiplied by the age value corresponding to the image block to obtain the corresponding age brain image block label.
[0028] In the technical solution, the model Tri-UNet in step (4) includes a feature learning main body Tri-UNet and an age prediction network ResSame;
[0029] (4-1) Feature learning main body Tri-UNet: Tri-UNet is an organic combination of U-Net and multiple connection modes of residual structure. Trible Res Block is designed based on Inception structure and residual structure, which can enhance the fusion of multi-layer feature information. Two Trible Res Blocks are added in the encoding stage of Tri-UNet, so that the network model layer is deepened without the problem of training difficulty. In the decoding stage of Tri-UNet, the output features of Trible Res Block are combined with the output features of the lower Trible Res Block, which expands the range of fusing features of different sizes and further strengthens the connection between deep features and shallow features. Finally, the features learned by the encoding and decoding network are integrated by ResSame structure and the final brain age is predicted.
[0030] Among them, the three-branch residual block (Trible Res Block): each feature layer can learn the information of the upper and lower layers, and at the same time solve the problem of deep network degradation. The model structure of Trible Res Block contains three paths: the first is the normal input path, which does not do any operation and is used to save the original input feature content; the second is the up-sampling path, which enlarges the size of the input feature and then performs two convolution operations, the purpose is to learn the structural image features of enlarged size, which helps to improve the information fitting ability; the third is the down-sampling path, which reduces the size of the input feature map set through the maximum pooling operation, and then performs two consecutive convolution operations, the purpose is to learn the deep features of the structural image. Then, the feature map sets completed by the three paths are spliced together. Among them, the up-sampling path needs to perform down-sampling operation to reduce the size of the feature map set, and at the same time the down-sampling path needs to perform up-sampling operation to enlarge the size of the feature map set, so that the sizes of the feature map sets output by the three paths are the same. can be aligned and spliced. Finally, the spliced feature map set is taken as the output. The advantage of the three-branch residual block model structure is that after the Trible Res Block operation, the original input feature carries deep and shallow feature information at the same time, and the Trible Res Block fuses the original input feature with the up-sampling feature and the down-sampling feature, which improves the correlation between the upper and lower layers;
[0031] (4-2) Age prediction network ResSame: The ResSame network structure is located at the end of the Tri-Unet network, which generates an output age result map, where each voxel has an independent brain predicted age. Finally, the brain predicted age of each voxel in the age result map is averaged to obtain the final brain predicted age;
[0032] (4-3) Loss function: Select mean square error (MSE) as the loss function of the model, and the MSE is expressed as:
[0033]
[0034] Where N represents the number of samples, y i represents the true age, and y represents the predicted age.
[0035] In the above technical solution, the evaluation index in step (5) is MAE, RMSE, r and R 2 .
[0036] (5-1) Mean Absolute Error (MAE)
[0037] MAE calculates the average of the absolute values of the errors between the predicted age and the true age. MAE assumes that all individual differences have the same weight, and MAE is expressed as:
[0038]
[0039] (5-2) Root Mean Square Error (RMSE)
[0040] RMSE is a quadratic scoring rule that calculates the square root of the ratio of the sum of the squares of the differences between the predicted age and the true age to the number of samples N. RMSE is expressed as:
[0041]
[0042] (5-3) Correlation coefficient r
[0043] The correlation coefficient represents the degree of linear correlation between the predicted age and the true age, and its value ranges from -1 to 1. The correlation coefficient r is expressed as:
[0044]
[0045] (5-4) Determination coefficient R 2
[0046] The determination coefficient R 2 measures the goodness of fit of the predicted age to the true age. The determination coefficient R2 is represented as:
[0047]
[0048] where N represents the number of samples, y i represents the true age, represents the predicted age, and respectively represent the corresponding sample mean values.
[0049] The present application is a Tri-UNet brain age prediction method based on MRI different scale features, and a network model Tri-UNet based on 3D ResNet and 3D U-Net is proposed for brain age prediction task. Among them, the U-Net multi-layer encoding and decoding structure can combine deep features and shallow features, so as to better learn the context semantic information of input features, improve the prediction accuracy, and also learn the refined features; the residual structure in ResNet can convert the end-to-end feature mapping into residual mapping, so as to better learn the feature information of the front and back layers of the network and improve the prediction accuracy. BRIEF DESCRIPTION OF DRAWINGS
[0050] Figure 1 is the network structure diagram of Tri-UNet in the present application.
[0051] Figure 2 is the flow chart of the Tri-UNet brain age prediction method based on MRI different scale features of the present application. DETAILED DESCRIPTION
[0052] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application will be further described in detail below in combination with the drawings and implementation examples. It should be understood that the specific implementation examples described herein are only used to explain the present application and are not used to limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.
[0053] Implementation data set: the data set of Cambridge Aging and Neuroimaging Center (Cam-CAN) repository was used, which collected structural (T1 weighted MRI) neuroimaging data of 651 healthy subjects (male / female = 322 / 329, average age = 54.7±18.6, age range 18-89 years old).
[0054] Evaluation index: mean absolute error (Mean Absolute Error, MAE) and root mean square error (Root Mean Square Error, RMSE).
[0055] The embodiment provides a Tri-UNet brain age prediction method based on MRI different scale features, and a specific implementation manner is as shown in the description Figure 2 The method comprises the following steps:
[0056] (1) performing data preprocessing on the MRI; specifically, the data preprocessing comprises skull stripping, image correction, image registration, image segmentation, spatial normalization and spatial smoothing processing.
[0057] (2) cropping the image processed in the step (1), and cropping the original (256, 256, 256) size whole brain image into a (32, 32, 32) size brain image block.
[0058] (3) recording the size of the brain image block cropped in the step (2), and then generating an age brain image block label with the same size by using matrix operation. In the age brain image block label, the value of the non-brain region pixel point is set to 0. The value of the pixel point containing brain tissue is set to the corresponding brain age. The image block cropped in the step (2) and the generated age brain image block label are paired and saved.
[0059] (4) inputting the brain image block cropped in the step (2) and the age brain image block label obtained in the step (3) into the Tri-UNet network model for training, adjusting the model parameters by calculating the MSE loss, and saving the model with the optimal effect.
[0060] The model Tri-UNet comprises a feature learning main body Tri-UNet and an age prediction network ResSame.
[0061] (4-1) the feature learning main body Tri-UNet: Tri-UNet is an organic combination of U-Net and multiple connection modes of residual structure. Trible Res Block is designed based on the Inception structure and the residual structure, and can enhance the fusion of multi-layer feature information. Two three-branch residual blocks are added in the encoding stage of Tri-UNet, so that the network model layer is deepened without the problem of training difficulty. In the decoding stage of Tri-UNet, the output features of Trible Res Block are combined with the output features of the lower Trible Res Block, the range of fusing features of different sizes is expanded, and the connection between deep features and shallow features is further strengthened. Finally, the features learned by the encoding and decoding network are integrated by ResSame and the brain age is predicted.
[0062] Among them, the three-branch residual block (Trible Res Block): makes each feature layer learn the information of the upper and lower layers while solving the degradation problem of the deep network. As shown in the description Figure 1The model structure of the Trible Res Block contains three paths, as shown by the two dashed boxes in FIG. 4. The first path is a normal input path that does not perform any operation and is used to save the original input feature content. The second path is an up-sampling path that enlarges the size of the input feature and then performs two convolution operations, aiming to learn the structural image features of the enlarged size, which helps to improve the information fitting capability. The third path is a down-sampling path that reduces the size of the input feature map set through a max-pooling operation, and then performs two consecutive convolution operations, aiming to learn the deep features of the structural image. Then, the feature map sets completed by the three paths are spliced together. Among them, the up-sampling path needs to perform a down-sampling operation to reduce the size of the feature map set, and the down-sampling path needs to perform an up-sampling operation to enlarge the size of the feature map set, so that the sizes of the feature map sets output by the three paths are the same and can be aligned and spliced. Finally, the spliced feature map set is taken as the output. The advantage of the three-branch residual block model structure is that the original input feature carries deep and shallow feature information after the Trible Res Block operation. The Trible Res Block fuses the original input feature with the up-sampling feature and the down-sampling feature, improving the correlation between the upper and lower layers of information.
[0063] (4-2) Age prediction network ResSame: The ResSame network structure is located at the end of the Tri-Unet network, which generates an output age result map. Each voxel in the map has an independent brain predicted age. Finally, the brain predicted age of each voxel in the age result map is averaged to obtain the final brain predicted age.
[0064] (4-3) Loss function: The mean square error (MSE) is selected as the loss function of the model, and the MSE is expressed as:
[0065]
[0066] where N represents the number of samples, y i represents the true age, and y represents the predicted age.
[0067] (5) The Tri-UNet model trained in step (4) is used to predict the brain age, and the MAE, RMSE, r and R 2 evaluation indexes are used to evaluate the prediction effect of the model.
[0068] The contents not described in detail in the specification belong to the prior art known to those skilled in the art.
[0069] Those skilled in the art will readily understand that the above described are only the preferred embodiments of the present application and are not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the scope of protection of the present application.
Claims
1. A Tri-UNet brain age prediction method based on different scale features of MRI, characterized in that The method comprises the following steps: (1) Preprocessing the image: using the full-automatic processing stream recon-all of the medical image processing software FreeSurfer to preprocess the original image, including skull stripping, image correction, image registration, image segmentation, spatial normalization, and spatial smoothing processing; (2) Cropping the image: cropping the whole brain image into small image blocks according to the server GPU memory limit, and then taking the cropped image blocks as model inputs; (3) Image alignment: mapping the age label value into the image blocks of the same size as the cropped image in step (2) to form an age brain image block label; (4) Model training: putting the cropped image blocks in step (2) and the age brain image block label obtained in step (3) into the Tri-UNet model for full-supervised training; the model Tri-UNet comprises a feature learning main body Tri-UNet and an age prediction network ResSame; (4-1) Feature learning main body Tri-UNet: adding two Trible Res Blocks in the encoding stage of Tri-UNet, combining the output features of the Trible Res Blocks with the output features of the lower Trible Res Blocks in the decoding stage of Tri-UNet, integrating the features learned by the encoding and decoding networks through ResSame, and predicting the brain age; Wherein, the Trible Res Block model structure comprises three paths: the first path is a normal input path, which does not perform any operation and is used to save the original input feature content, the second path is an up-sampling path, which enlarges the size of the input features and then performs two convolution operations, the purpose being to learn the structural image features of the enlarged size, which helps to improve the information fitting capability, and the third path is a down-sampling path, which executes two successive convolution operations after reducing the size of the input feature map set through a maximum pooling operation, the purpose being to learn the deep features of the structural image, then the feature map sets completed by the three paths are spliced together, wherein the up-sampling path needs to perform a down-sampling operation to reduce the size of the feature map set, and the down-sampling path needs to perform an up-sampling operation to enlarge the size of the feature map set, so that the sizes of the feature map sets output by the three paths are the same and can be aligned and spliced, finally, the spliced feature map set is taken as the output, after the Trible Res Block operation, the original input features carry deep and shallow feature information at the same time, the Trible Res Block fuses the original input features with the up-sampling features and the down-sampling features, and improves the correlation between the upper and lower layers of information; (4-2) Age prediction network ResSame: the ResSame network structure is located at the end of the Tri-Unet network, which generates an output age result map, wherein each voxel has an independent brain prediction age, and finally the brain prediction ages of each voxel in the age result map are averaged to obtain the final brain prediction age; (4-3) Loss function: selecting MSE as the loss function of the model, and the MSE is represented as: where N represents the number of samples, y i represents the true age, represents the predicted age; (5) Evaluation results: using the model trained in step (4) to predict the age of the test data, and calculating the relevant evaluation indicators of the prediction results.
2. The Tri-UNet brain age prediction method based on MRI features of different scales according to claim 1, characterized in that: The step (1) pre-processing image specifically includes: (1-1) skull stripping: stripping the non-brain structure in the whole brain image; (1-2) image correction: including anterior-posterior commissure correction, resampling the image size to 256*256*256, and correcting the non-uniform tissue intensity using multiple comparison algorithm; (1-3) image registration: the whole brain image is registered to the corresponding point on the medical image standard template fsaverage image through one or a series of spatial transformations, and the consistency refers to the same anatomical point on the human body having the same spatial position on the two matched images; (1-4) image segmentation: after determining the brain region with high correlation with age prediction, the required brain region is segmented as the input image of the network model, and then separate and joint analysis is performed; (1-5) spatial normalization: registering the image to the FreeSurfer standard brain template space fsaverage to unify the coordinate space of all images; (1-6) spatial smoothing processing: Gaussian kernel function is used for smoothing processing.
3. The Tri-UNet brain age prediction method based on MRI features of different scales according to claim 1, characterized in that In step (2), the (256, 256, 256) size brain image pre-processed in step (1) is cropped to a (32, 32, 32) size brain image block, specifically, in the training process, (32, 32, 32) size brain image blocks are randomly cropped at different positions of the brain image, and in the testing process, according to the principle of step length of 8, the (256, 256, 256) size brain image pre-processed in step (1) is cropped to a (32, 32, 32) size brain image block.
4. The Tri-UNet brain age prediction method based on MRI features of different scales according to claim 1, characterized in that In step (3), first, generate matrix A according to whether the pixel value in the image block cropped in step (2) is 0, that is, the pixel value in the image block is 0, the element value in matrix A is 0, and the pixel value in the image block is not 0, the element value in matrix A is 1, then multiply matrix A with the age value corresponding to the image block to obtain the corresponding age brain image block label.
5. The Tri-UNet brain age prediction method based on MRI features of different scales according to claim 1, characterized in that: The evaluation index in step (5) is MAE, RMSE, r and R 2 ; (5-1) MAE MAE calculates the average of the absolute values of the prediction age and the true age, MAE assumes that all individual differences have the same weight, and MAE is represented as: (5-2) RMSE RMSE is a quadratic scoring rule, which calculates the square root of the ratio of the sum of squares of the difference between the predicted age and the true age to the sample size N, and RMSE is represented as: (5-3) correlation coefficient r The correlation coefficient r represents the degree of linear correlation between the predicted age and the true age, and the value range is-1 to 1, and the correlation coefficient r is represented as: (5-4) Coefficient of determination R 2 The coefficient of determination R 2 The coefficient of determination R 2 is expressed as: where N denotes the number of samples, y i denotes the true age, denotes the predicted age, and denote the respective sample means.