A method, device and apparatus for detecting brain damage area in brain magnetic resonance images
Through the improved ResNet network model combined with T1WI and T2WI imaging sequences, a dual-branch structure was constructed, which solved the problem of brain injury detection in brain magnetic resonance images in infants and young children, and achieved high-accurate brain injury area detection.
Patent Information
- Application Number
- CN202411110253.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-08-14
AI Technical Summary
The prior art lacks brain injury area detection methods for infant brain magnetic resonance images, resulting in diagnostic difficulties and automated screening difficulties, especially in the rapid development of infant brains and complexity of MRI characteristics.
The improved ResNet network model is adopted, combined with T1WI and T2WI imaging sequences, and a dual-branch structure is constructed through the convolution enhancement module, Basic Block and cross-modal attention module to realize the detection of brain damage area of infant brain magnetic resonance images.
It improves the comprehensive expression ability of brain injury characteristics, improves detection accuracy, and can efficiently identify brain injury areas in brain magnetic resonance images of infants and young children.
Smart Images

Figure CN118628498B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a method, device and equipment for detecting brain damage areas in brain magnetic resonance images. Background Art
[0002] Currently, brain injury in infants and young children often leads to serious developmental outcomes (including motor and cognitive deficits), which can seriously affect the subsequent quality of life. For infants with suspected brain injury, medical imaging technology is essential for accurate diagnosis and development of appropriate treatment strategies. Among the commonly used brain imaging modalities, magnetic resonance imaging (MRI) is generally preferred over ultrasound (US) and computed tomography (CT) because of its higher resolution and better safety. The infant brain undergoes rapid maturation and myelination within the first year, resulting in significant changes in image appearance, which are strongly dependent on age and manifested as less tissue contrast. For example, in T1-weighted imaging (T1WI), the intensity range of gray matter (GM) and white matter (WM) in infants aged 3 to 9 months is very similar. However, after 9 months, WM intensity exceeds GM. Compared with other age groups, the unique dynamic development of the infant brain brings greater variability in size, shape, and cortical folding, thus affecting MRI feature values. These unique MRI features of infant brain injury present diagnostic challenges to radiologists and computer-aided diagnosis systems.
[0003] Multicenter studies are crucial in the early dynamic development of the infant brain because they increase the sample size and enhance the representativeness of the study, which can significantly advance the application and dissemination of research results in infant brain injury. Therefore, multicenter studies often need to screen out data containing brain injury from a large amount of data for more refined data screening and analysis. In addition, infant brain injury research requires not only imaging data of brain injury, but also brain imaging data without obvious brain injury. For example, a key step in MRI data analysis, spatial normalization, depends largely on the selection of a public spatial template. However, the lack of standardized public brain templates for infant research is a current research obstacle. Given the huge neuroanatomical differences between adult and infant brains, the adult MNI map commonly used for neuroimaging is not suitable for infant research. Therefore, manual screening of a large number of images without obvious brain injury is essential to create infant brain templates for MRI analysis. Therefore, screening through extensive data sets is time-consuming and laborious, regardless of the presence of brain injury, which highlights the need for computer and artificial intelligence methods to automate data screening to accelerate research and its application.
[0004] In the existing technology, deep learning technology in the field of artificial intelligence has become a prominent tool in image classification tasks and has achieved significant success in various medical imaging challenges, including those involving brain MRI data. For example, brain MR images are distinguished into normal (non-tumor) and abnormal (tumor) images through classic classification models such as ConvNet, LeNet, ResNet, and DenseNet. There is also a scheme that proposes a new neural network model VGG-SE-PCR, which aims to classify brain CT images into normal (without traumatic brain injury (TBI)) and abnormal (with TBI). In addition, there is also a scheme that proposes a deep learning method that simulates the diagnostic reasoning of radiologists by combining it with a 2D CNN model and a sequence model to effectively identify and classify acute cerebral hemorrhage.
[0005] However, current brain injury classification work mainly focuses on specific situations such as cerebral hemorrhage and brain tumors in adults, and there is a lack of solutions for classifying and detecting brain injury areas in brain magnetic resonance images of infants and young children. Summary of the invention
[0006] Based on this, it is necessary to provide a method, device and equipment for detecting brain damage areas in brain magnetic resonance images to address the above-mentioned technical problem of lack of classification detection of damage in brain magnetic resonance images of infants and young children.
[0007] The present invention adopts the following technical solutions:
[0008] The invention provides a method for detecting a brain injury region of a brain magnetic resonance image. First, historical brain magnetic resonance images of infants are obtained as sample images and annotated, and T1WI imaging sequences and T2WI imaging sequences thereof are extracted to obtain training data. Then, based on a ResNet network, a convolution enhancement module and a plurality of Basic Block modules are sequentially connected in series to form a branch, two of the branches are connected in parallel to form a double-branch structure, and a cross-modal attention module, a plurality of Basic Block modules and a classification layer are sequentially connected in series after the double-branch structure to construct an improved ResNet network model. Then, the improved ResNet network model is trained based on the training data to obtain a brain injury detection model. Finally, the T1WI imaging sequence and the T2WI imaging sequence of the brain magnetic resonance image of the infant to be detected are extracted, and brain injury region detection is performed based on the two imaging sequences of the brain magnetic resonance image of the infant to be detected by the brain injury detection model obtained after training.
[0009] The present invention provides a device for detecting brain damage areas in brain magnetic resonance images, and each module is used to execute the above-mentioned method for detecting brain damage areas in brain magnetic resonance images.
[0010] The present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method for detecting brain damage areas in brain magnetic resonance images is implemented.
[0011] The present invention provides a computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for detecting brain damage areas in brain magnetic resonance images when executing the program.
[0012] At least one of the above technical solutions adopted by the present invention can achieve the following beneficial effects:
[0013] The above technical scheme of the present invention simultaneously utilizes the T1WI imaging sequence and the T2WI imaging sequence of the infant brain magnetic resonance images, strengthens the utilization of unique features in different imaging sequences, and fuses the corresponding features of the two based on color channels and spatial positions, thereby improving the comprehensive expression ability of brain injury features, so as to effectively detect brain injury areas in the magnetic resonance images of infants with high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0015] Figure 1 A schematic diagram of a flow chart of a method for detecting brain injury areas in brain magnetic resonance images provided by the present invention;
[0016] Figure 2 A schematic diagram of brain injury conditions of data used by the present invention;
[0017] Figure 3 A schematic diagram of a flow chart of image preprocessing provided by the present invention;
[0018] Figure 4 This is a schematic diagram of a device for detecting brain damage areas in brain magnetic resonance images provided by the present invention. DETAILED DESCRIPTION
[0019] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0020] At present, brain injury classification work mainly focuses on specific situations such as cerebral hemorrhage and brain tumors in adults, while ignoring the critical task of classifying brain injuries in infants and young children. This situation is partly due to the fact that brain MRI studies in infants and young children rely on a manual data screening process, focusing on special sequences other than T1-weighted imaging (T1WI) and T2-weighted imaging (T2WI). In addition, MRI data collected in hospitals are usually thick layers (mostly between 4 and 6 mm thick), while most thin-layer datasets used for medical research on brain diseases are thin layers. These problems shown in infant brain MRI studies limit the practical application scenarios of deep learning methods.
[0021] The technical solutions provided by various embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.
[0022] Figure 1 The present invention is a flowchart of a method for detecting brain damage areas in brain magnetic resonance images, which specifically includes the following steps:
[0023] S101: Obtain historical brain magnetic resonance images of infants and young children as sample images, annotate target brain injury areas in each sample image, and extract T1WI imaging sequences and T2WI imaging sequences of each sample image.
[0024] Generally, for the classification prediction of brain injury areas, brain injury will be presented in the brain structure. Therefore, brain imaging can be detected to determine whether there is a brain injury area in the brain imaging. The current deep learning technology has been well applied in various medical image classification tasks. However, there are the following limitations in the research and application tasks related to infants and young children: infants and young children are in a stage of rapid brain development, with many patterns of brain structure changes and complex image features, which poses a challenge to the extraction of features by classic deep learning models. In addition, there is currently limited attention to pediatric MRI. Current research mainly focuses on adult diseases, ignoring the application of MRI in infants and young children with more complex brain structures. Multicenter studies need to screen out data with and without obvious brain injury, which requires a lot of time and human resources, and lacks high-performance automatic processing methods. At the same time, traditional methods usually directly use a single MRI sequence, or directly stack channels to combine the features of different sequences for training. However, this method fails to fully utilize the unique advantages of each sequence feature.
[0025] Based on this, in one or more embodiments of the present invention, the server of the business platform may first obtain historical brain magnetic resonance images of infants and young children as sample images, train the constructed model through the sample images, and then apply the trained model to detect brain damage areas in brain magnetic resonance images.
[0026] Typically, the server can obtain weighted imaging based on brain magnetic resonance images, such as T1-weighted imaging (T1WI) and T2-weighted imaging (T2WI), and perform feature extraction and brain damage area identification based on the weighted imaging.
[0027] T1WI is an imaging sequence in magnetic resonance imaging (MRI), which stands for T1-weighted imaging. In MRI, T1WI is an imaging technique that uses a specific pulse sequence to highlight the difference in T1 relaxation time in tissues. T1 relaxation time refers to the time required for tissue to return from a magnetized state to a state of equilibrium, and this time is different in different types of biological tissues. T1WI can usually provide good soft tissue contrast, so that the boundaries between different types of tissue structures are clear. Similar to T1WI, T2WI can usually provide good liquid contrast, so that liquids and tissues with more water appear brighter in the image. Therefore, in one or more embodiments of the present invention, it is possible to simulate radiologists combining two different conventional sequences (T1WI and T2WI) to detect brain injury areas.
[0028] Of course, the T1WI imaging sequence and the T2WI imaging sequence may also be preprocessed to reduce image differences, artifacts, uneven imaging, and other issues between different devices. The specific processing method may be determined as needed, and the present invention does not impose any limitation on this.
[0029] In addition, in current research, there are more studies on adult brain MRI and fewer on infant brain MRI, and thin-layer MRI is the main method (the scanning layer thickness of the brain magnetic resonance image is less than 4 mm), but most grassroots hospitals use conventional thick-layer MRI sequences (the scanning layer thickness of the brain magnetic resonance image is 4 mm to 6 mm), and there is a lack of deep learning models suitable for conventional thick-layer MRI. The focus of existing research is mainly on thin-layer and specialized MRI sequences, which are inconsistent with thick-layer MRI and more traditional sequences commonly obtained in hospitals. This difference leads to a lack of general MRI classification models suitable for conventional clinical environments. Compared with thin-layer MRI, thick-layer MRI images contain less feature information, and it is necessary to enhance the ability to capture features of different scales in the initial stage of feature extraction. In one or more embodiments of the present invention, the historical brain magnetic resonance images of infants and young children may be thick-layer brain magnetic resonance images, that is, the scanning layer thickness of the brain magnetic resonance images is 4 mm to 6 mm. And feature extraction is performed on thick-layer brain magnetic resonance images through subsequent steps.
[0030] The server mentioned in the present invention may be a server set on a business platform, or a device such as a desktop computer, a notebook computer, etc. that can execute the solution of the present invention. For the convenience of description, the following description is only based on the server as the execution subject.
[0031] S102: Based on the ResNet network, the convolution enhancement module and multiple Basic Block modules are connected in series in sequence to form a branch, the two branches are connected in parallel to form a double-branch structure, and the cross-modal attention module, multiple Basic Block modules and the classification layer are connected in series in sequence after the double-branch structure to construct an improved ResNet network model.
[0032] After obtaining different imaging sequences corresponding to the sample images as described above, the server can construct a model and perform model training based on the imaging sequences, so as to detect brain injury areas using the trained model.
[0033] Among them, the construction of the model can be based on the ResNet network. In the scenario of brain damage area detection in infant brain magnetic resonance images, the first half of the network model can use a dual-branch structure to perform adaptive feature extraction on two different imaging sequences. In the middle stage of the model, that is, after the dual-branch structure completes feature extraction, the cross-modal attention module (CMAM) can be used to achieve the fusion of dual-branch feature information in color channels and spatial positions. In the second half of the neural network, the basic module (Basic Block) used in ResNet can be used for feature extraction, and finally classification is performed through the classification layer.
[0034] Specifically, in one or more embodiments of the present invention, a convolutional enhancement module (CEM) and a plurality of Basic Block modules may be connected in series to form a branch, and two of the branches may be connected in parallel to form a dual-branch structure. For example, each branch of the dual-branch structure may include a convolutional enhancement module and four Basic Block modules connected in series, and the number of channels of the feature information output by the convolutional enhancement module after passing through the four Basic Block modules becomes 64, 64, 128, and 128 respectively.
[0035] Specifically, in the convolution enhancement module, the convolution enhancement module in one branch of the dual-branch structure may include the first convolution kernel and the second convolution kernel connected in series, that is, in the order of "first convolution kernel - second convolution kernel", and the convolution enhancement module in the other branch may include the third convolution kernel and the fourth convolution kernel connected in series, that is, in the order of "third convolution kernel - fourth convolution kernel". Among them, the first convolution kernel, the second convolution kernel, the third convolution kernel and the fourth convolution kernel all use convolution kernels of size 3×3×3, the first convolution kernel and the fourth convolution kernel have a step length of 2, and the second convolution kernel and the third convolution kernel have a step length of 1.
[0036] In addition, the Basic Block module may include a series structure formed by sequentially connecting a convolution kernel that is circulated twice, a batch normalization, and a ReLU activation function, and the classification layer may include: an average pooling layer, a fully connected layer, and a normalization layer.
[0037] S103: Inputting the T1WI imaging sequence and the T2WI imaging sequence of each sample image into the improved ResNet network model respectively to obtain the predicted brain injury area of each sample image; and training the improved ResNet network model with minimizing the deviation between the target brain injury area and the predicted brain injury area of the sample image as the optimization goal to obtain a brain injury detection model.
[0038] S104: extracting the T1WI imaging sequence and the T2WI imaging sequence of the brain magnetic resonance image of the infant to be tested, and correspondingly inputting the two branches of the dual-branch structure in the brain injury detection model, extracting the first brain injury feature map and the second brain injury feature map; performing feature fusion on the first brain injury feature map and the second brain injury feature map based on the color channel and the spatial position through the cross-modal attention module to obtain a fused feature map; then performing deep brain injury feature extraction through the subsequent multiple Basic Block modules to obtain a deep brain injury feature map, and performing detection through the classification layer to determine the brain injury area of the brain magnetic resonance image to be tested.
[0039] After obtaining the improved ResNet network model through the above construction, the server can input the T1WI imaging sequence of each sample image into one branch of the dual-branch structure, and input the T2WI imaging sequence into the other branch, and detect and judge the brain damage area through the improved ResNet network model to obtain the predicted brain damage area of each sample image.
[0040] Then, the deviation between the target brain injury area and the predicted brain injury area of the sample image is determined, and the improved ResNet network model is trained with minimizing the deviation as the optimization goal, thereby obtaining a brain injury detection model that can detect the brain injury area of the brain magnetic resonance image.
[0041] Based on this, in one or more embodiments of the present invention, for the brain magnetic resonance images of the infant to be tested, the server may first extract the imaging sequence thereof, including T1 imaging and T2 imaging, to obtain T1WI imaging sequence and T2WI imaging sequence.
[0042] Then, the server can input the T1WI imaging sequence into one of the branches, and extract the first brain injury feature map through the convolution enhancement module and multiple Basic Block modules of the branch, and at the same time input the T2WI imaging sequence into another branch, and extract the second brain injury feature map through the convolution enhancement module and multiple Basic Block modules of the other branch.
[0043] Then, the brain injury detection model can fuse the two through a cross-modal attention module. Specifically, the cross-modal attention module can include three interfaces, and the first interface of the cross-modal attention module can be used to convolve and resize the first brain injury feature map to obtain a reshaped first brain injury feature map.
[0044] The second brain injury feature map is convolved and resized through the second interface of the cross-modal attention module to obtain a reshaped second brain injury feature map, which is then point-multiplied and normalized with the reshaped first brain injury feature map to obtain an initial fused feature map.
[0045] The second brain injury feature map is convolved and resized through the third interface of the cross-modal attention module to obtain the reshaped second brain injury feature map, which is then dot-multiplied and resized with the initial fusion feature map to obtain the final fusion feature map.
[0046] From the perspective of the cross-modal attention module structure, each interface of the cross-modal attention module may include a convolution kernel and a reshape module (Reshape), assuming that the first interface includes Q conv and the first reshape module, the second interface includes K conv and the second reshape module, and the third interface includes V conv and the third reshape module. After the data of the two imaging sequences T1WI and T2WI are respectively extracted by the convolution enhancement modules of the two branches and the features of multiple basic modules, the obtained feature information is simultaneously input into different interfaces of the cross-modal attention module.
[0047] Among them, the feature information of the first branch can enter Q conv, using a 1×1×1 convolution kernel, and reducing the channel dimension to 1 / 8 of the previous one. The feature information of the second branch enters K conv and V conv at the same time, both using a 1×1×1 convolution kernel. The outputs obtained by Q conv, K conv, and V conv are reshaped (Reshape) to the same size through the corresponding reshaping modules. Then, the feature information corresponding to the reshaped Q conv and K conv is dot-multiplied and processed by the Softmax function, and then continues to be dot-multiplied with the feature information corresponding to the reshaped V conv. Finally, the obtained feature information is reshaped again and output, that is, the final fusion feature map is output.
[0048] Subsequently, the brain injury detection model can extract deep brain injury features based on the final fusion feature map through subsequent multiple Basic Block modules to obtain a deep brain injury feature map, and detect the deep brain injury feature map through the classification layer to determine the brain injury area of the brain magnetic resonance image to be detected. The average pooling layer here can use the AvgPool3d function, the fully connected layer can use the Linear function, and the activation function layer can use the Softmax layer. Of course, for infants and young children with normal development, the brain injury detection model can also not mark any brain injury area when detecting the brain magnetic resonance image to be detected.
[0049] based on Figure 1 The method for detecting brain damage areas in brain magnetic resonance images shown in the figure first obtains historical brain magnetic resonance images of infants and young children as sample images and annotates them, and extracts their T1WI imaging sequences and T2WI imaging sequences to obtain training data, and then based on the ResNet network, connects a convolution enhancement module and multiple Basic Block modules in series in sequence to form a branch, connects the two branches in parallel to form a double-branch structure, and connects a cross-modal attention module, multiple Basic Block modules and a classification layer in series in sequence after the double-branch structure to construct an improved ResNet network model, and then trains the improved ResNet network model based on the training data to obtain a brain damage detection model, and finally extracts the T1WI imaging sequence and T2WI imaging sequence of the brain magnetic resonance images of the infants and young children to be detected, and uses the brain damage detection model obtained after training to perform brain damage area detection based on the two imaging sequences of the brain magnetic resonance images of the infants and young children to be detected.
[0050] The present invention adopts a dual-branch structure to adaptively extract features from T1WI imaging sequences and T2WI imaging sequences in two independent branches, thereby enhancing the use of unique features of the two different sequences. At the same time, a convolution enhancement module (CEM) is designed to integrate multi-scale features at the beginning of feature extraction in each branch to enhance the feature expression capability of thick-layer MRI image data. In order to make full use of the features obtained from the above two branches, a cross-modal attention module (CMAM) is designed to achieve effective feature fusion of color channels and spatial position information of different imaging sequences, thereby improving the comprehensive expression capability of the features of the two different sequences, so as to effectively detect brain damage areas in magnetic resonance images of infants and young children with high accuracy.
[0051] In addition, the present invention also provides an application example of a method for detecting brain injury areas in brain magnetic resonance images. In this example, MRI images were obtained for 240 patients who underwent magnetic resonance scans from multiple institutions, including T1-weighted imaging and T2-weighted imaging. The slice spacing was distributed in the range of 4mm-6mm, and the age of the patients during the scan was less than 2 years old.
[0052] This example follows the diagnostic advice of senior radiology doctors, and all data are judged by two doctors with 10 years of experience in brain MRI diagnosis. Among them, there are 122 cases of brain MRI data with obvious brain damage and 118 cases of brain MRI data without obvious brain damage. Figure 2 The figure in the figure is a schematic diagram of brain damage of data used in the present invention. Figure 2 The same slice image with and without obvious brain damage is shown in Figure 2 The first row is a schematic diagram of areas with brain damage, and the second row is a schematic diagram of areas without brain damage. The horizontal axis represents the age and month of the infant, and "M" represents month. These data are randomly split into 5 parts, split into training sets and test sets in a ratio of 8:2, and a five-fold cross-validation experiment is completed. Table 1 is a schematic diagram of a five-fold cross-split data set in the present invention. The 5 rows of numbers in Table 1 correspond to a piece of data obtained by random splitting from top to bottom. The first column of numbers in each piece of data represents the amount of brain MRI data in the area with obvious brain damage in the data, and the second column of numbers represents the amount of brain MRI data in the area without obvious brain damage in the data.
[0053] Table 1 Split of infant brain injury MRI dataset in five-fold cross validation
[0054]
[0055] Figure 3 This is a flow chart of an image preprocessing process in the present invention. In order to reduce the difficulty of extracting features from the neural network model caused by image differences, artifacts, and uneven imaging between different devices, which leads to a decrease in the classification performance of the method, the T1WI and T2WI raw data of each sample are processed. The brain anatomical structure of the T1WI sequence is relatively clear, so the T2WI sequence is aligned to the T1WI. In order to eliminate the influence of the skull and noise signals outside the skull on the image quality, the skull of T1WI and T2WI is removed using FSL software. In order to reduce the influence of magnetic field inhomogeneity on image quality, ANTs are used to perform N4 bias field correction on T1WI and T2WI images.
[0056] Furthermore, the classification model uses the improved ResNet network model (DBAII-Net) in one or more of the above embodiments to detect brain injury areas in the images of the infant brain MRI dataset.
[0057] At the beginning of the two branches, the preprocessed T1WI and T2WI data are sorted, and the T1WI data is input into branch 1, and the T2WI data is input into branch 2. Branch 1 is arranged in the order of "convolution kernel 1-convolution kernel 2", and branch 2 is arranged in the order of "convolution kernel 3-convolution kernel 4". Among them, convolution kernel 1, convolution kernel 2, convolution kernel 3 and convolution kernel 4 all use the convolution kernel size of 3×3×3, but the stride of convolution kernel 1 and convolution kernel 4 is 2, and the stride of convolution kernel 2 and convolution kernel 3 is 1; after branch 1 and branch 2, feature splicing is performed according to the channel dimension, and then batch normalization (BatchNormalization) and ReLU activation function are used.
[0058] In the two branches, the feature information output by the convolution enhancement modules of the two branches passes through the same four basic modules in sequence. Each basic module consists of a convolution kernel (3×3×3) that is circulated twice, batch normalization, and a ReLU activation function. After the feature information passes through the four basic modules, the number of channels becomes 64, 64, 128, and 128 respectively.
[0059] After the T1WI and T2WI feature extraction of the two branches is completed, the obtained feature information is simultaneously input into different interfaces of the cross-modal attention module. Specifically, the feature information of branch 1 enters Q conv, using a 1×1×1 convolution kernel, and reducing the channel dimension to 1 / 8 of the previous one. The feature information of branch 2 enters K conv and V conv at the same time, both using a 1×1×1 convolution kernel. The outputs obtained by Q conv, K conv, and V conv are all reshaped (Reshape) to the same size. Next, the resized Q conv and K conv feature information are dot-multiplied and processed with the Softmax function, and then continue to be dot-multiplied with the resized V conv. Finally, the obtained feature information is resized again and output.
[0060] In the final stage of the entire model, the feature information obtained above is input into the classification layer including the average pooling layer, the fully connected layer, and the activation function layer. Among them, the average pooling layer uses the AvgPool3d function, the fully connected layer uses the Linear function, and the activation function layer uses the Softmax layer. Finally, the brain damage area of the brain magnetic resonance image to be detected is determined.
[0061] Table 2 shows the classification performance comparison results of the deep learning classification model DBAII-Net proposed in the present invention and other classic classification models obtained by five-fold cross validation in the same infant brain MRI dataset.
[0062] Table 2 Comparison of average classification performance indicators of different methods under five-fold cross validation
[0063]
[0064] As can be seen from Table 2, the DBAII-Net of the present invention has superior performance over traditional classical models in classifying brain injuries in five-fold cross validation. DBAII-Net achieved an average accuracy of up to 92.53%, and led in specificity, accuracy, F1 score and AUC value indicators, demonstrating its excellent classification ability. It is worth noting that although its sensitivity of 90.20% is slightly behind the peak of ShuffleNet 91.67%, DBAII-Net surpasses ShuffleNet in all other evaluation indicators. Compared with the classic ResNet model, DBAII-Net shows comprehensive enhancements, demonstrating its more powerful classification capabilities. Even compared with ViT under the currently popular Transformer series model, DBAII-Net has shown significant improvements in all performance indicators.
[0065] By performing a five-fold cross validation on the same thick-layer conventional MRI dataset to compare the performance of the DBAII-Net model proposed in the present invention with the classic classification model, the following points can be obtained:
[0066] 1. The DBAII-Net model achieved the best brain injury area detection performance in thick conventional brain MRI dataset;
[0067] 2. Compared with other classic models, the DBAII-Net model achieves the best computing performance;
[0068] 3. The DBAII-Net model makes good use of the feature information of two conventional MRI sequences.
[0069] When using the brain damage area detection method of the brain magnetic resonance image provided by the present invention, it is not necessary to Figure 1 The steps are executed in the order shown. The specific execution order of the steps can be determined according to needs, and the present invention does not limit this.
[0070] The above is a method for detecting brain damage areas in brain magnetic resonance images provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding device for detecting brain damage areas in brain magnetic resonance images, such as Figure 4 shown.
[0071] Figure 4 A schematic diagram of a brain damage area detection device for a brain magnetic resonance image provided by the present invention comprises:
[0072] An acquisition module 201 is used to acquire historical brain magnetic resonance images of infants and young children as sample images, mark the target brain injury area in each sample image, and extract the T1WI imaging sequence and T2WI imaging sequence of each sample image;
[0073] A construction module 202 is used to connect a convolution enhancement module and a plurality of Basic Block modules in series in sequence to form a branch based on a ResNet network, connect the two branches in parallel to form a double-branch structure, and connect a cross-modal attention module, a plurality of Basic Block modules and a classification layer in series in sequence after the double-branch structure to construct an improved ResNet network model;
[0074] The training module 203 is used to input the T1WI imaging sequence and the T2WI imaging sequence of each sample image into the improved ResNet network model respectively to obtain the predicted brain injury area of each sample image; and to train the improved ResNet network model with minimizing the deviation between the target brain injury area and the predicted brain injury area of the sample image as the optimization goal to obtain a brain injury detection model;
[0075] The detection module 204 is used to extract the T1WI imaging sequence and the T2WI imaging sequence of the brain magnetic resonance image of the infant to be detected, and extract the first brain injury feature map and the second brain injury feature map corresponding to the two branches of the dual-branch structure in the input brain injury detection model; the first brain injury feature map and the second brain injury feature map are subjected to feature fusion based on the color channel and the spatial position through the cross-modal attention module to obtain a fused feature map; and deep brain injury feature extraction is performed through the subsequent multiple Basic Block modules to obtain a deep brain injury feature map, and the classification layer is used for detection to determine the brain injury area of the brain magnetic resonance image to be detected.
[0076] Optionally, the scanning layer thickness of the historical brain magnetic resonance image of the infant is 4 mm to 6 mm.
[0077] Optionally, the Basic Block module includes: a series structure formed by sequentially connecting a convolution kernel that cycles twice, a batch normalization, and a ReLU activation function.
[0078] Optionally, the convolution enhancement module in one branch of the dual-branch structure includes a first convolution kernel and a second convolution kernel connected in series, and the convolution enhancement module in the other branch includes a third convolution kernel and a fourth convolution kernel connected in series, wherein the first convolution kernel, the second convolution kernel, the third convolution kernel and the fourth convolution kernel all use convolution kernels of size 3×3×3, the step size of the first convolution kernel and the fourth convolution kernel is 2, and the step size of the second convolution kernel and the third convolution kernel is 1.
[0079] Optionally, the cross-modal attention module includes three interfaces.
[0080] The detection module 204 convolves and resizes the first brain injury feature map through the first interface of the cross-modal attention module to obtain a reshaped first brain injury feature map, convolves and resizes the second brain injury feature map through the second interface of the cross-modal attention module to obtain a reshaped second brain injury feature map, and performs point multiplication and normalization on the reshaped second brain injury feature map with the reshaped first brain injury feature map to obtain an initial fused feature map, convolves and resizes the second brain injury feature map through the third interface of the cross-modal attention module to obtain a reshaped second brain injury feature map, and performs point multiplication and resize on the second brain injury feature map with the initial fused feature map to obtain a final fused feature map.
[0081] Optionally, the classification layer includes: an average pooling layer, a fully connected layer and a normalization layer.
[0082] For the specific definition of the brain damage area detection device of the brain magnetic resonance image, please refer to the definition of the brain damage area detection method of the brain magnetic resonance image above, which will not be repeated here. Each module in the above-mentioned brain damage area detection device of the brain magnetic resonance image can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0083] The present invention also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 A method for detecting brain lesion regions in brain magnetic resonance images is provided.
[0084] The present invention also provides a computer device. At the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, it may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 A method for detecting brain lesion regions in brain magnetic resonance images is provided.
[0085] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0086] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.
Claims
1. A method for detecting brain damage areas in brain magnetic resonance images, characterized in that: include: Obtain historical brain magnetic resonance images of infants and young children as sample images, annotate the target brain injury area in each sample image, and extract the T1WI imaging sequence and T2WI imaging sequence of each sample image; Based on the ResNet network, a convolution enhancement module and multiple Basic Block modules are sequentially connected in series to form a branch, two of the branches are connected in parallel to form a double-branch structure, and a cross-modal attention module, multiple Basic Block modules and a classification layer are sequentially connected in series after the double-branch structure to construct an improved ResNet network model; The T1WI imaging sequence and T2WI imaging sequence of each sample image are respectively input into the improved ResNet network model to obtain the predicted brain injury area of each sample image; and the improved ResNet network model is trained with the optimization goal of minimizing the deviation between the target brain injury area and the predicted brain injury area of the sample image to obtain a brain injury detection model; Extract the T1WI imaging sequence and T2WI imaging sequence of the brain magnetic resonance image of the infant to be tested, and extract the first brain injury feature map and the second brain injury feature map corresponding to the two branches of the double-branch structure in the input brain injury detection model; perform feature fusion on the first brain injury feature map and the second brain injury feature map based on color channel and spatial position through the cross-modal attention module to obtain a fused feature map; perform deep brain injury feature extraction through subsequent multiple Basic Block modules to obtain a deep brain injury feature map, and perform detection through the classification layer to determine the brain injury area of the brain magnetic resonance image to be tested; Wherein, the cross-modal attention module includes three interfaces; The method of fusing the first brain injury feature map and the second brain injury feature map based on color channels and spatial positions through a cross-modal attention module to obtain a fused feature map includes: Convolving and resizing the first brain injury feature map through a first interface of the cross-modal attention module to obtain a reshaped first brain injury feature map; Convolving and resizing the second brain injury feature map through the second interface of the cross-modal attention module to obtain a reshaped second brain injury feature map, and performing point multiplication and normalization processing on the reshaped first brain injury feature map to obtain an initial fusion feature map; The second brain injury feature map is convolved and resized through the third interface of the cross-modal attention module to obtain the reshaped second brain injury feature map, which is then dot-multiplied and resized with the initial fusion feature map to obtain the final fusion feature map.
2. The method for detecting brain damage areas in brain magnetic resonance images according to claim 1, characterized in that: The scanning layer thickness of the historical brain magnetic resonance images of the infants and young children is 4 mm to 6 mm.
3. The method for detecting brain damage area in brain magnetic resonance images according to claim 1, characterized in that: The BasicBlock module includes: a convolution kernel that is circulated twice, a batch normalization, and a ReLU activation function that are sequentially connected to form a series structure.
4. The method for detecting brain damage area in brain magnetic resonance images according to claim 1, characterized in that: The convolution enhancement module in one branch of the dual-branch structure includes a first convolution kernel and a second convolution kernel connected in series, and the convolution enhancement module in the other branch includes a third convolution kernel and a fourth convolution kernel connected in series; Among them, the first convolution kernel, the second convolution kernel, the third convolution kernel and the fourth convolution kernel all use convolution kernels of size 3×3×3, the step size of the first convolution kernel and the fourth convolution kernel is 2, and the step size of the second convolution kernel and the third convolution kernel is 1.
5. The method for detecting brain damage area in brain magnetic resonance images according to claim 1, characterized in that: The classification layer includes: an average pooling layer, a fully connected layer and a normalization layer.
6. A device for detecting brain damage areas in brain magnetic resonance images, characterized in that: include: An acquisition module is used to acquire historical brain magnetic resonance images of infants and young children as sample images, mark the target brain injury area in each sample image, and extract the T1WI imaging sequence and T2WI imaging sequence of each sample image; A construction module is used to connect a convolution enhancement module and multiple Basic Block modules in series to form a branch based on a ResNet network, connect the two branches in parallel to form a double-branch structure, and connect a cross-modal attention module, multiple Basic Block modules and a classification layer in series after the double-branch structure to construct an improved ResNet network model; A training module is used to input the T1WI imaging sequence and the T2WI imaging sequence of each sample image into the improved ResNet network model respectively to obtain the predicted brain injury area of each sample image; and to train the improved ResNet network model with the optimization goal of minimizing the deviation between the target brain injury area and the predicted brain injury area of the sample image to obtain a brain injury detection model; The detection module is used to extract the T1WI imaging sequence and T2WI imaging sequence of the brain magnetic resonance image of the infant to be detected, and extract the first brain injury feature map and the second brain injury feature map corresponding to the two branches of the double-branch structure in the input brain injury detection model; the first brain injury feature map and the second brain injury feature map are feature fused based on the color channel and spatial position through the cross-modal attention module to obtain a fused feature map; and then through the subsequent multiple Basic The Block module performs deep brain injury feature extraction to obtain a deep brain injury feature map, and detects through the classification layer to determine the brain injury area of the brain magnetic resonance image to be detected; wherein the cross-modal attention module includes three interfaces; the cross-modal attention module fuses the first brain injury feature map and the second brain injury feature map based on the color channel and the spatial position to obtain a fused feature map, including: convolving and reshaping the first brain injury feature map through the first interface of the cross-modal attention module to obtain a reshaped first brain injury feature map; convolving and reshaping the second brain injury feature map through the second interface of the cross-modal attention module to obtain a reshaped second brain injury feature map, and performing point multiplication and normalization processing on the second brain injury feature map with the reshaped first brain injury feature map to obtain an initial fused feature map; convolving and reshaping the second brain injury feature map through the third interface of the cross-modal attention module to obtain a reshaped second brain injury feature map, and performing point multiplication and reshaping processing on the second brain injury feature map with the initial fused feature map to obtain a final fused feature map.
7. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.
8. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 5 is implemented.
