Brain image evaluation model training method and device, electronic equipment and storage medium
The cerebrospinal fluid region in brain images was segmented through convolutional neural networks and deep learning network models, extracted features and integrated them into fusion features, and trained a classifier to solve the problem that brain images for acute ischemic stroke rely on manual analysis, realize automated evaluation, and improve efficiency and unity.
Patent Information
- Application Number
- CN202510006181.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-03
- Publication Date
- 2025-05-27
AI Technical Summary
In the prior art, brain imaging evaluation of acute ischemic stroke depends on doctors' manual analysis, which is time-consuming and labor-intensive, and the evaluation results are difficult to unify.
Through convolutional neural network and deep learning network model, the cerebrospinal fluid region in brain images is segmented, the radiomics and deep learning characteristics are extracted, and the preset classifier is integrated into fusion characteristics. The brain image evaluation model is obtained to achieve automated evaluation.
No manual interpretation is required, which significantly improves the unity of brain imaging evaluation efficiency and evaluation results, and improves the accuracy and efficiency of evaluation.
Smart Images
Figure CN120047720A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of medical image analysis, and particularly to a training method, device, electronic device and storage medium for a brain image evaluation model. Background Art
[0002] Acute ischemic stroke (AIS) is an ischemic injury of brain tissue caused by the obstruction of blood vessels in the brain, usually manifested as sudden loss of neurological function, such as limb weakness, language disorder and loss of consciousness. AIS is one of the main causes of death and disability globally, especially with a relatively high incidence in the elderly population. Therefore, it is crucial to quickly and accurately evaluate the brain images of acute ischemic stroke.
[0003] However, the current image interpretation of acute ischemic stroke relies on the manual analysis of doctors, which is time-consuming and laborious, and the judgment criteria of different doctors may vary, resulting in difficult-to-unify evaluation results. Summary of the Invention
[0004] To solve the technical problems that the existing brain image evaluation relies on the manual analysis of doctors, is time-consuming and laborious, and the evaluation results are also difficult to unify, the present invention provides a training method, device, electronic device and storage medium for a brain image evaluation model. Through a convolutional neural network and a deep learning network model, it can effectively segment the cerebrospinal fluid region in the brain image, extract radiomics features including global features and local features, extract deep learning features through the deep learning network model, integrate the radiomics features and the deep learning features into fusion features, train a preset classifier to obtain a brain image evaluation model, without manual interpretation of medical images, effectively improving the efficiency of brain image evaluation and the unity of evaluation results.
[0005] In a first aspect, an embodiment of the present application provides a training method for a brain image evaluation model, the method comprising:
[0006] Obtain the brain image of an object and the grading label corresponding to the brain image;
[0007] Determine the cerebrospinal fluid region of the brain image based on a convolutional neural network and a deep learning network model;
[0008] Obtain the radiomics features and deep learning features of the cerebrospinal fluid region; the radiomics features include the local features and global features of the cerebrospinal fluid region; the deep learning features are extracted based on the deep learning network model;
[0009] Determine fusion features based on the radiomics features and the deep learning features;
[0010] Obtain the clinical features of the object;
[0011] Training a preset classifier based on fusion features, clinical features, and grading labels to obtain a brain image evaluation model; the brain image evaluation model is used to perform grading evaluation on brain images.
[0012] In an alternative embodiment, determining the cerebrospinal fluid region in a brain image based on a convolutional neural network and a deep learning network model includes:
[0013] Normalizing the voxels in the brain image to obtain a normalized brain image; the intensity values of the voxels in the normalized brain image all belong to a preset intensity range;
[0014] Removing the skull from the normalized brain image to obtain a brain parenchyma image;
[0015] Inputting the brain parenchyma image into a convolutional neural network to obtain a cerebrospinal fluid candidate region;
[0016] Inputting the cerebrospinal fluid candidate region into a deep learning network model for segmentation to determine the cerebrospinal fluid region.
[0017] In an alternative embodiment, the deep learning network model includes an encoder, a decoder, skip connections, and a global context module; the skip connections include a scale interaction fusion module and a deformable convolution module; the scale interaction fusion module is used to fuse the local features and global features extracted by the encoder; the deformable convolution module is used to extract the morphological difference features of the cerebrospinal fluid region.
[0018] In an alternative embodiment, before obtaining the radiomics features and deep learning features of the cerebrospinal fluid region, it includes:
[0019] Obtaining a local threshold according to the normalized brain image;
[0020] Based on the cerebrospinal fluid region, obtaining a binary label for each voxel in the normalized brain image; the binary label includes a positive label and a negative label; the positive label indicates that the voxel belongs to the cerebrospinal fluid region; the negative label indicates that the voxel does not belong to the cerebrospinal fluid region;
[0021] If the intensity value of the voxel corresponding to the negative label is less than the local threshold, replacing the negative label with the positive label to obtain a binary label map;
[0022] Determining the cerebrospinal fluid region based on the binary label map and the normalized brain image.
[0023] In an alternative embodiment, obtaining the radiomics features and deep learning features of the cerebrospinal fluid region includes:
[0024] Extracting the radiomics features of the cerebrospinal fluid region using a preset image feature extraction algorithm;
[0025] Obtaining the high-frequency feature information of the cerebrospinal fluid region;
[0026] Input the high-frequency feature information and cerebrospinal fluid region into the deep learning network model to obtain deep learning features.
[0027] In an alternative embodiment, determining the fusion features based on the radiomics features and deep learning features includes:
[0028] Screen out multiple radiomics screening features from the radiomics features based on the first screening algorithm;
[0029] Screen out multiple deep learning screening features from the deep learning features based on the second screening algorithm;
[0030] Screen out the fusion features from multiple radiomics screening features and multiple deep learning screening features based on the third screening algorithm.
[0031] In an alternative embodiment, after constructing the brain image evaluation model based on the radiomics features, deep learning features, and clinical features, it further includes:
[0032] Obtain the brain image of the object to be evaluated;
[0033] Determine the cerebrospinal fluid region to be evaluated in the brain image of the object to be evaluated;
[0034] Obtain the radiomics features and deep learning features of the cerebrospinal fluid region to be evaluated;
[0035] Obtain the clinical data of the object to be evaluated;
[0036] Input the radiomics features, deep learning features, and clinical data into the brain image evaluation model to determine the grading data of the brain image of the object to be evaluated.
[0037] In a second aspect, an embodiment of the present application provides a training device for a brain image evaluation model, and the device includes:
[0038] The first acquisition module is used to acquire the brain image of the object;
[0039] The determination module is used to determine the cerebrospinal fluid region in the brain image;
[0040] The second acquisition module is used to acquire the radiomics features and deep learning features of the cerebrospinal fluid region; the radiomics features include local features and global features of the cerebrospinal fluid region;
[0041] The third acquisition module is used to acquire the clinical features of the object;
[0042] The model construction module is used to construct a brain image evaluation model based on the radiomics features, deep learning features, and clinical features; the brain image evaluation model is used to perform grading evaluation on the brain image.
[0043] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, a code set, or an instruction set. The at least one instruction, at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the training method of the brain image evaluation model in the first aspect.
[0044] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores at least one instruction or at least one program. The at least one instruction or at least one program is loaded and executed by the processor to implement the training method of the brain image evaluation model in the first aspect.
[0045] In a fifth aspect, an embodiment of the present application provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the training method of the brain image evaluation model in the first aspect.
[0046] The training method, device, electronic device, and storage medium of the brain image evaluation model provided by the embodiments of the present application have the following technical effects:
[0047] Obtain the brain image of the subject and the grading label corresponding to the brain image; determine the cerebrospinal fluid region of the brain image based on a convolutional neural network and a deep learning network model; obtain the radiomics features and deep learning features of the cerebrospinal fluid region; the radiomics features include local features and global features of the cerebrospinal fluid region; the deep learning features are extracted based on the deep learning network model; determine the fusion features based on the radiomics features and the deep learning features; obtain the clinical features of the subject; train a preset classifier based on the fusion features, the clinical features, and the grading label to obtain a brain image evaluation model; the brain image evaluation model is used to perform grading evaluation on the brain image. Through the convolutional neural network and the deep learning network model, the present application can effectively segment the cerebrospinal fluid region in the brain image, extract radiomics features including global features and local features, extract deep learning features through the deep learning network model, integrate the radiomics features and the deep learning features into fusion features, and train a preset classifier to obtain a brain image evaluation model, without the need for manual interpretation of medical images, effectively improving the efficiency of brain image evaluation and the unity of evaluation results. Description of the Drawings
[0048] To more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.
[0049] Figure 1 is a schematic diagram of an application environment provided by an embodiment of the present application;
[0050] Figure 2 is a schematic flow chart of a method for training a brain image evaluation model provided by an embodiment of the present application Figure 1 ;
[0051] Figure 3 is a schematic flow chart of a method for training a brain image evaluation model provided by an embodiment of the present application Figure 2 ;
[0052] Figure 4 is a schematic flow chart of a method for evaluating brain images provided by an embodiment of the present application;
[0053] Figure 5 is a schematic structural diagram of a device for training a brain image evaluation model provided by an embodiment of the present application;
[0054] Figure 6 is a block diagram of the hardware structure of a server for a method for training a brain image evaluation model provided by an embodiment of the present application. Detailed implementation manners
[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application.
[0056] It should be noted that the terms "first", "second", etc. in the description, claims and above-mentioned drawings of this application are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of this application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or server that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.
[0057] Please refer to Figure 1 , Figure 1 which is a schematic diagram of an application environment provided by an embodiment of this application, including an image acquisition device 101 and a server 102.
[0058] In a possible embodiment, the image acquisition device 101 is used to acquire brain images of an object. Since the brain images in this application may include Computed Tomography (CT) images, Magnetic Resonance Imaging (MRI) images, ultrasound images, etc., correspondingly, the information acquisition device 101 for acquiring the corresponding images may include a CT scanner, a magnetic resonance device, an ultrasound instrument, etc.
[0059] In a possible embodiment, the server 102 receives the brain image of the object and the grading label corresponding to the brain image sent by the image acquisition device 101; determines the cerebrospinal fluid region of the brain image based on a convolutional neural network and a deep learning network model; obtains the radiomics features and deep learning features of the cerebrospinal fluid region; the radiomics features include the local features and global features of the cerebrospinal fluid region; the deep learning features are extracted based on the deep learning network model; determines the correlation feature fusion feature based on the radiomics features and the deep learning features; obtains the clinical features of the object; trains a preset classifier based on the correlation feature fusion feature, the clinical features and the grading label to obtain a brain image evaluation model; the brain image evaluation model is used to perform grading evaluation on the brain image. Through the convolutional neural network and the deep learning network model, it can effectively segment the cerebrospinal fluid region in the brain image, extract radiomics features including global features and local features, extract deep learning features through the deep learning network model, and integrate the radiomics features and the deep learning features into a fusion feature, train a preset classifier to obtain a brain image evaluation model, without manual interpretation of medical images, effectively improving the efficiency of brain image evaluation and the unity of evaluation results.
[0060] The following introduces a specific embodiment of a training method for a brain image evaluation model of the present application. Figure 2 It is a flowchart showing a training method for a brain image evaluation model provided by an embodiment of the present application. Figure 1 This specification provides method operation steps such as in the embodiment or flowchart, but based on routine or non-creative labor, it may include more or fewer operation steps. The step order listed in the embodiment is only one way among the execution orders of numerous steps and does not represent the only execution order. When the actual system or server product executes, it can be executed in the order shown in the embodiment or the drawing or in parallel (for example, in an environment of parallel processors or multi-threaded processing). Specifically, as Figure 2 shown, this method is applied to a server and may include:
[0061] S201: Obtain the brain image of the object and the grading label corresponding to the brain image.
[0062] S202: Determine the cerebrospinal fluid region in the brain image based on a convolutional neural network and a deep learning network model.
[0063] S203: Obtain the radiomics features and deep learning features of the cerebrospinal fluid region; the radiomics features include the local features and global features of the cerebrospinal fluid region; the deep learning features are extracted based on the deep learning network model.
[0064] S204: Determine the fusion features based on the radiomics features and the deep learning features.
[0065] S205: Obtain the clinical features of the object.
[0066] S206: Train a preset classifier with the fusion features, clinical features, and grading labels to obtain a brain image evaluation model; the brain image evaluation model is used to perform grading evaluation on brain images.
[0067] Figure 3 It is a flowchart showing a training method for a brain image evaluation model provided by an embodiment of the present application. Figure 2 This method may include:
[0068] S301: Obtain the brain image of the object and the grading label corresponding to the brain image.
[0069] In the embodiment of the present application, the object refers to the object that needs to perform image analysis, which can be a person or other animal. In the present application, a stroke patient is taken as an example for illustration.
[0070] It should be particularly noted that a large number of brain images of different patients and corresponding grading labels under different disease severities of the same patient are required for model training. For the convenience of description, a set of brain images of one patient is taken as an example for elaboration.
[0071] In a possible embodiment, the grading label corresponding to the brain image is used to indicate the grading or classification information of the brain image in radiology. The grading label may include two types: a primary label and a secondary label. The label is used to indicate that the morphological or density characteristics of the brain image belong to a specific category. For example, images with higher morphological or density uniformity are classified into the first category, corresponding to the primary label; images with complex morphology or uneven density are classified into the second category, corresponding to the secondary label.
[0072] Furthermore, since there are various characteristics of the cerebrospinal fluid region in the brain image and various classification methods for the image, there are also corresponding multiple grading labels. By accurately and meticulously classifying the brain image into multiple types through features in multiple dimensions, it is possible to perform special processing on a certain type of brain image, that is, the brain image corresponding to a specific grading label in the subsequent process, which is convenient for doctors to conduct further analysis.
[0073] Optionally, the brain image of the subject may include CT images, MRI images, ultrasound images, etc. In the embodiments of the present application, a plain head CT image is preferably used.
[0074] S302: Determine the cerebrospinal fluid region in the brain image based on a convolutional neural network and a deep learning network model.
[0075] In a possible embodiment, determining the cerebrospinal fluid region in the brain image includes:
[0076] S312: Perform normalization processing on the voxels in the brain image to obtain a normalized brain image.
[0077] Those skilled in the art are aware that the voxel intensity value refers to the image intensity or density at a certain three-dimensional coordinate position, which reflects the characteristics of the tissue or structure at that position. The voxel intensity value in a CT image is usually expressed in Hounsfield units (HU).
[0078] In the embodiments of the present application, the intensity values of the voxels in the normalized brain image all belong to a preset intensity range. Normalizing the intensity values of all voxels to the preset intensity range can facilitate subsequent processing of the brain image.
[0079] Specifically, assuming that the original minimum voxel intensity value is -1024, the original maximum voxel intensity value is 1024, and the original voxel intensity value to be normalized is 240, the voxel intensity value after normalization to [0, 1024] is
[0080] S322: Remove the skull from the normalized brain image to obtain a brain parenchyma image.
[0081] In a possible embodiment, the purpose of skull stripping is to remove the skull part in the CT image and only retain the brain tissue. The method of skull stripping is not limited herein. For example, since the density of the skull is different from that of the brain tissue, the voxel intensity value of the skull is also different from that of the brain tissue. The brain tissue and the skull can be distinguished by setting an intensity value threshold. Therefore, a smaller voxel intensity value range can be set to extract the brain tissue region and obtain the brain parenchyma image.
[0082] S332: Input the brain parenchyma image into a convolutional neural network to obtain the cerebrospinal fluid candidate region.
[0083] In a possible embodiment, it is necessary to segment the cerebrospinal fluid region in the brain parenchyma to extract relevant features of the cerebrospinal fluid region. First, a convolutional neural network is used for rough segmentation to frame the cerebrospinal fluid candidate region on the image.
[0084] In the embodiment of the present application, the convolutional neural network for rough segmentation is specifically a convolutional neural network based on the encoder-decoder structure (U-shaped Convolutional Network, U-Net). Specifically, the U-Net convolutional neural network includes an encoder, a decoder, and skip connections. Among them, the encoder part of the U-Net is usually composed of a series of convolutional layers and pooling layers. The encoder can extract low-level features such as the edges and textures of the brain image, gradually reduce the image size, and increase the depth of the feature map at the same time. The decoder part of the U-Net gradually restores the spatial resolution of the image through upsampling operations and generates a feature map with a higher resolution. After each upsampling step, there will be a convolutional operation to ensure the fine restoration of the features. The skip connections can combine the feature maps in the earlier stages of the encoder with the corresponding stage feature maps in the decoder, and the finally output is the cerebrospinal fluid candidate region.
[0085] S342: Input the cerebrospinal fluid candidate region into a deep learning network model for segmentation to determine the cerebrospinal fluid region.
[0086] In the embodiment of the present application, the deep learning network model includes an encoder, a decoder, skip connections, and a global context module; the skip connections include a scale interaction fusion module and a deformable convolution module; the scale interaction fusion module is used to fuse the local features and global features extracted by the encoder; the deformable convolution module is used to extract the morphological difference features of the cerebrospinal fluid region.
[0087] Specifically, the deep learning network model of this application includes an encoder with four downsampling layers, a decoder with four upsampling layers, skip connections, and a global context module. Among them, the decoder uses a Convolutional Neural Network (CNN) as a feature extractor to perform downsampling layer by layer to extract multi-scale features; the decoder gradually restores the spatial resolution of the feature map through upsampling operations.
[0088] Among them, the encoder includes four downsampling layers, which extract multi-scale features through layer-by-layer convolution operations and pooling operations. Each convolutional layer captures low-level features by learning local features of the image (such as edges, textures, etc.), and the pooling operation gradually reduces the spatial resolution and increases the depth of the feature map to help extract the abstract representation of the image. The decoder gradually restores the spatial resolution of the feature map through four upsampling layers. The upsampling operation is used to enlarge the feature map in order to restore a higher-resolution image. The decoder gradually restores the detailed information of the image and fuses it with the high-level features extracted in the encoder to enhance the ability to restore details.
[0089] A channel attention mechanism module is introduced in each layer of the decoder to achieve intelligent fusion of encoder and decoder features using a channel cross-fusion Transformer and channel cross-attention. The channel attention mechanism can dynamically adjust the importance of different channel features and enhance the response of key features. By weighting the feature channels, the network can focus on the more important cerebrospinal fluid regions and improve the segmentation accuracy.
[0090] A scale interaction fusion module and a deformable convolution module are added to the skip connections of the deep learning network model of this application. Among them, the scale interaction fusion module is used to fuse local features and global features. In order to better capture the long-range dependence between global context information and local structural features, a scale interaction fusion module based on the Transformer self-attention mechanism is added. This module can combine local information with global information and improve the network performance by fusing feature maps of different scales. The deformable convolution module is used to generate image features and adds a learnable offset field, which contains the learnable offset amount at each position in the feature map, thereby enhancing the network's feature extraction ability and enabling the network to adaptively match the shape of the cerebrospinal fluid and capture the morphological differences of the cerebrospinal fluid. At the last stage of the decoder, a global context module is introduced to capture the global context information of the entire image. The global context module is implemented through operations such as self-attention mechanism, global average pooling, and global convolution.
[0091] A scale interaction fusion module is added to the skip connection, aiming to fuse local features and global features. Local features help capture details, while global features help understand the overall structure of the image. By fusing feature maps of different scales, the network can obtain richer information and improve the accuracy of segmentation. The deformable convolution module enhances the network's feature extraction ability by introducing a learnable offset field. This offset field contains learnable offset amounts for each feature map position, and the network automatically adapts to the shape differences of cerebrospinal fluid through these offset amounts, enabling it to capture details in brain images more precisely. The global context module extracts the overall global information in the image through operations such as self-attention mechanism, global average pooling, and global convolution, helping the network to understand the overall brain image, thereby improving the accuracy and robustness of the segmentation results.
[0092] The deep learning network model of this application outputs an initial binary map corresponding to the cerebrospinal fluid region, which is used to represent whether each voxel on the original brain image belongs to the cerebrospinal fluid region.
[0093] After the cerebrospinal fluid candidate region is finely segmented by the deep learning network model, before feature extraction, it is also necessary to fill the holes in the cerebrospinal fluid region to further improve the accuracy of the cerebrospinal fluid region. The specific steps include:
[0094] S352: Obtain the local threshold according to the standardized brain image.
[0095] S362: Based on the cerebrospinal fluid region, obtain the binary label of each voxel in the standardized brain image.
[0096] In a possible embodiment, the binary label includes a positive label and a negative label; the positive label represents that the voxel belongs to the cerebrospinal fluid region; the negative label represents that the voxel does not belong to the cerebrospinal fluid region.
[0097] In the embodiment of this application, the positive label is denoted as 1 and the negative label is denoted as 0. That is to say, when the binary label corresponding to a voxel is 1, it belongs to the cerebrospinal fluid region, and when the binary label corresponding to a voxel is 0, it does not belong to the cerebrospinal fluid region.
[0098] S372: If the intensity value of the voxel corresponding to the negative label is less than the local threshold, replace the negative label with the positive label to obtain the binary label map.
[0099] In the embodiment of this application, the standardized brain image is segmented into multiple regions, and hole filling is performed separately. Taking one region A as an example:
[0100] Obtain the average threshold a of this region according to the intensity values of the voxels in the standardized region A, which is the local threshold of the overall image.
[0101] For a voxel in region A, if the intensity value is less than the average threshold a of this region, it indicates that the voxel is a hole to be filled, and the binary label of this voxel is recorded as 1. On the contrary, if the intensity value is greater than or equal to the average threshold a of this region, it indicates that the voxel is not a hole to be filled, and the binary label of this voxel is recorded as 0.
[0102] After filling the holes of each voxel in region A, continue to fill the holes of the voxels in other regions, and finally complete the hole filling of the entire cerebrospinal fluid region, obtaining the binary label corresponding to each voxel in the cerebrospinal fluid region, and forming a binary label map.
[0103] S382: Determine the cerebrospinal fluid region based on the binary label map and the standardized brain image.
[0104] Multiply the binary label map obtained after the above hole filling operation by the standardized brain image (gray scale image), and the gray scale image of the cerebrospinal fluid region can be obtained. Use the gray scale image of the cerebrospinal fluid region as the cerebrospinal fluid region after fine segmentation.
[0105] S303: Extract the radiomics features of the cerebrospinal fluid region using a preset image feature extraction algorithm.
[0106] In the embodiment of the present application, use the PyRadiomics image feature extraction tool to extract radiomics features from the cerebrospinal fluid region through a variety of image feature extraction algorithms.
[0107] Specifically, a total of 1874 radiomics features are extracted, including 7 types of features: 19 first-order gray scale statistical features, 16 three-dimensional shape features, 24 gray level co-occurrence matrix features, 16 gray level run length matrix features, 16 gray level size zone matrix features, 5 adjacent gray level difference matrix features, and 14 gray level dependence matrix features. In addition, apply 8 image filters such as Laplacian of Gaussian, wavelet, logarithm, square, square root, exponential, gradient, and three-dimensional local binary pattern to each original image to obtain the transformed image features. In addition, the wavelet transform also generates 8 three-dimensional high-pass (H) and low-pass (L) filter combinations, such as LLH, LHL, LHH, HLL, HLH, HHL, HHH, and LLL.
[0108] Through the above settings, basic intensity statistical features such as the average value, standard deviation, kurtosis, and skewness of the cerebrospinal fluid region can be extracted, and the spatial structure features and texture detail features of the cerebrospinal fluid region can also be extracted. By extracting these multi-dimensional and multi-scale features, the imaging attributes of the cerebrospinal fluid region can be characterized more accurately, providing strong data support for subsequent model construction, and improving the efficiency and accuracy of image evaluation.
[0109] S304: Obtain the high-frequency feature information of the cerebrospinal fluid region.
[0110] In the embodiments of the present application, high-frequency feature information of the cerebrospinal fluid region is extracted through discrete wavelet transform. Discrete wavelet transform is a signal decomposition method that can decompose an image into low-frequency components and high-frequency components. Among them, the low-frequency components contain the overall contour or global information of the image, while the high-frequency components represent the edge information in the horizontal, vertical, and diagonal directions of the image, highlighting textures and boundaries.
[0111] S305: Input the high-frequency feature information and the cerebrospinal fluid region into the deep learning network model to obtain deep learning features.
[0112] Take the grayscale image of the cerebrospinal fluid region as the input image and import it into the deep learning network model together with the high-frequency feature information. Standardize the grayscale values of the input image, and extract the feature map from the fourth downsampling activation layer of the deep learning network model. Perform global average pooling on the feature map to obtain deep learning semantic segmentation features. These features are used to construct a feature similarity adaptive feature library. Finally, according to the unsupervised clustering algorithm, the features are divided into two clusters, and then the similarity between the two clusters and the feature library is tested to select the most effective feature combination. Finally, a total of 512 deep learning features are extracted from the images of each patient.
[0113] By introducing a multi-input module based on two-dimensional wavelet decomposition, the high-frequency features containing edge information are input into the deep learning network model together, providing more accurate edge information and richer texture information to help the deep learning network model learn the edges of the cerebrospinal fluid, thereby solving the problems of low contrast and blurred boundaries of the cerebrospinal fluid.
[0114] S306: Obtain the clinical features of the object.
[0115] In the embodiments of the present application, the clinical features of the object may include 14 clinical routine features such as gender, age, admission time, recanalization treatment surgery method, time from onset to CT examination, time from onset to treatment (OTT), National Institutes of Health Stroke Scale (NIHSS), stroke subtype, hypertension, hyperglycemia, hyperlipidemia, atrial fibrillation, previous stroke, smoking history, etc. In addition, the clinical features of the object may also include 3 pathological features such as cerebrospinal fluid volume change value, ratio of cerebrospinal fluid volume to cranial content volume, and ratio of ischemic lesion volume to cerebrospinal fluid volume, totaling 17 clinical features.
[0116] S307: Screen out multiple radiomics screening features from the radiomics features based on the first screening algorithm.
[0117] In a possible embodiment, all radiomics features need to be transformed into the range of 0 to 1 by a normalization method before screening features, so as to eliminate the difference in value ranges between data.
[0118] S308: Screen out a plurality of deep learning screening features from the deep learning features based on the second screening algorithm.
[0119] In a possible embodiment, 50 radiomics screening features are preliminarily screened out from the radiomics features through Least Absolute Shrinkage and Selection Operator (LASSO) regression.
[0120] In a possible embodiment, 15 deep learning screening features are preliminarily screened out from the deep learning features through LASSO.
[0121] S309: Screen out fusion features from the plurality of radiomics screening features and the plurality of deep learning screening features based on the third screening algorithm.
[0122] In a possible embodiment, it is also necessary to perform alignment and fusion screening on the preliminarily screened radiomics screening features and the preliminarily screened deep learning screening features. Specifically, 50 radiomics features and 15 deep learning features are taken as a whole, and screened again through LASSO. Finally, 16 radiomics features and 9 deep learning features (including DL-31, DL_92, DL_136, DL_170, DL_199, DL_241, DL_323, DL_462, DL_502) are obtained, totaling 25 fusion features.
[0123] For the 25 features, the screening algorithm generates a corresponding feature correlation coefficient as the automated feature weighting coefficient. In addition, for the above features, based on the prior knowledge of experts in the field and historical data, by extracting statistical patterns from the prior knowledge of experts, the importance or contribution degree of each feature is gradually determined, and a weight value based on the prior knowledge of experts is assigned to each feature. The prior knowledge of experts is combined with the automated feature weighting of the screening algorithm to assign and adjust the weights of the features. For specific features, such as cerebrospinal fluid volume and shape-related features, higher weights can be given based on the prior knowledge of experts in the field and referring to experience.
[0124] In the embodiment of the present application, the first screening algorithm, the second screening algorithm, and the third screening algorithm all select the LASSO regression algorithm. In other possible embodiments, the screening algorithm can also select other algorithms such as the Minimum Redundancy Maximum Relevance (mRMR) algorithm.
[0125] S310: Screen out multiple clinical screening features from clinical features based on the fourth screening algorithm.
[0126] In a possible embodiment, 15 clinical features are subjected to univariate and multivariate logistic regression analyses to screen out the most significant clinical features. That is, clinical features are first screened through univariate logistic regression analysis, and the features with significant significance (p < 0.05) enter the subsequent multivariate logistic regression analysis for further screening. Finally, five significant clinical features, namely the NIHSS grade, stroke subtype, hyperglycemia, cerebrospinal fluid volume change value, and the ratio of cerebrospinal fluid volume to cranial volume, are screened out.
[0127] S311: Determine a preset classifier based on radiomics features.
[0128] In a possible embodiment, all radiomics features are input into classifiers such as Convolutional Neural Network (CNN), NuSVC, Logistic Regression (LR), Support Vector Machine (SVM), Adaptive Boosting (Adaboost), and Random Forest (RF) for 5-fold cross-training and external test set verification. By comparison, the best-performing preset classifier NuSVC is obtained.
[0129] S312: Train the preset classifier based on the fusion features, clinical features, and grading labels to obtain a brain image evaluation model.
[0130] In the embodiment of the present application, the weighted features are input into the preset classifier, and finally a model capable of accurately evaluating brain images is trained. Finally, the trained brain image evaluation model can assign a reasonable label to the input brain image and clinical features.
[0131] The differences between the clinical model constructed by single features, the clinical-radiomics model constructed by two features, and the brain image evaluation model of the present application are compared through Delong test, and the ROC curve, nomogram, calibration curve, decision curve analysis (DCA), etc. of the corresponding models are drawn. According to the comparison of the performance such as the ROC curve and AUC value of different models, the brain image evaluation model of the present application has a significant improvement in evaluation accuracy compared with other models.
[0132] Figure 4 It is a schematic flowchart of a brain image evaluation method provided by an embodiment of the present application. The method may include:
[0133] S401: Obtain the brain image of the object to be evaluated.
[0134] S402: Determine the cerebrospinal fluid region to be evaluated in the brain image of the object to be evaluated.
[0135] S403: Obtain the radiomics features and deep learning features of the cerebrospinal fluid region to be evaluated.
[0136] S404: Obtain the clinical data of the object to be evaluated.
[0137] S405: Input the radiomics features, deep learning features and clinical data into the brain image evaluation model to determine the grading data of the brain image of the object to be evaluated.
[0138] In the embodiment of the present application, after the brain image evaluation model is trained, when it is necessary to evaluate and grade the brain image, first, obtain the brain image of the object to be evaluated, roughly segment and then finely segment the cerebrospinal fluid region from the brain image. Then, extract the radiomics features and deep learning features from the cerebrospinal fluid region. At the same time, obtain the clinical data of the object to be evaluated, especially three significant clinical features of the patient's NIHSS grading, stroke subtype and hyperglycemia. Finally, determine the fusion features based on the radiomics features and deep learning features, and input the fusion features and clinical data into the brain image evaluation model to finally obtain the grading data of the brain image of the object to be evaluated.
[0139] The embodiment of the present application also provides a training device for a brain image evaluation model. Figure 5 It is a schematic structural diagram of a training device for a brain image evaluation model provided by an embodiment of the present application. As Figure 5 shown, the device 500 includes:
[0140] The first acquisition module 501 is used to acquire the brain image of the acquisition object and the grading label corresponding to the brain image;
[0141] The first determination module 502 is used to determine the cerebrospinal fluid region in the brain image based on the convolutional neural network and the deep learning network model;
[0142] The second acquisition module 503 is used to acquire the radiomics features and deep learning features of the cerebrospinal fluid region; the radiomics features include the local features and global features of the cerebrospinal fluid region; the deep learning features are extracted based on the deep learning network model;
[0143] The second determination module 504 is used to determine the fusion features based on the radiomics features and deep learning features;
[0144] The third acquisition module 505 is configured to acquire the clinical features of the object;
[0145] The model training module 506 is configured to train a preset classifier based on the fusion features, clinical features, and grading labels to obtain a brain image evaluation model; the brain image evaluation model is used to perform grading evaluation on brain images.
[0146] In an optional implementation manner, it further includes:
[0147] The fourth acquisition module is configured to perform normalization processing on the voxels in the brain image to obtain a normalized brain image; the intensity values of the voxels in the normalized brain image all belong to a preset intensity range;
[0148] The fifth acquisition module is configured to remove the skull from the normalized brain image to obtain a brain parenchyma image;
[0149] The sixth acquisition module is configured to input the brain parenchyma image into a convolutional neural network to obtain a cerebrospinal fluid candidate region;
[0150] The first determination module is configured to input the cerebrospinal fluid candidate region into a deep learning network model for segmentation to determine the cerebrospinal fluid region.
[0151] In an optional implementation manner, the deep learning network model includes an encoder, a decoder, skip connections, and a global context module; the skip connections include a scale interaction fusion module and a deformable convolution module; the scale interaction fusion module is configured to fuse the local features and global features extracted by the encoder; the deformable convolution module is configured to extract the morphological difference features of the cerebrospinal fluid region.
[0152] In an optional implementation manner, it further includes:
[0153] The seventh acquisition module is configured to obtain a local threshold according to the normalized brain image;
[0154] The eighth acquisition module is configured to obtain a binary label for each voxel in the normalized brain image based on the cerebrospinal fluid region; the binary label includes a positive label and a negative label; the positive label indicates that the voxel belongs to the cerebrospinal fluid region; the negative label indicates that the voxel does not belong to the cerebrospinal fluid region;
[0155] The ninth acquisition module is configured to replace the negative label with the positive label when the intensity value of the voxel corresponding to the negative label is less than the local threshold to obtain a binary label map;
[0156] The third determination module is configured to determine the cerebrospinal fluid region based on the binary label map and the normalized brain image.
[0157] In an optional implementation manner, it further includes:
[0158] The first feature extraction module is used to extract radiomics features of the cerebrospinal fluid region by using a preset image feature extraction algorithm;
[0159] The tenth acquisition module is used to acquire high-frequency feature information of the cerebrospinal fluid region;
[0160] The second feature extraction module is used to input the high-frequency feature information and the cerebrospinal fluid region into a deep learning network model to obtain deep learning features.
[0161] In an optional implementation manner, it further includes:
[0162] The first screening module is used to screen out a plurality of radiomics screening features from the radiomics features based on a first screening algorithm;
[0163] The second screening module is used to screen out a plurality of deep learning screening features from the deep learning features based on a second screening algorithm;
[0164] The third screening module is used to screen out fusion features from the plurality of radiomics screening features and the plurality of deep learning screening features based on a third screening algorithm.
[0165] In an optional implementation manner, it further includes:
[0166] The twelfth acquisition module is used to acquire the brain image of the object to be evaluated;
[0167] The fifth determination module is used to determine the cerebrospinal fluid region to be evaluated in the brain image of the object to be evaluated;
[0168] The thirteenth acquisition module is used to acquire the radiomics features and deep learning features of the cerebrospinal fluid region to be evaluated
[0169] The fourteenth acquisition module is used to acquire the clinical data of the object to be evaluated;
[0170] The sixth determination module is used to input the radiomics features, deep learning features and clinical data into a brain image evaluation model to determine the grading data of the brain image of the object to be evaluated.
[0171] The device in the embodiments of the present application and the method embodiments are based on the same application concept.
[0172] The method embodiments provided by the embodiments of the present application can be executed on a computer terminal, a server or a similar computing device. Taking running on a server as an example, Figure 6 It is a hardware structure block diagram of a server for a training method of a brain image evaluation model provided by the embodiments of the present application. As Figure 6As shown, the server 600 can vary significantly due to different configurations or performances, and may include one or more central processing units (CPUs) 610 (the processor 610 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 630 for storing data, and one or more storage media 620 for storing application programs 623 or data 622 (such as one or more mass storage devices). Among them, the memory 630 and the storage media 620 can be transient storage or persistent storage. The programs stored in the storage media 620 may include one or more modules, and each module may include a series of instruction operations on the server. Further, the central processor 610 can be set to communicate with the storage media 620 and execute a series of instruction operations in the storage media 620 on the server 600. The server 600 may also include one or more power supplies 660, one or more wired or wireless network interfaces 650, one or more input / output interfaces 640, and / or one or more operating systems 621, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
[0173] The input / output interface 640 can be used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by the communication provider of the server 600. In one example, the input / output interface 640 includes a network interface controller (NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one example, the input / output interface 640 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0174] Those of ordinary skill in the art can understand that Figure 6 the structure shown is only schematic and does not limit the structure of the above electronic device. For example, the server 600 may also include more or fewer components than Figure 6 shown therein, or have a different configuration from Figure 6 that shown.
[0175] An embodiment of the present application provides an electronic device, which includes a processor and a memory. At least one instruction, at least one program, a code set, or an instruction set is stored in the memory, and the at least one instruction, at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the above method for training a brain image evaluation model.
[0176] Embodiments of the present application also provide a computer-readable storage medium, which can be disposed in a server to store at least one instruction, at least one program, a code set or an instruction set related to a training method of a brain image evaluation model in a method embodiment. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the above-mentioned training method of the brain image evaluation model.
[0177] Optionally, in this embodiment, the above storage medium may be located in at least one of multiple network servers in a computer network. Optionally, in this embodiment, the above storage medium may include, but is not limited to: various media that can store program codes such as USB flash drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), mobile hard disks, magnetic disks, or optical discs.
[0178] As can be seen from the embodiments of the training method, device, electronic device, or storage medium of the brain image evaluation model provided by the present application, in the present application, a brain image of an object and a grading label corresponding to the brain image are obtained; the cerebrospinal fluid region of the brain image is determined based on a convolutional neural network and a deep learning network model; radiomics features and deep learning features of the cerebrospinal fluid region are obtained; the radiomics features include local features and global features of the cerebrospinal fluid region; the deep learning features are extracted based on the deep learning network model; a fusion feature is determined based on the radiomics features and the deep learning features; clinical features of the object are obtained; a preset classifier is trained based on the fusion feature, the clinical feature, and the grading label to obtain a brain image evaluation model; the brain image evaluation model is used to perform grading evaluation on brain images. Through the convolutional neural network and the deep learning network model, the present application can effectively segment the cerebrospinal fluid region in the brain image, extract radiomics features including global features and local features, extract deep learning features through the deep learning network model, integrate the radiomics features and the deep learning features into a fusion feature, train a preset classifier, and obtain a brain image evaluation model, without manual interpretation of medical images, effectively improving the efficiency of brain image evaluation and the unity of evaluation results.
[0179] It should be noted that: the above sequence of embodiments of the present application is only for description and does not represent the advantages or disadvantages of the embodiments. And the above specific embodiments of this specification have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired results. Additionally, the processes depicted in the figures do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0180] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other, and the differences between each embodiment and other embodiments are emphasized. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for the relevant parts, reference can be made to the description of the method embodiments.
[0181] Those of ordinary skill in the art can understand that all or part of the steps for implementing the above embodiments can be completed by hardware or by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium, and the storage medium mentioned above can be a read-only memory, a magnetic disk, an optical disk, or the like.
[0182] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A training method for a brain imaging assessment model, characterized in that: include: Acquiring a brain image of a subject and a classification label corresponding to the brain image; Determining the cerebrospinal fluid region of the brain image based on a convolutional neural network and a deep learning network model; Acquiring radiomic features and deep learning features of the cerebrospinal fluid region; the radiomic features include local features and global features of the cerebrospinal fluid region; The deep learning features are extracted based on the deep learning network model; Determining a fusion feature based on the radiomics feature and the deep learning feature; obtaining clinical characteristics of the subject; A preset classifier is trained based on the fusion features, the clinical features and the grading labels to obtain a brain image evaluation model; the brain image evaluation model is used to perform graded evaluation on brain images.
2. A method for training a brain imaging assessment model according to claim 1, characterized in that: The determining of the cerebrospinal fluid area in the brain image based on the convolutional neural network and the deep learning network model includes: Performing standardization processing on the voxels in the brain image to obtain a standardized brain image; the intensity values of the voxels in the standardized brain image all belong to a preset intensity range; removing the skull from the standardized brain image to obtain a brain parenchyma image; Inputting the brain parenchyma image into the convolutional neural network to obtain a candidate cerebrospinal fluid region; The candidate cerebrospinal fluid region is input into the deep learning network model for segmentation to determine the cerebrospinal fluid region.
3. The training method of a brain imaging assessment model according to claim 2, characterized in that: The deep learning network model includes an encoder, a decoder, a skip connection and a global context module; the skip connection includes a scale interaction fusion module and a deformable convolution module; the scale interaction fusion module is used to fuse the local features and global features extracted by the encoder; The deformable convolution module is used to extract the morphological difference features of the cerebrospinal fluid region.
4. The training method of a brain imaging assessment model according to claim 2, characterized in that: Before obtaining the radiomic features and deep learning features of the cerebrospinal fluid region, the method includes: acquiring a local threshold according to the standardized brain image; Based on the cerebrospinal fluid region, a binary label of each voxel in the standardized brain image is obtained; the binary label includes a positive label and a negative label; the positive label indicates that the voxel belongs to the cerebrospinal fluid region; the negative label indicates that the voxel does not belong to the cerebrospinal fluid region; If the voxel intensity value corresponding to the negative label is less than the local threshold, the negative label is replaced by the positive label to obtain a binary label map; The cerebrospinal fluid region is determined based on the binary label map and the standardized brain image.
5. The training method of a brain image assessment model according to claim 2, characterized in that: The obtaining of the radiomics features and deep learning features of the cerebrospinal fluid region includes: Extracting the radiomics features of the cerebrospinal fluid region using a preset image feature extraction algorithm; Acquiring high-frequency characteristic information of the cerebrospinal fluid region; The high-frequency feature information and the cerebrospinal fluid region are input into the deep learning network model to obtain the deep learning features.
6. A method for training a brain image assessment model according to claim 5, characterized in that: The determining of the fusion feature based on the radiomics feature and the deep learning feature comprises: Screening out a plurality of radiomics screening features from the radiomics features based on a first screening algorithm; Filtering out a plurality of deep learning screening features from the deep learning features based on a second screening algorithm; The fusion feature is screened out from the plurality of radiomics screening features and the plurality of deep learning screening features based on a third screening algorithm.
7. The training method of a brain image assessment model according to claim 1, characterized in that: After constructing the brain imaging assessment model based on the radiomics features, the deep learning features and the clinical features, the method further includes: Obtaining brain images of the subject to be evaluated; determining a cerebrospinal fluid region to be evaluated in a brain image of the subject to be evaluated; Acquiring radiomic features and deep learning features of the cerebrospinal fluid area to be evaluated; Obtaining clinical data of the subject to be evaluated; The radiomics features, the deep learning features and the clinical data are input into a brain image evaluation model to determine the grading data of the brain image of the subject to be evaluated.
8. A training device for a brain image assessment model, characterized in that: include: A first acquisition module, used to acquire a brain image of a subject and a classification label corresponding to the brain image; A first determination module is used to determine the cerebrospinal fluid area in the brain image based on a convolutional neural network and a deep learning network model; A second acquisition module is used to acquire radiomic features and deep learning features of the cerebrospinal fluid region; the radiomic features include local features and global features of the cerebrospinal fluid region; The deep learning features are extracted based on the deep learning network model; A second determination module is used to determine a fusion feature based on the radiomics feature and the deep learning feature; A third acquisition module is used to acquire clinical characteristics of the subject; A model training module is used to train a preset classifier based on the fusion features, the clinical features and the grading labels to obtain a brain image evaluation model; the brain image evaluation model is used to perform graded evaluation on brain images.
9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the training method of the brain imaging assessment model as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the training method of the brain imaging assessment model as described in any one of claims 1-7.