Method, system and equipment for segmenting ICH region in combination with brain window and blood window images

Abstract

The invention belongs to the field of medical image analysis, and particularly relates to an ICH region segmentation method, system and device combining brain window and blood window images. According to the invention, an intracranial hemorrhage detection model comprising a global encoder, a local encoder, a reverse guidance module, a feature fusion module, a decoder and an output module is constructed; therefore, the ICH region contained in the original CT image can be detected. Wherein the global encoder and the local encoder are respectively used for performing feature extraction and encoding on input blood window and brain window images; and the reverse guidance module adopts a reverse attention mechanism to generate a pre-segmentation image containing a pseudo tag to guide the segmentation of the brain window image and help the model to accurately position a bleeding area. The decoder and the local encoder form a U-shaped network, and the output of the feature fusion module is introduced into the decoding features. And the output module generates a final segmented image by using the hybrid pyramid fusion module. According to the invention, the problem that the segmentation precision of the ICH region is insufficient in the existing scheme is solved.

Description

Technical Field

[0001] The present invention belongs to the field of medical image analysis, and in particular relates to a method, system and device for segmenting an ICH region combining brain window and blood window images. Background Art

[0002] Intracranial hemorrhage (ICH) is a common type of stroke. Once it occurs, it will cause rapid damage to brain tissue, with a mortality rate of up to 40%-50%. Computer tomography (CT) technology can quickly provide doctors with information about the patient's lesion area and is the gold standard tool for diagnosing intracranial hemorrhage. However, the existing ICH diagnosis based on CT images requires experienced physicians to manually locate the bleeding area, which is time-consuming and laborious. In order to reduce the workload of physicians, technicians have developed a variety of machine learning-based computer-aided detection methods to identify ICH areas in CT images. Such as random forests and fuzzy C-means clustering, etc. However, these methods all rely on manual feature extraction. Affected by the imaging equipment, the grayscale values ​​of different skull tissues are very different, and the difficulty of feature extraction makes segmenting the bleeding area a challenging task.

[0003] In order to overcome the difficulty of feature extraction, deep learning models such as U-Net, U-Net++, and 3DU-Net have achieved good performance based on the encoder-decoder structure design. However, most existing intracranial hemorrhage segmentation models based on convolutional neural networks (CNN) still face many challenges. For example, the traditional deep learning model uses repeated strides and pooling operations, which inevitably reduce the resolution and make it difficult to refine; at the same time, the convolution operation of small convolution kernels limits the area of ​​concern of the model, resulting in the loss of global information and multi-scale pathological semantic information. Due to the low contrast and blurred boundaries between the hemorrhage area and normal tissue, most existing methods only use single-modality images to perform intracranial hemorrhage area segmentation tasks, and do not use the complementary information of different modality images, making it difficult to locate the intracranial hemorrhage area.

[0004] On this basis, many feature fusion mechanisms have been proposed to preserve multi-scale features. For example, ASPP extracts features of different scales through dilated convolutions with different expansion rates and concatenates and fuses them; Wang et al. proposed a hard attention mechanism to improve the model's attention to difficult-to-segment areas; the UNS-UCLAN team used three CNNs to extract features of different spatial scales and adopted an independent multilayer perceptron (MLP) network for classification. However, these methods only focus on local features of larger areas and do not truly capture global features. Transformer-based models have great advantages in global feature modeling. TransUNet uses a self-attention mechanism in the encoder part to encode the tokenized image blocks from the convolutional neural network (CNN) feature map into an input sequence for extracting global context, and combines high-resolution feature maps from the convolutional neural network to refine local features. The dice value of the segmentation result reaches 0.77. Huang et al. proposed ReMix-FFN to reintegrate the local context and global dependencies. However, the complex structure of the network and the large computational overhead have become a major pain point in clinical applications. In recent years, models based on convolutional multi-layer perceptrons, such as UNeXt, have achieved great success, but their simple network structure leads to serious loss of local detail information.

[0005] Currently, most models have only a single branch to extract specific pathological semantic information. The latest Mobile-Former solution uses the parallel design of MobileNet and Transformer, and designs a bidirectional bridge between branches to fully utilize the advantages of MobileNet's ability to perform local processing and Transformer's ability to perform global extraction. Methods such as MedicalTransformer, DoubleU-Net, and S-UNet also build bridges between different branches to give full play to the advantages of each branch. However, most of the above methods use two isomorphic networks to form a dual-branch segmentation model, and the global-local feature interaction strategy using heterogeneous networks for complementary has not been fully explored.

[0006] In summary, existing solutions still have the following defects when dealing with the identification and segmentation tasks of ICH areas: (1) Traditional networks mostly ignore the extraction of global features, while methods that focus on extracting global features will lead to the loss of local position information, and smaller bleeding areas are very easy to be missed. (2) Traditional methods often use homogeneous branches to build multi-branch network structures, pay more attention to information fusion, and do not fully utilize complementary heterogeneous features. (3) Traditional solutions are limited to single-modality images and do not consider the complementary relationship between different modality images. The above defects will make it difficult for existing solutions to achieve the expected segmentation effect of the ICH area in CT images. Summary of the invention

[0007] In order to solve the problem that the existing solutions are difficult to achieve the expected segmentation effect on the ICH area in CT images, the present invention provides a method, system and device for segmenting the ICH area combining brain window and blood window images.

[0008] The present invention is implemented by the following technical solutions:

[0009] A method for segmenting an ICH region combining brain window and blood window images comprises the following steps:

[0010] S1: Construct an intracranial hemorrhage detection model including a global encoder, a local encoder, a reverse guidance module, a feature fusion module, a decoder and an output module. The features of the intracranial hemorrhage detection model are as follows:

[0011] (1) The global encoder uses PVT as the backbone network and is used to extract and encode the input blood window image to obtain multi-level global features. Where i represents the number of layers in the model. (2) The local encoder uses Res2Net50 as the backbone network and is used to extract and encode the input brain window image to obtain multi-level local features. (3) The reverse guidance module is used to generate the edge feature map F based on the primary features output by the local encoder Edge ; Then, combined with the reverse attention mechanism, the edge feature map and the global and local features of the corresponding level are used to generate a pre-segmentation map containing pseudo labels (4) The feature fusion module uses a multimodal fusion module CMF, a contextual anchor attention module CAA, a dilated convolution module, and an efficient multi-scale attention module EMA to fully fuse the pre-segmentation map and local features, and then generate a fusion feature Z i (5) The decoder and the local encoder form a U-shaped network, and upsample the local features and fusion features after feature concatenation to generate a multi-level decoding feature D i (6) The output module uses the hybrid pyramid fusion module HPPF and the convolution module to process the output of the decoder, and then obtains the segmentation image P representing the segmentation result.

[0012] S2: Obtain a large number of medical images of different cases, pre-process them as sample images, and then form the required sample data set; divide the sample data set into a training set and a test set.

[0013] S3: Using the inclusion segmentation loss L Seg , global information encoding loss L P With the boundary loss L EdgeThe mixed loss Loss is used as the loss function; the training set and test set are used to train and test the intracranial hemorrhage detection model.

[0014] S4: retain the intracranial hemorrhage detection model that meets the requirements after training and testing, and use it to identify and segment the intracranial hemorrhage area of ​​the input medical image of the case, and output the corresponding segmented image.

[0015] As a further improvement of the present invention, the global encoder includes 4 downsampling modules. The local encoder includes 5 downsampling modules, each of which is composed of convolution, pooling, batch normalization, etc., and is finally activated by the RELU activation function and output. Among them, the output of the first downsampling module in the local encoder is used to generate an edge feature map; the remaining four downsampling modules and the four upsampling modules in the decoder form a U-shaped network.

[0016] As a further improvement of the present invention, the data processing process of the reverse guidance module is as follows:

[0017] First, the output local features of the first downsampling module in the local encoder are processed by the convolutional attention module CBAM to obtain the edge feature map. Secondly, the multi-level local features are processed by the coarse-grained patch attention module. After processing, the attention map Pc is obtained. Finally, the edge feature map F Edge , attention map Pc and multi-level global features Perform the following processing to obtain the pre-segmentation map

[0018]

[0019] In the above formula, R i Represents global features The reverse feature map of the attention map Pc; H(·) represents the interpolation operation, C[·] represents the concatenation according to the channel dimension; f0 represents the attention map Pc and the reverse feature map R i The fusion feature of f0 and edge feature map F edge The feature map obtained after fusion; MLP represents the processing of multi-layer perceptron; and Represents pixel-by-pixel addition and multiplication operations respectively; represents Conv 3*3 Represents a 3×3 convolution operation.

[0020] As a further improvement of the present invention, the feature fusion module uses the pre-segmentation graph to process the global features of each level in turn, and then generates the corresponding fusion feature Z i ; The process is as follows:

[0021] First, the global features and pre-segmentation maps are input to the CMF module for cross-modal feature information fusion. Then, the output of CMF is processed by 3*3 convolution and divided into two paths, one of which is processed by the CAA module to capture remote context information, and the other is processed by three cascaded dilated convolution operations with different dilation rates. Then, the outputs of the two paths are input to the EMA module for processing to obtain the output feature map F. Finally, the output feature map F and the output feature ALGM of the previous layer obtained by the feature fusion module are combined. i+1 Subtract the complementary features, and add them to F after convolution operation to obtain the fusion feature Z of the corresponding level. i .

[0022] As a further improvement of the present invention, the data processing process of the output module is as follows:

[0023] The feature maps D of each level output by the decoder i (i∈{1,2,3}) are upsampled to the same size and then concatenated according to the channel dimension to obtain the feature map F C . For the feature map F C Perform global average pooling and two global maximum poolings of different scales. After 1*1 convolution and RELU operations, the three pooling results are converted into three pooling feature maps. Flatten the global average pooling result into the output vector Z F The result is then weighted with the global maximum pooling result, and the final segmented image P is obtained after the calculation result passes through a 3*3 convolution operation.

[0024] As a further improvement of the present invention, the expression of the output module outputting the segmented image is as follows:

[0025]

[0026] In the above formula, is the feature of global maximum pooling, is the feature of global average pooling, g(·) represents the transformation operation of two one-dimensional convolutions; σ(·) represents the Sigmoid function; h and w represent the height and width of the feature map, respectively; α is a weight factor used to control the overall weight of the feature map of global average pooling.

[0027] As a further improvement of the present invention, in step S2, the processing of the data set includes:

[0028] S21: Extract the case brain CT image slices and manually annotated masks.

[0029] S22: Generate a brain window image and a blood window image using preset window parameters, and use the two and mask information together as a group of sample images to obtain an initial sample data set containing a large number of sample images.

[0030] S23: Retrieve sample images containing the intracranial hemorrhage area from the initial sample data set according to the mask information, perform data enhancement and normalization on the images, and obtain an expanded data set containing more sample data.

[0031] S24: Divide the expanded data set into a test set and a training set according to a preset ratio; wherein the sample images expanded by the data enhancement method and their associated original sample images are divided into the same test set or training set.

[0032] As a further improvement of the present invention, the calculation formula of the mixing loss Loss is as follows:

[0033]

[0034] In the above formula, P represents the predicted segmentation image; G represents the actual mask; L Dice represents the Dice loss function; L bce represents the BCE loss function; L focal represents the focal loss function; λ1, λ2, and λ3 are preset weights used to balance the various loss functions.

[0035] The present invention also includes a segmentation system for an ICH region combining brain window and blood window images, which includes: a data acquisition unit, a preprocessing unit and a segmentation network.

[0036] The data acquisition unit is used to obtain the original CT image of the detected user. The preprocessing unit is used to generate the corresponding brain window image and blood window image according to the original CT image using the preset window parameters. After the segmentation network completes the training and testing in the aforementioned ICH area segmentation method combining the brain window and blood window images, the intracranial hemorrhage detection model that meets the requirements is retained; and then the intracranial hemorrhage detection model is used to generate a segmentation image reflecting the intracranial hemorrhage area contained therein according to the input brain window image and blood window image of the detected user.

[0037] The present invention also includes a segmentation device for an ICH region combining brain window and blood window images, which includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, a segmentation system for an ICH region combining brain window and blood window images as described above is created, thereby realizing target recognition and segmentation of an intracranial hemorrhage region contained in an input CT image, and generating a corresponding segmented image.

[0038] The technical solution provided by the present invention has the following three main beneficial effects:

[0039] 1. The present invention uses multimodal image information to achieve rapid and accurate segmentation of intracranial hemorrhage areas. The model receives blood window images and brain window images as input, and uses obvious lesion information in the blood window image to guide rapid and accurate segmentation of the lesion area in the brain window image, thereby achieving information complementarity between modalities.

[0040] 2. The present invention adopts dual encoders to optimize the information interaction mechanism, improve the computational efficiency, and refine the segmentation results. It includes the use of feedback-guidance-fusion information interaction mode, using the blood window image segmentation result to form a pseudo-label map to guide the segmentation of the brain window image through the reverse guidance module, helping the model to accurately locate the bleeding area while eliminating redundant information in the brain window image, improving the computational efficiency, and adding the edge information obtained from the shallow feature map obtained from the brain window image to refine the segmentation edge. Then, the feature fusion module is used to perform cross-modal information fusion and local and global information fusion.

[0041] 3. The present invention can take into account both global and local feature extraction, and improve the efficiency of segmentation of small-area hemorrhage. Most patients have small hemorrhage areas. Most previous intracranial hemorrhage segmentation algorithms only focus on local or global information, resulting in the omission of small-area hemorrhage areas and the lack of detailed information of hemorrhage areas. The method of the present invention adopts a dual-branch encoder architecture, in which the local encoder is responsible for extracting local information and the global encoder captures global dependencies to break semantic isolation. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flowchart of the steps of the ICH region segmentation method combining brain window and blood window images provided in Example 1 of the present invention.

[0043] Figure 2 This is a model architecture diagram of the intracranial hemorrhage detection model designed in Example 1 of the present invention.

[0044] Figure 3 for Figure 2 Data transmission link of internal feature images of intracranial hemorrhage detection model.

[0045] Figure 4 This is an example of data of the original medical image obtained in Example 1 of the present invention.

[0046] Figure 5 This is a system topology diagram of the ICH region segmentation system combining brain window and blood window images provided in Example 2 of the present invention.

[0047] Figure 6 It is a comparison chart of the segmentation effect of the scheme of the present invention and the control group scheme in the performance control test experiment.

[0048] Figure 7 Visualization results of the feature image extracted by the local encoder in the ablation experiment.

[0049] Figure 8 Visualization results of the feature image extracted by the reverse guidance module in the ablation experiment.

[0050] Fig. 9 This is the visualization result of the feature image extracted by the feature fusion module in the ablation experiment.

[0051] Fig.10 Visualization results of the feature images extracted by the decoder in the ablation experiment.

[0052] Fig.11 Visualization results of the feature image extracted by the global encoder in the ablation experiment.

[0053] Fig.12 This is a comparison diagram of the segmentation effects between the complete solution of the present invention and the solution after removing some core modules in the ablation experiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0055] Example 1

[0056] In order to solve the problem of detecting and segmenting intracranial hemorrhage areas in CT images, this embodiment provides a new computer-aided detection method based on deep learning. Considering that the brain window image has detailed texture information of brain tissue and intracranial hemorrhage area, but the hemorrhage area is not obvious; the blood window image has obvious hemorrhage area, but the texture information of brain tissue is blurred. In order to comprehensively utilize the rich information in the two types of image data, this embodiment designs a Unet network architecture with a dual-branch encoder. In the network with a dual-branch encoder, this embodiment further introduces a "feedback-guidance-fusion" information interaction method composed of a reverse guidance module and a feature fusion module.

[0057] Specifically, in the scheme of this embodiment, the feature map extracted from the blood window image is input into the reverse guidance module to form a pseudo-label map to roughly locate the bleeding area. Then, the reverse attention mechanism is used to form a constraint on the segmentation of the bleeding area in the feature map of the brain window image, thereby guiding the brain window image segmentation. At the same time, the scheme of this embodiment multiplies the feature map of each blood window image with the pseudo-label map of the blood window image after reverse attention, filters out useless texture detail information and image noise, excludes redundant information in the brain window image, and makes the model focus on the unpredicted information within the contour of the pseudo-label map, thereby realizing the blood window image-guided brain window image segmentation. In the reverse guidance module, the reverse attention mechanism can guide the model to pay attention to the details outside the overall position of the object, while adding edge information detail information from the shallow layer of the network, guiding the model to accurately segment the bleeding area while refining the edges of the segmentation results.

[0058] At the same time, in this embodiment, the feature map of the brain window image and the feature map output by the reverse guidance module are jointly input into the feature fusion module, thereby realizing cross-modal information fusion and fusion between local and global information, and the obtained output is input into the corresponding decoder. Finally, the output of the decoder module is jointly input into the hybrid pyramid fusion module, and multi-level feature fusion is guided by global average pooling and global maximum pooling. Based on the above improvements, the segmentation result finally output by this embodiment can take into account the rich information of image information of different branch encoders and different modalities, thereby achieving a more accurate segmentation effect, overcoming the problem that the existing scheme is prone to losing small details and prone to mis-segmentation at tissue adhesions.

[0059] In detail, Figure 1 As shown, the ICH region segmentation method provided in this embodiment combining brain window and blood window images includes the following steps:

[0060] S1: Construct an intracranial hemorrhage detection model including a global encoder, a local encoder, a reverse guidance module, a feature fusion module, a decoder and an output module. Figure 2 and Figure 3 As shown, the detailed features of the intracranial hemorrhage detection model are as follows:

[0061] (1) Global Encoder

[0062] In the solution of this embodiment, the global encoder is used as one of the branches of feature extraction, which uses PVT (Pyramid Vision Transformer) as the backbone network; this branch includes 4 downsampling modules. The global encoder receives the blood window image as input, and the extracted feature image is recorded as the global feature Wherein, i represents the number of layers of the model. In this embodiment, The image sizes of the global features obtained after encoding are 128, 64, 32, and 16, and the number of channels are 64, 128, 320, and 512, respectively.

[0063] In the global encoder adopted in this embodiment, the self-attention mechanism in the adopted Transformer structure is suitable for capturing the global dependencies of local features, and the progressively reduced pyramid structure of PVT enables the size of the feature map to be flexibly adjusted at each stage, thereby reducing the amount of parameters.

[0064] (2) Local Encoder

[0065] The local encoder of this embodiment serves as another feature extraction branch. This branch uses Res2Net50 as the backbone network. The branch includes a total of 5 downsampling modules. Each downsampling module consists of convolution, pooling, batch normalization, etc., and is finally activated by the RELU activation function and output. After each downsampling module, the image is reduced to half of its original size, which are 256, 128, 64, 32, and 16 respectively. The number of channels is 64, 256, 512, 1024, and 2048 respectively. In order to facilitate subsequent processing and reduce the amount of calculation, the number of feature map channels is unified to 64 through 1*1 convolution, followed by BatchNorm and RELU function activation. Finally, the local encoder of this embodiment can perform feature extraction and encoding on the input brain window image, thereby obtaining multi-level local features.

[0066] (3) Reverse guidance module

[0067] In this embodiment, the reverse guidance module includes a convolutional attention module CBAM, a convolutional module, a multi-layer perceptron (MLP), and a coarse-grained patch attention block. The reverse guidance module is used to obtain the primary features output by the local encoder. Generate edge feature map F Edge ; Then, combined with the reverse attention mechanism, the edge feature map and the global and local features of the corresponding level are used to generate a pre-segmentation map containing pseudo labels The data processing process of the reverse guidance module is as follows:

[0068] First, the local features of the output of the first downsampling module in the local encoder are processed by the convolutional attention module CBAM. Process and remove noise to obtain the edge feature map F Edge Secondly, the coarse-grained patch attention module is used to focus on the multi-level local features. The attention map Pc is generated by processing. The generation process of the attention map Pc specifically includes: Perform global average pooling on the channel, and then perform global average pooling on the feature map to extract the global information of each block; then calculate the weight of each block through full connection and convert it into the weight of the image shape. Finally, use the weight to weight the feature map to obtain the attention map Pc. The attention map contains the probability information of the existence of the target in different patches in the feature map.

[0069] Finally, the edge feature map F Edge , attention map Pc and multi-level global features Processing is performed to obtain the pre-segmentation map The detailed process includes: first, the global features output by the global encoder After the sigmoid function, it is inverted to form the reverse feature map R i , R i It is element-wise multiplied with the attention map Pc and then convolved with F Edge After the feature f is obtained, it is multiplied by the MLP operation to generate a boundary position information weight vector to guide its learning, and the obtained feature map is added to the coarse-grained attention map Pc. The above operation process is expressed as follows:

[0070]

[0071] In the above formula, R i Represents global features The reverse feature map of the attention map Pc; H(·) represents the interpolation operation, C[·] represents the concatenation according to the channel dimension; f0 represents the attention map Pc and the reverse feature map R i The fusion feature of f0 and edge feature map F edge The feature map obtained after fusion; MLP represents the processing of multi-layer perceptron; and Represents pixel-by-pixel addition and multiplication operations respectively; represents Conv 3*3 Represents a 3×3 convolution operation.

[0072] (4) Feature Fusion Module

[0073] The feature fusion module in this embodiment uses a multimodal fusion module CMF, a context anchor attention module CAA, a dilated convolution module, and an efficient multi-scale attention module EMA to perform an all-round fusion of the pre-segmentation map and local features, and then generate a fusion feature Z i .

[0074] Specifically, Figure 2 As shown in Figure 2, the first to third layers of the feature fusion module receive the pre-segmented images output by the corresponding layers of the reverse guidance module. The global information encoder corresponds to the global features of the output layer And the output ALGM of the next layer (i+1) of the feature fusion module i+1 As input. For the fourth adaptive local and global information fusion module A4, it receives the feature map from the local information encoder and the feature map from the global information encoder

[0075] In the data processing of the feature fusion module, and (or ) First, the input feature is fed into the cross-modal feature fusion module (CMF) for multi-modal feature information fusion. In CMF, the input feature is first passed through three 1*1 convolution Conv q , Conv k , Conv v Process the blood window image and brain window image and embed them into Q, K, and V. Then Q and K are deformed and transposed, and the attention weight is calculated through the self-attention mechanism. The attention output is obtained by weighting V with this weight. The attention output is convolved with the original input. (or ) are added together to get the final output.

[0076] The feature map output by CMF undergoes 3*3 convolution, and the number of feature map channels is adjusted to one-fourth of the original number, and then divided into two paths. One path passes through the contextual anchor attention module (CAA), which is responsible for capturing long-range context information. The corresponding operation can be expressed as:

[0077] G Global =C non-local (I1)

[0078] In the above formula, I1 represents the input feature of the CAA module; C non-local represents the processing of the contextual anchor attention module; G Global Represents the output features of the CAA module.

[0079] The other path passes through three cascaded dilated convolution modules with different dilation rates. The dilated convolution operations with different dilation rates can capture the local feature information of multi-level receptive fields. The dilation rates of the three dilated convolution modules are: 1*(6-i), 3*(6-i), and 5*(6-i); the input of each dilated convolution module is the concatenation of the output of the cross-modal feature fusion module and the output of the previous dilated convolution block.

[0080] Finally, all the feature maps input from the two paths are processed by the efficient multi-scale attention module (EMA) module, which captures spatial information at different scales. The feature map containing multi-scale spatial information output by the EMA module is then combined with the feature map ALGM from the i+1th layer of the feature fusion module. i+1 Subtract to obtain complementary features, which are added to the feature map containing multi-scale spatial information after convolution operation to obtain the output of the module, that is, the fusion feature Z i .

[0081] (5) Decoder

[0082] In this embodiment, combined with Figure 3 It can be seen that the decoder part includes 4 upsampling modules, each of which includes two 3*3 convolution operations and one interpolation operation to restore the image resolution to twice the original. Finally, the image is restored to the initial resolution through one interpolation operation. The 4 upsampling modules of the decoder in this embodiment and the last 4 downsampling modules in the local encoder form a U-shaped network. The decoder concatenates the local features and the fusion features and then upsamples them to generate a multi-level decoding feature D. i (i∈{1,2,3,4}).

[0083] (6) Output module

[0084] The output module provided in this embodiment is used to generate a final segmented image P according to the output of the decoder. Specifically, the output module processes the output of the decoder using a hybrid pyramid fusion module HPPF and a convolution module to obtain a segmented image P representing the segmentation result.

[0085] In detail, the output module of this embodiment converts the feature maps D of each level output by the decoder into i (i∈{1,2,3}) are upsampled to the same size and then concatenated according to the channel dimension to obtain the feature map F C Then the feature map F C Perform global average pooling and two global maximum pooling with pooling kernels of 4*4 and 8*8; the three pooling results are subjected to 1*1 convolution and RELU operations to obtain three pooling feature maps. Flatten the two global average pooling results into output vector Z F After that, it is weighted with the global maximum pooling result, and the result is processed through a 3*3 convolution operation to obtain the final segmented image P. The expression of the output module outputting the segmented image is as follows:

[0086]

[0087] In the above formula, is the feature of global maximum pooling, is the feature of global average pooling, g(·) represents the transformation operation of two one-dimensional convolutions; σ(·) represents the Sigmoid function; h and w represent the height and width of the feature map, respectively; α is a weight factor for controlling the overall weight of the feature map of global average pooling. In this embodiment, α=1.

[0088] S2: Obtain a large number of medical images of different cases, pre-process them as sample images, and then form the required sample data set; divide the sample data set into a training set and a test set.

[0089] In this embodiment, the process of collecting sample data and processing the training set and the test set includes the following steps:

[0090] S21: Extract the case brain CT image slices and manually annotated masks.

[0091] like Figure 4 As shown in the figure, two types of data can be obtained from the original CT medical image data in .nii format, one is the brain CT image slices of the case, and the other is the mask manually annotated by experts related to each CT image slice. Both types of data are saved in .png format. In the original data, different files are named in the format of "case number_slice number".

[0092] S22: Generate brain window image and blood window image using preset window parameters. In this embodiment, in order to obtain brain window image and blood window image, the corresponding window width and window position are adjusted as follows: brain window image (window position = 50, window width = 100); blood window image (window position = 60, window width = 40). The associated images obtained from the same CT image are kept separately, and the two and the mask information are used as a set of sample images. The same method is used to process each medical image, and finally an initial sample data set containing a large number of sample images can be obtained.

[0093] S23: Retrieve sample images containing the intracranial hemorrhage area from the initial sample data set according to the mask information, perform data enhancement and normalization on the images, and obtain an expanded data set containing more sample data.

[0094] Since the number of slices with hemorrhage areas in a single case is limited, in order to improve the model training efficiency, this embodiment first retrieves a completely black mask (corresponding to an image without hemorrhage areas), and deletes the brain CT images without hemorrhage areas by comparing the completely black mask. Finally, the remaining sample data are augmented by using operations such as horizontal flipping, vertical flipping, and random rotation of 90° to obtain an expanded data set. It is best to normalize all the data.

[0095] It should be additionally noted that in some schemes, sample images that do not contain intracranial hemorrhage areas in the sample data set can be retained, but when performing data enhancement, these data are excluded and not expanded.

[0096] S24: Divide the expanded data set into a test set and a training set according to a preset ratio; in this embodiment, the ratio of the training set to the test set is 90% for the training set and 10% for the test set. The sample images expanded by the data enhancement method and their associated original sample images are divided into the same test set or training set, thereby ensuring that there is no data image overlap between the test set and the validation set.

[0097] S3: Using the inclusion segmentation loss L Seg , global information encoding loss L P With the boundary loss L Edge The mixed loss Loss is used as the loss function; the training set and test set are used to train and test the intracranial hemorrhage detection model.

[0098] In this embodiment, the segmentation loss L Seg and the global information encoding loss L P A combination of Dice loss and BCE (binary cross-entropy) loss is used. This loss can use the Dice loss function to better evaluate the similarity between the segmentation result and the actual mask, and use the BCE loss to better promote the correctness of pixel classification.

[0099] Specifically, the calculation formula of the mixed loss Loss used in this embodiment is as follows:

[0100]

[0101] In the above formula, P represents the predicted segmentation image; G represents the actual mask; D i is the feature map output by the i-th layer decoder, is the feature map output by the i-th layer of the global encoder, F edge is the edge feature map, L Dice represents the Dice loss function; L bce represents the BCE loss function; L focal represents the focal loss function; λ1, λ2, and λ3 are preset weights used to balance the various loss functions.

[0102] S4: retain the intracranial hemorrhage detection model that meets the requirements after training and testing, and use it to identify and segment the intracranial hemorrhage area of ​​the input medical image of the case, and output the corresponding segmented image.

[0103] Combined with the previous content, it can be seen that most of the previous intracranial hemorrhage segmentation algorithms only focus on local or global information, resulting in the omission of small bleeding areas and the lack of detailed information of the bleeding area. The present embodiment and the provided solution adopt a dual-branch encoder architecture, in which the local information encoder is responsible for extracting local feature information, and the global information encoder is responsible for capturing the global dependencies of local features. Finally, through the "feedback-guidance-fusion" information interaction method constructed by the reverse guidance module and the feature fusion module and the hybrid pyramid fusion module in the output module, local and global information fusion and multi-level feature interaction are realized to break the local and global semantic isolation. The final segmentation result takes into account the image information of different branch encoders and different modalities; it not only plays the advantages of rich detailed information of brain window images, but also plays the obvious advantages of hemorrhage areas of blood window images, so that better segmentation results can be obtained.

[0104] Example 2

[0105] Based on the solution of Example 1, this embodiment further provides a segmentation system of the ICH region combining brain window and blood window images, such as Figure 5 As shown, it includes: a data acquisition unit, a preprocessing unit and a segmentation network.

[0106] The data acquisition unit is used to obtain the original CT image of the detected user. The preprocessing unit is used to generate the corresponding brain window image and blood window image according to the original CT image using the preset window parameters. After the segmentation network completes the training and testing in the aforementioned ICH area segmentation method combining the brain window and blood window images, the intracranial hemorrhage detection model that meets the requirements is retained; and then the intracranial hemorrhage detection model is used to generate a segmentation image reflecting the intracranial hemorrhage area contained therein according to the input brain window image and blood window image of the detected user.

[0107] The system provided in this embodiment can realize ICH detection, identification and image segmentation of the original CT image in one step, thereby solving the problem of intracranial hemorrhage areas missed by clinicians due to fatigue or lack of experience during the diagnosis and treatment process, and realizing rapid detection of intracranial hemorrhage, which has great clinical application value and market prospects.

[0108] Example 3

[0109] This embodiment provides a segmentation device for an ICH region combining brain window and blood window images, which includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, a segmentation system for an ICH region combining brain window and blood window images as shown in Example 2 is created, thereby realizing target recognition and segmentation of an intracranial hemorrhage region contained in an input CT image, and generating a corresponding segmented image.

[0110] The device for segmenting the ICH region of the combined brain window and blood window image provided in this embodiment is essentially a computer device. The computer device can be a smart phone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server or cabinet server (including an independent server, or a server cluster composed of multiple servers) capable of executing a program.

[0111] The computer device of this embodiment at least includes but is not limited to: a memory and a processor that can be connected to each other through a system bus. In this embodiment, the memory (i.e., a readable storage medium) includes a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory can be an internal storage unit of a computer device, such as a hard disk or a memory of the computer device. In other embodiments, the memory can also be an external storage device of a computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. Of course, the memory can also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the memory is generally used to store an operating system and various application software installed on the computer device. In addition, the memory can also be used to temporarily store various types of data that have been output or are to be output.

[0112] The processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor is generally used to control the overall operation of a computer device.

[0113] Performance Testing

[0114] In order to verify the effectiveness of the ICH region segmentation method of the combined brain window and blood window image provided by the present invention, the technicians formulated the following simulation experiment to train the network model and verify the segmentation performance of the network model.

[0115] 1. Experimental Content

[0116] In this experiment, we constructed Figure 2The intracranial hemorrhage detection model shown in the figure is then trained using sample data from the INSTANCE2022 dataset. The original data in the sample dataset contains a total of 100 cases. After data enhancement, this experiment uses 90% of the 893 images after removing all black images as the training set and 10% as the test set. After each round of training, 10% of the validation set data is used for testing immediately, and the best model is saved after 1,000 rounds of training.

[0117] During the training process, this experiment first loads the pre-trained model of Res2Net50 and the pre-trained model of PVT. During the training process, the batch size, learning rate and other parameters are adjusted until the model converges, that is, the loss of the training set decreases and tends to be stable, and the loss of the validation set also decreases and tends to be stable. During the training process, the final output of the model (that is, the result obtained by the final interpolation operation of the decoder part) and the decoder D i (i∈{1,2,3,4}) output, global information encoder The final output Output and edge feature map F Edge Conduct supervision and calculate losses.

[0118] 2. Control Experiment

[0119] In order to accurately evaluate the performance of the scheme of the present invention, this embodiment also compares the scheme of the present invention with several typical schemes that have been disclosed. The control group schemes include: Unet, U-net++, ConvUNeXt, UNeXt and MissFormer, among which Unet and U-net++ are classic medical image segmentation algorithms based on convolutional neural networks, ConvUNeXt and UNeXt are relatively new (2022) medical image segmentation algorithms based on convolutional MLP, and MissFormer is a medical image segmentation algorithm based on Transformer proposed in 2023.

[0120] 2.1. Visualization of Segmentation Results

[0121] First, this experiment visualizes the segmentation results of the scheme of the present invention and some control group schemes. The experimental results obtained are as follows: Figure 6 As shown. Figure 6 In the figure, (a) is the real mask, (b) is the segmentation result of the solution of the present invention, (c) is the segmentation result of Unet, (d) is the segmentation result of Unet++, (e) is the segmentation result of ConvUNeXt, and (f) is the segmentation result of UNeXt. The yellow arrows in the figure are the missing parts of the segmentation, the blue arrows are the parts where the segmentation does not match the mask, and the green circles are the parts where the model misjudges the brain tissue texture information as the hemorrhage area.

[0122] analyze Figure 6 The data can be found:

[0123] The segmentation result of the solution of the present invention is closest to the real mask, while the other four methods have different degrees of omission in detail area segmentation, and U-net++ and ConvUNeXt omission of details is more serious. In addition, the method proposed by the present invention is superior to the other four methods in the consistency between the segmented area and the mask and the distinction between brain tissue texture information and hemorrhage area.

[0124] 2.2 Quantification of Segmentation Performance

[0125] Furthermore, this experiment loads the weights of the trained model and inputs the images of the validation test set to test the model performance of each solution. During the experiment, the Recall, Specificity, Precision, Accuracy, Dice, IoU and other indicators are used to evaluate the model segmentation performance; the calculation formulas of each indicator are as follows:

[0126]

[0127] Among them, Recall represents the proportion of intracranial hemorrhage areas recognized by the model to the actual hemorrhage areas, Specificity represents the proportion of non-hemorrhage areas recognized by the model to the actual non-hemorrhage areas, Precision represents the proportion of actual positive classes in the positive classes predicted by the model, Accuracy represents the proportion of instances correctly predicted by the model to the total instances, Dice is used to measure the accuracy of image segmentation, and IoU is used to measure the similarity between the predicted target and the actual sample.

[0128] TP (True Positive): True positive, the actual area is the target area, and the predicted area is also the target area. FP (False Positive): False positive, the actual area is the target area, and the predicted area is the background. TN (True Negative): True negative, the actual area is the background, and the predicted area is the background; FN (False Negative): False negative, the actual area is the background, and the predicted area is the target area.

[0129] In the performance control experiment, the final performance of the present invention and the control group is shown in the following table:

[0130] Table 1: Performance test results of different schemes in the control experiment

[0131] recall specificity precision Dice / F1 IoU accuracy UNeX 0.7173 0.9989 0.7811 0.7124 0.6039 0.8581 Unet 0.7153 0.999 0.7627 0.7221 0.6155 0.8571 NestedUNet 0.6614 0.999 0.7252 0.6696 0.5708 0.8302 ConvUNeXt 0.7075 0.9989 0.7518 0.702 0.5994 0.8532 MissFormer 0.4301 0.5617 0.6511 0.4723 0.4297 0.5981 The present invention 0.7898 0.9992 0.8534 0.7959 0.6911 0.998

[0132] By analyzing the above experimental data, it can be found that the scheme of the present invention is superior to the three existing schemes in all indicators, and the segmentation performance is very outstanding.

[0133] 3. Ablation Experiment

[0134] In order to verify the contribution of different modules in the intracranial hemorrhage detection model to the outstanding performance achieved by the final solution in the solution provided by the present invention, technicians conducted ablation experiments.

[0135] 3.1 Visualization of module feature extraction results

[0136] This experiment firstly analyzes the feature image ( Figure 7-9 , Fig.11 ) and edge feature map F edge ( Fig.10 ) for visualization. In the feature map, the brighter the area, the higher the model's attention to the area. Analyzing the data in the figure, it can be found that the bleeding area in the feature map output by the local encoder is not very obvious, but there is an obvious bleeding area in the feature map output by the global feature encoder. The guided brain window image feature map segmentation from the blood window image is obtained through the edge feature map F edge By adding edge detail information, the bleeding area is more obvious in the feature map of the reverse guidance module. The feature map after the feature fusion module is blurred but the target area is still focused.

[0137] Therefore, the different functional modules in the intracranial hemorrhage detection model provided by the present invention can focus on different areas in the original medical image, which can help to extract more comprehensive feature information. This is actually the inherent reason why the scheme of the present invention ultimately obtains a more accurate segmentation result of the intracranial hemorrhage area.

[0138] 3.2 Comparison of Segmentation Effects

[0139] In this experiment, the technicians further compared the segmentation results of the complete solution of the present invention (b) with the true results (i.e., mask) (a), and the segmentation results after removing the feature fusion module ALGM (c), removing the reverse guidance module RCG (d), removing the hybrid pyramid fusion module HPPF in the output module (e), removing the feature fusion module and the hybrid pyramid fusion module (f), removing the reverse guidance module and the hybrid pyramid fusion module (g), and removing the reverse guidance module and the feature fusion module (h). The experimental results obtained are converted into thermal images such as Fig.12 shown. Fig.12 In the figure, the larger the colored area at the edge of the red target area, the more blurred the segmentation edge.

[0140] exist Fig.12In the figure, the closer the segmented images in each column are to the mask, the better the model segmentation effect is. As can be seen in the figure: the segmentation result of the complete solution in the second column (b) almost completely overlaps with the mask image, showing the most accurate segmentation effect. However, the segmentation performance after removing the reverse guidance module, feature fusion module, and hybrid pyramid fusion module has decreased. The segmentation accuracy has decreased more seriously after removing two modules. For example, the light blue part in the bottom image of the seventh column (g) is the part where the model misjudged the brain tissue as a hemorrhage area.

[0141] Furthermore, the test results of different schemes in various evaluation indicators in the ablation experiment are shown in the following table:

[0142] Table 2: Performance test results of different schemes in ablation experiments:

[0143] recall specificity precision Dice IoU accuracy Complete Solution 0.7898 0.9992 0.8534 0.7959 0.6911 0.998 No ALGM and HPPF 0.7961 0.9989 0.7866 0.7753 0.6662 0.9977 No RCG and HPPF 0.7576 0.9991 0.8454 0.7616 0.6534 0.9978 No ALGM and RCG 0.7416 0.9989 0.8169 0.7379 0.6304 0.9976 No HPPF 0.7966 0.999 0.833 0.7837 0.6774 0.9978 No RCG 0.784 0.999 0.8401 0.7766 0.6717 0.9979 No ALGM 0.7856 0.9988 0.8249 0.7784 0.6698 0.9977

[0144] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for segmenting an ICH region combining brain window and blood window images, characterized in that: It includes the following steps: S1: Construct an intracranial hemorrhage detection model including a global encoder, a local encoder, a reverse guidance module, a feature fusion module, a decoder and an output module; the intracranial hemorrhage detection model has the following features: (1) The global encoder uses PVT as the backbone network and is used to extract and encode the input blood window image to obtain multi-level global features. Wherein, i represents the number of layers of the model; (2) The local encoder uses Res2Net50 as the backbone network and is used to extract and encode the input brain window image to obtain multi-level local features. (3) The reverse guidance module is used to generate the primary features according to the output of the local encoder. Generate edge feature map F Edge ; Then, combined with the reverse attention mechanism, the edge feature map and the global features and local features of the corresponding level are used to generate a pre-segmentation map containing pseudo labels (4) The feature fusion module uses a multimodal fusion module CMF, a contextual anchor attention module CAA, a dilated convolution module, and an efficient multi-scale attention module EMA to comprehensively fuse the pre-segmentation map and local features, thereby generating a fused feature Z i (5) The decoder and the local encoder form a U-shaped network, and upsample the local features and fusion features after feature concatenation to generate a multi-level decoding feature D i ; (6) The output module processes the output of the decoder using a hybrid pyramid fusion module and a convolution module to obtain a segmentation image P representing the segmentation result; S2: obtaining a large number of medical images of different cases, pre-processing them as sample images, and then forming a required sample data set; and dividing the sample data set into a training set and a test set; S3: Using the inclusion segmentation loss L Seg , global information encoding loss L P With the boundary loss L Edge The mixed loss Loss is used as the loss function; the intracranial hemorrhage detection model is trained and tested using the training set and the test set; S4: retaining the intracranial hemorrhage detection model that meets the requirements after training and testing, and using it to identify and segment the intracranial hemorrhage area of ​​the input medical image of the case, and outputting the corresponding segmented image.

2. The ICH region segmentation method combining brain window and blood window images as claimed in claim 1, characterized in that: The global encoder includes 4 downsampling modules; The local encoder includes 5 downsampling modules, each of which is composed of convolution, pooling, batch normalization, etc., and is finally output after being activated by a RELU activation function; among them, the output of the first downsampling module is used to generate an edge feature map; the remaining four downsampling modules and the four upsampling modules in the decoder form a U-shaped network.

3. The ICH region segmentation method combining brain window and blood window images as claimed in claim 1, characterized in that: The data processing process of the reverse guidance module is as follows: First, the output local features of the first downsampling module in the local encoder are processed by the convolutional attention module CBAM to obtain the edge feature map F edge ; Secondly, the coarse-grained patch attention module is used to focus on the multi-level local features Processing is performed to obtain the attention map Pc; finally, the edge feature map F Edge , attention map Pc and multi-level global features Perform the following processing to obtain the pre-segmentation map In the above formula, R i Represents global features The reverse feature map of the attention map Pc; H(·) represents the interpolation operation, C[·] represents the concatenation according to the channel dimension; f0 represents the attention map Pc and the reverse feature map R i The fusion feature of f0 and edge feature map F edge The feature map obtained after fusion; MLP represents the processing of multi-layer perceptron; and Represents pixel-by-pixel addition and multiplication operations respectively; represents Conv 3*3 Represents a 3×3 convolution operation.

4. The ICH region segmentation method combining brain window and blood window images as claimed in claim 1, characterized in that: The feature fusion module uses the pre-segmentation graph to process the global features of each level in turn, and then generates the corresponding fusion feature Z i , the process is as follows: First, the global features and pre-segmentation maps are input into the CMF module for cross-modal feature information fusion; then, the output of CMF is processed by 3*3 convolution and divided into two paths, one of which is processed by the CAA module to capture long-range context information, and the other is processed by three cascaded dilated convolution operations with different dilation rates; then, the outputs of the two paths are input into the EMA module for processing to obtain the output feature map F; finally, the output feature map F and the output feature ALGM of the previous layer obtained by the feature fusion module are combined. i+1 Subtract the complementary features, and add them to F after convolution operation to obtain the fusion feature Z of the corresponding level. i .

5. The ICH region segmentation method combining brain window and blood window images as claimed in claim 1, characterized in that: The data processing process of the output module is as follows: The feature maps D of each level output by the decoder i (i∈{1,2,3}) are upsampled to the same size and then concatenated according to the channel dimension to obtain the feature map F C ; For the feature map F C Perform global average pooling and two global maximum poolings of different scales. The three pooling results are subjected to 1*1 convolution and RELU operations to obtain three pooling feature maps; flatten the global average pooling result into the output vector Z F The result is then weighted with the global maximum pooling result, and the final segmented image P is obtained after the calculation result passes through a 3*3 convolution operation.

6. The ICH region segmentation method combining brain window and blood window images as claimed in claim 5, characterized in that: The expression of the output module outputting the segmented image is as follows: In the above formula, is the feature of global maximum pooling, is the feature of global average pooling, g(·) represents the transformation operation of two one-dimensional convolutions; σ(·) represents the Sigmoid function; h and w represent the height and width of the feature map, respectively; α is a weight factor used to control the overall weight of the feature map of global average pooling.

7. The ICH region segmentation method combining brain window and blood window images as claimed in claim 1, characterized in that: In step S2, the data set processing process includes: S21: Extract the brain CT image slices of the case and the manually annotated masks; S22: Generate a brain window image and a blood window image using preset window parameters, and use the two and mask information as a group of sample images to obtain an initial sample data set containing a large number of sample images; S23: retrieving sample images containing the intracranial hemorrhage area in the initial sample data set according to the mask information, and performing data enhancement and normalization on the sample images to obtain an expanded data set containing more sample data; S24: Divide the expanded data set into a test set and a training set according to a preset ratio; wherein the sample images expanded by the data enhancement method and their associated original sample images are divided into the same test set or training set.

8. The ICH region segmentation method combining brain window and blood window images as claimed in claim 1, wherein in step S3, the calculation formula of the mixed loss Loss is as follows: In the above formula, P represents the predicted segmentation image; G represents the actual mask; L Dice represents the Dice loss function; L bce represents the BCE loss function; L focal represents the focal loss function; λ1, λ2, and λ3 are preset weights used to balance the various loss functions.

9. A system for segmenting ICH regions combining brain window and blood window images, characterized in that: It includes: A data acquisition unit, which is used to acquire the original CT image of the detected user; A preprocessing unit, which is used to generate corresponding brain window images and blood window images according to the original CT image using preset window parameters; A segmentation network, which adopts the intracranial hemorrhage detection model that meets the requirements and is retained after training and testing in the ICH area segmentation method combining brain window and blood window images as described in any one of claims 1 to 8; and then uses the intracranial hemorrhage detection model to generate a segmentation image reflecting the intracranial hemorrhage area contained therein according to the input brain window image and blood window image of the detection user.

10. A device for segmenting an ICH region combining brain window and blood window images, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, it creates the ICH area segmentation system combining the brain window and blood window images as described in claim 9, thereby realizing target recognition and segmentation of the intracranial hemorrhage area contained in the input CT image, and generating a corresponding segmented image.

Images