A stroke lesion segmentation method based on a progressive fusion network

Abstract

The application discloses a stroke lesion segmentation method based on a progressive fusion network, and comprises the following steps: a three-layer U-Net network is used to construct a multi-stage segmentation network, the multi-stage segmentation network is respectively a first-stage segmentation subnetwork, a second-stage segmentation subnetwork and a third-stage segmentation subnetwork, a stroke MRI image to be segmented is subjected to a shuffle operation to obtain three-level pyramid images with different resolutions, and the three-level pyramid images are used as input images of each stage; the input images of each stage are input into the segmentation subnetworks of each stage in turn for iteration, feature extraction is performed on the input images in the iteration process, and finally a segmentation image is output. The multi-stage segmentation network constructed by the application sets an encoding end cross-stage shared feature learning module and a decoding end terminal progressive adaptive fusion module, enhances the correlation between the segmentation subnetworks of each stage, improves the utilization rate of features extracted by the multi-stage structure and a segmentation result, and thus the precision of medical image segmentation is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image segmentation, and particularly relates to a stroke lesion segmentation method based on a progressive fusion network. BACKGROUND

[0002] Stroke, also known as "apoplexy" or "cerebral vascular accident" (CVA), is an acute cerebrovascular disease, and is one of the most common causes of adult death and disability in China. Every year, there are about 2.4 million new stroke patients and 1.1 million people die of stroke, which seriously endangers people's life safety. Ischemic stroke is a common type of stroke, accounting for more than 60% of stroke diseases. When ischemic stroke occurs, there are three states of brain tissue in the brain: infarcted infarcted area, ischemic penumbra at risk of infarction, and benign hypoperfusion area that will not infarct in a long time. If not treated in time, the ischemic penumbra area may evolve into an infarcted area within a few hours and be irreversible. Vascular recanalization therapy is the main treatment for ischemic stroke, but the intravenous thrombolysis strategy based on the 4.5-hour time window in the clinical guidelines is too harsh, resulting in more than 95% of cerebral ischemia patients being excluded. To solve this problem, individualized diagnosis and treatment based on diffusion weighted imaging (DWI)-perfusion weighted imaging (PWI) mismatching tissue window vascular recanalization is increasingly concerned in the clinic. The core of the tissue window is the accurate segmentation of the brain damage range. Due to the complexity of the disease, the manifestation of stroke damage in neuroimaging has great inter-individual and intra-individual differences. If the manual division by doctors with knowledge of neuroanatomy is relied on, the effect is not ideal. On the one hand, the accuracy of the quantitative identification result of the damage range size depends on the doctor's own experience, and on the other hand, the manual division work is time-consuming, which increases the doctor's work intensity. Therefore, how to automatically identify and segment the damage area in ischemic stroke medical images has attracted widespread attention.

[0003] In recent years, deep learning algorithms have been widely applied to the automatic segmentation of stroke lesions in medical images. Fully convolutional networks (FCN) and U-Net are two typical deep learning frameworks. FCN is a pioneering achievement in lesion segmentation using deep learning methods. It replaces the last fully connected layer of CNN with a convolutional layer, outputting images of the original size and labels for each voxel. The advantage of FCN is that it can achieve end-to-end segmentation, while the disadvantage is that the details of the segmentation results are not good enough. U-Net is a variant of FCN, which consists of a contraction path (encoder) and an expansion path (decoder). The contraction path consists of convolutional layers and down-sampling layers, which are used to encode images into abstract feature representations. The expansion path consists of convolutional layers and up-sampling layers, which are used to provide effective target information from abstract feature representations. Deep learning has a large number of parameters, multiple layers of abstraction, and strong representation learning capabilities, which can extract effective target information from complex background regions.

[0004] However, due to the varying sizes and shapes of stroke lesions, and the small differences between lesions and surrounding tissues, the single-stage U-Net network has difficulty in segmentation, easily losing smaller lesions, and cannot obtain satisfactory results. Currently, a multi-stage coarse-to-fine (MSCF) segmentation strategy is commonly used to solve the above problems. MSCF first uses a coarse segmentation model to obtain the approximate location information of the target, extracts the region of interest (ROI), and then uses a fine segmentation model to perform fine segmentation on the ROI region. Although the multi-stage method can effectively alleviate the sample imbalance problem caused by varying lesion sizes and improve the segmentation effect, the existing multi-stage method has the following defects: 1) The same input is used in different stages of the multi-stage model, making the segmentation tasks in different stages not targeted; 2) Most multi-stage models use a cascading approach, which ignores the relevance between different stages and only uses the segmentation results of the last stage, discarding other rich semantic information, making the multi-stage extracted features and multi-stage segmentation results not fully utilized; 3) In the constraints of the multi-stage model, the loss function only constrains the final segmentation result, which makes the deep features learned in the iteration training not directly fed back to the iteration training in the current stage.

[0005] To improve the accuracy of existing segmentation methods and improve the above-mentioned defects of the multi-stage model, the present application provides a medical image segmentation method based on a progressive fusion network, and applies it to ischemic stroke lesion segmentation. SUMMARY

[0006] The application aims to provide a stroke lesion segmentation method based on a progressive fusion network to improve the stroke lesion segmentation accuracy.

[0007] To achieve the above technical purposes, the application adopts the following technical scheme:

[0008] A stroke lesion segmentation method based on a progressive fusion network, specifically comprising the following steps:

[0009] Constructing a multi-stage model: a three-layer U-Net network is used to construct a multi-stage segmentation network, which is composed of a first-stage segmentation subnetwork, a second-stage segmentation subnetwork and a third-stage segmentation subnetwork from bottom to top. A cross-stage shared feature learning module is arranged between the encoding end of the first-stage segmentation subnetwork and the encoding end of the second-stage segmentation subnetwork, and between the encoding end of the second-stage segmentation subnetwork and the encoding end of the third-stage segmentation subnetwork, which is used for information interaction of different stage encoding processes. An adaptive fusion module is arranged at the end of the decoding end of the first-stage segmentation subnetwork, the decoding end of the second-stage segmentation subnetwork and the decoding end of the third-stage segmentation subnetwork, which is used for adaptive dynamic aggregation of the multi-scale segmentation results of the decoding end.

[0010] Image hierarchical processing: the stroke MRI image to be segmented is subjected to a shuffle operation to obtain three-level pyramid images of different resolutions, which are used as input images of each stage. The resolutions of the three-level pyramid images correspond to the first-stage segmentation subnetwork, the second-stage segmentation subnetwork and the third-stage segmentation subnetwork respectively.

[0011] Image segmentation: the obtained input images of each stage are input into the first-stage segmentation subnetwork, the second-stage segmentation subnetwork and the third-stage segmentation subnetwork in turn for iteration. Feature extraction is performed on the input images during the iteration process, and finally the segmentation image is output.

[0012] Further, the convolutional layer of the U-Net network adopts a convolutional block composed of a depthwise separable convolution.

[0013] Further, the process of the cross-stage shared feature learning module for information interaction of different stage encoding processes comprises the following steps:

[0014] The segmentation tasks of the first stage, the second stage and the third stage are defined as segmentation tasks X, Y and Z respectively. When the shared features are obtained at the i-th step, a forgetting gate is defined to select the information in the segmentation task Z of the previous stage segmentation subnetwork, and then the features obtained by the forgetting gate and the features extracted by the encoder of the next stage segmentation subnetwork are fused by a convolution operation to obtain the fusion features

[0015] define a memory gate two attention gates are set to determine the features that need to be memorized from the i-1th step to the i th step of the segmentation task Y of the next stage segmentation subnetwork and respectively used to output information of the forgetting gate and the memory gate , the attention gate is calculated by the Sigmoid function to obtain the output weight ratio of the forgetting gate and the memory gate , and the attention value is calculated by the Tanh function to determine the positive and negative of the information

[0016] aggregate the information controlled by the forgetting gate the memory gate the attention gate and the attention gate to construct the output feature of the segmentation task Y of the next stage segmentation subnetwork and input the output feature to the decoding end of the next stage segmentation subnetwork through the skip connection.

[0017] Further, the adaptive fusion module is used for the process of adaptively and dynamically aggregating the multi-scale segmentation results of the decoding end, which is:

[0018] upsample the output feature F z of the previous stage segmentation subnetwork to obtain a feature with the same size as the output feature F y of the next stage segmentation subnetwork; then use 1×1 convolution operation to compress the upsampled feature into single channel feature, and combine to obtain the fusion feature F s , F s =Con(up(F Z ))CCon(F y );

[0019] input the fusion feature F s into a convolution block with 4 layers and 3×3 convolution kernel size, extract features of the fusion feature through convolution, then extract the global certainty score predicted on the fusion scale through global average pooling GAP and global maximum pooling GMP, then connect the score from the fusion scale to the Sigmoid function to obtain the fusion attention feature map W s , W s =GAP(on( s ))+GMP(Con( s ));

[0020] fuse the attention feature map W sThe output feature map F of the former stage segmentation sub-network y The output of the progressive adaptive fusion module between the former stage segmentation sub-network and the latter stage segmentation sub-network is obtained after multiplication, and the progressive adaptive fusion module between the latter stage segmentation sub-network and the former stage segmentation sub-network outputs the final segmentation result.

[0021] Further, the bottom of each layer of the U-Net network is embedded with a non-local attention module for obtaining position-sensitive context information.

[0022] Further, the processing process of the non-local attention module is as follows:

[0023] The input feature map F is processed by a convolution layer with a convolution kernel size of 3x3 to reduce the channel number of the input feature map F from C0 to C, and a new feature map F is obtained. C0 / 8. H×W×C

[0024] Three new feature maps M1, M2 and M3 are generated by using a 1x1 convolution operation, and {M1, M2, M3}∈R H×W×C The feature maps M1 and M2 are deformed into R C×N where N=HxW is the number of pixels of the feature map, and the multiplication operation of the matrix M1 and the transposed matrix M2 is performed, and the result of the matrix multiplication is used to calculate a spatial attention map A∈R N×N , and where A ji is a measure of the influence of the i-th position on the j-th position.

[0025] The feature map M3 is deformed into R N×C , and the multiplication operation of the matrix M3 and the matrix A is performed, and the result is deformed into R C×H×W by transposition operation, and a non-local feature attention mechanism map is obtained. C×H×W The deformed result R is added to the feature map X2 element by element to obtain a feature map Z, and The obtained feature map Z is the sum of the relationship feature and the original feature.

[0026]

[0027] Further, the three-level pyramid images of different resolutions are obtained by shuffle operation, which are used as input images of each stage, and specifically:

[0028] ​The stroke MRI image to be segmented is input into a top-down shuffle operation based pyramid network input module, and three-level pyramid images of different resolutions are obtained through shuffle operation, the three-level pyramid images being 1 / 2 original image, 1 / 4 original image in size, and being input images of the first stage segmentation sub-network, the second stage segmentation sub-network and the third stage segmentation sub-network respectively.

[0029] Further, in the iterative process of the input image in the first stage segmentation sub-network, the second stage segmentation sub-network and the third stage segmentation sub-network, there is a corresponding segmentation result for each stage, and each stage of the segmentation result is constrained by a loss function through deep supervision, the loss function of each stage being composed of the sum of binary cross entropy loss and Dice loss, and the loss function expression of each stage being:

[0030] L s dice BCE

[0031]

[0032]

[0033] L s represents the loss function of each stage, L dice represents Dice loss, L BCE represents binary cross entropy loss, p i represents the output of the model, t i represents the segmentation label.

[0034] Further, the sum of the loss of each stage is the total loss L total , and there is:

[0035] L total s1 s2 s3

[0036] In the formula, L total represents the total loss of the segmentation result of the input image through the first stage segmentation sub-network, the second stage segmentation sub-network and the third stage segmentation sub-network, L s1 represents the loss function of the segmentation result of the first stage segmentation sub-network, L s2 represents the loss function of the segmentation result of the second stage segmentation sub-network, L s3 represents the loss function of the segmentation result of the third stage segmentation sub-network, and λ, β, δ are parameters obtained according to the different contribution degrees of the three stages to the overall segmentation task.

[0037] ​​​​​Preferably, for the parameters λ, β, δ, λ:β:δ=1:1:2.

[0038] After the above scheme, the present application has the following beneficial effects:

[0039] 1) The multi-stage progressive fusion medical image segmentation method provided by the present application provides a three-level pyramid image (multi-scale input image) for the multi-stage segmentation model through the pyramid network input module based on the shuffl operation; the multi-stage segmentation network is constructed, the correlation between the segmentation sub-networks of each stage is enhanced, the utilization rate of the features and segmentation results extracted by the multi-stage structure is improved, and finally a multi-stage progressive fusion segmentation network is formed by setting the encoding end cross-stage shared feature learning module and the decoding end progressive adaptive fusion module; finally, the network is constrained by three-stage deep supervision, so that the segmentation picture quality obtained by the network is better, thereby further improving the accuracy of medical image segmentation.

[0040] 2) In order to improve the accuracy of the existing segmentation method and improve the defects of the multi-stage model, the present application provides a medical image segmentation method based on a progressive fusion network, and better results are obtained when it is applied to ischemic stroke lesion segmentation

[0041] 3) The multi-stage segmentation network is constructed by using the U-Net network embedded with the depth separable convolution and the non-local attention module, so that more rich features can be extracted with less parameter amount and calculation amount. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other modified forms can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0043] Figure 1 is a step flow chart of a stroke lesion segmentation method based on a progressive fusion network of the present application;

[0044] Figure 2 is a multi-stage segmentation network structure framework diagram of the present application;

[0045] Figure 3 is a multi-stage progressive fusion segmentation network diagram of the present application;

[0046] Figure 4 is a cross-stage shared feature learning module structure diagram;

[0047] Figure 5 is a non-local attention module structure diagram;

[0048] Figure 6 Fig. 1 is a schematic diagram of a progressive fusion module structure;

[0049] Figure 7 Fig. 2 is a schematic diagram of three-stage deep supervision;

[0050] Figure 8 Fig. 3 is a schematic diagram of a stroke lesion MRI;

[0051] Figure 9 Fig. 4 is a schematic diagram of a segmentation result of a stroke ATLAS data set by the segmentation method of the present application. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0053] As shown in Figure 1 , the embodiment of the present application provides a stroke lesion segmentation method based on a progressive fusion network, which specifically comprises the following steps:

[0054] S1, constructing a multi-stage model: a three-layer U-Net network is used to construct a multi-stage segmentation network (three-stage multi-level encoding U-Net network), and the convolutional layer of the U-Net network adopts a convolutional block composed of a depth separable convolution to realize feature extraction; as shown in Figure 2 , the multi-stage segmentation network is composed of a first-stage segmentation sub-network, a second-stage segmentation sub-network and a third-stage segmentation sub-network from bottom to top, and a cross-stage shared feature learning module for information interaction in different stage encoding processes is arranged between the encoding end of the first-stage segmentation sub-network and the encoding end of the second-stage segmentation sub-network, and between the encoding end of the second-stage segmentation sub-network and the encoding end of the third-stage segmentation sub-network, i.e. a cross-stage shared feature learning module is arranged between the encoding ends of adjacent stage segmentation sub-networks, which realizes information interaction in different stage encoding processes; a progressive adaptive fusion module is arranged at the terminal end of the decoding end of the first-stage segmentation sub-network, the decoding end of the second-stage segmentation sub-network and the decoding end of the third-stage segmentation sub-network, i.e. a progressive adaptive fusion module is arranged at the terminal end of the decoding end of each stage segmentation sub-network, which is used for adaptively and dynamically aggregating the multi-scale segmentation results of the decoding end to form a multi-stage progressive fusion segmentation network as shown in Figure 3 .

[0055] S2, image hierarchical processing: the image to be segmented is processed in a hierarchical manner as shown inFigure 8 The stroke MRI image shown is obtained by a shuffle operation to obtain a three-level pyramid image (multi-scale input image) of different resolutions as an input image of each stage, which corresponds to the first stage segmentation subnetwork, the second stage segmentation subnetwork, and the third stage segmentation subnetwork in terms of resolution; in this embodiment, the three-level pyramid image of different resolutions obtained by the shuffle operation is used as the input image of each stage, specifically: the stroke MRI image to be segmented is input into a top-down pyramid network input module based on the shuffle operation, and a three-level pyramid image of different resolutions is obtained by the shuffle operation, which is 1 / 2 of the original image, 1 / 4 of the original image, and is used as the input image of the first stage segmentation subnetwork, the second stage segmentation subnetwork, and the third stage segmentation subnetwork, respectively. The original meaning of shuffle is to shuffle and mix, that is, to mix a set of regular data into irregular data as much as possible. The main reason for using the shuffle operation is that the shuffle operation can obtain multi-scale input images while maintaining the integrity of the original image information without increasing the number of network parameters.

[0056] S3, image segmentation: the obtained input images of each stage are input into the first stage segmentation subnetwork, the second stage segmentation subnetwork, and the third stage segmentation subnetwork of the multi-stage segmentation network (three-stage multi-level encoding U-Net network) in turn for iteration, and the input images are feature extracted in the iteration process, and finally a segmentation map is output. The feature extraction capability of the multi-stage segmentation network is improved in the iteration process, and the final segmentation map is output by the third stage segmentation subnetwork.

[0057] The above cross-stage shared feature learning module structure is as shown in Figure 4 The module can autonomously determine the retention and discard of feature information, segment the lesion from the surrounding tissue, and suppress low-level background noise (non-lesion area). The features obtained in the coarse segmentation process have rich details, and the selected features are applied to the fine segmentation of the next stage as supplementary information, so that beneficial features are transferred and useless or harmful information is filtered out; the following will take the cross-stage shared feature learning module between the first stage segmentation subnetwork encoding end and the second stage segmentation subnetwork encoding end as an example to illustrate the process of the cross-stage shared feature learning module for information interaction in different stage encoding processes, which specifically includes the following steps:

[0058] S101, define the segmentation tasks of the first stage, the second stage, and the third stage as segmentation tasks X, Y, and Z, respectively, and define a forgetting gate when sharing features at step i The information used to select the first stage segmentation subnetwork segmentation task Z is beneficial to the segmentation task Y of the second stage segmentation subnetwork, and then the information controlled by the forgetting gate and the features extracted by the second stage segmentation subnetwork encoder are fused by a convolution operation to obtain fused features

[0059] S102, define a memory gate used to determine the features that need to be memorized from the i-1th step to the i-th step of the segmentation task Y of the second stage segmentation subnetwork, and embed the attention mechanism into the proposed module, set two attention gates and respectively used to pay attention to the output information of the forgetting gate and the memory gate , prevent important information related to the lesion from being lost, the attention gate is calculated by the Sigmoid function forgetting gate and the output weight ratio of the memory gate ; the attention value is calculated by the Tanh function determine the positive and negative of the information; the Sigmoid function and the Tanh function are often used as the activation function of the neural network, the Sigmoid function maps the variable to (0, 1), and the Tanh function maps the variable to (-1, 1).

[0060] S103, aggregate the information controlled by the forgetting gate the memory gate the attention gate and the attention gate to construct the output features of the segmentation task Y of the second stage segmentation subnetwork and input the output features to the decoding end of the second stage segmentation subnetwork through the skip connection.

[0061] The above progressive adaptive module structure is shown in Figure 6 The module ensures that the final output of the model fully utilizes the segmentation results of each stage in the decoding path. The progressive adaptive fusion module dynamically aggregates the multi-scale segmentation results of the decoding end by determining the importance of different scale segmentation results in space, learns more global semantic representation relationships in the decoding path to adapt to the variable scale of cerebral apoplexy lesions, thereby improving the segmentation accuracy and refining the segmentation map. The process of dynamically aggregating the multi-scale segmentation results of the decoding end for adaptation is illustrated below by taking the adaptive fusion module of the decoding end of the first stage segmentation subnetwork and the decoding end of the second stage segmentation subnetwork as an example:

[0062] S201, the output features F zAn upsampling operation is performed to obtain the output features F of the second-stage segmentation subnetwork. y Features of the same size are used; then a 1×1 convolution operation is used to compress the upsampled features into single-channel features, which are then combined through channel concatenation operations to obtain the fused feature F. s F s =Con(up(F) Z ))CCon(F y );

[0063] S202, will fuse feature F s The input is fed into a 4-layer convolutional block with a 3×3 kernel. Convolution is used to extract features from the fused features. Then, global average pooling (GAP) and global max pooling (GMP) are used to extract the global deterministic score predicted at the fused scale. Finally, the score from the fused scale is concatenated into the sigmoid function to obtain the fused attention feature map W. s W s =GAP(on( s ))+GMP(Con( s ));

[0064] S203 will fuse the attention feature map W s Compared with the output feature map F of the second-stage segmentation sub-network y After multiplication, the output of the progressive adaptive fusion module between the first-stage segmentation subnetwork and the second-stage segmentation subnetwork is obtained. The progressive adaptive fusion module between the second-stage segmentation subnetwork and the first-stage segmentation subnetwork outputs the final segmentation result.

[0065] The structure of the non-local attention module described above is as follows: Figure 5 As shown, each layer of the U-Net network embeds a non-local attention module at its bottom for acquiring extensive location-sensitive contextual information. That is, the non-local attention module in each layer is embedded between the encoder and decoder of the U-Net network. The non-local attention module captures long-term dependencies between pixels in the image, acquiring extensive location-sensitive contextual information, thereby enhancing the shape feature information of lesions and achieving better single-stage segmentation. The processing procedure of the non-local attention module is as follows:

[0066] S301, The input feature map is processed by a convolutional layer with a kernel size of 3×3. The number of channels is reduced from C0 to C, resulting in a new feature map F∈R. H×W×C Where C = C0 / 8, this method can reduce the computational cost of feature extraction without significantly degrading performance;

[0067] S302. Use a 1×1 convolution operation to generate three new feature maps M1, M2, M3, where {M1, M2, M3} ∈ R.H ×W×C Transform feature maps M1 and M2 into R C×N Where N = H × W is the number of pixels in the feature map, then the matrix M1 is multiplied by its transpose M2, and the spatial attention map A ∈ R is calculated using the result of the matrix multiplication. N×N ,and Where A ji It measures the influence of the i-th position on the j-th position; the process of calculating the non-local attention mechanism is similar to Softmax. The closer two points are on feature maps M1 and M2, the higher the degree of correlation and relevance between them.

[0068] S303, Transform feature map M3 into R N×C Then perform multiplication of matrix M3 and matrix A, and transform the result into R by transpose. C×H×W Obtain a nonlocal feature attention mechanism map, and transform the deformed result R C×H×W Feature map Z is obtained by element-wise addition of feature map X2, and The resulting feature map Z is the sum of relational features and original features;

[0069] S304. Input the feature map Z into a convolutional layer with a kernel size of 3×3 to generate a feature map with the same shape as X1, and use X1 as a residual block for summation to avoid overfitting.

[0070] like Figure 7 As shown, in the image segmentation stage, the input image undergoes iterative processing through the first-stage segmentation subnetwork, the second-stage segmentation subnetwork, and the third-stage segmentation subnetwork. Each stage yields a corresponding segmentation result. This invention employs deep supervision to constrain the segmentation results of each stage with a loss function, which can preserve rich multi-scale features and enhance attention to target features. The loss function of each stage consists of the sum of binary cross-entropy loss and Dice loss. Dice loss can effectively alleviate the class imbalance problem in segmentation tasks. The specific expression is as follows:

[0071] L s =L dice +L BCE

[0072]

[0073]

[0074] L s L represents the loss function for each stage. dice Indicates Dice loss, L BCE p represents the binary cross-entropy loss. iThe output of the model, t i The label of the segmentation.

[0075] Binary cross-entropy loss is a widely used supervised learning technique that can be used to solve classification problems. It is an effective error loss function for calculating the performance error of the model; Dice loss is the most frequently used metric in medical image competitions, which is a set similarity metric, usually used to calculate the similarity of two samples, with a value threshold of [0, 1]. It is often used in medical images for image segmentation, and the best result of segmentation is 1 and the worst result is 0.

[0076] The sum of the loss of each stage is the total loss L total In order to improve the quality of the generated image of the network and give full play to the advantages of the multi-stage network, the application designs different proportion parameters for the loss function of each stage segmentation sub-network, to highlight the different contributions of different stages to the overall segmentation task, and has:

[0077] L total =λL s1 +βL s2 +δL s3

[0078] In the formula, L total represents the total loss of the segmentation result of the input image through the first stage segmentation sub-network, the second stage segmentation sub-network and the third stage segmentation sub-network, L s1 represents the loss function of the segmentation result of the first stage segmentation sub-network, L s2 represents the loss function of the segmentation result of the second stage segmentation sub-network, and L s3 represents the loss function of the segmentation result of the third stage segmentation sub-network, and λ, β and δ are parameters obtained according to the different contributions of the three stages to the overall segmentation task, and it is verified through experiments that when λ: β: δ = 1: 1: 2, the segmentation effect is the best.

[0079] As shown in Figure 9 , the result of the stroke lesion MRI image segmentation performed by the progressive fusion network constructed by the application is very close to the Groud Truth (true value).

[0080] As shown in Table 1, the Dice value obtained by the U-Net with different encoding and decoding layers is better.

[0081]

[0082] Table 1 Comparison of U-Net encoding and decoding layer results

[0083] As shown in Table 2 and Table 3, the effect of the application is better under the Dice value obtained by different loss function parameters.

[0084]

[0085] Table 2 Loss function ablation experiment result comparison table

[0086]

[0087] Table 3 Loss function parameter selection experiment result comparison table

[0088] In the description of the present specification, the description referring to the terms "one embodiment", "some embodiments", "an example", "a specific example" or "an optional embodiment" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0089] The above-described embodiments do not constitute a limitation on the protection scope of the technical solutions. Any modification, equivalent replacement and improvement made within the spirit and principles of the above-described embodiments shall be included in the protection scope of the technical solutions.

Claims

1. A method for stroke lesion segmentation based on progressive fusion networks, characterized in that... Including the following steps: Constructing a multi-stage model: A three-layer U-Net network is used to construct a multi-stage segmentation network, which consists of a first-stage segmentation sub-network, a second-stage segmentation sub-network, and a third-stage segmentation sub-network from bottom to top. Cross-stage shared feature learning modules for information exchange between different stages of the encoding process are set between the encoding ends of the first-stage segmentation sub-network and the second-stage segmentation sub-network, and between the encoding ends of the second-stage segmentation sub-network and the third-stage segmentation sub-network. Progressive adaptive fusion modules are set at the ends of the decoding ends of the first-stage segmentation sub-network, the second-stage segmentation sub-network, and the third-stage segmentation sub-network. These adaptive fusion modules are used to adaptively and dynamically aggregate the multi-scale segmentation results from the decoding ends. Image hierarchical processing: The stroke MRI images to be segmented are shuffled to obtain three-level pyramid images of different resolutions, which are used as input images for each stage. The resolution of the three-level pyramid images corresponds to the first stage segmentation subnetwork, the second stage segmentation subnetwork, and the third stage segmentation subnetwork, respectively. Image segmentation: The input images obtained at each stage are respectively input into the first stage segmentation sub-network, the second stage segmentation sub-network, and the third stage segmentation sub-network for sequential iteration. During the iteration process, features are extracted from the input images, and the final output is a segmentation map. The cross-stage shared feature learning module, used for information exchange in different stages of the encoding process, includes the following steps: The segmentation tasks in the first, second, and third stages are defined as segmentation tasks X, Y, and Z, respectively. During feature sharing in the i-th step, a forget gate is defined. This is used to select information from the previous segmentation subnetwork segmentation task Z, and then fused by the forget gate through convolution operations. The obtained features and the features extracted by the segmentation sub-network encoder in the next stage are used to obtain fused features. ; Define a memory gate This is used to determine the segmentation task Y from the next stage segmentation subnetwork. i Step 1 to the i The steps require memorizing certain features; two attention gates are set up. and They are used to pay attention to the forgetting gate. and memory gate The output information, note the gate. The forget gate is calculated using the Sigmoid function. and memory gate The output weight ratio; the attention value is calculated using the Tanh function. Determines whether the information is positive or negative; Aggregation from the Gate of Oblivion Memory gate And pay attention to the door The control information is used to construct the output features of the segmentation task Y in the next stage segmentation sub-network. The output features are then fed into the decoding end of the next-stage segmentation subnetwork via skip connections.

2. The stroke lesion segmentation method based on progressive fusion networks according to claim 1, characterized in that: The convolutional layers of the U-Net network employ convolutional blocks composed of depth-separable convolutions.

3. The stroke lesion segmentation method based on progressive fusion networks according to claim 1, characterized in that: The adaptive fusion module is used to adaptively aggregate the multi-scale segmentation results at the decoding end as follows: Output features of the previous stage segmentation subnetwork An upsampling operation is performed to obtain the output features of the subsequent segmentation subnetwork. Features of the same size are used; then a 1×1 convolution operation is used to compress the upsampled features into single-channel features, which are then combined through channel concatenation operations to obtain fused features. , ; Fusion features The input is fed into a 4-layer convolutional block with a 3×3 kernel. Convolution is used to extract features from the fused features. Then, global average pooling (GAP) and global max pooling (GMP) are used to extract the global deterministic score predicted at the fused scale. Finally, the score from the fused scale is concatenated into the sigmoid function to obtain the fused attention feature map. , ; Integrate attention feature maps Compared with the output feature map of the subsequent segmentation sub-network After multiplication, the output of the progressive adaptive fusion module between the previous and next stage segmentation subnetworks is obtained, and the output of the progressive adaptive fusion module between the next and previous stage segmentation subnetworks is the final segmentation result.

4. The stroke lesion segmentation method based on progressive fusion networks according to claim 1, characterized in that: At the bottom of each layer of the U-Net network, a non-local attention module is embedded for acquiring location-sensitive contextual information.

5. The stroke lesion segmentation method based on progressive fusion networks according to claim 1, characterized in that, The processing procedure for the non-local attention module is as follows: The input feature map is processed by a convolutional layer with a kernel size of 3×3. The number of channels is determined by Reduce to A new feature map is obtained. ,in ; Three new feature maps are generated using a 1×1 convolution operation. ,and , feature map Transformed into ,in It is the number of pixels in the feature map, then the matrix is ​​processed. With transpose matrix The multiplication operation is used to compute the spatial attention map using the result of the matrix multiplication. ,and ,in It measures the influence of the i-th position on the j-th position; feature map Transformed into Then perform matrix With matrix The multiplication operation, after transpose, transforms the result into Obtain a nonlocal feature attention mechanism map, and then transform the result. With feature map The feature map is obtained by adding elements together. ,and The resulting feature map Z is the sum of relational features and original features; The feature map Z is input into a convolutional layer with a kernel size of 3×3 to generate a sequence of features. Feature maps with the same shape, and with The residual blocks are used for summation to avoid overfitting.

6. The stroke lesion segmentation method based on progressive fusion networks according to claim 1, characterized in that: The process of obtaining three-level pyramid images of different resolutions through a shuffle operation, and using them as input images for each stage, specifically involves: The stroke MRI images to be segmented are input into a top-down pyramid network input module based on shuffle operation. The shuffle operation produces three-level pyramid images of different resolutions. These three-level pyramid images are the original image, half original image, and quarter original image, respectively, and are used as the input images for the first-stage segmentation subnetwork, the second-stage segmentation subnetwork, and the third-stage segmentation subnetwork.

7. The stroke lesion segmentation method based on progressive fusion networks according to claim 1, characterized in that: The input image is iterated through the first-stage segmentation subnetwork, the second-stage segmentation subnetwork, and the third-stage segmentation subnetwork. Each stage produces a corresponding segmentation result. Deep supervision is used to constrain the segmentation result of each stage with a loss function. The loss function of each stage consists of the sum of the binary cross-entropy loss and the Dice loss. The expression for the loss function of each stage is as follows: Represents the loss function at each stage. Indicates Dice loss, , Represents the model's output, Labels indicating separation.

8. A method for stroke lesion segmentation based on progressive fusion networks according to claim 7, characterized in that: The sum of the losses in each stage is the total loss. L total And there are: In this formula, L total This represents the total loss of the input image after being segmented by the first-stage segmentation subnetwork, the second-stage segmentation subnetwork, and the third-stage segmentation subnetwork. L s1 This represents the loss function of the first-stage segmentation subnetwork. L s2 This represents the loss function of the second-stage segmentation subnetwork. L s3 This represents the loss function for the segmentation results of the third-stage segmentation subnetwork. , , These are parameters derived from the different contributions of the segmentation tasks in the three stages to the overall segmentation task.

9. A method for stroke lesion segmentation based on progressive fusion networks according to claim 8, characterized in that: For parameters , , ,have =1:1:2.

Images