Brain tumor image segmentation system, method and device based on context attention mechanism

Abstract

The application discloses a brain tumor image segmentation system based on a context attention mechanism, which comprises a feature extraction subsystem, a feature fusion subsystem and a segmentation head subsystem connected in sequence; the feature extraction subsystem comprises a convolutional basic component, a plurality of series enhanced convolutional components, a batch normalization layer and an activation function layer connected in sequence; feature maps of different levels and different scales are output from the plurality of series enhanced convolutional components; the feature fusion subsystem comprises a plurality of splicing modules, the splicing modules input the feature maps of different levels and different scales, splice and then output; the segmentation head subsystem is used for performing an image segmentation task, and converts the extracted feature maps into pixel-level classification prediction; the cooperative work of the three subsystems enables the model to effectively deal with data heterogeneity in medical images while maintaining high efficiency and accuracy; the subsystems complement each other, fully excavate local and global features of the image, and make the segmentation result more accurate.

Description

TECHNICAL FIELD

[0001] The present application relates to an image segmentation method and device, in particular to a brain tumor image segmentation system, method and device based on a context attention mechanism. BACKGROUND

[0002] At present, early diagnosis and accurate segmentation of brain tumors are crucial for clinical treatment. With the continuous development of medical imaging technology, medical imaging devices such as magnetic resonance imaging (MRI) and computed tomography (CT) have become the main tools for diagnosing brain tumors. However, due to the complexity and heterogeneity of brain tumor images, automated brain tumor segmentation remains a challenging task. Existing deep learning methods have achieved good results in brain tumor image segmentation, but still face some technical problems. First, medical image data usually comes from different devices and institutions, and there is significant heterogeneity, which makes it difficult for the model to generalize to cross-hospital or cross-device datasets. Second, brain tumors have diverse shapes and sizes, and their boundaries with surrounding normal tissues are often blurred, increasing the difficulty of segmentation. In addition, the presence of noise and artifacts in the image and the lack of labeled data further affect the segmentation accuracy and the robustness of the model.

[0003] Due to the complexity and heterogeneity of brain tumor images, the computational complexity is high, and due to the problems of heterogeneous data, noise interference and blurred boundaries of tumor regions, the use of image processing methods in the prior art will result in low segmentation accuracy and model robustness. SUMMARY

[0004] The present application provides a brain tumor image segmentation system, method and device based on a context attention mechanism to solve the technical problems in the prior art.

[0005] The technical solution adopted by the present application to solve the technical problems in the prior art is:

[0006] A brain tumor image segmentation system based on a context attention mechanism, comprising a feature extraction subsystem, a feature fusion subsystem and a segmentation head subsystem connected in sequence; a module composed of a convolution layer, a batch normalization layer and an activation function layer connected in sequence is called a convolution basic component; a module composed of a plurality of convolution basic components connected in series, at least one attention mechanism module and a splicing module connected in series, and at least one attention mechanism module connected in parallel is called an enhanced convolution component;

[0007] The feature extraction subsystem is used to extract multi-level feature information from the input medical image, and comprises a convolution basic component, a plurality of enhanced convolution components connected in series, a batch normalization layer and an activation function layer connected in sequence; different levels and different scales of feature maps are output from the plurality of enhanced convolution components connected in series

[0008] The feature fusion subsystem is used for integrating feature maps from different levels and different scales; it includes a plurality of splicing modules, which input feature maps from different levels and different scales and output after splicing;

[0009] The segmentation head subsystem is used for performing an image segmentation task, which converts the extracted feature maps into a classification prediction at the pixel level; it performs multiple convolution and upsampling processes on the feature information integrated by the feature fusion subsystem, gradually restores the spatial resolution of the image, and finally generates a segmentation mask with the same size as the input image; the mask represents the class of each target in the image, and the target refers to a lesion area.

[0010] Further, the feature extraction subsystem includes enhanced convolution components A1 to A i , enhanced convolution components A i The input image size is 2 i a*2 i a*2 i b, a is 2, 4, 8 or 16, b is 8, 16, 32 or 64, and i≥1.

[0011] Further, in the feature fusion subsystem, each splicing module is connected with an enhanced convolution component.

[0012] Further, the feature fusion subsystem includes splicing modules B1 to B4, enhanced convolution components C1 to C4, convolution base components D1 to D4, and deconvolution modules E1 to E2; wherein the input ends of the splicing modules B1 and B2 are connected with the output ends of the two enhanced convolution components in the feature extraction subsystem one by one; the output end of the splicing module B1 is connected with the input end of the segmentation head subsystem through the enhanced convolution component C1; the output end of the splicing module B2 is connected with the input end of the splicing module B3 through the enhanced convolution component C2 and the convolution base component D2 in turn; the output end of the convolution base component D2 is connected with the input end of the splicing module B1 through the deconvolution module E1; the output end of the enhanced convolution component C1 is connected with the input end of the splicing module B3 through the convolution base component D1; the output end of the splicing module B3 is connected with the input end of the splicing module B4 through the enhanced convolution component C3 and the convolution base component D3 in turn; the splicing module B4 is connected with the input end of the feature extraction subsystem through the convolution base component D4, and its output end is connected with the input end of the segmentation head subsystem through the enhanced convolution component C4; the output end of the convolution base component D4 is connected with the input end of the splicing module B2 through the deconvolution module E2; the output end of the enhanced convolution component C3 is connected with the output end of the enhanced convolution component C4.

[0013] Further, the segmentation head subsystem comprises a Protonet network, a prediction head, a fast NMS module, a crop module and a threshold module; an input end of the Protonet network is connected with an output end of the enhanced convolution component C1; input ends of the prediction head are connected with output ends of the enhanced convolution component C1, the enhanced convolution component C3 and the enhanced convolution component C4 respectively; after an output signal of the prediction head is processed by the fast NMS module, the output signal is tensor multiplied with an output signal of the Protonet network, and then is input to the crop module, and then is sequentially processed by the crop processing of the crop module and the threshold segmentation processing of the threshold module and then is output.

[0014] Further, the Protonet network comprises a plurality of dilated convolution modules and an up-sampling module connected in sequence, and the input size is W / 8*H / 8 and the output size is W*H*T, wherein H represents the height of the image, W represents the width of the image, and T represents the number of prototype masks.

[0015] The application further provides a brain tumor image segmentation method based on the context attention mechanism, wherein the feature extraction subsystem extracts multi-level feature information from the input medical image; the feature fusion subsystem inputs different levels and different scales of features from the feature extraction subsystem, splices and then outputs to the segmentation head subsystem; the segmentation head subsystem performs segmentation prediction on the spliced feature map, generates a segmentation mask with the same size as the input image and then outputs.

[0016] Further, the attention mechanism module is built based on the Transformer architecture, and comprises a cross-attention mechanism module and / or a self-attention mechanism module.

[0017] Further, the feature fusion subsystem calculates from bottom to top, first performs convolution processing on the feature map with the lowest resolution using the convolution base component, and then performs up-sampling through the deconvolution module.

[0018] The application further provides a device of the brain tumor image segmentation method based on the context attention mechanism, comprising a memory and a processor, wherein the memory is used for storing a computer program; the processor is used for executing the computer program and realizing the steps of the brain tumor image segmentation method based on the context attention mechanism as described above when executing the computer program.

[0019] The application has the advantages and positive effects that:

[0020] The application comprises a feature extraction subsystem, a feature fusion subsystem and a segmentation head subsystem connected in sequence; wherein:

[0021] Feature extraction subsystem: The main task of this system is to extract multi-level feature information from the input medical images. Through the convolutional neural network (CNN) layer, the system captures low-level features in the image (such as edges, textures, etc.), as well as high-level features (such as tissue structure, tumor area, etc.).

[0022] Feature fusion subsystem: Mainly used to integrate features from different levels and different scales, so as to better capture the context semantic information and fine-grained boundary details in the image when performing medical image segmentation. When facing brain tumor medical data, the feature fusion subsystem can effectively improve the perception and representation ability of the model at different resolutions through multi-level feature fusion, thereby improving the performance of the model in the medical image segmentation task.

[0023] Segmentation head subsystem: This system is responsible for mapping the feature information obtained from the previous two systems to the final segmentation result. It gradually restores the spatial resolution of the image through a series of convolutional layers and upsampling operations, and finally generates a segmentation mask with the same size as the input image. The mask represents the class of each target in the image, such as classifying into tumor area and normal tissue area, and the target refers to the lesion area. The construction of the segmentation head subsystem can handle high-resolution detail information, ensuring accurate segmentation of the brain tumor area in the image.

[0024] The coordinated work of the three subsystems enables the model to effectively deal with data heterogeneity in medical images while maintaining efficiency and accuracy. Each subsystem complements each other, fully exploiting local and global features of the image, making the segmentation result more accurate, especially when dealing with complex medical images such as brain tumors, it can provide more reliable segmentation results.

[0025] The present invention sets up a convolutional layer with context perception ability, which can flexibly extract local and global semantic information. The present invention is a lightweight brain tumor segmentation method based on context attention mechanism. This method introduces a context semantic attention mechanism, dynamically adjusts the model's attention to local and global information during feature learning, thereby enhancing the model's perception of brain tumor regions. By constructing a lightweight network structure, the method of the present invention can effectively deal with heterogeneous data, noise interference and fuzzy boundary problems of tumor regions while maintaining low computational complexity. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 is a brain tumor image segmentation system structure diagram based on the context attention mechanism of the present invention.

[0027] Figure 2 is a convolutional basic component structure diagram of the present invention.

[0028] Figure 3is a structure diagram of an attention mechanism module of the present application.

[0029] Figure 4 is a structure diagram of an enhanced convolution component of the present application.

[0030] In the figure:

[0031] The arrow direction represents the signal flow direction.

[0032] Key: feature key.

[0033] Query: query.

[0034] Value: value.

[0035] K represents a feature key.

[0036] V represents a feature value.

[0037] X represents an input feature map.

[0038] Y represents an output feature map.

[0039] V: 1x1, indicates that the feature value V applies a 1x1 grouped convolution operation.

[0040] K: kxk, indicates that the feature key K applies a kxk grouped convolution operation.

[0041] θ: 1x1, indicates that the feature value K applies a 1x1 grouped convolution operation.

[0042] δ: 1x1, indicates that the feature value Q applies a 1x1 grouped convolution operation.

[0043] HxWxC, HxWxD, HxW(kxkxC h ), in which:

[0044] H (Height) represents the height of the image, that is, the vertical resolution of the image.

[0045] W (Width) represents the width of the image, that is, the horizontal resolution of the image.

[0046] C (Channels) represents the number of channels of the image.

[0047] D (Depth) represents the feature map depth.

[0048] kxC h , in which k represents the convolution kernel size, and C h represents the number of channels.

[0049] W / 8xH / 8 represents one-eighth of the feature map width W and one-eighth of the feature map height H.

[0050] Tensor product.

[0051] P L Match feature map generated for a feature map of W / 8xH / 8 resolution.

[0052] P M Match feature map generated for a feature map of W / 16xH / 16 resolution.

[0053] P S Match feature map generated for a feature map of W / 32xH / 32 resolution. DETAILED DESCRIPTION

[0054] The present application will be described in detail below with reference to the attached drawings and embodiments, and it should be understood that the preferred embodiments described herein are intended for illustration and explanation only and are not intended to limit the present application.

[0055] In the description of the present application, the terms "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", and the like indicate the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and do not require that the present application must be constructed and operated in a particular orientation, and therefore should not be understood as limiting the present application. The terms "connected", "connected" used in the present application should be understood broadly, for example, it can be fixedly connected or detachably connected; it can be directly connected or indirectly connected through an intermediate component; it can be electrically connected or signal transmission; the specific meaning of the above terms can be understood by those skilled in the art according to the specific circumstances.

[0056] In this application, the Chinese meanings of the following English words, abbreviations and phrases are as follows:

[0057] CNN: Convolutional Neural Network.

[0058] NMS: Non-Maximum Suppression.

[0059] Protonet network: Prototype network.

[0060] Conv layer: Convolutional layer.

[0061] BN layer: Batch Normalization layer.

[0062] Meta AconC: Meta Activation Function.

[0063] Key: Key.

[0064] Queries: Query.

[0065] values: Value.

[0066] Transformer: a deep learning model based on self-attention mechanism.

[0067] YOLACT: a real-time instance segmentation algorithm, whose core idea is to predict the prototype mask of the current picture and the mask coefficient of each instance in parallel, and then generate the instance mask by linearly combining the prototype and the mask coefficient.

[0068] See Figures 1 to 4 A brain tumor image segmentation system based on context attention mechanism, comprising a feature extraction subsystem, a feature fusion subsystem and a segmentation head subsystem connected in turn; a module composed of convolutional layers, batch normalization layers and activation function layers connected in turn is called a convolutional basic component; a module composed of a plurality of convolutional basic components connected in series, at least one attention mechanism module and a splicing module connected in series, and at least one attention mechanism module connected in parallel is called an enhanced convolutional component.

[0069] The feature extraction subsystem is used to extract multi-level feature information from the input medical image, and comprises convolutional basic components connected in turn, a plurality of enhanced convolutional components connected in series, batch normalization layers, activation function layers and a spatial fusion convolutional component; the spatial fusion convolutional component is used to extract multi-scale features.

[0070] The feature fusion subsystem is used to integrate feature maps from different levels and different scales; it comprises a plurality of splicing modules, which input feature maps of different levels and different scales and output after splicing.

[0071] The segmentation head subsystem is used to perform image segmentation tasks, which converts the extracted feature map into a pixel-level classification prediction; it performs multiple convolutions and upsampling processing on the feature information integrated by the feature fusion subsystem, gradually restores the spatial resolution of the image, and finally generates a segmentation mask with the same size as the input image; the mask represents the class of each target in the image, and the target refers to the lesion area.

[0072] Preferably, the feature extraction subsystem can comprise enhanced convolutional components A1 to A i , enhanced convolutional components A i The input image size is 2 i a*2 i a*2 i b, a takes 2, 4, 8 or 16, b takes 8, 16, 32 or 64, i≥1.

[0073] Preferably, in the feature fusion subsystem, an enhanced convolutional component can be connected after each splicing module.

[0074] Preferably, the feature fusion subsystem can comprise a splicing module B1, a splicing module B2, a splicing module B3 to a splicing module B4, an enhanced convolution component C1, an enhanced convolution component C2, an enhanced convolution component C3 to an enhanced convolution component C4, a convolution base component D1, a convolution base component D2, a convolution base component D3 to a convolution base component D4, and an inverse convolution module E1 to an inverse convolution module E2; wherein the input ends of the splicing module B1 and the splicing module B2 are connected in one-to-one correspondence with the output ends of two enhanced convolution components in the feature extraction subsystem; the output end of the splicing module B1 is connected with the input end of the segmentation head subsystem through the enhanced convolution component C1; the output end of the splicing module B2 is connected with the input end of the splicing module B3 through the enhanced convolution component C2 and the convolution base component D2 in sequence; the output end of the convolution base component D2 is connected with the input end of the splicing module B1 through the inverse convolution module E1; the output end of the enhanced convolution component C1 is connected with the input end of the splicing module B3 through the convolution base component D1; the output end of the splicing module B3 is connected with the input end of the splicing module B4 through the enhanced convolution component C3 and the convolution base component D3 in sequence; the splicing module B4 is connected with the input end of the feature extraction subsystem through the convolution base component D4, and is connected with the input end of the segmentation head subsystem through the enhanced convolution component C4; the output end of the convolution base component D4 is connected with the input end of the splicing module B2 through the inverse convolution module E2; and the output end of the enhanced convolution component C3 is connected with the output end of the enhanced convolution component C4.

[0075] Preferably, the segmentation head subsystem can comprise a Protonet network, a prediction head, a fast NMS module, a clipping module and a threshold module; the input end of the Protonet network is connected with the output end of the enhanced convolution component C1; the input end of the prediction head is connected with the output ends of the enhanced convolution component C1, the enhanced convolution component C3 and the enhanced convolution component C4 respectively; the output signal of the prediction head is input to the clipping module after being processed by the fast NMS module and being tensor multiplied with the output signal of the Protonet network, and can be output after being processed by the clipping processing of the clipping module and the threshold segmentation processing of the threshold module in sequence.

[0076] The fast NMS module is a post-processing module in a target detection framework, which is mainly used for deleting highly redundant bounding boxes, and is applied in both a fast regional convolutional neural network and a method of detecting objects in an image using a single deep neural network, and is used on extracted candidate boxes to obtain the final detection box of each class with the highest confidence and no overlap, which is input to the subsequent network, so as to effectively improve the accuracy and speed of the detection network.

[0077] Preferably, the Protonet network can comprise a plurality of dilated convolution modules and up-sampling modules connected in sequence, the input size is W / 8*H / 8, and the output size is W*H*T, wherein H represents the height of the image, W represents the width of the image, and T represents the number of prototype masks.

[0078] The application also provides a brain tumor image segmentation method based on the context attention mechanism, wherein a multi-level feature information is extracted from an input medical image by the feature extraction subsystem; different levels and different scales of features from the feature extraction subsystem are input into the feature fusion subsystem, spliced and then output to the segmentation head subsystem; the segmentation head subsystem performs segmentation prediction on the spliced feature map to generate a segmentation mask with the same size as the input image and then outputs.

[0079] Preferably, the attention mechanism module can be built based on a Transformer architecture, which can comprise a cross-attention mechanism module and / or a self-attention mechanism module.

[0080] The attention mechanism module can encode the input feature key through convolution to obtain a static context representation of the input, connect the encoded feature key with the input query to obtain a dynamic multi-head attention matrix, multiply the attention matrix with the input value to obtain a dynamic context representation of the input, and fuse the static and dynamic context representations as the output.

[0081] Preferably, the feature fusion subsystem can perform calculation from bottom to top, first perform convolution processing on the feature map with the lowest resolution using a convolution base component, and then perform up-sampling through the deconvolution module.

[0082] The application also provides a device for the brain tumor image segmentation method based on the context attention mechanism, comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program and realize the steps of the brain tumor image segmentation method based on the context attention mechanism as described above when executing the computer program.

[0083] The working process and working principle of the application are further described below with reference to a preferred embodiment of the application:

[0084] Brain medical images often contain noise and artifacts, especially in low-quality images or under certain imaging conditions. These noise and artifacts can obscure tumor regions, leading to inaccurate segmentation results. Noise can come from imaging equipment, patient motion, or other environmental factors, while artifacts are usually caused by imaging techniques such as metal artifacts in magnetic resonance images. These factors make the segmentation task of brain tumor images more difficult. The boundary between brain tumors and surrounding normal tissue (such as gray matter, white matter, etc.) is often blurred, especially when the tumor shape is irregular or the tumor overlaps with normal brain tissue. This makes it very difficult to accurately segment the tumor region. In some cases, the tumor may be similar to the surrounding blood vessels or other structures, further increasing the difficulty of segmentation.

[0085] Therefore, the present application provides a brain tumor image segmentation system and method based on context attention mechanism. The method integrates context attention mechanism in the feature learning process, so that the system model can dynamically adjust its focus on local and global information. Therefore, the system model can better understand the long-distance dependency relationship between different regions in the image, thereby improving the perception of context semantic information. This is particularly important for medical image segmentation tasks, as disease manifestations often involve global structures and local details in images.

[0086] A brain tumor image segmentation system based on context attention mechanism structure contains three key systems, each responsible for different tasks, working together to achieve medical image segmentation, and the network structure diagram is as shown in Figure 1 .

[0087] (1) Feature extraction subsystem: The main task of this subsystem is to extract multi-level feature information from the input medical image. Through convolutional neural network layers, this subsystem captures low-level features (such as edges, textures, etc.) and high-level features (such as tissue structure, tumor region, etc.) in the image. In order to better handle the heterogeneity in the image, this subsystem constructs a convolutional layer with context awareness capability, which can flexibly extract local and global semantic information.

[0088] (2) Feature fusion subsystem: mainly used to integrate features from different levels and different scales, so as to better capture the context semantic information and fine-grained boundary details in the image when performing medical image segmentation. When facing brain tumor medical data, the feature fusion subsystem can effectively improve the perception and representation ability of the model at different resolutions through multi-level feature fusion, thereby improving the performance of the model in the medical image segmentation task.

[0089] (3) Segmentation Head Subsystem: This subsystem is responsible for mapping the feature information obtained from the previous two subsystems to the final segmentation result. It gradually restores the spatial resolution of the image through a series of convolutional layers and upsampling operations, and finally generates a segmentation mask with the same size as the input image. The mask represents the class of each target in the image, where the target refers to the lesion area. For example, the tumor area mask and the normal tissue area mask. The construction of the segmentation head subsystem can handle high-resolution detail information and ensure accurate segmentation of brain tumor areas in the image.

[0090] The coordinated work of these three subsystems enables the model to effectively deal with data heterogeneity in medical images while maintaining efficiency and accuracy. Each subsystem complements each other, fully exploiting local and global features of the image, making the segmentation result more accurate, especially when dealing with complex medical images such as brain tumors, it can provide more reliable segmentation results.

[0091] The structure and working principle of the feature extraction subsystem, feature fusion subsystem and segmentation head subsystem are described in detail below:

[0092] A. Feature Extraction Subsystem

[0093] This subsystem is responsible for extracting rich features from raw brain tumor medical data. In this process, a context semantic attention (Context Transformer Attention) mechanism is constructed to help the model better capture context information during feature extraction. To achieve this goal, several key components are included in this subsystem.

[0094] The convolutional base component is an abbreviation of the three English words convolution, batch normalization and activation, which is a basic component of the brain tumor image segmentation system based on the context attention mechanism of the present invention, as shown in Figure 2 The convolutional base component consists of three main parts: convolutional layer (Conv layer), batch normalization layer (BN layer) and activation function (Meta AconC).

[0095] Convolutional layer (Conv layer): The convolutional layer captures local patterns and structures from the input brain tumor medical image through convolution operations. This operation helps to extract low-level features in the image, such as tumor edges, tissue structures, and tumor boundaries with surrounding normal tissue, thereby providing a foundation for subsequent feature learning.

[0096] Batch normalization layer (BN layer): The BN layer normalizes the mean and variance of each input channel, thereby stabilizing the data distribution and preventing problems such as gradient vanishing or gradient explosion. With the introduction of this layer, network training is more stable, accelerating the convergence process.

[0097] Activation function (Meta AconC): The activation function introduces nonlinearity into the network, enabling it to learn more complex mapping relationships. In the Meta AconC function, there is a hyperparameter β that demonstrates its adaptability, which can control the linear or nonlinear characteristics of the activation function. Specifically, when β approaches positive infinity, the function exhibits nonlinearity; when β approaches 0, the function exhibits linear form. The adaptability of the Meta AconC function enables the network to more flexibly choose whether to activate neurons, thereby improving the accuracy of the network, which replaces the traditional activation layer.

[0098] Through the construction of the convolutional base component, the network can better adapt to the diversity in brain tumor medical images and extract rich feature information, providing a solid foundation for subsequent feature fusion and segmentation tasks.

[0099] Contextual semantic attention mechanism module:

[0100] This attention mechanism module uses the contextual information of the input data to guide the learning of the dynamic attention matrix, enhancing the visual representation capability, as shown in Figure 3 The attention mechanism module first encodes the feature map of the input feature key through a 3x3 convolution operation, obtaining the static context representation of the input. Then, through two consecutive 1x1 convolution operations, the encoded feature key is connected with the input query, learning a dynamic multi-head attention matrix. The learned attention matrix is multiplied by the input value to obtain the dynamic context representation of the input. Finally, the module combines the results of the static and dynamic context representations as the output.

[0101] As shown in Figure 3 Let:

[0102] K represents the feature key.

[0103] Q represents the query.

[0104] V represents the feature value.

[0105] X represents the input feature map.

[0106] Y represents the output feature map.

[0107] Wv represents the weight matrix.

[0108] A represents the attention matrix.

[0109] H (Height) represents the height of the image, i.e., the vertical resolution of the image.

[0110] W (Width) represents the width of the image, i.e., the horizontal resolution of the image.

[0111] C (Channels) represents the number of channels of the image.

[0112] D (Depth) represents the depth of the feature map.

[0113] k x k x C h where k represents the size of the convolution kernel, C h represents the number of channels.

[0114] For example, 1 x 1 x 256 represents a convolution kernel size of 1 x 1 and a channel number of 256.

[0115] The specific implementation process is as follows:

[0116] Given a two-dimensional input with dimensions H x W x C, let K = X. Where the input feature map X is used as a feature key K, and local features are extracted through convolution operation. Query Q: Q = K. The input query Q is set to be equal to the feature key K, which interacts with the feature key in the subsequent attention calculation. Feature value V: V = X·Wv. The input data X is multiplied by the weight matrix Wv to obtain the feature value V, which is used for weighting in the dynamic context representation.

[0117] Let K 1 be the static context representation, and the calculation of the static context representation K 1 is as follows:

[0118] The context semantic features are passed to the feature key map (KeyMap) by applying a k x k grouping convolution operation on the feature key. The static context representation K 1 naturally reflects the context information of the local adjacent position. Where K 1 ∈R H×W×C . This static representation reflects the context information of the local region.

[0119] Learning of dynamic attention matrix: K 1 is concatenated with the query Q, and through two consecutive 1 x 1 convolution operations, the attention matrix A is calculated as follows:

[0120] A = [K 1 , Q]·W θ ·W δ Equation (1)

[0121] Where: W θ represents a weight layer with ReLU activation, and W δ represents a weight layer without activation function.

[0122] The local attention matrix of each head of the attention matrix A is based on the query features and the context features, not just the isolated query-key pair. This approach enhances self-attention learning and improves learning effectiveness through the guidance of static context K 1 .

[0123] Let K 2 be the dynamic context representation, which is calculated based on the context attention matrix A from the feature map K 2 , K 2 is calculated from the aggregated feature values V according to the following formula:

[0124]

[0125] where K 2 captures the dynamic feature interactions between input data, hence it is called the dynamic context representation of the input.

[0126] Output fusion Y: Finally, the static context representation K 1 and the dynamic context representation K 2 are fused to obtain the final output Y according to the following formula:

[0127] Y = Fusion(K 1 , K 2 ) Equation (3)

[0128] Fusion(K 1 , K 2 ) represents the fusion of the static context representation K 1 and the dynamic context representation K 2 .

[0129] In this way, the attention mechanism module can effectively combine static and dynamic context information, enhancing the model's context awareness ability when processing medical images, especially for capturing complex structures and detailed information in medical images such as brain tumors, improving the model's segmentation accuracy and robustness.

[0130] (3) Enhanced convolution component:

[0131] This is a key module in the feature extraction network, responsible for learning the basic features and residual features of the detected object, such as Figure 4The structure is composed of two systems: one system uses multiple stacked convolutional basic components with attention mechanism modules, and the other system only uses one attention mechanism module. Finally, the outputs of the two systems are spliced together. The enhanced convolutional component extracts features from the input image through multiple convolution operations, capturing features of different scales and semantics. Replacing the traditional residual module with the enhanced convolutional component effectively reduces the complexity of the model, achieving model compression and acceleration. At the same time, it ensures lightweight construction while obtaining more abundant gradient flow information.

[0132] B, feature fusion subsystem:

[0133] In the feature fusion subsystem, three different levels of feature representations are constructed, namely W / 8xH / 8, W / 16xH / 16, and W / 32xH / 32. By fusing features of different resolutions, this method can capture both contextual semantic information and fine boundary details. The feature fusion subsystem is calculated from bottom to top. First, the lowest resolution feature map is convolved, and then processed using the convolutional basic component, and then up-sampled through the deconvolution module. It is worth noting that, unlike traditional algorithms that use nearest neighbor interpolation, this study uses transposed convolution for up-sampling. This choice is to meet the high-precision requirement of medical segmentation tasks for fine boundary segmentation. Transposed convolution helps to restore finer image details or generate finer feature maps, which has a significant advantage in segmentation tasks. In addition, it also allows the model to automatically learn these parameters during backpropagation, which means that the model can dynamically adjust the up-sampling process according to the task requirements, thereby improving network performance. It is particularly important to note that proper initialization of transposed convolution parameters is crucial, and incorrect initialization may lead to unstable training or convergence to a local optimal solution during optimization.

[0134] In this subsystem, the enhanced convolutional component is introduced again, further enhancing the model's perception of the importance or relevance between different features. This method allows the model to flexibly focus on information that is more critical or relevant to the current task when fusing features. Different tasks may require attention to different features, and this method can flexibly switch between tasks, thereby improving the model's performance and generalization ability. After feature fusion, three feature maps of different resolutions are generated, denoted as P L , P M , and P S . P L is the matching feature map generated for the W / 8xH / 8 resolution feature map; P M is the matching feature map generated for the W / 16xH / 16 resolution feature map; and P S is the matching feature map generated for the W / 32xH / 32 resolution feature map.

[0135] C. Segmentation head subsystem:

[0136] The final system focuses on generating the final segmentation result. The present application builds a brain tumor image segmentation head based on YOLACT. After feature fusion, the largest scale feature map P l is input into the Protonet network. Unlike YOLACT, Protonet is composed of multiple dilated convolutions and up-sampling modules, with an input size of W / 8 x H / 8 and an output size of W x H x the number of prototype masks. Dilated convolution is used to expand the receptive field without sacrificing resolution, thereby capturing multi-scale contextual information. In addition, P L , P M and P S are input into the prediction head. If the dimension of P L is W / 8 x H / 8 x 128, the generated feature map will be used to create anchors. Each pixel generates 3 anchors, and the number of anchors is A L =W / 8 x H / 8 x 3. After passing through the prediction head, three types of outputs are produced: class confidence, dimension A L x class; position offset, dimension A L x 4; and mask coefficients, dimension A L x 32. After the three scale feature maps pass through the prediction head, the results are spliced together, and the final number of anchors is represented as A O =A L +A M +A S . Among them: A L is the anchor created for the feature map corresponding to the W / 8 x H / 8 resolution, A M is the anchor created for the feature map corresponding to the W / 16 x H / 16 resolution, A S is the anchor created for the feature map corresponding to the W / 32 x H / 32 resolution, and A O is the final number of anchors after splicing different resolution feature maps.

[0137] After obtaining the position offset, the position of the region of interest (RoI) can be determined by adding the position offset to the position of the anchor. Then, the redundant regions of interest are removed through the fast NMS module (fast non-maximum suppression). Finally, the mask coefficients are multiplied by the prototype mask to generate each individual mask in the image. The mask that exceeds the boundary is set to zero by cropping, and the generated mask is binarized by thresholding. The segmentation head not only generates pixel-level segmentation results, but also guides the feature fusion subsystem to learn the relationship between instances more effectively. By establishing a connection between instances, the network can more accurately understand different structures and lesions in medical images, improving the accuracy of fine-grained segmentation.

[0138] The above-mentioned feature extraction subsystem, feature fusion subsystem, segmentation head subsystem, convolution layer, batch normalization layer, activation function layer, convolution base component, attention mechanism module, enhanced convolution component, functional modules or systems including Protonet network, prediction head, fast NMS module, cropping module, threshold module, dilated convolution module, up-sampling module, deconvolution module, and spatial fusion convolution component, etc. can adopt the applicable functional modules or systems in the prior art, or adopt the functional modules, systems and software in the prior art and adopt conventional technical means to construct.

[0139] The above-mentioned embodiments are only used to illustrate the technical ideas and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the content of the present application and to implement it, and cannot be limited to the inventive scope of the present application only by the present embodiment, that is, any equivalent changes or modifications made in the spirit disclosed by the present application still fall within the inventive scope of the present application.

Claims

1. A brain tumor image segmentation system based on context attention mechanism, characterized in that, The system comprises a feature extraction subsystem, a feature fusion subsystem and a segmentation head subsystem connected in sequence; a module composed of a convolution layer, a batch normalization layer and an activation function layer connected in sequence is referred to as a convolution basic component; a module composed of a plurality of convolution basic components connected in series, at least one attention mechanism module and a splicing module connected in series, and at least one attention mechanism module connected in parallel is referred to as an enhanced convolution component; The feature extraction subsystem is used for extracting multi-level feature information from an input medical image, and comprises a convolution basic component, a plurality of enhanced convolution components connected in series, a batch normalization layer and an activation function layer connected in sequence; different levels and different scales of feature maps are output from the plurality of enhanced convolution components connected in series The feature fusion subsystem is used for integrating feature information from different levels and different scales; the feature fusion subsystem comprises a plurality of splicing modules, which input different levels and different scales of feature maps, splice the feature maps and output spliced feature maps; The segmentation head subsystem is used for performing an image segmentation task, and converts the extracted feature maps into pixel-level classification predictions; The segmentation head subsystem performs multiple convolution and up-sampling processes on the feature information integrated by the feature fusion subsystem, gradually restores the spatial resolution of the image, and finally generates a segmentation mask with the same size as the input image; the mask represents the category of each target in the image, and the target refers to a lesion region; In the feature fusion subsystem, an enhanced convolution component is connected after each splicing module. The feature fusion subsystem comprises splicing modules B1 to B4, enhanced convolution components C1 to C4, convolution basic components D1 to D4 and deconvolution modules E1 to E2. The input ends of the splicing modules B1 and B2 are connected to the output ends of the two enhanced convolution components in the feature extraction subsystem in a one-to-one correspondence; the output end of the splicing module B1 is connected to the input end of the segmentation head subsystem through the enhanced convolution component C1; the output end of the splicing module B2 is connected to the input end of the splicing module B3 in sequence through the enhanced convolution component C2 and the convolution basic component D2; the output end of the convolution basic component D2 is connected to the input end of the splicing module B1 through the deconvolution module E1; the output end of the enhanced convolution component C1 is connected to the input end of the splicing module B3 through the convolution basic component D1; the output end of the splicing module B3 is connected to the input end of the splicing module B4 in sequence through the enhanced convolution component C3 and the convolution basic component D3; the input end of the splicing module B4 is connected to the final output end of the feature extraction subsystem through the convolution basic component D4, and the output end of the splicing module B4 is connected to the input end of the segmentation head subsystem through the enhanced convolution component C4; the output end of the convolution basic component D4 is connected to the input end of the splicing module B2 through the deconvolution module E2; the output end of the enhanced convolution component C3 is connected to the output end of the enhanced convolution component C4.

2. The brain tumor image segmentation system based on context attention mechanism according to claim 1, wherein, The feature extraction subsystem comprises an enhanced convolution component A1 to an enhanced convolution component A i , the enhanced convolution component A i The input image size is , a takes 2, 4, 8 or 16, b takes 8, 16, 32 or 64, i ≥ 1. 3.The brain tumor image segmentation system based on context attention mechanism according to claim 1, wherein, The segmentation head subsystem comprises a Protonet network, a prediction head, a fast NMS module, a cropping module and a threshold module; an input end of the Protonet network is connected with an output end of the enhanced convolution component C1; input ends of the prediction head are respectively connected with output ends of the enhanced convolution component C1, the enhanced convolution component C3 and the enhanced convolution component C4; after output signals of the prediction head are processed by the fast NMS module, the output signals are tensor multiplied with output signals of the Protonet network, and then input to the cropping module, and then output after being processed by cropping of the cropping module and threshold segmentation processing of the threshold module in sequence. 4.The brain tumor image segmentation system based on context attention mechanism according to claim 1, wherein, The Protonet network comprises a plurality of dilated convolution modules and up-sampling modules connected in sequence, and the input size is , and the output size is , wherein H represents the height of the image, W represents the width of the image, and T represents the number of prototype masks.

5. A context attention mechanism-based brain tumor image segmentation method using the context attention mechanism-based brain tumor image segmentation system according to any one of claims 1 to 4, characterized by, The feature extraction subsystem extracts multi-level feature information from the input medical image; The feature fusion subsystem inputs features of different levels and different scales from the feature extraction subsystem, splices and outputs to the segmentation head subsystem; the segmentation head subsystem performs segmentation prediction on the spliced feature map, generates a segmentation mask with the same size as the input image and then outputs.

6. The method of claim 5, wherein the method further comprises: The attention mechanism module is built based on a Transformer architecture, and comprises a cross-attention mechanism module and / or a self-attention mechanism module.

7. The method of claim 5, wherein the method further comprises: The feature fusion subsystem calculates from bottom to top, first performs convolution processing on the feature map with the lowest resolution using the convolution base component, and then performs up-sampling through the deconvolution module.

8. A device of a brain tumor image segmentation method based on a context attention mechanism, comprising a memory and a processor, characterized in that, The memory is configured to store a computer program; and the processor is configured to execute the computer program and implement the brain tumor image segmentation method based on the context attention mechanism according to any one of claims 5 to 7 when executing the computer program.

Images