Colorectum and colorectal cancer segmentation system based on multi-view attention scale mixing

Abstract

The invention discloses a colorectal and colorectal cancer segmentation system based on multi-view attention scale mixing. The colorectal and colorectal cancer segmentation system comprises N continuous down-sampling blocks and up-sampling blocks with a jump connection mechanism, and a scale mixing module, a multi-view attention layer is arranged in front of each down-sampling block and is used for calculating attention scores obtained from wide, deep and high views and carrying out weighted summation calculation and channel dimension processing to obtain coding feature output; and the scale mixing module unifies the scale of all the decoding features and combines the decoding features to obtain a mixed feature map, and processes the mixed feature map to obtain pixel-level segmentation output. According to the method, in the encoding stage, the height, width and depth dimensions are considered to highlight semantic related information and position the focus position, secondary refining is carried out on the feature maps from the channel dimension to suppress irrelevant noise, decoding feature maps of all scales are connected in the decoding stage, then mixed feature extraction is carried out, and more accurate colorectal cancer segmentation is achieved.

Description

Technical Field

[0001] This invention belongs to the technical field of machine learning and artificial intelligence, specifically relating to a colorectal and colorectal cancer segmentation system based on multi-view attention scale hybridization. Background Technology

[0002] Colorectal cancer (CRC) is the third most common and second deadliest cancer worldwide. Medical imaging provides location and morphological information about the colorectum and CRC, which can be used for colorectal structural assessment, CRC detection, diagnosis, staging, and treatment response evaluation. In clinical practice, radiologists need to examine medical images layer by layer, identify and manually mark CRC tumor areas, a process that is extremely time-consuming, labor-intensive, and has low reproducibility. Therefore, automated segmentation of CRC is particularly important.

[0003] Accurate detection and segmentation of red blood cells (CRC) can provide strong support for subsequent treatment planning and clinical prognosis. However, CRC presents significant challenges for accurate segmentation due to its low contrast, blurred tumor boundaries, and highly variable location in conventional enhanced CT images. Traditional image processing techniques, such as thresholding, region growing, and edge detection, rely on manually designed features and prior knowledge to distinguish tumors from normal tissue, making it difficult to achieve satisfactory performance in CRC segmentation. With the rapid development of machine learning, especially the rise of end-to-end automated technologies like deep learning, which can learn complex patterns of targets in large amounts of images, it has been widely applied to medical image segmentation tasks.

[0004] UNet, based on a fully convolutional neural network, is a groundbreaking model that achieves efficient image segmentation by employing a symmetric encoder and decoder. Its unique network design and training strategy enable UNet to achieve high segmentation accuracy even with a small number of training images, significantly improving the efficiency and accuracy of biomedical image analysis. Many subsequent studies on image semantic segmentation have also revolved around UNet. A representative work is UNet++, which utilizes multi-scale features for image segmentation by redesigning skip connections. UNet++ improves segmentation performance by embedding U-Net structures of varying depths, allowing the decoder to process features from different scales in the encoder. Furthermore, through deep supervision and model pruning strategies, UNet++ accelerates inference speed without sacrificing too much performance.

[0005] Attention mechanisms, originally designed to mimic human attention and encourage deep learning models to focus on specific regions or targets, are now widely used in medical image segmentation tasks. A representative example is Attention UNet, which introduces an attention gate and uses a self-attention mechanism to generate an attention score map between the encoder and decoder, helping the model focus on important features. Furthermore, SE-Block utilizes channel attention to dynamically adjust the weights of channels in the feature map, enhancing feature representation capabilities. Spatial attention mechanisms can combine information from all channels to perform segmentation target localization in the feature space. However, most of these methods only use a single form of attention mechanism; further development is needed to utilize multi-dimensional and multi-view attention mechanisms to enhance feature representation capabilities.

[0006] UNet fails to fully extract valuable information from features at different scales. A common solution is to use a deep supervision strategy, which generates independent segmentation results for each layer, allowing gradients to propagate between different layers. This effectively utilizes multi-scale information and alleviates the gradient vanishing problem. However, this also requires higher computational and storage requirements. Zhang et al. proposed a staged deep supervision method that integrates feature information from two different stages to improve features, avoids generating multiple segmentation results, and saves computational resources. Dou et al. proposed a staged attention refinement method that uses an attention mechanism to improve decoded features and directly fuses them for cortical plate segmentation. However, this direct fusion approach may lead to problems such as loss of detailed information due to scale inconsistencies. Summary of the Invention

[0007] The main objective of this invention is to overcome the shortcomings and deficiencies of the prior art and provide a colorectal cancer segmentation system based on multi-view attention scale hybridization, which enables high-precision automatic detection and segmentation of CRC, facilitating subsequent clinical applications such as computer-aided diagnosis and treatment, computer-aided surgery, and radiotherapy, thereby reducing the burden on clinicians to manually mark lesion areas and make layer-by-layer judgments and segmentations.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: In one aspect, the present invention provides a colorectal and colorectal cancer segmentation system based on multi-view attention scale mixing, including an encoder and a decoder, wherein the encoder includes N consecutive downsampling blocks, the decoder includes N consecutive upsampling blocks and a scale mixing module, and a multi-view attention layer is provided before each downsampling block. The multi-view attention layer includes an attention block, a weighted summation module, and an SE module. The attention block is used to calculate the attention score of the original feature map from the width, depth, and height views. The weighted summation module is used to perform a weighted summation calculation on the attention score and the original feature map. The SE module is used to process the channel dimension of the output of the weighted summation module to obtain the encoded feature output. The upsampling block is used to restore and decode the coded feature output to obtain the decoded feature; A skip connection mechanism is provided between the upsampling block and the corresponding downsampling block; The scale fusion module includes a scale-aware activation module and a scale attention module. The scale-aware activation module is used to unify the scale of all decoded features and merge them according to interpolation upsampling operations and convolution to obtain a fusion feature map. The scale attention module is used to process the fusion feature map according to channel attention mechanism and spatial attention mechanism to obtain pixel-level segmentation output.

[0009] As a preferred technical solution, the attention block calculates the attention score of the original feature map from the perspectives of width, depth, and height, specifically as follows: ; in, F i For the first i Input features of each encoding stage f i The original feature map extracted by convolution. Conv ⨁ represents a 1×1 convolution operation, and ⨁ represents an element-wise addition operation. σ This represents the Sigmoid activation function. Pavg and Pmax These represent average pooling and max pooling operations, respectively. A i d This represents the attention score obtained from the depth perspective. A i w This represents the attention score obtained from the wide viewpoint. A i h The attention score is represented by the height of the viewpoint. C Indicates the depth of the feature map. W Indicates the width of the feature map. H This indicates the height of the feature map.

[0010] As a preferred technical solution, the weighted summation module performs a weighted summation calculation on the attention score and the original feature map, specifically as follows: ; in, Conv ⨂ represents a 1×1 convolution operation, ⨂ represents element-wise multiplication, and © represents a concatenation operation along the channel dimension. A i d This represents the attention score obtained from the depth perspective. A i w This represents the attention score obtained from the wide viewpoint. A i h The attention score is represented by the height of the viewpoint. f i For the first i Input features of each coding stage F i The original feature map obtained by convolution extraction. M i This is the output of the weighted summation module.

[0011] As a preferred technical solution, the SE module performs channel-dimensional processing on the output of the weighted summation module to obtain the encoded feature output. T i Specifically: ; in, Conv ⨁ represents a 1×1 convolution operation, and ⨁ represents an element-wise addition operation. σ This represents the Sigmoid activation function. Pavg c and Pmax c They represent the average pooling operator and the max pooling operator, respectively. M i This is the output of the weighted summation module.

[0012] As a preferred technical solution, the scale-aware activation module unifies the scale of all decoded features and merges them according to the interpolation upsampling operation and convolution to obtain a hybrid feature map, specifically: ; in, Conv This represents a 1×1 convolution operation. N The number of upsampling blocks. n ∈(0,1,2,…, N -1) represents the sequence number of the upsampling block and the feature being processed. Up ×2^(N-1-n) The symbol represents an interpolation upsampling operation, and © represents a stitching operation along the channel dimension. D 0、 D 1, ... D N-1 The decoding features are the input.D’ This is the output mixed feature map.

[0013] As a preferred technical solution, the channel attention mechanism in the scale attention module is specifically as follows: ; Where ⨁ represents element-level addition. σ This represents the Sigmoid activation function. Pavg c and Pmax c They represent the average pooling operator and the max pooling operator, respectively. M s Represents shared MLP, D scale The scale attention result output by the channel attention mechanism. D’ This is a mixed feature map.

[0014] As a preferred technical solution, the spatial attention mechanism in the scale attention module is specifically as follows: ; in, Pavg s and Pmax s These represent the average pooling operator and the max pooling operator in the spatial attention mechanism, respectively. D scale The scale attention result output by the channel attention mechanism; M Represents MLP; σ This represents the Sigmoid activation function. D’ For the mixed feature map, ⨂ represents element-level multiplication, and © represents concatenation along the channel dimension; Output of spatial attention mechanism D mix Perform the following operations to obtain the final pixel-level segmentation output. Pred : ; in, SoftMax for SoftMax function.

[0015] As a preferred technical solution, the loss used during training... L seg Specifically: ; ; L seg = L CE +L DC ; in, L CE For cross-entropy loss, L DC For Dice's loss, N For the data sample size, I {•} is an MLP network. h , w , d For input data x Size, P s t,n For the first t Individual elements in the first n Output the segmentation probability of the class.

[0016] Another aspect of the present invention provides a method for segmenting colorectal and rectal cancer based on multi-view attention scale mixing, which, using the above-mentioned colorectal and rectal cancer segmentation system based on multi-view attention scale mixing, includes the following steps: The image to be segmented is resampled, randomly cropped, and labeled. Encoding stage: The processed image to be segmented is input into the encoder. Attention scores from the width, depth and height perspectives of the original feature map are calculated through a multi-view attention layer. These scores are then weighted and fused with the original feature map. Finally, channel dimension processing is performed. Feature extraction is performed through downsampling blocks. The feature output of each downsampling block is passed to the corresponding layer in the decoding stage through a skip connection mechanism. Decoding stage: Upsampling blocks are used to reconstruct and decode the feature maps from the feature output of the encoding stage; a scale-aware activation module is used to unify the scale of all decoded features and merge them to obtain a hybrid feature map; a scale attention module is used to process the hybrid feature map according to the channel attention mechanism and the spatial attention mechanism to obtain pixel-level segmentation output.

[0017] In another aspect, the present invention also provides a storage medium storing a program that, when executed by a processor, implements the above-described method for colorectal and colorectal cancer segmentation based on multi-view attention scale hybridization.

[0018] Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) In the encoding stage, the present invention uses a multi-view attention module to reshape the feature map, highlighting semantically relevant information by considering the height, width and depth dimensions, and locating the lesion location. Then, the SE module refines the feature map from the channel dimension to suppress irrelevant background noise, thereby achieving accurate segmentation of colorectal and colorectal cancer.

[0019] (2) In the decoding stage, the present invention constructs a scale attention module, which integrates the decoding feature maps of various scales, thereby enhancing the model’s ability to identify small lesions and achieving more accurate colorectal cancer segmentation.

[0020] (3) The implementation of the present invention can help clinicians to more clearly diagnose colorectal abnormalities and detect colorectal cancer by accurately identifying and delineating the colorectal and colorectal cancer areas, and provide support for subsequent treatment plan design and treatment response monitoring, thereby promoting the development of personalized and precise clinical diagnosis and treatment. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the structure of the colorectal and colorectal cancer segmentation system based on multi-view attention scale hybridization according to an embodiment of the present invention; Figure 2 This is a flowchart of a colorectal and colorectal cancer segmentation method based on multi-view attention scale hybridization according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the storage medium according to an embodiment of the present invention. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0023] Example 1: like Figure 1 As shown, this embodiment provides a colorectal and colorectal cancer segmentation system based on multi-view attention scale mixing. The system includes an encoder and a decoder. The encoder includes N consecutive downsampling blocks, and the decoder includes N consecutive upsampling blocks and a scale mixing module. Each downsampling block is preceded by a multi-view attention layer (multi-view attention module). A skip connection mechanism is provided between the upsampling blocks and the corresponding downsampling blocks.

[0024] (1) Multi-view attention layer: Due to the varied structure and morphology of the colorectal region and the significant differences between individuals, traditional segmentation networks struggle to accurately segment the entire colorectal region. This embodiment employs a multi-view attention mechanism to rescale the original feature map from three perspectives: height, width, and depth. Combined with the channel processing mechanism of the SE module, semantically relevant features are fully highlighted, while the expression of irrelevant features is suppressed, thereby enhancing the model's encoding capability.

[0025] The multi-view attention layer includes an attention block (attention mechanism), a weighted summation module, and an SE (Squeeze-and-Excitation) module. The attention block is used to calculate the attention score of the original feature map from the width, depth, and height views. The weighted summation module is used to perform a weighted summation calculation on the attention score and the original feature map. The SE module is used to process the channel dimension of the output of the weighted summation module to obtain the encoded feature output.

[0026] like Figure 1 As shown, the three branches from top to bottom represent the attention mechanism's operations from the width, depth, and height perspectives, respectively. Operating the attention mechanism from these three perspectives allows for an intuitive representation of the importance of the region of interest in the feature map using a one-dimensional weight axis. Through the coordinated operation of these three different dimensional axes, the model can accurately locate CRC lesions of different shapes, sizes, and distributions in high-dimensional space, helping the model achieve more comprehensive and accurate segmentation.

[0027] After obtaining attention scores from three different perspectives, the output of the multi-view attention mechanism can be calculated by weighted summation of the scores with the original feature maps. The final output is then obtained by processing the channels through the SE module.

[0028] In one or more preferred embodiments, the attention block calculates attention scores for the original feature map from the perspectives of width, depth, and height, specifically: ; in, F i For the first i Input features of each encoding stage f i The original feature map extracted by convolution. Conv ⨁ represents a 1×1 convolution operation, and ⨁ represents an element-wise addition operation. σ This represents the Sigmoid activation function. Pavg and Pmax These represent average pooling and max pooling operations, respectively. A i d This represents the attention score obtained from the depth perspective. A i w This represents the attention score obtained from the wide viewpoint. A i h The attention score is represented by the height of the viewpoint. C Indicates the depth of the feature map. W Indicates the width of the feature map. H This indicates the height of the feature map.

[0029] In one or more preferred embodiments, the weighted summation module performs a weighted summation calculation on the attention score and the original feature map, specifically as follows: ; in, Conv ⨂ represents a 1×1 convolution operation, ⨂ represents element-wise multiplication, and © represents a concatenation operation along the channel dimension. A i d This represents the attention score obtained from the depth perspective. A i w This represents the attention score obtained from the wide viewpoint. A i h The attention score is represented by the height of the viewpoint. f i For the first i Input features of each coding stage F i The original feature map obtained by convolution extraction. M i This is the output of the weighted summation module.

[0030] In one or more preferred embodiments, the SE module performs channel-dimensional processing on the output of the weighted summation module to obtain the encoded feature output. T i Specifically: ; in, Conv ⨁ represents a 1×1 convolution operation, and ⨁ represents an element-wise addition operation. σ This represents the Sigmoid activation function. Pavg c and Pmax c They represent the average pooling operator and the max pooling operator, respectively. M i This is the output of the weighted summation module.

[0031] (2) Downsampling block: The downsampling block is used to receive the output of the multi-view attention layer and perform feature extraction. In addition, the feature output of each downsampling block is passed to the upsampling block of the corresponding layer of the decoder through a skip connection mechanism.

[0032] In one or more preferred embodiments, the number of downsampling blocks is set to 5.

[0033] (3) Upsampling block: The downsampling block is used to receive the feature output from the downsampling block of the corresponding layer of the encoder, perform feature map reconstruction and decoding, and obtain the decoded features.

[0034] In one or more preferred embodiments, the number of upsampling blocks is set to 5.

[0035] (4) Scale Blending Module: The scale blending module includes a scale-aware activation module and a scale attention module; the scale-aware activation module is used to unify the scale of all decoded features and merge them according to the interpolation upsampling operation and convolution to obtain a blended feature map; the scale attention module is used to process the blended feature map according to the channel attention mechanism and the spatial attention mechanism to obtain pixel-level segmentation output.

[0036] (4.1) Scale-Aware Activation Module: Traditional U-shaped networks, as a deep structure, suffer from gradient vanishing when using only the decoding results of the last layer for segmentation prediction and loss calculation. It is reasonable to combine all feature information generated by decoding for the final segmentation prediction. In order to selectively extract and merge useful information for decoding features at different scales to achieve more accurate segmentation prediction, this embodiment introduces a scale-aware activation module to help the model better segment CRC. The scale-aware activation module receives all decoding features as input, first uses interpolation upsampling operations and convolution to unify them to the final scale size, and then merges them to obtain a hybrid feature map.

[0037] In one or more preferred embodiments, the scale-aware activation module unifies the scale of all decoded features and merges them according to the interpolation upsampling operation and convolution to obtain a hybrid feature map, specifically: ; in, Conv This represents a 1×1 convolution operation. N The number of upsampling blocks. n ∈(0,1,2,…, N -1) represents the sequence number of the upsampling block and the feature being processed. Up ×2^(N-1-n) The symbol represents an interpolation upsampling operation, and © represents a stitching operation along the channel dimension. D 0、 D 1, ... D N-1 The decoding features are the input. D’ The resulting blended feature map is... D’ and D The size is the same as 0, but the number of channels is its N times.

[0038] (4.2) Scale attention module: used for processing mixed feature maps D’The process involves adaptively extracting and activating semantically relevant information at each scale to refine the decoded features. First, based on a channel attention mechanism, the scale attention results are filtered and refined at each scale. Then, further processing is performed at the spatial level to obtain a global spatial information representation and calculate the spatial confidence score to compute the final result of the scale attention module.

[0039] In one or more preferred embodiments, the channel attention mechanism in the scale attention module specifically comprises: ; Where ⨁ represents element-level addition. σ This represents the Sigmoid activation function. Pavg c and Pmax c They represent the average pooling operator and the max pooling operator, respectively. M s Represents shared MLP, D scale The scale attention result output by the channel attention mechanism. D’ This is a mixed feature map.

[0040] In one or more preferred embodiments, the spatial attention mechanism in the scale attention module specifically comprises: ; in, Pavg s and Pmax s These represent the average pooling operator and the max pooling operator in the spatial attention mechanism, respectively. D scale The scale attention result output by the channel attention mechanism; M Represents MLP; σ This represents the Sigmoid activation function. D’ For the mixed feature map, ⨂ represents element-level multiplication, and © represents concatenation along the channel dimension; Output of spatial attention mechanism D mix Perform the following operations to obtain the final pixel-level segmentation output. Pred : ; in, SoftMax for SoftMax function.

[0041] (5) Training and testing of colorectal and colorectal cancer segmentation system based on multi-view attention scale hybridization.

[0042] (5.1) Training.

[0043] Colorectal and Rectal Cancer Segmentation System Based on Multi-View Attention Scale Hybridization f seg The loss function used during training L seg It is cross-entropy loss L CE and Dice loss L DC Summary: ; ; ; in, N For the data sample size, I {•} is an MLP network. h , w , d For input data x Size, P s t,n For the first t Individual elements in the first n Output the segmentation probability of the class.

[0044] (5.2) Test.

[0045] Input data in a colorectal and colorectal cancer segmentation system based on multi-view attention scale hybridization f seg The prediction results are obtained through calculation: ; During the testing phase, the trained network, based on a multi-view attention-scale hybrid colorectal and colorectal cancer segmentation system, only requires CT images. X As input, segmentation prediction can be obtained. P pred As the final output.

[0046] It should be noted that the system provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure can be divided into different functional modules to complete all or part of the functions described above. The system can apply the colorectal and colorectal cancer segmentation method based on multi-view attention scale hybridization in the following embodiments.

[0047] Example 2: like Figure 2As shown, this embodiment provides a method for colorectal and colorectal cancer segmentation based on multi-view attention scale hybridization. By instantiating the scheme in Embodiment 1, the calculation process of the present invention is refined so that those skilled in the art can more easily understand the scheme of this application.

[0048] S1. Input the original CT image X Resample to a space of a preset size, then randomly crop it into patches of a specified size. x And by labeling, the corresponding pixel-level labels are obtained. y And coordinate graph e They are two of the same size X The same matrix.

[0049] In one or more preferred embodiments, the preset size in the resampling step is 0.75mm, 0.75mm, or 3mm.

[0050] In one or more preferred embodiments, the specified size in the random cropping step is 192×192×48 voxels.

[0051] In one or more preferred embodiments, pixel-level labeling y and coordinate graph e The matrix size is the same as the original input CT image. X Consistent.

[0052] In one or more preferred embodiments, pixel-level labeling y The value range is {0, 1, 2}, where different values ​​represent the semantic class to which the corresponding pixel belongs: 0 represents background, 1 represents colorectal cancer, and 2 represents colorectal cancer.

[0053] S2, patch x The input is fed into the encoder, where a one-dimensional attention map is captured from three angles: height, depth, and width, through a multi-view attention layer. H ×1, D ×1, W ×1), which is the attention score; then it is weighted and fused with the original feature map, and finally processed by the SE module in the channel dimension, the steps are the same as above, which is to capture a one-dimensional attention map ( C ×1) is then weighted with the input feature map.

[0054] S3. Feature extraction is performed using downsampling blocks; S4. After the downsampling feature extraction is completed, it is passed to the corresponding layer upsampling block of the decoder through a skip connection mechanism to restore and decode the feature map, thus obtaining the decoded features.

[0055] S5. For all decoded features, the scale-aware activation module is used to upsample them to the size of the last one (i.e. the size of the original input) and then concatenate them to obtain a hybrid feature map.

[0056] S6. The scale attention module is used to process the hybrid feature map according to the channel attention mechanism and the spatial attention mechanism to obtain the pixel-level segmentation output.

[0057] In one or more preferred embodiments, segmentation loss is employed. L seg = L CE + L DC Backpropagation is used to update network weights; an SGD optimizer with an initial learning rate of 0.01 is used; polynomial decay is used as the learning rate decay strategy; the batch size during training is set to 2, and the training is iterated 250,000 times.

[0058] Example 3: like Figure 3 As shown, this embodiment provides a storage medium storing a program. When the program is executed by a processor, it implements a colorectal and colorectal cancer segmentation method based on multi-view attention scale hybridization as described in Embodiment 2.

[0059] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0060] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A multi-view attention scale mixture based colorectal and colorectal cancer segmentation system, characterized in that, The system comprises an encoder and a decoder, the encoder comprises N continuous downsampling blocks, the decoder comprises N continuous upsampling blocks, a scale mixing module, and a multi-view attention layer is arranged before each downsampling block; The multi-view attention layer comprises an attention block, a weighted sum module, and an SE module; the attention block is used for calculating attention scores of an original feature map from width, depth, and height perspectives; the weighted sum module is used for weighted sum calculation of the attention scores and the original feature map; and the SE module is used for channel dimension processing of an output of the weighted sum module to obtain an encoded feature output; The upsampling block is used for restoring and decoding the encoded feature output to obtain a decoded feature; The upsampling block and the corresponding downsampling block are provided with a skip connection mechanism; The scale mixing module comprises a scale perception activation module and a scale attention module; The scale perception activation module is used for unifying scales of all decoded features and merging the decoded features to obtain a mixed feature map according to an interpolation upsampling operation and convolution; and the scale attention module is used for processing the mixed feature map according to a channel attention mechanism and a spatial attention mechanism to obtain a pixel-level segmentation output.

2. The multi-view attention scale mixture based colorectal and colorectal cancer segmentation system according to claim 1, wherein, The attention block calculates attention scores of an original feature map from width, depth, and height perspectives, and the calculation is specifically as follows: ； where, F i is the input feature of the i th encoding stage, f i is the original feature map extracted by convolution, Conv denotes a 1 × 1 convolution operation, and σ denotes a Sigmoid activation function, Pavg and Pmax denote average pooling and max pooling operations, respectively, A i d denotes the attention score obtained from the depth perspective, A i w denotes the attention score obtained from the width perspective, A i h denotes the attention score obtained from the height perspective, C denotes the depth of the feature map, W denotes the width of the feature map, H denotes the height of the feature map.

3. The multi-view attention scale mixture based colorectal and colorectal cancer segmentation system according to claim 1, wherein, The weighted sum module performs weighted sum calculation of the attention scores and the original feature map, and the calculation is specifically as follows: ； wherein, Conv denotes a 1x1 convolution operation, ⨂ denotes an element-wise multiplication, © denotes a concatenation operation along the channel dimension, A i d denotes the attention score obtained from the depth perspective, A i w denotes the attention score obtained from the width perspective, A i h denotes the attention score obtained from the height perspective, f i is the input feature for the first i encoding stage F i is the original feature map obtained by performing convolution extraction on the input feature M i is the output of the weighted sum module.

4. The multi-view attention scale mixture based colorectal and colorectal cancer segmentation system according to claim 1, wherein, The SE module performs channel dimension processing on the output of the weighted summation module to obtain an encoded feature output T i Specifically, ； wherein, Conv denotes a 1 x 1 convolution operation, denotes an element-wise addition operation, σ denotes a Sigmoid activation function, Pavg c and Pmax c denote an average pooling operator and a max pooling operator, respectively, M i is the output of the weighted sum module.

5. The multi-view attention scale mixture based colorectal and colorectal cancer segmentation system according to claim 1, wherein, The scale perception activation module unifies scales of all decoded features and merges the decoded features to obtain a mixed feature map according to an interpolation upsampling operation and convolution, and the calculation is specifically as follows: ； wherein, Conv denotes a 1 x 1 convolution operation, N is the number of up-sampling blocks, n ∈ (0, 1, 2, … N -1) is the index of the up-sampling block and the processed feature, Up ×2^(N-1-n) denotes an interpolation up-sampling operation, © denotes a concatenation operation along the channel dimension, D 0, D 1, … D N-1 is the input decoded feature, D’ is the output mixed feature map.

6. The multi-view attention scale mixture based colorectal and colorectal cancer segmentation system according to claim 1, wherein, The channel attention mechanism in the scale attention module is specifically as follows: ； where denotes an element-wise addition operation, σ denotes a Sigmoid activation function, Pavg c and Pmax c denote an average pooling operator and a max pooling operator, respectively, M s denotes a shared MLP, D scale is a channel attention result output by the channel attention mechanism, D’ is a mixed feature map.

7. The multi-view attention scale mixture based colorectal and colorectal cancer segmentation system according to claim 1, wherein, The spatial attention mechanism in the scale attention module is specifically as follows: ； wherein, Pavg s and Pmax s respectively represent the average pooling operator and the max pooling operator in the spatial attention mechanism, D scale is the scale attention result output by the channel attention mechanism; M represents the MLP; σ represents the Sigmoid activation function, D’ is the mixed feature map, ⨂ is the element-level multiplication, © is the concatenation operation along the channel dimension; The output of the spatial attention mechanism D mix The following operations are performed to obtain the final pixel-level segmentation output Pred : ； wherein SoftMax is SoftMax function.

8. The multi-view attention scale mixture based colorectal and colorectal cancer segmentation system according to claim 1, wherein, Loss used in the training process L seg Specifically: ； ； L seg = L CE + L DC ； where, L CE is the cross-entropy loss, L DC is the Dice loss, N is the number of data samples, I {•} is an MLP network, h , w , d is the size of the input data x , P s t,n is the segmentation probability output of the t th voxel for the n th class.

9. A multi-view attention scale mixture based colorectal and colorectal cancer segmentation method, characterized in that, The multi-view attention scale mixing-based colorectal and colorectal cancer segmentation system according to any one of claims 1-8 comprises the following steps: Resampling, random cropping, and labeling processing are performed on a to-be-segmented image; In the encoding stage, the processed to-be-segmented image is input into the encoder, attention scores of an original feature map from width, depth, and height perspectives are calculated through a multi-view attention layer, weighted fusion is performed on the original feature map and the attention scores respectively, and finally channel dimension processing is performed; feature extraction is performed through a downsampling block; and the feature output of each layer of the downsampling block is transmitted to a corresponding layer in the decoding stage through a skip connection mechanism; In the decoding stage, the feature output from the encoding stage is subjected to feature map restoration decoding through an upsampling block; all decoded features are unified in scale and merged to obtain a mixed feature map through a scale perception activation module; and the mixed feature map is processed through a scale attention module according to a channel attention mechanism and a spatial attention mechanism to obtain a pixel-level segmentation output.

10. A storage medium storing a program, characterized by: The program is executed by a processor to implement the multi-view attention scale mixing-based colorectal and colorectal cancer segmentation method according to claim 9.

Images