Frequency and scale co-aware network applied to pancreatic tumor segmentation

Abstract

The present application relates to medical image segmentation, in particular to a frequency and scale co-cognition network applied to pancreatic tumor segmentation, the network adopts a classical Encoder-Decoder architecture, in the Encoder, a frequency and scale aggregation module FSM is used to extract deep semantic features of the input image; in the Decoder, a frequency and scale enhancement module FSE is used to perform lightweight multi-scale feature fusion on the output features of different stages in the Encoder, and the features are enhanced through frequency domain enhancement to alleviate the semantic ambiguity and noise between different features and improve the feature expression capability of the frequency and scale co-cognition network; the technical scheme provided by the present application can effectively overcome the defect that the prior art cannot accurately segment the pancreatic tumor of the medical image.

Description

Technical Field

[0001] This invention relates to medical image segmentation, specifically to a frequency domain and scale co-sensing network applied to pancreatic tumor segmentation. Background Technology

[0002] In recent years, deep learning technology has gradually become the mainstream research method for medical image segmentation due to its powerful feature extraction capabilities, and has been applied to various medical image segmentation tasks (such as liver segmentation, kidney segmentation, pancreas segmentation, spleen segmentation, etc.). However, accurate segmentation of pancreatic tumors remains a significant challenge, mainly due to the large size variation and blurred boundaries of pancreatic tumors.

[0003] To address these issues, many studies have focused on enhancing the scale-awareness of models to improve pancreatic tumor segmentation. Among these methods, many studies are based on convolutional neural networks (CNNs), employing strategies such as convolutional pyramids, pooling pyramids, or dilated convolutional pyramids to extract multi-scale features and enhance the network's scale-awareness. However, due to the inherent local bias of CNNs, they cannot effectively extract long-range dependencies, which are particularly important for distinguishing target regions from complex backgrounds.

[0004] To address the aforementioned issues, Transformer-based methods have been widely applied in medical image segmentation, demonstrating superior performance compared to CNN methods on multiple datasets. This is primarily attributed to the effective representation of long-range features through the attention mechanism employed by Transformer-based methods. Consequently, many studies have attempted to implement local-global multi-scale perception using a hybrid CNN-Transformer architecture. However, the quadratic complexity of the attention mechanism exacerbates the computational and storage burden of the model, severely hindering the clinical deployment and application of the CNN-Transformer hybrid architecture.

[0005] Recently, the Mamba method has effectively modeled long-range dependencies using a state-space modeling (SSM) mechanism. Therefore, the CNN-Mamba framework is a multi-scale modeling method worthy of further investigation. Although some studies have attempted to use the CNN-Mamba framework for feature perception, these methods have significant shortcomings in perceiving interactive features, which limits the network's feature representation capabilities.

[0006] Furthermore, current deep learning methods focus on extracting spatial features, neglecting the guidance of frequency domain directional features. In traditional computer vision, particularly in wavelet analysis, research has shown that utilizing frequency domain information can effectively enhance target perception. Specifically, high-frequency information contains details and features of rapidly changing parts of the signal (such as edges and textures), suitable for enhancing the perception of detailed target features; while low-frequency information is an approximate representation of the signal, preserving the global structure and filtering out detail noise, which is beneficial for overall target perception. Simultaneously, changes in frequency domain information are lossless transformations, effectively mitigating information loss during deep learning. Therefore, combining frequency domain and multi-scale information to enhance the network's frequency and scale co-perception capabilities plays a crucial role in improving the network's performance in medical image segmentation. Summary of the Invention

[0007] (a) Technical problems to be solved

[0008] In view of the above-mentioned shortcomings of the existing technology, the present invention provides a frequency domain and scale co-sensing network for pancreatic tumor segmentation, which can effectively overcome the shortcomings of the existing technology in accurately segmenting pancreatic tumors in medical images.

[0009] (II) Technical Solution

[0010] To achieve the above objectives, the present invention provides the following technical solution:

[0011] A frequency domain and scale co-sensing network applied to pancreatic tumor segmentation. This network adopts a classic Encoder-Decoder architecture and includes:

[0012] In the Encoder, the Frequency Domain-Scale Aggregation (FSM) module is used to extract deep semantic features from the input image;

[0013] In the Decoder, the Frequency Domain-Scale Enhancement (FSE) module is used to perform lightweight multi-scale feature fusion on the output features of different stages in the Encoder, and the features are enhanced by frequency domain enhancement to alleviate semantic ambiguity and noise between different features and improve the feature representation capability of the frequency domain and scale co-sensing network.

[0014] The operation of the frequency domain-scale aggregation module (FSM) includes:

[0015] S11. Use Discrete Wavelet Transform (DWT) to transform the input features. I It is decomposed into different frequency domain sub-bands, including the low-frequency and high-frequency parts of the image;

[0016] S12. For the high-frequency part of the image, local feature extraction is performed using convolutional networks, and channel attention (CA) is introduced to dynamically adjust the weight distribution of features in each frequency band to form detail branches.

[0017] S13. For the low-frequency part of the image, use the Mamba module with linear complexity to perform global feature extraction and form a global branch.

[0018] S14. Use Inverse Discrete Wavelet Transform (IDWT) to restore the features of the outputs of the detail branch and the global branch.

[0019] Preferably, the Encoder includes multiple cascaded units consisting of a depthwise separable convolution (DSC) and a frequency domain-scale aggregation (FSM) module. The frequency domain-scale aggregation (FSM) module in each cascaded unit transmits the output features to the depthwise separable convolution (DSC) in the next cascaded unit, and the Decoder includes a frequency domain-scale enhancement (FSE) module.

[0020] Preferably, the Decoder includes a frequency domain-scale enhancement module FSE and a convolution module Conv. The frequency domain-scale enhancement module FSE receives the output features of the frequency domain-scale aggregation module FSM in each level of the Encoder and transmits the enhanced features to the convolution module Conv.

[0021] Preferably, the low-frequency portion of the image I LL The LL subband, considered as a thumbnail of the original image, includes the high-frequency components of the image. I LH , I HL , I HH ,in, I LH The LH subband reflects the horizontal edge information in the original image. I HL The HL subband reflects the vertical feature changes in the original image. I HH The HH subband reflects the diagonal detail features in the original image.

[0022] Preferably, in S12, for the high-frequency components of the image, local feature extraction is performed using a convolutional network, and channel attention (CA) is introduced to dynamically adjust the weight distribution of features in each frequency band, forming a detail branch, including:

[0023] S121. Employ the dilated convolutional pyramid strategy to obtain local features at different scales:

[0024] S1211. For each high-frequency part of the image, the depthwise convolution (DWC) is used to realize the perception of multi-scale features by setting different dilation coefficients, and a linear strategy is adopted to extract multi-scale features, that is, different depthwise convolutions (DWC) are used to extract the multi-scale features of each high-frequency part in order of increasing dilation coefficient.

[0025] S1212. Multi-scale features of each high-frequency part are fused by adding them pixel by pixel, and feature smoothing is performed by pixel-by-pixel convolution PWC to obtain multi-scale fused features of each high-frequency part.

[0026] S122. The multi-scale fusion features of each high-frequency part are fused by splicing, and channel attention (CA) is introduced to dynamically adjust the weight distribution of each frequency band feature.

[0027] Preferably, the operation of the frequency domain-scale enhancement module (FSE) includes:

[0028] S21. For the output features of the frequency domain-scale aggregation module FSM in each level unit of the Encoder, the channels are adjusted by pixel-wise convolution PWC, and then the spatial resolution is aligned by upsampling. The aligned features are then fused by pixel-wise addition.

[0029] S22. The fused features of all cascaded units in the Encoder are smoothed using pixel-wise convolution PWC.

[0030] S23. The discrete wavelet transform (DWT) is used to obtain spatial feature perception in the frequency domain for the fused smooth features, and a spatial attention strategy is used for feature enhancement.

[0031] S24. For each high-frequency part of the fused smooth features, spatial dependency modeling is performed using depthwise convolution (DWC), and channel dependency modeling is performed using pixel-wise convolution (PWC).

[0032] S25. The convolution results of each high-frequency part of the fused smooth feature are fused by splicing.

[0033] S26. For the low-frequency part of the fused smooth features, use the Mamba module with linear complexity to extract long-distance dependencies.

[0034] S27. Use Inverse Discrete Wavelet Transform (IDWT) to restore the high-frequency splicing features and low-frequency dependent features;

[0035] S28. Use the restored features as an attention map and perform feature enhancement on the fused smooth features by multiplying pixel by pixel.

[0036] (III) Beneficial Effects

[0037] Compared with existing technologies, the frequency domain and scale co-sensing network for pancreatic tumor segmentation provided by this invention has the following advantages:

[0038] I. Regarding the Frequency Domain-Scale Aggregation Module (FSM) in this invention:

[0039] 1) Frequency domain-multi-scale collaborative modeling: By integrating frequency domain perception and multi-scale feature extraction, the feature expression capability of the network is improved. The multi-scale structure ensures the expressive capability of semantic features, while frequency domain perception effectively alleviates the information loss problem in the feature extraction process.

[0040] 2) Frequency domain decoupling feature extraction: Differentiated feature extraction strategies are adopted for different frequency domain components, enabling the network to more accurately focus on the discriminative features of the target region;

[0041] 3) Multi-scale enhancement of low-frequency features: By modeling low-frequency information at multiple scales, the ability to capture local details is further improved, the fine-grained perception of the network is enhanced, and the image segmentation performance is optimized. In particular, a linear dilated convolutional pyramid is adopted, and a lightweight deep convolutional pyramid (DWC) is used to extract spatial features, which effectively reduces the network parameters and helps the model to be used in clinical applications.

[0042] 4) Dynamic high-frequency feature fusion: Introducing a channel attention mechanism to adaptively adjust the fusion weights of different high-frequency information, improving the model's sensitivity to differentiated features, and thus enhancing the ability to accurately characterize the target structure;

[0043] II. Regarding the Frequency Domain-Scale Enhancement Module (FSE) in this invention:

[0044] We propose a lightweight multi-scale perception architecture that improves feature representation capabilities while reducing computational complexity through the synergistic optimization of direct feature fusion and frequency domain enhancement strategies. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of the structure of the frequency domain and scale co-sensing network in this invention;

[0047] Figure 2 This is a schematic diagram of the frequency domain-scale aggregation module (FSM) of the present invention;

[0048] Figure 3 This is a schematic diagram of the frequency domain-scale enhancement module (FSE) of the present invention;

[0049] Figure 4 This is a schematic diagram of the pancreatic tumor segmentation results in the comparative experiment of this invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0051] The following describes the specific network architecture of the Frequency-Scale Co-Awareness Network (FSCA-Net) for pancreatic tumor segmentation provided by this invention, using concrete examples (e.g., ...). Figure 1 (As shown) and technical effects:

[0052] This network adopts the classic Encoder-Decoder architecture, specifically including:

[0053] In the Encoder, the Frequency Domain-Scale Aggregation (FSM) module is used to extract deep semantic features from the input image;

[0054] In the Decoder, the Frequency Domain-Scale Enhancement (FSE) module is used to perform lightweight multi-scale feature fusion on the output features of different stages in the Encoder, and the features are enhanced by frequency domain enhancement to alleviate semantic ambiguity and noise between different features and improve the feature representation capability of the frequency domain and scale co-sensing network.

[0055] The technical solution of this application, in the Encoder, is as follows: Figure 1 As shown, it includes multiple cascaded units consisting of a depthwise separable convolution (DSC) and a frequency domain-scale aggregation module (FSM). The frequency domain-scale aggregation module (FSM) in each cascaded unit transmits the output features to the depthwise separable convolution (DSC) in the next cascaded unit, as well as the frequency domain-scale enhancement module (FSE) in the Decoder.

[0056] like Figure 2 As shown, the working process of the Frequency Domain-Scale Aggregation Module (FSM) includes:

[0057] S11. The input feature I is decomposed into different frequency domain sub-bands using Discrete Wavelet Transform (DWT):

[0058] DWT(I)=I LL ,I LH ,I HL ,I HH ;

[0059] Among them, I LL The low-frequency portion of the image, represented by the LL subband, can be considered a thumbnail of the original image. LH I HL I HH All are high-frequency components of the image, I LH The LH subband reflects the horizontal edge information in the original image. HL The HL subband reflects the vertical feature changes in the original image. HH The HH subband reflects the diagonal detail features in the original image.

[0060] S12, For the high-frequency part I of the image LH I HL and I HH The advantages of convolutional networks in perceiving detailed features are utilized to extract local features, and channel attention (CA) is introduced to dynamically adjust the weight distribution of features in each frequency band, forming detail branches and achieving adaptive feature optimization.

[0061] S13, For the low-frequency part of the image I LL The Mamba module with linear complexity is used to extract global features and form global branches to distinguish the target region from the complex background.

[0062] S14. Use Inverse Discrete Wavelet Transform (IDWT) to restore the features of the outputs of the detail branch and the global branch.

[0063] Specifically, by decoupling local and global features through S12 and S13, the hierarchical perception capability of the frequency domain and scale co-sensing network for multi-scale features is improved.

[0064] Specifically, in S12, for the high-frequency part I of the image LH I HL and I HH Each method leverages the advantages of convolutional networks in perceiving detailed features to extract local features, and introduces channel attention (CA) to dynamically adjust the weight distribution of features in each frequency band, forming a detail branch, including:

[0065] S121. To improve the feature extraction capability of the frequency domain and scale co-sensing network for high-frequency components, a dilated convolutional pyramid strategy is adopted to obtain local features at different scales:

[0066] S1211. For each high-frequency component of the image, depthwise convolution (DWC) is used to perceive multi-scale features by setting different dilation coefficients, and a linear strategy is adopted to extract multi-scale features, that is, different depthwise convolutions (DWC) are used to extract multi-scale features of each high-frequency component in order of increasing dilation coefficient:

[0067] For LH subband I LH :

[0068] f DWC,1 (I LH )=I' LH f DWC,2 (I' LH )=I'' LH f DWC,4 (I'' LH )=I''' LH ;

[0069] Among them, f DWC,k This represents a depthwise convolution (DWC) operation with a 3x3 kernel and dilation coefficient k, where k = 1, 2, 4. LH For LH subband I LH In the depthwise convolution result with a dilation factor of 1, I'' LH For I' LH In the depthwise convolution result with a dilation factor of 2, I''' LH For I'' LH The result of a depthwise convolution with a dilation factor of 4;

[0070] For HL subband I HL :

[0071] f DWC,1 (I HL )=I' HL f DWC,2 (I' HL )=I'' HL f DWC,4 (I'' HL )=I''' HL ;

[0072] Among them, I' HL For HL subband I HL In the depthwise convolution result with a dilation factor of 1, I'' HL For I' HL In the depthwise convolution result with a dilation factor of 2, I''' HL For I'' HL Results of depthwise convolution with a dilation factor of 4;

[0073] For HH subband I HH :

[0074] f DWC,1 (I HH )=I' HH f DWC,2 (I' HH )=I'' HH f DWC,4 (I'' HH )=I''' HH ;

[0075] Among them, I' HH HH subband I HH In the depthwise convolution result with a dilation factor of 1, I'' HH For I' HH In the depthwise convolution result with a dilation factor of 2, I''' HH For I'' HH Results of depthwise convolution with a dilation factor of 4;

[0076] S1212. To further enhance the feature representation capability of the frequency domain and scale co-sensing network, the multi-scale features of each high-frequency part are fused by pixel-wise addition, and pixel-wise convolution PWC is used for feature smoothing.

[0077] ;

[0078] ;

[0079] ;

[0080] Among them, f PWC This indicates a pixel-wise convolution (PWC) operation. , , LH subband I LH HL sub-band I HL HH subband I HH Multi-scale fusion features;

[0081] S122. The multi-scale fusion features of each high-frequency component are fused by splicing, and channel attention (CA) is introduced to dynamically adjust the weight distribution of features in each frequency band:

[0082] ;

[0083] Where Cat represents the feature concatenation operation, and I H High-frequency fusion features output for detail branches.

[0084] Specifically, in S13, for the low-frequency part I of the image LLGlobal feature extraction is performed using the linearly complex Mamba module, forming a global branch, including:

[0085] For the low-frequency part I of the image LL Long-distance dependency modeling using Mamba modules with linear complexity:

[0086] f Mamba (I LL )=I' LL ;

[0087] Among them, f Mamba This indicates that the Mamba module is handling it, I' LL This represents the low-frequency characteristics of the global branch output.

[0088] Specifically, in S14, the inverse discrete wavelet transform (IDWT) is used to restore the features of the outputs of the detail branch and the global branch, including:

[0089] IDWT(I H ,I' LL )=I';

[0090] Where I' is the output feature of the frequency domain-scale aggregation module FSM.

[0091] The above technical solution applies to the frequency domain-scale aggregation module FSM in this invention:

[0092] 1) Frequency domain-multi-scale collaborative modeling: By integrating frequency domain perception and multi-scale feature extraction, the feature expression capability of the network is improved. The multi-scale structure ensures the expressive capability of semantic features, while frequency domain perception effectively alleviates the information loss problem in the feature extraction process.

[0093] 2) Frequency domain decoupling feature extraction: Differentiated feature extraction strategies are adopted for different frequency domain components, enabling the network to more accurately focus on the discriminative features of the target region;

[0094] 3) Multi-scale enhancement of low-frequency features: By modeling low-frequency information at multiple scales, the ability to capture local details is further improved, the fine-grained perception of the network is enhanced, and the image segmentation performance is optimized. In particular, a linear dilated convolutional pyramid is adopted, and a lightweight deep convolutional pyramid (DWC) is used to extract spatial features, which effectively reduces the network parameters and helps the model to be used in clinical applications.

[0095] 4) Dynamic high-frequency feature fusion: Introducing a channel attention mechanism to adaptively adjust the fusion weights of different high-frequency information, improving the model's sensitivity to differentiated features, and thus enhancing the ability to accurately characterize the target structure.

[0096] The technical solution of this application, in the Decoder, is as follows: Figure 1 As shown, it includes a frequency domain-scale enhancement module (FSE) and a convolution module (Conv). The frequency domain-scale enhancement module (FSE) receives the output features of the frequency domain-scale aggregation module (FSM) in each level of the encoder and transmits the enhanced features to the convolution module (Conv).

[0097] like Figure 3 As shown, the working process of the Frequency Domain-Scale Enhancement (FSE) module includes:

[0098] S21. For the output features of the Frequency-Scale Aggregation Module (FSM) in each coordinating unit of the Encoder, the channels are adjusted using pixel-wise convolution (PWC), and then the spatial resolution is aligned by upsampling. The aligned features are then fused by pixel-wise addition.

[0099] ;

[0100] Among them, E i Let UP represent the output feature of the frequency-scale aggregation module FSM in cascaded unit i of the Encoder, where UP represents the upsampling operation and D is the fused feature of all cascaded units in the Encoder.

[0101] S22. The fused feature D of all cascaded units in the Encoder is smoothed using pixel-wise convolution PWC:

[0102] f PWC (D)=D';

[0103] Where D' represents the fusion smoothing feature;

[0104] S23. The fused smooth feature D' is obtained by using Discrete Wavelet Transform (DWT) to capture spatial features in the frequency domain, and a spatial attention strategy is used for feature enhancement:

[0105] DWT(D')=D LL D LH D HL D HH ;

[0106] Among them, D LL To fuse the low-frequency portion of the smoothing feature D', we have the LL subband, which contains the low-frequency information of the original image. LH D HL D HH All are high-frequency components of the fused smooth feature D', D LH The LH subband contains horizontal edge information from the original image, i.e., high-frequency variations in the vertical direction of the image. HLThe HL subband contains vertical edge information from the original image, i.e., high-frequency variations in the image along the horizontal direction. HH The HH subband reflects the diagonal detail features in the original image.

[0107] S24. For each high-frequency component of the fused smooth feature D', spatial dependency modeling is performed using depthwise convolution (DWC), and channel dependency modeling is performed using pixel-wise convolution (PWC). This reduces the complexity of the model while extracting local detail information.

[0108] f PWC [f DWC,1 (D LH )]=D' LH ;

[0109] f PWC [f DWC,1 (D HL )]=D' HL ;

[0110] f PWC [f DWC,1 (D HH )]=D' HH ;

[0111] Among them, D' LH D' HL D' HH They are LH subband D LH HL sub-band D HL HH subband D HH The convolution result;

[0112] S25. The convolution results of each high-frequency part of the fused smooth feature D' are fused by concatenation:

[0113] Cat(D' LH ,D' HL ,D' HH )=D' H ;

[0114] Among them, D' H It is characterized by high-frequency splicing;

[0115] S26. For the low-frequency component D' of the fusion smoothing feature D' LL Extracting long-distance dependencies using Mamba modules with linear complexity:

[0116] f Mamba (D LL )=D' LL ;

[0117] Among them, D' LLIt is a low-frequency dependent feature;

[0118] S27. Using Inverse Discrete Wavelet Transform (IDWT) to analyze the high-frequency splicing feature D' H and low-frequency dependent features D' LL Perform feature restoration:

[0119] IDWT(D' H ,D' LL )=D score ;

[0120] Among them, D score Features for restoration;

[0121] S28, restore feature D score As an attention map, feature enhancement is performed on the fused smooth feature D' using pixel-wise multiplication:

[0122] D out =D'×D score ;

[0123] Among them, D out This refers to the output features of the Frequency Domain-Scale Aggregation Module (FSM), i.e., the enhanced features.

[0124] To better illustrate the technical solution of this application, a comparative experiment will be used to provide a detailed explanation below.

[0125] I. Sample Preprocessing

[0126] A thresholding method is used to remove some tissues irrelevant to the target in CT images, enabling the network to effectively extract features from the target region. Specifically, the HU value is first normalized to [-150, 300], and then mean normalization is used to normalize the processed HU value to [0, 1].

[0127] II. Network Construction

[0128] according to Figure 1 The network architecture shown is constructed using the PyTorch library to build the frequency domain and scale co-sensing network FSCA-Net proposed in this invention. This network adopts the classic Encoder-Decoder architecture, in which the Encoder uses the frequency domain-scale aggregation module FSM stack to extract deep semantic features from the input image, and the Decoder uses the frequency domain-scale enhancement module FSE to perform lightweight multi-scale feature fusion and enhancement on the output features of different stages in the Encoder.

[0129] The encoder consists of 5 stages (corresponding to 5 cascaded units), with each stage having 32, 64, 128, 256, and 512 output channels, respectively. The spatial resolution and channels are adjusted between different stages using depthwise separable convolution (DSC) with a stride of 2.

[0130] III. Model Training

[0131] The FSCA-Net network was trained on the publicly available pancreatic tumor segmentation dataset MSD. MSD contains 281 CT scans with slice thicknesses ranging from 0.5 to 2.5 mm. 200 scans were selected as the training set, 20 as the validation set, and the remaining 61 as the test set. First, the samples were preprocessed according to the method described in step one, and then the FSCA-Net network was constructed using the method described in step two.

[0132] During training, the AdamW optimizer was used for model optimization, with an initial learning rate of 0.001, a batch size of 32, an input image size of 1*512*512, and a maximum number of iterations of 10000. Dice loss and cross-entropy loss were used for model optimization during training, and the total loss function was:

[0133] ;

[0134] Where L is the total loss function, , Representing Dice loss and cross-entropy loss respectively, y, These are the labels and the prediction results, respectively.

[0135] For convenience, model validation was performed every 100 iterations in this experiment, and the best model was selected for the final model test.

[0136] IV. Model Testing and Performance Evaluation

[0137] The optimal model obtained in step three and other existing models were used to test the model on the test set. The test results are shown in Table 1.

[0138] Table 1. Comparison of pancreatic tumor segmentation performance of different models

[0139]

[0140] The frequency domain and scale co-aware network FSCA-Net proposed in this invention effectively reduces the number of network parameters and achieves competitive segmentation results for pancreatic tumor segmentation. Specifically, compared with the classic UNet, the FSCA-Net network exhibits advantages in segmentation performance and computational complexity, with a 54.09% improvement in pancreatic tumor segmentation performance (54.87% vs 35.61%), while the number of parameters and computational complexity are only 6.82% (2.137M vs 31.322) and 16.26% (47.859G vs 294.423G) of UNet, respectively. This is mainly attributed to the fact that the FSCA-Net network enhances the model's feature representation ability through co-awareness in the frequency domain and scale, thereby improving the overall segmentation performance.

[0141] As shown in Table 1, although UNETR has the highest number of parameters and computational complexity (90.639M and 893.432G), indicating that it has a strong feature extraction capability, UNETR has the worst pancreatic tumor segmentation performance. This is mainly because UNETR uses a single scale for feature extraction, which cannot effectively perceive the variable characteristics of tumor scale, thus leading to unsatisfactory segmentation results.

[0142] Although UNet++ outperformed FSCA-Net in pancreatic tumor segmentation (55.24% vs 54.87%), its overall pancreatic segmentation performance was significantly weaker than FSCA-Net. This is primarily attributed to the Mamba operation in FSCA-Net, which enhances the network's ability to express long-range dependencies. Furthermore, FSCA-Net's parameter count and computational complexity were only 5.81% (2.137M vs 36.768M) and 6.83% (47.859G vs 700.398G) of UNet++, respectively.

[0143] The pancreatic tumor segmentation results of the above comparative experiments are as follows: Figure 4 As shown, the red area represents the pancreas, and the green area represents pancreatic tumors. In summary, the frequency domain and scale co-sensing network FSCA-Net proposed in this invention exhibits segmentation results closer to the labels, proving the effectiveness of the technical solution presented in this application.

[0144] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A frequency domain and scale co-sensing network applied to pancreatic tumor segmentation, characterized by: This network adopts the classic Encoder-Decoder architecture, including: In the Encoder, the Frequency Domain-Scale Aggregation (FSM) module is used to extract deep semantic features from the input image; In the Decoder, the Frequency Domain-Scale Enhancement (FSE) module is used to perform lightweight multi-scale feature fusion on the output features of different stages in the Encoder, and the features are enhanced by frequency domain enhancement to alleviate semantic ambiguity and noise between different features and improve the feature representation capability of the frequency domain and scale co-sensing network. The operation of the frequency domain-scale aggregation module (FSM) includes: S11. Use Discrete Wavelet Transform (DWT) to transform the input features. I It is decomposed into different frequency domain sub-bands, including the low-frequency and high-frequency parts of the image; S12. For the high-frequency part of the image, local feature extraction is performed using convolutional networks, and channel attention (CA) is introduced to dynamically adjust the weight distribution of features in each frequency band to form detail branches. S13. For the low-frequency part of the image, use the Mamba module with linear complexity to perform global feature extraction and form a global branch. S14. Use Inverse Discrete Wavelet Transform (IDWT) to restore the features of the outputs of the detail branch and the global branch. The operation of the frequency domain-scale enhancement module (FSE) includes: S21. For the output features of the frequency domain-scale aggregation module FSM in each level unit of the Encoder, the channels are adjusted by pixel-wise convolution PWC, and then the spatial resolution is aligned by upsampling. The aligned features are then fused by pixel-wise addition. S22. The fused features of all cascaded units in the Encoder are smoothed using pixel-wise convolution PWC. S23. The discrete wavelet transform (DWT) is used to obtain spatial feature perception in the frequency domain for the fused smooth features, and a spatial attention strategy is used for feature enhancement. S24. For each high-frequency part of the fused smooth features, spatial dependency modeling is performed using depthwise convolution (DWC), and channel dependency modeling is performed using pixel-wise convolution (PWC). S25. The convolution results of each high-frequency part of the fused smooth feature are fused by splicing. S26. For the low-frequency part of the fused smooth features, use the Mamba module with linear complexity to extract long-distance dependencies. S27. Use Inverse Discrete Wavelet Transform (IDWT) to restore the high-frequency splicing features and low-frequency dependent features; S28. Use the restored features as an attention map and perform feature enhancement on the fused smooth features by multiplying pixel by pixel.

2. The frequency domain and scale co-sensing network for pancreatic tumor segmentation according to claim 1, characterized in that: The Encoder consists of multiple cascaded units composed of a depthwise separable convolution (DSC) and a frequency domain-scale aggregation (FSM) module. The FSM module in each cascaded unit transmits the output features to the depthwise separable convolution (DSC) in the next cascaded unit. The Decoder also includes a frequency domain-scale enhancement (FSE) module.

3. The frequency domain and scale co-sensing network for pancreatic tumor segmentation according to claim 2, characterized in that: The Decoder includes a frequency-scale enhancement module (FSE) and a convolution module (Conv). The frequency-scale enhancement module (FSE) receives the output features of the frequency-scale aggregation module (FSM) in each stage of the Encoder and transmits the enhanced features to the convolution module (Conv).

4. The frequency domain and scale co-sensing network for pancreatic tumor segmentation according to claim 2, characterized in that: The low-frequency component of the image I LL The LL subband, considered as a thumbnail of the original image, includes the high-frequency components of the image. I LH , I HL , I HH ,in, I LH The LH subband reflects the horizontal edge information in the original image. I HL The HL subband reflects the vertical feature changes in the original image. I HH The HH subband reflects the diagonal detail features in the original image.

5. The frequency domain and scale co-sensing network for pancreatic tumor segmentation according to claim 4, characterized in that: In S12, for the high-frequency components of the image, local feature extraction is performed using convolutional networks, and channel attention (CA) is introduced to dynamically adjust the weight distribution of features in each frequency band, forming detail branches, including: S121. Employ the dilated convolutional pyramid strategy to obtain local features at different scales: S1211. For each high-frequency part of the image, the depthwise convolution (DWC) is used to realize the perception of multi-scale features by setting different dilation coefficients, and a linear strategy is adopted to extract multi-scale features, that is, different depthwise convolutions (DWC) are used to extract the multi-scale features of each high-frequency part in order of increasing dilation coefficient. S1212. Multi-scale features of each high-frequency part are fused by adding them pixel by pixel, and feature smoothing is performed by pixel-by-pixel convolution PWC to obtain multi-scale fused features of each high-frequency part. S122. The multi-scale fusion features of each high-frequency part are fused by splicing, and channel attention (CA) is introduced to dynamically adjust the weight distribution of each frequency band feature.

Images