Colorectal cancer pathological image segmentation method and device based on frequency domain features and readable storage medium thereof

Abstract

The application provides a colorectal cancer pathological image segmentation method and device based on frequency domain features and a readable storage medium thereof. In view of the problems of an existing model, such as insufficient boundary detail recovery, poor multi-scale feature fusion effect and limited generalization ability in colorectal cancer pathological image segmentation, the application provides an improved TransUnet model. The model extracts high and low frequency features of an image by introducing a multi-scale frequency reconstruction module (MFRM), combines an attention aggregation module (AAM) to fuse frequency domain and spatial domain features, adopts F-Ctrans skip connection to optimize feature transmission, and finally outputs a high-precision segmentation result through a decoder. Experiments show that the method is superior to the existing model in terms of Dice, IoU and HD95 and the like, significantly improves the segmentation accuracy and boundary detail recovery ability, and has stronger generalization ability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, in particular to a colorectal cancer pathological image segmentation method and device based on frequency domain features and a readable storage medium thereof. BACKGROUND

[0002] Colorectal adenocarcinoma is a common malignant tumor, and early diagnosis is crucial for improving patient prognosis. Pathological images, as the core basis for auxiliary diagnosis of imaging, their accurate analysis is the key to early diagnosis. Traditional pathological image analysis relies on the experience and naked eye observation of doctors, which is significantly affected by subjective factors, resulting in insufficient stability and accuracy of the diagnosis results. Moreover, when dealing with a large amount of image data, manual inspection is inefficient and prone to missed diagnosis or misdiagnosis, especially in the accurate segmentation of tumor regions and the identification of complex edges.

[0003] With the development of artificial intelligence and deep learning technology, convolutional neural networks (CNN) and Transformer technology have been widely applied in medical image analysis, significantly improving the level of automation and intelligence. Among them, the existing TransUnet model combines the advantages of CNN and Transformer, and has made progress in medical image segmentation tasks, but still has defects: when processing tumor boundaries, the segmentation accuracy and detail recovery ability are insufficient, and the processing effect of complex edge regions is worse than manual segmentation; existing improvement schemes are mostly limited to single spatial domain or frequency domain feature extraction, and cannot achieve efficient fusion of multi-domain features.

[0004] At present, there is no comprehensive and automated deep learning framework that can effectively improve the accuracy of colorectal cancer pathological image segmentation while considering the generalization ability and processing efficiency of the model. Therefore, an improved technical solution is needed to solve the problems of blurred boundaries and insufficient multi-scale feature fusion, to meet the clinical demand for high-precision pathological image segmentation. SUMMARY

[0005] The present application provides a colorectal cancer pathological image segmentation method and device based on frequency domain features and a readable storage medium thereof, which addresses the problems of insufficient boundary detail recovery, poor multi-scale feature fusion, and limitation to single spatial domain or frequency domain feature extraction in existing technologies (such as TransUnet and related improvement schemes) in colorectal cancer pathological image segmentation, and lack of a comprehensive framework that can simultaneously improve segmentation accuracy, generalization ability, and processing efficiency.

[0006] The core technology of the present application is mainly to improve the TransUnet model (FD-TransUnet), introduce a multiscale frequency reconstruction module (MFRM), F-Ctrans skip connection and an adaptive attention mechanism (AAM), fuse frequency domain and spatial domain features, realize high-precision segmentation of colorectal cancer pathological images, and especially improve the boundary detail recovery and multiscale feature fusion capability.

[0007] In a first aspect, the present application provides a colorectal cancer pathological image segmentation method based on frequency domain features, comprising the following steps:

[0008] Step 1: Obtain the colorectal cancer pathological original input image, process the original input image through the multiscale frequency reconstruction module, extract the high-frequency component and the low-frequency component in the frequency domain, wherein the high-frequency component corresponds to the details and edge information of the image, and the low-frequency component corresponds to the overall contour information of the image;

[0009] Step 2: Extract spatial features of the original input image through an encoder, the encoder combines a convolutional neural network and a Vision Transformer (VIT), and outputs spatial features, the spatial features containing global context information of the image;

[0010] Step 3: Fuse the frequency features output in step 1 and the spatial features output in step 2 through an attention aggregation module, the fusion process including channel splicing, 1x1 convolution processing and height-width attention mechanism weighting, to obtain fused features;

[0011] Step 4: Transfer and process the features output by the encoder and the fused features through F-Ctrans skip connection, the transfer and processing including Fourier transform to convert the features to the frequency domain, learnable filter weighting, inverse Fourier transform to convert back to the space-time domain, and combining residual connection to optimize feature expression;

[0012] Step 5: Process the features transferred through F-Ctrans skip connection through a decoder, the decoder using an up-sampling operator to recover spatial information and an SE (Squeeze-and-Excitation) layer to adaptively adjust channel features, and output the segmentation result of the colorectal cancer pathological image.

[0013] Further, in step 1, the processing process of the multiscale frequency reconstruction module includes:

[0014] The original input image is blocked, and the frequency characteristics are extracted from each image block by two-dimensional discrete cosine transform, and the frequency characteristic map containing high frequency components and low frequency components is obtained by recombining the frequency characteristics.

[0015] Further, in step 3, the height-width attention mechanism includes: performing global pooling along the height direction and the width direction respectively, calculating the channel attention weights of the two directions, multiplying the attention weights back to the fusion features after 1×1 convolution processing, enhancing the key features and suppressing the redundant information.

[0016] Further, in step 4, the F-Ctrans jump connection processing further includes: channel fusion, flattening and position coding are performed on the input features, and then Fourier transform is performed after multi-layer perception processing, and after the frequency characteristics and spatial characteristics are interactively fused, the original feature dimension is restored through the feature reconstruction module.

[0017] Further, in step 5, the decoder uses the Dysample upsampling operator to recover the spatial information, and the SE layer adjusts the weights of each channel feature by squeezing and exciting the channel features.

[0018] In a second aspect, the present application provides a colorectal cancer pathological image segmentation device based on frequency domain features, comprising:

[0019] An input module is configured to obtain a colorectal cancer pathological original input image.

[0020] A multi-scale frequency reconstruction module is configured to process the original input image and extract high frequency components and low frequency components in the frequency domain, wherein the high frequency components correspond to the details and edge information of the image, and the low frequency components correspond to the overall contour information of the image.

[0021] An encoder module is configured to extract spatial features of the original input image, and the encoder module combines a convolutional neural network and a Vision Transformer (VIT) to output spatial features containing global context information.

[0022] An attention aggregation module is configured to fuse the frequency features output by the multi-scale frequency reconstruction module and the spatial features output by the encoder module, and the fusion process includes channel splicing, 1×1 convolution processing and height-width attention mechanism weighting to obtain fusion features.

[0023] An F-Ctrans jump connection module is configured to transfer and process the features output by the encoder module and the fusion features, and the transfer processing includes Fourier transform to convert the features to the frequency domain, learnable filter weighting, inverse Fourier transform to convert back to the space-time domain, and combining residual connection to optimize feature expression.

[0024] The decoder module is used for processing the features transmitted by the F-Ctrans skip connection module, restoring spatial information by using an up-sampling operator, and adaptively adjusting channel features by an SE (Squeeze-and-Excitation) layer, and outputs a segmentation result of the colorectal cancer pathological image.

[0025] Further, the multi-scale frequency reconstruction module comprises:

[0026] The block unit is used for block processing of the original input image.

[0027] The DCT processing unit is used for extracting frequency features of each image block by two-dimensional discrete cosine transform.

[0028] The reorganization unit is used for reorganizing the frequency features output by the DCT processing unit to obtain a frequency feature map containing high-frequency components and low-frequency components.

[0029] The attention aggregation module comprises:

[0030] The channel splicing unit is used for splicing the frequency features and the spatial features in the channel dimension.

[0031] The convolution fusion unit is used for extracting and fusing the spliced features by a 1x1 convolution layer.

[0032] The attention weighting unit is used for adopting a height-width attention mechanism, performing global pooling along the height and width directions, and calculating channel attention weights, multiplying the weights back to the fused features to enhance key information.

[0033] Further, the F-Ctrans skip connection module comprises:

[0034] The preliminary processing unit is used for channel fusion, flattening and position encoding of the input features.

[0035] The frequency domain enhancement unit is used for converting the features to the frequency domain by a multi-layer perceptron and a Fourier transform, weighting the selected frequency modes by a learnable filter, and converting back to the space-time domain by an inverse Fourier transform.

[0036] The fusion optimization unit is used for interactive fusion of the frequency domain enhanced features and the spatial features, and combines residual connection and multi-layer perceptron to optimize feature expression.

[0037] The feature reconstruction unit is used for recovering the original feature dimension by a reconstruction operation to generate output features.

[0038] The decoder module comprises:

[0039] The up-sampling unit is used for restoring spatial information by using a Dysample up-sampling operator.

[0040] an SE processing unit configured to adaptively adjust weights of channel features through SE layer operation;

[0041] an output unit configured to output a segmentation result of the colorectal cancer pathological image based on the processed features.

[0042] In a third aspect, the present application provides an electronic device comprising a memory and a processor, the memory storing a computer program, and the processor being configured to run the computer program to perform the above-mentioned colorectal cancer pathological image segmentation method based on frequency domain features.

[0043] In a fourth aspect, the present application provides a readable storage medium, the readable storage medium storing a computer program, the computer program comprising program codes for controlling a process to perform the process, and the process comprising the above-mentioned colorectal cancer pathological image segmentation method based on frequency domain features.

[0044] The main contributions and innovative points of the present application are as follows:

[0045] 1. Significantly improve segmentation accuracy: through the multi-scale frequency reconstruction module (MFRM) to extract frequency domain high and low frequency features, the attention aggregation module (AAM) to fuse frequency domain and spatial domain features, the Dice coefficient, IoU and other indicators are better than existing models (such as TransUnet), and the overall segmentation effect is better.

[0046] 2. Optimize boundary detail recovery: F-Ctrans skip connection enhances the use of frequency domain information in feature transmission, combined with the ability of frequency domain high frequency component to capture details and edges, which makes the HD95 index significantly reduced, the boundary segmentation is closer to the real boundary, effectively solves the segmentation problem of complex edge and fuzzy area.

[0047] 3. Enhance the generalization ability of the model: through cross-domain (spatial domain + frequency domain) feature fusion and multi-module collaborative design, it performs stably on different data sets such as Glas and CRAG, overcoming the limitations of single domain feature extraction, and adapting to different types of segmentation tasks.

[0048] 4. Improve processing efficiency and automation level: the integrated deep learning framework constructed realizes the automatic processing of pathological image segmentation, reduces the dependence on manual intervention, improves the processing efficiency while ensuring the accuracy, and is more suitable for large-scale clinical image analysis scenarios.

[0049] The details of one or more embodiments of the present application are presented in the following drawings and description, so that other features, objects and advantages of the present application are more apparent. BRIEF DESCRIPTION OF DRAWINGS

[0050] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0051] Figure 1 is a structural diagram of the overall model according to an embodiment of the application;

[0052] Figure 2 is a core flow chart of the multi-scale frequency reconstruction module (MFRM) and the CNN encoder collaborative processing according to an embodiment of the application;

[0053] Figure 3 is a process chart of the image processing by the multi-scale frequency reconstruction module according to an embodiment of the application;

[0054] Figure 4 is a structural diagram of the AAM aggregation module according to an embodiment of the application;

[0055] Figure 5 is a structural diagram of the F-Ctrans skip connection according to an embodiment of the application;

[0056] Figure 6 is a structural diagram of the F-Ctrans channel cross attention according to an embodiment of the application;

[0057] Figure 7 is a structural diagram of the MLP according to an embodiment of the application;

[0058] Figure 8 is a prediction result and an original image (a) of the FD-TransUnet according to an embodiment of the application;

[0059] Figure 9 is a schematic diagram of the hardware structure of the electronic device according to an embodiment of the application. DETAILED DESCRIPTION

[0060] The exemplary embodiments will be described in detail herein below with reference to the drawings. In the following description, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all the implementations consistent with one or more embodiments of the present description. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of the present description as detailed in the appended claims.

[0061] It should be noted that the steps of the corresponding method are not necessarily performed in the order shown and described in the present specification in other embodiments. In some other embodiments, the steps included in the method thereof can be more or less than described in the present specification. In addition, a single step described in the present specification can be divided into multiple steps for description in other embodiments; and multiple steps described in the present specification can also be combined into a single step for description in other embodiments.

[0062] The prior art (such as TransUnet and related improved schemes) has the problems of insufficient boundary detail recovery, poor multi-scale feature fusion effect, and limitation to single spatial domain or frequency domain feature extraction in colorectal cancer pathological image segmentation, and lacks a comprehensive framework that can simultaneously improve segmentation accuracy, generalization ability and processing efficiency.

[0063] Based on this, the present application solves the problems existing in the prior art based on the improved TransUnet model.

[0064] Embodiment one

[0065] The method of the present application extracts high-frequency components and low-frequency components of the pathological image from the frequency domain, wherein the high-frequency components correspond to the details, edges and pixels of the image, and the low-frequency components correspond to the overall outline of the image. After the original input image extracts the frequency features through the multiscale frequency reconstruction module (MFRM), the spatial features output by the Vision Transformer (VIT) encoder are fused through the adaptive attention mechanism (AAM). The FCtrans skip connection is introduced in the skip connection part, and the core idea is that before applying the attention mechanism, the input sequence features are first converted from the time domain to the frequency domain through the Fourier transform (FFT). In the frequency domain, the selected frequency pattern is weighted through a learnable filter to filter the components of a specific frequency to extract more representative features. Subsequently, the frequency domain features are converted back to the space-time domain through the inverse Fourier transform (IFFT), thereby restoring the format consistent with the original time domain sequence. The output of this process is connected with the residual of the original input features layer by layer, thereby enhancing the learning ability of the network to the frequency domain pattern, improving the expression ability of the features and the performance of the model. This method effectively solves the problem of rough segmentation results and imprecise edges caused by the pure Transformer model in the segmentation task.

[0066] Specifically, the present application provides a colorectal cancer pathological image segmentation method based on frequency domain features, specifically, referring to Figure 1The structure of the overall model of the application, the segmentation method based on the model comprises the following steps:

[0067] Step 1: Obtain the original input image of colorectal cancer pathology, process the original input image through a multi-scale frequency reconstruction module (MFRM), and extract high-frequency components (components) and low-frequency components (components) in the frequency domain, wherein the high-frequency components correspond to the details and edge information of the image, and the low-frequency components correspond to the overall contour information of the image;

[0068] As Figure 1 The model takes the colorectal cancer pathology image as input, and realizes high-precision segmentation through four stages of “frequency feature extraction (MFRM) → spatial feature coding (CNN+ViT) → cross-domain fusion (AAM+F-Ctrans) → segmentation decoding”.

[0069] As Figure 2 The core process of the multi-scale frequency reconstruction module (MFRM) and the CNN encoder cooperative processing of the application is shown, and the specific analysis is as follows:

[0070] 1) Input layer:

[0071] The left side is a colorectal cancer pathology image (such as a gray pathological section, containing gland, cell and other structures, which is the original input of the segmentation task).

[0072] 2) Image blocking (orange module “image segmentation”):

[0073] It is not a traditional “segmentation result output”, but a preprocessing blocking operation - cutting a large size pathological image into several small image blocks (such as fixed size subgraphs), the purpose is to reduce the DCT calculation complexity, at the same time adapt to the input dimension of the subsequent model, realize multi-scale frequency analysis (different blocks can correspond to different frequency resolution).

[0074] 3) DCT module (green “DCT”):

[0075] Perform two-dimensional discrete cosine transform (DCT) on each image block to convert the spatial domain image into frequency domain features:

[0076] Low-frequency component: concentrates the “global contour” of the image (such as the overall shape of the tumor area), which corresponds to the upper left corner of the DCT coefficient matrix;

[0077] High-frequency component: describes “details and edges” (such as tumor boundaries, cell textures), which corresponds to the lower right corner of the DCT coefficient matrix.

[0078] This step is the core of frequency domain feature extraction, which provides “detail-contour” dimensional information for subsequent fusion of spatial domain features.

[0079] 4) CNN subnetwork (right gray feature map):

[0080] The frequency features processed by DCT are input into a convolutional neural network (CNN) for spatial domain feature enhancement:

[0081] The feature map dimensions (such as 64x18x18, 64x9x9, etc.) represent the number of channels x spatial size. Through the down-sampling of the convolutional layer (spatial size reduction, channel number increase), multi-scale spatial features are gradually extracted (small size feature maps correspond to global information, large size corresponds to local details);

[0082] This CNN is the front-end module of the encoder, which cooperates with the subsequent Vision Transformer (ViT) to complete the complementary extraction of "local-global" spatial features.

[0083] In this embodiment, as shown in Figure 3 The MFRM extracts frequency features through discrete cosine transform (DCT) and processes them in combination with CNN to adjust the output. To overcome the limitations of single domain analysis, the overall model of the present application introduces frequency domain information. The frequency domain describes the overall structure, rate of change, etc. of the image, among which the high-frequency components correspond to the details, edges and textures of the image, and the low-frequency components correspond to the overall outline and slow changes of the image. The specific steps are as follows:

[0084] 1) input image;

[0085] 2) The input image is processed by block, and each block extracts frequency features through DCT (two-dimensional discrete cosine transform). These features help to capture the frequency information of the image, including high-frequency details and low-frequency overall structure;

[0086] 3) After DCT processing, the obtained frequency features are reorganized and further processed. At this time, the frequency information in the image is rearranged so that the model can better analyze and utilize the frequency features;

[0087] 4) Output the frequency feature map to support subsequent segmentation.

[0088] The DCT two-dimensional discrete cosine transform formula is as follows:

[0089] Given an input matrix F of N x M (usually M equals N, N represents the horizontal length), the discrete cosine transform result G can be calculated by the following formula:

[0090]

[0091]

[0092] where the spatial domain matrix the gray value of a pixel point in the original image an element in the transformed coefficient matrix, representing the weight of different frequency components, u, v are frequency coordinates, corresponding to the frequency components in the horizontal / vertical direction. an orthogonal normalized coefficient, when , , otherwise .

[0093] Step 2: Spatial feature extraction of the original input image by the encoder, the encoder combines Convolutional Neural Network (CNN) and Vision Transformer (VIT), outputs spatial features, which contain global context information of the image;

[0094] In this embodiment, the CNN performs "layer-by-layer abstraction" on the input image through convolution, pooling, activation, etc.:

[0095] Shallow layer (such as the first 2-3 layers): capture fine-grained spatial details (such as cell edges, texture mutations, small-scale structures), corresponding to local features such as "tumor boundary wrinkles" and "cell arrangement disorder" in pathological images.

[0096] Deep layer (such as the last 2-3 layers): aggregate local features to form high-level semantic representation (such as the overall outline of the tumor area, the distinction of tissue types), corresponding to the macroscopic structural differences between "tumor area" and "normal gland".

[0097] ViT divides the feature map output by CNN into token sequences (or directly processes the feature map), and models long-range spatial correlations through multi-head self-attention (Multi-Head Attention):

[0098] Self-attention calculates the correlation weight of different spatial positions: if there is a semantic correlation (such as morphological similarity, distribution regularity) between "tumor edge area" and "distant background tissue" in the pathological image, ViT will give high weight, thereby modeling the global context relationship (such as the spatial distribution of the tumor in the whole image, the location correlation with the normal gland).

[0099] In this way, through multi-scale spatial feature extraction by CNN and global context modeling by ViT, the "detailed and accurate, semantically complete" spatial domain basic features are provided for subsequent cross-domain feature fusion (AAM module) and segmentation decoding, which is the core support for realizing high-precision segmentation of colorectal cancer pathological images.

[0100] Step 3: Fusion of the frequency features output in step 1 and the spatial features output in step 2 through the attention aggregation module (AAM), the fusion process includes channel splicing, 1x1 convolution processing and height-width attention mechanism weighting, to obtain the fused features;​

[0101] In this embodiment, as shown in Figure 4 , first, the feature maps from the spatial and frequency domains are spliced in the channel dimension. Subsequently, information extraction and fusion are performed on the spliced features through a 1x1 convolutional layer. The AAM module uses a height-width attention mechanism, respectively performs global pooling in the height and width directions, and calculates the channel attention weights along these two dimensions. Finally, these attention weights are multiplied back into the fused feature map to extract key information. This mechanism can learn parameters according to the spatial distribution of the features and their distribution in the frequency domain, to enhance key features and suppress redundant information, thereby improving the quality of the merged features.

[0102] Step 4: The features output by the encoder and the fused features are processed by F-Ctrans skip connection, which includes Fourier transform (FFT) to convert the features to the frequency domain, learnable filter weighting, inverse Fourier transform (IFFT) to convert back to the space-time domain, and residual connection to optimize feature expression.

[0103] In this embodiment, as shown in Figure 5 , the input image is processed by the channel fusion module for convolution, flattening, and position encoding to obtain a preliminary feature representation. Then, after MLP (Multi-Layer Perceptron) and Fourier transform (FFT) processing, frequency-enhanced features are generated. A learnable filter is applied to the selected frequency patterns for weighting, and then inverse Fourier transform (IFFT) is used to convert back to the space-time domain. Then, the frequency features and the spatial features are interactively fused, and residual connection and MLP are combined to further optimize feature expression. The subsequent feature reconstruction module restores the original features through reconstruction to generate the final output. In this process, the skip connection is used to enhance the detail restoration capability of the model and ensure efficient fusion of spatial and frequency features.

[0104] where the FFT formula is as follows:

[0105]

[0106] where is the time domain function; is its frequency domain representation, and the integral covers all time ranges ; is the angular frequency (unit rad / s, describing the "fast and slow" of signal oscillation, corresponding to the "frequency feature" of the image: high frequency → edge / detail, low frequency → outline / global); is the complex exponential base function (from Euler's formula , representing "sinusoidal and cosine components with a frequency of "). In essence, the time domain signal is converted to the frequency domain signal mapping, i.e. the continuous-time Fourier transform (CTFT).

[0107] where the IFFT formula is as follows:

[0108]

[0109] where the normalization factor guarantees energy conservation (no such factor for forward transform, complemented by inverse transform, so that holds). This formula realizes time-domain reconstruction of the signal through frequency-domain integration and complex exponential function, which is essentially the inverse continuous-time Fourier transform (Inverse CTFT) for reducing the frequency-domain signal to the time-domain signal f(t).

[0110] In this embodiment, as shown in Figure 6 , it can be seen from the F-Ctrans channel cross-attention structure that F-Ctrans globally pools the feature maps from the decoder and the skip connection, then respectively inputs the two pooled information into two independent multi-layer perceptrons to obtain attention weight information, and finally weights the skip connection feature map with the obtained attention weight information, and fuses the weighted feature map with the feature map of the decoder to obtain more effective feature map information.

[0111] In this embodiment, the structure of the MLP layer is as shown in Figure 7 , which is divided into:

[0112] 1) input layer (left black node):

[0113] Receives original feature input (such as local features of pathological images, frequency domain coefficients, or feature vectors output by other modules), and the number of nodes corresponds to the dimension of the input feature (3-dimensional input in the figure).

[0114] 2) hidden layer (middle gray node):

[0115] Linear transformation of input features is performed through full connection weight + nonlinear activation (activation function is not explicitly shown in the figure, and engineering usually contains ReLU, Sigmoid, etc.), realizing feature abstraction and dimension mapping (such as the hidden layer dimension in the figure is 3, which can compress / expand the feature dimension).

[0116] 3) output layer (right gray node):

[0117] Further transform the output of the hidden layer to output the target result (such as the class probability of pathological image segmentation, the output vector after feature fusion, and the output in the figure is 2-dimensional).

[0118] Step 5: The features transmitted by the F-Ctrans skip connection are processed by a decoder, which restores spatial information by using an up-sampling operator and adaptively adjusts channel features by an SE (Squeeze-and-Excitation) layer, and outputs the segmentation result of the colorectal cancer pathological image.

[0119] The SE (Squeeze-and-Excitation) layer can be referred to as an adaptive channel attention module, which first compresses the spatial dimension by global average pooling (Squeeze), then learns the channel weight through a fully connected layer (Excitation), enhances the key channel feature, is deployed in the decoder, adaptively adjusts the channel feature weight, and improves the detail performance of the segmentation result.

[0120] In order to verify the technical effect of the present application, the Monte Carlo cross-validation (MMCV) is performed on the baseline model and the FD-TransUnet on the Glas (containing colorectal adenocarcinoma pathological sections, labeling tumor regions, used for model training and testing) and CRAG (covering more pathological variations (such as different differentiation degrees, interstitial interference), used for verifying the generalization ability of the model) data sets, 10 independent random data set partitioning and training and evaluation are performed on each model, and the experimental data of the baseline model and the FD-TransUnet are shown in Table 1 and Table 2:

[0121] Table 1 Comparison test data of Glas data set

[0122]

[0123] Table 2 Comparison test data of CRAG

[0124]

[0125] It can be seen that the FD-TransUnet of the present application is superior to the baseline model TransUnet in the evaluation indexes such as Dice and HD95, the Dice coefficient is improved by 0.0266, which proves that the FD-TransUnet is better in overall segmentation, the HD95 is reduced by 10.53, which indicates that the FD-TransUnet also performs better in boundary segmentation, and the boundary prediction is closer to the true boundary.

[0126] As shown in Figure 8 The effect diagram of the FD-TransUnet of the present application, the traditional doctor cutting and the existing TransUnet cutting is shown, wherein:

[0127] (a) Physician segmentation (gold standard): Green areas are manually annotated colorectal cancer lesions (e.g. tumor regions), representing the "ideal segmentation result" for evaluating model accuracy.

[0128] (b) TransUnet segmentation: Blue areas are the segmentation results of the baseline model (TransUnet), reflecting the limitations of traditional methods.

[0129] (c) FD-TransUnet segmentation: Blue areas are the segmentation results of the improved model of the present application, embodying the optimization effect of the technical solution.

[0130] It can be seen that the segmentation results of the FD-TransUnet of the present application are closer to the gold standard in boundary accuracy (HD95 index) and regional consistency (Dice coefficient), verifying the effectiveness of the following technical innovations:

[0131] Cross-domain feature fusion of frequency domain and spatial domain (to make up for the information loss of single domain features);

[0132] Enhancement capability of frequency domain optimization module (MFRM, F-Ctrans) on details and boundaries;

[0133] Focusing effect of attention mechanism on key features.

[0134] This group of figures proves from an intuitive level that the present application effectively solves the core problems of boundary blurring and incomplete regions in traditional models in pathological image segmentation through the technical solution of "frequency domain enhancement + cross-domain fusion", providing a more accurate automated tool for clinical pathological analysis.

[0135] Example Two

[0136] Based on the same idea, the present application also proposes a colorectal cancer pathological image segmentation device based on frequency domain features, comprising:

[0137] An input module for obtaining a colorectal cancer pathological original input image;

[0138] A multi-scale frequency reconstruction module for processing the original input image to extract high-frequency components and low-frequency components in the frequency domain, wherein the high-frequency components correspond to the details and edge information of the image, and the low-frequency components correspond to the overall contour information of the image;

[0139] An encoder module for spatial feature extraction of the original input image, the encoder module combining convolutional neural networks and Vision Transformer (VIT) to output spatial features containing global context information;

[0140] an attention aggregation module configured to fuse the frequency features output by the multi-scale frequency reconstruction module and the spatial features output by the encoder module, the fusion process including channel concatenation, 1x1 convolution processing, and height-width attention mechanism weighting, to obtain fused features;

[0141] an F-Ctrans skip connection module configured to perform transfer processing on the features output by the encoder module and the fused features, the transfer processing including Fourier transform to convert the features to the frequency domain, learnable filter weighting, inverse Fourier transform to convert back to the space-time domain, and residual connection to optimize feature expression;

[0142] a decoder module configured to process the features transferred by the F-Ctrans skip connection module, recover spatial information using an up-sampling operator, and adjust channel features adaptively through an SE (Squeeze-and-Excitation) layer, to output a segmentation result of the colorectal cancer pathological image.

[0143] Further, the multi-scale frequency reconstruction module includes:

[0144] a block unit configured to perform block processing on the original input image;

[0145] a DCT processing unit configured to extract frequency features from each image block through two-dimensional discrete cosine transform;

[0146] a reorganization unit configured to reorganize the frequency features output by the DCT processing unit to obtain a frequency feature map containing high-frequency components and low-frequency components;

[0147] Further, the attention aggregation module includes:

[0148] a channel concatenation unit configured to concatenate the frequency features and the spatial features in the channel dimension;

[0149] a convolution fusion unit configured to extract and fuse information from the concatenated features through a 1x1 convolution layer;

[0150] an attention weighting unit configured to use a height-width attention mechanism to perform global pooling along the height and width directions and calculate channel attention weights, and multiply the weights back to the fused features to enhance key information.

[0151] Further, the F-Ctrans skip connection module includes:

[0152] a preliminary processing unit configured to perform channel fusion, flattening, and position encoding on the input features;

[0153] a frequency domain enhancement unit configured to convert the features to the frequency domain through a multi-layer perceptron and Fourier transform, weight selected frequency patterns using a learnable filter, and convert back to the space-time domain through inverse Fourier transform;

[0154] a fusion optimization unit, configured to interactively fuse the frequency domain enhanced features and the spatial features, and optimize feature expression by combining a residual connection and a multi-layer perception;

[0155] a feature reconstruction unit, configured to restore original feature dimensions by a reconstruction operation, and generate output features;

[0156] Further, the decoder module comprises:

[0157] an up-sampling unit, configured to restore spatial information by using a Dysample up-sampling operator;

[0158] an SE processing unit, configured to adaptively adjust weights of channel features by an SE layer operation;

[0159] an output unit, configured to output a segmentation result of the colorectal cancer pathological image based on the processed features.

[0160] Embodiment Three

[0161] The embodiment also provides an electronic device, referring to Figure 9 , comprising a memory 404 and a processor 402, the memory 404 storing a computer program, and the processor 402 being configured to run the computer program to perform the steps in any of the above method embodiments.

[0162] Specifically, the processor 402 can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0163] The memory 404 can include a mass storage that stores data or instructions. For example, and without limitation, the memory 404 can include a Hard Disk Drive (HDD), a floppy disk drive, a Solid State Drive (SSD), a flash drive, a Compact Disc Read Only Memory (CD-ROM), a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. The memory 404 can be removable and / or non-removable (or fixed) as appropriate. The memory 404 can be internal or external as appropriate. In particular embodiments, the memory 404 is a Non-Volatile memory. In particular embodiments, the memory 404 includes a Read-Only Memory (ROM) and a Random Access Memory (RAM). The ROM can be a mask-programmed ROM, a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), an Electrically Alterable ROM (EAROM), or a FLASH memory, or a combination of two or more of these, as appropriate. The RAM can be a Static Random-Access Memory (SRAM) or a Dynamic Random Access Memory (DRAM), which can be a Fast Page Mode Dynamic Random Access Memory (FPMDRAM), an Extended Data Output Dynamic Random Access Memory (EDODRAM), a Synchronous Dynamic Random-Access Memory (SDRAM), or the like, as appropriate.

[0164] The memory 404 can be used to store or buffer various data files needed for processing and / or communication, and possible computer program instructions executed by the processor 402.

[0165] The processor 402 implements any one of the above-mentioned frequency domain feature based colorectal cancer pathological image segmentation methods by reading and executing the computer program instructions stored in the memory 404.

[0166] Optionally, the above-mentioned electronic device can further comprise a transmission device 406 connected with the processor 402 and an input / output device 408 connected with the processor 402.

[0167] The transmission device 406 can be used to receive or send data via a network. The above-mentioned network can include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device comprises a network adapter (NIC) which can be connected with other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (RF) module which is used to communicate with the Internet in a wireless manner.

[0168] The input / output device 408 is used to input or output information.

[0169] Embodiment Four

[0170] The embodiment also provides a readable storage medium, and the readable storage medium stores a computer program. The computer program comprises program codes for controlling a process to execute the process. The process comprises the frequency domain feature based colorectal cancer pathological image segmentation method according to Embodiment One.

[0171] It should be noted that the specific examples in the embodiment can refer to the examples described in the above-mentioned embodiments and optional implementation manners, and the embodiment will not be described here again.

[0172] Generally, various embodiments can be implemented in hardware or special-purpose circuitry, software, logic or any combination thereof. Some aspects of the application can be implemented in hardware, while other aspects can be implemented by firmware or software executed by a controller, microprocessor or other computing device, but the application is not limited thereto. Although various aspects of the application can be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein can be implemented in hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

[0173] Embodiments of the application can be implemented by computer software executable by a data processor of the mobile device such as in the processor entity, or by hardware, or by a combination of software and hardware. Computer software or program, also called program product, including software routines, applets and / or macros, can be stored in any apparatus-readable data storage medium and they include program instructions to implement certain tasks. The program product can include one or more computer-executable components tangibly embodied in a computer- readable medium, such as one or more of the computer-executable components mentioned above. The one or more computer-executable components can be one or more Figure 1 Any block in the logical flow of the above described embodiments can represent a module, segment, or portion of code which comprises one or more executable instructions implemented in digital electronic or optical equipment or in tangible installation of a microprocessor or microcontroller, or in one or more computer programs suitable for use with or by an instruction execution system, apparatus or device. Such equipment or devices can include a processor of any type used in mobile devices, or general purpose computers, or other programmable processor configurations to produce the required tasks of the embodiments. The processes described with respect to the embodiments can be embodied in a system, apparatus, computer program, or computer program product. Program code means or computer program or program or computer programs, also called program product, including one or more computer-executable instructions can be stored in a computer-readable or machine-readable medium (or on a data carrier), which is tangible and non-transitory, such as a memory chip, or a memory block in a processor, or a hard disk, or a floppy disk, or an optical medium, such as DVD, or a CD-ROM, or a memory on a general purpose computer, or a memory on a mobile device. The physical medium is non-transitory.

[0174] It should be understood that all technical features of the above embodiments can be combined in any manner, and for brevity, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not contradict each other, they should be considered as within the scope of the present disclosure.

[0175] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the present application. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A frequency domain feature-based colorectal cancer pathological image segmentation method, characterized in that, The method comprises the following steps: Step 1: obtaining a colorectal cancer pathology original input image, processing the original input image through a multi-scale frequency reconstruction module, and extracting high-frequency components and low-frequency components in a frequency domain, wherein the high-frequency components correspond to details and edge information of the image, and the low-frequency components correspond to overall contour information of the image; Step 2: performing spatial feature extraction on the original input image through an encoder, wherein the encoder combines a convolutional neural network and a ViT, and outputs spatial features containing global context information of the image; Step 3: fusing the frequency features output in step 1 and the spatial features output in step 2 through an attention aggregation module, wherein the fusion process comprises channel splicing, 1x1 convolution processing and height-width attention mechanism weighting, and fused features are obtained; Step 4: performing transmission processing on the features output by the encoder and the fused features through an F-Ctrans jump connection, wherein the transmission processing comprises Fourier transform for converting the features to the frequency domain, learnable filter weighting, inverse Fourier transform for converting back to the space-time domain, and residual connection for optimizing feature expression; Step 5: processing the features transmitted through the F-Ctrans jump connection through a decoder, wherein the decoder recovers spatial information by using an up-sampling operator, and adjusts channel features adaptively through an SE layer, and outputs a segmentation result of the colorectal cancer pathology image.

2. The method of claim 1, wherein the method is based on frequency domain features. In step 1, the processing process of the multi-scale frequency reconstruction module comprises: performing block processing on the original input image, extracting frequency features through two-dimensional discrete cosine transform for each image block, and recombining the frequency features to obtain a frequency feature map containing high-frequency components and low-frequency components.

3. The method of claim 1, wherein the method is based on frequency domain features. In step 3, the height-width attention mechanism comprises: performing global pooling along the height direction and the width direction respectively, calculating channel attention weights in the two directions, multiplying the attention weights back to the fused features after 1x1 convolution processing, enhancing key features and suppressing redundant information.

4. The method of claim 1, wherein the method is based on frequency domain features. In step 4, the processing of the F-Ctrans jump connection further comprises: performing channel fusion, flattening and position coding on the input features, performing Fourier transform after multi-layer perceptron processing, and recovering the original feature dimension through a feature reconstruction module after the frequency features and the spatial features are interactively fused.

5. The method of any one of claims 1 to 4, wherein the method is based on frequency domain features. In step 5, the decoder recovers spatial information by using a Dysample up-sampling operator, and the SE layer adaptively adjusts the weights of the channel features by squeezing and exciting the channel features. 6.A device for segmenting a colorectal cancer pathological image based on frequency domain features, characterized in that, The method comprises: an input module configured to obtain a colorectal cancer pathology original input image; a multi-scale frequency reconstruction module configured to process the original input image and extract high-frequency components and low-frequency components in a frequency domain, wherein the high-frequency components correspond to details and edge information of the image, and the low-frequency components correspond to overall contour information of the image; an encoder module configured to perform spatial feature extraction on the original input image, wherein the encoder module combines a convolutional neural network and a ViT, and outputs spatial features containing global context information of the image; an attention aggregation module configured to fuse the frequency features output by the multi-scale frequency reconstruction module and the spatial features output by the encoder module, the fusion process including channel concatenation, 1x1 convolution processing, and height-width attention mechanism weighting, to obtain fused features; an F-Ctrans skip connection module configured to perform transfer processing on the features output by the encoder module and the fused features, the transfer processing including Fourier transform to convert the features to the frequency domain, learnable filter weighting, inverse Fourier transform to convert back to the spatial domain, and residual connection to optimize feature expression; a decoder module configured to process the features transferred by the F-Ctrans skip connection module, recover spatial information using an up-sampling operator, and adjust the weights of the channel features through an SE layer, and output a segmentation result of the colorectal cancer pathology image.

7. The frequency-domain feature-based colorectal cancer pathology image segmentation apparatus of claim 6, wherein The multi-scale frequency reconstruction module includes: a block unit configured to perform block processing on the original input image; a DCT processing unit configured to extract frequency features from each image block through two-dimensional discrete cosine transform; a reorganization unit configured to reorganize the frequency features output by the DCT processing unit to obtain a frequency feature map containing high-frequency components and low-frequency components; The attention aggregation module includes: a channel concatenation unit configured to concatenate the frequency features and the spatial features in the channel dimension; a convolution fusion unit configured to extract and fuse information from the concatenated features through a 1x1 convolution layer; an attention weighting unit configured to use a height-width attention mechanism to perform global pooling along the height and width directions and calculate channel attention weights, and multiply the weights back into the fused features to enhance key information.

8. The frequency-domain feature-based colorectal cancer pathology image segmentation apparatus of claim 6, wherein, The F-Ctrans skip connection module includes: a preliminary processing unit configured to perform channel fusion, flattening, and position encoding on the input features; a frequency domain enhancement unit configured to convert the features to the frequency domain through a multi-layer perceptron and a Fourier transform, weight selected frequency patterns using a learnable filter, and convert back to the spatial domain through an inverse Fourier transform; a fusion optimization unit configured to interactively fuse the frequency domain enhanced features and the spatial features, and combine residual connection and multi-layer perceptron to optimize feature expression; a feature reconstruction unit configured to recover the original feature dimension through a reconstruction operation to generate output features; The decoder module includes: an up-sampling unit configured to recover spatial information using a Dysample up-sampling operator; an SE processing unit configured to adaptively adjust the weights of the channel features through an SE layer operation; an output unit configured to output a segmentation result of the colorectal cancer pathology image based on the processed features. 9.An electronic device comprising a memory and a processor, the electronic device characterized by, The memory stores a computer program, and the processor is configured to run the computer program to execute the frequency domain feature based colorectal cancer pathology image segmentation method of any one of claims 1-5.

10. A readable storage medium, characterized by, The readable storage medium stores a computer program, and the computer program includes program code for controlling a process to execute the process, and the process includes the frequency domain feature based colorectal cancer pathology image segmentation method of any one of claims 1-5.

Images