Methods, systems, and devices for multi-organ segmentation in medical images based on frequency domain sensing

Abstract

This invention belongs to the field of medical detection, specifically relating to a method, system, and device for multi-organ segmentation in medical images using frequency domain perception. The invention designs a U-shaped network with PVT modules as the initial backbone. The encoder in the U-shaped network uses a four-layer PVT module for downsampling, and the decoder uses a four-layer PE module for upsampling. In addition to the four-layer PVT module branches in the encoder, a feature fusion branch consisting of a five-layer TAF module is added. The TAF module is used to extract frequency domain features and fuse them with spatial features. A three-layer SRA module is added to the decoder. The SRA module adds relative position encoding to the upsampling results between adjacent PE modules, and then uses two layers of attention to calculate refined features. Finally, the newly designed network model is trained using multi-organ medical image samples to obtain a high-precision multi-organ segmentation model. This invention solves the problem of insufficient accuracy of existing image segmentation techniques in multi-organ segmentation scenarios.

Description

Technical Field

[0001] This invention belongs to the field of medical testing, specifically relating to a multi-organ segmentation method in medical images based on frequency domain perception, and a corresponding multi-organ segmentation system and device based on frequency domain perception. Background Technology

[0002] With advancements in medical technology, various advanced medical diagnostic equipment and instruments enable doctors to conduct in-depth examinations of patients, uncovering many early-stage diseases that cannot be diagnosed manually. This development of new technologies aligns with people's growing health awareness and needs. Currently, medical images acquired using various medical devices have become a primary means for doctors to conduct auxiliary diagnosis and pre- and post-operative assessments.

[0003] In medical image-based diagnosis, to save doctors' time and effort, computer-aided analysis is performed first. Medical image segmentation is a crucial step in this process. Medical image segmentation aims to accurately distinguish each type of organ from medical images containing multiple organs (such as images of abdominal and thoracic examinations) so that further targeted examination and analysis of each organ can be conducted. The accuracy of organ segmentation in medical images lays a solid foundation for subsequent analysis and treatment.

[0004] Medical image segmentation is an intensive prediction task that classifies pixels of organs or lesions in given medical images (such as CT, MRI, endoscopy, OCT, etc.). U-Net and its variants are widely used in medical image segmentation tasks due to their superior performance and sophisticated architecture. Multi-organ segmentation in medical images has always been a challenging task. Boundary textures between organs or between organs and surrounding tissues are difficult to distinguish, and their size and shape vary significantly from individual to individual, often leading to misclassification or misidentification of similar organs. U-Net is an encoder-decoder architecture where features from the encoder are aggregated with upsampled features from the decoder to produce a high-resolution segmentation map. With the development of computer vision, the ResNet architecture has been widely adopted as the backbone of medical image segmentation. Pyramid pooling and dilated convolutions are also used for lesion and organ segmentation.

[0005] Building on this, Transformer-based methods have also achieved great success in medical image segmentation. Therefore, various combinations of convolution and Transformer have been developed. For example, TransUNet uses a hybrid CNN-transformer encoder to capture long-range dependencies and a cascaded CNN upsampler as a decoder to capture local contextual relationships between pixels. Another example is Swin-Unet, a pure transformer architecture based on the Swin Transformer. However, Swin-Unet uses transformers in both the encoder and decoder, which does not lead to performance improvements.

[0006] However, existing methods all rely on spatial features such as size, color, and texture in medical images for organ classification and image segmentation. This makes the accuracy of image segmentation highly susceptible to the quality of the medical images. Furthermore, the boundary textures between organs or between organs and surrounding tissues are difficult to distinguish, varying in size and shape, leading to widespread misclassification and omissions. For example, Figure 1 This paper presents some typical segmentation cases on abdominal multi-organ medical images from the Synapse dataset. In abdominal multi-organ segmentation, the left and right kidneys are extremely similar in shape, making it easy to misidentify them using some popular image segmentation methods (i.e., mistaking the left kidney for the right kidney, and vice versa). Furthermore, differences in physiological characteristics among patients can also affect the accuracy of the final segmentation results. For example, organ changes due to disease or surgery can significantly reduce the accuracy of organ segmentation. Summary of the Invention

[0007] To address the issue of insufficient accuracy in multi-organ segmentation scenarios using existing medical image segmentation techniques, this invention provides a multi-organ segmentation method for medical images that incorporates frequency domain perception, as well as a corresponding multi-organ segmentation system and device that incorporates frequency domain perception.

[0008] This invention is achieved using the following technical solution:

[0009] This embodiment provides a multi-organ segmentation method for medical images based on frequency domain awareness, used for identifying and segmenting individual organs in medical images containing multiple organs. It includes the following steps:

[0010] S1: Construct an image segmentation network that integrates image features and frequency domain awareness for segmenting multiple organs in medical images. The process is as follows:

[0011] S11: Design a U-shaped network that uses PVT modules as the initial backbone. The encoder in the U-shaped network uses four layers of PVT modules for downsampling, and the decoder uses four layers of PE modules for upsampling.

[0012] S12: In addition to the four-layer PVT module branch in the encoder, a feature fusion branch consisting of a five-layer TAF module is added. The TAF module is used to extract frequency domain features and fuse them with spatial features.

[0013] The improved encoder cross-fuses the spatial features extracted by the PVT module of each layer with the frequency domain features extracted by the TAF module of the previous layer, and then passes them to the next TAF layer after n iterations.

[0014] S13: Add 3 SRA modules to the decoder. The SRA modules are used to add relative position encoding to the upsampling results between adjacent PE modules, and then calculate the refined features through two layers of attention.

[0015] The outputs of each stage of the PE module and TAF module in the improved decoder are cascaded on the channel via skip connections, and then input into the SRA module of the corresponding scale for processing; the SRA module is used to introduce relative position encoding and semantically guided relative position embedding on the basis of traditional attention computation.

[0016] S2: Obtain a large number of medical image samples containing multiple organs, manually label the organs in each medical image sample to form the required dataset; and divide the dataset into training set and test set.

[0017] S3: Construct a multi-organ segmentation model to be trained using an image segmentation network.

[0018] The input to the multi-organ segmentation model is the original medical image. This image is divided into two branches: one branch is processed by the Stem module and then input into the TAF module of the first layer of the image segmentation network; the other branch undergoes cropping, convolution, and normalization before being input into the PVT module of the first layer of the image segmentation network. The output of the SRA module in the last layer of the decoder of the image segmentation network is upsampled by the PE module, then processed by the classification head to form a classification mask, and finally outputs the generated organ segmentation result.

[0019] S4: A loss function including cross-input loss and pixel-level constraint loss is used as the loss function L during the training phase, and the multi-organ segmentation model is trained and tested using training set, test set and validation set.

[0020] S5: Save the model parameters of the trained and validated best multi-organ segmentation model, and use the multi-organ segmentation model to process medical images to identify and segment various organs in the medical images.

[0021] As a further improvement of the present invention, the five TAF modules in the image segmentation network constructed in step S1 are TAF1 to TAF5, and the data processing process in TAF2 to TAF5 modules is as follows:

[0022] (1) The fusion features extracted from the previous layer TAF are obtained from the TAF. i-1 The circuit is split into two paths and then subjected to 3×3 convolution.

[0023] (2) The spatial features PVT extracted from the previous layer TAF i-1 It is divided into two paths: one path is processed by 3×3 convolution, and the other path is processed by the Spectral Transform module.

[0024] (3) The TAF after 3×3 convolution i-1 and PVT i-1 The summation, followed by LN normalization, yields the frequency domain feature M. i-1 .

[0025] (4) The TAF after 3×3 convolution i-1 PVT processed by the Spectral Transform module i-1 Add them together; then, after LN standardization, obtain the spatial feature T. i-1 .

[0026] (5) From the feature TAF i-1 To feature M i-1 and from the characteristic PVT i-1 To feature T i-1 The two-way feature transformation process is iterated n times, and then the results are channel-connected;

[0027] (6) Input the features obtained after channel concatenation in the previous step into the feedforward network FFN for processing to obtain the fused features TAF required for the next layer. i .

[0028] The first layer TAF1 module uses the same data processing strategy as TAF2 to TAF5 modules. The difference is that the features output by the STEM module are divided into two equal parts on the channel, and then the two parts are convolved and used as the two raw inputs required by the TAF1 module.

[0029] As a further improvement of the present invention, the expression for the data processing procedure in the TAF module is as follows:

[0030]

[0031] In the above formula, the subscript j represents the iteration round, and X i,j and X i,j+1 These represent the previous TAF layer before and after the iteration, respectively.i-1 Module fusion characteristics; Y i,j and Y i,j+1 These represent the previous PVT layer before and after the iteration, respectively. i-1 Spatial characteristics of modules; TAF i This represents the fusion feature output by the TAF module of the current layer; Indicates channel connections between features; Conv3×3 represents convolution processing; Spect represents the spectral transformation processing of the Spectral Transform module; ReLU and GELU represent the corresponding activation functions; LN represents LayerNorm normalization, DW conv This represents the processing of the feedforward network (FFN); X i,n Y represents the fused features after n iterations; i,n This represents the spatial features after n iterations.

[0032] As a further improvement of the present invention, the Spectral Transform module includes, in sequence, a Fast Fourier Transform layer, a Convolutional Normalized Activation Function layer, and an Inverse Fast Fourier Transform layer; in the Spectral Transform module, the original feature image is first transformed from the spatial domain to the frequency domain, then the weights are updated in the frequency domain, and finally the frequency domain features with updated weights are transformed back to the spatial domain.

[0033] As a further improvement of this invention, the scaling transformation process of image features in the three different functional layers of the Spectral Transform module is as follows:

[0034]

[0035] In the above formula, R H×W×C The input image features are represented by H, W, and C, which represent the length, width, and dimension of the feature image, respectively; RealFFT2d represents the Fast Fourier Transform layer; ReLU represents the activation function; BN represents batch normalization; Conv 1×1 This indicates a convolution operation with a kernel of 1; InverseRealFFT2d indicates an inverse fast Fourier transform layer.

[0036] As a further improvement of the present invention, the three SRA modules in the image segmentation network constructed in step S1 are SRA1 to SRA3, and each SRA module consists of a position index unit for extracting relative position encoding and relative position embedding, and a self-attention mechanism; the data processing procedure of the SRA module is as follows:

[0037] (1) The original feature vector is linearly transformed to the QKV space through the self-attention mechanism to obtain the query vector Q, the key vector K and the value vector V.

[0038] (2) Based on the absolute position of the pixel in the coordinate layer in the original multi-scale feature image, a two-dimensional relative position code R is extracted by the position indexing unit. pos and two corresponding one-dimensional relative position indices R in the height and width directions h and R w .

[0039] (3) Using relative position index R h and R w And the query vector Q, the semantically guided relative position embeddings in the corresponding height and width directions are calculated using the following formula.

[0040]

[0041] (4) Embedding based on two relative positions and A one-dimensional relative position offset S is calculated using the following formula. pos :

[0042]

[0043] (5) Encode the obtained relative position R pos relative position embedding with semantic guidance Treating them as relative positional information and incorporating them into QK in the self-attention mechanism T In Chinese, the expression is as follows:

[0044]

[0045] In the above formula, d is a parameter representing the scaling ratio.

[0046] As a further improvement of the present invention, the expression for the loss function L in step S4 is as follows:

[0047]

[0048] In the above formula, p represents the label value in the segmentation result. Represents artificial labels in the original image; ce This represents the cross-entropy loss based on pixel-level constraints; Dice represents the sample similarity loss used for semantic segmentation; β and γ represent the weights of the cross-entropy loss and sample similarity loss, respectively.

[0049] As a further improvement of the present invention, in step S11, the four-layer PVT module in the encoder adopts a Transformer with a pyramid structure. Each layer PVT module performs downsampling processing on the input image and calculates attention.

[0050] The four-layer PE module in the decoder adopts the Patch Expanding Block in Transformer. Each PE module upsamples the fused features input from the TAF module of the next layer. The upsampling result is input into the SRA module for processing, and the result processed by the SRA module is then passed to the PE module of the previous layer.

[0051] and / or

[0052] The iteration count for both the frequency domain features and spatial features in the TAF module is set to 2.

[0053] This invention also includes a multi-organ segmentation system that integrates frequency domain perception. This system employs the aforementioned multi-organ segmentation method for medical images using integrated frequency domain perception to identify and segment individual organs in a medical image containing multiple organs. The multi-organ segmentation system includes: an image acquisition unit, a preprocessing unit, an image segmentation network, and a classification module.

[0054] The image acquisition unit is used to acquire raw medical images containing multiple organs to be segmented.

[0055] The preprocessing unit includes a first preprocessing subunit and a second preprocessing subunit. The first preprocessing subunit uses the Stem module to downsample the input raw medical image by 1 / 2 after a convolution with a stride of 2, followed by a 1*1 convolution branch and a pooling branch, and then concatenates the outputs of the two branches to obtain a 1 / 2 feature map. The second preprocessing subunit uses a common Transformer preprocessing method. First, the original input image is divided into patches according to a predetermined size. Then, based on the patches obtained after segmentation, the total number is calculated. The convolution kernel size is set according to the total number, and the original input image is convolved. After layer normalization, a 1 / 4 feature map is obtained.

[0056] The image segmentation network employs the image segmentation network described above in the multi-organ segmentation method for medical images with fusion frequency domain awareness. The image segmentation network consists of 4 PVT modules, 5 TAF modules, 4 PE modules, and 3 SRA modules, forming a U-shaped network with a dual-branch encoder. The 4 PVT modules constitute the spatial domain feature extraction branch of the decoder backbone network. The 5 TAF modules constitute the frequency domain feature extraction branch of the decoder backbone network, achieving fusion of spatial and frequency domain features at each scale. The 4 PE modules constitute the decoder; the feature images passed between every two PE modules undergo semantic relative position embedding based on relative position encoding and attention calculation via an SRA module. The preprocessed features output from the first and second preprocessing subunits are input into the first TAF module and the first PVT module of the image segmentation network, respectively.

[0057] The classification module is used to obtain the upsampled result of the last PE module in the image segmentation network, and form a classification mask through the classification head, thereby generating a medical image after image segmentation according to organs.

[0058] The present invention also includes a multi-organ segmentation device incorporating frequency domain perception, comprising a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it creates the multi-organ segmentation system incorporating frequency domain perception as described above, thereby realizing target recognition and image segmentation of the input raw medical image, and outputting a medical image segmented according to organs.

[0059] The technical solution provided by this invention has the following beneficial effects:

[0060] This invention proposes a novel multi-organ segmentation network based on the fusion of frequency and spatial domains. The network model employs five multi-scale Transient Aspect-Oriented Functions (TAFs) as newly added frequency domain feature extraction branches in the encoder. The extracted frequency domain features are fused with the spatial domain features extracted by the pre-trained PVT module through the TAF module. Convolutional layers are also used in the TAF module to better learn local information, thereby enabling the encoder to learn both local and global information simultaneously.

[0061] To compensate for the loss of frequency domain position and consider the relative positional relationships between organs, this invention also uses three SRA (Semantic Relative Position Attention) modules at the decoder. This introduces relative position encoding and semantically guided relative position embedding on top of traditional attention computation.

[0062] This invention, by fusing frequency and spatial domain features and introducing corresponding positional coding, ultimately achieves accurate segmentation of multi-organ medical images with rich texture details, overcoming the shortcomings of existing solutions. Attached Figure Description

[0063] Figure 1 This section presents a case study of the failure of traditional methods described in the background section for abdominal multi-organ segmentation tasks.

[0064] Figure 2 This is a schematic diagram illustrating the principle of spatial domain and frequency fusion feature distribution used in Embodiment 1 of the present invention.

[0065] Figure 3 The overall architecture diagram of the image segmentation network that integrates image features and frequency domain awareness designed in Embodiment 1 of this invention.

[0066] Figure 4 This is a schematic diagram of the TAF module for feature fusion designed in Embodiment 1 of the present invention.

[0067] Figure 5 for Figure 3 The signal flow graph of the first layer TAF module in the network model.

[0068] Figure 6 This is a schematic diagram of the Spectral Transform module used in the TAF module.

[0069] Figure 7 This is a schematic diagram of the feedforward network FFN used in the TAF module.

[0070] Figure 8 This is a schematic diagram illustrating the principle of relative position encoding and embedding in the SRA model of the present invention.

[0071] Figure 9 This is a schematic diagram of the SRA model designed in Embodiment 1 of the present invention.

[0072] Figure 10 Examples of multi-organ segmentation schemes obtained from simulation experiments. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0074] Example 1

[0075] To address the issue of difficulty in distinguishing boundary textures from a spatial domain perspective, the overall approach of this embodiment is as follows: Figure 2As shown, the image is first transformed from the spatial domain to the frequency domain using image processing methods, and then convolution processing is performed on the image in the frequency domain. Then, the frequency and spatial domain information are fused in a neural network to achieve image segmentation. To ensure the robustness of frequency domain segmentation, this embodiment processes the input image in two branches: one branch uses a pyramid-structured Transformer, and the other branch uses a Fourier convolution structure formed by combining frequency domain and convolution. During the transformation from the spatial domain to the frequency domain, some loss of positional information occurs, while multiple organs in multi-organ medical images have relative positional relationships. Therefore, this embodiment supplements this positional information at the network model decoding end by employing relative positional encoding and strengthening the correlation of positional information through attention calculation.

[0076] Specifically, this embodiment provides a method for multi-organ segmentation in medical images that integrates frequency domain awareness, which includes the following steps:

[0077] S1: Construct an image segmentation network that integrates image features and frequency domain awareness for segmenting multiple organs in medical images. The process is as follows:

[0078] S11: Design a U-shaped network that uses PVT modules as the initial backbone. The encoder in the U-shaped network uses four layers of PVT modules for downsampling, and the decoder uses four layers of PE modules for upsampling.

[0079] Medical image segmentation is a dense prediction task used to classify organs or lesions in given medical images (such as CT, MRI, endoscopy, OCT, etc.) at the pixel level. U-Net and its variants are widely used in medical image segmentation due to their strong performance and complex architecture. This embodiment borrows the encoder-decoder architecture from U-Net, selecting a network similar to Swin-UNet as the base network, and replacing the encoder from the Swin Transformer block with a PVT (Pyramid Vision Transformer) module. The decoder part of the designed U-shaped network still uses the PatchExpanding Block from the Swin-UNet network. Both the encoder and decoder parts are four layers long, used to acquire deep features at different scales.

[0080] S12: In addition to the four-layer PVT module branch in the encoder, a feature fusion branch consisting of a five-layer TAF module is added. The TAF module is used to extract frequency domain features and fuse them with spatial features.

[0081] To overcome the problem of insufficient spatial domain feature information in multi-organ medical images, which makes it impossible to achieve accurate classification, this embodiment adds a feature extraction branch to the encoder to capture rich global information through frequency domain updates. Then, by combining the local branch formed by convolution and the global branch formed by Fourier transform, local details are enhanced while focusing on the global signal.

[0082] Specifically, such as Figure 3 As shown, the improved encoder includes a spatial domain feature extraction branch consisting of 4 PVT layers, and a frequency domain feature extraction and feature fusion branch consisting of 5 TAF (Transformer Attended Fourier Convolution Block) modules. In the decoder, each layer's TAF module cross-fuses the spatial features extracted by the PVT module of each layer with the frequency domain features extracted by the TAF module of the previous layer, and then passes this fusion to the next TAF layer after n iterations.

[0083] The TAF module is a newly designed feature extraction module in this embodiment, combined with... Figure 3 As can be seen, the constructed image segmentation network consists of five TAF modules, namely TAF1 to TAF5. Since the number of layers in the TAF modules is greater than that in the PVT modules, the data transmission paths of the TAF modules in different layers differ between the two branches. The five TAF modules can be divided into two categories: one is the later TAF2 to TAF5, whose inputs are the outputs of the TAF modules and PVT modules in the previous layer, respectively; the other is the earlier TAF1, which has no PVT module in the previous layer, so its two required inputs are the preprocessed original image.

[0084] Specifically, the data processing procedures in the TAF2 to TAF5 modules of the first type are as follows: Figure 4 As shown, the process includes the following:

[0085] (1) The fusion features extracted from the previous layer TAF are obtained from the TAF. i-1 The circuit is split into two paths and then subjected to 3×3 convolution.

[0086] (2) The spatial features PVT extracted from the previous layer TAF i-1 It is divided into two paths: one path is processed by 3×3 convolution, and the other path is processed by the Spectral Transform module.

[0087] (3) The TAF after 3×3 convolution i-1 and PVT i-1 The summation, followed by LN normalization, yields the frequency domain feature M. i-1 .

[0088] (4) The TAF after 3×3 convolution i-1 PVT processed by the Spectral Transform module i-1 Add them together; then, after LN standardization, obtain the spatial feature T. i-1 .

[0089] (5) From the feature TAF i-1 To feature M i-1 and from the characteristic PVT i-1 To feature T i-1 The two-way feature transformation process is iterated n times, and then the results are channel-connected.

[0090] (6) Input the features obtained after channel concatenation in the previous step into the feedforward network FFN for processing to obtain the fused features TAF required for the next layer. i .

[0091] The data processing procedures implemented in the TAF2~TAF5 modules can be expressed using the following expressions:

[0092]

[0093] In the above formula, the subscript j represents the iteration round, and X i,j and X i,j+1 These represent the previous TAF layer before and after the iteration, respectively. i-1 Module fusion characteristics; Y i,j and Y i,j+1 These represent the previous PVT layer before and after the iteration, respectively. i-1 Spatial characteristics of modules; TAF i Represents the fused features output by the TAF module of the current layer; Conv3×3 represents convolution processing; Spect represents the spectral transformation processing of the Spectral Transform module; ReLU and GELU represent the corresponding activation functions; LN represents LayerNorm normalization, DW conv This represents the processing of the feedforward network (FFN); X i,n Y represents the fused features after n iterations; i,n This represents the spatial features after n iterations; This represents the channel connection between features. In this embodiment, the iteration round is set to 2.

[0094] The signal flow diagram of the first-layer TAF1 module is as follows: Figure 5 As shown, comparison Figure 4 and Figure 5It can be seen that the TAF1 module actually adopts the same data processing strategy as the TAF2-TAF5 modules, the only difference being the source of the input data for TAF1 and TAF2-TAF5. Specifically, before the TAF1 module, the STEM module first divides the input raw feature image into two equal parts in terms of channels, and then uses the features obtained from the convolution of the two feature maps as the two raw inputs required by the TAF1 module. The two input data for TAF2-TAF5 come from the previous layer's PVT module and TAF module.

[0095] In the newly designed encoder with dual branches, the Spectral Transform module is used in the spatial domain feature extraction path based on the PVT module. The structure of this module is as follows: Figure 6 As shown, it includes, in sequence, a Fast Fourier Transform layer (RealFFT2d), a Convolutional Normalized Activation Function layer (ReLU·BN·Conv), and so on. 1×1 :) and the Inverse Fast Fourier Transform layer (InverseRealFFT2d). In the Spectral Transform module, the original feature image is first transformed from the spatial domain to the frequency domain, then the weights are updated in the frequency domain, and finally the weighted frequency domain features are transformed back to the spatial domain.

[0096] According to the spectral convolution theorem in Fourier theory, points in image space are transformed into the frequency domain through Fourier transform, and their weights are updated in the frequency domain. In the Fourier transform, the spatial location formation of the image is not considered; instead, global points are converted into frequencies measured by amplitude. Therefore, rich global information can be added to the updated points in the frequency domain.

[0097] Specifically, in the decoder of this embodiment, the spatial domain features extracted by the PVT module undergo the following scaling process in the three different functional layers as they are iterated through the SpectralTransform module:

[0098]

[0099] In the above formula, R H×W×C The input image features are represented by H, W, and C, which represent the length, width, and dimension of the feature image, respectively; RealFFT2d represents the Fast Fourier Transform layer; ReLU represents the activation function; BN represents batch normalization; Conv 1×1 This indicates a convolution operation with a kernel of 1; InverseRealFFT2d indicates an inverse fast Fourier transform layer.

[0100] In addition, the network structure of the feedforward network FFN in the decoder is as follows: Figure 7As shown. In FFN, the original input consists of concatenated spatial domain features and frequency domain features. The input features are processed by the Deep-Wise Conv module and then added to the original input. Finally, they are processed by layer normalization and the GELU activation function before being output.

[0101] S13: Add a 3-layer SRA (Semantic Relative Attention Block) module to the decoder. The SRA module is used to add relative position encoding to the upsampling results between adjacent PE modules, and then calculate the refined features through two layers of attention.

[0102] Combination Figure 3 It can be seen that the outputs of each stage of the PE and TAF modules in the improved decoder are cascaded on the channel via skip connections, and then input into the corresponding scale SRA module for processing. The SRA module is used to introduce relative position encoding and semantically guided relative position embedding on the basis of traditional attention computation.

[0103] In this embodiment, the SRA module and PE module belong to the decoder of the network model. At this stage, this embodiment aims to add relative position encoding calculations to the attention calculations at each stage while the network gradually upsamples to restore resolution, capturing the relative positional relationships between features. Considering that there are relative positional relationships between organs at the macroscopic level, and also between pixels at the corresponding microscopic level, the frequency layer better captures the edges of the image. This makes the addition of frequency information, to some extent, a coarse absolute positional information. This relative positional relationship is different from commonly used absolute position encoding; it is a strong prior information. The SRA module designed in this embodiment can incorporate the feature information of relative position encoding into the self-attention mechanism.

[0104] Specifically, in the newly designed SRA module of this embodiment, the overall strategy for implementing relative position encoding and corresponding attention calculation is as follows:

[0105] First, the original multi-scale features are divided into multiple patches, each containing multiple pixels. Based on the coordinates of these pixels within the patches, their absolute positions can be obtained. From these absolute positions on the coordinate plane, their two-dimensional and one-dimensional relative positions can be derived. Then, a unique relative position index is calculated. This index can be used to look up the relative position offset table and obtain the weight corresponding to each index. Finally, by combining attention calculations, relative position encoding and semantic-based relative position embedding can be obtained during the upsampling process.

[0106] In particular, to make the solution of this embodiment clearer, the following is combined with Figure 8 The two-dimensional relative position encoding R in this embodiment posand two one-dimensional relative position indices R h and R w The extraction process will be explained in detail:

[0107] First, the multi-scale features are divided into multiple patches (assuming 7×7), each containing multiple pixels (assuming 4). Based on the coordinates of these pixels within the patch, their absolute positions are obtained (as shown in step 1). Using these absolute positions on the coordinate plane, we obtain a two-dimensional (h×w) relative position index (for later relative position encoding) and a one-dimensional (h or w) relative position index (for later relative position embedding) (as shown in step 2). The relative position encoding R... pos The range is [-M+1, M+1].

[0108] To facilitate the statistical analysis of the relative positions of different elements within the same batch, we merged the relative positions of different elements within the same patch into a single matrix (e.g., ...). Figure 3 (As shown). Meanwhile, to ensure the values ​​of the matrix elements are positive and unique, M-1 is first added to the original relative position encoding. Then, all horizontal coordinates are multiplied by 2M-1 to ensure the uniqueness of the index after summing the horizontal and vertical coordinates (as shown in step 3).

[0109] Finally, row and column indices are added to obtain a unique one-dimensional index (as shown in step 4). After these steps, a unique relative position encoding is obtained. By querying the relative position offset table for each index (which will be updated during model training), the weight corresponding to each index can be obtained.

[0110] The image segmentation network constructed in this embodiment consists of three SRA modules, SRA1 to SRA3. Each SRA module comprises a position index unit for extracting relative position codes and relative position embeddings, and a self-attention mechanism. Based on the relative position codes and relative position embeddings extracted by the method described above, such as... Figure 9 As shown, the data processing procedure of the SRA module is as follows:

[0111] (1) The original feature vector is linearly transformed to the QKV space through the self-attention mechanism to obtain the query vector Q, the key vector K and the value vector V.

[0112] (2) Based on the absolute position of the pixel in the coordinate layer in the original multi-scale feature image, a two-dimensional relative position code R is extracted by the position indexing unit. pos and two corresponding one-dimensional relative position indices R in the height and width directions h and R w .

[0113] (3) Using relative position index R h and Rw And the query vector Q, the semantically guided relative position embeddings in the corresponding height and width directions are calculated using the following formula.

[0114]

[0115] (4) Embedding based on two relative positions and A one-dimensional relative position offset S is calculated using the following formula. pos :

[0116]

[0117] (5) Encode the obtained relative position R pos relative position embedding with semantic guidance Treating them as relative positional information and incorporating them into QK in the self-attention mechanism T In Chinese, the expression is as follows:

[0118]

[0119] In the above formula, d is a parameter representing the scaling ratio. In this embodiment, d is taken as the dimension of the input feature vector, which is used to control the range of attention weights in multi-head self-attention, so as to improve the stability and learning ability of the model.

[0120] S2: Obtain a large number of medical image samples containing multiple organs, manually label the organs in each medical image sample to form the required dataset; and divide the dataset into training set and test set.

[0121] S3: Construct a multi-organ segmentation model to be trained using an image segmentation network.

[0122] The input to the multi-organ segmentation model is the original medical image. This image is divided into two branches: one branch is processed by the Stem module and then input into the TAF module of the first layer of the image segmentation network; the other branch undergoes cropping, convolution, and normalization before being input into the PVT module of the first layer of the image segmentation network. The output of the SRA module in the last layer of the decoder of the image segmentation network is upsampled by the PE module, then processed by the classification head to form a classification mask, and finally outputs the generated organ segmentation result.

[0123] S4: A loss function including cross-input loss and pixel-level constraint loss is used as the loss function L during the training phase, and the multi-organ segmentation model is trained and tested using training set, test set and validation set.

[0124] In this embodiment, the expression for the loss function L is as follows:

[0125]

[0126] In the above formula, p represents the label value in the segmentation result. Represents artificial labels in the original image; ce This represents the cross-entropy loss based on pixel-level constraints; Dice represents the sample similarity loss used for semantic segmentation; β and γ represent the weights of the cross-entropy loss and sample similarity loss, respectively.

[0127] S5: Save the model parameters of the trained and validated best multi-organ segmentation model, and use the multi-organ segmentation model to process medical images to identify and segment various organs in the medical images.

[0128] Example 2

[0129] Based on the scheme in Example 1, this embodiment provides a multi-organ segmentation system that integrates frequency domain perception. It adopts the multi-organ segmentation method in medical images that integrates frequency domain perception as in Example 1 to identify and segment each organ in a medical image containing multiple organs.

[0130] The multi-organ segmentation system provided in this embodiment includes: an image acquisition unit, a preprocessing unit, an image segmentation network, and a classification module.

[0131] The image acquisition unit is used to acquire the original medical image containing multiple organs to be segmented.

[0132] The preprocessing unit includes a first preprocessing subunit and a second preprocessing subunit. The first preprocessing subunit uses the Stem module to downsample the input raw medical image by 1 / 2 after a convolution with a stride of 2, followed by a 1*1 convolution branch and a pooling branch, and then concatenates the outputs of the two branches to obtain a 1 / 2 feature map. The second preprocessing subunit uses a common Transformer preprocessing method. First, the original input image is divided into patches according to a predetermined size. Then, based on the patches obtained after segmentation, the total number is calculated. The convolution kernel size is set according to the total number, and the original input image is convolved. After layer normalization, a 1 / 4 feature map is obtained.

[0133] The image segmentation network adopts the image segmentation network in the multi-organ segmentation method for medical images with fusion frequency domain awareness as described in Example 1. The image segmentation network consists of 4 PVT modules, 5 TAF modules, 4 PE modules, and 3 SRA modules, forming a U-shaped network with a dual-branch encoder. The 4 PVT modules constitute the spatial domain feature extraction branch of the decoder backbone network. The 5 TAF modules constitute the frequency domain feature extraction branch in the decoder backbone network, achieving the fusion of spatial and frequency domain features at each scale. The 4 PE modules constitute the decoder, and the feature images passed between every two PE modules undergo semantic relative position embedding based on relative position encoding and attention calculation via an SRA module. The preprocessed features output from the first and second preprocessing subunits are input into the first TAF module and the first PVT module of the image segmentation network, respectively.

[0134] The classification module is used to obtain the upsampled result of the last PE module in the image segmentation network, and form a classification mask through the classification head, thereby generating a medical image after image segmentation according to organs.

[0135] Example 3

[0136] This embodiment provides a multi-organ segmentation device incorporating frequency domain awareness, comprising a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it creates a multi-organ segmentation system incorporating frequency domain awareness as described in Embodiment 2, thereby enabling target recognition and image segmentation of the input raw medical image, and outputting a medical image segmented according to organs.

[0137] The multi-organ segmentation device with fused frequency domain perception provided in this embodiment is essentially a computer device. This computer device can be a smart terminal capable of executing programs, such as a tablet computer, laptop computer, desktop computer, rack server, blade server, tower server, or cabinet server (including standalone servers or server clusters composed of multiple servers). The computer device in this embodiment includes, but is not limited to, a memory and a processor that can communicate with each other via a system bus.

[0138] In this embodiment, the memory (i.e., the readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory can be an internal storage unit of a computer device, such as the hard disk or RAM of the computer device. In other embodiments, the memory can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the computer device. Of course, the memory can also include both internal storage units and external storage devices of the computer device. In this embodiment, the memory is typically used to store the operating system and various application software installed on the computer device. In addition, the memory can also be used to temporarily store various types of data that have been output or will be output.

[0139] In some embodiments, a processor may be a central processing unit (CPU), a graphics processing unit (GPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor is typically used to control the overall operation of a computer device. In this embodiment, the processor is used to run program code stored in memory or process data.

[0140] Simulation Experiment

[0141] To verify the technical effectiveness of the multi-organ segmentation method in medical images using fusion frequency domain perception provided in Embodiment 1 of this invention, the scheme of this invention was trained and tested during simulation, and its performance was compared and evaluated with other existing schemes. Simultaneously, to analyze the performance of the various improvements made in this invention, corresponding ablation experiments were also designed in the simulation experiments.

[0142] I. Model Training and Testing

[0143] 1.1 Sample Data

[0144] This experiment was conducted for training and testing on two different datasets: the Multi-Organ Segmentation Dataset (Synapse) and the Automated Cardiac Diagnosis Challenge Dataset (ACDC). The Synapse dataset includes 30 abdominal CT scans and 3779 abdominal axial clinical CT images. The sample dataset used in the simulation consisted of 18 training scans and 12 randomized test scans, following previous work. Our method was evaluated using the mean Srensen coefficient (DSC) and mean Hausdorff distance (HD95) for eight abdominal organs: aorta (Aorta), gallbladder (GB), left kidney (KL), right kidney (KR), liver (Liver), pancreas (PC), spleen (SP), and stomach (SM). The ACDC dataset includes 100 MRI scans collected from different patients, each scan labeled with three organs: left ventricle (LV), right ventricle (RV), and myocardium (MYO). Consistent with previous work, 80 cases were used for training and 20 for testing, and the method was evaluated using the mean DSC.

[0145] 1.2 Simulation Environment

[0146] The model used in this experiment is based on PyTorch and trained on a single Nvidia GeForce TX 3090 GPU with 24GB of memory. Following the rules of Swin-Unet, all datasets have been moderately augmented. The input image size was uniformly set to 224×224, and the initial learning rate was 1e. -4 A multi-aggregate learning rate strategy was employed. The maximum number of training epochs was 200, the batch size was 24, and a weight decay of 1e was used. -4 AdamW optimizer.

[0147] 1.3 Experimental Results of the ACDC Dataset

[0148] This experiment uses MRI images from the ACDC dataset to evaluate the performance of the network model proposed in this invention. During the testing process, TransUnet (2021), SwinUnet (2022), MT-Unet (2022), MISSformer (2022), PVT-CASCADE (2023), and TransCASCADE (2023) were selected as control groups. The accuracy of the network model on RV, Myo, and LV segmentation, and the DICE value representing segmentation accuracy, were evaluated. The performance test results are shown in Table 1.

[0149] Table 1: Performance evaluation results of different algorithms on the ACDC dataset

[0150] Model framework DICE RV Myo LV TransUnet / 2021 89.71 88.86 84.53 95.73 SwinUnet / 2022 90.00 88.55 85.62 95.83 MT-Unet / 2022 90.43 86.64 89.04 95.62 MISSformer / 2022 90.86 89.55 88.04 94.99 PVT-CASCADE / 2023 91.46 88.90 89.97 95.50 TransCASCADE / 2023 91.63 89.14 90.25 95.50 Ours 92.35 91.54 89.23 96.27

[0151] Based on the data in the table, it can be seen that compared with various popular and efficient methods in the past three years, the DICE index of the proposed solution on the ACDC dataset is relatively high, which indicates that the proposed solution is more advantageous.

[0152] 1.4. This experiment uses CT images from the Synapse dataset to evaluate the performance of the network model in this invention. The control group includes U-net (2015), Attn-Unet (2018), R50+Unet (2021), SSFormerPVT (2021), TransUnet (2021), SwinUnet (2022), MT-Unet (2022), MISSformer (2022), PVT-CASCADE (2023), and TransCASCADE (2023). The segmentation accuracy of each network model on Aorta, GB, KL, KR, Liver, PC, SP, and SM, as well as the DICE and HD95 values ​​representing segmentation accuracy, are compared. Figure 10 To select some typical segmentation cases from the test set in the Synapse dataset, the overall performance test results are shown in Table 2:

[0153] Table 2: Performance evaluation results of different algorithms on the Synapse dataset

[0154]

[0155] Combination Figure 10 As shown in Table 2, the solution provided by this invention outperforms popular and efficient methods in recent years in terms of segmentation results for different cases. Furthermore, combining the ground truth values ​​clearly demonstrates that the segmentation method of this invention can better segment the edges of organs and avoids missegmentation of organs.

[0156] II. Ablation Experiment

[0157] To verify the effectiveness of different design modules in the network model designed in this invention, ablation experiments were conducted on each of the designed modules.

[0158] 2.1 Encoder and Decoder Ends

[0159] First, consider the encoder of the multi-organ segmentation network model designed in this invention, which adopts a structure combining Transformer and Fourier Convolution. We call this fused structure the TAF module. We first tested the encoder with only Fourier Convolution on the Synapse dataset, then tested the encoder with only Transformer, and finally tested the TAF fusion module designed in this invention. In all three cases, the decoder is the same: a conventional decoder that only includes the PVT module and does not include the SRA module.

[0160] Considering the decoder of the model of this invention, this experiment designed a semantic relative attention (SRA) module by combining relative position encoding and attention mechanism. We also conducted ablation experiments on the SRA module and tested the overall model accuracy (Dice) with and without the SRA module on Synapse.

[0161] The final DICE metrics for the four experimental groups during the testing process are shown in Table 3:

[0162] Table 3: Ablation Experimental Test Results of Different Modules in the Network Model

[0163]

[0164] The results in the table above show that, based on the Dice metrics of the first three experiments where differences only exist at the encoder end, the Feature Fusion Module (TAF) designed in this invention can fully combine the advantages of the spatial and frequency domains, resulting in a significant improvement in segmentation performance. Comparing the Dice metrics of the latter two experiments where differences only exist at the decoder end, the SRA module designed in this invention, by providing additional prior information on relative positions, also significantly improves segmentation performance.

[0165] 2.2 Ablation Experiment of Spectral Transform in TAF Module

[0166] This experiment further supplements Ablation Experiment 1, primarily to evaluate the performance improvement of the Spectral Transform module in the TAF module. To verify the effectiveness of frequency domain conversion, the control group in this experiment replaced the Spectral Transform module with a conv (convolution) module with the same computational cost and parameters in the TAF module, testing whether spatial-domain to frequency-domain conversion is useful under the condition of the same computational cost and parameters. The specific experimental design is as follows:

[0167] In experimental group 1, only PVT was used at the decoder end. In experimental group 2, PVT+TAF was used at the decoder end, but the Spectral Transform in the TAF was replaced with conv. In experimental group 3, PVT+TAF was used at the decoder end, and the Spectral Transform was retained in the TAF. None of the decoders in experimental groups 1-3 used SRA. The decoders in experimental groups 4 and 5 both used an improved scheme including SRA. At the encoder end, experimental group 4 used PVT+TAF, and the Spectral Transform in the TAF was replaced with conv. Experimental group 5 used PVT+TAF, and the Spectral Transform was retained in the TAF. The test results for the five experimental groups are shown in Table 4.

[0168] Table 4: Ablation Experiment Results of the Spectral Transform Module in the Network Model

[0169]

[0170] Analysis of the data from experimental groups 1-3 shows that after replacing the frequency domain module with a convolutional module with the same parameter computational cost, the model with the frequency domain module clearly has a higher Dice index. Further analysis of the data from experimental groups 4 and 5 reveals that adding a semantically relative position encoding (SRA) module to the decoder does not conflict with the frequency domain module; in fact, combining the two produces better segmentation results.

[0171] 2.3 Ablation Experiment of Semantic Phase Position Encoding in SRA Module

[0172] In the improved decoder of this invention, an additional semantic relative positional encoding (SRA) model is added. To compare whether the scheme of this invention can have a positive effect on segmentation compared with existing positional encoding methods of different types, relevant ablation experiments were designed.

[0173] This experiment consisted of four experimental groups. While keeping all other model components unchanged, the differences were made only at the decoder level. The four experimental groups were tested sequentially: without positional codes, with absolute positional codes, with traditional relative positional codes derived from the Swin Transformer, and with the newly designed semantic relative positional codes of this invention. The test results for the four experimental groups are shown in Table 5.

[0174] Table 5: Ablation test results of semantic relative position encoding in the network model

[0175]

[0176] Comparing the test results of the above experimental groups, it can be seen that the semantic relative position encoding designed in this invention can achieve the best improvement effect on the segmentation of the model.

[0177] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-organ segmentation method for medical images fused with frequency domain awareness, used for identifying and segmenting individual organs in a medical image containing multiple organs, characterized in that, It includes the following steps: S1: Construct an image segmentation network that integrates image features and frequency domain awareness for segmenting multiple organs in medical images. The process is as follows: S11: Design a U-shaped network that uses PVT modules as the initial backbone. The encoder in the U-shaped network uses a four-layer PVT module for downsampling, and the decoder uses a four-layer PE module for upsampling. S12: In addition to the four-layer PVT module branch in the encoder, a feature fusion branch consisting of a five-layer TAF module is added. The TAF module is used to extract frequency domain features and fuse them with spatial features. The improved encoder cross-fuses the spatial features extracted by the PVT module of each layer with the frequency domain features extracted by the TAF module of the previous layer, and then passes them to the next TAF layer after n iterations. S13: Add three SRA modules to the decoder. The SRA modules are used to add relative position encoding to the upsampling results between adjacent PE modules, and then obtain the refined features through two layers of attention calculation. In the improved decoder, the outputs of each stage of the PE module and TAF module are cascaded on the channel via skip connections, and then input into the SRA module of the corresponding scale for processing; the SRA module is used to introduce relative position encoding and semantically guided relative position embedding on the basis of traditional attention computation. S2: Obtain medical image samples containing multiple organs, manually label the organs in each medical image sample to form the required dataset; and divide the dataset into training set and test set; S3: Construct a multi-organ segmentation model to be trained using the image segmentation network. The input of the multi-organ segmentation model is the original medical image, which is divided into two branches in the multi-organ segmentation model. One branch is processed by the Stem module and then input into the TAF module of the first layer of the image segmentation network. The other branch is processed by cropping, convolution, and normalization and then input into the PVT module of the first layer of the image segmentation network. The output of the last layer SRA module in the decoder of the image segmentation network is upsampled by the PE module and then passed through the classification head to form a classification mask, thereby outputting the generated organ segmentation result. S4: The loss function L, which includes cross-input loss and pixel-level constraint loss, is used as the loss function L during the training phase, and the multi-organ segmentation model is trained and tested using training set, test set and validation set; S5: Save the model parameters of the trained and validated best multi-organ segmentation model, and use the multi-organ segmentation model to process medical images to identify and segment various organs in the medical images.

2. The multi-organ segmentation method in medical images based on fusion frequency domain perception as described in claim 1, characterized in that: The five TAF modules in the image segmentation network constructed in step S1 are TAF1 to TAF5, and the data processing procedures in TAF2 to TAF5 modules are as follows: (1) The fusion features extracted from the previous layer TAF are fused together. i-1 The components are split into two paths and then processed using 3×3 convolution. (2) The spatial features extracted from the previous PVT layer PVT i-1 It is divided into two paths: one path is processed by 3×3 convolution, and the other path is processed by the Spectral Transform module; (3) TAF after 3×3 convolution i-1 and PVT i-1 The summation, followed by LN normalization, yields the frequency domain feature M. i-1 ; (4) TAF after 3×3 convolution i-1 PVT processed by the Spectral Transform module i-1 Add them together; then, after LN standardization, obtain the spatial feature T. i-1 ; (5) For the feature TAF i-1 To feature M i-1 and from the characteristic PVT i-1 To feature T i-1 The two-way feature transformation process is iterated n times, and then the results are channel-connected; (6) Input the channel-connected features into the feedforward network FFN for processing to obtain the fusion features TAF required for the next layer. i ; The first layer's TAF1 module uses the same data processing strategy as the TAF2~TAF5 modules. The difference is that the features on the channel are divided into two equal parts before the TAF1 module. After convolution processing, the two parts are passed in parallel and used as the two raw inputs required by the TAF1 module.

3. The multi-organ segmentation method in medical images based on fusion frequency domain perception as described in claim 2, characterized in that: The expression for the data processing procedure in the TAF module is as follows: ； In the above formula, the subscript j Indicates the iteration round, and These represent the previous TAF layer before and after the iteration, respectively. i-1 The fusion characteristics of modules; and These represent the previous PVT layer before and after the iteration, respectively. i-1 Spatial characteristics of modules; TAF i This represents the fusion feature output by the TAF module of the current layer; Indicates convolution processing; Spect This indicates the spectral transformation processing performed by the Spectral Transform module; "Indicates a channel connection between features; ReLU and GELU These represent the corresponding activation functions; LN represents LayerNorm normalization. This indicates the processing of the feedforward network (FFN). This represents the fused features after n iterations; This represents the spatial features after n iterations.

4. The multi-organ segmentation method in medical images based on fusion frequency domain perception as described in claim 2, characterized in that: The Spectral Transform module includes, in sequence, a Fast Fourier Transform layer, a Convolutional Normalized Activation Function layer, and an Inverse Fast Fourier Transform layer. In the Spectral Transform module, the original feature image is first transformed from the spatial domain to the frequency domain, then the weights are updated in the frequency domain, and finally the frequency domain features with updated weights are transformed back to the spatial domain.

5. The multi-organ segmentation method in medical images based on fusion frequency domain perception as described in claim 4, characterized in that: The scaling transformation process of image features in the three different functional layers of the Spectral Transform module is as follows: ； In the above formula, The input image features are represented by H, W, and C, which represent the length, width, and dimension of the feature image, respectively; RealFFT2d represents the Fast Fourier Transform layer; ReLU represents the activation function; and BN represents batch normalization. This indicates a convolution operation with a kernel of 1; InverseRealFFT2d indicates an inverse fast Fourier transform layer.

6. The multi-organ segmentation method in medical images based on fusion frequency domain perception as described in claim 1, characterized in that: The image segmentation network constructed in step S1 consists of three SRA modules, SRA1 to SRA3. Each SRA module comprises a position index unit for extracting relative position encoding and relative position embedding, and a self-attention mechanism. The data processing procedure of the SRA module is as follows: (1) The original feature vector is linearly transformed to the QKV space through the self-attention mechanism to obtain the query vector Q, the key vector K and the value vector V; (2) Based on the absolute position of the pixel in the coordinate layer in the original multi-scale feature image, a two-dimensional relative position code R is extracted by the position indexing unit. pos and two corresponding one-dimensional relative position indices R in the height and width directions h and R w ; (3) Using relative position index R h and R w And the query vector Q, the semantically guided relative position embeddings in the corresponding height and width directions are calculated using the following formula. , ; ； (4) Embedding based on two relative positions and A one-dimensional relative position offset S is calculated using the following formula. pos : ； (5) Encode the obtained relative position R pos and relative position offset S pos Treating them as relative positional information and incorporating them into QK in the self-attention mechanism T In Chinese, the expression is as follows: ； In the above formula, d This is a parameter that characterizes the scaling ratio.

7. The multi-organ segmentation method in medical images based on fusion frequency domain perception as described in claim 1, characterized in that: The expression for the loss function L in step S4 is as follows: In the above formula, p This represents the label value in the segmentation result. Represents the artificial labels in the original image; This represents the cross-entropy loss based on pixel-level constraints; This represents the sample similarity loss used in semantic segmentation problems. and These represent the weights of cross-entropy loss and sample similarity loss in the loss function, respectively.

8. The multi-organ segmentation method in medical images based on fusion frequency domain perception as described in claim 7, characterized in that: In step S11, the four-layer PVT module in the encoder adopts a Transformer with a pyramid structure. Each layer PVT module performs downsampling processing on the input image and calculates attention. The four-layer PE module in the decoder adopts the Patch Expanding Block in Transformer. Each PE module upsamples the fused features input from the TAF module of the next layer. The upsampling result is input into the SRA module for processing, and the result processed by the SRA module is then passed to the PE module of the previous layer.

9. The multi-organ segmentation method in medical images based on fusion frequency domain perception as described in claim 1, characterized in that: The iteration count for both the frequency domain features and spatial features in the TAF module is set to 2.

10. A multi-organ segmentation system integrating frequency domain sensing, characterized in that, It employs the multi-organ segmentation method in medical images based on fusion frequency domain perception as described in any one of claims 1-9, to identify and segment each organ in a medical image containing multiple organs; the multi-organ segmentation system includes: Image acquisition unit, which is used to acquire raw medical images containing multiple organs to be segmented; The preprocessing unit includes a first preprocessing subunit and a second preprocessing subunit. The first preprocessing subunit uses the Stem module to downsample the input raw medical image by 1 / 2 after a convolution with a stride of 2, followed by a 1*1 convolution branch and a pooling branch, and then concatenates the outputs of the two branches to obtain a feature map of 1 / 2. The second preprocessing subunit uses the preprocessing method commonly used in Transformers. First, the original input image is divided into blocks according to a predetermined size. Then, based on the patches obtained after the blocks are divided, the total number is calculated. The convolution kernel size is set according to the total number, and the original input image is convolved. After layer normalization, a feature map of 1 / 4 is obtained. An image segmentation network in a multi-organ segmentation method for medical images based on fusion frequency domain awareness as described in any one of claims 1-9; the image segmentation network consists of 4 PVT modules, 5 TAF modules, 4 PE modules, and 3 SRA modules, forming a U-shaped network containing a dual-branch encoder; wherein, the 4 PVT modules constitute the spatial domain feature extraction branch of the encoder backbone network, the 5 TAF modules constitute the frequency domain feature extraction branch of the encoder backbone network, and the fusion of spatial and frequency domain features is achieved at each scale; the 4 PE modules constitute the decoder, and the feature images transmitted between every two PE modules are subjected to semantic relative position embedding based on relative position encoding and attention calculation by an SRA module; the preprocessed features output by the first preprocessing subunit and the second preprocessing subunit are respectively input into the first TAF module and the first PVT module in the image segmentation network; The classification module is used to obtain the upsampling result output by the last PE module in the image segmentation network, and form a classification mask through the classification head to generate a medical image after image segmentation according to organs.

11. A multi-organ segmentation device integrating frequency domain sensing, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, it creates the multi-organ segmentation system with fusion frequency domain perception as described in claim 10, thereby realizing target recognition and image segmentation of the input raw medical image, and outputting a medical image segmented according to organs.

Images