Medical image processing method based on multilayer double-flow sharing network framework

Abstract

The embodiment of the invention provides a medical image processing method based on a multilayer double-flow sharing network framework. The method is applied to the technical field of medical image processing, and comprises the following steps: acquiring a medical image, and preprocessing the acquired medical image; performing feature extraction on the preprocessed medical image through a shared encoder to obtain basic feature representation of the corresponding medical image; inputting the basic feature representation of the medical image into a segmentation task decoder and a high-resolution task decoder respectively, and analyzing and processing the basic features of the medical image through the segmentation task decoder and the high-resolution task decoder to obtain a segmentation feature map and a high-resolution feature map; and feature fusion is performed on the segmented feature map and the high-resolution feature map through the multi-scale frequency crossover gate module to obtain a target feature map, so that the processing effect and the processing efficiency of the medical image are improved.

Description

Technical Field

[0001] The present application relates to the field of medical image processing technology, and in particular to a medical image processing method based on a multi-layer dual-stream shared network framework. Background Art

[0002] Medical image segmentation is one of the core tasks of medical image processing. Its essence is to extract key areas from various medical images (such as CT, MRI, ultrasound, etc.), which can be used to assist in medical diagnosis, treatment plan formulation, surgical navigation and other scenarios.

[0003] Existing image segmentation models include U-Net, its improved versions such as U-Net++ and U-Net3+, and the Transformer architecture introduced for image segmentation based on traditional convolutional neural networks.

[0004] The above methods have the problem of high computing resource requirements when performing high-resolution medical image segmentation, which limits their application in resource-constrained equipment in the medical field. Existing lightweight models cannot fully capture rich image feature information, resulting in poor segmentation results. Existing research only focuses on the final output features of the decoder, ignoring the rich spatial details and contextual information carried by the intermediate features of the decoder, affecting the segmentation accuracy in high-resolution output scenarios. Summary of the Invention

[0005] The present application provides a medical image processing method based on a multi-layer dual-stream shared network framework to solve the problems of high computing resource requirements, insufficient lightweight model performance, and underutilization of decoder intermediate features in existing medical image segmentation technologies.

[0006] In a first aspect, the present application provides a medical image processing method based on a multi-layer dual-stream shared network framework, wherein the multi-layer dual-stream shared network framework includes: a shared encoder, a segmentation task decoder, a high-resolution task decoder, and a multi-scale frequency cross-gate module. The method includes:

[0007] Acquiring medical images and preprocessing the acquired medical images;

[0008] The pre-processed medical image is subjected to feature extraction through a shared encoder to obtain the basic feature representation of the corresponding medical image;

[0009] The basic feature representation of the medical image is input into the segmentation task decoder and the high-resolution task decoder respectively. The basic features of the medical image are analyzed and processed by the segmentation task decoder and the high-resolution task decoder to obtain the segmentation feature map and the high-resolution feature map;

[0010] The segmentation feature map and the high-resolution feature map are fused through a multi-scale frequency cross-gate module to obtain the target feature map.

[0011] Optionally, the multi-scale frequency cross-gate module includes: a multi-scale residual channel attention module, a residual space attention module and a multi-scale frequency-aware convolution module. The multi-scale frequency cross-gate module is used to perform feature fusion on the segmentation feature map and the high-resolution feature map to obtain the target feature map, including:

[0012] Merge the segmentation feature map and the high-resolution feature map to obtain a merged feature map, perform a 1×1 convolution operation on the merged feature map to obtain a processed first intermediate feature map;

[0013] The first intermediate feature map is subjected to feature weighted fusion processing through a multi-scale residual channel attention module to obtain a second intermediate feature map;

[0014] The second intermediate feature map is subjected to feature weighted fusion processing through the residual space attention module to obtain the third intermediate feature map;

[0015] The third intermediate feature map is grouped according to the channel size through the multi-scale frequency-aware convolution module. 1×1 convolution, frequency-adaptive dilated convolution and pooling operations are applied to different groups to obtain feature information of different frequencies and spaces. The feature information of different frequencies and spaces is integrated to obtain the target feature map.

[0016] Optionally, performing feature weighted fusion processing on the first intermediate feature map through a multi-scale residual channel attention module to obtain a second intermediate feature map includes:

[0017] Extracting global features of the first intermediate feature map through adaptive maximum pooling and adaptive average pooling operations, generating channel attention weights of the global features through a nonlinear activation function, and weighting the first intermediate feature map according to the channel attention weights of the global features to obtain a first global feature map;

[0018] Use two 3×3 depthwise convolutions to obtain the local features of the first intermediate feature map, generate the channel attention weights of the local features through a nonlinear activation function, and weight the first intermediate feature map according to the channel attention weights of the local features to obtain the first local feature map;

[0019] Feature fusion is performed on the first intermediate feature map, the first global feature map, and the first local feature map to obtain a second intermediate feature map.

[0020] Optionally, performing feature weighted fusion processing on the second intermediate feature map through the residual space attention module to obtain a third intermediate feature map includes:

[0021] Determine the maximum value and the average value of the second intermediate feature map in the channel dimension, and concatenate the maximum value and the average value of the second intermediate feature map in the channel dimension to obtain a second concatenated feature map;

[0022] Input the second concatenated feature map into the large kernel convolution layer to enhance the local context relationship, thereby obtaining a second enhanced feature map;

[0023] Generating a spatial attention weight of the second enhanced feature map through a nonlinear activation function, and weighting the second intermediate feature map according to the generated spatial attention weight of the second enhanced feature map to obtain a second weighted feature map;

[0024] The second intermediate feature map and the second weighted feature map are fused to obtain a third intermediate feature map.

[0025] Optionally, after performing feature fusion on the segmentation feature map and the high-resolution feature map through the multi-scale frequency cross-gate module to obtain the target feature map, the method further includes:

[0026] The output results of the segmentation task decoder, high-resolution task decoder and multi-scale frequency cross-gate module are optimized respectively using objective functions, where the objective functions include: segmentation loss, high-resolution loss and task interaction constraint loss.

[0027] Optionally, the objective function is used to optimize the output results of the segmentation task decoder, the high-resolution task decoder, and the multi-scale frequency cross-gate module respectively, including:

[0028] Cross entropy loss is used as the segmentation loss to optimize the output of the segmentation task decoder;

[0029] The mean square error is used as the high-resolution loss to optimize the output of the high-resolution task decoder;

[0030] Structural similarity index loss and regional mutual information loss are used as task interaction constraint losses to optimize the output results of the multi-scale frequency cross-gate module.

[0031] In a second aspect, the present application provides a medical image processing device based on a multi-layer dual-stream shared network framework, the device comprising:

[0032] An acquisition module, used for acquiring medical images;

[0033] A first processing module, configured to pre-process the acquired medical image;

[0034] A second processing module is used to extract features from the preprocessed medical image using a shared encoder to obtain a basic feature representation of the corresponding medical image;

[0035] The second processing module is further configured to input the basic feature representation of the medical image into the segmentation task decoder and the high-resolution task decoder respectively, and analyze and process the basic features of the medical image through the segmentation task decoder and the high-resolution task decoder to obtain a segmentation feature map and a high-resolution feature map;

[0036] The second processing module is further configured to perform feature fusion on the segmentation feature map and the high-resolution feature map through a multi-scale frequency cross-gate module to obtain a target feature map.

[0037] In a third aspect, the present application provides a medical image processing device based on a multi-layer dual-stream shared network framework, comprising:

[0038] Memory;

[0039] processor;

[0040] wherein the memory stores computer-executable instructions;

[0041] The processor executes the computer-executable instructions stored in the memory to implement the medical image processing method based on the multi-layer dual-stream shared network framework as described in the above-mentioned first aspect and various possible implementation methods of the first aspect.

[0042] In a fourth aspect, the present application provides a computer storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the medical image processing method based on the multi-layer dual-stream shared network framework as described in the first aspect and various possible implementation methods of the first aspect.

[0043] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the medical image processing method based on the multi-layer dual-stream shared network framework as described in the first aspect and various possible implementation methods of the first aspect.

[0044] The present application provides a medical image processing method based on a multi-layer dual-stream shared network framework. The method obtains a medical image and preprocesses the obtained medical image; extracts features from the preprocessed medical image through a shared encoder to obtain a basic feature representation of the corresponding medical image; inputs the basic feature representation of the medical image into a segmentation task decoder and a high-resolution task decoder respectively, analyzes and processes the basic features of the medical image through the segmentation task decoder and the high-resolution task decoder to obtain a segmentation feature map and a high-resolution feature map; and fuses the segmentation feature map and the high-resolution feature map through a multi-scale frequency cross-gate module to obtain a target feature map, thereby improving the processing effect and efficiency of the medical image. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0046] Figure 1 Schematic diagram of the process of medical image processing method based on multi-layer dual-stream shared network framework provided in the embodiment of the present application Figure 1 ;

[0047] Figure 2 A schematic diagram of the structure of a medical image processing device based on a multi-layer dual-stream shared network framework provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of the structure of a medical image processing device based on a multi-layer dual-stream shared network framework provided in an embodiment of the present application.

[0049] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0050] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions in this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0051] The terms "first," "second," "third," "fourth," and so forth (if any) in the description and claims of the present invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequential sequence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in sequences other than those illustrated or described herein.

[0052] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0053] With the development of deep learning, image segmentation technology has made significant progress. U-Net extracts features at different levels by downsampling the encoder, and the decoder uses jump connections to upsample the low-level features obtained to restore the image resolution, thereby segmenting the objects in the image. On this basis, improved versions such as U-Net++ and U-Net3+ have been proposed. Transformer-based methods introduce the Transformer architecture based on traditional convolutional neural networks, which can capture long-range dependencies, overcome the limitations of traditional methods in dealing with complex structural relationships, and improve segmentation accuracy. However, the above methods generally have the problem of large computing resource requirements when performing high-resolution medical image segmentation, which limits their application on resource-constrained equipment in the medical field.

[0054] In order to reduce the computational cost, a lightweight model based on U-Net and convolutional neural network was proposed. However, the lightweight model cannot fully capture rich image feature information, resulting in poor segmentation effect.

[0055] In addition, in the research on combining single image super-resolution networks to assist medical image segmentation, most methods only focus on sharing the final output features of the decoder, ignoring the rich spatial details and contextual information carried by the intermediate features of the decoder.

[0056] To address the above problems, this application proposes a medical image processing method based on a multi-layer dual-stream shared network framework. This method effectively integrates the intermediate features of the decoder and improves the segmentation accuracy by proposing a multi-layer dual-stream shared network framework; introduces a multi-scale frequency cross-gate module and frequency-adaptive dilated convolution to enhance the feature extraction capability, while ensuring the segmentation accuracy and reducing the computational cost, meeting the needs of high-resolution medical image segmentation in medical scenarios with limited computing resources, and is suitable for key area extraction and clinical application scenarios of various medical images.

[0057] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0058] Figure 1 A schematic diagram of a medical image processing method based on a multi-layer dual-stream shared network framework provided in an embodiment of the present application Figure 1 .like Figure 1 As shown, the medical image processing method based on the multi-layer dual-stream shared network framework provided in this embodiment includes:

[0059] S1: Acquire a medical image and preprocess the acquired medical image.

[0060] As you can understand, preprocessing medical images involves normalizing the images. Specifically, pixel values are mapped uniformly to the interval [0, 1] to eliminate brightness differences between images. Furthermore, according to medical image analysis standards, the normalized images are cropped to remove content unrelated to the target area of interest, focusing the image on the target area.

[0061] In an optional embodiment, accurate region labels are annotated for each image as supervisory information for model training. The dataset is divided into training, validation, and test sets in a ratio of 7:2:1 to ensure that the images in each set are representative and diverse.

[0062] S2: Perform feature extraction on the preprocessed medical image through a shared encoder to obtain the basic feature representation of the corresponding medical image.

[0063] The shared encoder may be a multi-layer network composed of a convolutional neural network or a Transformer, such as ResNet50, DenseNet121, or VisionTransformer.

[0064] Specifically, the shared encoder is used to extract features from the preprocessed medical image to obtain the basic feature representation of the corresponding medical image. The formula is as follows:

[0065] F en =En share (X)

[0066] Among them, En share represents the shared encoder, X is the input medical image, F en Represents the feature representation of the input image after being transformed by the shared encoder.

[0067] S3: The basic feature representation of the medical image is input into the segmentation task decoder and the high-resolution task decoder respectively. The basic features of the medical image are analyzed and processed by the segmentation task decoder and the high-resolution task decoder to obtain the segmentation feature map and the high-resolution feature map.

[0068] Among them, the segmentation task decoder, high-resolution decoder and shared encoder have the same structure and can be directly connected.

[0069] Specifically, the feature F en Input to the segmentation task decoder respectively and high-resolution task decoder Get task-specific feature map and The formula is as follows:

[0070]

[0071] in, represents the segmentation feature map, represents the segmentation task decoder, i represents the number of layers of the decoder, represents a high-resolution feature map, Represents a high-resolution task decoder.

[0072] S4: The segmentation feature map and the high-resolution feature map are fused through the multi-scale frequency cross-gate module to obtain the target feature map.

[0073] Among them, the multi-scale frequency cross-gate module includes a multi-scale residual channel attention module, a residual space attention module and a multi-scale frequency-aware convolution module.

[0074] The segmentation feature map and the high-resolution feature map are fused through the multi-scale frequency cross-gate module to obtain the target feature map, which specifically includes the following steps:

[0075] S41: Merge the segmentation feature map and the high-resolution feature map to obtain a merged feature map, perform a 1×1 convolution operation on the merged feature map, and obtain a processed first intermediate feature map.

[0076] The segmentation feature map is concatenated and high-resolution feature maps Merge and then perform 1×1 convolution to obtain the processed first intermediate feature map. The specific formula is:

[0077]

[0078] in, Represents the first intermediate feature map, C1 represents 1×1 convolution processing, and Con represents the concatenation operation.

[0079] The purpose of this step is to preliminarily integrate features from different task branches, laying the foundation for subsequent deeper feature fusion and information extraction.

[0080] S42: Perform feature weighted fusion processing on the first intermediate feature map through a multi-scale residual channel attention module to obtain a second intermediate feature map.

[0081] It can be understood that the multi-scale residual channel attention module performs feature weighted fusion processing on the first intermediate feature map to obtain the second intermediate feature map, which specifically includes the following steps:

[0082] S421: Extract the global features of the first intermediate feature map through adaptive maximum pooling and adaptive average pooling operations, generate the channel attention weights of the global features through a nonlinear activation function, and weight the first intermediate feature map according to the channel attention weights of the global features to obtain a first global feature map.

[0083] It is described by the following formula:

[0084]

[0085] in, is the first global feature map, C1 and C2 are 1×1 convolutions to adjust the number of channels, σ is the activation function Sigmoid, R is the activation function LeakyReLU, P m is the adaptive maximum pooling, P a is the adaptive average pooling, Represents element-wise multiplication.

[0086] S422: Use two 3×3 depth convolutions to obtain local features of the first intermediate feature map, generate channel attention weights of the local features through a nonlinear activation function, and weight the first intermediate feature map according to the channel attention weights of the local features to obtain a first local feature map.

[0087] It is described by the following formula:

[0088]

[0089] Among them, M i is the local feature of the first intermediate feature map, Represents the first local feature map, and C3 represents 3×3 convolution processing.

[0090] S423: Perform feature fusion on the first intermediate feature map, the first global feature map, and the first local feature map to obtain a second intermediate feature map.

[0091] It is described by the following formula:

[0092]

[0093] in, is the second intermediate feature map.

[0094] S43: Perform feature weighted fusion processing on the second intermediate feature map through the residual space attention module to obtain a third intermediate feature map.

[0095] In an optional embodiment, the second intermediate feature map is subjected to feature weighted fusion processing by the residual space attention module to obtain a third intermediate feature map, which specifically includes the following steps:

[0096] Determine the maximum value and the average value of the second intermediate feature map in the channel dimension, and concatenate the maximum value and the average value of the second intermediate feature map in the channel dimension to obtain a second concatenated feature map;

[0097] Input the second concatenated feature map into the large kernel convolution layer to enhance the local context relationship, thereby obtaining a second enhanced feature map;

[0098] Generating a spatial attention weight of the second enhanced feature map through a nonlinear activation function, and weighting the second intermediate feature map according to the generated spatial attention weight of the second enhanced feature map to obtain a second weighted feature map;

[0099] Perform feature fusion on the second intermediate feature map and the second weighted feature map to obtain a third intermediate feature map;

[0100] The specific formula is as follows:

[0101]

[0102] in, is the third intermediate feature map, is the maximum value in the channel dimension, is the average value in the channel dimension, C b It is a large kernel convolution layer.

[0103] S44: The third intermediate feature map is grouped according to the channel size through the multi-scale frequency-aware convolution module. 1×1 convolution, frequency-adaptive dilated convolution and pooling operations are performed on different groups to obtain feature information of different frequencies and spaces. The feature information of different frequencies and spaces is integrated to obtain the target feature map.

[0104] In an optional embodiment, the multi-scale frequency-aware convolution module divides the third intermediate feature map into k groups based on channel size. 1×1 convolution, frequency-adaptive dilated convolution, and pooling operations are applied to different groups to capture information at different frequencies and spatial scales. Finally, batch normalization and activation functions are used to integrate the information. The specific formula is:

[0105]

[0106] in, is the target feature map, G is the activation function GELU, BN is batch normalization, P represents the feature group after channel segmentation, FAMC is the frequency adaptive dilated convolution, and Pool is the pooling operation.

[0107] In an optional embodiment, after performing feature fusion on the segmentation feature map and the high-resolution feature map through the multi-scale frequency cross-gate module to obtain the target feature map, the method further includes:

[0108] The objective function is used to optimize the output results of the segmentation task decoder, high-resolution task decoder and multi-scale frequency cross-gate module respectively.

[0109] Among them, the objective functions include: segmentation loss, high-resolution loss and task interaction constraint loss.

[0110] Specifically, the output results of the segmentation task decoder, high-resolution task decoder, and multi-scale frequency cross-gate module are optimized using the objective function, including the following steps:

[0111] Cross entropy loss is used as the segmentation loss to optimize the output of the segmentation task decoder. The specific formula is:

[0112]

[0113] in, Represents the segmentation loss, C represents the number of categories, for example, in the brain MRI image tumor segmentation task, C = 2, namely the tumor area and the non-tumor area; y i is the true label of category i, the tumor area is 1 and the non-tumor area is 0; It is the predicted probability after the segmentation output is processed by the softmax function, which indicates the probability value of the model predicting that the sample belongs to category i.

[0114] By minimizing this loss, the model is guided to adjust its parameters so that the prediction results are closer to the true label.

[0115] The mean square error is used as the high-resolution loss to optimize the output of the high-resolution task decoder. The specific formula is as follows:

[0116]

[0117] in, represents high-resolution loss, and N represents the total number of pixels in the image; is the value of the jth pixel in the high-resolution task output image; HR j is the true value of the j-th pixel in the target high-resolution image.

[0118] Structural similarity index loss and regional mutual information loss are used as task interaction constraint losses to optimize the output results of the multi-scale frequency cross-gate module. The specific formula is as follows:

[0119]

[0120] in, represents the task interaction constraint loss; i represents the number of decoder layers; μ and ν are weights, which are both set to 0.5 in this embodiment after experiments to adjust the relative importance of the two losses in the total loss; SSIM is the structural similarity index loss; RMI is the regional mutual information loss; is the feature after fusion of the i-th layer; Y is the true label of the segmentation task; is the output of the i-th layer high-resolution task after processing by the multi-scale frequency cross-gate module; HR is the target high-resolution image.

[0121] In an optional embodiment, the segmentation loss, high-resolution loss, and task interaction constraint loss are added together to obtain the final loss function for overall model optimization. The specific formula is:

[0122]

[0123] in, is the final loss function, and N is the total number of decoder layers.

[0124] During the model training process, the gradient of the final loss function with respect to the model parameters is calculated through the back-propagation algorithm, and the optimizer is used to iteratively update the parameters to continuously reduce the final loss value, so that the model can achieve better performance in terms of segmentation accuracy and high-resolution reconstruction quality.

[0125] The embodiment of the present application provides a medical image processing method based on a multi-layer dual-stream shared network framework. The method obtains a medical image and preprocesses the obtained medical image; extracts features of the preprocessed medical image through a shared encoder to obtain a basic feature representation of the corresponding medical image; inputs the basic feature representation of the medical image into a segmentation task decoder and a high-resolution task decoder respectively, analyzes and processes the basic features of the medical image through the segmentation task decoder and the high-resolution task decoder to obtain a segmentation feature map and a high-resolution feature map; and fuses the segmentation feature map and the high-resolution feature map through a multi-scale frequency cross-gate module to obtain a target feature map, thereby improving the processing effect and efficiency of the medical image.

[0126] Figure 2 This is a schematic diagram of the structure of a medical image processing device based on a multi-layer dual-stream shared network framework provided in an embodiment of the present application. Figure 2 As shown, the medical image processing device 200 based on the multi-layer dual-stream sharing network framework provided in this embodiment includes:

[0127] An acquisition module 201 is used to acquire medical images;

[0128] A first processing module 202 is configured to pre-process the acquired medical image;

[0129] The second processing module 203 is configured to extract features from the pre-processed medical image using a shared encoder to obtain a basic feature representation of the corresponding medical image;

[0130] The second processing module 203 is further configured to input the basic feature representation of the medical image into the segmentation task decoder and the high-resolution task decoder respectively, and analyze and process the basic features of the medical image through the segmentation task decoder and the high-resolution task decoder to obtain a segmentation feature map and a high-resolution feature map;

[0131] The second processing module 203 is further configured to perform feature fusion on the segmentation feature map and the high-resolution feature map through a multi-scale frequency cross-gate module to obtain a target feature map.

[0132] The medical image processing device based on the multi-layer dual-stream shared network framework provided in this embodiment can execute the medical image processing method based on the multi-layer dual-stream shared network framework provided in the above method embodiment. Its implementation principles and technical effects are similar, and are not described in detail in this embodiment.

[0133] Figure 3 This is a schematic diagram of the structure of a medical image processing device based on a multi-layer dual-stream shared network framework provided in an embodiment of the present application. Figure 3 As shown, an embodiment of the present application provides a medical image processing device based on a multi-layer dual-stream sharing network framework. The medical image processing device 300 based on a multi-layer dual-stream sharing network framework includes: a receiver 301, a transmitter 302, a processor 303 and a memory 304.

[0134] Receiver 301, for receiving instructions and data;

[0135] Transmitter 302, used to send instructions and data;

[0136] Memory 304, for storing computer-executable instructions;

[0137] Processor 303 is configured to execute computer-executable instructions stored in memory 304 to implement the steps of the medical image processing method based on the multi-layer dual-stream shared network framework described in the above embodiment. For details, please refer to the description of the medical image processing method based on the multi-layer dual-stream shared network framework described above.

[0138] Optionally, the memory 304 may be independent or integrated with the processor 303 .

[0139] When the memory 304 is independently provided, the electronic device further includes a bus for connecting the memory 304 and the processor 303 .

[0140] An embodiment of the present application also provides a computer storage medium, which stores computer execution instructions. When a processor executes the computer execution instructions, it implements the medical image processing method based on the multi-layer dual-stream shared network framework as performed by the medical image processing device based on the multi-layer dual-stream shared network framework as mentioned above.

[0141] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned medical image processing method based on the multi-layer dual-stream shared network framework.

[0142] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable, and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media generally embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0143] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0144] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A medical image processing method based on a multi-layer dual-stream shared network framework, characterized in that: The multi-layer dual-stream shared network framework includes: a shared encoder, a segmentation task decoder, a high-resolution task decoder and a multi-scale frequency cross-gate module, and the method includes: Acquiring medical images and preprocessing the acquired medical images; The pre-processed medical image is subjected to feature extraction through a shared encoder to obtain the basic feature representation of the corresponding medical image; The basic feature representation of the medical image is input into the segmentation task decoder and the high-resolution task decoder respectively. The basic features of the medical image are analyzed and processed by the segmentation task decoder and the high-resolution task decoder to obtain the segmentation feature map and the high-resolution feature map; The segmentation feature map and the high-resolution feature map are fused through a multi-scale frequency cross-gate module to obtain the target feature map.

2. The method according to claim 1, characterized in that The multi-scale frequency cross-gate module includes: a multi-scale residual channel attention module, a residual space attention module and a multi-scale frequency perception convolution module. The multi-scale frequency cross-gate module is used to perform feature fusion on the segmentation feature map and the high-resolution feature map to obtain the target feature map, including: Merge the segmentation feature map and the high-resolution feature map to obtain a merged feature map, perform a 1×1 convolution operation on the merged feature map to obtain a processed first intermediate feature map; The first intermediate feature map is subjected to feature weighted fusion processing through a multi-scale residual channel attention module to obtain a second intermediate feature map; The second intermediate feature map is subjected to feature weighted fusion processing through the residual space attention module to obtain the third intermediate feature map; The third intermediate feature map is grouped according to the channel size through the multi-scale frequency-aware convolution module. 1×1 convolution, frequency-adaptive dilated convolution and pooling operations are applied to different groups to obtain feature information of different frequencies and spaces. The feature information of different frequencies and spaces is integrated to obtain the target feature map.

3. The method according to claim 2, characterized in that The method of performing feature weighted fusion processing on the first intermediate feature map through the multi-scale residual channel attention module to obtain the second intermediate feature map includes: Extracting global features of the first intermediate feature map through adaptive maximum pooling and adaptive average pooling operations, generating channel attention weights of the global features through a nonlinear activation function, and weighting the first intermediate feature map according to the channel attention weights of the global features to obtain a first global feature map; Use two 3×3 depthwise convolutions to obtain the local features of the first intermediate feature map, generate the channel attention weights of the local features through a nonlinear activation function, and weight the first intermediate feature map according to the channel attention weights of the local features to obtain the first local feature map; Feature fusion is performed on the first intermediate feature map, the first global feature map, and the first local feature map to obtain a second intermediate feature map.

4. The method according to claim 2, characterized in that The method of performing feature weighted fusion processing on the second intermediate feature map through the residual space attention module to obtain a third intermediate feature map includes: Determine the maximum value and the average value of the second intermediate feature map in the channel dimension, and concatenate the maximum value and the average value of the second intermediate feature map in the channel dimension to obtain a second concatenated feature map; Input the second concatenated feature map into the large kernel convolution layer to enhance the local context relationship, thereby obtaining a second enhanced feature map; Generating a spatial attention weight of the second enhanced feature map through a nonlinear activation function, and weighting the second intermediate feature map according to the generated spatial attention weight of the second enhanced feature map to obtain a second weighted feature map; The second intermediate feature map and the second weighted feature map are fused to obtain a third intermediate feature map.

5. The method according to claim 1, wherein After performing feature fusion on the segmentation feature map and the high-resolution feature map through the multi-scale frequency cross-gate module to obtain the target feature map, the method further includes: The output results of the segmentation task decoder, high-resolution task decoder and multi-scale frequency cross-gate module are optimized respectively using objective functions, where the objective functions include: segmentation loss, high-resolution loss and task interaction constraint loss.

6. The method according to claim 5, characterized in that The objective function is used to optimize the output results of the segmentation task decoder, the high-resolution task decoder, and the multi-scale frequency cross-gate module, respectively, including: Cross entropy loss is used as the segmentation loss to optimize the output of the segmentation task decoder; The mean square error is used as the high-resolution loss to optimize the output of the high-resolution task decoder; Structural similarity index loss and regional mutual information loss are used as task interaction constraint losses to optimize the output results of the multi-scale frequency cross-gate module.

7. A medical image processing device based on a multi-layer dual-stream shared network framework, characterized in that: The device comprises: An acquisition module, used for acquiring medical images; A first processing module, configured to pre-process the acquired medical image; A second processing module is used to extract features from the preprocessed medical image using a shared encoder to obtain a basic feature representation of the corresponding medical image; The second processing module is further configured to input the basic feature representation of the medical image into the segmentation task decoder and the high-resolution task decoder respectively, and analyze and process the basic features of the medical image through the segmentation task decoder and the high-resolution task decoder to obtain a segmentation feature map and a high-resolution feature map; The second processing module is further configured to perform feature fusion on the segmentation feature map and the high-resolution feature map through a multi-scale frequency cross-gate module to obtain a target feature map.

8. A medical image processing device based on a multi-layer dual-stream shared network framework, characterized in that: The device comprises: Memory; processor; wherein the memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the medical image processing method based on the multi-layer dual-stream sharing network framework according to any one of claims 1 to 6.

9. A computer storage medium, characterized in that The computer storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the medical image processing method based on a multi-layer dual-stream shared network framework according to any one of claims 1 to 6.

10. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, is used to implement the medical image processing method based on a multi-layer dual-stream shared network framework as described in any one of claims 1 to 6.

Images