Medical image segmentation method, device and medium based on self-supervised reconstruction assistance

Abstract

The application discloses a medical image segmentation method and device based on self-supervised reconstruction assistance, a medium, acquires a medical image, and inputs a reconstruction subnetwork to separate the foreground and background of the medical image to obtain a foreground image and a background image; the medical image, the foreground image and the background image are respectively input into an encoder to extract foreground features, background features and reconstruction features; based on the foreground features, the background features and the reconstruction features, multiplicative modulation parameters and additive modulation parameters are respectively generated, and the reconstruction features are adaptively modulated to obtain modulation features; the medical image is input into a segmentation subnetwork to extract segmentation features, and based on a cross attention mechanism, the segmentation features and the modulation features are bidirectionally cross-fused to generate fusion features; and a segmentation result of the medical image is output based on the fusion features. The application significantly enhances the performance of medical image segmentation under the premise of guaranteeing the semantic expression ability of the segmentation task.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image information processing, and in particular to a medical image segmentation method and device based on self-supervised reconstruction assistance and a medium. BACKGROUND

[0002] With the in-depth development of public health and the continuous progress of medical imaging technology, the core value of medical images in disease screening, auxiliary diagnosis and treatment evaluation is increasingly prominent. With the help of artificial intelligence technology, medical image analysis gradually realizes automatic feature extraction, lesion recognition and accurate segmentation, providing strong technical support for clinical diagnosis and treatment decision-making. Among them, medical image segmentation as the key link of accurate positioning of lesion area, algorithm research has become the core direction of promoting the intelligentization of image diagnosis in recent years.

[0003] Medical image segmentation algorithms based on deep learning mainly include two categories: one is the convolutional neural network (CNN) segmentation method, which learns multi-level features of the input image through the encoder-decoder architecture to realize high-precision segmentation of the lesion area; the other is the Transformer segmentation method, which uses the self-attention mechanism to model the global feature association of the image, and improves the segmentation performance of the lesion contour and overall morphology.

[0004] In recent years, image reconstruction algorithms driven by self-supervised learning have gradually become a research hotspot. This kind of method can strengthen the modeling ability of the segmentation model for edge, texture, structure and other detailed features by reconstructing the original image, effectively making up for the neglect of low-level features by traditional segmentation models, and improving the reliability of medical image auxiliary diagnosis. However, the medical image segmentation task focuses on the extraction of high-level semantic information of the target, which often leads to insufficient segmentation accuracy of the edge and local details of the model; while the core of the image reconstruction task is to restore fine-grained information, especially the edge contour and local structure, which have natural complementarity in feature modeling. Therefore, how to enhance the detailed perception ability of the model by introducing the reconstruction task while ensuring the semantic expression ability of the segmentation task is a problem to be solved in the field of joint modeling. SUMMARY

[0005] Based on the shortcomings of the prior art, the present application provides a medical image segmentation method, device and medium based on self-supervised reconstruction assistance, which significantly enhances the performance of medical image segmentation while ensuring the semantic expression ability of the segmentation task.

[0006] To solve the above technical problems, the first aspect of the present application discloses a medical image segmentation method based on self-supervised reconstruction assistance, which is executed on a model including a reconstruction subnetwork and a segmentation subnetwork, the reconstruction subnetwork is trained in a self-supervised manner, including:

[0007] The medical image is acquired and input into a reconstruction subnetwork to separate the foreground and background of the medical image to obtain a foreground image and a background image; the medical image, the foreground image and the background image are input into an encoder to extract foreground features, background features and reconstruction features;

[0008] Multiplicative modulation parameters and additive modulation parameters are generated based on the foreground features, the background features and the reconstruction features, and the reconstruction features are adaptively modulated to obtain modulation features;

[0009] The medical image is input into a segmentation subnetwork to extract segmentation features, and the segmentation features and the modulation features are bidirectionally cross-fused based on a cross-attention mechanism to generate fusion features;

[0010] The segmentation result of the medical image is output based on the fusion features.

[0011] In some embodiments, the multiplicative modulation parameters and the additive modulation parameters are generated based on the foreground features, the background features and the reconstruction features, and the reconstruction features are adaptively modulated to obtain modulation features, including:

[0012] The foreground features and the background features, and the reconstruction features are spliced to obtain spliced features;

[0013] A global average pooling operation is performed on the spliced features to obtain a global context feature vector;

[0014] The global context feature vector is input into a first convolution stream and a second convolution stream to obtain the multiplicative modulation parameters and the additive modulation parameters;

[0015] The reconstruction features are adaptively modulated based on the multiplicative modulation parameters and the additive modulation parameters to obtain modulation features.

[0016] In some embodiments, the segmentation features and the modulation features are bidirectionally cross-fused based on a cross-attention mechanism to generate fusion features, including:

[0017] The modulation features are taken as query vectors, the segmentation features are taken as key vectors and value vectors, a first cross-attention map is calculated, and enhanced segmentation features are calculated;

[0018] The segmentation features are taken as query vectors, and the modulation features are taken as key vectors and value vectors, a second cross-attention map is calculated, and enhanced reconstruction features are calculated;

[0019] The enhanced segmentation features and the enhanced reconstruction features are aggregated and residual-spliced to generate fusion features.

[0020] In some embodiments, the feature aggregation employs a convolutional layer with a kernel size of 3*3 to adjust the enhanced segmentation features and the enhanced reconstruction features, and the adjusted enhanced segmentation features and the adjusted enhanced reconstruction features are connected through a residual connection to maintain original semantic information, to generate fusion features.

[0021] In some embodiments, a segmentation result of the medical image is output based on the fusion features, including:

[0022] The fusion features are input into a segmentation head, and the spatial resolution is recovered through layer-by-layer upsampling and convolution operation, to output a segmentation prediction result with the same size as the medical image.

[0023] In some embodiments, the segmentation subnetwork is trained and optimized by minimizing a segmentation training loss function, and the segmentation training loss function includes a binary cross-entropy loss and a Dice loss.

[0024] In some embodiments, the reconstruction subnetwork is trained by minimizing a reconstruction loss function, and the reconstruction loss function includes a foreground reconstruction loss, a background reconstruction loss, and a medical image reconstruction loss.

[0025] In some embodiments, the modulation features are generated by the following formula:

[0026]

[0027] wherein, the modulation features are represented by fmod; is the concatenation feature;

[0028] is a multiplicative modulation parameter, is an additive modulation parameter, . is a global context feature vector, is a Sigmoid activation function, represents a convolutional layer with two convolution kernels of 1*1.

[0029] In a second aspect, a computer device is disclosed, including a processor and a memory; wherein the memory stores a computer program, and the computer program is adapted to be loaded and executed by the processor to implement the steps of the medical image segmentation method based on self-supervised reconstruction assistance according to any one of the above.

[0030] In a third aspect, a computer storage medium is disclosed, and the computer storage medium stores a computer program, and the computer program is executed by a processor to implement the medical image segmentation method based on self-supervised reconstruction assistance according to any one of the above.

[0031] Compared with the prior art, the beneficial effects of the present application are as follows:

[0032] The medical image segmentation method, device and medium based on self-supervised reconstruction assistance provided by the application introduce a self-supervised image reconstruction task in the segmentation task, utilize the information of the input medical image itself without additional labeling, extract foreground features and background features through foreground and background separation, generate multiplicative and additive modulation parameters based on the foreground features, background features and reconstruction features to adaptively modulate the reconstruction features, so as to enhance the fine-grained information related to the lesion; and the segmentation features and the modulation features are bidirectionally fused through the cross attention mechanism in the segmentation subnetwork, the complementarity of semantics and details is realized, the fusion features with global structure and local details are generated, the deficiency of the traditional segmentation method in low-level feature reservation is made up, and the performance of the medical image segmentation model in lesion target boundary recognition and segmentation precision is significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 The reconstruction subnetwork and the segmentation subnetwork of the medical image segmentation method based on self-supervised reconstruction assistance provided by the application are shown in the overall training flowchart of the medical image segmentation method based on self-supervised reconstruction assistance provided by the application;

[0034] Figure 2 The flowchart of the medical image segmentation method based on self-supervised reconstruction assistance provided by the application is shown in the flowchart of the medical image segmentation method based on self-supervised reconstruction assistance provided by the application;

[0035] Figure 3 The adaptive feature modulation module in the medical image segmentation method based on self-supervised reconstruction assistance provided by the application is shown in the schematic diagram of the adaptive feature modulation module in the medical image segmentation method based on self-supervised reconstruction assistance provided by the application;

[0036] Figure 4 The flowchart of step S2 of the medical image segmentation method based on self-supervised reconstruction assistance provided by the application is shown in the flowchart of step S2 of the medical image segmentation method based on self-supervised reconstruction assistance provided by the application;

[0037] Figure 5 The bidirectional cross fusion module in the medical image segmentation method based on self-supervised reconstruction assistance provided by the application is shown in the schematic diagram of the bidirectional cross fusion module in the medical image segmentation method based on self-supervised reconstruction assistance provided by the application;

[0038] Figure 6 The flowchart of step S3 of the medical image segmentation method based on self-supervised reconstruction assistance provided by the application is shown in the flowchart of step S3 of the medical image segmentation method based on self-supervised reconstruction assistance provided by the application. DETAILED DESCRIPTION

[0039] In order to better understand and implement, the technical solutions in the embodiments of the application will be clearly and completely described below in conjunction with the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.

[0040] The terms "comprising" and "having" and any variations thereof in the embodiments of the present application are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or apparatus that includes a list of steps or modules as non- limiting to those steps or modules that are expressly listed, but can include other steps or modules that are not expressly listed or inherent to such processes, methods, products or apparatus.

[0041] Embodiments of the present application disclose a medical image segmentation method based on self-supervised reconstruction assistance, which is implemented by a reconstruction subnetwork and a segmentation subnetwork as shown in the drawings. Figure 1 Under the premise of guaranteeing the semantic expression ability of the segmentation task, the detail perception ability of the model is enhanced by introducing the reconstruction task, which significantly enhances the performance of medical image segmentation.

[0042] Specifically, as shown in Figure 2 The method comprises the following steps:

[0043] Step S1, acquire a medical image, and input the medical image into a reconstruction subnetwork to separate the foreground and background of the medical image to obtain a foreground image and a background image; input the medical image, the foreground image and the background image into an encoder respectively to extract foreground features, background features and reconstruction features;

[0044] The medical image is input into the reconstruction subnetwork to separate the foreground and background of the medical image to obtain a foreground image and a background image .The segmentation mask image is marked as 1 for the foreground and 0 for the background, and the corresponding calculation method is:

[0045] (1)

[0046] (2)

[0047] Wherein, represents a pixel-by-pixel multiplication operation.

[0048] The foreground image and the background image are input into a foreground encoder and a background encoder respectively to extract foreground features and background features of each layer, and the process is as follows:

[0049] (3)

[0050] (4)

[0051] wherein, denotes the number of layers of the foreground encoder or the background encoder. Meanwhile, the medical image is input to the reconstruction decoder , the reconstruction feature of each layer is extracted , which can be represented as:

[0052] (5)

[0053] Step S2, based on the foreground feature, the background feature and the reconstruction feature, respectively generating multiplicative modulation parameters and additive modulation parameters, and adaptively modulating the reconstruction feature to obtain a modulated feature.

[0054] In the reconstruction decoder , the reconstruction feature output by each layer is introduced into an adaptive feature modulation module Figure 3 as shown in the following formula: to help more accurately capture the fine-grained information of the lesion target and enhance the foreground feature closely related to the segmentation task and the auxiliary background feature. Specifically, as shown in the following formula: Figure 4 includes the following steps:

[0055] Step S21, splicing the foreground feature and the background feature, the reconstruction feature to obtain a spliced feature;

[0056] The foreground feature , the background feature and the reconstruction feature are spliced in the channel dimension to obtain a spliced feature :

[0057] (6)

[0058] wherein, denotes a feature splicing operation. The foreground feature generally contains fine-grained information of the lesion area, such as texture, edge, etc. The background feature contains background structure information, which helps to distinguish the lesion from the surrounding tissue, and the reconstruction feature f combines global structure and local detail information, which is the feature expression learned by the reconstruction task. Splicing the above three features in the channel dimension integrates the fine-grained information of the lesion area, the background structure information and the global detail information into a comprehensive feature representation, which not only preserves the independent semantics of each type of feature, but also provides complete information input for subsequent global context modeling.

[0059] Step S22, performing a global average pooling operation on the spliced feature to obtain a global context feature vector :

[0060] (7)

[0061] wherein, represents a global average pooling operation.

[0062] Step S23, inputting the global context feature vector into the first convolution stream and the second convolution stream to obtain a multiplicative modulation parameter and an additive modulation parameter;

[0063] are input into two double-layer convolution streams respectively to generate a multiplicative modulation parameter and an additive modulation parameter respectively:

[0064] (8)

[0065] (9)

[0066] wherein, and represent a convolution layer with a 1×1 convolution kernel, represents a Sigmoid activation function to ensure that the modulation parameter is in the range of [0, 1], so as to realize controllable scaling of the channel response. The multiplicative modulation parameter is used to scale the feature channel weight in proportion, so as to suppress irrelevant information and highlight the response of the target region; the additive modulation parameter is used to introduce an offset in the feature, further compensating for the lack of local detail information, and improving the completeness and adaptability of the feature expression.

[0067] Step S24, adaptively modulating the reconstructed feature based on the multiplicative modulation parameter and the additive modulation parameter to obtain a modulation feature.

[0068] The reconstructed feature is adaptively modulated, and the corresponding modulation process is as follows:

[0069] (10)

[0070] In the modulation process, the reconstructed feature is first multiplied element by element with the multiplicative modulation parameter to adjust the channel weight, and then added element by element with the additive modulation parameter to complete the feature offset correction, so as to obtain a modulation feature with fine granularity and global information. The modulation feature effectively fuses the foreground feature, the background feature and the global context information, dynamically optimizes the channel distribution of the reconstructed feature, so that the reconstructed subnetwork has stronger perception and expression ability for the detail features such as texture, edge and local structure of the lesion region, and more comprehensively extracts the fine-grained feature information of the lesion region.

[0071] ​More, the reconstruction sub-network adopts a self-supervised training manner, and by minimizing the foreground reconstruction loss, the background reconstruction loss and the overall medical image reconstruction loss, the model can automatically learn the fine-grained information of the image, thereby providing high-fidelity detailed feature support for the segmentation task. The reconstruction loss function includes the foreground reconstruction loss, the background reconstruction loss and the medical image reconstruction loss. In the training process, the foreground feature and the background feature , the modulation feature are respectively reconstructed in the foreground decoder , the background decoder and the reconstruction decoder . Among them, each decoder is composed of three layers of convolution and up-sampling structure, which can gradually improve the spatial resolution while maintaining the integrity of the feature semantic, and avoid the loss of details caused by one-time up-sampling.

[0072] In the reconstruction process, the feature map is sequentially restored in spatial resolution by layer-by-layer convolution and up-sampling operation, and finally generates the foreground reconstruction image , the background reconstruction image and the overall medical reconstruction image consistent with the size of the input medical image:

[0073] = (11)

[0074] = (12)

[0075] = (13)

[0076] wherein, represents the foreground reconstruction image, represents the background reconstruction image and represents the medical reconstruction image. Through the above reconstruction, the decomposition reconstruction manner can respectively optimize the detailed expression of the foreground area and the background area, and retain the global structure information in the overall reconstruction, thereby improving the restoration ability of the edge contour and local details.

[0077] In order to further improve the reconstruction quality, the model is trained by jointly minimizing the reconstruction loss function to optimize the parameters of the reconstruction sub-network. The reconstruction loss function includes the foreground reconstruction loss , the background reconstruction loss and the medical image reconstruction loss :

[0078] (14)

[0079] (15)

[0080] (16)

[0081] wherein represents the L1 norm, used to measure the pixel-level absolute error between the reconstructed image and the original image. By jointly optimizing the medical segmentation sub-image according to the above loss, the modeling ability of local details and global structure can be balanced, and the feature deviation caused by a single loss function can be avoided. The above reconstruction loss adopts the L1 norm to measure the pixel-level absolute error between the reconstructed image and the original image, which can directly reflect the difference at the detail level, reduce the blurring tendency, and thus ensure a high-fidelity image restoration effect in the reconstruction task.

[0082] Step S3, inputting the medical image into the segmentation sub-network, extracting segmentation features, and performing bidirectional cross fusion of the segmentation features and the modulation features based on a cross attention mechanism to generate fusion features.

[0083] In the medical image segmentation process provided in the application, the low-level detail representation in the segmentation features is enhanced by using the fine-grained pixel-level features output by the segmentation sub-network. To further exert the complementary advantages between the segmentation features and the modulated reconstruction features, the embodiment fuses the reconstruction features containing global structure information and the segmentation features with fine-grained expression capability; wherein the reconstruction features provide global morphology and spatial layout, and the segmentation features retain high-level semantic information and position accuracy, and by fusion, global semantics and local details can be considered at the same time, thereby effectively improving the boundary recognition accuracy and overall segmentation performance.

[0084] Specifically, the medical image is input into the segmentation decoder , and the segmentation features of the first layer are extracted :

[0085] (17)

[0086] To realize deep fusion of the segmentation features and the modulated reconstruction features, the segmentation features and the modulation features are bidirectionally cross-fused based on a cross attention mechanism. In the embodiment, a bidirectional cross fusion module as shown in Figure 5 is introduced into each layer of the segmentation decoder in the segmentation sub-network, to establish interaction between different features, thereby improving the structure expression and semantic perception ability of the segmentation features. Specifically, as shown in Figure 6As shown, the segmentation features and the modulation features are bidirectionally cross-fused based on a cross-attention mechanism to generate fusion features, including:

[0087] In step S31, the modulation features are taken as query vectors, the segmentation features are taken as key vectors and value vectors, a first cross-attention map is calculated, and enhanced segmentation features are calculated.

[0088] Taking the modeling of the attention of the segmentation features guided by the modulation features as an example, first, the reconstructed features after modulation are mapped to query vectors , the segmentation features are mapped to key vectors and value vectors ; cross-attention dot product calculation is performed based on the query vectors and the corresponding key vectors to generate a first cross-attention map :

[0089] (18)

[0090] wherein denotes a feature dimension scaling factor, denotes a normalized distribution.

[0091] The enhanced segmentation features refer to the multiplication of the first cross-attention map and the corresponding value vectors , the feature is adjusted by a convolution layer with a convolution kernel of , and is added to the segmentation features through a residual connection. The obtained enhanced segmentation features are:

[0092] (19)

[0093] wherein, denotes a convolution layer with a convolution kernel of , and denotes the enhanced segmentation features. The fine-grained information in the reconstruction branch is used to enhance the detail expression ability of the segmentation features, a feature dimension scaling factor is introduced to balance the response amplitude of different channels, and a normalized distribution is introduced to ensure the stability of the attention weight.

[0094] In step S32, the segmentation features are taken as query vectors, and the modulation features are taken as key vectors and value vectors, a second cross-attention map is calculated, and enhanced reconstructed features are calculated.

[0095] Similarly, when the reconstructed features after modulation are modeled by the segmentation features, the segmentation features are mapped to query vectors , the modulation features are mapped to key vectors and value vectors , and a second cross-attention map is calculated. denotes the cross-attention map constructed by the reconstructed features guided by the segmentation features, i.e., the query vector generated by the segmentation features , and the key generated by the modulation reconstructed features The results obtained by cross-attention modeling. Accordingly, the second cross-attention map is multiplied by the corresponding value vector The convolution layer with a convolution kernel of is used to adjust the features, and the segmentation features are added through a residual connection to obtain the enhanced reconstructed features .

[0096] Step S33, the enhanced segmentation features and the enhanced reconstructed features are aggregated and residually spliced to generate fusion features.

[0097] The enhanced segmentation features and the enhanced reconstructed features further integrate complementary information at the fine-grained and structural levels, thereby forming a complete feature representation. This process can be represented as:

[0098] (20)

[0099] wherein, denotes two convolution layers with a convolution kernel of , and denotes the fusion features.

[0100] Step S4, outputting a segmentation result of the medical image based on the fusion features.

[0101] The fusion features are input into the segmentation head , and through layer-by-layer upsampling and convolution operations, the spatial resolution of the features is gradually restored, and finally a segmentation prediction result with the same size as the original input image is generated. This process can be represented as:

[0102] (21)

[0103] The medical segmentation subnetwork is trained by minimizing a segmentation loss function, wherein the segmentation loss function includes a binary cross-entropy loss and a Dice loss . Specifically,

[0104] (22)

[0105] wherein, denotes a segmentation training loss function, denotes a binary cross-entropy loss,​ representing Dice loss.

[0106] The segmentation loss function includes a binary cross-entropy loss, which directly compares the predicted probability with the real segmentation label at the pixel level, can provide a stable gradient signal, and is helpful for the model to converge quickly in the early training. Meanwhile, a Dice loss is introduced to measure the overlap degree of the predicted segmentation result and the real segmentation label, improve the boundary integrity and consistency of the segmentation result, and give consideration to the overall segmentation contour and pixel details in the training, thereby improving the adaptability and segmentation precision of the segmentation subnetwork on different scale targets.

[0107] The medical image segmentation method, device and medium based on self-supervised reconstruction assistance provided in the application extract differential features through foreground and background separation, generate multiplicative and additive modulation parameters for the reconstruction features based on the foreground features, background features and reconstruction features to adaptively modulate the reconstruction features, so as to enhance the fine-grained information related to the lesion; the segmentation features and the modulation features are bidirectionally fused through a cross-attention mechanism in the segmentation subnetwork, the complementarity of semantics and details is realized, the fusion features with global structure and local details are generated, the deficiency of the traditional segmentation method in low-level feature reservation is made up, and the performance of the medical image segmentation model in lesion target boundary recognition and segmentation precision is significantly improved.

[0108] Based on the same inventive concept, the application further provides a computer device, comprising a processor and a memory; wherein the memory stores a computer program, and the computer program is adapted to be loaded and executed by the processor to perform the steps of the medical image segmentation method based on self-supervised reconstruction assistance as above.

[0109] The processing method of the computer device can refer to the description of the above method, and will not be described here.

[0110] The application embodiment further provides a non-transitory machine-readable storage medium, which stores an executable program, and when the executable program is executed by a microprocessor, the processor executes the method provided by the above embodiment.

[0111] The computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program causes a computer to execute the described method.

[0112] The computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute the described method.

[0113] The above-described embodiments are merely illustrative for the present application, wherein the modules described as separate components can or can not be physically separated, and the components shown as modules can or can not be physical modules, i.e., can be located in one place or distributed to multiple network modules. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs, and those skilled in the art can understand and implement them without creative labor.

[0114] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software and necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software product, and the computer software product can be stored in a computer readable storage medium, including Read-Only Memory (ROM), Random Access Memory (RAM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), One-time Programmable Read-Only Memory (OTPROM), Electrically-Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, magnetic disk storage, magnetic tape storage or any other computer readable medium capable of carrying or storing data.

[0115] Finally, it should be noted that: the embodiments disclosed by the present application are only the preferred embodiments of the present application, and are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A medical image segmentation method based on self-supervised reconstruction assistance, characterized in that, The process is performed on a model that includes a reconstruction subnetwork and a segmentation subnetwork, wherein the reconstruction subnetwork is trained in a self-supervised manner, including: A medical image is acquired and input into a reconstruction subnetwork to separate the foreground and background of the medical image, resulting in a foreground image and a background image. The medical image, foreground image, and background image are then input into an encoder to extract foreground features, background features, and reconstructed features. Multiplicative modulation parameters and additive modulation parameters are generated based on the foreground features, background features, and reconstructed features, respectively, and the reconstructed features are adaptively modulated to obtain the modulation features; The medical image is input into the segmentation sub-network, segmentation features are extracted, and the segmentation features and modulation features are bidirectionally cross-fused based on the cross-attention mechanism to generate fused features. The segmentation result of the medical image is output based on the fusion features; Specifically, multiplicative modulation parameters and additive modulation parameters are generated based on the foreground features, background features, and reconstructed features, respectively, and adaptive modulation is performed on the reconstructed features to obtain modulation features, including: By splicing the foreground features with the background features and reconstructing the features, the spliced ​​features are obtained; Perform global average pooling on the concatenated features to obtain a global context feature vector; The global context feature vector is input into the first convolutional stream and the second convolutional stream to obtain the multiplicative modulation parameters and the additive modulation parameters; The reconstructed features are adaptively modulated based on the multiplicative modulation parameters and the additive modulation parameters to obtain modulation features, which are generated by the following formula: in, Indicates modulation characteristics; Features of splicing; These are multiplicative modulation parameters. These are additive modulation parameters. ; For global context feature vectors, , This indicates a global average pooling operation. , This indicates a feature concatenation operation. These are foreground features, background features, and reconstructed features, respectively. It is the Sigmoid activation function. This indicates a convolutional layer with two 1×1 kernels.

2. The medical image segmentation method based on self-supervised reconstruction assistance according to claim 1, characterized in that, The segmentation features and modulation features are bidirectionally cross-fused based on a cross-attention mechanism to generate fused features, including: Using modulation features as query vectors and segmentation features as key and value vectors, calculate the first cross-attention map and then calculate enhanced segmentation features. Using segmentation features as query vectors and modulation features as key and value vectors, calculate the second cross-attention map and then calculate the enhanced reconstruction features. Enhanced segmentation features and enhanced reconstruction features are aggregated and residuals are spliced ​​to generate fused features.

3. The medical image segmentation method based on self-supervised reconstruction assistance according to claim 2, characterized in that, The feature aggregation uses a 3×3 convolutional layer to adjust the enhanced segmentation features and enhanced reconstruction features. The adjusted enhanced segmentation features and enhanced reconstruction features are then connected through residual connections to maintain the original semantic information and generate fused features.

4. The medical image segmentation method based on self-supervised reconstruction assistance according to claim 1, characterized in that, The segmentation results of the medical image are output based on the fusion features, including: The fusion feature input segmentation head restores spatial resolution through layer-by-layer upsampling and convolution operations, and outputs segmentation prediction results of the same size as the medical image.

5. The medical image segmentation method based on self-supervised reconstruction assistance according to claim 1, characterized in that, The segmentation subnetwork is trained and optimized by minimizing the segmentation training loss function, which includes binary cross-entropy loss and Dice loss.

6. The medical image segmentation method based on self-supervised reconstruction assistance according to claim 1, characterized in that, The reconstruction subnetwork is trained by minimizing the reconstruction loss function; the reconstruction loss function includes foreground reconstruction loss, background reconstruction loss and medical image reconstruction loss.

7. A computer device, characterized in that, include: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the steps of the self-supervised reconstruction-assisted medical image segmentation method as claimed in any one of claims 1-6.

8. A computer storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements the steps of the self-supervised reconstruction-assisted medical image segmentation method as described in any one of claims 1-6.

Images