Adaptive mask medical image segmentation method based on self-supervised mask and deep reinforcement learning

Abstract

The invention discloses an adaptive mask medical image segmentation method based on a self-supervised mask and deep reinforcement learning, and the method comprises the steps: employing a classic encoder-decoder architecture for a self-supervised mask reconstruction network, fusing a Swin Transform encoder, and carrying out the feature fusion of local image blocks through a self-attention mechanism; according to the self-adaptive mask model, a PPO deep reinforcement learning algorithm is adopted, a strategy network and a value network are constructed, mask actions are dynamically regulated and controlled, reconstruction errors are gradually reduced, a mask strategy is continuously optimized in multiple times of strategy updating for self-adaptive optimization, and high-quality reconstruction of a medical image influenced by missing information is achieved; according to the method, high-quality feature representation can be obtained in an unlabeled data environment, and relatively high precision and accuracy are presented on a public data set.

Description

Technical Field

[0001] The present invention relates to the fields of medical image segmentation and artificial intelligence, and in particular to an adaptive mask medical image segmentation method based on self-supervised masking and deep reinforcement learning. Background Art

[0002] In recent years, medical image segmentation technology has played an increasingly important role in clinical diagnosis, surgical planning, treatment design, and other fields. Medical image segmentation aims to accurately separate different anatomical structures or lesion areas in medical images from the background, providing a basis for subsequent quantitative analysis and auxiliary diagnosis.

[0003] Traditional image segmentation methods, such as threshold segmentation, region growing, and graph cuts, can, to a certain extent, extract target regions. However, they often rely on prior knowledge and manual parameter adjustment, making them difficult to handle complex and variable medical imaging data. With the rapid development of deep learning technology, segmentation models based on convolutional neural networks have become the mainstream method for medical image segmentation. These methods can automatically learn the hierarchical features of images, significantly improving segmentation accuracy and robustness. However, medical image data often suffers from problems such as limited sample size, high data annotation costs, and diverse lesion morphology. The annotation cost is much higher than that of natural images, and there is a large amount of unlabeled data. Therefore, fully supervised medical image segmentation is often unrealistic.

[0004] In order to alleviate the problems of high labeling costs and data scarcity, weakly supervised and semi-supervised segmentation methods have emerged. Weakly supervised methods use weak labels provided by point annotations, box annotations, or rough outlines to improve model performance through multi-task learning or pseudo-label self-training. Semi-supervised methods combine a small amount of labeled data with a large amount of unlabeled data, and improve the network's generalization ability for unlabeled samples through strategies such as consistency regularization and generative adversarial networks. However, weak supervision signals are usually too sparse to accurately guide pixel-level segmentation. The noise of pseudo-labels will accumulate during the training process, and the model is prone to falling into suboptimal solutions. Semi-supervised methods are sensitive to the quality of the initial labeled samples, and when the distribution of labeled data is highly biased, the generalization effect is poor.

[0005] In order to address the core bottleneck of insufficient labeled data and fully explore the potential information in massive amounts of unlabeled medical images, self-supervised learning has gradually been introduced into segmentation tasks in recent years. Self-supervised learning forces the network to extract useful feature representations from the data by designing pre-tasks such as contrastive learning and mask modeling. These pre-trained features often play a good transfer role in downstream tasks. However, contrastive learning methods can only learn global representations but lack the ability to model local missing information, which affects the accuracy of downstream segmentation tasks. The position and size of the mask in current mask modeling strategies are either fixed or completely random. They rarely consider the individual differences between different patients and different anatomical structures in medical images and lack the ability to adaptively adjust the content of a single image. Summary of the Invention

[0006] The present invention aims to solve the technical pain points of existing medical image segmentation methods, such as high dependence on large-scale accurately labeled data, insufficient modeling of fine-grained lesion areas, and difficulty in flexibly adjusting the accuracy-efficiency trade-off in the inference stage. An adaptive mask medical image segmentation method based on self-supervised mask and deep reinforcement learning is proposed. This method combines deep reinforcement learning with self-supervised mask modeling. Self-supervised mask modeling learns general visual representations through large-scale unlabeled images, and deep reinforcement learning learns local masking strategies under the guidance of the deep reinforcement learning algorithm (PPO). Ultimately, the downstream migration of medical image segmentation tasks can be robustly adapted to medical segmentation tasks in multimodal, weakly labeled, and even zero-labeled scenarios.

[0007] The adaptive mask medical image segmentation method based on self-supervised mask and deep reinforcement learning in the present invention is as follows:

[0008] 1. Data preprocessing

[0009] Perform preprocessing operations such as enhancement, transformation, and cropping on the original medical image to obtain preprocessed medical image data X∈R of uniform dimension C×H×W (C represents the number of channels, H represents the height, and W represents the width), specifically including performing data enhancement and transformation operations on the image, uniformly cropping the 3D samples to a certain size, and then cropping each 3D sample into several 2D slices;

[0010] 2. Self-supervised mask reconstruction network model training

[0011] This method uses self-supervised learning to restore medical images affected by missing information. It adopts a classic encoder-decoder architecture and integrates the Swin Transformer encoder to efficiently model local image details and global semantic information.

[0012] Considering the computational complexity of image segmentation and facilitating the network's capture of local information, we first constructed an adaptive mask network model based on the deep reinforcement learning PPO algorithm. This model includes Patch Embedding, a Swin Transformer encoder, and a symmetric decoder. We then used a self-supervised mask reconstruction network model to train the pre-processed medical image data from step 1.

[0013] 2.1 Preprocessed input image X∈R C×H×W Perform Patch Embedding. The Patch Embedding layer divides the original input image into fixed-size image patches through a convolutional layer. The convolution kernel size and stride are both set to (p, p). The formula is as follows:

[0014] f = Conv(X; W c ,b c ))∈R E×H′×W′

[0015] Where E is the number of output channels, W c is the convolution kernel weight, b c is the bias term, the original image is divided into patches of size (p, p), and each patch is represented by an E-dimensional vector in the high-dimensional space;

[0016] 2.2 Embedding features f∈R of the image using Swin Transformer encoder E×H′×W′ Perform feature extraction to obtain low-resolution global features f t

[0017] Specifically, the image blocks in step 2.1 are randomly sampled globally according to the mask ratio of 60-80%, 60-80% of the image blocks are masked, and the unmasked image blocks are sent to the Swin Transformer encoder. The embedded features f of the sent image blocks are divided into local windows (p′, p′), each window contains N patches (N=p′×p′), and for the feature X in each window w ∈R N×E , N represents the number of patches. The output of each patch obtained through the self-attention mechanism not only contains its own information, but also integrates the context information of other patches in the window. The calculation formula is as follows:

[0018]

[0019] Among them, Q, K, and V are respectively represented by X w Obtained by linear mapping of the fully connected layer, d k is the dimension of each attention head, is a scaling factor used to prevent the dot product from being too large, and T is the transpose operation.

[0020] To enhance global information modeling capabilities, the Swin Transformer encoder also employs a shifted window strategy. In some Transformer layers, features within a local window are shifted by a certain amount, allowing patches originally located in different windows to participate in self-attention calculations together after the shift, enabling cross-window information interaction.

[0021] In each local window, Attention(X w ) and X w Perform residual addition and LayerNorm normalization processing, then pass through two layers of MLP feedforward network, perform residual addition and LayerNorm normalization processing again to complete a deep fusion. After multiple layers of deep fusion and several times of downsampling, the encoder finally outputs a low-resolution global feature f t ∈R E×H′×W′ ;

[0022] 2.3 Symmetric decoder uses transposed convolution to transform low-resolution global features f t ∈R E×H′×W′ Upsampling is restored to the original image size. The convolution kernel size and stride of the transposed convolution layer are consistent with those of the Patch Embedding layer to ensure that the output mask reconstructed medical image after upsampling

[0023] The training objective of the model is to minimize the reconstruction loss:

[0024]

[0025] Where: N is the number of image blocks, i = 1, 2, 3..., X i is the i-th preprocessed image, is the medical image reconstructed by the i-th mask;

[0026] The model not only needs to output a reconstructed image but also generate auxiliary mask information, which has a positive impact on downstream segmentation tasks. Traditional hard thresholding operations are often non-differentiable, which can easily lead to gradient breakage and affect end-to-end training. To this end, the model introduces the Gumbel-Softmax differentiable discretization method to convert continuous mask logits into soft one-hot representations. The specific process is as follows:

[0027]

[0028] Where π is the logits distribution of the model output, g is the noise sampled from the Gumbel (0, 1) distribution, and τ is the temperature parameter, which is gradually annealed during the training process. As τ is gradually reduced, the output y will tend to be discrete, but still maintain differentiability, ensuring that the gradient can be smoothly transferred, ensuring stable training of the model, and obtaining the trained self-supervised mask reconstruction network model;

[0029] 3. Adaptive mask segmentation model training

[0030] An adaptive mask network model based on the deep reinforcement learning PPO algorithm is constructed. The model includes a feature extraction network, a policy network, and a value network. Based on the adaptive reconstruction network, deep reinforcement learning is used to automatically learn the local area masking strategy, thereby improving the segmentation effect and reducing the dependence on labeled data.

[0031] 3.1 Using the feature extraction network to obtain the low-resolution global features f of the image blocks extracted from the supervised mask reconstruction network model training t ∈R E×H′×W′ , from the low-resolution global feature f t Extract the local embedding vector s∈R corresponding to each image block E ; In order to obtain the global state information of the region, a global average pooling operation is used to fuse the spatial dimension information into a fixed-dimensional state vector. The embedding of all voxels in a patch can be expressed as:

[0032]

[0033] Among them, f t (i) represents the E-dimensional feature vector of the position of the i-th voxel, N p Indicates the total number of voxels contained in the patch;

[0034] 3.2 The policy network is used to map the local embedding vector s to the action probability space through the fully connected layer, and finally the network output is normalized into a probability distribution through the softmax function normalization:

[0035] π(s)=Softmax(W p s+b p )

[0036] Where: π(s) represents the probability of two possible actions, action 0 means retaining the information of the patch; action 1 means applying a mask to the patch, W p is the policy network weight, b p is the policy network bias term;

[0037] 3.3 Use the value network to evaluate the state of the local embedding vector s of each patch and predict the value corresponding to the state under the current strategy. The value network structure consists of only one fully connected layer, and the formula is:

[0038] V(s)=W v s+b b

[0039] V(s) is used to measure the importance of the patch in the overall reconstruction task and guide the strategy update. b is the value network weight, b v is the value network bias term.

[0040] 3.4 Adaptive Mask Network Model Using PPO Algorithm to Update Policy Network and Value Network

[0041] By using the PPO algorithm, the model adopts a ratio clipping objective function when updating the policy network parameters. The objective function is defined as:

[0042] L p =-E[min(r(θ)A(s,a),clip(r(θ),1-∈,1+∈)A(s,a))]

[0043] Where clip represents a clipping operation, which limits r(θ) to the interval [1-∈, 1+∈]. The policy ratio r(θ) is defined as:

[0044]

[0045] r(θ) represents the ratio of the probability of executing action a under the current policy parameters θ to the probability under the old policy parameters.

[0046] π θ (a|s) represents the probability of executing action a under the current policy parameters θ; represents the old policy parameter θ old The probability of executing action a is obtained, and the advantage function A(s,a) is calculated by generalized advantage estimation (GAE):

[0047] A(s t ,a t )=δ t +(γλ)δ t+1 +…,δ t =r t +γV(s t+1 )-V(s t )

[0048] δ t It reflects the deviation between the actual reward and the predicted value after transferring from the current state to the next state.t If it is positive, it means that the actual reward is higher than expected, otherwise it is lower than expected. t is the reward observed at time step t, V(s t ) represents state s t is the value estimate of , γ is the discount factor, and λ is the parameter that controls the smoothness of the multi-step advantage estimate.

[0049] 3.5 Use the updated policy network and value network to perform a sampling action a (0 or 1) on each image patch according to the probability distribution π(s) generated by the policy network, and use the sampling result to control whether to apply a mask operation to the patch;

[0050] After the patch is restored to the complete image through the trained self-supervised mask reconstruction network model, a reconstruction error will be generated. To encourage the model to adaptively select masks based on local information while preventing over-masking, this study uses the maximization of the reward function as the training objective during training. The trained self-supervised mask reconstruction network model is then used to train medical images after masking to obtain medical images reconstructed with dynamic masks.

[0051]

[0052] Where N is the number of image blocks, i = 1, 2, 3, ..., α is the penalty coefficient that controls the cost of the mask operation, X i is the i-th preprocessed image, is the medical image reconstructed by the i-th mask;

[0053] 4. Linearly map the channel features of each pixel in the medical image reconstructed by the dynamic mask in step 3 through a 1×1 convolution branch, and calculate the probability p of each pixel for different categories through the Softmax function in the channel dimension. i ;

[0054] When the segmentation type is binary classification, the Dice coefficient is used as the target, and a grid search is performed in the interval [0, 1] with a step size of 0.01 to select the global threshold τ with the best performance. i , the probability p of each pixel corresponding to the category i With the global threshold τ i For comparison, if p i ≥τ i , then the pixel is located in the target area (such as a tumor, lesion or a specific organ), otherwise the pixel is located in the background. The pixels in the target area are counted to obtain the medical image segmentation result;

[0055] When the segmentation type is multi-classification, the category probability p obtained by calculating the pixels for multiple types isi The category corresponding to the largest probability is taken as the target type of the pixel, and the categories of all pixels are counted to obtain the medical image segmentation result.

[0056] Compared with the prior art, the present invention has the following advantages:

[0057] (1) Using self-supervised mask reconstruction to release the value of massive unlabeled images and reduce dependence on manual labeling;

[0058] (2) Through PPO learning, an adaptive masking strategy is adopted, which enables the model to dynamically balance speed and accuracy under different task and device constraints;

[0059] (3) The window shift mechanism and multi-scale upsampling design of the Swin Transformer encoder take into account both local boundary details and global anatomical semantics, improving the robustness of lesion segmentation;

[0060] (4) The mask penalty term in the reward function suppresses meaningless over-occlusion and significantly improves the convergence stability of the model;

[0061] (5) This method can be seamlessly integrated into mainstream medical imaging workstations and has good generalization and deployment feasibility for three-dimensional multimodal data such as CT, MRI, and PET-CT.

[0062] In summary, the present invention provides an effective path to deeply integrate the representation ability of self-supervised learning with the decision-making ability of deep reinforcement learning. It is aimed at the medical image segmentation task with scarce annotations and complex lesions in real clinical environments, significantly improving the segmentation accuracy and resource utilization efficiency, and has significant academic value and industrial promotion prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 is a 2D schematic diagram of the preprocessed brain tumor image;

[0064] Figure 2 is a 2D schematic diagram of the medical image after mask reconstruction;

[0065] Figure 3 is a schematic diagram of applying a mask operation;

[0066] Figure 4 Schematic diagram of medical image segmentation results. DETAILED DESCRIPTION

[0067] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention. Unless otherwise specified, the methods in the embodiments are all conventional methods.

[0068] Example 1: Adaptive mask medical image segmentation method based on self-supervised mask and deep reinforcement learning

[0069] 1. This example uses the public BraTS dataset, which is a 3D brain tumor segmentation dataset. It includes 484 3D sample data, including four major region annotations: background, enhancement core, necrotic / non-enhancement core, and water. The 3D samples are uniformly cropped to 192×192×144, and each 3D sample is generated into 144 2D sample data of size 192×192 according to the axis of the data ( Figure 1 );

[0070] In order to achieve unified training and evaluation, this study divided the integrated data into a training set (80%), a validation set (10%), and a test set (10%). In order to improve the stability of the experimental results, the entire data division process adopted a five-fold cross-validation and repeated the experiment multiple times to obtain reliable evaluation indicators.

[0071] This study was conducted under the Ubuntu 22.04 operating system. The hardware environment included an Intel Xeon Gold 5218 CPU, 256GB of memory, and three NVIDIA RTX 3090 graphics cards (24GB of video memory each). The deep learning framework used was PyTorch 2.0, and the programming language was Python 3.12.

[0072] 2. Build an adaptive mask network model based on the deep reinforcement learning PPO algorithm. This network adopts the "convolutional patch embedding + Swin Transformer encoder + symmetric decoder" architecture and uses a self-supervised mask reconstruction network model to train the medical image data pre-processed in step 1. Specifically:

[0073] 2.1 Preprocessed input image X∈R 1×192×192 Perform Patch Embedding. The PatchEmbedding layer divides the original input image into fixed-size image patches through a convolutional layer. The convolution kernel size and stride are both set to (16, 16). The formula is as follows:

[0074] f = Conv(X; WC ,b C ))∈R E×H′×W′

[0075] Where E is the number of output channels, W C is the convolution kernel weight, b C is the bias term, the original image is divided into patches of size (16,16), Each patch is represented by a 128-dimensional vector in high-dimensional space;

[0076] 2.2 A fixed mask ratio of 75% is used to perform global random sampling on the embedded features f, 75% of the patch masks are blocked, and only 25% of the embedded features are retained and fed into the Swin Transformer encoder. The fixed mask ratio of 75% ensures that the network always blocks the same proportion of areas in each input, and under stable mask conditions, the model gradually learns how to fill in the missing information; the embedded features are extracted and shifted self-attention is implemented in the local window, taking into account both local fine texture and cross-window global context. The features output by the Transformer module are fused at multiple layers to obtain the low-resolution global feature f t ∈R 128×12×12 ;

[0077] 2.3 Symmetric decoder uses transposed convolution to transform low-resolution global features f t ∈R 128×12×12 Upsampling is performed to restore the original image size and reconstruct the image damaged by the random mask. The convolution kernel size and stride of the transposed convolution layer are consistent with those of the PatchEmbedding layer to ensure that the output mask reconstructed medical image after upsampling ( Figure 2 ), and obtain the trained self-supervised mask reconstruction network model at the same time;

[0078] The training objective of the model is to minimize the reconstruction loss:

[0079]

[0080] Where: N is the number of image blocks, i = 1, 2, 3..., X i is the i-th preprocessed image, is the medical image reconstructed by the i-th mask;

[0081] 3. Build an adaptive mask network model based on the deep reinforcement learning PPO algorithm. The model includes feature extraction network, policy network, and value network.

[0082] 3.1 Using the feature extraction network to obtain the low-resolution global features f of the image blocks extracted from the supervised mask reconstruction network model training t∈R 128×12×12 , from the low-resolution global feature f t Extract the local embedding vector s∈R corresponding to each image block 128 ; In order to obtain the global state information of the region, a global average pooling operation is used to fuse the spatial dimension information into a fixed-dimensional state vector. The embedding of all voxels in a patch can be expressed as:

[0083]

[0084] 3.2 The policy network is used to map the local embedding vector s to the action probability space through the fully connected layer, and finally the network output is normalized into a probability distribution through the softmax function normalization:

[0085] π(s)=softmax(W p s+b p )

[0086] Where: π(s) represents the probability of two possible actions, action 0 means retaining the information of the patch; action 1 means applying a mask to the patch, W p is the policy network weight, b p is the policy network bias term;

[0087] 3.3 Use the value network to evaluate the state of the local embedding vector s of each patch and predict the value corresponding to the state under the current strategy. The value network structure consists of only one fully connected layer, and the formula is:

[0088] V(s)=W v s+b v

[0089] V(s) is used to measure the importance of the patch in the overall reconstruction task and guide the strategy update. v is the value network weight, b v is the value network bias term.

[0090] 3.4 Adaptive Mask Network Model uses the PPO algorithm to update the policy network and value network. The updated policy network and value network are used to perform sampling action a (0 or 1) on each image block according to the probability distribution π(s) generated by the policy network, and the sampling result is used to control whether to apply a mask operation to the patch ( Figure 3 ); The random mask rate is set to 30%. By randomizing the mask position and ratio, the model generalization ability is enhanced and the model overfitting is prevented;

[0091] After the patch is restored to the complete image through the trained self-supervised mask reconstruction network model, a reconstruction error will be generated. To encourage the model to adaptively select masks based on local information while preventing over-masking, this study uses the maximization of the reward function as the training objective during training. The trained self-supervised mask reconstruction network model is then used to train medical images after masking to obtain medical images reconstructed with dynamic masks.

[0092]

[0093] Where N is the number of image blocks, i = 1, 2, 3, ..., α is the penalty coefficient that controls the cost of the mask operation, X i is the i-th preprocessed image, is the medical image reconstructed by the i-th mask;

[0094] 4. Linearly map the channel features of each pixel in the medical image reconstructed by the dynamic mask in step 3 through a 1×1 convolution branch, and calculate the category probability p of each pixel in the channel dimension through the Softmax function. i ;

[0095] In this example, the segmentation type is tumor and background. The Dice coefficient is used as the target. A grid search is performed in the interval [0, 1] with a step size of 0.01 to select the best performing global threshold τ i = 0.5, the category probability p of each pixel i With the global threshold τ i For comparison, if p i ≥τ i , then the pixel is located in the tumor area (marked with color in the figure, and the color from blue to red corresponds to the prediction probability from low to high), otherwise the pixel is located in the background (marked with black in the figure), and the pixels in the target area are counted to obtain the medical image segmentation result ( Figure 4 ),Should Figure 4 The results were consistent with the actual tumor area;

[0096] 5. The method of the present invention tried different annotation scales and models in the downstream image segmentation task, and used standard MAE and U-net as comparative experiments. MAE learns global semantic features in a self-supervised manner by performing a large-scale random mask on the image, encoding only the visible blocks and reconstructing the masked area with a lightweight decoder. U-net consists of a symmetric convolutional encoder-decoder and a cross-layer jump connection, which can efficiently fuse multi-scale features and achieve fine pixel-level segmentation of medical images. At the same time, a self-supervised mask (denoted as Random in Table 1) with the random mask of the PPO algorithm removed was used as an ablation experiment; the experimental results were measured using DSC and IoU. DSC evaluates the similarity between the image segmentation result and the true label; IoU measures the overlap between the predicted mask and the true mask. The formula is as follows:

[0097]

[0098]

[0099] Among them, |Y|, are the total number of pixels in the true and predicted regions, respectively. is the number of intersection pixels between the real and predicted pixels, is the number of pixels in the union of the true and predicted pixels. The experimental results of the present invention achieved the best results on 5%, 10%, and 100% labeled data. DSC and DSC achieved 0.707 and 0.614 on 5% labeled data, respectively. They also outperformed other models on 10% and 100% labeled data, as shown in Table 1.

[0100] Table 1 Comparison between other models and the present invention

[0101]

Claims

1. An adaptive mask medical image segmentation method based on self-supervised masking and deep reinforcement learning, characterized by: Here are the steps: (1) Enhance, transform, and crop the medical image to obtain pre-processed medical image data with unified data dimensions; (2) constructing a self-supervised mask reconstruction network model based on a Swin Transformer encoder, and using the self-supervised mask reconstruction network model to train the medical image data pre-processed in step (1), thereby obtaining a medical image after mask reconstruction and a trained self-supervised mask reconstruction network model; (3) Construct an adaptive mask network model based on deep reinforcement learning PPO algorithm. The model includes feature extraction network, policy network and value network. The feature extraction network is used to obtain the low-resolution global features f of the image blocks extracted from the training of the self-supervised mask reconstruction network model. t , from the low-resolution global feature f t Extract the local embedding vector s corresponding to each image block; use the policy network to generate the probability distribution of the mask action for the local embedding vector s of the image block, and use the value network to evaluate the state of the local embedding vector s of the image block and predict the value corresponding to the state under the current policy; The adaptive mask network model uses the PPO algorithm to update the policy network and value network, and uses the updated policy network and value network to control the sampling action of each image block to decide whether to apply a mask operation to the image block; After the mask operation is completed, the self-supervised mask reconstruction network model trained in step (2) is used to train the medical image after the mask operation is applied to obtain a medical image after dynamic mask reconstruction; (4) Linearly map the channel features of each pixel in the medical image reconstructed by the dynamic mask in step (3) through a 1×1 convolution branch, and calculate the probability p of each pixel for different categories through the Softmax function in the channel dimension. i ; When the segmentation type is binary classification, the Dice coefficient is used as the target, and a grid search is performed in the interval [0, 1] with a step size of 0.01 to select the global threshold τ with the best performance. i , the probability p of each pixel corresponding to the category i With the global threshold τ i For comparison, if p i ≥τ i , then the pixel is in the target area, otherwise the pixel is in the background, and the pixels in the target area are counted to obtain the medical image segmentation result; When the segmentation type is multi-classification, the probability p of the pixel is calculated for multiple types. i The category corresponding to the largest probability is taken as the target type of the pixel, and the categories of all pixels are counted to obtain the medical image segmentation result.

2. The adaptive mask medical image segmentation method based on self-supervised mask and deep reinforcement learning according to claim 1, characterized in that: The following method is used for training to obtain the medical image after mask reconstruction: (1) PatchEmbedding, a self-supervised mask reconstruction network model based on the Swin Transformer encoder, is used to divide the preprocessed image X into image blocks of fixed size and obtain the embedded features f of the image blocks, where E represents that each image block is represented by an E-dimensional vector in high-dimensional space; (2) Perform global random sampling of the image blocks in step (1) at a mask ratio of 60-80%, mask 60-80% of the image blocks, and send the unmasked image blocks to the Swin Transformer encoder. The Swin Transformer encoder extracts the embedded features f of the image blocks to obtain low-resolution global features f t ; (3) The low-resolution global feature f t Input symmetric decoder, upsample to restore to the preprocessed image size, and output mask reconstructed medical image 3. The adaptive mask medical image segmentation method based on self-supervised mask and deep reinforcement learning according to claim 2, characterized in that: The training goal during training is to minimize the reconstruction loss: Where: N is the number of image blocks, i = 1, 2, 3..., X i is the i-th preprocessed image, is the medical image reconstructed by the i-th mask.

4. The adaptive mask medical image segmentation method based on self-supervised mask and deep reinforcement learning according to claim 1, characterized in that: The training objective in step (3) is to maximize the reward function: Where: N is the number of image blocks, i = 1, 2, 3, ..., α is the penalty coefficient that controls the cost of the mask operation, X i is the i-th preprocessed image, is the medical image reconstructed by the i-th mask.

5. The adaptive mask medical image segmentation method based on self-supervised mask and deep reinforcement learning according to claim 1, characterized in that: The probability distribution of mask actions is: π(s)=Softmax(W p s+b p ) Where: π(s) represents the probability of two possible actions, action 0 means retaining the information of the patch; action 1 means applying a mask to the patch, W p is the policy network weight, b p is the policy network bias term.

Images