A weakly supervised medical image segmentation method based on causal invariance

Abstract

The application discloses a kind of weak supervision medical image segmentation method based on causality invariance, the application uses two-stage training process, the first stage is label generation, and the picture-level label is generated pixel-level pseudo-label by class activation mapping mechanism, solve the problem that existing algorithm highly depends on a large number of artificial marking pixel-level label.The second stage is segmentation model training, and a segmentation model is trained using the pixel-level pseudo-label output in the first stage, to provide effective segmentation results for the actual clinical application of medical image analysis.Furthermore, a multi-branch feature extraction network is designed in the label generation stage of the application, and a causality invariance conversion strategy is proposed, so that the model can better extract the features of the same target in different states.Meanwhile, the application designs a loss function for different branch networks to ensure the classification accuracy of the model.Finally, the application designs a causality invariance constraint loss between multiple branches to ensure the segmentation accuracy of the model.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image analysis and processing, and in particular relates to a weakly supervised medical image segmentation method based on causal invariance. Background Art

[0002] Deep learning methods have been widely used in computer vision tasks for medical imaging. In particular, semantic segmentation of human organs or lesions in medical imaging data is a crucial step in clinical diagnosis and treatment.

[0003] Existing semantic segmentation methods based on deep learning are highly dependent on large amounts of pixel-level annotated data. However, due to the unclear boundaries of targets in medical images, determining the target location and outline requires a high level of professional knowledge from the annotators. In addition, pixel-by-pixel annotation is time-consuming and labor-intensive. As a result, academia and industry have begun to propose using weakly supervised learning methods to complete the semantic segmentation of medical images. This type of task is called weakly supervised semantic segmentation. In weakly supervised semantic segmentation tasks, the annotated data that the model training relies on does not need to be pixel-level annotations, but rather weakly labeled data that is easier to obtain. Common weakly labeled data include image-level annotations, points, scribbles, and bounding rectangles. Among them, image-level annotation only requires providing the target category information contained in the image, which is the simplest to obtain. Therefore, most weakly supervised semantic segmentation work revolves around image-level annotated data.

[0004] Existing weakly supervised semantic segmentation methods based on image-level annotations generally employ the Class Activation Mapping (CAM) technique to map image-level annotations to pixel-level annotations. These CAM-based segmentation methods use regions highly correlated with a specific class as approximate estimates of the target region corresponding to that class. However, correlation does not equate to causality, and the approximate estimated regions are not the true target class regions. This leads to significant errors in the form of large numbers of false positives and false negatives. False positives arise when certain background regions are highly correlated with the target foreground, causing the model to mistakenly identify these background regions as belonging to the foreground. False negatives arise when the classification model tends to focus on salient regions of the target, resulting in activation regions that fail to fully cover the target region. Both of these situations can lead to significant errors in the pseudo-segmentation labels generated based on the target class activation regions, which can be further amplified in segmentation models based on these pseudo-segmentation labels. Therefore, establishing a causal relationship between image-level labels and segmentation results to reduce the accuracy bias of pseudo-segmentation labels caused by correlation is a major research topic in weakly supervised medical image segmentation. Summary of the Invention

[0005] The purpose of the present invention is to provide a weakly supervised medical image segmentation method based on causal invariance to overcome the shortcomings of the existing technology.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A weakly supervised medical image segmentation method based on causal invariance includes the following steps:

[0008] Step 1: Pre-train the classification model; use the image data in the training dataset and its corresponding image-level category annotation data to train the classification model;

[0009] Step 2: Expand the input image data; perform a causal-invariant transformation operation on each input original image to obtain multiple transformed images;

[0010] Step 3: Train a multi-branch feature extractor. Design a multi-branch feature extraction network, including a base branch and multiple conversion branches. Use the original input image and the converted image obtained in step 2 as input to the base branch and the conversion branch, respectively, to extract different forms of features from the same image. At the same time, record the conversion parameters of each conversion branch compared to the original image.

[0011] Step 4: Calculate the multi-branch classification loss. Based on the multi-branch feature extraction network in step 3, design different classification head networks for the base branch where the original input image is located and the conversion branch where the converted image is located. Predict the classification prediction probability of the input image of each branch and calculate the cross entropy loss value between the prediction result and the image-level category labeling in step 1.

[0012] Step 5: Calculate the multi-branch class activation features; use the different forms of features obtained in step 3 and the model parameters of the different classification head networks obtained in step 4 to calculate the target category activation features of each branch;

[0013] Step 6: Calculate the causal invariance constraint loss between multiple branches. Use the image conversion parameters saved in step 3 to perform the inverse transformation of the causal invariance transformation on the class activation features of the converted branches obtained in step 5, and calculate the distance between each inverse transformed class activation feature and the class activation feature of the base branch as the causal invariance constraint loss between multiple branches.

[0014] Step 7: Jointly optimize the multi-branch feature extraction network model; design a joint optimization loss function, use a gradient descent strategy to simultaneously backpropagate the multi-branch classification loss in step 4 and the causal invariance constraint loss in step 6, and end the optimization process by iterative optimization until the loss value converges to a minimum value. The model parameters of the basic branch are saved to form a pseudo-label generation model for medical images.

[0015] Step eight, generating pseudo segmentation labels; based on the pseudo label generation model in step seven, the input picture is executed in the basic branch process to generate a class activation feature map with causal invariance constraints, and a binary map is generated using thresholding operation according to the pixel value in the feature map, 0 represents background, and 1 represents foreground as a pseudo segmentation label;

[0016] Step nine, training the segmentation model; the picture data in step one and the pseudo segmentation label obtained in step eight are used as training data together to train a segmentation model; the segmentation model encodes the input picture, and the decoding operation finally outputs the probability that each pixel point in the picture belongs to the target category; in the training stage, the gradient descent strategy is used to make the predicted segmentation result infinitely close to the pseudo segmentation label as the target, and iterative optimization is performed until the model converges; the model parameters are saved as the segmentation model of the medical image.

[0017] Further improvement of the application is that in step one, the classification network used by the pre-trained classification model is composed of a feature extraction network and a classification head network; wherein the feature extraction network is composed of five convolutional layers, the first convolutional layer is composed of a convolution operation with a kernel size of 7x7 and a step of 2, a batch normalization operation, a ReLU operation and a maximum pooling operation with a kernel size of 3x3 and a step of 2; the last four convolutional layers are composed of repeated splicing of multiple basic operation units, and the number of repeated splicing is 3, 4, 6 and 3 respectively; each basic operation unit is composed of three convolutional operations, three batch normalization operations, three ReLU operations and a residual connection operation; the classification head network is composed of a global average pooling operation and a fully connected operation with a parameter quantity of 2048x1.

[0018] Further improvement of the application is that in step one, the picture data in the training data set is a gray scale image or an RGB picture converted from multiple modal medical image data, including magnetic resonance images, computed tomography images and ultrasound images; the class label information used in the training data set is whether the input picture contains the target region belonging to the category, if it contains, it is marked as 1, and if it does not contain, it is marked as 0; wherein the target region is a specific organ region or a specific lesion region.

[0019] Further improvement of the application is that in step two, the causal invariant conversion operation used has segmentation invariance, and its formula is represented as:

[0020] f Φ (f Tr (I))=f Tr (f Φ (I))

[0021] Wherein, f Φ represents the process of extracting class activation features from the input picture, f Trrepresents a causal-invariant transformation operation, and I represents the input image data.

[0022] A further improvement of the present invention is that, in step three, the network model parameters are shared among the multi-branch feature extractors, and the network parameters of each branch are initialized by the pre-trained classification model parameters obtained in step one.

[0023] A further improvement of the present invention is that, in step 4, the base branch and the conversion branch use different classification head networks respectively; the classification head network used by the base branch consists of a global average pooling operation and a fully connected operation, and its formula is expressed as follows:

[0024]

[0025] Among them, FC represents the fully connected operation, GAP represents the global average pooling operation, F represents the output features of each branch feature extractor, and F i,j Represents the feature vector of the pixel position (i, j), M and N represent the height and width of the input image respectively, W fc Represents the parameters of the fully connected layer;

[0026] The classification head network used by the conversion branch consists of a 1×1 convolution operation and a global average pooling operation, and its formula is expressed as:

[0027]

[0028] Among them, GAP represents the global average pooling operation, Conv represents the convolution operation, F represents the output features of each branch feature extractor, · represents the convolution operator, (i, j) represents the pixel position in the image, M and N represent the height and width of the input image respectively, and W conv Represents the parameters of the convolutional layer.

[0029] A further improvement of the present invention is that, in step 5, the target category activation features are calculated for different branches respectively, wherein the calculation formula of the basic branch is:

[0030]

[0031] Among them, S k represents the activation feature corresponding to the k-th target category, C represents the number of channels of the output feature of the basic branch feature extractor, and F c represents the feature map of the c-th channel, Represents the fully connected layer parameters of the k-th category in the c-th channel;

[0032] The calculation formula for the conversion branch is:

[0033] S k =(F·Wconv ) k

[0034] Among them, (F·W conv ) k It means directly extracting the feature map of the kth channel of the output feature after the convolution operation in the classification head network as the activation feature corresponding to the kth target category.

[0035] A further improvement of the present invention is that, in step 6, the inverse transformation operation of the class activation features of the transformed branch is approximated by performing a causal invariant transformation operation on the class activation features of the base branch. The specific causal invariance constraint loss formula is expressed as follows:

[0036] L cons =L1(f Φ (f Tr (I,γ)),f Tr (f Φ (I),γ))

[0037] Among them, L cons represents the causal invariance constraint loss, L1 represents the L1 distance, and γ represents the control variable of each specific conversion operation, which has different physical meanings in different conversion operations.

[0038] A further improvement of the present invention is that the conversion operation includes: scale invariance conversion, receptive field invariance conversion and angle invariance conversion;

[0039] For scale-invariant transformation, γ represents the scaling factor of the input image. Theoretically, γ can be any value greater than 0. Considering the computational burden in actual operation, the value of γ is randomly selected from 0.5, 0.75, 1.5, and 1.75.

[0040] For the receptive field invariance transformation, γ represents the coordinate of the upper left corner of the randomly selected image cropping box. The size of the cropping box is fixed to 320×320, and the coordinate point selection of γ is limited to the range of [0:(M-320),0:(N-320)];

[0041] For angle-invariant transformation, γ represents the angle value of the original image rotation. The value of γ is limited to the range of [-90°, 90°], where negative values ​​represent counterclockwise rotation and positive values ​​represent clockwise rotation.

[0042] A further improvement of the present invention is that, in step 7, the loss function calculation formula of the joint optimization is:

[0043] L bcls =-(glog(p b )+(1-g)log(1-p b ))

[0044] L tcls =-(glog(p′ b )+(1-g)log(1-p′ b ))

[0045] L=L bcls +L tcls +λL cons

[0046] Among them, L bcls represents the classification loss of the base branch, g represents the real image-level category label, and p b represents the basic branch prediction classification probability, L tcls represents the conversion branch classification loss, p′ b represents the predicted classification probability of the conversion branch, λ represents the weight parameter of the causal invariance constraint loss, and its value changes with the number of iterations. The change formula is expressed as:

[0047]

[0048] Among them, λ max Represents the maximum weight parameter value, curr_itr represents the current number of iterations, max_itr represents the maximum number of iterations, and τ represents the λ value growth rate control variable.

[0049] Compared with the prior art, the present invention has at least the following beneficial technical effects:

[0050] The present invention provides a weakly supervised medical image segmentation method based on causal invariance. The method uses a two-stage training process. The first stage is the label generation stage. Through the improved class activation mapping mechanism, the image-level labels are generated into high-precision pixel-level pseudo-labels, solving the problem that the medical image segmentation algorithm is highly dependent on a large number of manually labeled pixel-level labels. The second stage is the segmentation model training stage. The results output by the first stage are used as pseudo-labels to retrain a segmentation model. The self-correction ability of the segmentation network is used to further improve the accuracy of the segmentation results, providing effective segmentation results for practical clinical applications of medical image analysis. Secondly, the present invention designs a multi-branch feature extraction network in the pseudo-label generation stage and designs a causal invariant conversion strategy, so that the model can better extract the features of the same target in different states. At the same time, the present invention specifically designs classification head networks and classification loss functions for different branch networks, ensuring the model's prediction accuracy for the target object category. Finally, the present invention designs a causal invariance constraint loss between multiple branches to solve the problem that deep network models cannot learn causal essential features, ensuring the model's prediction accuracy for the target object shape. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flow chart of the present invention;

[0052] Figure 2 It is a model structure diagram of the multi-branch feature extraction network in the present invention;

[0053] Figure 3 This is a model structure diagram of the two types of classification head networks in the present invention. DETAILED DESCRIPTION

[0054] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, unless there is a conflict, the embodiments of the present disclosure and the features described in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0055] See also Figure 1 The present invention provides a weakly supervised medical image segmentation method based on causal invariance, comprising the following steps:

[0056] Step 101: Obtain medical image data such as magnetic resonance imaging, computed tomography imaging, and ultrasound imaging, and convert them into grayscale or RGB image formats using professional image processing tools according to the image characteristics.

[0057] Step 102: Use the processed image data to train a classification network until the classification model prediction accuracy reaches the specified requirement.

[0058] Step 103: Design a multi-branch feature extraction network. Use the pre-trained classification model to initialize the parameters of the feature extractors of the base branch and the conversion branch. The classification head network parameters are randomly initialized. The base branch and the conversion branch use different classification head networks. The classification head network used by the base branch consists of a global average pooling operation and a fully connected operation. Its formula is expressed as:

[0059]

[0060] Among them, FC represents the fully connected operation, GAP represents the global average pooling operation, F represents the output features of each branch feature extractor, and F i,j Represents the feature vector of the pixel position (i, j), M and N represent the height and width of the input image respectively, W fc Represents the parameters of the fully connected layer;

[0061] The classification head network used by the conversion branch consists of a 1×1 convolution operation and a global average pooling operation, and its formula is expressed as:

[0062]

[0063] Among them, GAP represents the global average pooling operation, Conv represents the convolution operation, F represents the output features of each branch feature extractor, · represents the convolution operator, (i, j) represents the pixel position in the image, M and N represent the height and width of the input image respectively, and W conv Represents the parameters of the convolutional layer.

[0064] Step 104 : Perform a causally invariant transformation operation on each input original image to expand it into multiple images.

[0065] Step 105: Use scale-invariant transformation to randomly scale the input original image. The selectable values ​​of the scaling scale are 0.5, 0.75, 1.5, and 1.75.

[0066] In step 106, the input original image is cropped at random positions using receptive field invariance transformation, and the size of the cropping box is fixed to 320×320.

[0067] Step 107: Use angle-invariant transformation to rotate the input original image by a random angle, with the rotation angle range being [-90°, 90°].

[0068] In step 108, the original image is input into the basic branch, and the converted images are input into different conversion branches respectively, and the feature extractors of each branch extract features from the input image respectively.

[0069] In step 109, the features extracted by all branches are input into the classification head network corresponding to each branch to predict the classification result and calculate the classification loss. The class activation features of each branch are calculated using the classification head network parameters, and the causal invariance constraint loss between each branch is calculated. The class activation feature calculation formula of the base branch is:

[0070]

[0071] Among them, S k represents the activation feature corresponding to the k-th target category, C represents the number of channels of the output feature of the basic branch feature extractor, and F c represents the feature map of the c-th channel, Represents the fully connected layer parameters of the k-th category in the c-th channel;

[0072] The class activation feature calculation formula of the conversion branch is:

[0073] S k =(F·Wconv ) k

[0074] Among them, (F·W conv ) k It means directly extracting the feature map of the kth channel of the output feature after the convolution operation in the classification head network as the activation feature corresponding to the kth target category.

[0075] The specific causal invariance constraint loss formula is as follows:

[0076] L cons =L1(f Φ (f Tr (I,γ)),f Tr (f Φ (I),γ))

[0077] Among them, L cons represents the causal invariance constraint loss, L1 represents the L1 distance, and γ represents the control variable of each specific conversion operation, which has different physical meanings in different conversion operations.

[0078] Step 110: Jointly optimize the classification loss and causal invariance constraint loss of all branches, and dynamically update the weight of the causal invariance constraint loss as the number of iterations increases. The loss function calculation formula for the joint optimization is:

[0079] L bcls =-(glog(p b )+(1-g)log(1-p b ))

[0080] L tcls =-(glog(p′ b )+(1-g)log(1-p′ b ))

[0081] L=L bcls +L tcls +λL cons

[0082] Among them, L bcls represents the classification loss of the base branch, g represents the real image-level category label, and p b represents the basic branch prediction classification probability, L tcls represents the conversion branch classification loss, p′ b represents the predicted classification probability of the conversion branch, λ represents the weight parameter of the causal invariance constraint loss, and its value changes with the number of iterations. The change formula is expressed as:

[0083]

[0084] Among them, λ max Represents the maximum weight parameter value, curr_itr represents the current number of iterations, max_itr represents the maximum number of iterations, and τ represents the λ value growth rate control variable.

[0085] In step 111, whether the model has converged is determined based on the change in the loss value of the joint optimization. If the model has converged, the process proceeds to step 112; otherwise, the process returns to step 108 and continues to repeat the extraction of multi-branch features to iteratively update the model.

[0086] In step 112, the converged model parameters are saved, and the basic branches in the saved model are used to infer the training set images to generate a class activation feature map with causal invariance constraints, and a thresholding operation is used to generate a binary pseudo segmentation label based on the pixel values ​​in the feature map.

[0087] In step 113 , a segmentation network such as U-Net is trained using the pseudo segmentation labels in combination with the original image until the segmentation model converges, and the model parameters are saved and output as a segmentation model of the medical image.

[0088] See also Figure 2 ,The model structure diagram of the multi-branch feature extraction network includes the following parts:

[0089] A basic branch network consists of a feature extraction network and a basic classification head network.

[0090] Three conversion branch networks, each conversion branch network consists of a feature extraction network and a conversion classification network.

[0091] The feature extraction network model parameters of each branch are shared.

[0092] The classification loss is calculated by comparing the output of the transformed classification head network of each branch with the actual image-level label.

[0093] The causal invariance constraint loss is calculated between the target category activation feature maps obtained by the class activation mapping of each branch.

[0094] See also Figure 3 ,The model structure diagram of the two-class classification head network includes the following parts:

[0095] The base branch classification head network consists of a global average pooling operation and a fully connected operation in series. It inputs the output features of the base branch feature extractor and outputs the classification prediction result.

[0096] The conversion branch classification head network consists of a 1×1 convolution operation and a global average pooling operation in series. It inputs the output features of the feature extractor of the conversion branch and outputs the classification prediction results.

[0097] Although the present invention has been described in detail above using general descriptions and specific embodiments, it will be apparent to those skilled in the art that modifications and improvements may be made based on the present invention. Therefore, such modifications and improvements, which do not depart from the spirit of the present invention, are intended to be within the scope of protection claimed herein.

Claims

1. A weakly supervised medical image segmentation method based on causal invariance, characterized in that: The following steps are involved: Step 1: Pre-train the classification model; use the image data in the training dataset and its corresponding image-level category annotation data to train the classification model; Step 2: Expand the input image data; perform a causal-invariant transformation operation on each input original image to obtain multiple transformed images; Step 3: Train a multi-branch feature extractor. Design a multi-branch feature extraction network, including a base branch and multiple conversion branches. Use the original input image and the converted image obtained in step 2 as input to the base branch and the conversion branch, respectively, to extract different forms of features from the same image. At the same time, record the conversion parameters of each conversion branch compared to the original image. Step 4: Calculate the multi-branch classification loss. Based on the multi-branch feature extraction network in step 3, design different classification head networks for the base branch where the original input image is located and the conversion branch where the converted image is located. Predict the classification prediction probability of the input image of each branch and calculate the cross entropy loss value between the prediction result and the image-level category labeling in step 1. Step 5: Calculate multi-branch class activation features; Calculate the target category activation features of each branch using the features of different forms obtained in step 3 and the model parameters of different classification head networks obtained in step 4; Step 6: Calculate the causal invariance constraint loss between multiple branches; Use the image conversion parameters saved in step 3 to perform the inverse transformation of the causal invariance transformation on the class activation features of the transformed branches obtained in step 5, and calculate the distance between each inverse transformed class activation feature and the class activation feature of the base branch as the causal invariance constraint loss between multiple branches; Step 7: Jointly optimize the multi-branch feature extraction network model; design a joint optimization loss function, use a gradient descent strategy to simultaneously backpropagate the multi-branch classification loss in step 4 and the causal invariance constraint loss in step 6, and end the optimization process by iterative optimization until the loss value converges to a minimum value. The model parameters of the basic branch are saved to form a pseudo-label generation model for medical images. Step 8: Generate pseudo-segmentation labels. Based on the pseudo-label generation model in step 7, execute the basic branch process on the input image to generate a class activation feature map with causal invariance constraints. Then, use the thresholding operation to generate a binary map based on the pixel values ​​in the feature map, where 0 represents background and 1 represents foreground, which serves as the pseudo-segmentation label. Step 9: Train the segmentation model. Use the image data from step 1 and the pseudo-segmentation labels obtained from step 8 as training data to train a segmentation model. The segmentation model encodes and decodes the input image, and ultimately outputs the probability that each pixel in the image belongs to the target category. During the training phase, a gradient descent strategy is used to iteratively optimize the model until the model converges, with the goal of making the predicted segmentation results as close as possible to the pseudo-segmentation labels. The model parameters are saved as the segmentation model for medical images.

2. The weakly supervised medical image segmentation method based on causal invariance according to claim 1, characterized in that: In step 1, the classification network used by the pre-trained classification model consists of a feature extraction network and a classification head network; the feature extraction network consists of 5 convolutional layers, the first convolutional layer consists of a convolution operation with a kernel size of 7×7 and a stride of 2, a batch normalization operation, a ReLU operation and a maximum pooling operation with a kernel size of 3×3 and a stride of 2; the last four convolutional layers are composed of repeated splicing of multiple basic operation units, and the number of repeated splicing is 3, 4, 6, and 3 respectively; each basic operation unit consists of three convolution operations, three batch normalization operations, three ReLU operations and a residual connection operation; the classification head network consists of a global average pooling operation and a fully connected operation with a parameter size of 2048×1.

3. The weakly supervised medical image segmentation method based on causal invariance according to claim 1, characterized in that: In step 1, the image data in the training dataset are grayscale images or RGB images converted from medical imaging data of multiple modalities, including magnetic resonance imaging, computed tomography imaging, and ultrasound imaging. The category labeling information used in the training dataset is whether the input image contains the category to which the target area belongs. If it does, it is marked as 1, and if it does not, it is marked as 0. The target area is a specific organ area or a specific lesion area.

4. The weakly supervised medical image segmentation method based on causal invariance according to claim 1, characterized in that: In step 2, the causal invariant transformation operation used has segmentation invariance, which is expressed as: f Φ (f Tr (I))=f Tr (f Φ (I)) Among them, f Φ Represents the process of extracting class activation features from the input image, f Tr represents a causal-invariant transformation operation, and I represents the input image data.

5. The weakly supervised medical image segmentation method based on causal invariance according to claim 1, characterized in that: In step three, the network model parameters are shared among the multi-branch feature extractors, and the network parameters of each branch are initialized by the pre-trained classification model parameters obtained in step one.

6. The weakly supervised medical image segmentation method based on causal invariance according to claim 1, characterized in that: In step 4, the base branch and the conversion branch use different classification head networks respectively; the classification head network used by the base branch consists of a global average pooling operation and a fully connected operation, and its formula is expressed as: Among them, FC represents the fully connected operation, GAP represents the global average pooling operation, F represents the output features of each branch feature extractor, and F i,j Represents the feature vector of the pixel position (i, j), M and N represent the height and width of the input image respectively, W fc Represents the parameters of the fully connected layer; The classification head network used by the conversion branch consists of a 1×1 convolution operation and a global average pooling operation, and its formula is expressed as: Among them, GAP represents the global average pooling operation, Conv represents the convolution operation, F represents the output features of each branch feature extractor, · represents the convolution operator, (i, j) represents the pixel position in the image, M and N represent the height and width of the input image respectively, and W conv Represents the parameters of the convolutional layer.

7. The weakly supervised medical image segmentation method based on causal invariance according to claim 1, characterized in that: In step 5, the target category activation features are calculated for different branches respectively. The calculation formula of the basic branch is: Among them, S k represents the activation feature corresponding to the k-th target category, C represents the number of channels of the output feature of the basic branch feature extractor, and F c represents the feature map of the c-th channel, Represents the fully connected layer parameters of the k-th category in the c-th channel; The calculation formula for the conversion branch is: S k =(F·W conv ) k Among them, (F·W conv ) k It means directly extracting the feature map of the kth channel of the output feature after the convolution operation in the classification head network as the activation feature corresponding to the kth target category.

8. The weakly supervised medical image segmentation method based on causal invariance according to claim 1, characterized in that: In step 6, the inverse transformation operation of the class activation features of the transformed branch is approximated by performing a causal invariant transformation operation on the class activation features of the base branch. The specific causal invariance constraint loss formula is expressed as follows: L cons =L1(f Φ (f Tr (I,γ)),f Tr (f Φ (I),γ)) Among them, f Φ Represents the process of extracting class activation features from the input image, f Tr represents a causally invariant transformation operation, L cons represents the causal invariance constraint loss, L1 represents the L1 distance, and γ represents the control variable of each specific conversion operation, which has different physical meanings in different conversion operations.

9. The weakly supervised medical image segmentation method based on causal invariance according to claim 8, characterized in that: The transformation operations include: scale invariance transformation, receptive field invariance transformation and angle invariance transformation; For scale-invariant transformation, γ represents the scaling factor of the input image. Theoretically, γ can be any value greater than 0. Considering the computational burden in actual operation, the value of γ is randomly selected from 0.5, 0.75, 1.5, and 1.

75. For the receptive field invariance transformation, γ represents the coordinate of the upper left corner of the randomly selected image cropping box. The size of the cropping box is fixed to 320×320, and the coordinate point selection of γ is limited to the range of [0:(M-320),0:(N-320)]; For angle-invariant transformation, γ represents the angle value of the original image rotation. The value of γ is limited to the range of [-90°, 90°], where negative values ​​represent counterclockwise rotation and positive values ​​represent clockwise rotation.

10. The weakly supervised medical image segmentation method based on causal invariance according to claim 8, characterized in that: In step 7, the loss function calculation formula for joint optimization is: L bcls =-(glog(p b )+(1-g)log(1-p b )) L tcls =-(glog(p′ b )+(1-g)log(1-p′ b )) L=L bcls +L tcls +λL cons Among them, L bcls represents the classification loss of the base branch, g represents the real image-level category label, and p b represents the basic branch prediction classification probability, L tcls represents the conversion branch classification loss, p ′ b represents the predicted classification probability of the conversion branch, λ represents the weight parameter of the causal invariance constraint loss, and its value changes with the number of iterations. The change formula is expressed as: Among them, λ max Represents the maximum weight parameter value, curr_itr represents the current number of iterations, max_itr represents the maximum number of iterations, and τ represents the λ value growth rate control variable.

Images