Pheochromocytoma medical image segmentation method and device, equipment and medium

Abstract

The embodiment of the invention provides a pheochromocytoma medical image segmentation method and device, equipment and a medium. The method comprises the following steps: acquiring a pheochromocytoma medical image; the pheochromocytoma medical image is input into a pheochromocytoma segmentation model for image segmentation, a segmentation result of the pheochromocytoma medical image is obtained, the pheochromocytoma segmentation model is composed of a semi-supervised segmentation model and an SAM-Med3D segmentation model, the semi-supervised segmentation model comprises a shared encoder and a plurality of independent decoders, and the SAM-Med3D segmentation model is used for segmenting the pheochromocytoma medical image. A multi-scale attention module is arranged on an encoding path of the encoder, the multi-scale attention module is used for enhancing pheochromocytoma key feature extraction, pseudo labels are selected through competition comparison in output of the multiple decoders, and position information generated through prediction of the multiple decoders serves as a mask prompt information source of the SAM-Med3D segmentation model. Based on this, the embodiment of the invention can improve the automatic segmentation performance of the pheochromocytoma medical image.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of medical image processing, and in particular to a method and apparatus, device and medium for segmenting medical images of pheochromocytoma. Background Art

[0002] Pheochromocytoma is a rare neuroendocrine tumor with an annual incidence of approximately 0.8 / 100,000 people. Accurate diagnosis and timely surgical intervention can prevent tumor deterioration and metastasis and reduce mortality. However, the clinical manifestations of pheochromocytoma are diverse, which makes it difficult for doctors to make accurate diagnoses. In addition, pheochromocytomas are usually located in the abdomen and are surrounded by important multi-organ anatomical structures. Tumors that deteriorate and grow larger will invade connected organs, further increasing surgical risks. Therefore, it is very important to accurately identify the tumor location and precisely segment the tumor boundaries to assist in surgical treatment.

[0003] Existing pheochromocytoma segmentation methods require sufficient labels to ensure the segmentation performance of the model. For 3D medical images, it is very time-consuming and difficult to obtain pixel-level annotations because accurate annotations can only be provided by experts with domain knowledge. For the pheochromocytoma segmentation task, the annotation of the CT volume may take several hours. The complex shape and appearance of the tumor further increase the workload and time required for annotation, which makes it difficult to annotate a large number of 3D pheochromocytoma CT scans for training. In addition, the inherent properties of pheochromocytoma also make automated segmentation challenging. First, the size of the tumor varies greatly with the severity of the disease. A small tumor lesion may contain only a few voxels, while a large tumor can occupy a large space, which easily causes class imbalance problems. Second, the location of the tumor is not fixed, appearing in different locations in the abdomen with a complex spatial distribution. In addition, the tumor morphology varies greatly. Smaller tumors are usually uniform, while larger tumors may have uneven grayscale and show central necrosis, which may lead to under-segmentation. In addition, the proximity of the tumor to the abdominal organs may cause the model to misunderstand the morphological changes of these organs as tumor boundaries and mistakenly segment them as part of the tumor, resulting in over-segmentation. These factors, together with the low soft tissue contrast in abdominal CT images, further exacerbate the difficulty of the pheochromocytoma segmentation task, resulting in deficiencies in automated segmentation performance. Summary of the invention

[0004] The embodiments of the present invention provide a method, apparatus, device and medium for segmenting medical images of pheochromocytoma, which effectively alleviates the problem of insufficient label data by applying semi-supervised technology to pheochromocytoma segmentation. Pseudo-labels are selected from multiple decoder outputs, which helps to improve the reliability of pseudo-labels, and introduces uncertainty assessment to reduce the impact of pseudo-label noise. A multi-scale attention module is used to enhance the extraction of key features of pheochromocytoma, effectively addressing challenges such as changes in size and position and low contrast of pheochromocytoma. The pseudo-labels generated by the semi-supervised segmentation model are used as the source of prompt information for SAM-Med3D, which effectively solves the defects of labor-intensive manual prompts, thereby further improving the automated segmentation performance of pheochromocytoma medical images.

[0005] In a first aspect, an embodiment of the present invention provides a pheochromocytoma medical image segmentation method, comprising:

[0006] Get medical images of pheochromocytoma;

[0007] The pheochromocytoma medical image is input into a pre-trained pheochromocytoma segmentation model for image segmentation to obtain a segmentation result of the pheochromocytoma medical image, wherein the pheochromocytoma segmentation model is composed of a semi-supervised segmentation model and a SAM-Med3D segmentation model, the semi-supervised segmentation model includes a shared encoder and multiple independent decoders, a multi-scale attention module is arranged on the encoding path of the encoder, the multi-scale attention module is used to enhance the extraction of key features of pheochromocytoma, pseudo labels are selected through competitive comparison in multiple decoder outputs, and the position information predicted by multiple decoders is used as the mask prompt information source of the SAM-Med3D segmentation model.

[0008] In some embodiments, the step of inputting the pheochromocytoma medical image into a pre-trained pheochromocytoma segmentation model to perform image segmentation to obtain a segmentation result of the pheochromocytoma medical image includes:

[0009] Inputting the pheochromocytoma medical image into the encoder for feature extraction to obtain features of the pheochromocytoma medical image;

[0010] Inputting the pheochromocytoma medical image features into a plurality of decoders to obtain a plurality of output results;

[0011] A dynamic uncertainty correction strategy is used to select pseudo labels from multiple output results;

[0012] Filtering the pseudo-labels to generate mask prompt information;

[0013] The SAM-Med3D segmentation model is guided to perform image segmentation on the pheochromocytoma medical image according to the mask prompt information to obtain a segmentation result of the pheochromocytoma medical image.

[0014] In some embodiments, the multi-scale attention module includes an MSA-H variant applied at high resolution and an MSA-L variant applied at low resolution, and the pheochromocytoma medical image is input into the encoder for feature extraction to obtain pheochromocytoma medical image features, including:

[0015] Inputting the pheochromocytoma medical image into the encoder to learn and obtain variable pheochromocytoma features;

[0016] The variable pheochromocytoma features are input into the multi-scale attention module to extract the multi-scale medical image features of the pheochromocytoma.

[0017] In some embodiments, the adopting a dynamic uncertainty correction strategy to select a pseudo label from a plurality of output results comprises:

[0018] Select the best prediction result as the pseudo label from multiple output results through dynamic competition;

[0019] Introducing an uncertainty correction method to perform voxel-level evaluation on the selected pseudo-labels to obtain the confidence of each voxel;

[0020] A weight factor value corresponding to each voxel is assigned according to the confidence level of each voxel.

[0021] In some embodiments, assigning an uncertainty weight factor corresponding to each voxel according to the confidence level of each voxel comprises:

[0022] increasing the weight factor value of the voxel with high confidence;

[0023] The weight factor value of the voxel with low confidence is reduced.

[0024] In some embodiments, the method further comprises:

[0025] The output of the optimal decoder is selected from the multiple decoder output results as a pseudo label to guide the suboptimal decoder.

[0026] In some embodiments, the training method of the pheochromocytoma segmentation model comprises:

[0027] Determine a final loss function, wherein the final loss function is determined based on multiple supervised loss functions, multiple uncertain loss functions, and multiple balanced hyperparameters;

[0028] The pheochromocytoma segmentation model is trained based on the final loss function to obtain the trained pheochromocytoma segmentation model.

[0029] In a second aspect, an embodiment of the present invention further provides a pheochromocytoma medical image segmentation device, the device comprising:

[0030] An acquisition module, used for acquiring medical images of pheochromocytoma;

[0031] A segmentation module is used to input the pheochromocytoma medical image into a pre-trained pheochromocytoma segmentation model for image segmentation to obtain a segmentation result of the pheochromocytoma medical image, wherein the pheochromocytoma segmentation model is composed of a semi-supervised segmentation model and a SAM-Med3D segmentation model, the semi-supervised segmentation model includes a shared encoder and multiple independent decoders, a multi-scale attention module is arranged on the encoding path of the encoder, the multi-scale attention module is used to enhance the extraction of key features of pheochromocytoma, pseudo labels are selected through competitive comparison in multiple decoder outputs, and the position information predicted by multiple decoders is used as the source of mask prompt information of the SAM-Med3D segmentation model.

[0032] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the pheochromocytoma medical image segmentation method as described in the first aspect when executing the computer program.

[0033] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the pheochromocytoma medical image segmentation method as described in the first aspect.

[0034] According to the pheochromocytoma medical image segmentation method and apparatus, device and medium provided by the embodiment of the present invention, the pheochromocytoma medical image segmentation method includes: obtaining a pheochromocytoma medical image; inputting the pheochromocytoma medical image into a pre-trained pheochromocytoma segmentation model for image segmentation, and obtaining a segmentation result of the pheochromocytoma medical image, wherein the pheochromocytoma segmentation model is composed of a semi-supervised segmentation model and a SAM-Med3D segmentation model, and the semi-supervised segmentation model includes a shared encoder and multiple independent decoders, and a multi-scale attention module is provided on the encoding path of the encoder, and the multi-scale attention module is used to strengthen the extraction of key features of pheochromocytoma, and pseudo labels are selected by competitive comparison in multiple decoder outputs, and the position information generated by multiple decoder predictions is used as the source of mask prompt information of the SAM-Med3D segmentation model. The present invention effectively alleviates the problem of insufficient label data by applying semi-supervised technology to pheochromocytoma segmentation. And pseudo labels are selected in multiple decoder outputs, which helps to improve the reliability of pseudo labels, and introduces uncertainty evaluation to reduce the impact of pseudo label noise. A multi-scale attention module is used to enhance the extraction of key features of pheochromocytoma, effectively addressing challenges such as size, location, and low contrast of pheochromocytoma. Pseudo-labels generated by the semi-supervised segmentation model are used as the source of prompt information for SAM-Med3D, effectively solving the defects of labor-intensive manual prompts, thereby further improving the performance of automated segmentation of pheochromocytoma medical images. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a flow chart of a pheochromocytoma medical image segmentation method provided by one embodiment of the present invention;

[0036] Figure 2 is a schematic diagram of the structure of a pheochromocytoma segmentation model provided by one embodiment of the present invention;

[0037] Figure 3 is a schematic diagram of the structure of a multi-scale attention module provided by an embodiment of the present invention;

[0038] Figure 4 is an algorithm diagram of a pseudo label selection process provided by an embodiment of the present invention;

[0039] Figure 5 It is a structural schematic diagram of a pheochromocytoma medical image segmentation device provided by an embodiment of the present invention;

[0040] Figure 6 is a schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0042] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification and claims and the following drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0043] In the embodiments of the present invention, the words "further", "exemplarily" or "optionally" are used to indicate examples, illustrations or descriptions, and should not be interpreted as being more preferred or more advantageous than other embodiments or designs. The use of the words "further", "exemplarily" or "optionally" is intended to present related concepts in a specific way.

[0044] In order to more conveniently describe the working principle of the embodiment of the present invention later, an introduction to relevant technical scenarios is first given below.

[0045] Pheochromocytoma is a rare neuroendocrine tumor with an annual incidence of approximately 0.8 / 100,000 people. Accurate diagnosis and timely surgical intervention can prevent tumor deterioration and metastasis and reduce mortality. However, the clinical manifestations of pheochromocytoma are diverse, which makes it difficult for doctors to make accurate diagnoses. In addition, pheochromocytomas are usually located in the abdomen and are surrounded by important multi-organ anatomical structures. Tumors that deteriorate and grow larger will invade connected organs, further increasing surgical risks. Therefore, it is very important to accurately identify the tumor location and precisely segment the tumor boundaries to assist in surgical treatment.

[0046] Existing pheochromocytoma segmentation methods require sufficient labels to ensure the segmentation performance of the model. For 3D medical images, it is very time-consuming and difficult to obtain pixel-level annotations because accurate annotations can only be provided by experts with domain knowledge. For the pheochromocytoma segmentation task, the annotation of the CT volume may take several hours. The complex shape and appearance of the tumor further increase the workload and time required for annotation, which makes it difficult to annotate a large number of 3D pheochromocytoma CT scans for training. In addition, the inherent properties of pheochromocytoma also make automated segmentation challenging. First, the size of the tumor varies greatly with the severity of the disease. A small tumor lesion may contain only a few voxels, while a large tumor can occupy a large space, which easily causes class imbalance problems. Second, the location of the tumor is not fixed, appearing in different locations in the abdomen with a complex spatial distribution. In addition, the tumor morphology varies greatly. Smaller tumors are usually uniform, while larger tumors may have uneven grayscale and show central necrosis, which may lead to under-segmentation. In addition, the proximity of the tumor to the abdominal organs may cause the model to misunderstand the morphological changes of these organs as tumor boundaries and mistakenly segment them as part of the tumor, resulting in over-segmentation. These factors, together with the low soft tissue contrast in abdominal CT images, further exacerbate the difficulty of the pheochromocytoma segmentation task, resulting in deficiencies in automated segmentation performance.

[0047] Based on this, the present invention provides a pheochromocytoma medical image segmentation method and device, equipment and medium. Among them, the pheochromocytoma medical image segmentation method includes: obtaining a pheochromocytoma medical image; inputting the pheochromocytoma medical image into a pre-trained pheochromocytoma segmentation model for image segmentation, and obtaining a segmentation result of the pheochromocytoma medical image, wherein the pheochromocytoma segmentation model is composed of a semi-supervised segmentation model and a SAM-Med3D segmentation model, and the semi-supervised segmentation model includes a shared encoder and multiple independent decoders, and a multi-scale attention module is provided on the encoding path of the encoder. The multi-scale attention module is used to strengthen the extraction of key features of pheochromocytoma, and pseudo labels are selected through competitive comparison in multiple decoder outputs, and the position information generated by multiple decoder predictions is used as the source of mask prompt information of the SAM-Med3D segmentation model. The present invention effectively alleviates the problem of insufficient label data by applying semi-supervised technology to pheochromocytoma segmentation. And pseudo labels are selected in multiple decoder outputs, which helps to improve the reliability of pseudo labels, and introduces uncertainty evaluation to reduce the impact of pseudo label noise. A multi-scale attention module is used to enhance the extraction of key features of pheochromocytoma, effectively addressing challenges such as size, location, and low contrast of pheochromocytoma. Pseudo-labels generated by the semi-supervised segmentation model are used as the source of prompt information for SAM-Med3D, effectively solving the defects of labor-intensive manual prompts, thereby further improving the performance of automated segmentation of pheochromocytoma medical images.

[0048] The embodiments of the present invention are further described below in conjunction with the accompanying drawings.

[0049] like Figure 1 As shown, Figure 1 1 is a flowchart of a pheochromocytoma medical image segmentation method provided by an embodiment of the present invention. The pheochromocytoma medical image segmentation method may include but is not limited to steps S101 to S102.

[0050] Step S101, obtaining a medical image of pheochromocytoma;

[0051] Step S102, inputting the pheochromocytoma medical image into a pre-trained pheochromocytoma segmentation model for image segmentation, and obtaining a segmentation result of the pheochromocytoma medical image, wherein the pheochromocytoma segmentation model is composed of a semi-supervised segmentation model and a SAM-Med3D segmentation model, the semi-supervised segmentation model includes a shared encoder and multiple independent decoders, a multi-scale attention module is arranged on the encoding path of the encoder, the multi-scale attention module is used to enhance the extraction of key features of pheochromocytoma, pseudo labels are selected through competitive comparison in the outputs of multiple decoders, and the position information predicted by the multiple decoders is used as the mask prompt information source of the SAM-Med3D segmentation model.

[0052] It is understandable that if Figure 2 As shown, the pheochromocytoma segmentation model of the present invention is mainly composed of a semi-supervised segmentation model and a SAM-Med3D segmentation model. The semi-supervised segmentation model is a dynamic uncertainty correction model, which is named DR-Net. The model has a shared encoder and three independent decoders. In order to strengthen the extraction of key features, a multi-scale attention module is designed on the encoding path. For different decoder outputs, the prediction with the best performance is selected as a pseudo-label to correct the model through competition. DR-Net+ is formed by adding SAM-Med3D on the basis of DR-Net. SAM-Med3D serves as an additional supervision branch to strengthen the utilization of unlabeled data and assist in pheochromocytoma segmentation. Taking into account the problem of computing resources, during DR-Net training, SAM-Med3D does not participate in back propagation, but directly performs reasoning based on the mask prompts provided by DR-Net. The reasoning result acts on DR-Net as a supervision item, thereby improving the segmentation performance of DR-Net.

[0053] It can be understood that the present invention proposes a new semi-supervised framework for pheochromocytoma segmentation. The present invention selects pseudo-labels from multiple decoder outputs, which helps to improve the reliability of pseudo-labels, and introduces uncertainty assessment to reduce the impact of pseudo-label noise. The present invention proposes a multi-scale attention module to enhance the extraction of key features of pheochromocytoma, and effectively address challenges such as size, position, and low contrast of pheochromocytoma. The present invention uses pseudo-labels generated by a semi-supervised segmentation model as a source of prompt information for SAM-Med3D, effectively solving the defects of labor-intensive manual prompts.

[0054] It can be understood that the present invention applies semi-supervised technology to pheochromocytoma segmentation for the first time, effectively alleviating the problem of insufficient label data. The existing pheochromocytoma segmentation method requires all label data to ensure segmentation performance. In addition, the traditional level set-based method also achieves good segmentation results, but is still limited by manual parameter adjustment, while the pheochromocytoma medical image segmentation method of the present invention is an automated segmentation method that does not require manual parameter adjustment. The present invention also introduces the prior knowledge of SAM-Med3D to further improve the segmentation performance.

[0055] It can be understood that, in order to facilitate the description of the method of the present invention, the present invention expresses the problem of the SSL task as follows, with n labeled data represented as y i yes The label of m unlabeled data is represented as Where n<<m. The goal of the present invention is to improve the performance of pheochromocytoma segmentation framework using small amount of labeled data and abundant unlabeled data.

[0056] It is understandable that in the field of medical image segmentation, convolutional neural networks with integrated attention mechanisms have shown strong advantages in challenging scenarios such as low contrast, irregular shapes, and small target processing. In the pheochromocytoma segmentation task, when faced with challenges such as low contrast, shape and size changes, especially when the pheochromocytoma is squeezed by surrounding abdominal organs and causes irregular deformation, it is difficult for the model to make the most favorable decision based on the consistency of the target context. In addition, the close connection with the surrounding abdominal organs will also cause the model to have over-segmentation and under-segmentation problems.

[0057] In order to effectively deal with the above problems, the present invention designs a multi-scale attention module (MSA) in the model encoding stage to enhance the key feature extraction. MSA appears at the beginning of each stage of the encoder, receives the output (or input image) of the previous stage as input, learns the variable pheochromocytoma features, and extracts multi-scale tumor features at the same time. Figure 3As shown in Figure 1, the multi-scale attention module has two variants, namely MSA-H applied on high resolution and MSA-L applied on low resolution. MSA-H and MSA-L are described below.

[0058] and Respectively represent the feature input and output of MSA, W×H×D represents the size of the space, C i and C o Represent the number of input and output channels respectively. MSA-H first passes F through convolution layers with kernel sizes of 1 and 3 respectively, and the convolution layers are followed by a batch normalization layer and a ReLU activation layer. Then, the downsampling layer reduces the feature resolution by half, and then uses point-by-point convolution to realize the feature interaction mapping between channels, and then uses upsampling to restore the feature resolution to obtain F", and then sends F" to the Sigmoid activation function to obtain the attention coefficient α. The present invention uses an average pooling layer with a step size and kernel size of 2 to achieve downsampling, and upsampling is achieved through trilinear interpolation. Using downsampling can increase the receptive field of the convolution layer and reduce the computational cost of calculating spatial attention. The obtained attention coefficient is multiplied by F, and after assigning different attention weights to all voxels in F, it is added to F to obtain The expression is:

[0059]

[0060] At the bottleneck of the segmentation network, the resolution of the feature map is very low. Continuing to use the downsampling layer in MSA-H makes the feature map smaller and unable to retain spatial detail information. Therefore, the present invention proposes MSA-L, which is a variant of MSA-H. Due to the low spatial resolution, the present invention removes the downsampling and upsampling operations and only retains the point convolution.

[0061] It is understandable that generating pseudo labels for unannotated images has been shown to be effective for semi-supervised medical image segmentation. The key is to improve the quality of pseudo labels. Using wrong pseudo labels for correction will lead to a catastrophic decline in model performance. Selecting the best performing pseudo label among multiple options is considered to be an effective method, which filters out all predictions with suboptimal performance. In the present invention, a dynamic uncertainty correction strategy is proposed to process unlabeled data. The strategy mainly consists of two parts. The first is to select the best performing prediction result as the pseudo label among multiple different decoder outputs. The second is to introduce an uncertainty evaluation method to perform voxel-level evaluation on the selected pseudo label to reduce the impact of high uncertainty voxels.

[0062] It can be understood that, unlike selecting from multiple subnets, the present invention dynamically selects pseudo labels from multiple independent decoder outputs, and these decoders share the same encoder. This design has three advantages. First, it helps to reduce the representation differences introduced by different network parameters at the encoder stage. Second, selecting from multiple outputs can effectively improve the reliability of pseudo labels. Third, the multi-branch setting avoids the deviation of a single prediction head and improves the diversity of pseudo labels.

[0063] like Figure 4 As shown, Algorithm 1 describes the pseudo-label selection process. The present invention uses dice loss to detect the segmentation performance of multiple decoders in real time and select the best performing decoder. The dice loss is calculated for the labeled data in each batch. Among all the decoder output results obtained, it is generally believed that the decoder with the best segmentation performance has higher reliability in predicting unlabeled data and can be regarded as a pseudo-label. It should be pointed out that although the pseudo-label has higher reliability in all decoder outputs, there is still a certain gap with the real label.

[0064] It is understandable that the acquired pseudo-labels can be used to correct the suboptimal decoder output and improve the segmentation performance of the suboptimal decoder. The present invention uses consistency regularization loss to achieve correction:

[0065]

[0066] P l represents a pseudo label, Represents the kth decoder output. However, directly using the pseudo-labels generated by the optimal decoder to participate in model correction is not the best choice, because without the real label for guidance, the prediction results may be unreliable and noisy, which will affect the model training. Therefore, the present invention needs to evaluate the uncertainty of the pseudo-label, obtain the confidence of each voxel, and then guide the model to learn from more reliable information. The present invention first uses the KL divergence between the optimal decoder and the suboptimal decoder as the uncertainty:

[0067]

[0068] represents KL divergence, n v represents the number of voxels, represents the prediction of the k-th decoder output at the i-th voxel, P i represents the prediction of the i-th voxel in the pseudo-label. If the predictions of the optimal decoder and the k-th suboptimal decoder are significantly different, The value will be larger. Then As the uncertainty weight factor, the uncertainty loss function can be expressed as:

[0069]

[0070] By applying the uncertainty weight factor, voxels with higher uncertainty are automatically assigned lower weights, and conversely, voxels with lower uncertainty are automatically assigned higher weights, so that the model can learn more reliable information. Introducing constraints can improve the robustness of the model. The model effectively learns reliable representations from the predictions to keep the decoder outputs close under different perturbations, improving the segmentation performance of the model overall.

[0071] In addition, according to the semi-supervised smoothness assumption, the present invention hopes that the outputs of two independent suboptimal decoders are similar. Therefore, the present invention introduces another consistency regularization to further constrain the model, which is achieved through cosine distance:

[0072]

[0073] Shrinking The difference between suboptimal decoder outputs can be reduced. In fact, the uncertainty loss is reduced It is also possible to implicitly reduce But in pseudo-supervised correction, the explicit loss term contributes more regularization during training, which helps stabilize the training process.

[0074] It can be understood that in order to further enhance the utilization of unlabeled data, the present invention introduces SAM-Med3D with superior performance as an additional supervision branch to promote semi-supervised learning. Since SAM-Med3D contains rich and robust semantic information, the present invention uses the prior knowledge of SAM-Med3D to enhance the semantic representation of unlabeled pheochromocytoma data. The specific process is that the present invention uses SAM-Med3D's prediction of unlabeled data as an additional supervision signal to supervise the suboptimal decoder:

[0075]

[0076] represents the predicted output of the suboptimal decoder, Represents the reasoning results of SAM-Med3D by reducing To further optimize the suboptimal decoder and improve the performance of the segmentation model.

[0077] In addition, SAM-Med3D also requires experts to provide prompt information. This reliance on labor-intensive manual annotations as prompt information obviously hinders its clinical applicability. In order to solve this problem, the present invention uses the output of the decoder as position information to generate prompt information. Since the coarse segmentation contains a certain amount of noise, it is necessary to perform filtering processing first to retain reliable prompt information. Specifically, the present invention selects voxels with relatively large foreground probability values ​​in the pseudo-label. The higher the foreground probability, the more reliable the corresponding prediction is, which plays a greater role in the prediction of pheochromocytoma:

[0078]

[0079] Where T is the threshold for determining high-quality mask hints, P i represents the prediction of the pseudo label at the i-th voxel. It is the mask hint information required by SAM-Med3D. Through this operation, the false positive errors in mask prediction are reduced, high-probability voxels are retained, and low-confidence voxels that are prone to misclassification are filtered out.

[0080] It is understandable that the final loss function of the present invention takes the following form:

[0081]

[0082] in, and is the supervision loss term, dice loss is a region-based loss function, and focal loss helps alleviate the sample imbalance problem. They are both widely used in segmentation tasks. γ and β are balanced supervision losses and The two hyperparameters of is the balance uncertainty loss and It should be noted that and It is applied to both labeled and unlabeled datasets. Based on the above final loss function, the pheochromocytoma segmentation model is trained to obtain a trained pheochromocytoma segmentation model.

[0083] The pheochromocytoma medical image segmentation method proposed in the present invention is verified by experiments as follows.

[0084] For the dataset, the present invention evaluates the semi-supervised segmentation framework on the pheochromocytoma (PPGL) dataset. The PPGL dataset is a 3D private dataset collected by Hospital A, with a total of 100 CT samples. The dataset was annotated by two experienced medical experts and then handed over to a senior expert for final inspection. In addition, the isotropic resolution of each sample is 1×1×1mm, and the window width and window position are set to 255Hu and 127.5Hu respectively. In order to meet the input size requirements of SAM-Med3D, the size of each sample needs to be cropped to 128×128×128. We crop the corresponding image according to the label position, retain the lesion area while expanding its edges, and the expansion range in three dimensions is [10~20,10~20,5~10] voxels. Then, all samples are normalized to zero mean and unit variance. 80 cases were randomly selected as the training set, and the remaining 20 cases were used as the test set.

[0085] For evaluation indicators, the present invention uses DICE coefficient (DI), Jaccard (JA), average surface distance (ASD) and 95% Hausdorff distance (95HD). DICE coefficient and Jaccard are used to measure the pixel-level overlap between the segmentation map and the true label. The higher the values ​​of these two indicators, the better the performance. Average surface distance (ASD) and 95% Hausdorff distance are used to evaluate boundary similarity. The lower the values ​​of these two indicators, the better the performance.

[0086] Based on this, the present invention proposes a semi-supervised segmentation framework for pheochromocytoma segmentation. The framework is mainly composed of a semi-supervised segmentation model and SAM-Med3D. The semi-supervised segmentation model processes both labeled data and unlabeled data, and SAM-Med3D provides additional supervision information to process only unlabeled data. Among them, the semi-supervised segmentation model is an encoder-decoder model based on the popular V-Net architecture, which consists of a shared encoder and multiple independent decoders. In order to effectively deal with the challenges of shape and size changes, unfixed position and low contrast of pheochromocytoma, which lead to insufficient acquisition of key information, the present invention designs a multi-scale attention module to strengthen the extraction of key features, guide the model to focus on the tumor area, and reduce the interference of background noise. For pseudo-label correction, the present invention sets a dynamic uncertainty correction strategy, selects the decoder output with the best performance from multiple decoder output results as a pseudo-label to guide the suboptimal decoder, and dynamically competes to select pseudo-labels in multiple prediction results to reduce the inherent bias caused by the pseudo-labels of unlabeled pixels processed by a single model. Considering the possible noise interference in the pseudo-labels, which affects the model training, the present invention introduces an uncertainty correction method on this basis. The present invention uses Kullback-Leibler (KL) divergence to estimate the confidence of labeled and unlabeled samples. Then, the present invention enforces consistency between the prediction of the optimal decoder and the prediction of the suboptimal decoder. The uncertainty estimate is a weight factor used to control the importance of each sample in the consistency loss. This method highlights predictions with higher reliability while assigning less weight to less reliable predictions. In addition, in view of the defect that SAM-Med3D requires manual prompt information, the present invention generates high-quality mask prompts to guide SAM-Med3D segmentation by filtering the pseudo-labels obtained by the semi-supervised segmentation model. This method of using the position information generated by multiple decoder predictions as a prompt source effectively solves the defects of labor-intensive manual prompts.

[0087] For all experiments, the present invention uses the SGD optimizer to optimize the model, sets the learning rate of SGD to 0.01, and the weight decay factor to 0.0001. The batch size is set to 4, including two labeled patches and two unlabeled patches. The weights and φ are both set to 0.5, the weights and φ are set to the time-dependent Gaussian warm-up function, and the model is trained for 10k iterations. During training, random cropping, rotation, and flipping are used for data enhancement. In the test phase, only the first decoder output is used as the final prediction. All experiments are run in the same environment, the hardware device is NVIDIA 4090, the software configuration is Pytorch 2.1.0+cuda12.0, and the random seed is set to 1337.

[0088] The experimental results are as follows:

[0089] Table 1 Test results on the internal PPGL dataset, using 5% of the labels, the best results are shown in bold

[0090]

[0091]

[0092] Table 2 Test results on the internal PPGL dataset, using 10% of the labels, the best results are shown in bold

[0093]

[0094] Table 3 Test results on the internal PPGL dataset, using 20% ​​of the labels, the best results are shown in bold

[0095]

[0096] Based on the data comparison of Table 1, Table 2 and Table 3 above, the comprehensive experimental results on the internal pheochromocytoma dataset show that the method proposed in the present invention is superior to other existing semi-supervised methods.

[0097] In summary, compared with the prior art, the pheochromocytoma medical image segmentation method of the present invention has at least the following beneficial effects:

[0098] 1. This paper proposes a semi-supervised segmentation framework embedded with SAM-Med3D prior knowledge, which provides an automatic solution for pheochromocytoma segmentation. The framework consists of a semi-supervised segmentation model and a SAM-Med3D.

[0099] 2. In order to improve the pseudo-label correction capability, the present invention proposes a dynamic uncertainty correction strategy, selects pseudo-labels through competitive comparison, and introduces uncertainty evaluation to pseudo-labels, so that the model can learn from unlabeled data more reliably. At the same time, the present invention designs a multi-scale attention module in the semi-supervised segmentation model to guide the model to focus on specific domain features and improve network performance.

[0100] 3. Comprehensive experimental results on the internal pheochromocytoma dataset show that the method proposed in this paper outperforms other existing semi-supervised methods.

[0101] In addition, if Figure 5 As shown, one embodiment of the present invention further discloses a pheochromocytoma medical image segmentation device, the device comprising:

[0102] An acquisition module 110 is used to acquire a medical image of pheochromocytoma;

[0103] The segmentation module 120 is used to input the pheochromocytoma medical image into a pre-trained pheochromocytoma segmentation model for image segmentation to obtain a segmentation result of the pheochromocytoma medical image, wherein the pheochromocytoma segmentation model is composed of a semi-supervised segmentation model and a SAM-Med3D segmentation model, the semi-supervised segmentation model includes a shared encoder and multiple independent decoders, a multi-scale attention module is arranged on the encoding path of the encoder, the multi-scale attention module is used to enhance the extraction of key features of the pheochromocytoma, pseudo labels are selected through competitive comparison in the outputs of multiple decoders, and the position information predicted by the multiple decoders is used as the mask prompt information source of the SAM-Med3D segmentation model.

[0104] The pheochromocytoma medical image segmentation device of the embodiment of the present invention is used to execute the pheochromocytoma medical image segmentation method in the above embodiment. Its specific processing process is the same as the pheochromocytoma medical image segmentation method in the above embodiment, and will not be repeated here.

[0105] In addition, if Figure 6 As shown, an embodiment of the present invention further discloses an electronic device, comprising: at least one processor 210; at least one memory 220, for storing at least one program; when the at least one program is executed by the at least one processor 210, a pheochromocytoma medical image segmentation method as in any of the previous embodiments is implemented.

[0106] In addition, an embodiment of the present invention further discloses a computer-readable storage medium, in which computer-executable instructions are stored. The computer-executable instructions are used to execute the pheochromocytoma medical image segmentation method as described in any of the previous embodiments.

[0107] The system architecture and application scenarios described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention, and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Those skilled in the art will appreciate that with the evolution of the system architecture and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are equally applicable to similar technical problems.

[0108] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0109] In hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0110] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components may reside in a process or an execution thread, and a component may be located on a computer or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may communicate, for example, through local or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, or a network, such as the Internet interacting with other systems through signals).

Claims

1. A pheochromocytoma medical image segmentation method, comprising: Get medical images of pheochromocytoma; The pheochromocytoma medical image is input into a pre-trained pheochromocytoma segmentation model for image segmentation to obtain a segmentation result of the pheochromocytoma medical image, wherein the pheochromocytoma segmentation model is composed of a semi-supervised segmentation model and a SAM-Med3D segmentation model, the semi-supervised segmentation model includes a shared encoder and multiple independent decoders, a multi-scale attention module is arranged on the encoding path of the encoder, the multi-scale attention module is used to enhance the extraction of key features of pheochromocytoma, pseudo labels are selected through competitive comparison in multiple decoder outputs, and the position information predicted by multiple decoders is used as the mask prompt information source of the SAM-Med3D segmentation model.

2. The method according to claim 1, characterized in that The step of inputting the pheochromocytoma medical image into a pre-trained pheochromocytoma segmentation model to perform image segmentation to obtain a segmentation result of the pheochromocytoma medical image includes: Inputting the pheochromocytoma medical image into the encoder for feature extraction to obtain features of the pheochromocytoma medical image; Inputting the pheochromocytoma medical image features into a plurality of decoders to obtain a plurality of output results; A dynamic uncertainty correction strategy is used to select pseudo labels from multiple output results; Filtering the pseudo-labels to generate mask prompt information; The SAM-Med3D segmentation model is guided to perform image segmentation on the pheochromocytoma medical image according to the mask prompt information to obtain a segmentation result of the pheochromocytoma medical image.

3. The method according to claim 2, characterized in that The multi-scale attention module includes an MSA-H variant applied at high resolution and an MSA-L variant applied at low resolution, and the pheochromocytoma medical image is input into the encoder for feature extraction to obtain pheochromocytoma medical image features, including: Inputting the pheochromocytoma medical image into the encoder to learn and obtain variable pheochromocytoma features; The variable pheochromocytoma features are input into the multi-scale attention module to extract the multi-scale medical image features of the pheochromocytoma.

4. The method according to claim 2, characterized in that: The method of selecting a pseudo label from a plurality of output results using a dynamic uncertainty correction strategy includes: Select the best prediction result as the pseudo label from multiple output results through dynamic competition; Introducing an uncertainty correction method to perform voxel-level evaluation on the selected pseudo-labels to obtain the confidence of each voxel; A weight factor value corresponding to each voxel is assigned according to the confidence level of each voxel.

5. The method according to claim 4, characterized in that The step of allocating an uncertainty weight factor corresponding to each voxel according to the confidence level of each voxel comprises: increasing the weight factor value of the voxel with high confidence; The weight factor value of the voxel with low confidence is reduced.

6. The method according to claim 2, characterized in that The method further comprises: The output of the optimal decoder is selected from the multiple decoder output results as a pseudo label to guide the suboptimal decoder.

7. The method according to claim 1, characterized in that The training method of the pheochromocytoma segmentation model includes: Determine a final loss function, wherein the final loss function is determined based on multiple supervised loss functions, multiple uncertain loss functions, and multiple balanced hyperparameters; The pheochromocytoma segmentation model is trained based on the final loss function to obtain the trained pheochromocytoma segmentation model.

8. A pheochromocytoma medical image segmentation device, characterized in that: The device comprises: An acquisition module, used for acquiring medical images of pheochromocytoma; A segmentation module is used to input the pheochromocytoma medical image into a pre-trained pheochromocytoma segmentation model for image segmentation to obtain a segmentation result of the pheochromocytoma medical image, wherein the pheochromocytoma segmentation model is composed of a semi-supervised segmentation model and a SAM-Med3D segmentation model, the semi-supervised segmentation model includes a shared encoder and multiple independent decoders, a multi-scale attention module is arranged on the encoding path of the encoder, the multi-scale attention module is used to enhance the extraction of key features of pheochromocytoma, pseudo labels are selected through competitive comparison in multiple decoder outputs, and the position information predicted by multiple decoders is used as the source of mask prompt information of the SAM-Med3D segmentation model.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for medical image segmentation of pheochromocytoma as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the pheochromocytoma medical image segmentation method according to any one of claims 1 to 7.

Images