An interactive 3D medical image segmentation method

Abstract

The present invention relates to a segmentation method for three-dimensional medical images, and specifically to an interactive three-dimensional medical image segmentation method. The method comprises the following steps: performing pre-processing such as cropping and standardization on the data to be processed; constructing a neural network based on a visual base model and three-dimensional coding, and inputting the training data into the interactive three-dimensional medical image segmentation model for learning during the training phase; and performing online learning on the data to be segmented using a three-dimensional coding strategy during the testing phase to achieve further updating of the model. The present invention proposes an interactive image segmentation model framework based on a visual base model, which adopts an efficient fine-tuning strategy and network structure to reduce the amount of model parameters and the computing power required for training, while ensuring the high accuracy and real-time performance of the model; adopts a three-dimensional click coding extraction strategy to better input the user's click information into the network, and continuously improve the online learning performance of the model.

Description

Technical Field

[0001] The present invention relates to a segmentation method for three-dimensional medical images, and in particular to an interactive three-dimensional medical image segmentation method based on a visual base model and three-dimensional coding. Background Art

[0002] With the continuous development and widespread application of medical imaging technology, computed tomography has become a key means for early detection, accurate diagnosis and effective treatment of diseases. In clinical diagnosis, 3D medical image segmentation plays a vital role. It has irreplaceable clinical value for accurate diagnosis of diseases, detailed planning of surgical plans, evaluation of treatment effects and subsequent follow-up. However, 3D medical image segmentation faces a significant challenge: the process of obtaining gold standard data is complex and time-consuming, requiring professional doctors to carefully outline and annotate layer by layer. This process is not only time-consuming, but also has extremely high requirements for doctors' professional skills and experience.

[0003] Deep learning technology has broad application prospects in the field of 3D medical image segmentation, but its performance is highly dependent on a large amount of high-quality annotated data. Therefore, developing fast and accurate annotation methods is crucial to achieve high-precision 3D medical image segmentation. In order to further improve the accuracy and efficiency of medical image annotation, it is particularly important to study interactive 3D medical image segmentation algorithms that can respond in real time, are highly accurate, and support online learning. The algorithm can significantly speed up the annotation process, improve the accuracy of segmentation, optimize the work experience of doctors, reduce their burden, and efficiently generate a large amount of high-quality annotated data.

[0004] At present, interactive segmentation models for 3D medical images generally have problems such as large model size, numerous parameters, long inference time and high consumption of computing resources, which limit the practical application and deployment of the models. In addition, how to effectively convert doctors' interactive information, such as click actions, into prompts that the model can understand, thereby improving the accuracy of segmentation; and how to enable the model to continuously learn and absorb doctors' professional interpretation experience and enhance its online learning ability are of great significance to the application of medical image segmentation. Summary of the invention

[0005] The purpose of the present invention is to construct a neural network based on a visual base model and three-dimensional coding for three-dimensional medical images, so as to improve the accuracy and efficiency of interactive image segmentation of CT data.

[0006] The present invention is achieved by the following technical solution, comprising the following steps:

[0007] 1. Preprocessing stage: the original CT data is cropped and standardized for preprocessing, as follows:

[0008] (1) For the original CT data, a certain window size is used to crop the target and its surroundings, and the resulting data is regarded as a sample data;

[0009] (2) For each CT sample data, standardization preprocessing is performed according to the mean and variance of its lesion;

[0010] 2. Training phase: the preprocessed CT sample data is input into an interactive 3D medical image segmentation neural network based on the visual base model SAM (SegmentAnything Model) for learning; the segmentation neural network includes two parts: an image encoder based on the visual base model and a lightweight decoder, wherein the image encoder based on the visual base model maintains the visual Transformer feature extraction structure of the visual base model SAM, combines the low-rank adapter with the fully connected adapter, and inserts it into the backbone network of the visual base model SAM to encode the slices of the 3D image in a 2D form; in the decoding part, the lightweight decoder uses 3D convolution to enhance the information transfer between different slices, inputs the extracted features F into the convolutional neural network decoder, realizes the segmentation of the input image, and uses the gradient descent method to update the network parameters;

[0011] 3. Testing phase: The data to be segmented is subjected to online learning using the 3D coding extraction strategy to further update the model, as follows:

[0012] (1) Based on the trained network model, the test data is pre-segmented and manually labeled, and the labeled click information obtained in this process is 3D-encoded;

[0013] (2) For the three-dimensional spherical code Q, use the prompt encoder , improve the feature extraction ability of three-dimensional spherical coding; the prompt encoder uses convolutional layers with strides of 4 and 2 to downsample the click coding twice. After each downsampling, the paired attention module is used for feature extraction, and finally the mapped features are obtained through the fully connected layer. , mapping user information to the three-dimensional image feature space;

[0014] (3) The 3D coded features clicked by the user Token encoding features of images Additive fusion, fusion features Input to the image feature extraction module to obtain the comprehensive feature extraction result F;

[0015] (4) Input the comprehensive feature extraction result F into the decoder to achieve segmentation of the input image;

[0016] (5) Based on the segmentation results obtained by user clicks and the model prediction results, a combination of the cross entropy loss function and the Dice loss function is used to update the model online using gradient back propagation. User experience is organically integrated into the model training process in the form of click feedback, further improving the accuracy of the model.

[0017] Furthermore, in the training phase of the step, the specific method of the image encoder based on the visual base model is as follows:

[0018] First, each two-dimensional slice image I is encoded by word tokens to obtain the encoded features ,in is the token embedding layer; specifically, a two-dimensional convolutional layer with a convolution kernel size of 16×16 and a stride of 16 is used to downsample the token encoding of image I, and a one-dimensional convolutional layer with a convolution kernel length of 16 and a stride of 16 is used to downsample in the slice dimension;

[0019] Then, three-dimensional position encoding is performed on each pixel point on the image I, including learnable two-dimensional spatial position encoding and learnable one-dimensional deep position encoding, and the two are combined to obtain three-dimensional spatial position encoding; wherein, the position encoding generates a fixed-dimensional vector for each pixel position, and the vector contains the sine and cosine values ​​of the position, and the two are stacked along the channel dimension to obtain the position encoding feature of the point;

[0020] Finally, the image encoder of the visual base model is used to extract feature F; the image encoder includes 2 downsampling convolutional layers and 12 attention mechanism modules. The parameters of the attention mechanism modules use the pre-trained parameters of the visual base model, and 2 low-rank adapters and 1 fully connected adapter are added to improve the adaptability to new data.

[0021] Furthermore, the specific method of extracting feature F by the image encoder using the visual basis model is as follows:

[0022] (1) In the low-rank adapter, for the pre-trained weight matrix , restricting its update by low-rank decomposition:

[0023]

[0024] Among them, h is the output of the low-rank adapter, x is the input of the low-rank adapter. During the training process, Frozen and not receiving gradient updates, low-rank matrices A and B replace the original high-rank weight matrices and contain trainable parameters;

[0025] (2) In the fully connected adapter, the output of the linear layer of the visual Transformer module in the visual base model SAM , by adding a linear layer, the module output for:

[0026] ,

[0027] in, is a network composed of linear layers. is a learnable vector whose dimension is consistent with the model latent space.

[0028] Furthermore, in the training phase of the step, the specific method of the lightweight decoder in the decoding part is as follows:

[0029] First, the extracted feature F is input into two convolutional layers with different step sizes to extract features of different scales:

[0030] ,

[0031] in, , and They are convolutional layers with strides of 1, 2, and 4, respectively. This convolutional layer uses grouped convolution to divide the input channels into three groups, and each group is convolved with a shared convolution kernel;

[0032] Secondly, a shared linear transformation is used Will and Two multi-scale features are further mapped:

[0033] ,

[0034] The mapped features are combined with , use the interpolation algorithm to adjust to the same size and stack them to get :

[0035] ,

[0036] Then, the deep attention mechanism is used to transfer information in the depth direction of the three-dimensional image:

[0037] 1) Extract global features in height direction:

[0038] ,

[0039] in, It is a two-dimensional global feature extraction function, that is, an efficient average pooling operation, which compresses the height and width dimensions, retains the depth dimension, and converts the feature from three dimensions to one dimension;

[0040] 2) Use the self-attention mechanism to obtain the features after the depth direction information is transferred :

[0041] ,

[0042] Among them, SA is the self-attention operation;

[0043] 3) Adoption The original features are scaled and adjusted to achieve the transmission of information between different slices of 3D medical images:

[0044] ;

[0045] Finally, the prediction network consisting of a fully connected layer is used , to achieve the segmentation of input data:

[0046] .

[0047] Furthermore, the specific method of performing three-dimensional encoding in the steps is as follows:

[0048] First, for each voxel in the three-dimensional space , calculate its position Click location with user Distance ,Right now

[0049] ,

[0050] For each voxel, find the closest distance to all user clicks ,Right now

[0051] ,

[0052] Where N is the number of user clicks;

[0053] Then, the distance is binarized to obtain a three-dimensional spherical code Q, ensuring that the spherical area near the user's click position is 1 and the rest of the area is 0, marking the user's click position, and the value describes the distance between the current voxel and the user's click;

[0054] Finally, the user's clicks are divided into two point sets: positive clicks and negative clicks. The results of three-dimensional spherical encoding are calculated and stacked into two-channel three-dimensional voxels to encode the click information into three-dimensional form.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] (1) The present invention proposes an interactive 3D medical image segmentation model framework and designs an efficient fine-tuning strategy and network structure, which combines a low-rank adapter with a fully connected adapter, encodes slices of a 3D image in a 2D form, and uses 3D convolution in the decoding part to enhance information transfer between different slices, and finally obtains a 3D segmentation result. The framework designed by the present invention reduces the number of model parameters and the computing power required for training, while ensuring the high accuracy and real-time performance of the model;

[0057] (2) The present invention proposes a three-dimensional click coding feature extraction strategy, which can better input the user's click information into the network and continuously improve the online learning performance of the model. The user's interpretation experience is organically integrated into the model training process in the form of click feedback, further improving the accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a flow chart of the interactive three-dimensional medical image segmentation algorithm of the present invention. DETAILED DESCRIPTION

[0059] The following is a detailed description of an embodiment of the present invention in conjunction with the accompanying drawings. This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and process are given, but the protection scope of the present invention is not limited to the following embodiment.

[0060] This study used tumor and organ segmentation datasets from multiple institutions including Stanford University Medical Center, University of Minnesota Medical Center, and Sloan Kettering Cancer Center to conduct a series of experimental verifications in four lesions of lung cancer, pancreatic cancer, kidney cancer, liver cancer, and spleen organs. All training and test data were randomly repeated five times. The kidney tumor segmentation dataset originated from the 2023 Kidney and Kidney Tumor Segmentation Challenge (KiTS23), which contains a total of 489 CT scan images of kidney tumors. The lung cancer segmentation dataset uses the non-small cell lung cancer dataset, which contains 502 annotated CT data and PET / CT images. To increase the amount of data, the lung cancer segmentation dataset also integrates another 64 annotated CT data, including a total of 566 annotated data. The pancreatic dataset (Pancreas) was obtained from portal venous phase CT scans of 421 patients who underwent pancreatic mass resection surgery. The focus of this dataset is to segment the pancreatic parenchyma and pancreatic masses (cysts or tumors). There is a label imbalance problem between large structures (background), medium structures (pancreas), and small structures (tumors). It is part of the pancreatic tumor segmentation task (MSD-Pancreas) in the 2018 MICCAI Medical Segmentation All-Around Challenge. The liver vascular and tumor dataset includes 443 portal venous phase CT scans of various primary and metastatic liver tumors. Its target region of interest (ROI) is the blood vessels and tumors in the liver, which is of great significance for studying the relationship between liver tumors and surrounding blood vessels and the morphology and anatomical structure of liver blood vessels.

[0061] like Figure 1 As shown, the embodiment process of the interactive three-dimensional medical image segmentation algorithm of the present invention is as follows:

[0062] The first step is to perform preprocessing such as cropping and standardization on each CT sample data. The details are as follows:

[0063] First, for the original CT data, the target and its surroundings are cropped by using a certain window size, and the obtained data is regarded as a sample;

[0064] Then, each CT sample data is preprocessed by standardization according to the mean and variance of its lesion.

[0065] The second step is to input the training data into the interactive 3D image segmentation neural network based on the visual base model SAM, namely Segment Anything Model, for learning. The details are as follows:

[0066] (1) The image encoder based on the visual base model maintains the visual Transformer feature extraction structure of the visual base model SAM, combines the low-rank adapter and the fully connected adapter, inserts them into the backbone network of the visual base model SAM, and encodes the slices of the three-dimensional image in two-dimensional form.

[0067] First, each two-dimensional slice image I is encoded by word tokens to obtain the encoded features ,in Specifically, a two-dimensional convolutional layer with a convolution kernel size of 16×16 and a stride of 16 is used to downsample the image I to encode the tokens, and a one-dimensional convolutional layer with a convolution kernel length of 16 and a stride of 16 is used to downsample the image I in the slice dimension.

[0068] Then, three-dimensional position encoding is performed on each pixel point on the image I, including learnable two-dimensional spatial position encoding and learnable one-dimensional deep position encoding, and the two are combined to obtain three-dimensional spatial position encoding. Among them, the position encoding generates a fixed-dimensional vector for each pixel position, which contains the sine and cosine values ​​of the position. The two are stacked along the channel dimension to obtain the position encoding feature of the point.

[0069] Finally, the image encoder of the visual base model is used to extract feature F. The image encoder includes 2 downsampling convolutional layers and 12 attention mechanism modules. The parameters of the attention mechanism modules use the pre-trained parameters of the visual base model, and 2 low-rank adapters and 1 fully connected adapter are added to improve the adaptability to new data. The details are as follows:

[0070] 1) In the low-rank adapter, for the pre-trained weight matrix , restricting its update by low-rank decomposition:

[0071] ,

[0072] Among them, h is the output of the low-rank adapter, x is the input of the low-rank adapter. During the training process, Frozen and do not receive gradient updates, the low-rank matrices A and B replace the original high-rank weight matrices and contain trainable parameters.

[0073] 2) In the fully connected adapter, the output of the linear layer of the visual Transformer module in the visual base model SAM , by adding a linear layer, the module output for:

[0074] ,

[0075] in, is a network composed of linear layers. is a learnable vector whose dimension is consistent with the model latent space.

[0076] (2) Lightweight decoder

[0077] In the decoding part, three-dimensional convolution is used to enhance the information transfer between different slices. The extracted feature F is input into the convolutional neural network decoder to achieve the segmentation of the input image, and the gradient descent method is used to update the network parameters.

[0078] First, the extracted feature F is input into two convolutional layers with different step sizes to extract features of different scales:

[0079] ,

[0080] in, , and They are convolutional layers with strides of 1, 2, and 4 respectively. The convolutional layer uses grouped convolution to divide the input channels into three groups, and each group is convolved with a shared convolution kernel.

[0081] Secondly, a shared linear transformation is used Will and Two multi-scale features are further mapped:

[0082] ,

[0083] The mapped features are combined with , use the interpolation algorithm to adjust to the same size and stack them to get :

[0084] ,

[0085] Then, the deep attention mechanism is used to transfer information in the depth direction of the three-dimensional image:

[0086] 1) Extract global features in height direction:

[0087] ,

[0088] in, It is a two-dimensional global feature extraction function, that is, an efficient average pooling operation, which compresses the height and width dimensions, retains the depth dimension, and converts the features from three dimensions to one dimension.

[0089] 2) Use the self-attention mechanism to obtain the features after the depth direction information is transferred :

[0090] ,

[0091] Among them, SA is the self-attention operation.

[0092] 3) Adoption The original features are scaled and adjusted to achieve the transmission of information between different slices of 3D medical images:

[0093] ;

[0094] Finally, the prediction network consisting of a fully connected layer is used , to achieve the segmentation of input data:

[0095] .

[0096] The third step is to use the 3D coding extraction strategy to perform online learning on the data to be segmented during the testing phase to further update the model. The details are as follows:

[0097] (1) Based on the trained network model, the test data is pre-segmented and manually labeled, and the labeled click information obtained in this process is three-dimensionally encoded.

[0098] First, for each voxel in the three-dimensional space , calculate its position Click location with user Distance ,Right now

[0099] ,

[0100] For each voxel, find the closest distance to all user clicks ,Right now

[0101] ,

[0102] Where N is the number of user clicks.

[0103] Then, the distance is binarized to obtain a three-dimensional spherical code Q, ensuring that the spherical area near the user click position is 1 and the rest of the area is 0, marking the user click position, and the value describes the distance between the current voxel and the user click.

[0104] Finally, the user's clicks are divided into two point sets: positive clicks and negative clicks. The results of three-dimensional spherical encoding are calculated and stacked into two-channel three-dimensional voxels to encode the click information into three-dimensional form.

[0105] (2) For the three-dimensional spherical code Q, use the prompt encoder , improving the feature extraction capability of 3D spherical coding. The prompt encoder uses convolutional layers with strides of 4 and 2 to downsample the click coding twice. After each downsampling, the paired attention module is used for feature extraction, and finally the mapped features are obtained through the fully connected layer. , mapping user information to the three-dimensional image feature space.

[0106] (3) The 3D coded features clicked by the user Token encoding features of images Additive fusion, fusion features Input to the image feature extraction module to obtain the comprehensive feature extraction result F.

[0107] (4) Input the comprehensive feature extraction result F into the decoder to achieve segmentation of the input image.

[0108] (5) Based on the segmentation results obtained by user clicks and the model prediction results, a combination of the cross entropy loss function and the Dice loss function is used to update the model online using gradient back propagation. User experience is organically integrated into the model training process in the form of click feedback, further improving the accuracy of the model.

[0109] In this embodiment, the system used is Ubuntu 22.04 system, the GPU configuration is NVIDIA GeForce 309024G, and the development software environment is a server with Python 3.8. All comparison methods select Adam as the optimizer, and train 100 rounds with a batch size of 2 and a learning rate of 0.001.

[0110] In order to demonstrate the effectiveness of the present invention in segmentation accuracy, robustness, and real-time performance, kidney cancer, lung cancer, pancreatic cancer, and liver cancer datasets were selected as test sets, and the present invention method was compared with other existing methods on different lesion CT data. The evaluation indicators of segmentation accuracy use the Dice score and the 95% Hausdorff Distance 95th percentile (HD95%) commonly used in segmentation tasks, and the specific calculation method is as follows:

[0111] ,

[0112] in, To predict the segmentation result, is the actual segmentation result, To predict the area of ​​the segmentation result, is the area of ​​the actual segmentation result.

[0113] ,

[0114] in, The one-sided HD distance between the representative point sets A and B is:

[0115] ,

[0116] in, Represents the Euclidean distance. HD95% is defined as the 95% quantile of the HD distance.

[0117] As shown in Table 1, Table 2, Table 3, and Table 4, the method of the present invention has great advantages in segmentation accuracy and robustness, and has high computational efficiency and training parameter volume (the parameter volume only accounts for 12.1% to 51.4% of the comparison method) while achieving high segmentation accuracy. This makes it more feasible in practical applications, especially in resource-constrained environments.

[0118] In order to demonstrate the effect of the present invention on online learning, the labeled data is obtained by simulated clicks and user interactive clicks, and used for online updating of the model. Table 5 shows that the labeled data is first obtained and the model is fine-tuned under the simulated click strategy, and then the updated model is compared with the test data to evaluate the segmentation accuracy. In the lung cancer CT data experiment, 30 samples were labeled and fine-tuned, and then verified and tested with 30 samples respectively. The experimental results show the adaptability of the method under new labeled data and the segmentation accuracy on different data sets. In the model that has not been updated, the segmentation accuracy of the method of the present invention is higher, indicating that it has strong generalization ability. After the update, the Dice score and HD95% of the method of the present invention are further optimized.

[0119] In order to verify the application effect of the method in actual scenarios, this study invited an experienced radiologist to participate in the interactive annotation experiment. The experiment used the aforementioned lung cancer target data test set. The doctor annotated a total of 8 samples, and then used these annotated data to update the model and tested it on 12 samples. Before the start of the experiment, the doctor was asked to read the operation manual carefully to familiarize himself with the software operation and avoid interference caused by unfamiliarity. At the same time, the doctor took a break every half an hour during the annotation process to reduce the potential impact of fatigue and energy on the experimental results. After interactive annotation, the annotated data obtained by the doctor using the model of the present invention showed higher accuracy, while requiring less time and clicks.

[0120] In order to further verify the effectiveness of the three-dimensional spherical encoding strategy of the base model of the present invention, the strategy was applied under the model framework of the present invention and the Medical SAM Adapter, and experienced doctors were invited to use the two models for annotation. As shown in Table 6, compared with only using segmentation annotation updates, the introduction of doctor clicks improved the segmentation accuracy of the model of the present invention and the Medical SAM Adapter, indicating that the model can learn from the experience of doctor clicks. In addition, under the same training strategy and click encoding module, the model of the present invention achieved better results than the Medical SAM Adapter, proving the advantages of the model of the present invention in encoding.

[0121] Table 1 Comparison of segmentation performance of kidney cancer dataset

[0122]

[0123] Table 2 Comparison of segmentation performance of lung cancer dataset

[0124]

[0125] Table 3 Comparison of segmentation performance of pancreatic cancer dataset

[0126]

[0127] Table 4 Comparison of segmentation performance of liver cancer dataset

[0128]

[0129] Table 5 Comparison of online update capabilities of lung cancer CT datasets

[0130]

[0131] Table 6 Comparison of online update capabilities of lung cancer target scan dataset

[0132]

[0133] The role of each algorithm module in the present invention is shown in the model ablation experiment results in Table 7. In this experiment, lung cancer, pancreatic cancer, kidney cancer, and liver cancer are used as training data, and the trained network model is tested on four lesion test sets. Compared with the complete method, when the efficient fine-tuning strategy is not used, the training parameter quantity requirement is not high due to the reduction of the additional adapter module, but the segmentation accuracy will decrease: the model Dice score decreased by 0.039, and HD95% deteriorated by 0.45, which reflects that the introduction of low-rank adapters and fully connected adapters can improve the model's adaptability to new data and improve the model's segmentation accuracy; compared with the complete method, when the base model three-dimensional spherical encoding strategy is not used, the model's Dice score decreased by 0.081, and HD95% deteriorated by 1.91, which reflects that for three-dimensional interactive segmentation tasks, the click is combined with the image word feature in a three-dimensional spherical encoding manner, and then the visual base model is used for feature extraction, which can effectively improve the segmentation accuracy.

[0134] Table 7 Model ablation experiment results

[0135]

[0136] In summary, in the analysis of CT data of various lesions, the method of the present invention can more effectively realize interactive image segmentation, reduce the complexity of the model while improving the segmentation accuracy, and obtain better segmentation effect with fewer model parameters; at the same time, the method of the present invention can effectively integrate the user's interpretation experience into the training process of the model in the form of annotation feedback, thereby further improving the accuracy of the model.

[0137] The above description has fully introduced the specific embodiments of the present invention. It should be pointed out that any changes made by a person skilled in the art to the specific embodiments of the present invention do not deviate from the scope of the claims of the present invention. Accordingly, the scope of the claims of the present invention is not limited to the above specific embodiments.

Claims

1. An interactive three-dimensional medical image segmentation method, the method is applied to computer tomography data, i.e., CT data, and is characterized in that: The method comprises the following steps: (1.1) Preprocessing stage: the raw CT data is cropped and standardized for preprocessing, as follows: (1.1.1) For the original CT data, a certain window size is used to crop the target and its surroundings, and the resulting data is regarded as a sample data; (1.1.2) For each CT sample data, standardize and preprocess it according to the mean and variance of its lesion; (1.2) Training phase: the preprocessed CT sample data is input into an interactive 3D medical image segmentation neural network based on the visual base model SAM (SegmentAnything Model) for learning; the segmentation neural network includes two parts: an image encoder based on the visual base model and a lightweight decoder, wherein the image encoder based on the visual base model maintains the visual Transformer feature extraction structure of the visual base model SAM, combines the low-rank adapter and the fully connected adapter, and inserts them into the backbone network of the visual base model SAM to encode the slices of the 3D image in a 2D form; in the decoding part, the lightweight decoder uses 3D convolution to enhance the information transfer between different slices, inputs the extracted features F into the convolutional neural network decoder, realizes the segmentation of the input image, and uses the gradient descent method to update the network parameters; (1.3) Testing phase: The data to be segmented is subjected to online learning using the 3D coding extraction strategy, and the model is further updated as follows: (1.3.1) Based on the trained network model, the test data is pre-segmented and manually labeled, and the labeled click information obtained in this process is 3D-encoded; (1.3.2) For the three-dimensional spherical code Q, a prompt encoder is used , improve the feature extraction ability of three-dimensional spherical coding; the prompt encoder uses convolutional layers with strides of 4 and 2 to downsample the click coding twice. After each downsampling, the paired attention module is used for feature extraction, and finally the mapped features are obtained through the fully connected layer. , mapping user information to the three-dimensional image feature space; (1.3.3) The 3D coded feature clicked by the user Token encoding features of images Additive fusion, fusion features Input to the image feature extraction module to obtain the comprehensive feature extraction result F; (1.3.4) Input the comprehensive feature extraction result F into the decoder to achieve segmentation of the input image; (1.3.5) Based on the segmentation results obtained by user clicks and the model prediction results, a combination of the cross entropy loss function and the Dice loss function is used to update the model online by gradient back propagation. The user experience is organically integrated into the model training process in the form of click feedback, further improving the accuracy of the model.

2. The interactive three-dimensional medical image segmentation method according to claim 1, characterized in that: The specific method of the image encoder based on the visual basis model in the training phase of step (1.2) is as follows: First, each two-dimensional slice image I is encoded by word tokens to obtain the encoded features ,in is the token embedding layer; specifically, a two-dimensional convolutional layer with a convolution kernel size of 16×16 and a stride of 16 is used to downsample the token encoding of image I, and a one-dimensional convolutional layer with a convolution kernel length of 16 and a stride of 16 is used to downsample in the slice dimension; Then, three-dimensional position encoding is performed on each pixel point on the image I, including learnable two-dimensional spatial position encoding and learnable one-dimensional deep position encoding, and the two are combined to obtain three-dimensional spatial position encoding; wherein, the position encoding generates a fixed-dimensional vector for each pixel position, and the vector contains the sine and cosine values ​​of the position, and the two are stacked along the channel dimension to obtain the position encoding feature of the point; Finally, the image encoder of the visual base model is used to extract feature F; the image encoder includes 2 downsampling convolutional layers and 12 attention mechanism modules. The parameters of the attention mechanism modules use the pre-trained parameters of the visual base model, and 2 low-rank adapters and 1 fully connected adapter are added to improve the adaptability to new data.

3. The interactive three-dimensional medical image segmentation method according to claim 2, characterized in that: The specific method of extracting feature F using the image encoder of the visual basis model is as follows: (3.1) In the low-rank adapter, for the pre-trained weight matrix , restricting its update by low-rank decomposition: ， Among them, h is the output of the low-rank adapter, x is the input of the low-rank adapter. During the training process, Frozen and not receiving gradient updates, low-rank matrices A and B replace the original high-rank weight matrices and contain trainable parameters; (3.2) In the fully connected adapter, the output of the linear layer of the visual Transformer module in the visual base model SAM , by adding a linear layer, the module output for: ， in, is a network composed of linear layers. is a learnable vector whose dimension is consistent with the model latent space.

4. The interactive three-dimensional medical image segmentation method according to claim 1, characterized in that: In the training phase of step (1.2), the specific method of the lightweight decoder in the decoding part is as follows: First, the extracted feature F is input into two convolutional layers with different step sizes to extract features of different scales: ， in, , and They are convolutional layers with strides of 1, 2, and 4, respectively. This convolutional layer uses grouped convolution to divide the input channels into three groups, and each group is convolved with a shared convolution kernel; Secondly, a shared linear transformation is used Will and Two multi-scale features are further mapped: ， The mapped features are combined with , use the interpolation algorithm to adjust to the same size and stack them to get : ， Then, the deep attention mechanism is used to transfer information in the depth direction of the three-dimensional image: 1) Extract global features in height direction: ， in, It is a two-dimensional global feature extraction function, that is, an efficient average pooling operation, which compresses the height and width dimensions, retains the depth dimension, and converts the feature from three dimensions to one dimension; 2) Use the self-attention mechanism to obtain the features after the depth direction information is transferred : ， Among them, SA is the self-attention operation; 3) Adoption The original features are scaled and adjusted to achieve the transmission of information between different slices of 3D medical images: ； Finally, the prediction network consisting of a fully connected layer is used , to achieve the segmentation of input data: 。 5. The interactive 3D medical image segmentation method according to claim 2, characterized in that: The specific method of performing three-dimensional encoding in step (1.3.1) is as follows: First, for each voxel in the three-dimensional space , calculate its position Click location with user Distance ,Right now ， For each voxel, find the closest distance to all user clicks ,Right now ， Where N is the number of user clicks; Then, the distance is binarized to obtain a three-dimensional spherical code Q, ensuring that the spherical area near the user's click position is 1 and the rest of the area is 0, marking the user's click position, and the value describes the distance between the current voxel and the user's click; Finally, the user's clicks are divided into two point sets: positive clicks and negative clicks. The results of three-dimensional spherical encoding are calculated and stacked into two-channel three-dimensional voxels to encode the click information into three-dimensional form.

Images