Three-dimensional medical image segmentation method and system based on improved SAM

Abstract

The invention provides a three-dimensional medical image segmentation method and system based on improved SAM, and belongs to the technical field of image processing. The method comprises the following steps: firstly, acquiring a medical image training set and inputting the medical image training set into a three-dimensional medical image segmentation network to train the three-dimensional medical image segmentation network; inputting a medical image to be segmented into the trained three-dimensional medical image segmentation network for image segmentation; specifically, a dual-channel image encoder in the segmentation network extracts embedding with depth correlation and spatial correlation based on a multi-head attention mechanism and an adapter; the prompt code generator divides the volume image into small blocks to obtain block image embedding; and finally, the block image embedding and the embedding with depth correlation and spatial correlation are input into a multi-layer aggregation decoder to obtain a segmentation result of the three-dimensional medical image. According to the invention, local and boundary detail features of the medical image can be effectively distinguished, and the problem of boundary blurring during three-dimensional medical image segmentation is solved.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a three-dimensional medical image segmentation method and system based on improved SAM. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Three-dimensional medical image segmentation is a core task in the field of medical image processing. It can accurately locate the lesion area, identify and extract specific areas or structures from three-dimensional medical image data, thereby greatly improving the accuracy and reliability of medical diagnosis and providing doctors with more accurate diagnostic information.

[0004] The inventors found that in recent years, three-dimensional medical image segmentation has attracted the attention of researchers. Three-dimensional medical images can be pre-processed by noise removal and then feature extraction to locate specific lesion areas. CT images are a kind of medical image composed of many slices that are interconnected and combined. When segmenting such three-dimensional medical images, existing models will miss the depth correlation (vertical) and spatial correlation (horizontal) between three-dimensional slices, so the segmentation results may be inaccurate. In addition, the existing three-dimensional medical image segmentation model does not have the ability to segment any image like SAM (Segment Anything Model), that is, the ability to segment any image without pre-training on a specific image. In addition, the traditional three-dimensional medical image segmentation method only focuses on the overall image, ignoring the local detail information and boundary information of the image. In medical images, some diseases often occur in local details or boundary positions, and this information is not utilized. Summary of the invention

[0005] In order to overcome the above-mentioned deficiencies of the prior art, the present invention provides a three-dimensional medical image segmentation method and system based on improved SAM, which can effectively distinguish the local and boundary details of medical images and solve the problem of blurred boundaries during three-dimensional medical image segmentation.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0007] A first aspect of the present invention provides a three-dimensional medical image segmentation method based on improved SAM.

[0008] A three-dimensional medical image segmentation method based on improved SAM, comprising:

[0009] Acquire a medical image training set and a medical image to be segmented, wherein the medical image training set includes medical images of known healthy areas and diseased areas;

[0010] Inputting the medical image training set into a three-dimensional medical image segmentation network, and training the three-dimensional medical image segmentation network;

[0011] The medical image to be segmented is input into a trained three-dimensional medical image segmentation network for image segmentation; the three-dimensional medical image segmentation network includes a dual-channel image encoder, a prompt code generator and a multi-layer aggregate decoder, specifically: the dual-channel image encoder simultaneously extracts embeddings with depth correlation and spatial correlation based on a multi-head attention mechanism and an adapter; the prompt code generator divides the volume image into small blocks to obtain block image embeddings; the block image embeddings and the embeddings with depth correlation and spatial correlation are input into a multi-layer aggregate decoder to obtain the segmentation result of the three-dimensional medical image.

[0012] Furthermore, the dual-channel image encoder includes two channels, namely a spatial branch channel and a depth branch channel; both channels are provided with a multi-head attention mechanism and an adapter.

[0013] Furthermore, the multi-head attention mechanisms in the spatial branch channel and the depth branch channel are respectively used to obtain the spatial correlation and depth correlation of the medical image to be segmented; and the adapter is used to fine-tune the image model in the channel.

[0014] Furthermore, the hint code generator uses a sliding window method to divide the volume image embedded by the dual-channel image encoder into small blocks.

[0015] Furthermore, the hint code generator uses trilinear interpolation to obtain point embedding based on point coordinates to ensure that the hint and block image embedding have the same semantic features.

[0016] Furthermore, the three-dimensional medical image segmentation network is trained, including: inputting the training set into a dual-channel image encoder, using the dual-channel image encoder to extract image features from the images in the training set and outputting an image feature map; inputting the prompt information of the training set into a prompt encoding generator to obtain a prompt embedding; inputting the obtained image feature map and the prompt embedding into a multi-layer aggregation decoder, embedding the prompt information, and obtaining an image feature map embedded with the prompt information; calculating the feature similarity between the image features and the original image features, and generating a predicted label; constructing a combined loss function of cross entropy loss and dice loss based on the pixel-level gap between the predicted label and the true label; when the combined loss function value of the network no longer decreases, stopping the training to obtain a trained three-dimensional medical image segmentation network.

[0017] Furthermore, in each round of combined loss function operation, the loss is calculated by taking the weighted sum of the cross entropy loss and the dice loss, and the calculated result is the loss between each generated prediction mask and the true label.

[0018] The second aspect of the present invention provides a three-dimensional medical image segmentation system based on improved SAM.

[0019] A three-dimensional medical image segmentation system based on improved SAM, comprising:

[0020] An image acquisition module is configured to: acquire a medical image training set and a medical image to be segmented, wherein the medical image training set includes medical images of known healthy areas and diseased areas;

[0021] A model training module is configured to: input the medical image training set into a three-dimensional medical image segmentation network, and train the three-dimensional medical image segmentation network;

[0022] The image segmentation module is configured to: input the medical image to be segmented into a trained three-dimensional medical image segmentation network for image segmentation; the three-dimensional medical image segmentation network includes a dual-channel image encoder, a prompt code generator and a multi-layer aggregate decoder, specifically: the dual-channel image encoder simultaneously extracts embeddings with depth correlation and spatial correlation based on a multi-head attention mechanism and an adapter; the prompt code generator divides the volume image into small blocks as input to obtain block image embedding; the block image embedding and the embedding with depth correlation and spatial correlation are input into the multi-layer aggregate decoder to obtain the segmentation result of the three-dimensional medical image.

[0023] The third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in a three-dimensional medical image segmentation method based on an improved SAM as described in the first aspect of the present invention.

[0024] The fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in a three-dimensional medical image segmentation method based on an improved SAM as described in the first aspect of the present invention are implemented.

[0025] One or more of the above technical solutions have the following beneficial effects:

[0026] The present invention designs a three-dimensional medical image segmentation model including a dual-channel image encoder, a hint coding generator and a multi-layer aggregation decoder; wherein, the dual-channel image encoder can effectively apply the SAM (Segmentation Arbitrary Image Model) applied to two-dimensional natural images to three-dimensional medical image segmentation after improvement, so as to better obtain the depth correlation and spatial correlation of three-dimensional slices at the same time, and any image can be segmented without pre-training on specific images. In addition, the hint coding generator uses the three-dimensional medical image segmented into small blocks with internal structures as input, which helps to retain more local details, thereby solving the problem of local detail loss in the position embedding of the three-dimensional medical image; at the same time, the multi-layer aggregation decoder distinguishes the details of the medical image through the U-Net with multi-layer jump connections, effectively solving the problem of blurred boundaries during the segmentation of three-dimensional medical images.

[0027] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0029] Figure 1 This is a method flow chart of a three-dimensional medical image segmentation method based on improved SAM in Embodiment 1 of the present invention.

[0030] Figure 2 Schematic diagram of the model structure of a three-dimensional medical image segmentation method based on improved SAM in Example 1 of the present invention.

[0031] Figure 3 This is a schematic diagram of the model structure of the adapter in the first embodiment of the present invention.

[0032] Figure 4 This is a comparison chart of 3D medical image segmentation visualization results of a model of a 3D medical image segmentation method based on an improved SAM in Example 1 of the present invention and an existing baseline model. DETAILED DESCRIPTION

[0033] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0034] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0035] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0036] The overall idea proposed by the present invention is: in order to effectively improve the accuracy of medical image segmentation and distinguish the detailed features such as local and boundary features of medical images, and solve the problem of blurred boundaries during three-dimensional medical image segmentation, the present invention uses a dual-channel mechanism to extract the depth correlation and spatial correlation of three-dimensional medical images. It is worth noting that the model of the present invention uses three-dimensional medical images in both the training stage and the test stage. Compared with the prior art, the model of the present invention ensures that the medical image is segmented in areas with dense medical semantics rather than useless medical background areas, thereby improving the segmentation accuracy of the model. On this basis, the present invention uses a hint coding generator to take small blocks of three-dimensional medical images with internal structures as input; in addition, the present invention also proposes a multi-layer aggregation mechanism, which enables the model to obtain boundary information of three-dimensional medical images.

[0037] Embodiment 1

[0038] This embodiment discloses a three-dimensional medical image segmentation method based on improved SAM.

[0039] like Figure 1 As shown, a three-dimensional medical image segmentation method based on improved SAM includes:

[0040] Step S1, obtaining a medical image training set and a medical image to be segmented, wherein the medical image training set includes medical images of known healthy areas and diseased areas;

[0041] Step S2, inputting the medical image training set into a three-dimensional medical image segmentation network, and training the three-dimensional medical image segmentation network;

[0042] Step S3, inputting the medical image to be segmented into a trained three-dimensional medical image segmentation network for image segmentation; the three-dimensional medical image segmentation network includes a dual-channel image encoder, a prompt code generator and a multi-layer aggregate decoder, specifically: the dual-channel image encoder simultaneously extracts embeddings with depth correlation and spatial correlation based on a multi-head attention mechanism and an adapter; the prompt code generator takes a small block of a three-dimensional medical image with an internal structure as input to obtain a block image embedding; the block image embedding and the embedding with depth correlation and spatial correlation are input into a multi-layer aggregate decoder to obtain a segmentation result of the three-dimensional medical image.

[0043] Based on the above-mentioned 3D medical image segmentation method of the present invention, it is possible to effectively distinguish the detailed features such as local and boundary features of medical images, and solve the problem of blurred boundaries when segmenting 3D medical images. To facilitate the understanding of the technical solution of the present invention, the above-mentioned implementation method and process are further explained and illustrated:

[0044] In step S1, a medical image training set and a medical image to be segmented are obtained; wherein the medical image training set includes medical images of known healthy areas and diseased areas;

[0045] The specific division ratio of the data set can be set according to actual needs, and this embodiment does not impose any specific restrictions on this. At the same time, this embodiment can also apply a series of data enhancement technologies to improve the quality of medical images, such as execution, scaling, flipping, moving and rotating.

[0046] In step S2, the medical image training set is input into the 3D medical image segmentation network, and the 3D medical image segmentation network is trained. Specifically:

[0047] The three-dimensional medical image segmentation network includes a dual-channel image encoder, a prompt encoding generator, and a multi-layer aggregate decoder; the training of the three-dimensional medical image segmentation network can be achieved by the following steps:

[0048] Step S2-1, input the training set into a dual-channel image encoder, use the dual-channel image encoder to extract image features from the images in the training set and output an image feature map; at the same time, divide the training set into small blocks and combine them with manually constructed point prompts as prompt information to input the prompt encoding generator to obtain prompt embedding; input the obtained image feature map and prompt embedding into a multi-layer aggregation decoder, embed the prompt information, and obtain an image feature map embedded with the prompt information;

[0049] Step S2-2, calculate the feature similarity between the image features and the original image features, and generate a predicted label; construct a combined loss function of cross entropy loss and dice loss based on the pixel-level difference between the predicted label and the true label; when the combined loss function value of the network no longer decreases, stop training to obtain a trained three-dimensional medical image segmentation network.

[0050] Among them, the 3D medical image segmentation network model is trained to predict the pixel-level difference between the mask and the true label. The total loss of the segmentation is the combined loss of the cross entropy loss and the dice loss. In each round of combined loss function calculation, the loss is calculated by weighted summing the cross entropy loss and the dice loss. The result is the loss of each generated predicted mask and the true label. The definition formula of the combined loss function is as follows:

[0051]

[0052] Among them, L() represents the combined loss function, Y represents the predicted segmentation result, is the real label; L CE represents the cross entropy loss function, L Dicerepresents the dice loss function; N represents the number of classes, K represents the number of voxels, and Y k,n represents the prediction of class i at voxel j, represents the true label of class i at voxel j.

[0053] In step S3, the medical image to be segmented is input into a trained three-dimensional medical image segmentation network, image features are extracted from the medical image to be segmented, and an image feature map is obtained; position features are embedded in the image feature map, and a three-dimensional medical image is given.

[0054] In this embodiment, the purpose of training the 3D medical image segmentation network model is to predict the location of tumors and organs. The structure of the 3D medical image segmentation network model is as follows: Figure 2 As shown; in the specific implementation process, the dual-channel image encoder simultaneously extracts embeddings with depth correlation and spatial correlation based on the multi-head attention mechanism and the adapter; the prompt code generator takes small blocks of the three-dimensional medical image with internal structure as input to obtain block image embedding; here, the block image embedding and point prompts are used to calculate the point embedding with local details. Finally, the block image embedding and the embedding with depth correlation and spatial correlation are input into the multi-layer aggregate decoder to obtain the segmentation result of the three-dimensional medical image. Among them, the input end of the dual-channel image encoder is used to input the medical image, the input end of the prompt code generator is used to input the three-dimensional medical image block prompt code, and the input end of the multi-layer aggregate decoder is used to input the output of the dual-channel image encoder and the prompt code generator. More specifically, the segmentation of the three-dimensional medical image can be achieved by the following process:

[0055] Step S3-1, the dual-channel image encoder simultaneously extracts embeddings with depth correlation and spatial correlation based on the multi-head attention mechanism and the adapter.

[0056] In order to simultaneously obtain the depth correlation and spatial correlation of the three-dimensional slices and extract more complete three-dimensional image features, this embodiment proposes a method such as Figure 2 The dual-channel image encoder shown. Specifically, this embodiment divides the image encoder into two channels according to the order of the input dimensions, namely the spatial branch channel and the depth branch channel, and both channels are provided with a multi-head attention mechanism and an adapter. Specifically, the channel is divided into two channels, the longitudinal channel and the transverse channel, the longitudinal channel serves as the channel of the depth branch, and the transverse channel serves as the channel of the spatial branch. Among them, the multi-head attention mechanism in the spatial branch channel and the depth branch channel is used to obtain the spatial correlation and depth correlation of the medical image to be segmented, respectively; the adapter is used to fine-tune the image model in the channel.

[0057] Furthermore, the definition formula of the multi-head attention mechanism (MHA) is as follows:

[0058] MHA(Q,K,V)=Concat(head1,…head i …, head h )W O ;

[0059] Among them, MHA() represents the multi-head attention mechanism, Q,K,V∈R D×N×L Represents query tensor, key tensor and value tensor respectively; head i represents the i-th attention head, W O ∈R L×L Represents the weight matrix, and Concat(,) represents the concatenation operation.

[0060] Furthermore, this embodiment also fine-tunes the image model by introducing an improved adapter. The general adapter only includes upsampling, downsampling and relu activation function; the PEFT adapter is an effective image model fine-tuning tool. On this basis, this embodiment improves the adapter by adding three-dimensional convolution after downsampling and fine-tuning the SAM encoder to better extract the depth and spatial correlation of the three-dimensional medical image. Figure 3 The improved adapter formula is defined as follows:

[0061] Adap t er(X)=X + Ac t (3DConv(XW down ))W up ;

[0062] Among them, A d apter() represents the adapter function, X represents the input of the adapter, W up and W down They represent upsampling and downsampling respectively. Act() represents the activation function, and 3DConv() represents the 3D convolutional layer.

[0063] Subsequently, a slice operation is performed on the three-dimensional medical image, and the thickness of the slice is D. The formula for the slice operation is defined as follows:

[0064]

[0065] Among them, S i,j,k Represents the pixel value at the (i, j, k)th position after slicing, V i,j,k represents the pixel value at the (i, j, k)th position before slicing, and D represents the number of samples of V in the K dimension.

[0066] In the spatial branch channel, the sample S∈R D×N×LInput to the multi-head attention mechanism, where N represents the number of embeddings, L represents the length of the embedding, and D represents the number of operations. Keep the D dimension of the sample S unchanged and perform multi-head attention operations on the N×L dimension to obtain an embedding with spatial correlation. Next, pass this embedding to the improved adapter, which is calculated as follows:

[0067] S space =Adapter(MultiHead(LN(S)));

[0068] Among them, S space Represents embedding with spatial correlation, Adapter() represents the introduced improved adapter, MultiHead() represents the multi-head attention mechanism, LN represents layer normalization, and S represents the input sample.

[0069] In the deep branching channel, the input S∈R D×N×L Convert to S ′ ∈R N×D×L , the transposition formula is defined as follows:

[0070] S ′ =Transpose(S);

[0071] Among them, S′∈R N×D×L The N dimension of the sample S is kept unchanged, and a multi-head attention operation is performed on the D*L dimension to obtain an embedding with deep correlation; then, it is embedded into the improved adapter; next, the result of the deep branch channel is converted back to the original shape S depth ,Right now:

[0072] S depth =Transpose(Adapter(MultiHead(LN(S ′ ))));

[0073] Finally, while preserving the depth correlation and spatial correlation, the results generated by the spatial branch channel and the depth branch channel are added together, that is:

[0074] S correlation =Add(S space ,S depth );

[0075] Step S3-2: prompt the code generator to divide the volume image into small blocks to obtain block image embedding.

[0076] In order to reduce the local detail loss of 3D image position embedding in the previous model, this embodiment redesigns the 3D hint code generator, which generates effective hint information by dividing the 3D medical image into small blocks and capturing local detail information between the small blocks. The whole process of capturing local detail information is as follows: Figure 2 shown.

[0077] Furthermore, the prompt code generator uses a sliding window method to divide the volume image embedded in the dual-channel image encoder into small blocks, namely:

[0078] B i，j，k =I(d start , d start +d-1,h start ,h start +h-1,w start , w start +w-1);

[0079] Among them, i, j, k represent the depth, height and width coordinates respectively; d, h, w represent the size of the small block respectively, d start ,h start and w start It represents the starting position of the small block in depth, height and width, and I(,) is a function that returns the corresponding position coordinates.

[0080] Next, the hint code generator uses trilinear interpolation to obtain point embedding based on the point coordinates to ensure that the hint and block image embedding have the same semantic features. The trilinear interpolation formula is defined as follows:

[0081]

[0082] Where V represents point embedding, (u,v,w) represents point cue coordinates, and ω(·) represents the weight function.

[0083] Then, this embodiment randomly selects a point embedding as a global query and applies a self-attention mechanism between the global query and the remaining point embeddings to better capture the detailed information between the point embeddings. The formula of the self-attention mechanism is defined as follows:

[0084]

[0085] Among them, Attention(,) represents the self-attention mechanism function, Q represents the global query vector, K = VW Q is the calculated attention weight, W Q represents the weight matrix, d k is the embedding length of each point, and n represents the number of point embeddings.

[0086] Finally, a cross-attention mechanism is applied between the patch image embedding and the global query to obtain the final patch hint embedding with hint information, which contains more local detail information, namely:

[0087]

[0088] where γ represents the block hint embedding, B i,j,k represents the image embedding vector, Q represents the query vector, d is used to control the value range, and softmax() represents the activation function.

[0089] Step S3-3: input the block image embedding and the embedding with depth correlation and spatial correlation into a multi-layer aggregation decoder to obtain a segmentation result of a three-dimensional medical image.

[0090] In order to solve the problem of blurred boundaries in three-dimensional medical images, this embodiment adopts a U-Net with multi-layer skip connections. Figure 1 The whole process of multi-layer aggregation decoder is defined as follows:

[0091] First, this embodiment designs a convolutional activation block CIRU, which includes three-dimensional convolution, normalization, ReLU activation function and upsampling. The embedding X generated by the dual-channel image encoder and the block prompt embedding Y generated by the prompt encoding generator are sent to the convolutional activation block CIRU. The formula of the above process is defined as follows:

[0092] X raw =CIRU(X);

[0093] Y raw =CIRU(Y);

[0094] Among them, X raw and Y raw $ represents an embedding with the same resolution as the original image Z. Next, X, Y are merged with the original image to obtain the parallel image feature F con ,Right now:

[0095] F con =Concatenated(X raw ,Y raw ,Z);

[0096] Among them, Z represents the embedding of the original image, and Concatenated(,) represents the concatenation operation.

[0097] On this basis, this embodiment designs another convolution activation block CIRC, which includes 3D convolution, normalization, ReLU activation function and 3D convolution. conSent to the convolutional activation block CIRC, the formula of the above process is defined as follows:

[0098] Mask pre =CIRC(F con );

[0099] Based on the above process, the segmentation of three-dimensional medical images can be achieved. In order to show the difference between the present invention and the existing segmentation methods, this embodiment shows the visual comparison of the 3D medical image segmentation results of the present invention model and the existing baseline model on the KiTS21 dataset. The specific comparison results are as follows: Figure 4 As shown; green represents tumors and red represents organs.

[0100] Embodiment 2

[0101] This embodiment discloses a three-dimensional medical image segmentation system based on improved SAM.

[0102] A three-dimensional medical image segmentation system based on improved SAM, comprising:

[0103] An image acquisition module is configured to: acquire a medical image training set and a medical image to be segmented, wherein the medical image training set includes medical images of known healthy areas and diseased areas;

[0104] A model training module is configured to: input the medical image training set into a three-dimensional medical image segmentation network, and train the three-dimensional medical image segmentation network;

[0105] The image segmentation module is configured to: input the medical image to be segmented into a trained three-dimensional medical image segmentation network for image segmentation; the three-dimensional medical image segmentation network includes a dual-channel image encoder, a prompt code generator and a multi-layer aggregate decoder, specifically: the dual-channel image encoder extracts embeddings with depth correlation and spatial correlation based on a multi-head attention mechanism and an adapter at the same time; the prompt code generator divides the volume image into small blocks as input to obtain a block image embedding; the block image embedding and the embedding with depth correlation and spatial correlation are input into the multi-layer aggregate decoder to obtain the segmentation result of the three-dimensional medical image. The output end of the dual-channel image encoder and the output end of the prompt code generator are respectively connected to the input end of the multi-layer aggregate decoder.

[0106] Embodiment 3

[0107] The purpose of this embodiment is to provide a computer-readable storage medium.

[0108] A computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps in a three-dimensional medical image segmentation method based on an improved SAM as described in the first embodiment of the present disclosure.

[0109] Embodiment 4

[0110] The purpose of this embodiment is to provide an electronic device.

[0111] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in a three-dimensional medical image segmentation method based on an improved SAM as described in the first embodiment of the present disclosure are implemented.

[0112] The steps involved in the apparatuses of the above embodiments 2, 3 and 4 correspond to the method embodiment 1, and the specific implementation methods can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0113] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0114] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A three-dimensional medical image segmentation method based on improved SAM, characterized in that: include: Acquire a medical image training set and a medical image to be segmented, wherein the medical image training set includes medical images of known healthy areas and diseased areas; Inputting the medical image training set into a three-dimensional medical image segmentation network, and training the three-dimensional medical image segmentation network; The medical image to be segmented is input into a trained three-dimensional medical image segmentation network for image segmentation; the three-dimensional medical image segmentation network includes a dual-channel image encoder, a prompt code generator and a multi-layer aggregate decoder, specifically: the dual-channel image encoder simultaneously extracts embeddings with depth correlation and spatial correlation based on a multi-head attention mechanism and an adapter; the prompt code generator divides the volume image into small blocks to obtain a block image embedding; The block image embedding and the embedding with depth correlation and spatial correlation are input into a multi-layer aggregation decoder to obtain a segmentation result of a three-dimensional medical image.

2. A three-dimensional medical image segmentation method based on improved SAM as claimed in claim 1, characterized in that: The dual-channel image encoder includes two channels, namely a spatial branch channel and a depth branch channel; a multi-head attention mechanism and an adapter are arranged in both channels.

3. A three-dimensional medical image segmentation method based on improved SAM as claimed in claim 2, characterized in that: The multi-head attention mechanisms in the spatial branch channel and the depth branch channel are respectively used to obtain the spatial correlation and depth correlation of the medical image to be segmented; the adapter is used to fine-tune the image model in the channel.

4. A three-dimensional medical image segmentation method based on improved SAM as claimed in claim 1, characterized in that: The hint code generator uses a sliding window method to divide the volume image embedded in the dual-channel image encoder into small blocks.

5. A three-dimensional medical image segmentation method based on improved SAM as claimed in claim 4, characterized in that: The hint code generator uses trilinear interpolation to obtain point embedding based on point coordinates to ensure that the hint and block image embedding have the same semantic features.

6. A three-dimensional medical image segmentation method based on improved SAM as claimed in claim 1, characterized in that: Training of 3D medical image segmentation network, including: The training set is input into a dual-channel image encoder, and the dual-channel image encoder is used to extract image features from the images in the training set and output an image feature map; the prompt information of the training set is input into a prompt encoding generator to obtain a prompt embedding; The obtained image feature map and prompt embedding are input into the multi-layer aggregation decoder to embed the prompt information, thereby obtaining the image feature map embedded with the prompt information; the feature similarity between the image features and the original image features is calculated to generate the predicted label; based on the pixel-level gap between the predicted label and the true label, a combined loss function of the cross entropy loss and the dice loss is constructed; when the combined loss function value of the network no longer decreases, the training is stopped to obtain the trained three-dimensional medical image segmentation network.

7. A three-dimensional medical image segmentation method based on improved SAM as claimed in claim 6, characterized in that: In each round of combined loss function operation, the loss is calculated by weighted summing the cross entropy loss and the dice loss, and the result is the loss between each generated prediction mask and the true label.

8. A three-dimensional medical image segmentation system based on improved SAM, characterized in that: include: An image acquisition module is configured to: acquire a medical image training set and a medical image to be segmented, wherein the medical image training set includes medical images of known healthy areas and diseased areas; A model training module is configured to: input the medical image training set into a three-dimensional medical image segmentation network, and train the three-dimensional medical image segmentation network; The image segmentation module is configured to: input the medical image to be segmented into a trained three-dimensional medical image segmentation network for image segmentation; the three-dimensional medical image segmentation network includes a dual-channel image encoder, a prompt code generator and a multi-layer aggregate decoder, specifically: the dual-channel image encoder simultaneously extracts embeddings with depth correlation and spatial correlation based on a multi-head attention mechanism and an adapter; the prompt code generator divides the volume image into small blocks to obtain block image embeddings; The block image embedding and the embedding with depth correlation and spatial correlation are input into a multi-layer aggregation decoder to obtain a segmentation result of a three-dimensional medical image.

9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps in the three-dimensional medical image segmentation method based on improved SAM as described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the three-dimensional medical image segmentation method based on improved SAM as described in any one of claims 1 to 7 are implemented.

Images