Construction method and application of abdominal organ segmentation sam model based on prompt enhancement

Abstract

The scheme provides a construction method and application of an abdominal organ segmentation SAM model based on prompt enhancement, acquires an abdominal organ three-dimensional CT image; inputs the abdominal organ three-dimensional CT image into a pre-trained lightweight organ segmentation model to output a first initial segmentation mask, inputs the abdominal organ three-dimensional CT image and a full prompt box into a SAM model to output a second initial segmentation mask; generates a first prompt point and a first prompt box based on the first initial segmentation mask and a prompt strategy, generates a second prompt point and a second prompt box based on the second initial segmentation mask and the prompt strategy, randomly fuses the first prompt point and the second prompt point to obtain a fused prompt point, takes a union of the first prompt box and the second prompt box to generate a fused prompt box, and collects the fused point information and the fused prompt box as prompt information; inputs the abdominal organ three-dimensional CT image and the prompt information into the abdominal organ segmentation SAM model based on prompt enhancement to output a segmentation result of the abdominal organ.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image processing, in particular to a construction method and application of an abdominal organ segmentation SAM model based on prompt enhancement. BACKGROUND

[0002] In recent years, machine learning and artificial intelligence technologies have made significant progress in the medical field, especially in the segmentation of abdominal organs in abdominal CT images. The segmentation of abdominal organs in abdominal CT images is the basis for computer-aided diagnosis, surgical simulation, image-guided intervention, and radiotherapy planning.

[0003] Although existing segmentation algorithms based on deep learning (such as U-Net, Mednext, Swin-Unet, nnU-Net, V-net, etc.) can achieve good results in the segmentation of remote sensing images, such ordinary segmentation algorithms still have limitations in segmenting common abdominal organs and dealing with complex situations such as poor image quality and organ shape variation.

[0004] To solve the problem of poor segmentation effect of medical images based on deep learning segmentation algorithms, Meta's SegmentAnythingModel (SAM) introduces prompt engineering and pre-trains on the SA-1B dataset containing 110 million images and 1.1 billion masks. SAM model exhibits good generalization performance in the zero-shot segmentation scenario. Its core architecture uses MAE pre-trained VisionTransformer as an image encoder, and cooperates with a decoder that can flexibly embed points, boxes, and text, etc. multi-modal prompts to achieve the general ability of "segmenting everything". SAM model can significantly improve the segmentation accuracy of abdominal organs, but there are still defects when using SAM model for abdominal organ segmentation:

[0005] 1. Difficulty in obtaining qualified and effective prompts: Automatic segmentation requires SAM to obtain qualified and effective prompts. Currently, human interaction labeling is still used to provide prompt information, which means that a large amount of human and time costs are still needed for prompt information labeling, and such labeling is prone to human bias. If the prompt information is inaccurate or incomplete, it will lead to large deviations in SAM segmentation results.

[0006] 2. Insufficient domain adaptability: The SAM model is pre-trained on natural image datasets, resulting in significantly poorer segmentation performance on medical images compared to natural images. The average Dice coefficient is relatively low. It should be noted that there are fundamental differences between medical images and remote sensing natural images. Medical images have more noise, more complex organ tissues, and strong specificity. Medical images display internal organs, tissues, and structures of the human body. The boundaries of these structures are often not as clear as the boundaries of objects in natural images, and the morphology is uncertain. For example, there is no clear physical boundary between the soft tissues of the human body (such as the liver and muscles), and their density and reflectivity can be very similar, resulting in blurred edges in the image. Compared to natural images and remote sensing images, edge segmentation and recognition of medical images are generally more complex and difficult.

[0007] 3. Cutting precision needs to be improved: Medical image segmentation tasks have very high requirements for segmentation precision. Relying solely on the SAM model cannot meet the needs of abdominal organ segmentation. In medical image segmentation tasks, segmentation accuracy is crucial because accurately identifying organ boundaries and lesion areas has a profound impact on clinical diagnosis and treatment planning. Therefore, even a small improvement in precision has important value and can improve the accuracy and reliability of clinical decision-making.

[0008] In summary, although the SAM model can improve the precision of organ segmentation to some extent, it still has the problems of insufficient domain adaptability, lack of prompt sources, and low segmentation precision in the abdominal organ segmentation scenario of CT images, which limits its application in abdominal organ segmentation scenarios. SUMMARY

[0009] The embodiments of the present application provide a method for constructing a SAM model for abdominal organ segmentation based on prompt enhancement and its application, which can solve the problems of insufficient domain adaptability and lack of prompt sources of the SAM model in the abdominal organ segmentation scenario of CT images, reduce the consumption of computing resources, and improve the accuracy of abdominal segmentation.

[0010] In a first aspect, the embodiments of the present application provide a method for constructing a SAM model for abdominal organ segmentation based on prompt enhancement, which includes the following steps:

[0011] S1: Obtain prompt information:

[0012] Process the abdominal organ three-dimensional CT image;

[0013] Input the preprocessed abdominal organ three-dimensional CT image into the pre-trained lightweight organ segmentation model to output a first initial segmentation mask, and generate a first prompt point and a first prompt box based on the first initial segmentation mask and the prompt strategy;

[0014] input the preprocessed abdominal organ three-dimensional CT image and the full prompt box into the SAM model to output a second initial segmentation mask, and the prompt points and the second prompt box are generated based on the second initial segmentation mask and a prompt strategy;

[0015] The first prompt point and the second prompt point are fused to obtain a fusion prompt point, the union of the first prompt box and the second prompt box is taken to generate a fusion prompt box, and the fusion point information and the fusion prompt box are summarized as prompt information;

[0016] S2: generate a training data set containing prompt information:

[0017] Obtain abdominal organ three-dimensional CT images and corresponding prompt information as a training data set;

[0018] S3: input the training data set into the SAM model for fine-tuning training to obtain an abdominal organ segmentation SAM model, wherein the image encoder and the prompt encoder of the SAM model are frozen during the training process.

[0019] In a second aspect, an embodiment of the present application provides a segmentation method of an abdominal organ SAM model based on prompt enhancement, including the following steps:

[0020] Process abdominal organ three-dimensional CT images;

[0021] input the processed abdominal organ three-dimensional CT image into a pre-trained lightweight organ segmentation model to output a first initial segmentation mask, and generate first prompt points and a first prompt box based on the first initial segmentation mask and a prompt strategy;

[0022] input the preprocessed abdominal organ three-dimensional CT image and the full prompt box into the SAM model to output a second initial segmentation mask, and the prompt points and the second prompt box are generated based on the second initial segmentation mask and a prompt strategy;

[0023] The first prompt point and the second prompt point are fused to obtain a fusion prompt point, the union of the first prompt box and the second prompt box is taken to generate a fusion prompt box, and the fusion point information and the fusion prompt box are summarized as prompt information;

[0024] input the processed abdominal organ three-dimensional CT image and the prompt information into the abdominal organ segmentation SAM model trained in the embodiment one to output a segmentation result of the abdominal organ.

[0025] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute the construction method of the abdominal organ SAM model based on prompt enhancement.

[0026] In a fourth aspect, the embodiments of the present application provide a readable storage medium, wherein the readable storage medium stores a computer program, the computer program comprises program codes for controlling a process to execute the process, and the process comprises the method for constructing the abdominal organ SAM model based on prompt enhancement.

[0027] The main contributions and innovations of the present application are as follows:

[0028] The scheme provided by the present scheme can solve the problems of insufficient field adaptability and lack of prompt source of the SAM model in the field of abdominal organ segmentation of medical images. First, the preprocessed abdominal organ three-dimensional CT image data is respectively obtained through a lightweight organ segmentation model to obtain a first segmentation mask, and through a SAM full prompt box to obtain a second segmentation mask. Then, the first segmentation mask and the second segmentation mask are respectively disturbed and processed to obtain corresponding prompt information. Finally, the prompt information from different models is fused to obtain prompt information that combines the segmentation advantages of different models. The abdominal organ three-dimensional CT image data with prompt information is further used for subsequent training of the SAM model. The prompt box of the present scheme combines the advantages of high segmentation prompt precision of the lightweight organ segmentation model and excellent generalization of the full box segmentation of the SAM model to improve the segmentation precision and generalization of the subsequent SAM fine segmentation result. Moreover, the mask decoder of the SAM model is only trained to reduce the training cost of the SAM model.

[0029] The details of one or more embodiments of the present application are presented in the following drawings and description to make other features, objects and advantages of the present application more apparent. BRIEF DESCRIPTION OF DRAWINGS

[0030] The drawings described herein are used to provide further understanding of the present application, and form a part of the present application. The schematic embodiments of the present application and the description thereof are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0031] Figure 1 is a flowchart of a method for constructing an abdominal organ segmentation SAM model based on prompt enhancement according to an embodiment of the present application;

[0032] Figure 2 is a basic architecture diagram of a SAM model according to the present application;

[0033] Figure 3 is a flowchart of an application method of an abdominal organ segmentation SAM model based on prompt enhancement according to an embodiment of the present application;

[0034] Figure 4 is a schematic diagram of a segmentation method of an abdominal organ SAM model based on prompt enhancement;

[0035] Figure 5 Different acquisition manners of the prompt point;

[0036] Figure 6 Fig. 1 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0037] The exemplary embodiments will be described in detail hereinbelow with reference to the drawings. In the following description, the same drawings refer to the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not meant to represent all embodiments consistent with one or more embodiments of the present description. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of the present description as detailed in the appended claims.

[0038] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in the present description in other embodiments. In some other embodiments, the steps included in the methods can be more or less than those described in the present description. Furthermore, a single step described in the present description can be broken down into multiple steps for description in other embodiments; and multiple steps described in the present description can be combined into a single step for description in other embodiments.

[0039] Embodiment One

[0040] As shown in Figure 1 The present scheme provides a construction method of an abdominal organ segmentation SAM model based on prompt enhancement, comprising the following steps:

[0041] S1: Obtain prompt information:

[0042] Process the abdominal organ three-dimensional CT image;

[0043] Input the preprocessed abdominal organ three-dimensional CT image into the pre-trained lightweight organ segmentation model to output a first initial segmentation mask, and generate a first prompt point and a first prompt box based on the first initial segmentation mask and a prompt strategy;

[0044] Input the preprocessed abdominal organ three-dimensional CT image and the full prompt box into the SAM model to output a second initial segmentation mask, and generate a second prompt point and a second prompt box based on the second initial segmentation mask and the prompt strategy;

[0045] Fuse the first prompt point and the second prompt point to obtain a fused prompt point, take the union of the first prompt box and the second prompt box to generate a fused prompt box, and collect the fused point information and the fused prompt box as the prompt information;

[0046] S2: Generate a training data set containing prompt information:

[0047] obtain three-dimensional CT images of abdominal organs and corresponding prompt information as a training data set;

[0048] S3: input the training data set into the SAM model for fine-tuning training to obtain an abdominal organ segmentation SAM model, wherein the image encoder and the prompt encoder of the SAM model are frozen during the training process.

[0049] The present scheme is based on the SAM model to construct an abdominal organ segmentation SAM model specially for abdominal organ segmentation of three-dimensional CT images. It is worth mentioning that the present scheme does not require expert manual annotation of the prompt information in the training data set, but uses different models to automatically generate enhanced prompt information.

[0050] In step S1:

[0051] In the field of medical image segmentation, different models have different advantages in abdominal organ segmentation tasks: lightweight organ segmentation models have higher accuracy, while SAM models have better generalization ability. Therefore, the present scheme combines the respective advantages of the segmentation masks generated by the two different models to generate prompt information.

[0052] In some embodiments, the abdominal organ three-dimensional CT images are preprocessed before being input into the lightweight organ segmentation model and the SAM model to adjust the size of the abdominal organ three-dimensional CT images to meet the segmentation requirements of the lightweight organ segmentation model and the SAM model.

[0053] In some embodiments, the pre-trained lightweight organ segmentation model is selected as a pre-trained BasicUNet and SegResNet, and the lightweight organ segmentation model is trained to extract an initial mask of the abdominal organs in the abdominal organ three-dimensional CT images as a first initial segmentation mask.

[0054] When the pre-trained lightweight organ segmentation model is selected as BasicUNet, BasicUNet is a simple and efficient U-Net architecture that extracts features and up-samples through multiple convolution blocks to generate a segmentation mask of the target abdominal organ.

[0055] When the pre-trained lightweight organ segmentation model is selected as SegResNet, SegResNet is constructed based on the residual learning concept. By introducing a residual block mechanism, it solves the problem of gradient vanishing and explosion in deep networks and performs well in image segmentation.

[0056] The training method of the lightweight organ segmentation model is a conventional method, that is, it can be trained using abdominal organ three-dimensional CT images labeled with abdominal organ masks, and no redundant description is made here.

[0057] The scheme not only inputs the three-dimensional CT image of the abdominal organ into the lightweight organ segmentation model to segment the first initial mask, but also inputs the same three-dimensional CT image of the abdominal organ and the corresponding full prompt box into the SAM model to segment the second initial mask. The SAM model can segment the general outline of the abdominal organ based on the full prompt box, wherein the full prompt box refers to the outer outline of the whole image picture.

[0058] The first initial mask and the second initial mask of the scheme can both provide basic shape and position information of the abdominal organ as the starting point for subsequent SAM fine segmentation.

[0059] Further, due to the complexity of the anatomical structure of the abdominal organ in the medical image, the prompt information for the SAM model has high sensitivity to the spatial position. Whether the position of a single prompt point of the first initial mask or the second initial mask is offset will cause significant inconsistency in the segmentation result of the subsequent SAM model. Based on the first initial mask and the second initial mask, the scheme also needs to use a corresponding prompt acquisition strategy to obtain the corresponding prompt information.

[0060] The prompt strategy for generating the second prompt point and the second prompt box from the second initial segmentation mask and the prompt strategy for generating the first prompt point and the first prompt box from the first initial segmentation mask are the same.

[0061] Regarding the prompt strategy of the first prompt box and the second prompt box, the scheme generates the first prompt box and the second prompt box by adding perturbations. Specifically, the diagonal coordinate points of the first initial segmentation mask are selected, and a certain number of pixel points are expanded outward based on the diagonal coordinate points to construct the first prompt box. The diagonal coordinate points of the second initial segmentation mask are selected, and a certain number of pixel points are expanded outward based on the diagonal coordinate points to construct the second prompt box.

[0062] Preferably, 5 pixel points are expanded outward based on the diagonal coordinate points to construct the first prompt box and the second prompt box.

[0063] Regarding the prompt strategy of the first prompt point and the second prompt point, the scheme generates the first prompt point and the second prompt point by emphasizing the edge weight. Specifically, the first prompt point is collected in the first initial mask by using the method that the edge ratio is greater than that of the remaining area, and the second prompt point is also collected in the second initial mask by using the method that the edge ratio is greater than that of the remaining area.

[0064] In some embodiments, edge points of the mask edge of the first initial mask are obtained, the point weight of each point in the first initial mask is calculated according to the distance between all points of the first initial mask and the mask edge, and the point weight of the edge point is amplified, and a first prompt point in the first initial mask is selected according to the point weight; edge points of the mask edge of the second initial mask are obtained, the point weight of each point in the second initial mask is calculated according to the distance between all points of the second initial mask and the mask edge, and the point weight of the edge point is amplified, and a second prompt point in the second initial mask is selected according to the point weight.

[0065] Specifically, in some embodiments, the distance between all points of the first initial mask or the second initial mask and the mask edge is calculated, and the point weight of each point is calculated according to the following formula:

[0066] Point weight = 1 / (distance + 1).

[0067] In some embodiments, the weight of the edge point of the first initial mask or the second initial mask is multiplied by 50.

[0068] In some embodiments, 100 prompt points are selected from the first initial mask, and 100 prompt points are selected from the second initial mask.

[0069] It should be noted that, as Figure 5 is a different way of obtaining prompt points, the application team has found through experiments that the training effect of the edge weight focused way to generate the first prompt point and the second prompt point is the best. In addition, since the first prompt point and the first prompt box of the first initial mask, and the second prompt point and the second prompt box of the second initial mask, contain the advantages of the respective models, the different prompt points and prompt boxes also need to be fused and merged.

[0070] Regarding the acquisition of the fusion prompt point;

[0071] Since the prompt points generated by the lightweight organ segmentation model and the SAM model present differences in spatial distribution and information content, the first prompt point and the second prompt point are fused to obtain a fusion prompt point. A random sampling fusion strategy is adopted, that is, half of the points in the first prompt point are randomly selected, and half of the points in the second prompt point are also randomly selected. These two parts of points are integrated into a new set of fusion prompt points for subsequent fine segmentation of the SAM model. That is, half of the prompt points in the first prompt point are randomly selected, half of the prompt points in the second prompt point are randomly selected, and the prompt points from the first prompt point and the second prompt point are fused to obtain a fusion prompt point.

[0072] Regarding the acquisition of the fusion prompt box:

[0073] Given that the SAM model pays great attention to the information within the frame in the segmentation model, the prompt box in an ideal state should be able to completely contain the organ to be segmented. However, some prompt boxes generated by the initial mask may be small in size and cannot completely cover the contours of the abdominal organs. Therefore, this solution uses a union-like fusion method to fuse the first prompt box and the second prompt box.

[0074] Specifically, the union of the first and second boxes is calculated. The smaller of the upper-left corner coordinates of the first and second boxes is taken, while the larger of the lower-right corner coordinates of the first and second boxes is taken. A fused box is generated based on these smaller and larger values. This approach allows the new fused box to encompass both the first and second boxes, increasing the likelihood that the fused box will fully cover the entire organ and optimizing the segmentation results.

[0075] In step S3:

[0076] This solution adopts a more efficient training strategy in the SAM model architecture, namely freezing the parameters of the image encoder and hint encoder and only fine-tuning the parameters of the mask decoder. This strategy reduces the demand for computing resources by optimizing the training process, improves training efficiency, and maintains model performance. The solution achieves effective training optimization of the SAM model under limited resource conditions and can provide support for subsequent image segmentation tasks.

[0077] like Figure 2 As shown, Figure 2 The following diagram shows the basic architecture of the SAM model. The SAM model consists of a parallel image encoder and a cue encoder, as well as a mask decoder connected to the outputs of the image encoder and the cue encoder. The cue information is input to the cue encoder, and the preprocessed CT image is input to the image encoder. This solution inputs the training dataset into the SAM model for fine-tuning, resulting in a SAM model for abdominal organ segmentation. During training, the image encoder and cue encoder of the SAM model are frozen, and only the parameters of the mask decoder are trained.

[0078] Example 2

[0079] Based on the same concept, Figure 3 As shown, the present application also proposes an application method of the abdominal organ segmentation SAM model based on the hint enhancement, or a segmentation method of the abdominal organ SAM model based on the hint enhancement, comprising the following steps:

[0080] Obtain three-dimensional CT images of abdominal organs;

[0081] Input the 3D CT image of the abdominal organs into the pre-trained lightweight organ segmentation model to output a first initial segmentation mask, and input the 3D CT image of the abdominal organs and the full prompt box into the SAM model to output a second initial segmentation mask; generate a first prompt point and a first prompt box based on the first initial segmentation mask and the prompt strategy, generate a second prompt point and a second prompt box based on the second initial segmentation mask and the prompt strategy, randomly fuse the first prompt point and the second prompt point to obtain a fused prompt point, take the union of the first prompt box and the second prompt box to generate a fused prompt box, and summarize the fused point information and the fused prompt box as prompt information;

[0082] The abdominal organ 3D CT image and prompt information are input into the abdominal organ segmentation SAM model trained in Example 1 to output the abdominal organ segmentation result.

[0083] The schematic diagram of the segmentation method of the abdominal organ SAM model based on hint enhancement is as follows Figure 4 As shown, the original three-dimensional CT image of the abdominal organs is input into the pre-trained lightweight organ segmentation model to obtain a first initial mask, and a first prompt point and a first prompt box are generated based on the first initial mask; the original three-dimensional CT image of the abdominal organs and the full prompt box are input into the SAM model to obtain a second initial mask, and a second prompt point and a second prompt box are generated based on the second initial mask, the first prompt point and the second prompt point are fused to obtain a fused prompt point, and the first prompt box and the second prompt box are fused to obtain a fused prompt box, and the fused prompt point and the fused prompt box are used as prompt information to input into the abdominal organ segmentation SAM model to output a fine abdominal organ segmentation result. It can be seen that the final generated abdominal organ segmentation result has a significantly improved fineness compared with the segmentation result of the early stage using the pre-trained lightweight organ segmentation model or the SAM model alone.

[0084] Example 3

[0085] This embodiment also provides an electronic device, referring to Figure 6 , including a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above-mentioned embodiments of the method for constructing a SAM model for abdominal organ segmentation based on hint enhancement or the method for segmenting a SAM model for abdominal organ segmentation based on hint enhancement.

[0086] Specifically, the processor 402 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0087] The memory 404 can include a mass storage device 404 for data or instructions. The memory 404 can be used to store or cache various data files needed for processing and / or communication, and possible computer program instructions executed by the processor 402.

[0088] The processor 402 implements any one of the above-mentioned methods for constructing a prompt-enhanced abdominal organ segmentation SAM model or the method for segmenting a prompt-enhanced abdominal organ segmentation SAM model by reading and executing the computer program instructions stored in the memory 404.

[0089] Optionally, the above-mentioned electronic device can further include a transmission device 406 connected to the processor 402 and an input / output device 408 connected to the processor 402.

[0090] The transmission device 406 can be used to receive or send data via a network. Specific examples of the network can include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (NIC) which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (RF) module for communicating with the Internet in a wireless manner.

[0091] The input / output device 408 is used for inputting or outputting information. In the present embodiment, the input information can be an abdominal organ three-dimensional CT image, and the output information can be a segmented abdominal organ.

[0092] Optionally, in the present embodiment, the processor 402 can be configured to perform the following steps by computer program:

[0093] S1: obtaining prompt information:

[0094] inputting the abdominal organ three-dimensional CT image into the pre-trained lightweight organ segmentation model to output a first initial segmentation mask, and inputting the abdominal organ three-dimensional CT image and the full prompt box into the SAM model to output a second initial segmentation mask;

[0095] generating a first prompt point and a first prompt box based on the first initial segmentation mask and the prompt strategy, generating a second prompt point and a second prompt box based on the second initial segmentation mask and the prompt strategy, randomly fusing the first prompt point and the second prompt point to obtain a fused prompt point, taking the union of the first prompt box and the second prompt box to generate a fused prompt box, and collecting the fused point information and the fused prompt box as the prompt information;

[0096] S2: Construct a training data set:

[0097] Obtain three-dimensional CT images containing prompt information to make a training data set;

[0098] S3: Input the training data set into the SAM model for fine-tuning training to obtain an abdominal organ segmentation SAM model, wherein the image encoder and the prompt encoder of the SAM model are frozen during the training process.

[0099] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation manners, which will not be described here again.

[0100] Generally, various embodiments can be implemented in hardware or special-purpose circuitry, software, logic or any combination thereof. Some aspects of the application can be implemented in hardware, while other aspects can be implemented by firmware or software executed by a controller, microprocessor or other computing device, but the application is not limited thereto. Although various aspects of the application can be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein can be implemented in hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

[0101] Embodiments of the application can be implemented by computer software executable by a data processor of the mobile device such as in the processor entity, or by hardware, or by a combination of software and hardware. Computer software or program, also called program product, including software routines, applets and / or macros, can be stored in any apparatus-readable data storage medium and they include program instructions for implementing certain aspects of embodiments. The program product can include one or more computer-executable components or modules for implementing embodiments of the application. The one or more computer-executable components or modules can be one or more software codes or portions thereof. Further, at this regard, it should be noted that any blocks of the logical flow of the figures can represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software can be stored on such physical media as memory chips, or memory blocks implemented in the processor, magnetic media such as hard disk or floppy disks, and optical media such as for example DVD and the data variants thereof, CD, etc. The physical media are non-transitory media.

[0102] Those skilled in the art should understand that each technical feature of the above embodiments can be combined arbitrarily, and for the sake of brevity, each technical feature in the above embodiments is not described in all possible combinations, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the description.

[0103] The above embodiments only express several implementation manners of the present application, the description is more specific and detailed, but it should not be understood as the limitation of the scope of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for constructing a SAM model for abdominal organ segmentation based on hint enhancement, characterized in that: include: S1: Get prompt information: Processing of three-dimensional CT images of abdominal organs; Inputting the processed abdominal organ 3D CT image into a pre-trained lightweight organ segmentation model to output a first initial segmentation mask, and generating a first cue point and a first cue box based on the first initial segmentation mask and a cue strategy; Input the processed abdominal organ 3D CT image and the full prompt box into the SAM model to output a second initial segmentation mask, and generate a second prompt point and a second prompt box based on the second initial segmentation mask and the prompt strategy; Fusing the first prompt point and the second prompt point to obtain a fused prompt point, taking the union of the first prompt frame and the second prompt frame to generate a fused prompt frame, and summarizing the fused prompt point and the fused prompt frame as prompt information; S2: Generate a training dataset containing prompt information: Obtain three-dimensional CT images of abdominal organs and corresponding prompt information as training data sets; S3: The training dataset is input into the SAM model for fine-tuning training to obtain the abdominal organ segmentation SAM model, wherein the image encoder and prompt encoder of the SAM model are frozen during the training process.

2. The method for constructing a SAM model for abdominal organ segmentation based on hint enhancement according to claim 1, characterized in that: Select the diagonal coordinate points of the first initial segmentation mask, and expand a certain number of pixels outward based on the diagonal coordinate points to construct a first prompt box; select the diagonal coordinate points of the second initial segmentation mask, and expand a certain number of pixels outward based on the diagonal coordinate points to construct a second prompt box.

3. The method for constructing a SAM model for abdominal organ segmentation based on hint enhancement according to claim 1, characterized in that: Obtain edge points of the mask edge of the first initial mask, calculate the point weight of each point in the first initial mask according to the distance between all points of the first initial mask and the mask edge, amplify the point weight of the edge point, and select the first hint point in the first initial mask according to the point weight; Obtain edge points of the mask edge of the second initial mask, calculate the point weight of each point in the second initial mask according to the distance between all points of the second initial mask and the mask edge, amplify the point weight of the edge point, and select the second hint point in the second initial mask according to the point weight.

4. The method for constructing a SAM model for abdominal organ segmentation based on hint enhancement according to claim 1, characterized in that: Half of the prompt points are randomly selected from the first prompt point, and half of the prompt points are randomly selected from the second prompt point. The prompt points from the first prompt point and the prompt points from the second prompt point are fused to obtain fused prompt points.

5. The method for constructing a SAM model for abdominal organ segmentation based on hint enhancement according to claim 1, characterized in that: The union of the first prompt frame and the second prompt frame is obtained, the smaller value of the upper left corner coordinates of the first prompt frame and the second prompt frame is taken, and the larger value of the lower right corner coordinates of the first prompt frame and the second prompt frame is taken, and a fused prompt frame is generated based on the smaller value and the larger value.

6. The method for constructing a SAM model for abdominal organ segmentation based on hint enhancement according to claim 1, characterized in that: The pre-trained lightweight organ segmentation model is the pre-trained BasicUNet or SegResNet.

7. The method for constructing a SAM model for abdominal organ segmentation based on hint enhancement according to claim 1, characterized in that: The first cue point and the second cue point each contain 100 cue points.

8. A segmentation method for abdominal organs based on a hint-enhanced SAM model, characterized in that: include: Processing of three-dimensional CT images of abdominal organs; Inputting the processed abdominal organ 3D CT image into a pre-trained lightweight organ segmentation model to output a first initial segmentation mask, and generating a first cue point and a first cue box based on the first initial segmentation mask and a cue strategy; Input the processed abdominal organ 3D CT image and the full prompt box into the SAM model to output a second initial segmentation mask, and generate a second prompt point and a second prompt box based on the second initial segmentation mask and the prompt strategy; Fusing the first prompt point and the second prompt point to obtain a fused prompt point, taking the union of the first prompt frame and the second prompt frame to generate a fused prompt frame, and summarizing the fused prompt point and the fused prompt frame as prompt information; The three-dimensional CT image of the abdominal organs and the prompt information are input into the prompt-enhanced abdominal organ segmentation SAM model to output the segmentation results of the abdominal organs, wherein the prompt-enhanced abdominal organ segmentation SAM model is constructed according to the construction method of the prompt-enhanced abdominal organ segmentation SAM model according to any one of claims 1 to 7.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the method for constructing a SAM model for abdominal organ segmentation based on hint enhancement according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for constructing a SAM model for abdominal organ segmentation based on hint enhancement according to any one of claims 1 to 7 is implemented.

Images