Image segmentation method and device for pheochromocytoma, and medium
By incorporating block embedding layers, detail encoders, Transformer layers, and attention mechanisms into the image segmentation model, and combining these with loss function optimization, the issues of dataset integrity and accuracy in multi-organ segmentation around pheochromocytoma were addressed, resulting in more efficient image segmentation.
Patent Information
- Application Number
- CN202511001479.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-21
- Publication Date
- 2025-11-21
AI Technical Summary
Existing image segmentation methods for pheochromocytoma and its surrounding organs are insufficient in terms of dataset completeness and segmentation accuracy, which affects model training efficiency and generalization ability.
An image segmentation model is adopted, which transforms images through a block embedding layer and an image adapter. It combines a detail encoder, a Transformer layer and an attention mechanism to capture multi-scale features, and decodes them through a feature fusion decoder. The segmentation results are optimized using mean squared error and focus loss function.
It significantly improves the segmentation accuracy and integrity of pheochromocytoma and surrounding organs, enhances the ability to identify small and irregularly shaped targets, provides more detailed boundary information, and improves the accuracy and integrity of segmentation.
Smart Images
Figure CN120997497A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] Embodiments of the present application relate to the field of image processing, and in particular to a pheochromocytoma image segmentation method, device and medium. BACKGROUND
[0002] In the field of pheochromocytoma and its surrounding multi-organ segmentation, some current image segmentation methods (such as image segmentation methods combining multi-scale analysis and spatial frequency features) still face many challenges in the segmentation task involving the surrounding tissues of pheochromocytoma (such as renal artery and renal vein, etc.). The current available data set has significant deficiencies in completeness and fails to cover comprehensive annotation information of all related segmentation targets, which limits the efficiency and generalization ability of model training and affects the accuracy and completeness of image segmentation. SUMMARY
[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.
[0004] The purpose of the present application is to at least partially solve one of the technical problems existing in the related art. The embodiments of the present application provide a pheochromocytoma image segmentation method, device and medium, which improves the accuracy and completeness of image segmentation.
[0005] In an embodiment of the first aspect of the present application, a pheochromocytoma image segmentation method comprises:
[0006] obtaining a pheochromocytoma image, and inputting the pheochromocytoma image into an image segmentation model;
[0007] converting the pheochromocytoma image into an image embedding through a block embedding layer and an image adapter of the image segmentation model;
[0008] performing convolution on the pheochromocytoma image through a detail encoder of the image segmentation model to obtain a detail feature;
[0009] obtaining a multi-scale feature according to the image embedding through a Transformer layer of the image segmentation model using convolution layers of different scales;
[0010] splicing the image embedding and the multi-scale feature to obtain a new image embedding;
[0011] obtaining a query of an attention mechanism according to the new image embedding, obtaining a key and a value of the attention mechanism according to the detail feature, performing attention calculation to obtain a global feature, and the global feature comprises a first global feature, a second global feature, a third global feature and a fourth global feature;
[0012] encoding the point prompt as a prompt embedding by a prompt encoder of the image segmentation model;
[0013] decoding by a mask decoder of the image segmentation model according to the prompt embedding and the fourth global feature to obtain a first encoded feature;
[0014] decoding by a feature fusion decoder of the image segmentation model according to the first global feature, the second global feature, the third global feature and the first encoded feature to obtain a second encoded feature;
[0015] performing image segmentation according to the first encoded feature and the second encoded feature to obtain an image segmentation result.
[0016] According to certain embodiments of the first aspect of the present application, the converting the pheochromocytoma image into an image embedding by the patch embedding layer and the image adapter of the image segmentation model comprises:
[0017] converting the pheochromocytoma image into a patch embedding by the patch embedding layer of the image segmentation model;
[0018] converting the patch embedding into an image embedding by the image adapter.
[0019] According to certain embodiments of the first aspect of the present application, the convolving the pheochromocytoma image by the detail encoder of the image segmentation model to obtain a detail feature comprises:
[0020] passing the pheochromocytoma image through two convolution layers and a ReLU activation function layer to obtain a first convolution feature;
[0021] passing the pheochromocytoma image through a 3D linear deformation convolution layer to obtain a second convolution feature;
[0022] adding the first convolution feature and the second convolution feature to obtain a third convolution feature;
[0023] passing the third convolution feature through a plurality of convolution modules composed of a max-pooling layer, a convolution layer and a ReLU activation function layer in sequence to obtain a detail feature.
[0024] According to certain embodiments of the first aspect of the present application, the obtaining a multi-scale feature from the image embedding by the Transformer layer of the image segmentation model comprises:
[0025] passing the image embedding through a convolution layer and a ReLU activation function layer to obtain a first sub-multi-scale feature;
[0026] The first sub multi-scale features are respectively input into a plurality of convolution layers with different scales to obtain a plurality of second sub multi-scale features.
[0027] The sum of the second sub multi-scale features is input into a convolution layer and a batch normalization layer to obtain a multi-scale feature.
[0028] According to some embodiments of the first aspect of the present application, the plurality of convolution layers with different scales include a first standard convolution layer, a second standard convolution layer, a first dilated convolution layer, a second dilated convolution layer, and a third dilated convolution layer; the kernel sizes of the first standard convolution layer and the second standard convolution layer are different, and the dilation rates of the first dilated convolution layer, the second dilated convolution layer, and the third dilated convolution layer are different.
[0029] According to some embodiments of the first aspect of the present application, the attention calculation based on the new image embedding as the query of the attention mechanism and the detail feature as the key and value of the attention mechanism includes:
[0030] The new image embedding is taken as the query of the attention mechanism, the sum of the result of the detail feature passing through a multi-layer perceptron, the result of the detail feature passing through a 3D linear deformation convolution layer, and the result of the detail feature passing through a plurality of 3D convolution layers is taken as the updated detail feature, the updated detail feature is taken as the key and value of the attention mechanism, and attention calculation is performed to obtain a global feature.
[0031] According to some embodiments of the first aspect of the present application, the decoding by the feature fusion decoder of the image segmentation model based on the first global feature, the second global feature, the third global feature, and the first encoded feature to obtain a second encoded feature includes:
[0032] The feature fusion decoder of the image segmentation model fuses the features by residual connection and multi-level upsampling of the first global feature, the second global feature, the third global feature, and the first encoded feature to obtain a second encoded feature.
[0033] According to some embodiments of the first aspect of the present application, the loss function of the image segmentation model includes a mean square error loss function and a focal loss function.
[0034] Embodiments of the second aspect of the present application, an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image segmentation method for pheochromocytoma as described in embodiments of the first aspect of the present application.
[0035] The embodiment of the third aspect of the application is a computer storage medium storing computer executable instructions for performing the image segmentation method of the pheochromocytoma as described in the embodiment of the first aspect of the application.
[0036] The above scheme has at least the following beneficial effects: by supplementing local information, the model's ability to capture detailed features is significantly enhanced, thereby improving the segmentation accuracy of small-sized and irregularly-shaped targets; it can capture and fuse global features of different scales to provide the model with more rich semantic information, significantly improving the overall recognition accuracy of the target data set; by integrating global features at different levels and combining the constraints of the loss function, more detailed boundary information is provided for the segmentation mask output by the decoder, thereby further improving the accuracy and completeness of the segmentation. BRIEF DESCRIPTION OF DRAWINGS
[0037] The accompanying drawings are used to provide a further understanding of the technical solutions of the application, and constitute a part of the specification, and are used to explain the technical solutions of the application together with the embodiments of the application, and do not constitute a limitation on the technical solutions of the application.
[0038] Figure 1 is a step diagram of the image segmentation method provided by the embodiment of the application;
[0039] Figure 2 is a sub-step diagram of step S300 provided by the embodiment of the application;
[0040] Figure 3 is a sub-step diagram of step S400 provided by the embodiment of the application;
[0041] Figure 4 is a structure diagram of the image segmentation model provided by the embodiment of the application;
[0042] Figure 5 is a structure diagram of the detail encoder provided by the embodiment of the application;
[0043] Figure 6 is a structure diagram of the Transformer layer and the spatial multi-scale adapter provided by the embodiment of the application;
[0044] Figure 7 is a principle diagram of the bidirectional cross-attention mechanism provided by the embodiment of the application;
[0045] Figure 8 is a principle diagram of the feature fusion decoder provided by the embodiment of the application. DETAILED DESCRIPTION
[0046] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application.
[0047] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a manner different from the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims or above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.
[0048] The embodiments of the present application will be further described below with reference to the accompanying drawings.
[0049] The embodiments of the present application provide an image segmentation method for pheochromocytoma.
[0050] With reference to Figure 1 and Figure 4 , the image segmentation method comprises the following steps:
[0051] Step S100, acquiring a pheochromocytoma image, and inputting the pheochromocytoma image into an image segmentation model;
[0052] Step S200, converting the pheochromocytoma image into an image embedding through a patch embedding layer and an image adapter of the image segmentation model;
[0053] Step S300, performing convolution on the pheochromocytoma image through a detail encoder of the image segmentation model to obtain a detail feature;
[0054] Step S400, obtaining a multi-scale feature according to the image embedding through a Transformer layer of the image segmentation model using different scale convolution layers;
[0055] Step S500, obtaining a new image embedding by splicing the image embedding and the multi-scale feature;
[0056] Step S600, obtaining a query of an attention mechanism according to the new image embedding, obtaining a key and a value of the attention mechanism according to the detail feature, performing attention calculation to obtain a global feature; wherein the global feature comprises a first global feature, a second global feature, a third global feature and a fourth global feature;
[0057] Step S700, encoding a point prompt into a prompt embedding through a prompt encoder of the image segmentation model;
[0058] Step S800, decoding the first encoding feature according to the prompt embedding and the fourth global feature through the mask decoder of the image segmentation model;
[0059] Step S900, decoding the second encoding feature according to the first global feature, the second global feature, the third global feature and the first encoding feature through the feature fusion decoder of the image segmentation model;
[0060] Step S1000, performing image segmentation according to the first encoding feature and the second encoding feature to obtain an image segmentation result.
[0061] For step S100, a pheochromocytoma image of a CT image type is acquired, and the pheochromocytoma image is input to the image segmentation model. The pheochromocytoma image can be acquired from a large online database or from a CT image captured in real time.
[0062] The data set includes a pheochromocytoma data set (PPGLs), a multi-organ data set (BVTAMOS) and a kidney blood vessel data set (KiPA22). The labels included in the PPGLs include pheochromocytoma; the labels included in the BVTAMOS include spleen, right kidney, left kidney, stomach, esophagus, liver, aorta, gallbladder, pancreas, inferior vena cava, left adrenal gland, right adrenal gland, duodenum; and the labels included in the KiPA22 include renal vein and renal artery.
[0063] When training the image segmentation model, the pheochromocytoma image, the multi-organ image and the renal artery and vein image are used as training images together.
[0064] For step S200, the input image is converted into an image embedding through the block embedding layer and the image adapter of the image segmentation model, including the following steps: converting the input image into a block embedding through the block embedding layer of the image segmentation model; and converting the block embedding into an image embedding through the image adapter.
[0065] Reference Figure 2 For step S300, the input image is convolved through the detail encoder of the image segmentation model to obtain a detail feature, including the following steps:
[0066] Step S310, the input image is passed through two convolution layers and a ReLU activation function layer to obtain a first convolution feature;
[0067] Step S320, the input image is passed through a 3D linear deformation convolution layer to obtain a second convolution feature;
[0068] Step S330, the first convolution feature and the second convolution feature are added to obtain a third convolution feature;
[0069] Step S340, sequentially passing the third convolutional feature through a plurality of convolutional modules composed of a max-pooling layer, a convolutional layer and a ReLU activation function layer to obtain a detail feature.
[0070] With reference to Figure 5 In this embodiment, the detail encoder is used as a detail supplement convolutional branch of the image segmentation model. The detail encoder combines a standard convolutional layer and a 3D linear deformation convolutional layer. The 3D linear deformation convolutional layer automatically generates sampling points, which include not only regular grid points but also irregular sampling points that are dynamically adjusted according to the shape and size of the target, to overcome the limitations of using fixed sampling points in traditional convolutional operations when dealing with complex shapes and irregular structures. This not only supplements the feature maps generated by the visual transformer but also enhances the extraction capability of the features of deformable targets. The 3D linear deformation convolutional layer can realize sampling of various shapes according to different initial sampling shapes, thereby providing support for shape-variable targets. As the size of the convolutional kernel increases, the sampling pattern will become richer.
[0071] The algorithm formula for automatic sampling is as follows:
[0072]
[0073] C regular ={(z,x,y)|z∈[0,d),x∈[0,r),y∈[0,b)};
[0074] C mod ={(d,r,y')|y'∈[0,mod)};
[0075]
[0076] P' n =reshape([z1,...,z n ,x1,...,x n ,y1,...,y n ])。
[0077] Given the input convolutional kernel size N, define the two-dimensional plane grid size as b, the three-dimensional space depth as d, the number of rows in each layer as r, and the additional irregular sampling points mod. All integer regular coordinates C regular are generated in a three-dimensional space with a size of dxrxb, containing dxrxb points. When mod is greater than 0, all integer irregular coordinates C mod are generated in a three-dimensional space with a size of dxrxmod, containing mod points. Finally, the regular coordinates C regular and the irregular coordinates C modThe merged complete coordinate set Pn is obtained, and the tensor Pn' is output.
[0078] The detail supplement convolution branch significantly enhances the model's ability to capture detailed features by supplementing local information, thereby improving the segmentation accuracy of small-sized and irregularly shaped targets.
[0079] Referring to Figure 3 For step S400, the Transformer layer of the image segmentation model uses convolution layers of different scales to obtain multi-scale features from image embeddings, including the following steps:
[0080] Step S410: The image embedding is passed through a convolution layer and a ReLU activation function layer to obtain a first sub-multi-scale feature;
[0081] Step S420: The first sub-multi-scale feature is passed through multiple convolution layers of different scales to obtain multiple second sub-multi-scale features;
[0082] Step S430: The sum of the multiple second sub-multi-scale features is passed through a convolution layer and a batch normalization layer to obtain a multi-scale feature.
[0083] Referring to Figure 6 The spatial multi-scale adapter improves the recognition accuracy of the target data set. The spatial multi-scale adapter captures and fuses multi-scale features by constructing convolution layers of different scales. When performing dilated convolution on the input feature map X, the parameter r is the stride width of the convolution layer, thereby adaptively adjusting its receptive field size. The multiple convolution layers of different scales include a first standard convolution layer, a second standard convolution layer, a first dilated convolution layer, a second dilated convolution layer, and a third dilated convolution layer; the kernel sizes of the first standard convolution layer and the second standard convolution layer are different, and the dilation rates of the first dilated convolution layer, the second dilated convolution layer, and the third dilated convolution layer are different. The adapter provides two standard convolution layers with kernel sizes of 1x1x1 and 3x3x3, and three dilated convolution layers (Dilated Conv) with dilation rates of 12, 24, and 32, a total of five different receptive field configurations. By adding these features of different scales, the model can learn multi-scale features covering multiple receptive fields.
[0084] The multi-scale feature Y is represented as: Y[i] = ∑ k X(i+r*k)W[k]; where the parameter i represents the position of each pixel, the parameter W represents the convolution kernel, and the parameter k represents the convolution kernel size.
[0085] The spatial multi-scale adapter can capture and fuse global features of different scales, providing the model with more rich semantic information and significantly improving the overall recognition accuracy of the target data set.
[0086] For step S500, a new image embedding is obtained by image embedding and multi-scale feature concatenation.
[0087] Referring to Figure 7 For step S600, a query of the attention mechanism is obtained according to the new image embedding, a key and a value of the attention mechanism are obtained according to the detail feature, attention calculation is performed, and a first global feature is obtained, including the following steps: taking the new image embedding as the query Query of the attention mechanism, taking the sum of the result of the detail feature passing through the multi-layer perceptron (MLP), the result of the detail feature passing through the 3D linear deformation convolutional layer (LDConv3D), and the result of the detail feature passing through multiple 3D convolutional layers (Conv3D) as the updated detail feature, taking the updated detail feature as the key Key and the value Value of the attention mechanism, performing attention calculation to obtain the first global feature.
[0088] The detail supplementary convolution branch is dedicated to extracting fine features of the image, and the output of the detail feature is denoted as CNNX. At the same time, the multiple global Transformer layers are dedicated to obtaining the overall information of the image, and the output of the new image embedding is denoted as X.
[0089] The local feature supplement is fused in the model encoder part through the cross-attention mechanism. In this mechanism, the global feature X is used as the query (Query), and the detail feature CNNX is used as the key (Key) and the value (Value) respectively. Through the matching between the query and the key, the model can identify the part associated with the global feature and the detail feature, and update the global feature in a residual connection manner. The process can be represented as: CNNX = MLP(CNNX) + Conv3D(Conv3D(Conv3D(CNNX))) + LDConv3D(CNNX).
[0090] Among them, in order to enable the model to better capture the key details in the image, after each feature fusion, the output CNNX of the detail supplementary convolution branch is optimized by MLP, LDConv3D and Conv3D, so as to more effectively integrate the detail information in the next global Transformer layer.
[0091] By combining the advantages of convolutional neural network (CNN) in local feature extraction and the powerful ability of Transformer architecture in global feature extraction, it is beneficial to improve the image segmentation capability. By integrating global features of different levels and combining the constraints of the loss function, more detailed boundary information is provided for the segmentation mask output by the decoder, thereby further improving the accuracy and integrity of the segmentation.
[0092] The second global feature, the third global feature, and the fourth global feature are obtained in sequence by iterating steps S400, S500, and S600.
[0093] For step S700, the point prompt is encoded into a prompt embedding by a prompt encoder of the image segmentation model.
[0094] In the prompt encoder, encoding a point prompt into an embedding representation is a process of converting discrete prompt information into continuous and operable vectors. A point prompt generally refers to providing auxiliary information by specifying one or more points in an image in certain tasks. For example, in an image segmentation task, by clicking on the foreground or background points in the image, the model can better understand the target region. The embedding representation is a low-dimensional, continuous vector that can capture the semantic information and spatial position information of the point prompt. The role of the prompt encoder is to convert these discrete point prompts into embedding representations that the model can understand and process.
[0095] For step S800, the first encoded feature is obtained by decoding the prompt embedding and the fourth global feature through a mask decoder of the image segmentation model.
[0096] The mask decoder can generate a segmentation mask of the target object according to the input feature (the fourth global feature) and the prompt information (the prompt embedding). The mask decoder aligns the prompt embedding with the third global feature, then fuses the prompt embedding with the third global feature, restores the fused feature map to the same resolution as the input image through upsampling operations, and converts the fused feature map into a segmentation mask through a series of convolutional layers or Transformer layers. The final output is a two-dimensional mask with the same resolution as the input image, and the value of each pixel represents the probability that the pixel belongs to the target object (usually a value between 0 and 1).
[0097] Referring to Figure 8 For step S900, the second encoded feature is obtained by decoding the first global feature, the second global feature, the third global feature, and the first encoded feature through a feature fusion decoder of the image segmentation model, including the following steps: fusing the multiple global features and the first encoded feature through residual connection and multi-level upsampling by the feature fusion decoder of the image segmentation model to obtain the second encoded feature.
[0098] The output features are extracted from the 2nd, 5th, and 8th global Transformer layers, and the spatial resolution is represented as These features are not directly and simply fused into the mask of the decoder bidirectional cross-attention output. Instead, a skip connection is introduced to fuse image embeddings at different scales, and the image embedding after the residual connection is upsampled to avoid directly splicing low-resolution global features. In the multi-level upsampling path, the feature size is first upsampled from (8, 8, 8) to (16, 16, 16), and the channel number is reduced from 768 to 192. Subsequently, the feature is further upsampled from (16, 16, 16) to (32, 32, 32), and the channel number is further reduced to 96. This process uses deconvolution operations to achieve spatial resolution improvement. This design aims to better ensure the propagation of information flow and gradient, while ensuring that the resolution of the final generated segmentation mask matches the input image. The decoder adopts a double-head output structure, and the original mask output head generates the basic segmentation result, and the newly added boundary perception head displays the optimized boundary accuracy through attention weighting.
[0099] This process can be represented by the following formula:
[0100]
[0101] Extracting multi-level global features from the global Transformer Block Different scale image embeddings are fused through residual connection and step-by-step upsampling, which can be expressed as: The decoder bidirectional cross-attention output is defined as D∈R B×4×32×32×32 .
[0102] By capturing global features at different levels and combining loss function constraints, detailed boundary information is provided for the final generation of the decoder output mask.
[0103] For step S1000, image segmentation is performed according to the first encoded feature and the second encoded feature to obtain an image segmentation result.
[0104] In training the image segmentation model, the parameters of the image segmentation model are adjusted through the loss function to optimize the image segmentation model. During the training process, the original SAM-Med3D part parameters are frozen, while the parameters of the adapter module and other modules are updated.
[0105] The loss function includes a mean square error loss function and a focal loss function. A method of joint supervision of main loss and auxiliary loss is adopted. The main loss combines Dice loss and Focal loss, where Focal loss reduces the weight of easy-to-classify samples, making the model pay more attention to difficult-to-classify samples, thereby effectively alleviating the class imbalance problem and improving the performance and convergence speed of the model. The auxiliary loss uses the mean square error loss, which focuses on calculating the boundary loss to further optimize the boundary accuracy of the segmentation mask.
[0106] The loss calculation formula is as follows:
[0107]
[0108] L loss (p,t)=L DiceFocalLoss (p,t)+10*L MSELoss (p,t)。
[0109] where p represents the probability value predicted by the model, t represents the binary value of the true label (i.e. 0 or 1), p i and t i represent the predicted value and true value of the i-th pixel, respectively. In order to avoid division by zero during calculation, a small smoothing term ∈ is usually introduced.
[0110] Three datasets were used to train and test the proposed model: one containing 192 cases of pheochromocytoma (PPGLs), one containing 110 cases of abdominal multi-organ (BVTAMOS), and one containing 70 cases of renal artery and renal vein (KiPA22). During data preprocessing, the labels of these datasets were screened. The three datasets together provided 372 samples, of which 294 were selected for model training and the remaining 78 were used for testing.
[0111] For evaluation indicators, Dice score and 95% Hausdorff distance (HD95 distance) were used. Dice score was used to measure the pixel-level overlap between the segmentation map and the true label, and the higher the indicator value, the better the performance. 95% Hausdorff distance was used to evaluate the boundary similarity, and the lower the indicator value, the better the performance.
[0112] During the experiment, all experiments were run on a system equipped with NVIDIA RTX 4090 24GB GPU, with Linux 22.04 as the operating system. In the comparative experiment, multiple network models were verified, including SAM-Med3D, SegVol, SAT-Pro, nnUNet ResEnc L, nnFormer and CoTr, labeled as A, B, C, D, E and F respectively. These models were all tested using the default configuration. For the 3D UX-Net and SwinUNETR-V2 models, labeled as G and H respectively; custom configurations were used. The proposed model sets the batch size to 1, uses the Adamw optimizer, and sets the learning rate to 1e-4. In addition, a multi-step learning rate adjustment strategy is used, i.e. adjusting the learning rate every fixed step, with a learning rate decay coefficient of 0.1, a weight decay coefficient also set to 0.1, and a gradient accumulation step number of 20.
[0113] The Dice scores of the respective models on the internal test dataset are shown in Table 1.
[0114] Table 1. Dice scores of the models on the internal test dataset
[0115]
[0116]
[0117] The HD95 distance values of the respective models on the internal test dataset are shown in Table 1.
[0118] Table 2. HD95 distance values of the models on the internal test dataset
[0119]
[0120] Embodiments of the present application provide an electronic device. The electronic device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the image segmentation method of pheochromocytoma as described above when executing the computer program.
[0121] The electronic device can be any intelligent terminal including a computer.
[0122] Generally, for the hardware structure of the electronic device, the processor can be implemented in the form of a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the embodiments of the present application.
[0123] The memory can be implemented in the form of a ROM (Read Only Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), etc. The memory can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are saved in the memory and called and executed by the processor to implement the method of the embodiments of the present application.
[0124] The input / output interface is used to realize information input and output.
[0125] The communication interface is configured to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (for example, a USB, a network cable, and the like) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, and the like).
[0126] The bus transmits information between various components (for example, the processor, the memory, the input / output interface, and the communication interface) of the device. The processor, the memory, the input / output interface, and the communication interface are connected to each other through the bus to realize the communication connection between the device.
[0127] Embodiments of the present application provide a computer storage medium. The computer storage medium stores computer executable instructions for executing the image segmentation method of pheochromocytoma as described above.
[0128] Those of ordinary skill in the art can understand that all or some steps in the above disclosed method and system can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on a computer readable medium, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those of ordinary skill in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage, or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as known to those of ordinary skill in the art, communication media typically includes computer readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transport mechanisms, and can include any information delivery medium. In the above description of the specification, the description of the terms "one embodiment", "another embodiment", or "some embodiments" and the like means that the specific features, structures, materials, or characteristics described in connection with the embodiments or examples are included in at least one embodiment or example of the present application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in one or more embodiments or examples.
[0129] Those of ordinary skill in the art understand that all or some of the steps in the method disclosed above and the functional modules / units in the system and device can be implemented as software, firmware, hardware, and appropriate combinations thereof.
[0130] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.
[0131] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.
[0132] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.
[0133] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. For example, the apparatus embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices, or units, and can be in electrical, mechanical, or other forms.
[0134] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or replacements without departing from the spirit and principles of the present application. These equivalent modifications or replacements are all included in the scope of the claims of the present application and their equivalents.
Claims
1. A method of image segmentation of a pheochromocytoma, characterized by, The method comprises the following steps: acquiring a pheochromocytoma image, and inputting the pheochromocytoma image into an image segmentation model; converting the pheochromocytoma image into an image embedding through a patch embedding layer and an image adapter of the image segmentation model; performing convolution on the pheochromocytoma image through a detail encoder of the image segmentation model to obtain a detail feature; obtaining a multi-scale feature according to the image embedding through a Transformer layer of the image segmentation model using convolution layers of different scales; splicing the image embedding and the multi-scale feature to obtain a new image embedding; obtaining a query of an attention mechanism according to the new image embedding, obtaining a key and a value of the attention mechanism according to the detail feature, performing attention calculation to obtain a global feature, and the global feature comprises a first global feature, a second global feature, a third global feature and a fourth global feature; encoding a point prompt into a prompt embedding through a prompt encoder of the image segmentation model; decoding the prompt embedding and the fourth global feature through a mask decoder of the image segmentation model to obtain a first encoded feature; decoding the first global feature, the second global feature, the third global feature and the first encoded feature through a feature fusion decoder of the image segmentation model to obtain a second encoded feature; performing image segmentation according to the first encoded feature and the second encoded feature to obtain an image segmentation result.
2. The image segmentation method of pheochromocytoma according to claim 1, characterized by, The method comprises the following steps: converting the pheochromocytoma image into a patch embedding through a patch embedding layer of the image segmentation model; converting the patch embedding into an image embedding through the image adapter.
3. The image segmentation method of pheochromocytoma according to claim 1, characterized by, The method comprises the following steps: performing convolution on the pheochromocytoma image through two convolution layers and a ReLU activation function layer to obtain a first convolution feature; performing convolution on the pheochromocytoma image through a 3D linear deformation convolution layer to obtain a second convolution feature; adding the first convolution feature and the second convolution feature to obtain a third convolution feature; sequentially passing the third convolution feature through a plurality of convolution modules composed of a max-pooling layer, a convolution layer and a ReLU activation function layer to obtain a detail feature.
4. The image segmentation method of pheochromocytoma according to claim 1, characterized by, The method comprises the following steps: passing the image embedding through a convolution layer and a ReLU activation function layer to obtain a first sub-multi-scale feature; passing the first sub-multi-scale feature through a plurality of convolution layers of different scales respectively to obtain a plurality of second sub-multi-scale features; passing the sum of the plurality of second sub-multi-scale features through a convolution layer and a batch normalization layer to obtain a multi-scale feature.
5. The image segmentation method of pheochromocytoma according to claim 4, characterized by, The plurality of convolutional layers of different scales comprises a first standard convolutional layer, a second standard convolutional layer, a first dilated convolutional layer, a second dilated convolutional layer, and a third dilated convolutional layer; the kernel sizes of the first standard convolutional layer and the second standard convolutional layer are different, and the dilation rates of the first dilated convolutional layer, the second dilated convolutional layer, and the third dilated convolutional layer are different.
6. The image segmentation method of pheochromocytoma according to claim 1, characterized by, The query of the attention mechanism is obtained according to the new image embedding, the key and value of the attention mechanism are obtained according to the detail feature, attention calculation is performed, and a global feature is obtained, including: The new image embedding is taken as the query of the attention mechanism, the sum of the result of the detail feature passing through the multilayer perceptron, the result of the detail feature passing through the 3D linear deformation convolutional layer, and the result of the detail feature passing through the plurality of 3D convolutional layers is taken as the updated detail feature, the key and value of the attention mechanism are taken as the updated detail feature, attention calculation is performed, and a global feature is obtained.
7. The image segmentation method of pheochromocytoma according to claim 1, characterized by, The feature fusion decoder of the image segmentation model decodes the first global feature, the second global feature, the third global feature, and the first encoded feature to obtain a second encoded feature, including: The feature fusion decoder of the image segmentation model fuses the first global feature, the second global feature, the third global feature, and the first encoded feature through residual connection and multi-level upsampling to obtain a second encoded feature.
8. The image segmentation method of pheochromocytoma according to claim 1, characterized by, The loss function of the image segmentation model comprises a mean square error loss function and a focus loss function.
9. An electronic device, comprising: Including: A memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the image segmentation method of pheochromocytoma according to any one of claims 1 to 8.
10. A computer storage medium, characterized in that, Computer executable instructions are stored, and the computer executable instructions are used to execute the image segmentation method of pheochromocytoma according to any one of claims 1 to 8.