Method and system for segmenting kidney region in dynamic kidney development image based on mixed attention branches
By integrating SAM with VMamba model and combining cross-branch attention mechanisms, fusing global semantic and fine-grained features, using continuous feature decoder and implicit feature alignment module, the existing kidney dynamic imaging image segmentation method is solved in complex details and The problem of insufficient accuracy and high computing resources in low-contrast image processing achieves high-precision and efficient renal dynamic imaging image segmentation.
Patent Information
- Application Number
- CN202410962521.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-07-18
AI Technical Summary
The existing renal dynamic imaging image segmentation method has limited segmentation capabilities when processing complex details and low-contrast images, consumes large computing resources, complex model structure and cumbersome training. The SAM model prompts that the driver input mechanism cannot fully play its role in medical image segmentation.
Using a hybrid attention branch-based approach, the Segment Anything model (SAM) is integrated with the Visual Mamba (VMamba) model, combined with the cross-branch attention mechanism, and integrate global semantic features and fine-grained local details through cross-branch attention partial fusion, using continuous The feature decoder and the implicit feature alignment module perform feature fusion, eliminating the prompt drive input mechanism, and simplifying the segmentation workflow.
Improve segmentation accuracy and generalization capabilities, simplify workflow, enhance the robustness and stability of the model, improve training efficiency and effect, and solve the insufficient accuracy and high computing resources of existing methods in complex medical image processing. The problem.
Smart Images

Figure CN118918126A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the fields of computer science and medical imaging, and more specifically, to a method and system for segmenting a kidney region in a renal dynamic imaging image based on a hybrid attention branch. Background Art
[0002] RDS image segmentation plays a key role in computer-aided diagnosis systems, especially in identifying and analyzing specific anatomical structures. In recent years, with the rapid development of deep learning technology, RDS image segmentation technology has been significantly improved. However, existing segmentation methods still face many challenges when processing RDS images with complex details and low contrast.
[0003] Existing renal dynamic imaging image segmentation technologies mainly include three categories: renal dynamic imaging image segmentation methods based on classic UNet and its variants (such as UNet++, Attention UNet), renal dynamic imaging image segmentation methods based on Transformer models (such as SwinUNet), and renal dynamic imaging image segmentation methods based on hybrid architectures (such as MambaUNet). In addition, the Segment Anything model (SAM for short) has also received attention in recent years. Although it is mainly used in natural image segmentation tasks, it has gradually begun to be applied to the field of medical image segmentation.
[0004] However, the above four existing renal dynamic imaging image segmentation methods all have some defects that cannot be ignored:
[0005] First, the renal dynamic imaging image segmentation method based on the classic UNet and its variants enhances feature transfer through skip connections, but its segmentation ability is limited when processing renal dynamic imaging images with rich details; especially when facing complex anatomical structures and low-contrast areas, this model is difficult to accurately capture important features (for example, UNet++ improves feature propagation by introducing nested skip connections, but there is still a problem of blurred boundaries in complex renal dynamic imaging images; Attention UNet enhances feature selection through the attention mechanism, but when processing low-contrast images, the attention mechanism may not be able to effectively distinguish important features from background noise), which will lead to inaccurate renal dynamic imaging image segmentation results.
[0006] Second, the renal dynamic imaging image segmentation method based on the Transformer model captures global dependencies through the self-attention mechanism, but when processing large-scale renal dynamic imaging images, it often requires a lot of computing resources and has limited ability to capture detailed features.
[0007] Third, although the renal dynamic imaging image segmentation method based on the hybrid architecture combines the advantages of the state-space model and the convolutional neural network and performs well in processing long-distance dependencies, its model structure is complex, the training process is cumbersome, and it requires high computing resources when dealing with high-detail and high-resolution images.
[0008] Fourth, the Segment Anything Model (SAM) performs well in zero-shot and few-shot segmentation tasks in natural images, but there are some challenges in directly applying it to the segmentation of renal dynamic imaging images. Renal dynamic imaging images usually have subtle differences, such as low contrast, fuzzy boundaries, and complex anatomical structures, which are different from the challenges in natural image segmentation. Although SAM has strong adaptability, its cue-driven input mechanism and mask decoder may not work fully when facing renal dynamic imaging images. Summary of the invention
[0009] In view of the above defects or improvement needs of the prior art, the present invention provides a method and system for segmenting kidney areas in renal dynamic imaging images based on hybrid attention branches, the purpose of which is to solve the technical problems that the existing renal dynamic imaging image segmentation method based on classic UNet and its variants enhances feature transfer through jump connections, but its segmentation ability is limited when processing renal dynamic imaging images with rich details; especially when facing complex anatomical structures and low-contrast areas, since this model is difficult to accurately capture important features, it will lead to inaccurate renal dynamic imaging image segmentation results. The existing renal dynamic imaging image segmentation method based on the Transformer model often requires a large amount of computing resources when processing large-scale renal dynamic imaging images, and has limited ability to capture detailed features. The existing renal dynamic imaging image segmentation method based on a hybrid architecture has a complex model structure, a cumbersome training process, and high requirements for computing resources when dealing with high-detail and high-resolution images. The technical problem that the prompt-driven input mechanism and mask decoder of the existing SAM model cannot fully play their role when facing renal dynamic imaging images.
[0010] To achieve the above object, according to one aspect of the present invention, a method for segmenting a kidney region in a renal dynamic imaging image based on a hybrid attention branch is provided, comprising the following steps:
[0011] (1) Obtaining a renal dynamic imaging image acquired by a single photon emission computed tomography (SPECT) scanner, and preprocessing the renal dynamic imaging image to obtain a plurality of preprocessed images.
[0012] (2) Each preprocessed image obtained in step (1) is input into a pre-trained renal dynamic imaging image segmentation model to obtain the corresponding kidney region.
[0013] Preferably, the renal dynamic imaging image obtained in step (1) is in the Digital Imaging and Communications in Medicine (DICOM) format.
[0014] The preprocessing process in step (1) is as follows: first, convert the renal dynamic imaging image in DICOM format into an image in Neural Image Information Technology Initiative (NIfTI) format; then, extract all renal imaging slices from the NIfTI format image; then, delete all renal imaging slices without renal labels from all extracted renal imaging slices, and retain the slices with renal labels; finally, convert each retained renal imaging slice into a PNG format image as a preprocessed image.
[0015] Preferably, the renal dynamic imaging image segmentation model includes a SAM encoder branch, a VMamba encoder branch, and a CBA part for connecting the SAM encoder branch, the cross-branch attention, and the image decoder part.
[0016] The SAM encoder branch consists of an overlapping patch embedding module, a position embedding module, a window Transformer module, and a detailed feature extraction module.
[0017] The input of the overlapping patch embedding module is an image of dimension 1024*1024*3. It divides the input image into multiple overlapping patches of the same size, and converts each patch into a high-dimensional vector through a linear projection. The final output is a set of vectors corresponding to all patches, that is, the first eigenvector of dimension 256*256*768.
[0018] The input of the position embedding module is the first feature vector with a dimension of 256*256*768 output by the overlapping patch embedding module. It performs position embedding processing on the vector corresponding to each patch in the first feature vector, and finally outputs a second feature vector with a dimension of 256*256*768.
[0019] The windowed Transformer module consists of a layer normalization submodule, a multi-head attention submodule, and a feed-forward neural network.
[0020] The input of the layer normalization submodule is the second feature vector with a dimension of 256*256*768 output by the position embedding module. It performs layer normalization on the second feature vector and outputs a normalized feature vector with the same dimension as the input second feature vector.
[0021] The input of the multi-head attention submodule is the normalized feature vector output by the layer normalization submodule, which performs multi-head attention processing on the vector and outputs an attention feature vector with a dimension of 256*256*768.
[0022] The input of the feedforward neural network is the attention feature vector output by the multi-head attention submodule, which uses a nonlinear activation function to process the attention feature vector, performs feature transformation through a fully connected layer, and finally outputs a third feature vector with a dimension of 64*64*128.
[0023] The input of the detailed feature extraction module is the third feature vector with a dimension of 64*64*128 output by the window Transformer module. It uses a convolutional neural network to extract features from the third feature vector, applies a spatial pyramid for pooling, uses residual connections to retain the original information, and applies the attention mechanism. It finally outputs a final feature vector with a dimension of 64*64*128.
[0024] Preferably, the VMamba encoder branch is in parallel with the SAM encoder branch and comprises an input processing module, 4 visual state space VSS blocks, and 3 block merging layers.
[0025] The input processing module input is the same as the SAM encoder branch, which is an image of dimension 1024*1024*3. The module first divides the input image into multiple non-overlapping patches of the same size, then flattens each patch into a one-dimensional vector and concatenates all patch vectors in sequence to form a long sequence. Finally, a linear embedding layer adjusts the feature dimension of this sequence, and the final output is the adjusted feature sequence;
[0026] The input of the first VSS block is the adjusted feature sequence output by the input processing module. This block first applies a selective scanning mechanism to capture local dependencies by performing local scanning in the sequence. Then, a cross-gated multi-layer perceptron (MLP) is used for feature interaction to enhance the expressiveness of the features. Next, residual connections are implemented to preserve the original information, and layer normalization is applied to stabilize the training process. After these processing steps, the first feature vector with a dimension of 512*512*32 is finally output.
[0027] The input of the second VSS block and the first merging layer is the first feature vector of dimension 512*512*32 output by the first VSS block. In the second VSS block, a selective scan with a larger receptive field is first applied to capture a wider range of dependencies. Then a higher-dimensional cross-gated MLP is used for more complex feature interactions. Residual connections and layer normalization are also implemented. After that, the first merging layer reduces the spatial dimension and increases the number of channels through operations such as pooling or strided convolution, and finally outputs a second feature vector of dimension 256*256*64.
[0028] The input of the third VSS block and the second merging layer is the second feature vector of dimension 256*256*64 output by the second VSS block and the first merging layer. The third VSS block further increases the receptive field of the selective scan, uses a higher-dimensional cross-gated MLP, and continues to apply residual connections and layer normalization. The second merging layer again reduces the spatial dimension and increases the number of channels, and outputs the third feature vector of dimension 128*128*128.
[0029] The input of the fourth VSS block and the third merging layer is the third eigenvector of dimension 128*128*128 output by the third VSS block and the second merging layer. The fourth VSS block uses the selective scanning of the maximum receptive field and the highest dimensional cross-gated MLP, while maintaining the application of residual connection and layer normalization. The third merging layer performs the final dimensionality reduction and channel number increase, and finally outputs the fourth eigenvector of dimension 64*64*128.
[0030] Preferably, the cross-branch attention part is used to fuse the final feature vector with an output dimension of 64*64*128 output by the SAM encoder branch and the fourth feature vector with a dimension of 64*64*128 output by the VMamba encoder branch. The cross-branch attention part includes a query mapping module, a key mapping module, a value mapping module, a scaled dot product attention module, a weight calculation module and a feature fusion module;
[0031] The input of the query mapping module is the fourth eigenvector of dimension 64*64*128 output by the VMamba encoder branch. It transforms the eigenvector into a set of high-dimensional vectors through a linear projection and finally outputs a query vector of dimension 64*64*q, where q represents the dimension of the query vector.
[0032] The input of the key mapping module is the feature vector of dimension 64*64*128 output by the SAM encoder branch. It performs a linear transformation on the feature vector, maps the features of each spatial position to the key space, and finally outputs a key vector of dimension 64*64*q.
[0033] The input of the value mapping module is also the feature vector output by the SAM encoder branch with a dimension of 64*64*128. It maps the feature vector to the value space through another linear transformation, and finally outputs a value vector with a dimension of 64*64*r, where r represents the dimension of the value vector;
[0034] The input of the scaled dot product attention module is the query vector output by the query mapping module, the key vector output by the key mapping module, and the value vector output by the value mapping module. It calculates the dot product between the query vector and the key vector, and scales the result by dividing it by √q. The scaled result is then processed using the softmax function and output as an attention weight matrix of dimension 64*64, where √q represents the square root of q.
[0035] The input of the weight calculation module is the attention weight matrix output by the scaled dot product attention module. It normalizes the attention weight matrix to ensure that the sum of the weights is 1, and outputs the normalized attention weight matrix with a dimension of 64*64.
[0036] The input of the feature fusion module is the value vector output by the value mapping module and the normalized attention weight matrix output by the weight calculation module. It applies the attention weight to the value vector, generates fused features through weighted summation, and finally outputs a fused feature map with a dimension of 64*r.
[0037] Preferably, the image decoder part includes a continuous feature decoder and an implicit feature alignment IFA module;
[0038] The continuous feature decoder includes an input block, a feature expansion block, and a final convolutional layer, and the implicit feature alignment module includes a feature collection submodule, a feature mapping submodule, a feature alignment submodule, and a feature fusion submodule;
[0039] The input block takes a feature map of dimension 64*64*256 (i.e., the sum of the outputs of the SAM encoder branch and the VMamba encoder branch), which is processed by convolutional layers, batch normalization, and ReLU activation functions, and finally outputs a feature map of dimension 128*128*256.
[0040] The input of the feature expansion block is the feature map of the dimension 128*128*256 output by the input block. It uses a combination of convolutional layer and transposed convolutional layer to expand and refine the feature map, and finally outputs a feature map of dimension 512*512*256.
[0041] The input of the final convolutional layer is the feature map of dimension 512*512*256 output by the feature expansion block. It performs a convolution operation on the feature map and outputs a feature map of dimension 512*512*128 that can be aligned.
[0042] The input of the feature collection submodule is a feature map with a dimension of 64*64*256, a feature map with a dimension of 128*128*256 output by the input block, a feature map with a dimension of 512*512*256 output by the feature expansion block, and a feature map with a dimension of 512*512*128 output by the final convolutional layer. All these feature maps are merged according to their respective channel dimensions and output as a multi-scale feature map set.
[0043] The input of the feature mapping submodule is the multi-scale feature map set output by the feature collection submodule, which uses a multi-layer perceptron MLP to map the multi-scale feature map set to an implicit feature space, and the output is an implicit feature representation of dimension 512*512*D, where D is the dimension of the implicit feature, which is usually smaller than the number of channels of each multi-scale feature in the multi-scale feature map set.
[0044] The input of the feature alignment submodule is the implicit feature representation with a dimension of 512*512*D output by the feature mapping submodule. It realizes feature alignment through an iterative optimization process, and the output dimension is still the aligned implicit feature of 512*512*D.
[0045] The input of the feature fusion submodule is the aligned implicit features output by the feature alignment submodule. It first projects the implicit features back to the original feature space through a learnable mapping to obtain features of different scales, and then uses weighted summation or adaptive fusion strategy to fuse all features of different scales together to obtain a feature map. Finally, through upsampling or transposed convolution operations, the feature map is gradually enlarged to the target resolution, and finally the output dimension is 1024*1024*1 segmentation mask.
[0046] Preferably, the renal dynamic imaging image segmentation model is trained by the following steps:
[0047] (2-1) Obtain a pediatric renal dynamic imaging dataset consisting of pediatric renal dynamic imaging images, preprocess the pediatric renal dynamic imaging dataset to obtain a preprocessed pediatric renal dynamic imaging dataset, and divide the preprocessed pediatric renal dynamic imaging dataset into a training set and a test set in a ratio of 4:1.
[0048] (2-2) initializing the renal dynamic imaging image segmentation model to obtain an initialized renal dynamic imaging image segmentation model;
[0049] (2-3) For each pediatric renal dynamic imaging image in the training set obtained in step (2-1), the pediatric renal dynamic imaging image is input into the SAM encoder branch and the VMamba encoder branch respectively to obtain two feature vectors corresponding to the pediatric renal dynamic imaging image respectively.
[0050] (2-4) For each pediatric renal dynamic imaging image in the training set obtained in step (2-1), the two feature vectors corresponding to the pediatric renal dynamic imaging image obtained in step (2-3) are input into the cross-branch attention part for fusion to obtain a fused feature map corresponding to the pediatric renal dynamic imaging image.
[0051] (2-5) For each pediatric renal dynamic imaging image in the training set obtained in step (2-1), the fusion feature map corresponding to the pediatric renal dynamic imaging image obtained in step (2-4) is input into the image decoder part to obtain the segmentation result corresponding to the pediatric renal dynamic imaging image.
[0052] (2-6) according to the segmentation result corresponding to each pediatric renal dynamic imaging image obtained in step (2-5), and using the loss function to iteratively train the renal dynamic imaging image segmentation model until the renal dynamic imaging image segmentation model converges, thereby obtaining a preliminarily trained renal dynamic imaging image segmentation model;
[0053] (2-7) Use the test set obtained in step (2-1) to verify the renal dynamic imaging image segmentation model preliminarily trained in step (2-6) until the classification accuracy reaches the optimal level or the set total number of iterations (epochs) is reached, thereby obtaining a trained renal dynamic imaging image segmentation model.
[0054] Preferably, step (2-2) is to initialize the total number of iterations and the learning rate of the renal dynamic imaging image segmentation model, wherein the total number of iterations ranges from 100 to 300, preferably 200, and the learning rate ranges from 0 to 1, preferably 0.001;
[0055] Step (2-6) uses the Dice Loss function as the loss function, and its calculation formula is:
[0056]
[0057] Among them, is a constant, which is equal to 1*10 -8 , which is used to prevent the denominator from being 0. X represents the true segmentation result of the pediatric renal dynamic imaging image obtained in step (2-5), Y represents the predicted segmentation result of the pediatric renal dynamic imaging image, and |X| and |Y| represent the sum of voxels of the true segmentation result and the sum of voxels of the predicted segmentation result, respectively.
[0058] According to another aspect of the present invention, a system for segmenting a kidney region in a renal dynamic imaging image based on a hybrid attention branch is provided, comprising:
[0059] The first module is used to obtain a renal dynamic imaging image acquired by a single photon emission computed tomography (SPECT) scanner, and preprocess the renal dynamic imaging image to obtain a plurality of preprocessed images.
[0060] The second module is used to input each preprocessed image obtained by the first module into a pre-trained renal dynamic imaging image segmentation model to obtain the corresponding kidney area.
[0061] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:
[0062] 1. Improve segmentation accuracy and generalization ability: Due to the use of steps (2-3) and (2-4), the present invention effectively integrates global semantic features and fine-grained local details through a cross-branch attention mechanism, so that the model maintains high accuracy and high reliability in complex medical images. Therefore, it can solve the problem that the existing renal dynamic imaging image segmentation method based on classic UNet and its variants is difficult to accurately capture important features when facing complex anatomical structures and low-contrast areas, thereby avoiding the technical problem of inaccurate renal dynamic imaging image segmentation results;
[0063] 2. Simplify the workflow and reduce manual input: The present invention adopts steps (2-2) and (2-5), which eliminate the prompt-driven input mechanism through the autonomous prediction model and the image decoder part, improve the operation efficiency, and reduce the reliance on manual input. Therefore, it can solve the problem that the existing renal dynamic imaging image segmentation method based on the Transformer model requires a lot of manual adjustment and computing resources when processing large-scale renal dynamic imaging images, thereby improving the simplicity and efficiency of the workflow;
[0064] 3. Enhance the robustness and stability of the model: The present invention adopts steps (2-5) and (2-6), which ensure the multi-level integration and high-quality segmentation output of feature maps through the collaborative work of the continuous feature decoder and the implicit feature alignment module, thereby enhancing the robustness of the model when processing complex medical images. Therefore, it can solve the problems of the existing renal dynamic imaging image segmentation method based on hybrid architecture, such as complex model structure, cumbersome training process, and high computing resource requirements when facing high-detail and high-resolution images, thereby improving the robustness and stability of the model;
[0065] 4. Improve the efficiency and effect of model training: The present invention adopts steps (2-6) and (2-7), which ensures excellent segmentation performance in a variable medical image environment by optimizing training strategies and model parameters, accelerates the convergence speed of the model, and improves the training efficiency and effect. Therefore, it can solve the problem that the prompt-driven input mechanism and mask decoder cannot fully play their role and the problem of insufficient computing resources when the SAM model is directly applied to the segmentation of renal dynamic imaging images, thereby improving the efficiency and effect of model training. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] Figure 1 is an overall flow chart of a method for segmenting a kidney region in a renal dynamic imaging image based on a hybrid attention branch of the present invention;
[0067] Figure 2 It is a structural schematic diagram of the renal dynamic imaging image segmentation model used in the present invention. DETAILED DESCRIPTION
[0068] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0069] The basic idea of the present invention is to design a segmentation method based on hybrid attention branches, by integrating the SegmentAnything model (SAM) with the Visual Mamba (VMamba) model, and combining the cross-branch attention (CBA) mechanism to segment the kidney area in the renal dynamic imaging image. By eliminating the prompt-driven input mechanism, an autonomous prediction model is used to simplify the segmentation workflow, reduce the reliance on manual input, and improve operational efficiency. Compared with traditional segmentation methods, the present invention significantly improves the segmentation accuracy and the generalization ability of the model when processing complex details and low-contrast medical images.
[0070] like Figure 1 As shown, the present invention provides a method for segmenting a kidney region in a renal dynamic imaging image based on a hybrid attention branch, comprising the following steps:
[0071] (1) Obtaining a renal dynamic imaging image acquired by a single photon emission computed tomography (SPECT) scanner, and preprocessing the renal dynamic imaging image to obtain a plurality of preprocessed images.
[0072] Specifically, the renal dynamic imaging image format obtained in this step is the Digital Imaging and Communications in Medicine (DICOM) format.
[0073] The preprocessing process in this step is as follows: first, convert the renal dynamic imaging images in DICOM format into images in Neuroimaging Informatics Technology Initiative (NIfTI) format; then, extract all renal imaging slices from the NIfTI format images; then, delete all renal imaging slices without renal labels from all extracted renal imaging slices, and retain the slices with renal labels; finally, convert each retained renal imaging slice into a PNG format image as a preprocessed image.
[0074] (2) inputting each preprocessed image obtained in step (1) into a pre-trained renal dynamic imaging image segmentation model to obtain the corresponding kidney region;
[0075] Specifically, the renal dynamic imaging image segmentation model of the present invention includes a SAM encoder branch, a VMamba encoder branch, and a part for connecting the SAM encoder branch, a cross-branch attention (CBA) part, and an image decoder part.
[0076] The SAM encoder branch consists of an overlapping patch embedding module, a position embedding module, a window Transformer module, and a detailed feature extraction module.
[0077] The input of the overlapping patch embedding module is an image of dimension 1024*1024*3. It divides the input image into multiple overlapping patches of the same size, and converts each patch into a high-dimensional vector through a linear projection (i.e., embedding layer). The final output is a set of vectors corresponding to all patches, i.e., the first eigenvector of dimension 256*256*768.
[0078] The input of the position embedding module is the first feature vector with a dimension of 256*256*768 output by the overlapping patch embedding module. It performs position embedding processing on the vector corresponding to each patch in the first feature vector (specifically, embedding information representing its position in the image into the vector corresponding to the first patch of the first feature vector, embedding information representing its position into the vector corresponding to the second patch, and so on), and finally outputs a second feature vector with a dimension of 256*256*768.
[0079] The windowed Transformer module (which is the core component of the ViT branch) includes a layer normalization submodule, a multi-head attention submodule, and a feed-forward neural network.
[0080] The input of the layer normalization submodule is the second feature vector with a dimension of 256*256*768 output by the position embedding module. It performs layer normalization on the second feature vector (to speed up training and improve model stability) and outputs a normalized feature vector with the same dimension as the input second feature vector.
[0081] The input of the multi-head attention submodule is the normalized feature vector output by the layer normalization submodule, which performs multi-head attention processing on the vector (the purpose is to enable the model to capture different contextual information in the input, each attention head independently calculates the attention score, and concatenates and linearly transforms their outputs), and outputs an attention feature vector with a dimension of 256*256*768.
[0082] The input of the feedforward neural network is the attention feature vector output by the multi-head attention submodule, which uses a nonlinear activation function (such as ReLU) to process the attention feature vector and performs feature transformation through a fully connected layer, and finally outputs a third feature vector with a dimension of 64*64*128.
[0083] The input of the detailed feature extraction module is the third eigenvector with a dimension of 64*64*128 output by the window Transformer module. It uses a convolutional neural network to extract features from the third eigenvector, applies a spatial pyramid for pooling (for capturing multi-scale information), uses residual connections to retain the original information, and applies an attention mechanism (for highlighting important features). The final output is a final feature vector with a dimension of 64*64*128.
[0084] The SAM encoder branch of the present invention can effectively extract multi-scale features in the image and provide rich feature expressions for subsequent segmentation tasks. By combining overlapping patch embedding, position encoding, Transformer structure and detailed feature extraction, the encoder can capture local details and global semantic information of the image, thereby improving the accuracy and robustness of the segmentation task.
[0085] In parallel with the SAM encoder branch, the VMamba encoder branch focuses on extracting fine-grained features of the image and includes an input processing module, 4 Visual State Space (VSS) blocks, and 3 block merging layers.
[0086] The input processing module input is the same as the SAM encoder branch, which is an image of dimension 1024*1024*3. The module first splits the input image into multiple non-overlapping patches of the same size (e.g. 16x16 pixels). Then, each patch is flattened into a one-dimensional vector and all patch vectors are concatenated in sequence to form a long sequence. Finally, a linear embedding layer is used to adjust the feature dimension of this sequence, and the final output is the adjusted feature sequence.
[0087] The input of the first VSS block is the adjusted feature sequence output by the input processing module. This block first applies a selective scanning mechanism to capture local dependencies by performing local scanning in the sequence. Then, a cross-gated multi-layer perceptron (MLP) is used for feature interaction to enhance the expressiveness of the features. Next, residual connections are implemented to preserve the original information, and layer normalization is applied to stabilize the training process. After these processing steps, the first feature vector with a dimension of 512*512*32 is finally output.
[0088] The input of the second VSS block and the first merging layer is the first feature vector of dimension 512*512*32 output by the first VSS block. In the second VSS block, a selective scan with a larger receptive field is first applied to capture a wider range of dependencies. Then a higher-dimensional cross-gated MLP is used for more complex feature interactions. Residual connections and layer normalization are also implemented. After that, the first merging layer reduces the spatial dimension and increases the number of channels through operations such as pooling or strided convolution, and finally outputs a second feature vector of dimension 256*256*64.
[0089] The input of the third VSS block and the second merging layer is the second feature vector of dimension 256*256*64 output by the second VSS block and the first merging layer. The third VSS block further increases the receptive field of the selective scan, uses a higher-dimensional cross-gated MLP, and continues to apply residual connections and layer normalization. The second merging layer again reduces the spatial dimension and increases the number of channels, and outputs the third feature vector of dimension 128*128*128.
[0090] The input of the fourth VSS block and the third merging layer is the third eigenvector of dimension 128*128*128 output by the third VSS block and the second merging layer. The fourth VSS block uses the selective scanning of the maximum receptive field and the highest dimensional cross-gated MLP, while maintaining the application of residual connection and layer normalization. The third merging layer performs the final dimensionality reduction and channel number increase, and finally outputs the fourth eigenvector of dimension 64*64*128.
[0091] Through this hierarchical process, the VMamba encoder branch can effectively extract fine-grained features in the image, gradually increasing the complexity and abstraction level of the features while maintaining sensitivity to local details. This design enables the model to capture image features at different scales and provide rich local detail information for subsequent segmentation tasks.
[0092] The cross-branch attention part of the present invention is used to fuse the final feature vector with an output dimension of 64*64*128 output by the SAM encoder branch and the fourth feature vector with a dimension of 64*64*128 output by the VMamba encoder branch. The cross-branch attention part includes a query mapping module, a key mapping module, a value mapping module, a scaled dot product attention module, a weight calculation module, and a feature fusion module.
[0093] The input of the query mapping module is the fourth eigenvector of dimension 64*64*128 output by the VMamba encoder branch. It converts the eigenvector into a set of high-dimensional vectors through a linear projection, and finally outputs a query vector of dimension 64*64*q, where q represents the dimension of the query vector, which usually ranges from 32 to 256. The specific value is selected according to the model complexity and computing resources.
[0094] The input of the key mapping module is the feature vector of dimension 64*64*128 output by the SAM encoder branch. It performs a linear transformation on the feature vector, maps the features of each spatial position to the key space, and finally outputs a key vector of dimension 64*64*q.
[0095] The input of the value mapping module is also the feature vector output by the SAM encoder branch with a dimension of 64*64*128. It maps the feature vector to the value space through another linear transformation, and finally outputs a value vector with a dimension of 64*64*r, where r represents the dimension of the value vector, which is usually equal to or slightly different from q, and the value range is also between 32 and 256.
[0096] The input of the scaled dot product attention module is the query vector output by the query mapping module, the key vector output by the key mapping module, and the value vector output by the value mapping module. It calculates the dot product between the query vector and the key vector, and scales the result by dividing it by √q (where √q represents the square root of q). The scaled result is then processed using the softmax function and output as an attention weight matrix with a dimension of 64*64.
[0097] The input of the weight calculation module is the attention weight matrix output by the scaled dot product attention module. It normalizes the attention weight matrix to ensure that the sum of the weights is 1, and outputs the normalized attention weight matrix with a dimension of 64*64.
[0098] The input of the feature fusion module is the value vector output by the value mapping module and the normalized attention weight matrix output by the weight calculation module. It applies the attention weight to the value vector, generates fused features through weighted summation, and finally outputs a fused feature map with a dimension of 64*r.
[0099] The cross-branch attention part of the present invention can effectively fuse features from the SAM encoder and the VMamba encoder. By dynamically calculating attention weights and feature fusion, the mechanism can capture global semantic information and local details, thereby improving the accuracy and robustness of the segmentation task.
[0100] The image decoder part includes a continuous feature decoder and an implicit feature alignment (IFA) module, which is used to convert the fused feature map with a dimension of 64*r output by the cross-branch attention part into a high-resolution segmentation result.
[0101] The continuous feature decoder includes an input block, a feature expansion block and a final convolutional layer, and the implicit feature alignment module includes a feature collection submodule, a feature mapping submodule, a feature alignment submodule, and a feature fusion submodule.
[0102] The input of the input block is a feature map of dimension 64*64*256 (which is the sum of the outputs of the SAM encoder branch and the VMamba encoder branch). The feature map is processed by convolutional layers, batch normalization, and ReLU activation functions, and the final output is a feature map of dimension 128*128*256.
[0103] The input of the feature expansion block is the feature map of the dimension 128*128*256 output by the input block. It uses a combination of convolutional layer and transposed convolutional layer to expand and refine the feature map, and finally outputs a feature map of dimension 512*512*256.
[0104] The input of the final convolutional layer is the feature map of dimension 512*512*256 output by the feature expansion block. It performs a convolution operation on the feature map and outputs a feature map of dimension 512*512*128 that can be aligned.
[0105] The input of the feature collection submodule is a feature map with a dimension of 64*64*256, a feature map with a dimension of 128*128*256 output by the input block, a feature map with a dimension of 512*512*256 output by the feature expansion block, and a feature map with a dimension of 512*512*128 output by the final convolutional layer. All these feature maps are merged according to their respective channel dimensions and output as a multi-scale feature map set.
[0106] The input of the feature mapping submodule is the multi-scale feature map set output by the feature collection submodule, which uses a multi-layer perceptron (MLP) to map the multi-scale feature map set to an implicit feature space, and the output is an implicit feature representation of dimension 512*512*D, where D is the dimension of the implicit feature, which is usually smaller than the number of channels of each multi-scale feature in the multi-scale feature map set (for example, D=64).
[0107] The input of the feature alignment submodule is the implicit feature representation with a dimension of 512*512*D output by the feature mapping submodule. It realizes feature alignment through an iterative optimization process, and the output dimension is still the aligned implicit feature of 512*512*D.
[0108] Specifically, the feature alignment submodule uses learnable positional encoding and attention mechanisms to adaptively align features of different scales in the implicit feature space. This process allows the model to capture long-range dependencies between features of different scales.
[0109] The input of the feature fusion submodule is the aligned implicit features output by the feature alignment submodule. It first projects the implicit features back to the original feature space through a learnable mapping to obtain features of different scales, and then uses weighted summation or adaptive fusion strategy to fuse all features of different scales together to obtain a feature map. Finally, through upsampling or transposed convolution operations, the feature map is gradually enlarged to the target resolution (1024*1024), and the final output dimension is a segmentation mask of 1024*1024*1.
[0110] The IFA module of the present invention overcomes the scale inconsistency problem in traditional feature fusion methods by performing feature alignment and fusion in implicit feature space. It can adaptively learn the relationship between features of different scales and achieve more accurate feature alignment and fusion. This method not only maintains the continuity and consistency of features, but also effectively integrates multi-scale information, thereby capturing global semantic information while maintaining high-resolution details.
[0111] By combining a continuous feature decoder and an innovative IFA module, the present invention can effectively convert the multi-level features extracted by the encoder into high-resolution segmentation results. This design significantly improves the accuracy and detail retention of segmentation tasks, especially when dealing with complex scenes and fine structures.
[0112] Specifically, the renal dynamic imaging image segmentation model of the present invention is trained by the following steps:
[0113] (2-1) Obtain a pediatric renal dynamic imaging dataset consisting of pediatric renal dynamic imaging images, preprocess the pediatric renal dynamic imaging dataset to obtain a preprocessed pediatric renal dynamic imaging dataset, and divide the preprocessed pediatric renal dynamic imaging dataset into a training set and a test set in a ratio of 4:1.
[0114] Specifically, this step first converts the pediatric renal dynamic imaging dataset into a dataset in NIfTI format (the purpose is to facilitate storage and transmission. The NIfTI format contains head information and image data, such as voxel size and orientation, which makes data access and analysis more convenient). Then, the dataset in NIfTI format is divided into a training set and a test set in a ratio of 8:2 (the specific method is to randomly shuffle all preprocessed images and evenly divide them into 5 parts, take any 1 part as the test set, and the remaining 4 parts as the training set. This data division method ensures the representativeness of the training set and the test set, thereby improving the generalization ability and robustness of the model).
[0115] (2-2) initializing the renal dynamic imaging image segmentation model to obtain an initialized renal dynamic imaging image segmentation model;
[0116] The purpose of this step is to ensure effective training of the renal dynamic imaging image segmentation model.
[0117] Specifically, this step is to initialize the total number of iterations (epochs) and the learning rate of the renal dynamic imaging image segmentation model, wherein the total number of iterations ranges from 50 to 100, preferably 60, and the learning rate ranges from 0 to 1, preferably 0.001.
[0118] (2-3) For each pediatric renal dynamic imaging image in the training set obtained in step (2-1), the pediatric renal dynamic imaging image is input into the SAM encoder branch and the VMamba encoder branch respectively to obtain two feature vectors corresponding to the pediatric renal dynamic imaging image respectively.
[0119] The purpose of this step is to extract the global semantic information and local detail features of the image through the encoder structure.
[0120] Specifically, this step adopts an encoder architecture that combines the sam encoder and the VMamba encoder.
[0121] SAM encoder branch: The visual transformer (ViT) structure is used to process pediatric renal dynamic imaging images, and a multi-head self-attention mechanism and a feedforward neural network are used to encode pediatric renal dynamic imaging images to generate a high-dimensional feature vector containing global semantic information.
[0122] VMamba encoder branch: A 2D state space model (SS2D) is used to process pediatric nephrography SPECT images, and a selective scanning mechanism (S6) and a cross scanning module (CSM) are used to capture fine-grained local features and generate multi-scale high-dimensional feature vectors.
[0123] (2-4) For each pediatric renal dynamic imaging image in the training set obtained in step (2-1), the two feature vectors corresponding to the pediatric renal dynamic imaging image obtained in step (2-3) are input into the cross-branch attention part for fusion to obtain a fused feature map corresponding to the pediatric renal dynamic imaging image.
[0124] The purpose of this step is to effectively combine global semantic features and local detail features to improve the accuracy of subsequent segmentation. Specifically, this step uses the cross-branch attention mechanism (CBA) for feature fusion:
[0125] The feature map of the VMamba encoder branch is mapped to the query space, and the feature map of the SAM encoder branch is mapped to the key and value space. The similarity between the query and the key is calculated, and the stable gradient flow is promoted by scaling the dot product. The values are weighted summed using the calculated similarity weights to generate a fused feature map. The final fused feature map combines global semantic information and fine-grained details, laying the foundation for subsequent accurate segmentation.
[0126] (2-5) For each pediatric renal dynamic imaging image in the training set obtained in step (2-1), the fusion feature map corresponding to the pediatric renal dynamic imaging image obtained in step (2-4) is input into the image decoder part to obtain the segmentation result corresponding to the pediatric renal dynamic imaging image.
[0127] The purpose of this step is to restore the spatial resolution of the feature map and generate the final segmentation result. Specifically, this step uses a continuous feature decoder and an implicit feature alignment (IFA) module:
[0128] Continuous feature decoder: Based on the CNN architecture, the feature map is expanded and refined through multiple stages of convolution and transposed convolution operations. It includes input blocks (adjusting feature depth), feature expansion blocks (further refining feature maps), and final convolution layers (generating feature maps for alignment).
[0129] IFA module: Feature alignment is achieved through the learned decoding function, ensuring that feature maps of different scales are seamlessly combined in the same space, retaining the details and spatial accuracy required for high-quality semantic segmentation.
[0130] (2-6) according to the segmentation result corresponding to each pediatric renal dynamic imaging image obtained in step (2-5), and using the loss function to iteratively train the renal dynamic imaging image segmentation model until the renal dynamic imaging image segmentation model converges, thereby obtaining a preliminarily trained renal dynamic imaging image segmentation model;
[0131] The purpose of this step is to optimize the model parameters through iterative training and improve the segmentation accuracy.
[0132] Specifically, this step uses the Dice Loss function as the loss function, and its calculation formula is:
[0133]
[0134] Among them, is a constant, which is equal to 1*10 -8 , which is used to prevent the denominator from being 0. X represents the true segmentation result of the pediatric renal dynamic imaging image obtained in step (2-5), Y represents the predicted segmentation result of the pediatric renal dynamic imaging image, and |X| and |Y| represent the sum of voxels of the true segmentation result and the sum of voxels of the predicted segmentation result, respectively.
[0135] (2-7) Use the test set obtained in step (2-1) to verify the renal dynamic imaging image segmentation model preliminarily trained in step (2-6) until the classification accuracy reaches the optimal level or the set total number of iterations is reached, thereby obtaining a trained renal dynamic imaging image segmentation model.
[0136] The purpose of this step is to evaluate the segmentation accuracy of the model and optimize it.
[0137] Specifically, this step is to verify the preliminarily trained model using the test set divided in step (2-1). The performance of the model on the test set is evaluated by calculating indicators such as the Dice coefficient, Hausdorff distance, and mean intersection over union (MIoU). According to the test results, the model parameters and training strategies are adjusted, and the above steps (2-3) to (2-6) are repeated until the set total number of iterations is reached or the segmentation accuracy on the test set reaches the optimal value, thereby obtaining the final trained medical image segmentation model.
[0138] In summary, the present invention provides a method and system for segmenting the kidney region in a renal dynamic imaging image based on a hybrid attention branch. The purpose is to solve the problems of insufficient accuracy and poor generalization ability of existing segmentation methods when processing complex details and low-contrast medical images. Traditional segmentation methods rely on a prompt-driven input mechanism, which increases the complexity of manual input and performs poorly when processing complex details. The present invention integrates the Segment Anything model (SAM) with the Visual Mamba (VMamba) model by designing a segmentation method based on a hybrid attention branch, and combines the cross-branch attention (CBA) mechanism to segment the kidney region in the renal dynamic imaging image. By eliminating the prompt-driven input mechanism and adopting an autonomous prediction model to simplify the segmentation workflow, reduce the reliance on manual input, and improve operational efficiency. Specific implementation methods include data preprocessing, a dual-branch encoder architecture, a cross-branch attention mechanism, a continuous feature decoder and an implicit feature alignment (IFA) module, as well as model training and evaluation. The data preprocessing step converts renal dynamic imaging images into standardized PNG format images; the dual-branch encoder architecture uses SAM and VMamba encoders to process images respectively; the cross-branch attention mechanism fuses global semantics and fine-grained features by calculating attention weights; the continuous feature decoder and IFA module ensure multi-level integration of feature maps and high-quality segmentation output; finally, the Dice Loss function is used to optimize the model parameters and evaluate the model performance.
[0139] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for segmenting kidney regions in renal dynamic imaging images based on hybrid attention branches, characterized in that: The following steps are involved: (1) Obtaining a renal dynamic imaging image acquired by a single photon emission computed tomography scanner, and preprocessing the renal dynamic imaging image to obtain a plurality of preprocessed images. (2) Each preprocessed image obtained in step (1) is input into a pre-trained renal dynamic imaging image segmentation model to obtain the corresponding kidney region.
2. The method for segmenting the kidney region in a renal dynamic imaging image based on hybrid attention branch according to claim 1, characterized in that: The renal dynamic imaging image obtained in step (1) is in the Digital Imaging and Communications in Medicine DICOM format. The preprocessing process in step (1) is as follows: first, convert the renal dynamic imaging image in DICOM format into an image in Neural Image Information Technology Initiative (NIfTI) format; then, extract all renal imaging slices from the NIfTI format image; then, delete all renal imaging slices without renal labels from all extracted renal imaging slices, and retain the slices with renal labels; finally, convert each retained renal imaging slice into a PNG format image as a preprocessed image.
3. The method for segmenting the kidney region in a renal dynamic imaging image based on hybrid attention branch according to claim 1 or 2, characterized in that: The renal dynamic imaging image segmentation model includes a SAM encoder branch, a VMamba encoder branch, a CBA part for connecting the SAM encoder branch, a cross-branch attention, and an image decoder part. The SAM encoder branch consists of an overlapping patch embedding module, a position embedding module, a window Transformer module, and a detailed feature extraction module. The input of the overlapping patch embedding module is an image of dimension 1024*1024*3. It divides the input image into multiple overlapping patches of the same size, and converts each patch into a high-dimensional vector through a linear projection. The final output is a set of vectors corresponding to all patches, that is, the first eigenvector of dimension 256*256*768. The input of the position embedding module is the first feature vector with a dimension of 256*256*768 output by the overlapping patch embedding module. It performs position embedding processing on the vector corresponding to each patch in the first feature vector, and finally outputs a second feature vector with a dimension of 256*256*768. The windowed Transformer module consists of a layer normalization submodule, a multi-head attention submodule, and a feed-forward neural network. The input of the layer normalization submodule is the second feature vector with a dimension of 256*256*768 output by the position embedding module. It performs layer normalization on the second feature vector and outputs a normalized feature vector with the same dimension as the input second feature vector. The input of the multi-head attention submodule is the normalized feature vector output by the layer normalization submodule, which performs multi-head attention processing on the vector and outputs an attention feature vector with a dimension of 256*256*768. The input of the feedforward neural network is the attention feature vector output by the multi-head attention submodule, which uses a nonlinear activation function to process the attention feature vector, performs feature transformation through a fully connected layer, and finally outputs a third feature vector with a dimension of 64*64*128. The input of the detailed feature extraction module is the third feature vector with a dimension of 64*64*128 output by the window Transformer module. It uses a convolutional neural network to extract features from the third feature vector, applies a spatial pyramid for pooling, uses residual connections to retain the original information, and applies the attention mechanism. It finally outputs a final feature vector with a dimension of 64*64*128.
4. The method for segmenting the kidney region in a renal dynamic imaging image based on a hybrid attention branch according to any one of claims 1 to 3, characterized in that: The VMamba encoder branch is parallel to the SAM encoder branch and includes an input processing module, 4 visual state space VSS blocks, and 3 block merging layers. The input processing module input is the same as the SAM encoder branch, which is an image of dimension 1024*1024*3. The module first divides the input image into multiple non-overlapping patches of the same size, then flattens each patch into a one-dimensional vector and concatenates all patch vectors in sequence to form a long sequence. Finally, a linear embedding layer adjusts the feature dimension of this sequence, and the final output is the adjusted feature sequence; The input of the first VSS block is the adjusted feature sequence output by the input processing module. This block first applies a selective scanning mechanism to capture local dependencies by performing local scanning in the sequence. Then, a cross-gated multi-layer perceptron (MLP) is used for feature interaction to enhance the expressiveness of the features. Next, residual connections are implemented to preserve the original information, and layer normalization is applied to stabilize the training process. After these processing steps, the first feature vector with a dimension of 512*512*32 is finally output. The input of the second VSS block and the first merging layer is the first feature vector of dimension 512*512*32 output by the first VSS block. In the second VSS block, a selective scan with a larger receptive field is first applied to capture a wider range of dependencies. Then a higher-dimensional cross-gated MLP is used for more complex feature interactions. Residual connections and layer normalization are also implemented. After that, the first merging layer reduces the spatial dimension and increases the number of channels through operations such as pooling or strided convolution, and finally outputs a second feature vector of dimension 256*256*64. The input of the third VSS block and the second merging layer is the second feature vector of dimension 256*256*64 output by the second VSS block and the first merging layer. The third VSS block further increases the receptive field of the selective scan, uses a higher-dimensional cross-gated MLP, and continues to apply residual connections and layer normalization. The second merging layer again reduces the spatial dimension and increases the number of channels, and outputs the third feature vector of dimension 128*128*128. The input of the fourth VSS block and the third merging layer is the third eigenvector of dimension 128*128*128 output by the third VSS block and the second merging layer. The fourth VSS block uses the selective scanning of the maximum receptive field and the highest dimensional cross-gated MLP, while maintaining the application of residual connection and layer normalization. The third merging layer performs the final dimensionality reduction and channel number increase, and finally outputs the fourth eigenvector of dimension 64*64*128.
5. The method for segmenting the kidney region in a renal dynamic imaging image based on hybrid attention branch according to claim 4, characterized in that: The cross-branch attention part is used to fuse the final feature vector with an output dimension of 64*64*128 output by the SAM encoder branch and the fourth feature vector with a dimension of 64*64*128 output by the VMamba encoder branch. The cross-branch attention part includes a query mapping module, a key mapping module, a value mapping module, a scaled dot product attention module, a weight calculation module, and a feature fusion module; The input of the query mapping module is the fourth eigenvector of dimension 64*64*128 output by the VMamba encoder branch. It transforms the eigenvector into a set of high-dimensional vectors through a linear projection and finally outputs a query vector of dimension 64*64*q, where q represents the dimension of the query vector. The input of the key mapping module is the feature vector of dimension 64*64*128 output by the SAM encoder branch. It performs a linear transformation on the feature vector, maps the features of each spatial position to the key space, and finally outputs a key vector of dimension 64*64*q. The input of the value mapping module is also the feature vector output by the SAM encoder branch with a dimension of 64*64*128. It maps the feature vector to the value space through another linear transformation, and finally outputs a value vector with a dimension of 64*64*r, where r represents the dimension of the value vector; The input of the scaled dot product attention module is the query vector output by the query mapping module, the key vector output by the key mapping module, and the value vector output by the value mapping module. It calculates the dot product between the query vector and the key vector, and scales the result by dividing it by √q. The scaled result is then processed using the softmax function and output as an attention weight matrix of dimension 64*64, where √q represents the square root of q. The input of the weight calculation module is the attention weight matrix output by the scaled dot product attention module. It normalizes the attention weight matrix to ensure that the sum of the weights is 1, and outputs the normalized attention weight matrix with a dimension of 64*64. The input of the feature fusion module is the value vector output by the value mapping module and the normalized attention weight matrix output by the weight calculation module. It applies the attention weight to the value vector, generates fused features through weighted summation, and finally outputs a fused feature map with a dimension of 64*r.
6. The method for segmenting the kidney region in a renal dynamic imaging image based on hybrid attention branch according to claim 5, characterized in that: The image decoder part includes a continuous feature decoder and an implicit feature alignment IFA module; The continuous feature decoder includes an input block, a feature expansion block, and a final convolutional layer, and the implicit feature alignment module includes a feature collection submodule, a feature mapping submodule, a feature alignment submodule, and a feature fusion submodule; The input block takes a feature map of dimension 64*64*256 (i.e., the sum of the outputs of the SAM encoder branch and the VMamba encoder branch), which is processed by convolutional layers, batch normalization, and ReLU activation functions, and finally outputs a feature map of dimension 128*128*256. The input of the feature expansion block is the feature map of the dimension 128*128*256 output by the input block. It uses a combination of convolutional layer and transposed convolutional layer to expand and refine the feature map, and finally outputs a feature map of dimension 512*512*256. The input of the final convolutional layer is the feature map of dimension 512*512*256 output by the feature expansion block. It performs a convolution operation on the feature map and outputs a feature map of dimension 512*512*128 that can be aligned. The input of the feature collection submodule is a feature map with a dimension of 64*64*256, a feature map with a dimension of 128*128*256 output by the input block, a feature map with a dimension of 512*512*256 output by the feature expansion block, and a feature map with a dimension of 512*512*128 output by the final convolutional layer. All these feature maps are merged according to their respective channel dimensions and output as a multi-scale feature map set. The input of the feature mapping submodule is the multi-scale feature map set output by the feature collection submodule, which uses a multi-layer perceptron MLP to map the multi-scale feature map set to an implicit feature space, and the output is an implicit feature representation of dimension 512*512*D, where D is the dimension of the implicit feature, which is usually smaller than the number of channels of each multi-scale feature in the multi-scale feature map set. The input of the feature alignment submodule is the implicit feature representation with a dimension of 512*512*D output by the feature mapping submodule. It realizes feature alignment through an iterative optimization process, and the output dimension is still the aligned implicit feature of 512*512*D. The input of the feature fusion submodule is the aligned implicit features output by the feature alignment submodule. It first projects the implicit features back to the original feature space through a learnable mapping to obtain features of different scales, and then uses weighted summation or adaptive fusion strategy to fuse all features of different scales together to obtain a feature map. Finally, through upsampling or transposed convolution operations, the feature map is gradually enlarged to the target resolution, and finally the output dimension is 1024*1024*1 segmentation mask.
7. The method for segmenting the kidney region in a renal dynamic imaging image based on hybrid attention branch according to claim 6, characterized in that: The renal dynamic imaging image segmentation model is trained through the following steps: (2-1) Obtain a pediatric renal dynamic imaging dataset consisting of pediatric renal dynamic imaging images, preprocess the pediatric renal dynamic imaging dataset to obtain a preprocessed pediatric renal dynamic imaging dataset, and divide the preprocessed pediatric renal dynamic imaging dataset into a training set and a test set in a ratio of 4:
1. (2-2) initializing the renal dynamic imaging image segmentation model to obtain an initialized renal dynamic imaging image segmentation model; (2-3) For each pediatric renal dynamic imaging image in the training set obtained in step (2-1), the pediatric renal dynamic imaging image is input into the SAM encoder branch and the VMamba encoder branch respectively to obtain two feature vectors corresponding to the pediatric renal dynamic imaging image respectively. (2-4) For each pediatric renal dynamic imaging image in the training set obtained in step (2-1), the two feature vectors corresponding to the pediatric renal dynamic imaging image obtained in step (2-3) are input into the cross-branch attention part for fusion to obtain a fused feature map corresponding to the pediatric renal dynamic imaging image. (2-5) For each pediatric renal dynamic imaging image in the training set obtained in step (2-1), the fusion feature map corresponding to the pediatric renal dynamic imaging image obtained in step (2-4) is input into the image decoder part to obtain the segmentation result corresponding to the pediatric renal dynamic imaging image. (2-6) according to the segmentation result corresponding to each pediatric renal dynamic imaging image obtained in step (2-5), and using the loss function to iteratively train the renal dynamic imaging image segmentation model until the renal dynamic imaging image segmentation model converges, thereby obtaining a preliminarily trained renal dynamic imaging image segmentation model; (2-7) Use the test set obtained in step (2-1) to verify the renal dynamic imaging image segmentation model preliminarily trained in step (2-6) until the classification accuracy reaches the optimal level or the set total number of iterations (epochs) is reached, thereby obtaining a trained renal dynamic imaging image segmentation model.
8. The method for segmenting the kidney region in a renal dynamic imaging image based on hybrid attention branch according to claim 7, characterized in that: Step (2-2) is to initialize the total number of iterations and the learning rate of the renal dynamic imaging image segmentation model, wherein the total number of iterations ranges from 100 to 300, preferably 200, and the learning rate ranges from 0 to 1, preferably 0.001; Step (2-6) uses the Dice Loss function as the loss function, and its calculation formula is: Among them, is a constant, which is equal to 1*10 -8 , which is used to prevent the denominator from being 0. X represents the true segmentation result of the pediatric renal dynamic imaging image obtained in step (2-5), Y represents the predicted segmentation result of the pediatric renal dynamic imaging image, and |X| and |Y| represent the sum of voxels of the true segmentation result and the sum of voxels of the predicted segmentation result, respectively.
9. A system for segmenting kidney regions in renal dynamic imaging images based on hybrid attention branches, characterized in that: include: The first module is used to obtain a renal dynamic imaging image acquired by a single photon emission computed tomography (SPECT) scanner, and preprocess the renal dynamic imaging image to obtain a plurality of preprocessed images. The second module is used to input each preprocessed image obtained by the first module into a pre-trained renal dynamic imaging image segmentation model to obtain the corresponding kidney area.
Citation Information
Patent Citations
Liver tumor segmentation method based on mixed attention and multi-scale supervision
CN117764948A
SAM-based cross-modal domain generalization medical image segmentation method
CN117808834A
CT pancreatic tumor automatic segmentation method and system, terminal and storage medium
WO2024000161A1
Automated detection of keratorefractive surgeries on anterior segment optical coherence tomography (as-oct) scans, creation of the as-oct scans, and detection of implantable collamer lens vault
WO2024036140A2
Cited By
Image-based rectal cancer auxiliary diagnosis method and system
CN119324034A
Column chromatography automatic control analysis method and system based on DS-Transform image segmentation
CN119693405A
A column chromatography automated control and analysis method and system based on DS-Transformer image segmentation
CN119693405B
Gastrointestinal tract disease classification method and system based on convolution and state space model
CN119992200A
A classification method and system for gastrointestinal diseases based on convolution and state-space models
CN119992200B
