Kidney ultrasound image segmentation method based on HieraUnet model
By using the HieraUnet model and a self-built dataset, combined with transfer learning and a specific loss function, the problems of time-consuming and labor-intensive traditional methods and inaccurate segmentation of deep learning methods were solved, achieving efficient and accurate segmentation of kidney ultrasound images and improving diagnostic accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-30
- Publication Date
- 2026-03-31
AI Technical Summary
Traditional kidney ultrasound image segmentation methods are time-consuming, labor-intensive, and easily affected by the operator's experience, making it difficult to provide accurate diagnoses in a short time. Deep learning methods are also difficult to segment completely and accurately when the boundary between the kidney and surrounding structures is unclear or the image edges are blurred.
A kidney ultrasound image segmentation method based on the HieraUnet model is adopted. By building a self-built dataset and utilizing the Hiera backbone network through transfer learning, combined with the Adapter module and a loss function that is the sum of BCE loss and IOU loss, accurate kidney segmentation is achieved.
It improves the accuracy and efficiency of kidney ultrasound image segmentation, reduces the need for human intervention, and can better process complex kidney ultrasound images, providing high-quality diagnostic information.
Smart Images
Figure CN121767262A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical image segmentation technology, specifically a kidney ultrasound image segmentation method based on the HieraUnet model. Background Technology
[0002] Kidney health is crucial to overall health, as the kidneys are responsible for regulating the body's water and electrolyte balance, participating in waste excretion, maintaining acid-base balance, and secreting various hormones, thus playing a key role in metabolic processes. Early detection and accurate diagnosis of kidney diseases are essential to ensure the proper functioning of these vital organs. In recent years, with the continuous development and improvement of ultrasound imaging technology, ultrasound diagnosis has become a painless, non-invasive, and low-cost routine examination method, widely used in the screening and diagnosis of kidney diseases. Kidney ultrasound images can provide detailed structural information about the kidneys, including cross-sectional area, lengths of the major and minor axes, and echo values of various regions. These parameters are important for assessing kidney health and diagnosing different types of kidney diseases (such as kidney stones, kidney cysts, and kidney tumors). Although ultrasound technology plays an important role in the early diagnosis of kidney diseases, traditional image segmentation methods still present many challenges. Manual segmentation of kidney ultrasound images requires meticulous operation by professional physicians, which is not only time-consuming and labor-intensive but also easily affected by the operator's experience level, potentially leading to low segmentation accuracy or even misidentification. Furthermore, due to the potential interference of various factors affecting the morphology and echogenicity of the kidneys, traditional image processing methods often perform poorly when processing complex kidney ultrasound images, making it difficult to provide accurate diagnostic information in a short time. With the rapid development of deep learning technology, deep learning-based automatic image segmentation methods have gradually become an effective way to solve these problems. Deep learning models, especially convolutional neural networks (CNNs), can automatically learn the features of kidney images through training on massive amounts of data, achieving efficient and accurate segmentation. Compared with traditional methods, deep learning-based automatic segmentation methods can significantly improve the accuracy and efficiency of segmentation, reducing the need for human intervention. However, despite the enormous potential of deep learning in kidney ultrasound image segmentation, it still faces some technical challenges. Although existing deep learning methods can improve segmentation accuracy to some extent, it is still difficult to achieve completely accurate segmentation in cases where the boundary between the kidney and surrounding structures is unclear or the image edges are blurred. To address these technical problems, this invention designs a kidney ultrasound image segmentation method based on the HieraUnet model. The HieraUnet model uses the Hiera network as its backbone. Based on transfer learning, the model can make full use of the powerful feature extraction capabilities of the Hiera backbone to more accurately apply it to the downstream task of kidney ultrasound image segmentation, which has important medical research and application value. Summary of the Invention
[0003] This invention designs a kidney ultrasound image segmentation method based on the HieraUnet model. By collecting and building a self-constructed kidney ultrasound image segmentation dataset, and using the HieraUnet network model, accurate segmentation of kidney ultrasound images is achieved. Based on transfer learning, a large Hiera backbone network model is employed. By freezing the pre-trained weight parameters of the backbone network, an adapter module is used to enable the model to fully utilize the powerful feature extraction capabilities of the Hiera backbone network for more accurate application to the downstream task of kidney ultrasound image segmentation. Furthermore, BCE loss (L...) is used... BCE ) and IOU loss (L IOU The sum of these factors serves as the loss function, guiding the HieraUnet network model to achieve more refined kidney segmentation results.
[0004] A kidney ultrasound image segmentation method based on the HieraUnet model includes the following steps:
[0005] Step S1: First, acquire high-quality kidney ultrasound images as training samples.
[0006] Step S2: The gold standard for creating kidney ultrasound images can be achieved by a professional sonographer using Labelme software;
[0007] Step S3: Construct a kidney ultrasound image segmentation dataset based on kidney ultrasound images and the gold standard;
[0008] Step S4: Preprocess the kidney ultrasound image dataset and divide it into training and validation sets according to a preset ratio;
[0009] Step S5: Feed the training set of the dataset into the HieraUnet network model for training and optimize the model;
[0010] Step S6: Accurate segmentation results of ultrasound kidney images based on the training-optimized model prediction validation set.
[0011] Furthermore, the kidney ultrasound images are obtained by professional sonographers using color Doppler ultrasound diagnostic instruments.
[0012] Furthermore, the gold standard for kidney ultrasound images is created using Labelme image labeling software written in Python. Each gold standard is created under the guidance of a professional ultrasound physician.
[0013] Furthermore, a kidney ultrasound image segmentation dataset was constructed based on kidney ultrasound images and the gold standard.
[0014] Furthermore, the preset ratio for dividing the kidney ultrasound image dataset into a training set and a validation set is 8:2.
[0015] Furthermore, the HieraUnet network model's backbone network is based on a pre-trained Hiera network model using transfer learning. Only the backbone network is retained, and its pre-trained weight parameters are frozen. On this basis, an Adapter module is introduced, enabling the model to fully utilize the powerful feature extraction capabilities of the Hiera backbone network for more accurate application to the downstream task of kidney ultrasound image segmentation.
[0016] Furthermore, the HieraUnet backbone network comprises four encoding modules, each consisting of multiple stacked basic encoding unit modules. The Adapter module and the Transformer module together form a basic encoding unit. The standard Transformer encoder layer is the basic encoding unit of the original model, while the Adapter module is used to fine-tune the frozen pre-trained weights.
[0017] Furthermore, the ViT module used in the backbone of the HieraUnet network model is a standard Transformer encoder layer. A standard Transformer encoder layer includes layer normalization, multi-head self-attention layers, layer normalization, and MLP (feedforward neural network), as well as residual connections to maintain information flow and training stability.
[0018] Furthermore, the Adapter module consists of a linear layer for downsampling, a GeLU activation function layer, then another linear layer for upsampling, and a final GeLU activation function layer. The Adapter module precedes each standard Transformer encoder layer and is assembled via residual connections.
[0019] Furthermore, the feature map sizes output by the four encoding modules in the backbone network of the HieraUnet network model are 144×88×88, 288×44×44, 576×22×22, and 1152×11×11.
[0020] Furthermore, the processing steps of the four feature maps obtained after passing through the backbone network and inputting them to the feature fusion decoding layer include: the fourth downsampled feature map is subjected to bilinear upsampling once, then concatenated with the third downsampled feature map through a skip connection, and then processed by two layers of convolution to obtain the third upsampled feature map; the third upsampled feature map is subjected to bilinear upsampling once, then concatenated with the second downsampled feature map through a skip connection, and then processed by two layers of convolution to obtain the second upsampled feature map; the second upsampled feature map is subjected to bilinear upsampling once, then concatenated with the first downsampled feature map through a skip connection, and then processed by two layers of convolution to finally obtain the first upsampled feature map, which is the final output feature map.
[0021] Furthermore, the final output feature map is fed into the segmentation head to obtain the final segmentation prediction result.
[0022] Furthermore, the loss function used in the HieraUnet network model is the BCE loss (L... BCE ) and IOU loss (L IOU The sum of these values allows the model to quantify and compensate for the difference between the predicted results and the true labels. BCE and L IOU The formula for the sum is as follows:
[0023]
[0024] In the formula, y i Indicates the truth value. This represents the predicted probability value, and N represents the number of pixels.
[0025] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0026] 1. This invention introduces the Hiera backbone network based on transfer learning in the field of kidney ultrasound image segmentation, which has significant medical research and application value;
[0027] 2. The HieraUnet network model designed in this invention uses the sum of BCE loss and IOU loss as the loss function, which can better guide the model to optimize and achieve fine segmentation of kidney ultrasound images;
[0028] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. Attached Figure Description
[0029] Figure 1 This is a schematic diagram of the network structure of the HieraUnet model of this invention;
[0030] Figure 2 This is a schematic diagram of the kidney ultrasound image segmentation dataset built by this invention.
[0031] Figure 3 This is a schematic diagram of the gold standard for kidney ultrasound image segmentation dataset built by this invention. Detailed Implementation
[0032] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.
[0033] Figure 1 A schematic diagram of the network structure of the HieraUnet model proposed in this invention is shown. This model consists of a backbone encoder and a feature fusion decoder. Multi-level features extracted by the encoder are fused with the decoder features through a concatenation operation, thereby achieving effective integration of image semantics and detailed information. Here, Input represents the model's input image, Output represents the model's output predicted image, Encoder Block1, Encoder Block2, Encoder Block3, and Encoder Block4 represent the four stages of image feature encoding, Decoder Block1, Decoder Block2, Decoder Block3, and Decoder Block4 represent the four stages of image feature decoding, and Concat represents concatenating encoded and decoded features of the same scale.
[0034] Figure 2 This is a schematic diagram of some images from the self-built kidney ultrasound image segmentation dataset of this invention. The figure shows the morphology of the kidney under ultrasound images, which is used for model input and training.
[0035] Figure 3 This is a schematic diagram illustrating the gold standard in the dataset. The labels were created by professional physicians based on the corresponding ultrasound images, and are shown in the diagram. Figure 2 The images in the dataset are one-to-one correspondences, meaning that images with the same name and their labels together constitute a set of training samples.
[0036] I. Backbone Network of the HieraUnet Network Model
[0037] Based on transfer learning, a model pre-trained using Hiera is used. Other components are removed, leaving only the backbone network, and the pre-trained weights of the backbone network are frozen. An Adapter module is then introduced to fine-tune the pre-training, enabling the model to fully utilize the powerful feature extraction capabilities of the Hiera backbone network for more accurate application to the downstream task of kidney ultrasound image segmentation.
[0038] The Hiera backbone network consists of four encoding modules, each composed of multiple stacked basic encoding unit modules. An adapter module and a standard Transformer encoder layer together form a basic encoding unit. The standard Transformer encoder layer is the basic encoding unit of the original model, while the adapter module is used to fine-tune the frozen pre-trained weights.
[0039] The Adapter module and the standard Transformer encoder layer together form a basic encoding unit. The Transformer module used in the backbone of the HieraUnet network model is the same as that used in the original Hiera model, which is the standard Transformer encoder layer. The standard Transformer encoder layer includes layer normalization, multi-head self-attention layer, layer normalization, and MLP (feedforward neural network), as well as residual connections to maintain information flow and training stability. The Adapter module consists of a linear layer for downsampling, a GeLU activation function layer, then another linear layer for upsampling, and a final GeLU activation function layer. The Adapter module precedes each standard Transformer encoder layer and is combined through residual connections.
[0040] The HieraUnet network model has four outputs in its backbone network, with feature map sizes of 144×88×88, 288×44×44, 576×22×22, and 1152×11×11, respectively.
[0041] II. Feature Fusion Decoding Module
[0042] The feature fusion decoding module integrates feature information extracted from four different levels of the backbone network to generate two richer and more comprehensive feature maps, which are then fused again to obtain an even more comprehensive feature map. The advantages of the feature fusion decoding module include capturing information that a single feature cannot provide by fusing features from multiple different levels of abstraction; different feature sources may capture different aspects of the data, and fusing these features helps the HieraUnet network model better understand and adapt to diverse inputs, which helps improve the model's performance on unseen data, i.e., enhances its generalization ability. Furthermore, the feature fusion decoding module not only fuses features but also participates in the decoding process, transforming the fused features into the final output. Through the designed fusion strategy, the decoder can be effectively guided to generate high-quality results.
[0043] The four feature maps obtained after passing through the backbone network are processed by the following steps in the feature fusion decoding layer: The fourth downsampled feature map undergoes bilinear upsampling once, then is concatenated with the third downsampled feature map via a skip connection, followed by two convolutional layers to obtain the third upsampled feature map; the third upsampled feature map undergoes bilinear upsampling once, then is concatenated with the second downsampled feature map via a skip connection, followed by two convolutional layers to obtain the second upsampled feature map; the second upsampled feature map undergoes bilinear upsampling once, then is concatenated with the first downsampled feature map via a skip connection, followed by two convolutional layers to obtain the first upsampled feature map, which is the final output feature map. The final output feature map is then fed into the segmentation head to obtain the final segmentation prediction result.
[0044] III. Loss Function
[0045] In the field of medical image segmentation technology, BCE loss (L BCE ) and IOU loss (L IOU BCE loss and IOU loss are two commonly used loss functions. BCE loss calculates the difference between the predicted segmentation map and the gold standard, while IOU loss calculates the degree of mismatch between the predicted segmentation map and the gold standard. Since a single loss function cannot achieve good kidney segmentation results, BCE loss and IOU loss are used in combination, as shown in the following formula:
[0046]
[0047] In the formula, y i Indicates the truth value. This represents the predicted probability value, and N represents the number of pixels.
[0048] IV. Experiment
[0049] To verify the performance of the model designed in this invention on kidney segmentation tasks using ultrasound images, this invention was experimentally evaluated on a self-built kidney ultrasound image segmentation dataset. The kidney ultrasound image segmentation dataset includes kidney ultrasound images acquired by professional sonographers using color Doppler ultrasound diagnostic instruments with the consent of volunteers, and the gold standard for kidney ultrasound image segmentation produced under the guidance of professional sonographers.
[0050] The model designed in this invention is implemented based on the hardware environment of an NVIDIA RTX 4060 and the software environment of Python and PyTorch. The kidney ultrasound image dataset is randomly divided into training and validation sets at a preset ratio of 8:2. To ensure data consistency, all images are resized to a fixed size. During model training, the initial parameters are obtained using PyTorch's default initialization method, and the network parameters are updated using the Adam optimizer. The initial learning rate is 0.0001, the weight decay coefficient is 0.0005, and the total number of iterations is 200.
[0051] To quantitatively evaluate the segmentation performance of different methods for kidney segmentation in ultrasound images, two commonly used segmentation metrics, Dice and IOU, were used, with the following formulas:
[0052]
[0053] TP is the number of features that are predicted as positive and actually are positive; FP is the number of features that are predicted as positive but actually are negative; TN is the number of features that are predicted as negative but actually are negative; FN is the number of features that are predicted as negative but actually are positive. The values of the above 5 indicators are all in the range of [0,1]. The higher the value, the better the result of kidney segmentation.
Claims
1. A method for kidney ultrasound image segmentation based on HieraUnet model, The features include the following steps: Step S1, first obtain high-quality kidney ultrasound images as training samples; Step S2, use Labelme software to make the gold standard of kidney ultrasound images; Step S3, construct a kidney ultrasound image segmentation dataset based on the kidney ultrasound images and the gold standard; Step S4, pre-process the kidney ultrasound image dataset, and divide the training set and the validation set according to the preset proportion; Step S5, send the training set in the data set into the HieraUnet network model for training, and optimize the model; Step S6, based on the training and optimization of the model, predict the accurate segmentation result of the ultrasound kidney image of the validation set.
2. The kidney ultrasound image segmentation method based on the HieraUnet model according to claim 1, characterized in that The kidney ultrasound image is derived from the kidney image collected by a color Doppler ultrasound diagnostic instrument.
3. The kidney ultrasound image segmentation method based on the HieraUnet model according to claim 2, characterized in that The tool for making the gold standard of the kidney ultrasound image is the Labelme image marking software written in Python language.
4. The kidney ultrasound image segmentation method based on the HieraUnet model according to claim 3, characterized in that The preset proportion for dividing the training set and the validation set from the kidney ultrasound image dataset is 8:
2.
5. The kidney ultrasound image segmentation method based on the HieraUnet model according to claim 4, characterized in that: The backbone network of the HieraUnet network model is a Hiera network based on transfer learning using a pre-trained model, only the backbone network of the model is retained, and the pre-training weight parameters of the backbone network are frozen; On this basis, the Adapter module is introduced, so that the model can fully utilize the feature extraction capability of the Hiera backbone network to more accurately apply to the downstream task of kidney ultrasound image segmentation.
6. The kidney ultrasound image segmentation method based on the HieraUnet model according to claim 5, characterized in that: The HieraUnet backbone network includes four encoding modules, and each encoding module is stacked by a plurality of basic encoding unit modules; the Adapter module and the Transformer module jointly constitute a basic encoding unit; the standard Transformer encoder layer is the basic encoding unit of the original model, and the Adapter module is used to fine-tune the frozen pre-training weight; The Transformer module used in the backbone network of the HieraUnet network model is a standard Transformer encoder layer; the standard Transformer encoder layer includes layer normalization, multi-head self-attention layer, layer normalization and MLP feedforward neural network, and further includes residual connection to maintain information flow and training stability.
7. The kidney ultrasound image segmentation method based on the HieraUnet model according to claim 6, characterized in that: The Adapter module includes a linear layer for downsampling, a GeLU activation function layer, then another linear layer for upsampling, and a final GeLU activation function layer. The Adapter module is before each standard Transformer encoder layer and is combined through residual connection.
8. The kidney ultrasound image segmentation method based on the HieraUnet model according to claim 7, characterized in that: The feature map size output by the four encoding modules in the backbone network of the HieraUnet network model is 144×88×88, 288×44×44, 576×22×22, 1152×11×11; The four feature maps obtained after the backbone network are input to a processing step of a feature fusion decoding layer, which includes: the fourth down-sampling feature map is subjected to one time of bilinear up-sampling, then spliced with the third down-sampling feature map through a jump connection, and then subjected to double-layer convolution processing to obtain a third up-sampling feature map; the third up-sampling feature map is subjected to one time of bilinear up-sampling, then spliced with the second down-sampling feature map through a jump connection, and then subjected to double-layer convolution processing to obtain a second up-sampling feature map; the second up-sampling feature map is subjected to one time of bilinear up-sampling, then spliced with the first down-sampling feature map through a jump connection, and then subjected to double-layer convolution processing to finally obtain a first up-sampling feature map, which is the final output feature map; The final output feature map is input into a segmentation head to obtain a final segmentation prediction result.
9. The kidney ultrasound image segmentation method based on the HieraUnet model according to claim 8, characterized in that: The loss function used in the HieraUnet network model is the sum of the BCE loss (L BCE ) and the IOU loss (L IOU ), so that the model can quantify and compensate for the difference between the prediction result and the true label; the formula of the sum of L BCE and L IOU is as follows: In the formula, y i represents a true value, represents a predicted probability value, and N represents the number of pixels.