A lightweight medical image segmentation system and method based on Hadamard product

Abstract

The present invention discloses a lightweight medical image segmentation system and method based on Hadamard product. The system includes: a model construction module, a model improvement module, a model training module, and an image segmentation module; the model construction module is used to construct an initial network model; the model improvement module is used to improve the initial network model to obtain an improved HPGA-UNet model; the model training module is used to train the HPGA-UNet model to obtain an image segmentation model; and the image segmentation module segments the collected medical images based on the image segmentation model to obtain semantic segmentation results of the images. The improved image segmentation model of the present invention can automatically learn global features and adjust parameters for different input images, has certain attention and adaptability, effectively reduces the number of model parameters and floating-point calculations, and also has certain improvements in medical image segmentation accuracy compared to traditional convolutional neural networks.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image segmentation, and in particular relates to a lightweight medical image segmentation system and method based on Hadamard product. Background Art

[0002] Research on lightweight medical image semantic segmentation algorithms is crucial for improving resource efficiency, real-time performance, portability, deployability, and privacy protection. This research can promote the widespread application of medical image semantic segmentation technology in clinical practice and healthcare, providing better support for medical services and patient health.

[0003] Existing models have large number of parameters and computational complexity, making them difficult to apply to mobile medical devices. Many researchers have attempted to achieve the goal of lightweighting the model by reducing the number of model parameters. However, reducing the number of parameters often leads to a decrease in the network's segmentation accuracy for edge parts. In addition, a large number of convolution and downsampling operations in the model will lose some important information, and upsampling cannot fully restore spatial information. Therefore, there is an urgent need to study ways to make the module lighter while ensuring the segmentation accuracy as much as possible. Summary of the Invention

[0004] The present invention aims to solve the deficiencies of the prior art and provides the following solutions:

[0005] A lightweight medical image segmentation system based on Hadamard product, comprising: a model building module, a model improvement module, a model training module and an image segmentation module;

[0006] The model building module is used to build an initial network model;

[0007] The model improvement module is used to improve the initial network model to obtain an improved HPGA-UNet model;

[0008] The model training module is used to train the HPGA-UNet model to obtain an image segmentation model;

[0009] The image segmentation module segments the collected medical image based on the image segmentation model to obtain a semantic segmentation result of the image.

[0010] Preferably, the initial network model includes: an encoder, a decoder, a skip connection layer and a loss function;

[0011] The encoder is composed of several double convolutional layers, activation functions and pooling layers, and is used for feature extraction;

[0012] The decoder consists of several transposed double convolutional layers, activation functions and upsampling layers. The decoder is used to restore the spatial dimension while reducing the number of channels.

[0013] The skip connection layer is used to splice the feature maps of different levels in the encoder with the feature maps of the decoder at the corresponding level to complete semantic feature fusion;

[0014] The loss function is used to optimize the performance index of the segmentation task.

[0015] Preferably, the model improvement module includes: a global attention improvement unit and a convolution replacement unit;

[0016] The global attention improvement unit is used to design a Hadamard product global attention module HPGA based on the Transformer multi-head attention mechanism;

[0017] The convolution replacement unit is used to replace the double convolution layers of the U-Net encoder and decoder segments in the initial network model with the Hadamard product global attention module HPGA to obtain the improved HPGA-UNet model.

[0018] Preferably, the model training module includes: a data set collection unit, a data preprocessing unit and a model training unit;

[0019] The data set collection unit is used to collect a medical semantic segmentation data set, wherein the medical semantic segmentation data set includes original medical images and labels;

[0020] The data preprocessing unit is used to perform size adjustment, normalization and image enhancement processing on the original medical image to obtain a preprocessed data set;

[0021] The model training unit is used to divide the preprocessed data set into a training set and a test set, train the HPGA-UNet model based on the training set, and evaluate the trained model using the test set to obtain an evaluation result;

[0022] The model training unit is also used to adjust the hyperparameters of the model based on the evaluation results, and perform visual analysis on the model after the hyperparameters are adjusted, compare the semantic segmentation results predicted by the model with the labels, adjust the boundary details of the model, and obtain the image segmentation model.

[0023] The present invention also provides a lightweight medical image segmentation method based on Hadamard product, comprising the following steps:

[0024] Build an initial network model;

[0025] Improving the initial network model to obtain an improved HPGA-UNet model;

[0026] Training the HPGA-UNet model to obtain an image segmentation model;

[0027] The collected medical image is segmented based on the image segmentation model to obtain a semantic segmentation result of the image.

[0028] Preferably, the initial network model includes: an encoder, a decoder, a skip connection layer and a loss function;

[0029] The encoder is composed of several double convolutional layers, activation functions and pooling layers, and is used for feature extraction;

[0030] The decoder consists of several transposed double convolutional layers, activation functions and upsampling layers. The decoder is used to restore the spatial dimension while reducing the number of channels.

[0031] The skip connection layer is used to splice the feature maps of different levels in the encoder with the feature maps of the decoder at the corresponding level to complete semantic feature fusion;

[0032] The loss function is used to optimize the performance index of the segmentation task.

[0033] Preferably, the improved method comprises:

[0034] Design the Hadamard product global attention module HPGA based on the Transformer multi-head attention mechanism;

[0035] The Hadamard product global attention module HPGA replaces the double convolutional layers of the U-Net encoder and decoder segments in the initial network model to obtain the improved HPGA-UNet model.

[0036] Preferably, the training method includes:

[0037] Collect a medical semantic segmentation dataset, wherein the medical semantic segmentation dataset includes original medical images and labels;

[0038] performing size adjustment, normalization, and image enhancement processing on the original medical image to obtain a preprocessed data set;

[0039] Dividing the preprocessed data set into a training set and a test set, training the HPGA-UNet model based on the training set, and evaluating the trained model using the test set to obtain an evaluation result;

[0040] The hyperparameters of the model are adjusted based on the evaluation results, and the model after the hyperparameter adjustment is visually analyzed. The semantic segmentation results predicted by the model are compared with the labels, and the boundary details of the model are adjusted to obtain the image segmentation model.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] Based on the Transformer multi-head attention mechanism design, this paper proposes a Hadamard product global attention module HPGA, and replaces the U-Net double convolutional layer in the constructed initial model with the Hadamard product global attention module HPGA for improvement. The improved image segmentation model can automatically learn global features and adjust parameters for different input images, has certain attention and adaptability, effectively reduces the number of model parameters and floating-point calculations, and also has certain improvements in medical image segmentation accuracy compared to traditional convolutional neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 Schematic diagram of the system structure of an embodiment of the present invention;

[0045] Figure 2 Schematic diagram of the structure of the Hadamard product global attention module HPGA according to an embodiment of the present invention;

[0046] Figure 3 Schematic diagram of the structure of HPA in the Hadamard product global attention module of an embodiment of the present invention;

[0047] Figure 4 Schematic diagram of the image segmentation model structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0049] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] Example 1

[0051] In this embodiment, if Figure 1As shown, a lightweight medical image segmentation system based on Hadamard product includes: a model building module, a model improvement module, a model training module and an image segmentation module.

[0052] The model building module is used to build the initial network model.

[0053] The initial network model includes an encoder, a decoder, a skip connection layer, and a loss function. The encoder, consisting of several dual convolutional layers, activation functions, and pooling layers, is used for feature extraction. The decoder, consisting of several transposed dual convolutional layers, activation functions, and upsampling layers, is used to restore spatial dimensions while reducing the number of channels. The skip connection layer is used to concatenate feature maps from different levels of the encoder with those from the corresponding decoder layers to achieve semantic feature fusion. The loss function is used to optimize the performance of the segmentation task.

[0054] In this embodiment, specifically, the encoder (Encoder): consists of a series of double convolution layers, activation functions and pooling layers, which plays the role of feature extraction. The encoder gradually reduces the spatial dimension while increasing the number of channels to extract more and more abstract features. The decoder (Decoder): consists of a series of transposed double convolution layers (Transposed Convolution), activation functions and upsampling layers; the decoder gradually restores the spatial dimension while reducing the number of channels to generate pixel-level prediction outputs. The transposed convolution operation of each layer will double the size of the feature map. Skip Connections: The feature maps of different levels in the encoder are spliced with the decoder feature maps of the corresponding levels. This skip connection structure can fuse the detailed features of the bottom layer with the semantic features of the high layer to improve the segmentation accuracy. Loss function: The DiceLoss loss function is used to directly optimize the performance indicators of the segmentation task, such as mIoU, Dice coefficient, etc.

[0055] The model improvement module is used to improve the initial network model to obtain the improved HPGA-UNet model.

[0056] The model improvement module includes a global attention improvement unit and a convolution replacement unit. The global attention improvement unit is used to design a Hadamard product global attention module (HPGA) based on the Transformer multi-head attention mechanism. The convolution replacement unit replaces the double convolutional layers in the U-Net encoder and decoder sections of the initial network model with the Hadamard product global attention module (HPGA), resulting in the improved HPGA-UNet model.

[0057] In this embodiment, based on the inspiration of MHSA (multi-head attention mechanism), a global attention module based on Hadamard product is designed, such as Figure 2As shown in the figure, Hadamard product, also known as element-wise product or element-level product, refers to the element-by-element multiplication of two matrices or vectors with the same dimension. Specifically, 4 sets of input images are obtained, and 3 sets of input image data are subjected to HPA operation to extract feature information. The structure of HPA is as follows Figure 3 As shown in the figure, for the last set of input image data, only the DW operation is performed on the feature map and a weight parameter p4 is assigned. Finally, the four sets of data are concat-operated in the channel dimension, and the DW operation is continued to be used to integrate information from different perspectives.

[0058] The model training module is used to train the HPGA-UNet model to obtain an image segmentation model.

[0059] The model training module includes: data set collection unit, data preprocessing unit and model training unit. The data set collection unit is used to collect medical semantic segmentation data sets such as ISIC2017 (International Skin Imaging Collaboration 2017) International Skin Imaging Collaboration 2017 Challenge Dataset, ISIC2018 ((International Skin Imaging Collaboration 2018) International Skin Imaging Collaboration 2018 Challenge Dataset, BreCaHAD (Breast Cancer Histology Dataset) breast cancer histology dataset, medical semantic segmentation dataset includes original medical images and labels. The data preprocessing unit is used to resize, normalize and enhance the original medical images to obtain a preprocessed dataset. In this embodiment, the workflow of the data preprocessing unit is as follows: determine the uniform size required for the model input, use the interpolation algorithm to scale the image to the required size, and ensure that all input images have the same size; scale the pixel value to a standard range, usually [0,1] or [-1,1], complete the normalization operation, and ensure that the pixel values of different images are distributed in the same range, which is conducive to model convergence; by randomly flipping, rotating, scaling, cropping, color space transformation, and adding noise to the original image, complete image enhancement and generate new training samples, namely the preprocessed dataset. The model training unit is used to divide the preprocessed dataset into a training set and a test set, and train the HPGA-UNet model based on the training set. In this embodiment, during the training process, GT_BceDiceLoss is used to measure the difference between the model output and the true label. Considering the balance between convergence speed and learning ability, the initial learning rate defaults to 1e-3, and batch The size (number of samples used in each training) is 8, the epoch is set to 300, and the optimization algorithm AdamW is used to update the model parameters. The trained model is evaluated using the test set to obtain the evaluation results. The model training unit is also used to adjust the model's hyperparameters based on the evaluation results, and to perform a visual analysis of the model after the hyperparameter adjustment. The semantic segmentation results predicted by the model are compared with the labels, and the boundary details of the model are adjusted to obtain the image segmentation model, such as Figure 4 shown.

[0060] In this embodiment, the specific steps of hyperparameter adjustment are as follows: determine the hyperparameters that need to be adjusted, including learning rate and batch size; set a reasonable search range for each hyperparameter that needs to be adjusted, set the learning rate to the logarithmic space of [1e-5, 1e-1], and set the batch size to discrete values such as [32, 64, 128, 256]; use the Bayesian statistical model to adaptively explore the search space and iteratively optimize the hyperparameters.

[0061] The steps of visualization analysis include: visual analysis of the distribution and feature correlation of input data, and use scatter plots and other methods to display data characteristics; draw the network structure diagram of the model to clearly show the connection relationship between each layer; visualize the feature representation learned by the model in different hidden layers, and understand the semantic information learned by the model in each layer through feature maps, activation maps, etc.; visualize the distribution of attention weights, and analyze which input areas or features receive more attention when the model makes predictions; draw feature maps of error samples, analyze the samples predicted by the model incorrectly, and find out the key factors that lead to the error; draw learning curves and loss function curves, visualize the verification indicators in the hyperparameter tuning process, and observe the performance of different parameter combinations.

[0062] The image segmentation module segments the collected medical images based on the image segmentation model to obtain the semantic segmentation results of the images.

[0063] The image segmentation model is deployed to a mobile medical device, which collects medical images. The collected medical images are segmented using the image segmentation model to obtain semantic segmentation results of the images.

[0064] Example 2

[0065] In this embodiment, a lightweight medical image segmentation method based on Hadamard product includes the following steps:

[0066] S1. Build the initial network model.

[0067] The initial network model includes an encoder, a decoder, a skip connection layer, and a loss function. The encoder, consisting of several dual convolutional layers, activation functions, and pooling layers, is used for feature extraction. The decoder, consisting of several transposed dual convolutional layers, activation functions, and upsampling layers, is used to restore spatial dimensions while reducing the number of channels. The skip connection layer is used to concatenate feature maps from different levels of the encoder with those from the corresponding decoder layers to achieve semantic feature fusion. The loss function is used to optimize the performance of the segmentation task.

[0068] S2. Improve the initial network model to obtain the improved HPGA-UNet model.

[0069] The improved method includes: designing a Hadamard product global attention module HPGA based on the Transformer multi-head attention mechanism; replacing the double convolutional layers of the U-Net encoder and decoder segments in the initial network model with the Hadamard product global attention module HPGA to obtain the improved HPGA-UNet model.

[0070] S3. Train the HPGA-UNet model to obtain an image segmentation model.

[0071] The training method includes: collecting a medical semantic segmentation dataset, which includes original medical images and labels; resizing, normalizing and image enhancement processing the original medical images to obtain a preprocessed dataset; dividing the preprocessed dataset into a training set and a test set, training the HPGA-UNet model based on the training set, and evaluating the trained model using the test set to obtain evaluation results; adjusting the model's hyperparameters based on the evaluation results, and visually analyzing the model after the hyperparameter adjustment, comparing the semantic segmentation results predicted by the model with the labels, and adjusting the model's boundary details to obtain an image segmentation model.

[0072] S4. Segment the collected medical images based on the image segmentation model to obtain the semantic segmentation results of the images.

[0073] The image segmentation model is deployed to a mobile medical device, which collects medical images. The collected medical images are segmented using the image segmentation model to obtain semantic segmentation results of the images.

[0074] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A lightweight medical image segmentation system based on Hadamard product, characterized in that: include: Model building module, model improvement module, model training module and image segmentation module; The model building module is used to build an initial network model; The model improvement module is used to improve the initial network model to obtain an improved HPGA-UNet model; The model training module is used to train the HPGA-UNet model to obtain an image segmentation model; The image segmentation module segments the collected medical image based on the image segmentation model to obtain a semantic segmentation result of the image; The initial network model includes: an encoder, a decoder, a skip connection layer and a loss function; The encoder is composed of several double convolutional layers, activation functions and pooling layers, and is used for feature extraction; The decoder consists of several transposed double convolutional layers, activation functions and upsampling layers. The decoder is used to restore the spatial dimension while reducing the number of channels. The skip connection layer is used to splice the feature maps of different levels in the encoder with the feature maps of the decoder at the corresponding level to complete semantic feature fusion; The loss function is used to optimize the performance index of the segmentation task; The model improvement module includes: a global attention improvement unit and a convolution replacement unit; The global attention improvement unit is used to design a Hadamard product global attention module HPGA based on the Transformer multi-head attention mechanism; The convolution replacement unit is used to replace the double convolution layers of the U-Net encoder and decoder segments in the initial network model with the Hadamard product global attention module HPGA to obtain the improved HPGA-UNet model.

2. The lightweight medical image segmentation system based on Hadamard product according to claim 1, characterized in that: The model training module includes: a data set collection unit, a data preprocessing unit and a model training unit; The data set collection unit is used to collect a medical semantic segmentation data set, wherein the medical semantic segmentation data set includes original medical images and labels; The data preprocessing unit is used to perform size adjustment, normalization and image enhancement processing on the original medical image to obtain a preprocessed data set; The model training unit is used to divide the preprocessed data set into a training set and a test set, train the HPGA-UNet model based on the training set, and evaluate the trained model using the test set to obtain an evaluation result; The model training unit is also used to adjust the hyperparameters of the model based on the evaluation results, and perform visual analysis on the model after the hyperparameters are adjusted, compare the semantic segmentation results predicted by the model with the labels, adjust the boundary details of the model, and obtain the image segmentation model.

3. A lightweight medical image segmentation method based on Hadamard product, characterized in that: The following steps are involved: Build an initial network model; Improving the initial network model to obtain an improved HPGA-UNet model; Training the HPGA-UNet model to obtain an image segmentation model; Segmenting the collected medical image based on the image segmentation model to obtain a semantic segmentation result of the image; The initial network model includes: an encoder, a decoder, a skip connection layer and a loss function; The encoder is composed of several double convolutional layers, activation functions and pooling layers, and is used for feature extraction; The decoder consists of several transposed double convolutional layers, activation functions and upsampling layers. The decoder is used to restore the spatial dimension while reducing the number of channels. The skip connection layer is used to splice the feature maps of different levels in the encoder with the feature maps of the decoder at the corresponding level to complete semantic feature fusion; The loss function is used to optimize the performance index of the segmentation task; The improved method comprises: Design the Hadamard product global attention module HPGA based on the Transformer multi-head attention mechanism; The Hadamard product global attention module HPGA replaces the double convolutional layers of the U-Net encoder and decoder segments in the initial network model to obtain the improved HPGA-UNet model.

4. The lightweight medical image segmentation method based on Hadamard product according to claim 3, characterized in that: The training method includes: Collect a medical semantic segmentation dataset, wherein the medical semantic segmentation dataset includes original medical images and labels; performing size adjustment, normalization, and image enhancement processing on the original medical image to obtain a preprocessed data set; Dividing the preprocessed data set into a training set and a test set, training the HPGA-UNet model based on the training set, and evaluating the trained model using the test set to obtain an evaluation result; The hyperparameters of the model are adjusted based on the evaluation results, and the model after the hyperparameter adjustment is visually analyzed. The semantic segmentation results predicted by the model are compared with the labels, and the boundary details of the model are adjusted to obtain the image segmentation model.

Images