Two-dimensional echocardiography left ventricle segmentation method based on improved U-Net

Abstract

The invention belongs to the technical field of medical instruments, and particularly relates to a two-dimensional echocardiography left ventricle segmentation method based on improved U-Net. The method comprises an image preprocessing module, a feature extraction and fusion module and a classifier module. The method comprises the following steps: firstly, carrying out preprocessing operations such as center cutting and downsampling on an acquired two-dimensional echocardiography image; inputting the processed image into an improved U-Net network, expanding a receptive field of a convolution kernel through expansion convolution, extracting image features, and enhancing feature representation of a key feature channel and a key space position through a channel attention and space attention module; and finally, logistic regression is adopted as a classifier, pixel-level classification is performed on the image, and left ventricle segmentation of the two-dimensional echocardiography image is realized. According to the method, the problem that automatic left ventricle segmentation is difficult due to individual differences and pathological changes can be effectively solved, a more accurate left ventricle segmentation result is provided for medical research and clinical diagnosis, and early discovery and treatment of heart diseases are facilitated.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical instruments, and particularly relates to a method for segmenting the left ventricle of two-dimensional echocardiogram. Background Art

[0002] As a serious clinical syndrome, heart failure is commonly induced by myocardial infarction, cardiomyopathy, myocarditis, etc. The high-risk population includes the elderly over 70 years old, with an incidence rate exceeding 10%, and the 5-year mortality rate can reach 50%. The left ventricular ejection fraction is a key parameter for measuring heart function, mainly used to measure the efficiency of the left ventricle in pumping blood during systole. It reflects the systolic function of the heart and is a key parameter for diagnosing heart failure, evaluating the severity of heart diseases, and monitoring the treatment effect. In clinical applications, the most common method for measuring the left ventricular ejection fraction is based on segmenting the left ventricle of echocardiogram to obtain the end-diastolic volume and end-systolic volume.

[0003] Currently, the left ventricle segmentation of two-dimensional echocardiogram is usually manually performed by professional physicians through manual tracing of the left ventricle contour in the image. This process is not only time-consuming and laborious, but also greatly affected by human factors in terms of segmentation accuracy. Therefore, a method suitable for automatic left ventricle segmentation is needed to ensure the accuracy and stability of the segmentation results. Summary of the Invention

[0004] In view of the above requirements for the left ventricle segmentation algorithm and the deficiencies of existing research technologies, the present invention provides a method for segmenting the left ventricle of two-dimensional echocardiogram based on improved U-Net to solve the problem of difficult automatic left ventricle segmentation caused by individual differences and pathological changes.

[0005] The method for segmenting the left ventricle of two-dimensional echocardiogram based on improved U-Net provided by the present invention includes an image preprocessing module, a feature extraction and fusion module, and a classifier; wherein:

[0006] The image preprocessing module is used to preprocess the collected two-dimensional echocardiogram images; specifically, it includes extracting the region of interest from the two-dimensional echocardiogram, that is, deleting the interference information around the image through central cropping and retaining the main part; then downsampling the cropped image to 112 pixels × 112 pixels using bilinear interpolation to meet the subsequent network input requirements.

[0007] The feature extraction and fusion module takes the preprocessed image as input and applies an improved U-Net for feature extraction and feature fusion. It includes an encoder and a decoder. Specifically, the encoder consists of multiple convolutional layers and pooling layers. Dilated convolutions are used in the encoder part, and the dilation rate is increased layer by layer to expand the receptive field of the convolutional kernel, extract high-level features of the image, and at the same time reduce the resolution of the image. The decoder part restores the low-resolution feature map output by the encoder to the original image size step by step through upsampling operations, and performs channel splicing with the feature map of the corresponding level of the encoder through skip connections to avoid feature loss. In addition, an attention mechanism is introduced in the U-Net to weight the feature map in both the channel and spatial dimensions to enhance the model's ability to focus on key information. There are two ways to introduce attention:

[0008] (1) Add a channel attention module in the U-Net encoder. By learning the importance of different channels, different weights are assigned to each channel, thereby enhancing the response of key feature channels and suppressing unimportant channels;

[0009] (2) Add a spatial attention module in the U-Net decoder. By learning the importance of different spatial positions in the feature map, different weights are assigned to each position, thereby enhancing the feature response of key regions and suppressing unimportant regions;

[0010] The classifier module uses logistic regression as the classifier, measures the classification performance by combining the cross-entropy loss function and the DICE loss function, and introduces an L2 regularization term to prevent the feature coefficients from being too large, reduce the complexity of the model, and prevent overfitting.

[0011] Furthermore:

[0012] In the present invention, the specific working process of the image preprocessing module is as follows:

[0013] (1) Use central cropping to delete the interference information around the two-dimensional echocardiogram, retain the main part. For a two-dimensional echocardiogram image with an original width of W and a height of H, the target width is w and the target height is h. The starting coordinates of the central cropping are as shown in Equation (1): Figure 4 For a two-dimensional echocardiogram image with an original width of W and a height of H, the target width is w and the target height is h. The starting coordinates of the central cropping are as shown in Equation (1):

[0014]

[0015] where, represents rounding down.

[0016] (2) Use bilinear interpolation to downsample the cropped image, and estimate the value of the target point (x, y) by calculating the weighted average of the surrounding four known points (x1, y1), (x2, y1), (x1, y2) and (x2, y2). The calculation formula is as shown in Equation (2):

[0017]

[0018] where x1 ≤ x ≤ x2 and y1 ≤ y ≤ y2 (2).

[0019] In the present invention, the construction of the segmentation network model specifically includes the following steps:

[0020] (1) Use a four-layer basic U-Net model, and the input of the model is a preprocessed two-dimensional echocardiogram with a size of 112 pixels × 112 pixels;

[0021] (2) Replace the 3×3 convolution in each layer of the encoder with dilated convolutions with dilation rates of 1, 2, 4, and 8 respectively, and their receptive fields are 3×3, 7×7, 15×15, and 31×31 respectively.

[0022] (3) Add a channel attention module after the convolution operation in the encoder. The specific implementation steps are as follows:

[0023] ① Global pooling: Perform global max pooling and global average pooling on the input feature map F (with a size of H×W×C) respectively to obtain two feature vectors of 1×1×C;

[0024] ② Shared multi-layer perceptron: Send these two feature vectors into a shared multi-layer perceptron (MLP) respectively. The number of neurons in the first layer of the MLP is C / r, where r is the dimensionality reduction rate and C is the number of neurons in the second layer; the activation function is ReLU; learn the dependence relationship between channels through the MLP;

[0025] ③ Weight generation: Add the output results of the MLP, and then generate a channel attention weight matrix M c through the Sigmoid activation function, and each element of it represents the importance of the corresponding channel;

[0026] ④ Feature weighting: Multiply the channel attention weight matrix M c by the original feature map F to obtain a weighted feature map;

[0027] (4) Before each convolution block in the decoder stage, perform a skip connection between the feature map in the encoder stage and the feature map in the decoder stage. The specific steps are as follows:

[0028] ① Upsampling: Use a transposed convolution layer or nearest neighbor interpolation to double the spatial resolution of the feature map in the decoder stage;

[0029] ② Feature map concatenation: Concatenate the feature map in the encoder stage and the feature map in the decoder stage in the channel dimension;

[0030] ③Convolution operation: Perform convolution operation on the concatenated feature map to further extract features and reduce the number of channels of the feature map;

[0031] (5) Add a spatial attention module in the decoder. The specific implementation steps are as follows:

[0032] ①Feature extraction: Perform global average pooling and global maximum pooling on the input feature map to generate two independent feature maps;

[0033] ②Feature fusion: Concatenate these two feature maps along the channel dimension, and then use a convolutional layer to extract spatial information;

[0034] ③Weight generation: Normalize the weights to between 0 and 1 through the Sigmoid activation function to generate the final spatial attention map;

[0035] ④Feature weighting: Multiply the spatial attention map and the original feature map element by element to enhance the features in the key regions and suppress the unimportant regions;

[0036] In the present invention, the classification module uses logistic regression as the output layer, introduces L2 regularization to effectively prevent the model from overfitting, and evaluates the quality of the model in the way of combining the cross-entropy loss function and the DICE loss function. The specific calculation method of the loss function is as shown in formulas (3)-(6):

[0037] ①Cross-entropy loss L CE : For binary classification problems, its calculation formula is:

[0038]

[0039] where N is the number of samples, y i is the true label of the i-th sample, is the predicted probability of the i-th sample;

[0040] ②DICE loss L DICE is:

[0041]

[0042] ③L2 regularization term L L2 is:

[0043]

[0044] where λ is the regularization coefficient, M is the number of model parameters, and ω j is the j-th model parameter.

[0045] ④Total loss function L total As shown in formula (6):

[0046]

[0047] The present invention can effectively solve the problem of difficult automatic segmentation of the left ventricle caused by individual differences and pathological changes, provide more accurate left ventricle segmentation results for medical research and clinical diagnosis, and contribute to the early detection and treatment of heart diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a block diagram of the method for segmenting the left ventricle of two-dimensional echocardiogram based on the improved U-Net of the present invention.

[0049] Figure 2 They are echocardiogram images of the embodiments of the present invention and the images after preprocessing. Among them, (a), (b), and (c) are the original two-dimensional echocardiogram image and the images obtained by central cropping and bilinear interpolation respectively.

[0050] Figure 3 It is the left ventricle segmentation network model of the embodiments of the present invention.

[0051] Figure 4 It is the channel attention model of the embodiments of the present invention.

[0052] Figure 5 It is the spatial attention model of the embodiments of the present invention.

[0053] Figure 6 They are echocardiogram images of the embodiments of the present invention and the left ventricle segmentation results. Among them, (a) and (b) are the input image and the output result of the left ventricle segmentation network respectively. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0054] The following further describes the specific embodiments of the present invention with reference to the drawings. The following embodiments are used to illustrate the present invention, but do not limit the scope of the rights of the present invention.

[0055] In this embodiment, the method for segmenting the left ventricle of two-dimensional echocardiogram based on the improved U-Net, as Figure 1 shown, includes the following steps:

[0056] Step 1: The image preprocessing module preprocesses the collected two-dimensional echocardiogram to obtain the input of the segmentation network;

[0057] (1.1) Center-crop the image to delete the interference information around the image and retain the main part, as Figure 2 (b) shown;

[0058] (1.2) Downsample the cropped image to 112 pixels × 112 pixels using bilinear interpolation, as Figure 2 (c) shown, to meet the input requirements of the subsequent network;

[0059] Step 2: Feature extraction and fusion module. Construct a left ventricular segmentation network, as Figure 3 shown, to perform feature extraction and feature fusion on the input image;

[0060] (2.1) Use a four-layer basic U-Net model. The input of the model is a preprocessed two-dimensional echocardiogram with a size of 112 pixels × 112 pixels. Its encoder consists of two operations: convolution and downsampling. The convolution used is a 3×3 convolution kernel, the padding is set to 0, the stride is set to 1, and then a batch normalization layer and a ReLU activation function are connected in sequence. After the convolutional layer, a 2×2 max pooling layer is used to reduce the spatial dimension;

[0061] (2.2) Replace the 3×3 convolution in each layer of the encoder with dilated convolutions with dilation rates of 1, 2, 4, and 8 respectively, and their receptive fields are 3×3, 7×7, 15×15, and 31×31 respectively.

[0062] (2.3) Add a channel attention module after the convolutional operation in the encoder, as Figure 4 shown. The specific implementation steps are as follows:

[0063] ① Global pooling: Perform global max pooling and global average pooling on the input feature map F (with a size of H×W×C) respectively to obtain two 1×1×C feature vectors;

[0064] ② Shared multi-layer perceptron: Send these two feature vectors into a shared multi-layer perceptron (MLP) respectively. The number of neurons in the first layer of the MLP is C / r (where r is the dimensionality reduction rate), and the activation function is ReLU; the number of neurons in the second layer is C. Learn the dependency relationship between channels through the MLP;

[0065] ③ Weight generation: Add the output results of the MLP, and then generate a channel attention weight matrix M c through the Sigmoid activation function. Each element of it represents the importance of the corresponding channel;

[0066] ④ Feature weighting: Multiply the channel attention weight matrix M c by the original feature map F to obtain a weighted feature map;

[0067] (2.4) Before each convolutional block in the decoder stage, perform a skip connection between the feature map in the encoder stage and the feature map in the decoder stage. The specific steps are as follows:

[0068] ① 3×3 upsampling: Use a transposed convolutional layer or nearest neighbor interpolation to double the spatial resolution of the feature map in the decoder stage;

[0069] ② Feature map concatenation: Concatenate the feature maps in the encoder stage and the decoder stage along the channel dimension;

[0070] ③ Convolution operation: Perform convolution operation on the concatenated feature maps. The convolution block used is the same as that in the encoder to further extract features and reduce the number of channels of the feature maps;

[0071] (2.5) Add a spatial attention module in the decoder, as Figure 5 shown. The specific implementation steps are as follows:

[0072] ① Feature extraction: Perform global average pooling and global max pooling on the input feature map to generate two independent feature maps;

[0073] ② Feature fusion: Concatenate these two feature maps along the channel dimension, and then use a convolutional layer to extract spatial information;

[0074] ③ Weight generation: Normalize the weights between 0 and 1 through the Sigmoid activation function to generate the final spatial attention map;

[0075] ④ Feature weighting: Multiply the spatial attention map and the original feature map element by element to enhance the features in the key regions and suppress the unimportant regions;

[0076] Step 3: Classify and discriminate by the classifier to obtain the segmentation result as Figure 6 shown, where (a) and (b) are the input image and the output result of the left ventricle segmentation network respectively; The specific classification method is: Use logistic regression as the output layer, introduce L2 regularization to effectively prevent the model from overfitting, and evaluate the quality of the model by combining the cross-entropy loss function and the DICE loss function.

[0077] Perform five-fold cross-validation on 4924 two-dimensional echocardiograms of 145 subjects according to this embodiment. The Dice similarity coefficient (DSC) of the segmentation result is 93.79% ± 0.68%, and the 95th percentile of the Hausdorff distance (HD95) is 1.31 ± 0.07, indicating that the two-dimensional echocardiogram left ventricle segmentation system based on the improved U-Net provided by the present invention has a high segmentation accuracy.

[0078] The above is the preferred embodiment of the present invention, but the protection scope of the present invention is not limited to this example. Any other examples made by simple replacement and substitution by those skilled in the art within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. The protection scope of the present invention shall be subject to the protection scope defined by the claims.

Claims

1. A method for segmenting the left ventricle of two-dimensional echocardiogram based on improved U-Net, characterized in that, It includes an image preprocessing module, a feature extraction and fusion module, and a classifier; among which: The image preprocessing module is used to preprocess the collected two-dimensional echocardiogram images; specifically, it includes extracting the region of interest from the two-dimensional echocardiogram; that is, through central cropping, removing the interference information around the image and retaining the main part; then using bilinear interpolation to downsample the cropped image to 112 pixels × 112 pixels to meet the subsequent network input requirements; The feature extraction and fusion module takes the preprocessed image as input and applies an improved U-Net for feature extraction and feature fusion; it includes an encoder and a decoder; specifically, the encoder consists of multiple convolutional layers and pooling layers. Dilated convolutions are used in the encoder part, and the dilation rate is increased layer by layer to expand the receptive field of the convolutional kernel, extract high-level features of the image, and at the same time reduce the resolution of the image; the decoder part restores the low-resolution feature map output by the encoder to the original image size step by step through upsampling operations, and performs channel splicing with the feature map of the corresponding layer of the encoder through skip connections to avoid feature loss; in addition, an attention mechanism is introduced into the U-Net to weight the feature map from both the channel and spatial dimensions to enhance the model's ability to focus on key information; there are two ways to introduce attention: (1) Add a channel attention module in the U-Net encoder. By learning the importance of different channels, different weights are assigned to each channel, thereby enhancing the response of key feature channels and suppressing unimportant channels; (2) Add a spatial attention module in the U-Net decoder. By learning the importance of different spatial positions in the feature map, different weights are assigned to each position, thereby enhancing the feature response of key regions and suppressing unimportant regions; The classifier module uses logistic regression as the classifier, measures the classification performance by combining the cross-entropy loss function and the DICE loss function, and introduces an L2 regularization term to prevent the feature coefficients from being too large, reduce the complexity of the model, and prevent overfitting.

2. The two-dimensional echocardiogram left ventricle segmentation system according to claim 1, wherein In the image preprocessing module, the specific workflow is as follows: (1) Use central cropping to remove the interference information around the two-dimensional echocardiogram and retain the main part. For a two-dimensional echocardiogram image with an original width of W and a height of H, the target width is w and the target height is h. The starting coordinates of the central cropping are: Among them, represents rounding down; (2) Use bilinear interpolation to downsample the cropped image. The value of the target point (x, y) is estimated by calculating the weighted average of the four known points (x1, y1), (x2, y1), (x1, y2), and (x2, y2) around it. The calculation formula is:

3. The two-dimensional echocardiogram left ventricle segmentation system according to claim 1, characterized in that, The construction of the segmentation network model has the following specific process: (1) Use a four-layer basic U-Net model. The model input is the preprocessed two-dimensional echocardiogram, with a size of 112 pixels × 112 pixels; (2) Replace the 3×3 convolution in each layer of the encoder with dilated convolutions with dilation rates of 1, 2, 4, and 8 respectively, and their receptive fields are 3×3, 7×7, 15×15, and 31×31; (3) Add a channel attention module after the convolutional operation in the encoder. The specific implementation steps are as follows: ① Global pooling: Perform global max pooling and global average pooling on the input feature map F respectively to obtain two feature vectors of 1×1×C. ② Shared multi-layer perception mechanism: Send these two feature vectors into a shared multi-layer perceptron (MLP) respectively. The number of neurons in the first layer of the MLP is C / r, where r is the dimensionality reduction rate and C is the number of neurons in the second layer; the activation function is ReLU; learn the dependencies between channels through the MLP. ③ Weight generation: Add the output results of the MLP, and then generate the channel attention weight matrix M through the Sigmoid activation function c , where each element represents the importance of the corresponding channel; ④ Feature weighting: Multiply the channel attention weight matrix M c by the original feature map F to obtain the weighted feature map; (4) Before each convolutional block in the decoder stage, perform a skip connection between the feature map in the encoder stage and the feature map in the decoder stage. The specific steps are as follows: ① Upsampling: Use a transposed convolutional layer or nearest neighbor interpolation to double the spatial resolution of the feature map in the decoder stage. ② Feature map concatenation: Concatenate the feature map in the encoder stage and the feature map in the decoder stage along the channel dimension. ③ Convolutional operation: Perform a convolutional operation on the concatenated feature map to further extract features and reduce the number of channels of the feature map. (5) Add a spatial attention module in the decoder. The specific implementation steps are as follows: ① Feature extraction: Perform global average pooling and global max pooling on the input feature map to generate two independent feature maps. ② Feature fusion: Concatenate these two feature maps along the channel dimension, and then use a convolutional layer to extract spatial information. ③ Weight generation: Normalize the weights to between 0 and 1 through the Sigmoid activation function to generate the final spatial attention map. ④ Feature weighting: Multiply the spatial attention map and the original feature map element by element to enhance the features in the key regions and suppress the unimportant regions.

4. The two-dimensional echocardiogram left ventricle segmentation system according to claim 1, characterized in that, The classification module uses logistic regression as the output layer, introduces L2 regularization to prevent the model from overfitting, and evaluates the quality of the model by combining the cross-entropy loss function and the DICE loss function. Specific calculation of the loss function: ① Cross-entropy loss L CE : For binary classification problems, its calculation formula is: where N is the number of samples, and y i is the true label of the i-th sample, is the predicted probability of the i-th sample; ② DICE loss L DICE The calculation formula is as follows: ③ The L2 regularization term L L2 is calculated as follows: where λ is the regularization coefficient, M is the number of model parameters, and ω j is the j-th model parameter; ④Total loss function L total is as follows:

Images